2023-12-01T00:55:56.466 INFO:root:teuthology version: 0.0.1.dev214+gf2de108 2023-12-01T00:55:56.466 DEBUG:teuthology.run:Teuthology command: teuthology --name yuriw-2023-11-30_22:47:08-fs-reef-release-distro-default-smithi --verbose --archive /home/teuthworker/archive/yuriw-2023-11-30_22:47:08-fs-reef-release-distro-default-smithi/7473168 --owner scheduled_yuriw@teuthology --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/fsx}} -- /home/teuthworker/archive/yuriw-2023-11-30_22:47:08-fs-reef-release-distro-default-smithi/7473168/orig.config.yaml 2023-12-01T00:55:56.534 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-01T00:55:56.622 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-11-30_22:47:08-fs-reef-release-distro-default-smithi/7473168 branch: reef-release description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/fsx}} email: yweinste@redhat.com first_in_suite: false job_id: '7473168' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-11-30_22:47:08-fs-reef-release-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: reef-release ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: a6f82997ac57b70a895455dfed6256360a1e4c32 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.dir_split install: ceph: flavor: default sha1: a6f82997ac57b70a895455dfed6256360a1e4c32 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: reef-release sha1: a6f82997ac57b70a895455dfed6256360a1e4c32 owner: scheduled_yuriw@teuthology priority: 55 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8832 sha1: a6f82997ac57b70a895455dfed6256360a1e4c32 sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: reef-release suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_a6f82997ac57b70a895455dfed6256360a1e4c32/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: a6f82997ac57b70a895455dfed6256360a1e4c32 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi077.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi137.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi203.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/fsx.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: f2de108beb0a6dde7f3e6c96039071828a1f6269 timestamp: 2023-11-30_22:47:08 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-12-01T00:55:56.622 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_a6f82997ac57b70a895455dfed6256360a1e4c32/qa; will attempt to use it 2023-12-01T00:55:56.623 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_a6f82997ac57b70a895455dfed6256360a1e4c32/qa/tasks 2023-12-01T00:55:56.623 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-12-01T00:55:56.625 INFO:teuthology.task.internal:Checking packages... 2023-12-01T00:55:56.649 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'a6f82997ac57b70a895455dfed6256360a1e4c32' 2023-12-01T00:55:56.649 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-12-01T00:55:56.649 INFO:teuthology.packaging:ref: None 2023-12-01T00:55:56.649 INFO:teuthology.packaging:tag: None 2023-12-01T00:55:56.649 INFO:teuthology.packaging:branch: reef-release 2023-12-01T00:55:56.649 INFO:teuthology.packaging:sha1: a6f82997ac57b70a895455dfed6256360a1e4c32 2023-12-01T00:55:56.649 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=reef-release 2023-12-01T00:55:56.803 INFO:teuthology.task.internal:Found packages for ceph version 18.2.0-1249.ga6f82997 2023-12-01T00:55:56.803 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-12-01T00:55:56.810 INFO:teuthology.task.internal:no buildpackages task found 2023-12-01T00:55:56.810 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-12-01T00:55:56.819 INFO:teuthology.task.internal:Saving configuration 2023-12-01T00:55:56.844 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-12-01T00:55:56.852 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-01T00:55:56.878 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi077.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-30_22:47:08-fs-reef-release-distro-default-smithi/7473168', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-01 00:50:54.330090', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-12-01T00:55:56.901 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi137.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-30_22:47:08-fs-reef-release-distro-default-smithi/7473168', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-01 00:50:54.328310', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-12-01T00:55:56.925 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi203.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-30_22:47:08-fs-reef-release-distro-default-smithi/7473168', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-01 00:50:54.331166', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-12-01T00:55:56.925 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-12-01T00:55:56.933 INFO:teuthology.task.internal:roles: ubuntu@smithi077.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-12-01T00:55:56.933 INFO:teuthology.task.internal:roles: ubuntu@smithi137.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-12-01T00:55:56.933 INFO:teuthology.task.internal:roles: ubuntu@smithi203.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-12-01T00:55:56.933 INFO:teuthology.run_tasks:Running task console_log... 2023-12-01T00:55:57.042 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3e674a7dc0>, signals=[15]) 2023-12-01T00:55:57.042 INFO:teuthology.run_tasks:Running task internal.connect... 2023-12-01T00:55:57.050 INFO:teuthology.task.internal:Opening connections... 2023-12-01T00:55:57.050 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi077.front.sepia.ceph.com 2023-12-01T00:55:57.052 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-01T00:55:57.122 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi137.front.sepia.ceph.com 2023-12-01T00:55:57.123 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-01T00:55:57.189 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi203.front.sepia.ceph.com 2023-12-01T00:55:57.190 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-01T00:55:57.257 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-12-01T00:55:57.264 DEBUG:teuthology.orchestra.run.smithi077:> uname -m 2023-12-01T00:55:57.286 INFO:teuthology.orchestra.run.smithi077.stdout:x86_64 2023-12-01T00:55:57.286 DEBUG:teuthology.orchestra.run.smithi077:> cat /etc/os-release 2023-12-01T00:55:57.344 INFO:teuthology.orchestra.run.smithi077.stdout:NAME="Red Hat Enterprise Linux" 2023-12-01T00:55:57.345 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION="8.6 (Ootpa)" 2023-12-01T00:55:57.345 INFO:teuthology.orchestra.run.smithi077.stdout:ID="rhel" 2023-12-01T00:55:57.345 INFO:teuthology.orchestra.run.smithi077.stdout:ID_LIKE="fedora" 2023-12-01T00:55:57.345 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION_ID="8.6" 2023-12-01T00:55:57.345 INFO:teuthology.orchestra.run.smithi077.stdout:PLATFORM_ID="platform:el8" 2023-12-01T00:55:57.345 INFO:teuthology.orchestra.run.smithi077.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-01T00:55:57.345 INFO:teuthology.orchestra.run.smithi077.stdout:ANSI_COLOR="0;31" 2023-12-01T00:55:57.345 INFO:teuthology.orchestra.run.smithi077.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-01T00:55:57.345 INFO:teuthology.orchestra.run.smithi077.stdout:HOME_URL="https://www.redhat.com/" 2023-12-01T00:55:57.345 INFO:teuthology.orchestra.run.smithi077.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-01T00:55:57.345 INFO:teuthology.orchestra.run.smithi077.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-01T00:55:57.346 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T00:55:57.346 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-01T00:55:57.346 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-01T00:55:57.346 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-01T00:55:57.346 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-01T00:55:57.346 INFO:teuthology.lock.ops:Updating smithi077.front.sepia.ceph.com on lock server 2023-12-01T00:55:57.367 DEBUG:teuthology.orchestra.run.smithi137:> uname -m 2023-12-01T00:55:57.387 INFO:teuthology.orchestra.run.smithi137.stdout:x86_64 2023-12-01T00:55:57.387 DEBUG:teuthology.orchestra.run.smithi137:> cat /etc/os-release 2023-12-01T00:55:57.447 INFO:teuthology.orchestra.run.smithi137.stdout:NAME="Red Hat Enterprise Linux" 2023-12-01T00:55:57.447 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION="8.6 (Ootpa)" 2023-12-01T00:55:57.447 INFO:teuthology.orchestra.run.smithi137.stdout:ID="rhel" 2023-12-01T00:55:57.447 INFO:teuthology.orchestra.run.smithi137.stdout:ID_LIKE="fedora" 2023-12-01T00:55:57.447 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION_ID="8.6" 2023-12-01T00:55:57.447 INFO:teuthology.orchestra.run.smithi137.stdout:PLATFORM_ID="platform:el8" 2023-12-01T00:55:57.448 INFO:teuthology.orchestra.run.smithi137.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-01T00:55:57.448 INFO:teuthology.orchestra.run.smithi137.stdout:ANSI_COLOR="0;31" 2023-12-01T00:55:57.448 INFO:teuthology.orchestra.run.smithi137.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-01T00:55:57.448 INFO:teuthology.orchestra.run.smithi137.stdout:HOME_URL="https://www.redhat.com/" 2023-12-01T00:55:57.448 INFO:teuthology.orchestra.run.smithi137.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-01T00:55:57.448 INFO:teuthology.orchestra.run.smithi137.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-01T00:55:57.448 INFO:teuthology.orchestra.run.smithi137.stdout: 2023-12-01T00:55:57.448 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-01T00:55:57.448 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-01T00:55:57.448 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-01T00:55:57.448 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-01T00:55:57.448 INFO:teuthology.lock.ops:Updating smithi137.front.sepia.ceph.com on lock server 2023-12-01T00:55:57.475 DEBUG:teuthology.orchestra.run.smithi203:> uname -m 2023-12-01T00:55:57.495 INFO:teuthology.orchestra.run.smithi203.stdout:x86_64 2023-12-01T00:55:57.495 DEBUG:teuthology.orchestra.run.smithi203:> cat /etc/os-release 2023-12-01T00:55:57.555 INFO:teuthology.orchestra.run.smithi203.stdout:NAME="Red Hat Enterprise Linux" 2023-12-01T00:55:57.555 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION="8.6 (Ootpa)" 2023-12-01T00:55:57.555 INFO:teuthology.orchestra.run.smithi203.stdout:ID="rhel" 2023-12-01T00:55:57.555 INFO:teuthology.orchestra.run.smithi203.stdout:ID_LIKE="fedora" 2023-12-01T00:55:57.555 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION_ID="8.6" 2023-12-01T00:55:57.555 INFO:teuthology.orchestra.run.smithi203.stdout:PLATFORM_ID="platform:el8" 2023-12-01T00:55:57.555 INFO:teuthology.orchestra.run.smithi203.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-01T00:55:57.555 INFO:teuthology.orchestra.run.smithi203.stdout:ANSI_COLOR="0;31" 2023-12-01T00:55:57.555 INFO:teuthology.orchestra.run.smithi203.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-01T00:55:57.555 INFO:teuthology.orchestra.run.smithi203.stdout:HOME_URL="https://www.redhat.com/" 2023-12-01T00:55:57.555 INFO:teuthology.orchestra.run.smithi203.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-01T00:55:57.555 INFO:teuthology.orchestra.run.smithi203.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-01T00:55:57.555 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-12-01T00:55:57.556 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-01T00:55:57.556 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-01T00:55:57.556 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-01T00:55:57.556 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-01T00:55:57.556 INFO:teuthology.lock.ops:Updating smithi203.front.sepia.ceph.com on lock server 2023-12-01T00:55:57.574 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-12-01T00:55:57.586 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-12-01T00:55:57.591 INFO:teuthology.task.internal:Checking for old test directory... 2023-12-01T00:55:57.592 DEBUG:teuthology.orchestra.run.smithi077:> test '!' -e /home/ubuntu/cephtest 2023-12-01T00:55:57.594 DEBUG:teuthology.orchestra.run.smithi137:> test '!' -e /home/ubuntu/cephtest 2023-12-01T00:55:57.597 DEBUG:teuthology.orchestra.run.smithi203:> test '!' -e /home/ubuntu/cephtest 2023-12-01T00:55:57.615 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-12-01T00:55:57.622 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-12-01T00:55:57.623 DEBUG:teuthology.orchestra.run.smithi077:> test -z $(ls -A /var/lib/ceph) 2023-12-01T00:55:57.653 DEBUG:teuthology.orchestra.run.smithi137:> test -z $(ls -A /var/lib/ceph) 2023-12-01T00:55:57.655 DEBUG:teuthology.orchestra.run.smithi203:> test -z $(ls -A /var/lib/ceph) 2023-12-01T00:55:57.688 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-12-01T00:55:57.770 INFO:teuthology.run_tasks:Running task kernel... 2023-12-01T00:55:57.791 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2023-12-01T00:55:57.791 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2023-12-01T00:55:57.791 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2023-12-01T00:55:57.791 DEBUG:teuthology.orchestra.run.smithi077:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-01T00:55:57.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-01T00:55:57.811 DEBUG:teuthology.orchestra.run.smithi077:> uname -r 2023-12-01T00:55:57.871 INFO:teuthology.orchestra.run.smithi077.stdout:4.18.0-372.9.1.el8.x86_64 2023-12-01T00:55:57.871 INFO:teuthology.task.kernel:Running kernel on smithi077: 4.18.0-372.9.1.el8.x86_64 2023-12-01T00:55:57.871 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum install -y kernel 2023-12-01T00:55:59.087 INFO:teuthology.orchestra.run.smithi077.stdout:Updating Subscription Management repositories. 2023-12-01T00:55:59.087 INFO:teuthology.orchestra.run.smithi077.stdout:Unable to read consumer identity 2023-12-01T00:56:00.465 INFO:teuthology.orchestra.run.smithi077.stdout:Copr repo for python3-asyncssh owned by ceph 18 kB/s | 3.5 kB 00:00 2023-12-01T00:56:01.216 INFO:teuthology.orchestra.run.smithi077.stdout:Extra Packages for Enterprise Linux 22 MB/s | 16 MB 00:00 2023-12-01T00:56:04.674 INFO:teuthology.orchestra.run.smithi077.stdout:lab-extras 435 kB/s | 24 kB 00:00 2023-12-01T00:56:05.368 INFO:teuthology.orchestra.run.smithi077.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-12-01T00:56:05.417 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2023-12-01T00:56:05.418 INFO:teuthology.orchestra.run.smithi077.stdout:Nothing to do. 2023-12-01T00:56:05.418 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2023-12-01T00:56:05.537 DEBUG:teuthology.orchestra.run.smithi077:> echo no | sudo yum reinstall kernel || true 2023-12-01T00:56:05.867 INFO:teuthology.orchestra.run.smithi077.stdout:Updating Subscription Management repositories. 2023-12-01T00:56:05.868 INFO:teuthology.orchestra.run.smithi077.stdout:Unable to read consumer identity 2023-12-01T00:56:06.126 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:00:02 ago on Fri 01 Dec 2023 12:56:04 AM UTC. 2023-12-01T00:56:06.173 INFO:teuthology.orchestra.run.smithi077.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-12-01T00:56:06.201 INFO:teuthology.orchestra.run.smithi077.stderr:Error: No packages marked for reinstall. 2023-12-01T00:56:06.246 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum reinstall -y kernel || true 2023-12-01T00:56:06.579 INFO:teuthology.orchestra.run.smithi077.stdout:Updating Subscription Management repositories. 2023-12-01T00:56:06.579 INFO:teuthology.orchestra.run.smithi077.stdout:Unable to read consumer identity 2023-12-01T00:56:06.836 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:00:02 ago on Fri 01 Dec 2023 12:56:04 AM UTC. 2023-12-01T00:56:06.878 INFO:teuthology.orchestra.run.smithi077.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-12-01T00:56:06.907 INFO:teuthology.orchestra.run.smithi077.stderr:Error: No packages marked for reinstall. 2023-12-01T00:56:06.957 DEBUG:teuthology.orchestra.run.smithi077:> rpm -q kernel | sort -rV | head -n 1 2023-12-01T00:56:07.037 INFO:teuthology.orchestra.run.smithi077.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-12-01T00:56:07.037 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-12-01T00:56:07.037 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-12-01T00:56:07.037 DEBUG:teuthology.parallel:result is None 2023-12-01T00:56:07.038 INFO:teuthology.run_tasks:Running task internal.base... 2023-12-01T00:56:07.045 INFO:teuthology.task.internal:Creating test directory... 2023-12-01T00:56:07.045 DEBUG:teuthology.orchestra.run.smithi077:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-01T00:56:07.047 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-01T00:56:07.050 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-01T00:56:07.070 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-12-01T00:56:07.084 INFO:teuthology.run_tasks:Running task internal.archive... 2023-12-01T00:56:07.092 INFO:teuthology.task.internal:Creating archive directory... 2023-12-01T00:56:07.093 DEBUG:teuthology.orchestra.run.smithi077:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-01T00:56:07.104 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-01T00:56:07.110 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-01T00:56:07.155 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-12-01T00:56:07.162 INFO:teuthology.task.internal:Enabling coredump saving... 2023-12-01T00:56:07.162 DEBUG:teuthology.orchestra.run.smithi137:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-01T00:56:07.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-01T00:56:07.195 DEBUG:teuthology.orchestra.run.smithi203:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-01T00:56:07.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-01T00:56:07.213 DEBUG:teuthology.orchestra.run.smithi077:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-01T00:56:07.215 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-01T00:56:07.237 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-01T00:56:07.256 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-01T00:56:07.271 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-01T00:56:07.283 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-01T00:56:07.301 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-01T00:56:07.302 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-01T00:56:07.321 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-01T00:56:07.323 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-12-01T00:56:07.330 INFO:teuthology.task.internal:Configuring sudo... 2023-12-01T00:56:07.332 DEBUG:teuthology.orchestra.run.smithi077:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-01T00:56:07.334 DEBUG:teuthology.orchestra.run.smithi137:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-01T00:56:07.337 DEBUG:teuthology.orchestra.run.smithi203:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-01T00:56:07.365 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-12-01T00:56:07.374 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-12-01T00:56:07.374 DEBUG:teuthology.orchestra.run.smithi077:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-01T00:56:07.401 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-01T00:56:07.405 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-01T00:56:07.422 DEBUG:teuthology.orchestra.run.smithi077:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-01T00:56:07.499 DEBUG:teuthology.orchestra.run.smithi077:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-01T00:56:07.548 DEBUG:teuthology.orchestra.run.smithi077:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-01T00:56:07.633 DEBUG:teuthology.orchestra.run.smithi077:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-01T00:56:07.674 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-12-01T00:56:07.674 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-01T00:56:07.752 DEBUG:teuthology.orchestra.run.smithi137:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-01T00:56:07.800 DEBUG:teuthology.orchestra.run.smithi137:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-01T00:56:07.853 DEBUG:teuthology.orchestra.run.smithi137:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-01T00:56:07.938 DEBUG:teuthology.orchestra.run.smithi137:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-01T00:56:07.984 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2023-12-01T00:56:07.985 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-01T00:56:08.075 DEBUG:teuthology.orchestra.run.smithi203:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-01T00:56:08.123 DEBUG:teuthology.orchestra.run.smithi203:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-01T00:56:08.181 DEBUG:teuthology.orchestra.run.smithi203:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-01T00:56:08.269 DEBUG:teuthology.orchestra.run.smithi203:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-01T00:56:08.314 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-12-01T00:56:08.314 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-01T00:56:08.384 DEBUG:teuthology.orchestra.run.smithi077:> sudo service rsyslog restart 2023-12-01T00:56:08.387 DEBUG:teuthology.orchestra.run.smithi137:> sudo service rsyslog restart 2023-12-01T00:56:08.389 DEBUG:teuthology.orchestra.run.smithi203:> sudo service rsyslog restart 2023-12-01T00:56:08.445 INFO:teuthology.orchestra.run.smithi077.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-01T00:56:08.455 INFO:teuthology.orchestra.run.smithi137.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-01T00:56:08.504 INFO:teuthology.orchestra.run.smithi203.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-01T00:56:08.918 INFO:teuthology.run_tasks:Running task internal.timer... 2023-12-01T00:56:08.924 INFO:teuthology.task.internal:Starting timer... 2023-12-01T00:56:08.925 INFO:teuthology.run_tasks:Running task pcp... 2023-12-01T00:56:08.936 INFO:teuthology.run_tasks:Running task selinux... 2023-12-01T00:56:08.944 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2023-12-01T00:56:08.945 DEBUG:teuthology.orchestra.run.smithi077:> sudo service auditd rotate 2023-12-01T00:56:09.003 INFO:teuthology.orchestra.run.smithi077.stdout:Rotating logs: 2023-12-01T00:56:09.004 DEBUG:teuthology.orchestra.run.smithi137:> sudo service auditd rotate 2023-12-01T00:56:09.066 INFO:teuthology.orchestra.run.smithi137.stdout:Rotating logs: 2023-12-01T00:56:09.068 DEBUG:teuthology.orchestra.run.smithi203:> sudo service auditd rotate 2023-12-01T00:56:09.134 INFO:teuthology.orchestra.run.smithi203.stdout:Rotating logs: 2023-12-01T00:56:09.136 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-12-01T00:56:09.136 DEBUG:teuthology.orchestra.run.smithi077:> /usr/sbin/getenforce 2023-12-01T00:56:09.164 INFO:teuthology.orchestra.run.smithi077.stdout:Permissive 2023-12-01T00:56:09.164 DEBUG:teuthology.orchestra.run.smithi137:> /usr/sbin/getenforce 2023-12-01T00:56:09.188 INFO:teuthology.orchestra.run.smithi137.stdout:Permissive 2023-12-01T00:56:09.189 DEBUG:teuthology.orchestra.run.smithi203:> /usr/sbin/getenforce 2023-12-01T00:56:09.208 INFO:teuthology.orchestra.run.smithi203.stdout:Permissive 2023-12-01T00:56:09.208 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi077.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi137.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi203.front.sepia.ceph.com': 'permissive'} 2023-12-01T00:56:09.209 DEBUG:teuthology.orchestra.run.smithi077:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-01T00:56:09.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-01T00:56:09.235 DEBUG:teuthology.orchestra.run.smithi137:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-01T00:56:09.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-01T00:56:09.264 DEBUG:teuthology.orchestra.run.smithi203:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-01T00:56:09.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-01T00:56:09.291 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-12-01T00:56:09.291 DEBUG:teuthology.orchestra.run.smithi077:> sudo /usr/sbin/setenforce permissive 2023-12-01T00:56:09.319 DEBUG:teuthology.orchestra.run.smithi137:> sudo /usr/sbin/setenforce permissive 2023-12-01T00:56:09.345 DEBUG:teuthology.orchestra.run.smithi203:> sudo /usr/sbin/setenforce permissive 2023-12-01T00:56:09.390 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-12-01T00:56:09.399 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-12-01T00:56:09.404 INFO:teuthology.repo_utils:Fetching main from origin 2023-12-01T00:56:09.517 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-12-01T00:56:09.532 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-12-01T00:56:09.533 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi077.front.sepia.ceph.com,smithi137.front.sepia.ceph.com,smithi203.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-12-01T01:05:46.799 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi077.front.sepia.ceph.com'), Remote(name='ubuntu@smithi137.front.sepia.ceph.com'), Remote(name='ubuntu@smithi203.front.sepia.ceph.com')] 2023-12-01T01:05:46.800 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com' 2023-12-01T01:05:46.801 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-01T01:05:46.872 DEBUG:teuthology.orchestra.run.smithi077:> true 2023-12-01T01:05:46.945 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi077.front.sepia.ceph.com' 2023-12-01T01:05:46.945 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2023-12-01T01:05:46.946 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-01T01:05:47.015 DEBUG:teuthology.orchestra.run.smithi137:> true 2023-12-01T01:05:47.043 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi137.front.sepia.ceph.com' 2023-12-01T01:05:47.043 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2023-12-01T01:05:47.044 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-01T01:05:47.111 DEBUG:teuthology.orchestra.run.smithi203:> true 2023-12-01T01:05:47.180 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi203.front.sepia.ceph.com' 2023-12-01T01:05:47.181 INFO:teuthology.run_tasks:Running task clock... 2023-12-01T01:05:47.191 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-12-01T01:05:47.192 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-01T01:05:47.192 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-01T01:05:47.194 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-01T01:05:47.195 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-01T01:05:47.197 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-01T01:05:47.197 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-01T01:05:47.229 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-12-01T01:05:47.231 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-12-01T01:05:47.245 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-12-01T01:05:47.247 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-12-01T01:05:47.252 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-12-01T01:05:47.267 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-12-01T01:05:47.324 INFO:teuthology.orchestra.run.smithi077.stderr:sudo: ntpd: command not found 2023-12-01T01:05:47.327 INFO:teuthology.orchestra.run.smithi137.stderr:sudo: ntpd: command not found 2023-12-01T01:05:47.334 INFO:teuthology.orchestra.run.smithi203.stderr:sudo: ntpd: command not found 2023-12-01T01:05:47.337 INFO:teuthology.orchestra.run.smithi077.stdout:506 Cannot talk to daemon 2023-12-01T01:05:47.340 INFO:teuthology.orchestra.run.smithi137.stdout:506 Cannot talk to daemon 2023-12-01T01:05:47.346 INFO:teuthology.orchestra.run.smithi203.stdout:506 Cannot talk to daemon 2023-12-01T01:05:47.352 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-12-01T01:05:47.356 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-12-01T01:05:47.360 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-12-01T01:05:47.366 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-12-01T01:05:47.370 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-12-01T01:05:47.374 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-12-01T01:05:47.437 INFO:teuthology.orchestra.run.smithi203.stderr:bash: ntpq: command not found 2023-12-01T01:05:47.439 INFO:teuthology.orchestra.run.smithi137.stderr:bash: ntpq: command not found 2023-12-01T01:05:47.441 INFO:teuthology.orchestra.run.smithi203.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-01T01:05:47.441 INFO:teuthology.orchestra.run.smithi203.stdout:=============================================================================== 2023-12-01T01:05:47.441 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-01T01:05:47.441 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-01T01:05:47.441 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-01T01:05:47.441 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-01T01:05:47.442 INFO:teuthology.orchestra.run.smithi077.stderr:bash: ntpq: command not found 2023-12-01T01:05:47.443 INFO:teuthology.orchestra.run.smithi137.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-01T01:05:47.443 INFO:teuthology.orchestra.run.smithi137.stdout:=============================================================================== 2023-12-01T01:05:47.443 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-01T01:05:47.443 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-01T01:05:47.443 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-01T01:05:47.444 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-01T01:05:47.445 INFO:teuthology.orchestra.run.smithi077.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-01T01:05:47.445 INFO:teuthology.orchestra.run.smithi077.stdout:=============================================================================== 2023-12-01T01:05:47.445 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-01T01:05:47.445 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-01T01:05:47.445 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-01T01:05:47.445 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-01T01:05:47.446 INFO:teuthology.run_tasks:Running task pexec... 2023-12-01T01:05:47.457 INFO:teuthology.task.pexec:Executing custom commands... 2023-12-01T01:05:47.458 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi077.front.sepia.ceph.com 2023-12-01T01:05:47.458 DEBUG:teuthology.orchestra.run.smithi077:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-12-01T01:05:47.458 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi137.front.sepia.ceph.com 2023-12-01T01:05:47.458 DEBUG:teuthology.orchestra.run.smithi137:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-12-01T01:05:47.459 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi203.front.sepia.ceph.com 2023-12-01T01:05:47.459 DEBUG:teuthology.orchestra.run.smithi203:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-12-01T01:05:47.855 INFO:teuthology.orchestra.run.smithi137.stdout:Updating Subscription Management repositories. 2023-12-01T01:05:47.857 INFO:teuthology.orchestra.run.smithi077.stdout:Updating Subscription Management repositories. 2023-12-01T01:05:47.858 INFO:teuthology.orchestra.run.smithi203.stdout:Updating Subscription Management repositories. 2023-12-01T01:05:48.750 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:45 ago on Fri 01 Dec 2023 01:04:03 AM UTC. 2023-12-01T01:05:48.761 INFO:teuthology.orchestra.run.smithi203.stdout:Last metadata expiration check: 0:01:36 ago on Fri 01 Dec 2023 01:04:12 AM UTC. 2023-12-01T01:05:48.795 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:01:58 ago on Fri 01 Dec 2023 01:03:50 AM UTC. 2023-12-01T01:05:50.496 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2023-12-01T01:05:50.497 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2023-12-01T01:05:50.497 INFO:teuthology.orchestra.run.smithi203.stdout: Package Architecture Version Repository Size 2023-12-01T01:05:50.497 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2023-12-01T01:05:50.497 INFO:teuthology.orchestra.run.smithi203.stdout:Resetting modules: 2023-12-01T01:05:50.498 INFO:teuthology.orchestra.run.smithi203.stdout: container-tools 2023-12-01T01:05:50.498 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-12-01T01:05:50.498 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2023-12-01T01:05:50.498 INFO:teuthology.orchestra.run.smithi203.stdout:================================================================================ 2023-12-01T01:05:50.498 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-12-01T01:05:50.503 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2023-12-01T01:05:50.504 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2023-12-01T01:05:50.504 INFO:teuthology.orchestra.run.smithi137.stdout: Package Architecture Version Repository Size 2023-12-01T01:05:50.504 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2023-12-01T01:05:50.505 INFO:teuthology.orchestra.run.smithi137.stdout:Resetting modules: 2023-12-01T01:05:50.505 INFO:teuthology.orchestra.run.smithi137.stdout: container-tools 2023-12-01T01:05:50.505 INFO:teuthology.orchestra.run.smithi137.stdout: 2023-12-01T01:05:50.505 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2023-12-01T01:05:50.505 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2023-12-01T01:05:50.505 INFO:teuthology.orchestra.run.smithi137.stdout: 2023-12-01T01:05:50.513 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2023-12-01T01:05:50.514 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2023-12-01T01:05:50.514 INFO:teuthology.orchestra.run.smithi077.stdout: Package Architecture Version Repository Size 2023-12-01T01:05:50.514 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2023-12-01T01:05:50.514 INFO:teuthology.orchestra.run.smithi077.stdout:Resetting modules: 2023-12-01T01:05:50.515 INFO:teuthology.orchestra.run.smithi077.stdout: container-tools 2023-12-01T01:05:50.515 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:05:50.515 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2023-12-01T01:05:50.515 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2023-12-01T01:05:50.515 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:05:51.453 INFO:teuthology.orchestra.run.smithi203.stdout:Last metadata expiration check: 0:01:39 ago on Fri 01 Dec 2023 01:04:12 AM UTC. 2023-12-01T01:05:51.467 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:48 ago on Fri 01 Dec 2023 01:04:03 AM UTC. 2023-12-01T01:05:51.473 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:02:01 ago on Fri 01 Dec 2023 01:03:50 AM UTC. 2023-12-01T01:05:54.490 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2023-12-01T01:05:54.726 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2023-12-01T01:05:54.781 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2023-12-01T01:05:54.958 INFO:teuthology.orchestra.run.smithi137.stdout:Updating Subscription Management repositories. 2023-12-01T01:05:55.191 INFO:teuthology.orchestra.run.smithi203.stdout:Updating Subscription Management repositories. 2023-12-01T01:05:55.534 INFO:teuthology.orchestra.run.smithi077.stdout:Updating Subscription Management repositories. 2023-12-01T01:05:55.815 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:01:52 ago on Fri 01 Dec 2023 01:04:03 AM UTC. 2023-12-01T01:05:56.186 INFO:teuthology.orchestra.run.smithi203.stdout:Last metadata expiration check: 0:01:43 ago on Fri 01 Dec 2023 01:04:12 AM UTC. 2023-12-01T01:05:56.394 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:02:05 ago on Fri 01 Dec 2023 01:03:50 AM UTC. 2023-12-01T01:05:57.511 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2023-12-01T01:05:57.604 INFO:teuthology.orchestra.run.smithi137.stdout:============================================================================================================ 2023-12-01T01:05:57.604 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2023-12-01T01:05:57.604 INFO:teuthology.orchestra.run.smithi137.stdout:============================================================================================================ 2023-12-01T01:05:57.604 INFO:teuthology.orchestra.run.smithi137.stdout:Installing group/module packages: 2023-12-01T01:05:57.604 INFO:teuthology.orchestra.run.smithi137.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-12-01T01:05:57.604 INFO:teuthology.orchestra.run.smithi137.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-12-01T01:05:57.604 INFO:teuthology.orchestra.run.smithi137.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-12-01T01:05:57.604 INFO:teuthology.orchestra.run.smithi137.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-12-01T01:05:57.605 INFO:teuthology.orchestra.run.smithi137.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-12-01T01:05:57.605 INFO:teuthology.orchestra.run.smithi137.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-12-01T01:05:57.605 INFO:teuthology.orchestra.run.smithi137.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-12-01T01:05:57.605 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2023-12-01T01:05:57.605 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-12-01T01:05:57.605 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-12-01T01:05:57.605 INFO:teuthology.orchestra.run.smithi137.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-12-01T01:05:57.605 INFO:teuthology.orchestra.run.smithi137.stdout:Installing module profiles: 2023-12-01T01:05:57.605 INFO:teuthology.orchestra.run.smithi137.stdout: container-tools/common 2023-12-01T01:05:57.605 INFO:teuthology.orchestra.run.smithi137.stdout: 2023-12-01T01:05:57.605 INFO:teuthology.orchestra.run.smithi137.stdout:Enabling module streams: 2023-12-01T01:05:57.605 INFO:teuthology.orchestra.run.smithi137.stdout: container-tools rhel8 2023-12-01T01:05:57.605 INFO:teuthology.orchestra.run.smithi137.stdout: 2023-12-01T01:05:57.605 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2023-12-01T01:05:57.605 INFO:teuthology.orchestra.run.smithi137.stdout:============================================================================================================ 2023-12-01T01:05:57.606 INFO:teuthology.orchestra.run.smithi137.stdout:Install 10 Packages 2023-12-01T01:05:57.606 INFO:teuthology.orchestra.run.smithi137.stdout: 2023-12-01T01:05:57.606 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 18 M 2023-12-01T01:05:57.606 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 65 M 2023-12-01T01:05:57.606 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2023-12-01T01:05:57.721 INFO:teuthology.orchestra.run.smithi137.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 356 kB/s | 41 kB 00:00 2023-12-01T01:05:57.750 INFO:teuthology.orchestra.run.smithi137.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 173 kB/s | 25 kB 00:00 2023-12-01T01:05:57.775 INFO:teuthology.orchestra.run.smithi137.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 558 kB/s | 94 kB 00:00 2023-12-01T01:05:57.800 INFO:teuthology.orchestra.run.smithi137.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 628 kB/s | 49 kB 00:00 2023-12-01T01:05:57.835 INFO:teuthology.orchestra.run.smithi137.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-12-01T01:05:57.859 INFO:teuthology.orchestra.run.smithi137.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-12-01T01:05:57.892 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2023-12-01T01:05:57.893 INFO:teuthology.orchestra.run.smithi137.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 3.6 MB/s | 213 kB 00:00 2023-12-01T01:05:57.989 INFO:teuthology.orchestra.run.smithi203.stdout:============================================================================================================ 2023-12-01T01:05:57.989 INFO:teuthology.orchestra.run.smithi203.stdout: Package Arch Version Repository Size 2023-12-01T01:05:57.989 INFO:teuthology.orchestra.run.smithi203.stdout:============================================================================================================ 2023-12-01T01:05:57.989 INFO:teuthology.orchestra.run.smithi203.stdout:Installing group/module packages: 2023-12-01T01:05:57.990 INFO:teuthology.orchestra.run.smithi203.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-12-01T01:05:57.990 INFO:teuthology.orchestra.run.smithi203.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-12-01T01:05:57.990 INFO:teuthology.orchestra.run.smithi203.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-12-01T01:05:57.990 INFO:teuthology.orchestra.run.smithi203.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-12-01T01:05:57.990 INFO:teuthology.orchestra.run.smithi203.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-12-01T01:05:57.990 INFO:teuthology.orchestra.run.smithi203.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-12-01T01:05:57.990 INFO:teuthology.orchestra.run.smithi203.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-12-01T01:05:57.990 INFO:teuthology.orchestra.run.smithi203.stdout:Installing dependencies: 2023-12-01T01:05:57.990 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-12-01T01:05:57.990 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-12-01T01:05:57.990 INFO:teuthology.orchestra.run.smithi203.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-12-01T01:05:57.990 INFO:teuthology.orchestra.run.smithi203.stdout:Installing module profiles: 2023-12-01T01:05:57.990 INFO:teuthology.orchestra.run.smithi203.stdout: container-tools/common 2023-12-01T01:05:57.990 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-12-01T01:05:57.991 INFO:teuthology.orchestra.run.smithi203.stdout:Enabling module streams: 2023-12-01T01:05:57.991 INFO:teuthology.orchestra.run.smithi203.stdout: container-tools rhel8 2023-12-01T01:05:57.991 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-12-01T01:05:57.991 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2023-12-01T01:05:57.991 INFO:teuthology.orchestra.run.smithi203.stdout:============================================================================================================ 2023-12-01T01:05:57.991 INFO:teuthology.orchestra.run.smithi203.stdout:Install 10 Packages 2023-12-01T01:05:57.991 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-12-01T01:05:57.992 INFO:teuthology.orchestra.run.smithi203.stdout:Total download size: 18 M 2023-12-01T01:05:57.992 INFO:teuthology.orchestra.run.smithi203.stdout:Installed size: 65 M 2023-12-01T01:05:57.992 INFO:teuthology.orchestra.run.smithi203.stdout:Downloading Packages: 2023-12-01T01:05:58.025 INFO:teuthology.orchestra.run.smithi137.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 17 MB/s | 2.2 MB 00:00 2023-12-01T01:05:58.056 INFO:teuthology.orchestra.run.smithi203.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 638 kB/s | 41 kB 00:00 2023-12-01T01:05:58.073 INFO:teuthology.orchestra.run.smithi203.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 308 kB/s | 25 kB 00:00 2023-12-01T01:05:58.090 INFO:teuthology.orchestra.run.smithi203.stdout:(3/10): udica-0.2.6-4.module+el8.6.0+16771+28df 1.4 MB/s | 49 kB 00:00 2023-12-01T01:05:58.107 INFO:teuthology.orchestra.run.smithi203.stdout:(4/10): python3-pyxdg-0.25-16.el8.noarch.rpm 817 kB/s | 94 kB 00:00 2023-12-01T01:05:58.127 INFO:teuthology.orchestra.run.smithi137.stdout:(9/10): buildah-1.26.2-1.module+el8.6.0+16771+2 23 MB/s | 8.0 MB 00:00 2023-12-01T01:05:58.140 INFO:teuthology.orchestra.run.smithi203.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 7.6 MB/s | 524 kB 00:00 2023-12-01T01:05:58.166 INFO:teuthology.orchestra.run.smithi203.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-12-01T01:05:58.191 INFO:teuthology.orchestra.run.smithi203.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.1 MB/s | 213 kB 00:00 2023-12-01T01:05:58.210 INFO:teuthology.orchestra.run.smithi137.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 20 MB/s | 7.0 MB 00:00 2023-12-01T01:05:58.211 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2023-12-01T01:05:58.211 INFO:teuthology.orchestra.run.smithi137.stdout:Total 30 MB/s | 18 MB 00:00 2023-12-01T01:05:58.245 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2023-12-01T01:05:58.283 INFO:teuthology.orchestra.run.smithi203.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 24 MB/s | 2.2 MB 00:00 2023-12-01T01:05:58.336 INFO:teuthology.orchestra.run.smithi077.stdout:============================================================================================================ 2023-12-01T01:05:58.336 INFO:teuthology.orchestra.run.smithi077.stdout: Package Arch Version Repository Size 2023-12-01T01:05:58.336 INFO:teuthology.orchestra.run.smithi077.stdout:============================================================================================================ 2023-12-01T01:05:58.336 INFO:teuthology.orchestra.run.smithi077.stdout:Installing group/module packages: 2023-12-01T01:05:58.336 INFO:teuthology.orchestra.run.smithi077.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-12-01T01:05:58.336 INFO:teuthology.orchestra.run.smithi077.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-12-01T01:05:58.337 INFO:teuthology.orchestra.run.smithi077.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-12-01T01:05:58.337 INFO:teuthology.orchestra.run.smithi077.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-12-01T01:05:58.337 INFO:teuthology.orchestra.run.smithi077.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-12-01T01:05:58.337 INFO:teuthology.orchestra.run.smithi077.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-12-01T01:05:58.337 INFO:teuthology.orchestra.run.smithi077.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-12-01T01:05:58.337 INFO:teuthology.orchestra.run.smithi077.stdout:Installing dependencies: 2023-12-01T01:05:58.337 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-12-01T01:05:58.337 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-12-01T01:05:58.337 INFO:teuthology.orchestra.run.smithi077.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-12-01T01:05:58.337 INFO:teuthology.orchestra.run.smithi077.stdout:Installing module profiles: 2023-12-01T01:05:58.337 INFO:teuthology.orchestra.run.smithi077.stdout: container-tools/common 2023-12-01T01:05:58.337 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:05:58.337 INFO:teuthology.orchestra.run.smithi077.stdout:Enabling module streams: 2023-12-01T01:05:58.337 INFO:teuthology.orchestra.run.smithi077.stdout: container-tools rhel8 2023-12-01T01:05:58.338 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:05:58.338 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2023-12-01T01:05:58.338 INFO:teuthology.orchestra.run.smithi077.stdout:============================================================================================================ 2023-12-01T01:05:58.338 INFO:teuthology.orchestra.run.smithi077.stdout:Install 10 Packages 2023-12-01T01:05:58.338 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:05:58.338 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 18 M 2023-12-01T01:05:58.339 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 65 M 2023-12-01T01:05:58.339 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2023-12-01T01:05:58.400 INFO:teuthology.orchestra.run.smithi203.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 30 MB/s | 7.0 MB 00:00 2023-12-01T01:05:58.411 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2023-12-01T01:05:58.430 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2023-12-01T01:05:58.430 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2023-12-01T01:05:58.496 INFO:teuthology.orchestra.run.smithi077.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 597 kB/s | 94 kB 00:00 2023-12-01T01:05:58.513 INFO:teuthology.orchestra.run.smithi077.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 143 kB/s | 25 kB 00:00 2023-12-01T01:05:58.517 INFO:teuthology.orchestra.run.smithi203.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 19 MB/s | 8.0 MB 00:00 2023-12-01T01:05:58.517 INFO:teuthology.orchestra.run.smithi203.stdout:-------------------------------------------------------------------------------- 2023-12-01T01:05:58.517 INFO:teuthology.orchestra.run.smithi203.stdout:Total 35 MB/s | 18 MB 00:00 2023-12-01T01:05:58.530 INFO:teuthology.orchestra.run.smithi077.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 214 kB/s | 41 kB 00:00 2023-12-01T01:05:58.547 INFO:teuthology.orchestra.run.smithi077.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 976 kB/s | 49 kB 00:00 2023-12-01T01:05:58.580 INFO:teuthology.orchestra.run.smithi077.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 7.6 MB/s | 524 kB 00:00 2023-12-01T01:05:58.605 INFO:teuthology.orchestra.run.smithi077.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-12-01T01:05:58.613 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2023-12-01T01:05:58.618 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2023-12-01T01:05:58.631 INFO:teuthology.orchestra.run.smithi077.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.1 MB/s | 213 kB 00:00 2023-12-01T01:05:58.717 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction check 2023-12-01T01:05:58.731 INFO:teuthology.orchestra.run.smithi077.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 22 MB/s | 2.2 MB 00:00 2023-12-01T01:05:58.737 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction check succeeded. 2023-12-01T01:05:58.737 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction test 2023-12-01T01:05:58.848 INFO:teuthology.orchestra.run.smithi077.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 29 MB/s | 7.0 MB 00:00 2023-12-01T01:05:58.917 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction test succeeded. 2023-12-01T01:05:58.921 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction 2023-12-01T01:05:58.940 INFO:teuthology.orchestra.run.smithi077.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 19 MB/s | 8.0 MB 00:00 2023-12-01T01:05:58.940 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2023-12-01T01:05:58.940 INFO:teuthology.orchestra.run.smithi077.stdout:Total 30 MB/s | 18 MB 00:00 2023-12-01T01:05:59.138 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2023-12-01T01:05:59.149 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2023-12-01T01:05:59.156 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2023-12-01T01:05:59.157 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2023-12-01T01:05:59.337 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2023-12-01T01:05:59.341 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2023-12-01T01:05:59.348 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-12-01T01:05:59.440 INFO:teuthology.orchestra.run.smithi203.stdout: Preparing : 1/1 2023-12-01T01:05:59.554 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-12-01T01:05:59.613 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-12-01T01:05:59.759 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-12-01T01:05:59.765 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-12-01T01:05:59.834 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2023-12-01T01:05:59.968 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-12-01T01:05:59.995 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-12-01T01:06:00.032 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-12-01T01:06:00.151 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-12-01T01:06:00.199 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-12-01T01:06:00.371 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-12-01T01:06:00.410 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-12-01T01:06:00.512 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-12-01T01:06:00.621 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-12-01T01:06:00.977 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-12-01T01:06:01.152 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-12-01T01:06:01.249 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-12-01T01:06:01.750 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-12-01T01:06:02.066 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-12-01T01:06:02.123 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-12-01T01:06:02.299 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-12-01T01:06:02.338 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-12-01T01:06:02.464 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-12-01T01:06:02.497 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-12-01T01:06:02.641 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-12-01T01:06:02.648 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-12-01T01:06:02.682 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-12-01T01:06:02.858 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-12-01T01:06:02.904 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-12-01T01:06:02.904 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-12-01T01:06:02.904 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-12-01T01:06:02.905 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-12-01T01:06:02.905 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-12-01T01:06:02.905 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-12-01T01:06:02.905 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-12-01T01:06:02.905 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-12-01T01:06:02.905 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-12-01T01:06:02.905 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-12-01T01:06:02.947 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-12-01T01:06:02.947 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-12-01T01:06:02.948 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-12-01T01:06:02.948 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-12-01T01:06:02.948 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-12-01T01:06:02.948 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-12-01T01:06:02.948 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-12-01T01:06:02.948 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-12-01T01:06:02.948 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-12-01T01:06:02.948 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-12-01T01:06:03.023 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-12-01T01:06:03.216 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-12-01T01:06:03.482 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-12-01T01:06:03.482 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-12-01T01:06:03.483 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-12-01T01:06:03.483 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-12-01T01:06:03.483 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-12-01T01:06:03.483 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-12-01T01:06:03.483 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-12-01T01:06:03.483 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-12-01T01:06:03.483 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-12-01T01:06:03.483 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-12-01T01:06:04.117 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-12-01T01:06:04.118 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:02:00 ago on Fri 01 Dec 2023 01:04:03 AM UTC. 2023-12-01T01:06:04.167 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-12-01T01:06:04.168 INFO:teuthology.orchestra.run.smithi203.stdout:Last metadata expiration check: 0:01:51 ago on Fri 01 Dec 2023 01:04:12 AM UTC. 2023-12-01T01:06:04.566 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-12-01T01:06:04.566 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:02:14 ago on Fri 01 Dec 2023 01:03:50 AM UTC. 2023-12-01T01:06:07.924 INFO:teuthology.orchestra.run.smithi137.stdout:Installed products updated. 2023-12-01T01:06:07.936 INFO:teuthology.orchestra.run.smithi203.stdout:Installed products updated. 2023-12-01T01:06:07.979 INFO:teuthology.orchestra.run.smithi077.stdout:Installed products updated. 2023-12-01T01:06:08.124 INFO:teuthology.orchestra.run.smithi137.stdout: 2023-12-01T01:06:08.124 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2023-12-01T01:06:08.124 INFO:teuthology.orchestra.run.smithi137.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-01T01:06:08.125 INFO:teuthology.orchestra.run.smithi137.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-12-01T01:06:08.125 INFO:teuthology.orchestra.run.smithi137.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-01T01:06:08.125 INFO:teuthology.orchestra.run.smithi137.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-12-01T01:06:08.125 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-12-01T01:06:08.125 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-12-01T01:06:08.125 INFO:teuthology.orchestra.run.smithi137.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-01T01:06:08.125 INFO:teuthology.orchestra.run.smithi137.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-12-01T01:06:08.125 INFO:teuthology.orchestra.run.smithi137.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-12-01T01:06:08.125 INFO:teuthology.orchestra.run.smithi137.stdout: yajl-2.1.0-10.el8.x86_64 2023-12-01T01:06:08.125 INFO:teuthology.orchestra.run.smithi137.stdout: 2023-12-01T01:06:08.125 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2023-12-01T01:06:08.131 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-12-01T01:06:08.131 INFO:teuthology.orchestra.run.smithi203.stdout:Installed: 2023-12-01T01:06:08.131 INFO:teuthology.orchestra.run.smithi203.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-01T01:06:08.131 INFO:teuthology.orchestra.run.smithi203.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-12-01T01:06:08.132 INFO:teuthology.orchestra.run.smithi203.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-01T01:06:08.132 INFO:teuthology.orchestra.run.smithi203.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-12-01T01:06:08.132 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-12-01T01:06:08.132 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-12-01T01:06:08.132 INFO:teuthology.orchestra.run.smithi203.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-01T01:06:08.132 INFO:teuthology.orchestra.run.smithi203.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-12-01T01:06:08.132 INFO:teuthology.orchestra.run.smithi203.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-12-01T01:06:08.132 INFO:teuthology.orchestra.run.smithi203.stdout: yajl-2.1.0-10.el8.x86_64 2023-12-01T01:06:08.132 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-12-01T01:06:08.132 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2023-12-01T01:06:08.169 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:06:08.169 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2023-12-01T01:06:08.169 INFO:teuthology.orchestra.run.smithi077.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-01T01:06:08.169 INFO:teuthology.orchestra.run.smithi077.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-12-01T01:06:08.169 INFO:teuthology.orchestra.run.smithi077.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-01T01:06:08.169 INFO:teuthology.orchestra.run.smithi077.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-12-01T01:06:08.169 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-12-01T01:06:08.169 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-12-01T01:06:08.169 INFO:teuthology.orchestra.run.smithi077.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-01T01:06:08.170 INFO:teuthology.orchestra.run.smithi077.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-12-01T01:06:08.170 INFO:teuthology.orchestra.run.smithi077.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-12-01T01:06:08.170 INFO:teuthology.orchestra.run.smithi077.stdout: yajl-2.1.0-10.el8.x86_64 2023-12-01T01:06:08.170 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:06:08.170 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2023-12-01T01:06:08.279 DEBUG:teuthology.parallel:result is None 2023-12-01T01:06:08.280 DEBUG:teuthology.parallel:result is None 2023-12-01T01:06:08.302 DEBUG:teuthology.parallel:result is None 2023-12-01T01:06:08.302 INFO:teuthology.run_tasks:Running task install... 2023-12-01T01:06:08.314 DEBUG:teuthology.task.install:project ceph 2023-12-01T01:06:08.314 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a6f82997ac57b70a895455dfed6256360a1e4c32'}} 2023-12-01T01:06:08.314 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a6f82997ac57b70a895455dfed6256360a1e4c32'} 2023-12-01T01:06:08.314 INFO:teuthology.task.install:Using flavor: default 2023-12-01T01:06:08.321 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-12-01T01:06:08.321 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-12-01T01:06:08.321 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a6f82997ac57b70a895455dfed6256360a1e4c32', 'tag': None, 'wait_for_package': False} 2023-12-01T01:06:08.322 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a6f82997ac57b70a895455dfed6256360a1e4c32 2023-12-01T01:06:08.324 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a6f82997ac57b70a895455dfed6256360a1e4c32', 'tag': None, 'wait_for_package': False} 2023-12-01T01:06:08.324 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a6f82997ac57b70a895455dfed6256360a1e4c32 2023-12-01T01:06:08.325 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a6f82997ac57b70a895455dfed6256360a1e4c32', 'tag': None, 'wait_for_package': False} 2023-12-01T01:06:08.325 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a6f82997ac57b70a895455dfed6256360a1e4c32 2023-12-01T01:06:08.475 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/reef-release/a6f82997ac57b70a895455dfed6256360a1e4c32/centos/8/flavors/default/ 2023-12-01T01:06:08.476 INFO:teuthology.task.install.rpm:Package version is 18.2.0-1249.ga6f82997 2023-12-01T01:06:08.483 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/reef-release/a6f82997ac57b70a895455dfed6256360a1e4c32/centos/8/flavors/default/ 2023-12-01T01:06:08.483 INFO:teuthology.task.install.rpm:Package version is 18.2.0-1249.ga6f82997 2023-12-01T01:06:08.484 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/reef-release/a6f82997ac57b70a895455dfed6256360a1e4c32/centos/8/flavors/default/ 2023-12-01T01:06:08.484 INFO:teuthology.task.install.rpm:Package version is 18.2.0-1249.ga6f82997 2023-12-01T01:06:08.600 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/reef-release/a6f82997ac57b70a895455dfed6256360a1e4c32/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/reef-release/a6f82997ac57b70a895455dfed6256360a1e4c32/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/reef-release/a6f82997ac57b70a895455dfed6256360a1e4c32/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-12-01T01:06:08.600 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2023-12-01T01:06:08.600 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-12-01T01:06:08.608 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/reef-release/a6f82997ac57b70a895455dfed6256360a1e4c32/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/reef-release/a6f82997ac57b70a895455dfed6256360a1e4c32/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/reef-release/a6f82997ac57b70a895455dfed6256360a1e4c32/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-12-01T01:06:08.608 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-12-01T01:06:08.608 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-12-01T01:06:08.630 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-12-01T01:06:08.631 DEBUG:teuthology.orchestra.run.smithi137:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a6f82997ac57b70a895455dfed6256360a1e4c32/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-12-01T01:06:08.636 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-12-01T01:06:08.636 DEBUG:teuthology.orchestra.run.smithi203:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a6f82997ac57b70a895455dfed6256360a1e4c32/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-12-01T01:06:08.668 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/reef-release/a6f82997ac57b70a895455dfed6256360a1e4c32/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/reef-release/a6f82997ac57b70a895455dfed6256360a1e4c32/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/reef-release/a6f82997ac57b70a895455dfed6256360a1e4c32/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-12-01T01:06:08.668 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-12-01T01:06:08.669 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-12-01T01:06:08.696 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-12-01T01:06:08.696 DEBUG:teuthology.orchestra.run.smithi077:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a6f82997ac57b70a895455dfed6256360a1e4c32/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-12-01T01:06:08.698 DEBUG:teuthology.orchestra.run.smithi137:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-12-01T01:06:08.705 DEBUG:teuthology.orchestra.run.smithi203:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-12-01T01:06:08.763 DEBUG:teuthology.orchestra.run.smithi077:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-12-01T01:06:08.775 DEBUG:teuthology.orchestra.run.smithi137:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-12-01T01:06:08.783 DEBUG:teuthology.orchestra.run.smithi203:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-12-01T01:06:08.840 DEBUG:teuthology.orchestra.run.smithi077:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-12-01T01:06:08.841 INFO:teuthology.orchestra.run.smithi137.stdout:check_obsoletes = 1 2023-12-01T01:06:08.843 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum clean all 2023-12-01T01:06:08.849 INFO:teuthology.orchestra.run.smithi203.stdout:check_obsoletes = 1 2023-12-01T01:06:08.851 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum clean all 2023-12-01T01:06:08.906 INFO:teuthology.orchestra.run.smithi077.stdout:check_obsoletes = 1 2023-12-01T01:06:08.907 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum clean all 2023-12-01T01:06:09.213 INFO:teuthology.orchestra.run.smithi137.stdout:Updating Subscription Management repositories. 2023-12-01T01:06:09.222 INFO:teuthology.orchestra.run.smithi203.stdout:Updating Subscription Management repositories. 2023-12-01T01:06:09.277 INFO:teuthology.orchestra.run.smithi077.stdout:Updating Subscription Management repositories. 2023-12-01T01:06:09.490 INFO:teuthology.orchestra.run.smithi137.stdout:44 files removed 2023-12-01T01:06:09.499 INFO:teuthology.orchestra.run.smithi203.stdout:44 files removed 2023-12-01T01:06:09.537 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-12-01T01:06:09.544 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-12-01T01:06:09.556 INFO:teuthology.orchestra.run.smithi077.stdout:44 files removed 2023-12-01T01:06:09.603 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-12-01T01:06:09.867 INFO:teuthology.orchestra.run.smithi137.stdout:Updating Subscription Management repositories. 2023-12-01T01:06:09.877 INFO:teuthology.orchestra.run.smithi203.stdout:Updating Subscription Management repositories. 2023-12-01T01:06:09.935 INFO:teuthology.orchestra.run.smithi077.stdout:Updating Subscription Management repositories. 2023-12-01T01:06:10.663 INFO:teuthology.orchestra.run.smithi203.stdout:ceph packages for x86_64 231 kB/s | 80 kB 00:00 2023-12-01T01:06:10.741 INFO:teuthology.orchestra.run.smithi077.stdout:ceph packages for x86_64 192 kB/s | 80 kB 00:00 2023-12-01T01:06:10.787 INFO:teuthology.orchestra.run.smithi137.stdout:ceph packages for x86_64 194 kB/s | 80 kB 00:00 2023-12-01T01:06:10.972 INFO:teuthology.orchestra.run.smithi203.stdout:ceph noarch packages 34 kB/s | 9.5 kB 00:00 2023-12-01T01:06:11.062 INFO:teuthology.orchestra.run.smithi137.stdout:ceph noarch packages 39 kB/s | 9.5 kB 00:00 2023-12-01T01:06:11.125 INFO:teuthology.orchestra.run.smithi077.stdout:ceph noarch packages 27 kB/s | 9.5 kB 00:00 2023-12-01T01:06:11.248 INFO:teuthology.orchestra.run.smithi203.stdout:ceph source packages 5.8 kB/s | 1.5 kB 00:00 2023-12-01T01:06:11.367 INFO:teuthology.orchestra.run.smithi077.stdout:ceph source packages 6.7 kB/s | 1.5 kB 00:00 2023-12-01T01:06:11.407 INFO:teuthology.orchestra.run.smithi203.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2023-12-01T01:06:11.438 INFO:teuthology.orchestra.run.smithi137.stdout:ceph source packages 4.2 kB/s | 1.5 kB 00:00 2023-12-01T01:06:11.534 INFO:teuthology.orchestra.run.smithi077.stdout:Copr repo for python3-asyncssh owned by ceph 23 kB/s | 3.5 kB 00:00 2023-12-01T01:06:11.597 INFO:teuthology.orchestra.run.smithi137.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2023-12-01T01:06:13.504 INFO:teuthology.orchestra.run.smithi077.stdout:Extra Packages for Enterprise Linux 8.2 MB/s | 16 MB 00:01 2023-12-01T01:06:16.664 INFO:teuthology.orchestra.run.smithi137.stdout:Extra Packages for Enterprise Linux 3.2 MB/s | 16 MB 00:05 2023-12-01T01:06:17.044 INFO:teuthology.orchestra.run.smithi077.stdout:lab-extras 568 kB/s | 24 kB 00:00 2023-12-01T01:06:17.913 INFO:teuthology.orchestra.run.smithi077.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 59 MB/s | 47 MB 00:00 2023-12-01T01:06:20.205 INFO:teuthology.orchestra.run.smithi137.stdout:lab-extras 432 kB/s | 24 kB 00:00 2023-12-01T01:06:21.415 INFO:teuthology.orchestra.run.smithi137.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 47 MB/s | 53 MB 00:01 2023-12-01T01:06:26.097 INFO:teuthology.orchestra.run.smithi077.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 49 MB/s | 53 MB 00:01 2023-12-01T01:06:26.265 INFO:teuthology.orchestra.run.smithi203.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 16 MB 00:14 2023-12-01T01:06:29.765 INFO:teuthology.orchestra.run.smithi203.stdout:lab-extras 352 kB/s | 24 kB 00:00 2023-12-01T01:06:29.858 INFO:teuthology.orchestra.run.smithi137.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 44 MB/s | 47 MB 00:01 2023-12-01T01:06:31.034 INFO:teuthology.orchestra.run.smithi203.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 44 MB/s | 53 MB 00:01 2023-12-01T01:06:37.254 INFO:teuthology.orchestra.run.smithi077.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-01T01:06:37.254 INFO:teuthology.orchestra.run.smithi077.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-01T01:06:37.269 INFO:teuthology.orchestra.run.smithi077.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-12-01T01:06:37.270 INFO:teuthology.orchestra.run.smithi077.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-12-01T01:06:37.272 INFO:teuthology.orchestra.run.smithi077.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-12-01T01:06:37.273 INFO:teuthology.orchestra.run.smithi077.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-12-01T01:06:37.275 INFO:teuthology.orchestra.run.smithi077.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-12-01T01:06:37.279 INFO:teuthology.orchestra.run.smithi077.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-12-01T01:06:37.280 INFO:teuthology.orchestra.run.smithi077.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-12-01T01:06:37.412 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2023-12-01T01:06:37.424 INFO:teuthology.orchestra.run.smithi077.stdout:=========================================================================================================================== 2023-12-01T01:06:37.424 INFO:teuthology.orchestra.run.smithi077.stdout: Package Arch Version Repository Size 2023-12-01T01:06:37.424 INFO:teuthology.orchestra.run.smithi077.stdout:=========================================================================================================================== 2023-12-01T01:06:37.425 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2023-12-01T01:06:37.425 INFO:teuthology.orchestra.run.smithi077.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-12-01T01:06:37.425 INFO:teuthology.orchestra.run.smithi077.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-12-01T01:06:37.425 INFO:teuthology.orchestra.run.smithi077.stdout: ceph x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 6.4 k 2023-12-01T01:06:37.425 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-base x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 5.4 M 2023-12-01T01:06:37.425 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-fuse x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 851 k 2023-12-01T01:06:37.425 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-immutable-object-cache x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 165 k 2023-12-01T01:06:37.425 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 1.6 M 2023-12-01T01:06:37.425 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-cephadm noarch 2:18.2.0-1249.ga6f82997.el8 ceph-noarch 132 k 2023-12-01T01:06:37.425 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-dashboard noarch 2:18.2.0-1249.ga6f82997.el8 ceph-noarch 1.8 M 2023-12-01T01:06:37.425 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.0-1249.ga6f82997.el8 ceph-noarch 7.4 M 2023-12-01T01:06:37.425 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-rook noarch 2:18.2.0-1249.ga6f82997.el8 ceph-noarch 50 k 2023-12-01T01:06:37.425 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-radosgw x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 13 M 2023-12-01T01:06:37.426 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-test x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 47 M 2023-12-01T01:06:37.426 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-volume noarch 2:18.2.0-1249.ga6f82997.el8 ceph-noarch 262 k 2023-12-01T01:06:37.426 INFO:teuthology.orchestra.run.smithi077.stdout: cephadm noarch 2:18.2.0-1249.ga6f82997.el8 ceph-noarch 168 k 2023-12-01T01:06:37.426 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-mirror x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 258 k 2023-12-01T01:06:37.426 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-top noarch 2:18.2.0-1249.ga6f82997.el8 ceph-noarch 18 k 2023-12-01T01:06:37.426 INFO:teuthology.orchestra.run.smithi077.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-12-01T01:06:37.426 INFO:teuthology.orchestra.run.smithi077.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-12-01T01:06:37.426 INFO:teuthology.orchestra.run.smithi077.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-12-01T01:06:37.426 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs-devel x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 31 k 2023-12-01T01:06:37.426 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs2 x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 774 k 2023-12-01T01:06:37.426 INFO:teuthology.orchestra.run.smithi077.stdout: librados-devel x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 139 k 2023-12-01T01:06:37.426 INFO:teuthology.orchestra.run.smithi077.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-12-01T01:06:37.426 INFO:teuthology.orchestra.run.smithi077.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-12-01T01:06:37.426 INFO:teuthology.orchestra.run.smithi077.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-12-01T01:06:37.426 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-12-01T01:06:37.427 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-12-01T01:06:37.427 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-12-01T01:06:37.427 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cephfs x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 196 k 2023-12-01T01:06:37.427 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rados x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 371 k 2023-12-01T01:06:37.427 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rbd x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 364 k 2023-12-01T01:06:37.427 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rgw x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 107 k 2023-12-01T01:06:37.427 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-fuse x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 94 k 2023-12-01T01:06:37.427 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-mirror x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 3.4 M 2023-12-01T01:06:37.427 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-nbd x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 181 k 2023-12-01T01:06:37.427 INFO:teuthology.orchestra.run.smithi077.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-12-01T01:06:37.427 INFO:teuthology.orchestra.run.smithi077.stdout:Upgrading: 2023-12-01T01:06:37.427 INFO:teuthology.orchestra.run.smithi077.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-12-01T01:06:37.427 INFO:teuthology.orchestra.run.smithi077.stdout: librados2 x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 3.7 M 2023-12-01T01:06:37.427 INFO:teuthology.orchestra.run.smithi077.stdout: librbd1 x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 3.6 M 2023-12-01T01:06:37.428 INFO:teuthology.orchestra.run.smithi077.stdout:Installing dependencies: 2023-12-01T01:06:37.428 INFO:teuthology.orchestra.run.smithi077.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-12-01T01:06:37.428 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-common x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 22 M 2023-12-01T01:06:37.428 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-grafana-dashboards noarch 2:18.2.0-1249.ga6f82997.el8 ceph-noarch 23 k 2023-12-01T01:06:37.428 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mds x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 2.3 M 2023-12-01T01:06:37.428 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-modules-core noarch 2:18.2.0-1249.ga6f82997.el8 ceph-noarch 242 k 2023-12-01T01:06:37.428 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mon x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 4.6 M 2023-12-01T01:06:37.428 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-osd x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 18 M 2023-12-01T01:06:37.428 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-prometheus-alerts noarch 2:18.2.0-1249.ga6f82997.el8 ceph-noarch 15 k 2023-12-01T01:06:37.428 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-selinux x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 24 k 2023-12-01T01:06:37.428 INFO:teuthology.orchestra.run.smithi077.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-12-01T01:06:37.428 INFO:teuthology.orchestra.run.smithi077.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-12-01T01:06:37.428 INFO:teuthology.orchestra.run.smithi077.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-12-01T01:06:37.428 INFO:teuthology.orchestra.run.smithi077.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-12-01T01:06:37.429 INFO:teuthology.orchestra.run.smithi077.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-12-01T01:06:37.429 INFO:teuthology.orchestra.run.smithi077.stdout: libcephsqlite x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 141 k 2023-12-01T01:06:37.429 INFO:teuthology.orchestra.run.smithi077.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-12-01T01:06:37.429 INFO:teuthology.orchestra.run.smithi077.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-12-01T01:06:37.429 INFO:teuthology.orchestra.run.smithi077.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-12-01T01:06:37.429 INFO:teuthology.orchestra.run.smithi077.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-12-01T01:06:37.429 INFO:teuthology.orchestra.run.smithi077.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-12-01T01:06:37.429 INFO:teuthology.orchestra.run.smithi077.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-12-01T01:06:37.429 INFO:teuthology.orchestra.run.smithi077.stdout: libradosstriper1 x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 474 k 2023-12-01T01:06:37.429 INFO:teuthology.orchestra.run.smithi077.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-12-01T01:06:37.429 INFO:teuthology.orchestra.run.smithi077.stdout: librgw2 x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 6.6 M 2023-12-01T01:06:37.429 INFO:teuthology.orchestra.run.smithi077.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-12-01T01:06:37.429 INFO:teuthology.orchestra.run.smithi077.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-12-01T01:06:37.430 INFO:teuthology.orchestra.run.smithi077.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-12-01T01:06:37.430 INFO:teuthology.orchestra.run.smithi077.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-12-01T01:06:37.430 INFO:teuthology.orchestra.run.smithi077.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-12-01T01:06:37.430 INFO:teuthology.orchestra.run.smithi077.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-12-01T01:06:37.430 INFO:teuthology.orchestra.run.smithi077.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-12-01T01:06:37.430 INFO:teuthology.orchestra.run.smithi077.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-12-01T01:06:37.430 INFO:teuthology.orchestra.run.smithi077.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-12-01T01:06:37.430 INFO:teuthology.orchestra.run.smithi077.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-12-01T01:06:37.430 INFO:teuthology.orchestra.run.smithi077.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-12-01T01:06:37.430 INFO:teuthology.orchestra.run.smithi077.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-12-01T01:06:37.430 INFO:teuthology.orchestra.run.smithi077.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-12-01T01:06:37.430 INFO:teuthology.orchestra.run.smithi077.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-12-01T01:06:37.430 INFO:teuthology.orchestra.run.smithi077.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-12-01T01:06:37.430 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-12-01T01:06:37.431 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-argparse x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 45 k 2023-12-01T01:06:37.431 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-common x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 123 k 2023-12-01T01:06:37.431 INFO:teuthology.orchestra.run.smithi077.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-12-01T01:06:37.431 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-12-01T01:06:37.431 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-12-01T01:06:37.431 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-12-01T01:06:37.431 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-12-01T01:06:37.431 INFO:teuthology.orchestra.run.smithi077.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-12-01T01:06:37.431 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-12-01T01:06:37.431 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-12-01T01:06:37.431 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-12-01T01:06:37.431 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-12-01T01:06:37.431 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-12-01T01:06:37.431 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-01T01:06:37.432 INFO:teuthology.orchestra.run.smithi077.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-12-01T01:06:37.432 INFO:teuthology.orchestra.run.smithi077.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-12-01T01:06:37.432 INFO:teuthology.orchestra.run.smithi077.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-12-01T01:06:37.432 INFO:teuthology.orchestra.run.smithi077.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-12-01T01:06:37.432 INFO:teuthology.orchestra.run.smithi077.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-12-01T01:06:37.432 INFO:teuthology.orchestra.run.smithi077.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-12-01T01:06:37.432 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-12-01T01:06:37.432 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-12-01T01:06:37.432 INFO:teuthology.orchestra.run.smithi077.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-12-01T01:06:37.432 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-12-01T01:06:37.432 INFO:teuthology.orchestra.run.smithi077.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-12-01T01:06:37.432 INFO:teuthology.orchestra.run.smithi077.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-12-01T01:06:37.432 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-12-01T01:06:37.432 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-12-01T01:06:37.432 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-12-01T01:06:37.433 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-12-01T01:06:37.433 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-12-01T01:06:37.433 INFO:teuthology.orchestra.run.smithi077.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-12-01T01:06:37.433 INFO:teuthology.orchestra.run.smithi077.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-01T01:06:37.433 INFO:teuthology.orchestra.run.smithi077.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-12-01T01:06:37.433 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-12-01T01:06:37.433 INFO:teuthology.orchestra.run.smithi077.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-12-01T01:06:37.433 INFO:teuthology.orchestra.run.smithi077.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-12-01T01:06:37.433 INFO:teuthology.orchestra.run.smithi077.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-12-01T01:06:37.433 INFO:teuthology.orchestra.run.smithi077.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-12-01T01:06:37.433 INFO:teuthology.orchestra.run.smithi077.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-12-01T01:06:37.433 INFO:teuthology.orchestra.run.smithi077.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-12-01T01:06:37.433 INFO:teuthology.orchestra.run.smithi077.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-12-01T01:06:37.433 INFO:teuthology.orchestra.run.smithi077.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-12-01T01:06:37.433 INFO:teuthology.orchestra.run.smithi077.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-12-01T01:06:37.434 INFO:teuthology.orchestra.run.smithi077.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-12-01T01:06:37.434 INFO:teuthology.orchestra.run.smithi077.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-12-01T01:06:37.434 INFO:teuthology.orchestra.run.smithi077.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-12-01T01:06:37.434 INFO:teuthology.orchestra.run.smithi077.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-12-01T01:06:37.434 INFO:teuthology.orchestra.run.smithi077.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-12-01T01:06:37.434 INFO:teuthology.orchestra.run.smithi077.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-12-01T01:06:37.434 INFO:teuthology.orchestra.run.smithi077.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-12-01T01:06:37.434 INFO:teuthology.orchestra.run.smithi077.stdout:Enabling module streams: 2023-12-01T01:06:37.434 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql 10 2023-12-01T01:06:37.434 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:06:37.434 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2023-12-01T01:06:37.434 INFO:teuthology.orchestra.run.smithi077.stdout:=========================================================================================================================== 2023-12-01T01:06:37.434 INFO:teuthology.orchestra.run.smithi077.stdout:Install 128 Packages 2023-12-01T01:06:37.435 INFO:teuthology.orchestra.run.smithi077.stdout:Upgrade 3 Packages 2023-12-01T01:06:37.435 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:06:37.435 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 203 M 2023-12-01T01:06:37.435 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2023-12-01T01:06:37.664 INFO:teuthology.orchestra.run.smithi077.stdout:(1/131): ceph-18.2.0-1249.ga6f82997.el8.x86_64. 33 kB/s | 6.4 kB 00:00 2023-12-01T01:06:39.014 INFO:teuthology.orchestra.run.smithi077.stdout:(2/131): ceph-fuse-18.2.0-1249.ga6f82997.el8.x8 630 kB/s | 851 kB 00:01 2023-12-01T01:06:39.206 INFO:teuthology.orchestra.run.smithi077.stdout:(3/131): ceph-immutable-object-cache-18.2.0-124 863 kB/s | 165 kB 00:00 2023-12-01T01:06:39.419 INFO:teuthology.orchestra.run.smithi203.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 42 MB/s | 47 MB 00:01 2023-12-01T01:06:40.781 INFO:teuthology.orchestra.run.smithi137.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-01T01:06:40.783 INFO:teuthology.orchestra.run.smithi137.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-01T01:06:40.796 INFO:teuthology.orchestra.run.smithi137.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-12-01T01:06:40.797 INFO:teuthology.orchestra.run.smithi137.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-12-01T01:06:40.798 INFO:teuthology.orchestra.run.smithi137.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-12-01T01:06:40.800 INFO:teuthology.orchestra.run.smithi137.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-12-01T01:06:40.802 INFO:teuthology.orchestra.run.smithi137.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-12-01T01:06:40.805 INFO:teuthology.orchestra.run.smithi137.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-12-01T01:06:40.806 INFO:teuthology.orchestra.run.smithi137.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-12-01T01:06:40.939 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2023-12-01T01:06:40.951 INFO:teuthology.orchestra.run.smithi137.stdout:=========================================================================================================================== 2023-12-01T01:06:40.951 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2023-12-01T01:06:40.951 INFO:teuthology.orchestra.run.smithi137.stdout:=========================================================================================================================== 2023-12-01T01:06:40.952 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2023-12-01T01:06:40.952 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-12-01T01:06:40.952 INFO:teuthology.orchestra.run.smithi137.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-12-01T01:06:40.952 INFO:teuthology.orchestra.run.smithi137.stdout: ceph x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 6.4 k 2023-12-01T01:06:40.952 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-base x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 5.4 M 2023-12-01T01:06:40.952 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-fuse x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 851 k 2023-12-01T01:06:40.952 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-immutable-object-cache x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 165 k 2023-12-01T01:06:40.952 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 1.6 M 2023-12-01T01:06:40.952 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-cephadm noarch 2:18.2.0-1249.ga6f82997.el8 ceph-noarch 132 k 2023-12-01T01:06:40.952 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-dashboard noarch 2:18.2.0-1249.ga6f82997.el8 ceph-noarch 1.8 M 2023-12-01T01:06:40.952 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.0-1249.ga6f82997.el8 ceph-noarch 7.4 M 2023-12-01T01:06:40.952 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-rook noarch 2:18.2.0-1249.ga6f82997.el8 ceph-noarch 50 k 2023-12-01T01:06:40.952 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-radosgw x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 13 M 2023-12-01T01:06:40.953 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-test x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 47 M 2023-12-01T01:06:40.953 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-volume noarch 2:18.2.0-1249.ga6f82997.el8 ceph-noarch 262 k 2023-12-01T01:06:40.953 INFO:teuthology.orchestra.run.smithi137.stdout: cephadm noarch 2:18.2.0-1249.ga6f82997.el8 ceph-noarch 168 k 2023-12-01T01:06:40.953 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-mirror x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 258 k 2023-12-01T01:06:40.953 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-top noarch 2:18.2.0-1249.ga6f82997.el8 ceph-noarch 18 k 2023-12-01T01:06:40.953 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-12-01T01:06:40.953 INFO:teuthology.orchestra.run.smithi137.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-12-01T01:06:40.953 INFO:teuthology.orchestra.run.smithi137.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-12-01T01:06:40.953 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs-devel x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 31 k 2023-12-01T01:06:40.953 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs2 x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 774 k 2023-12-01T01:06:40.953 INFO:teuthology.orchestra.run.smithi137.stdout: librados-devel x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 139 k 2023-12-01T01:06:40.953 INFO:teuthology.orchestra.run.smithi137.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-12-01T01:06:40.953 INFO:teuthology.orchestra.run.smithi137.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-12-01T01:06:40.953 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-12-01T01:06:40.954 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-12-01T01:06:40.954 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-12-01T01:06:40.954 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-12-01T01:06:40.954 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cephfs x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 196 k 2023-12-01T01:06:40.954 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rados x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 371 k 2023-12-01T01:06:40.954 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rbd x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 364 k 2023-12-01T01:06:40.954 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rgw x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 107 k 2023-12-01T01:06:40.954 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-fuse x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 94 k 2023-12-01T01:06:40.954 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-mirror x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 3.4 M 2023-12-01T01:06:40.954 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-nbd x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 181 k 2023-12-01T01:06:40.954 INFO:teuthology.orchestra.run.smithi137.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-12-01T01:06:40.954 INFO:teuthology.orchestra.run.smithi137.stdout:Upgrading: 2023-12-01T01:06:40.954 INFO:teuthology.orchestra.run.smithi137.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-12-01T01:06:40.954 INFO:teuthology.orchestra.run.smithi137.stdout: librados2 x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 3.7 M 2023-12-01T01:06:40.955 INFO:teuthology.orchestra.run.smithi137.stdout: librbd1 x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 3.6 M 2023-12-01T01:06:40.955 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2023-12-01T01:06:40.955 INFO:teuthology.orchestra.run.smithi137.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-12-01T01:06:40.955 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-common x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 22 M 2023-12-01T01:06:40.955 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-grafana-dashboards noarch 2:18.2.0-1249.ga6f82997.el8 ceph-noarch 23 k 2023-12-01T01:06:40.955 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mds x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 2.3 M 2023-12-01T01:06:40.955 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-modules-core noarch 2:18.2.0-1249.ga6f82997.el8 ceph-noarch 242 k 2023-12-01T01:06:40.955 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mon x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 4.6 M 2023-12-01T01:06:40.955 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-osd x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 18 M 2023-12-01T01:06:40.955 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-prometheus-alerts noarch 2:18.2.0-1249.ga6f82997.el8 ceph-noarch 15 k 2023-12-01T01:06:40.955 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-selinux x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 24 k 2023-12-01T01:06:40.955 INFO:teuthology.orchestra.run.smithi137.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-12-01T01:06:40.955 INFO:teuthology.orchestra.run.smithi137.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-12-01T01:06:40.955 INFO:teuthology.orchestra.run.smithi137.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-12-01T01:06:40.955 INFO:teuthology.orchestra.run.smithi137.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-12-01T01:06:40.956 INFO:teuthology.orchestra.run.smithi137.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-12-01T01:06:40.956 INFO:teuthology.orchestra.run.smithi137.stdout: libcephsqlite x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 141 k 2023-12-01T01:06:40.956 INFO:teuthology.orchestra.run.smithi137.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-12-01T01:06:40.956 INFO:teuthology.orchestra.run.smithi137.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-12-01T01:06:40.956 INFO:teuthology.orchestra.run.smithi137.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-12-01T01:06:40.956 INFO:teuthology.orchestra.run.smithi137.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-12-01T01:06:40.956 INFO:teuthology.orchestra.run.smithi137.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-12-01T01:06:40.956 INFO:teuthology.orchestra.run.smithi137.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-12-01T01:06:40.956 INFO:teuthology.orchestra.run.smithi137.stdout: libradosstriper1 x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 474 k 2023-12-01T01:06:40.956 INFO:teuthology.orchestra.run.smithi137.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-12-01T01:06:40.956 INFO:teuthology.orchestra.run.smithi137.stdout: librgw2 x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 6.6 M 2023-12-01T01:06:40.956 INFO:teuthology.orchestra.run.smithi137.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-12-01T01:06:40.956 INFO:teuthology.orchestra.run.smithi137.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-12-01T01:06:40.956 INFO:teuthology.orchestra.run.smithi137.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-12-01T01:06:40.957 INFO:teuthology.orchestra.run.smithi137.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-12-01T01:06:40.957 INFO:teuthology.orchestra.run.smithi137.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-12-01T01:06:40.957 INFO:teuthology.orchestra.run.smithi137.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-12-01T01:06:40.957 INFO:teuthology.orchestra.run.smithi137.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-12-01T01:06:40.957 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-12-01T01:06:40.957 INFO:teuthology.orchestra.run.smithi137.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-12-01T01:06:40.957 INFO:teuthology.orchestra.run.smithi137.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-12-01T01:06:40.957 INFO:teuthology.orchestra.run.smithi137.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-12-01T01:06:40.957 INFO:teuthology.orchestra.run.smithi137.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-12-01T01:06:40.957 INFO:teuthology.orchestra.run.smithi137.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-12-01T01:06:40.957 INFO:teuthology.orchestra.run.smithi137.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-12-01T01:06:40.957 INFO:teuthology.orchestra.run.smithi137.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-12-01T01:06:40.957 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-12-01T01:06:40.957 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-argparse x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 45 k 2023-12-01T01:06:40.958 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-common x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 123 k 2023-12-01T01:06:40.958 INFO:teuthology.orchestra.run.smithi137.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-12-01T01:06:40.958 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-12-01T01:06:40.958 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-12-01T01:06:40.958 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-12-01T01:06:40.958 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-12-01T01:06:40.958 INFO:teuthology.orchestra.run.smithi137.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-12-01T01:06:40.958 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-12-01T01:06:40.958 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-12-01T01:06:40.958 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-12-01T01:06:40.958 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-12-01T01:06:40.958 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-12-01T01:06:40.958 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-01T01:06:40.958 INFO:teuthology.orchestra.run.smithi137.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-12-01T01:06:40.959 INFO:teuthology.orchestra.run.smithi137.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-12-01T01:06:40.959 INFO:teuthology.orchestra.run.smithi137.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-12-01T01:06:40.959 INFO:teuthology.orchestra.run.smithi137.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-12-01T01:06:40.959 INFO:teuthology.orchestra.run.smithi137.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-12-01T01:06:40.959 INFO:teuthology.orchestra.run.smithi137.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-12-01T01:06:40.959 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-12-01T01:06:40.959 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-12-01T01:06:40.959 INFO:teuthology.orchestra.run.smithi137.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-12-01T01:06:40.959 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-12-01T01:06:40.959 INFO:teuthology.orchestra.run.smithi137.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-12-01T01:06:40.959 INFO:teuthology.orchestra.run.smithi137.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-12-01T01:06:40.959 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-12-01T01:06:40.959 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-12-01T01:06:40.959 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-12-01T01:06:40.960 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-12-01T01:06:40.960 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-12-01T01:06:40.960 INFO:teuthology.orchestra.run.smithi137.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-12-01T01:06:40.960 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-01T01:06:40.960 INFO:teuthology.orchestra.run.smithi137.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-12-01T01:06:40.960 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-12-01T01:06:40.960 INFO:teuthology.orchestra.run.smithi137.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-12-01T01:06:40.960 INFO:teuthology.orchestra.run.smithi137.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-12-01T01:06:40.960 INFO:teuthology.orchestra.run.smithi137.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-12-01T01:06:40.960 INFO:teuthology.orchestra.run.smithi137.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-12-01T01:06:40.960 INFO:teuthology.orchestra.run.smithi137.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-12-01T01:06:40.960 INFO:teuthology.orchestra.run.smithi137.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-12-01T01:06:40.960 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-12-01T01:06:40.961 INFO:teuthology.orchestra.run.smithi137.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-12-01T01:06:40.961 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-12-01T01:06:40.961 INFO:teuthology.orchestra.run.smithi137.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-12-01T01:06:40.961 INFO:teuthology.orchestra.run.smithi137.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-12-01T01:06:40.961 INFO:teuthology.orchestra.run.smithi137.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-12-01T01:06:40.961 INFO:teuthology.orchestra.run.smithi137.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-12-01T01:06:40.961 INFO:teuthology.orchestra.run.smithi137.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-12-01T01:06:40.961 INFO:teuthology.orchestra.run.smithi137.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-12-01T01:06:40.961 INFO:teuthology.orchestra.run.smithi137.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-12-01T01:06:40.961 INFO:teuthology.orchestra.run.smithi137.stdout:Enabling module streams: 2023-12-01T01:06:40.961 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql 10 2023-12-01T01:06:40.961 INFO:teuthology.orchestra.run.smithi137.stdout: 2023-12-01T01:06:40.961 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2023-12-01T01:06:40.961 INFO:teuthology.orchestra.run.smithi137.stdout:=========================================================================================================================== 2023-12-01T01:06:40.962 INFO:teuthology.orchestra.run.smithi137.stdout:Install 128 Packages 2023-12-01T01:06:40.962 INFO:teuthology.orchestra.run.smithi137.stdout:Upgrade 3 Packages 2023-12-01T01:06:40.962 INFO:teuthology.orchestra.run.smithi137.stdout: 2023-12-01T01:06:40.962 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 203 M 2023-12-01T01:06:40.962 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2023-12-01T01:06:41.289 INFO:teuthology.orchestra.run.smithi077.stdout:(4/131): ceph-mds-18.2.0-1249.ga6f82997.el8.x86 1.1 MB/s | 2.3 MB 00:02 2023-12-01T01:06:41.367 INFO:teuthology.orchestra.run.smithi137.stdout:(1/131): ceph-18.2.0-1249.ga6f82997.el8.x86_64. 25 kB/s | 6.4 kB 00:00 2023-12-01T01:06:41.489 INFO:teuthology.orchestra.run.smithi077.stdout:(5/131): ceph-base-18.2.0-1249.ga6f82997.el8.x8 1.3 MB/s | 5.4 MB 00:04 2023-12-01T01:06:41.709 INFO:teuthology.orchestra.run.smithi137.stdout:(2/131): ceph-fuse-18.2.0-1249.ga6f82997.el8.x8 2.4 MB/s | 851 kB 00:00 2023-12-01T01:06:41.776 INFO:teuthology.orchestra.run.smithi137.stdout:(3/131): ceph-immutable-object-cache-18.2.0-124 2.4 MB/s | 165 kB 00:00 2023-12-01T01:06:42.231 INFO:teuthology.orchestra.run.smithi077.stdout:(6/131): ceph-mgr-18.2.0-1249.ga6f82997.el8.x86 1.6 MB/s | 1.6 MB 00:00 2023-12-01T01:06:42.251 INFO:teuthology.orchestra.run.smithi137.stdout:(4/131): ceph-mds-18.2.0-1249.ga6f82997.el8.x86 4.8 MB/s | 2.3 MB 00:00 2023-12-01T01:06:42.626 INFO:teuthology.orchestra.run.smithi137.stdout:(5/131): ceph-mgr-18.2.0-1249.ga6f82997.el8.x86 4.1 MB/s | 1.6 MB 00:00 2023-12-01T01:06:43.618 INFO:teuthology.orchestra.run.smithi137.stdout:(6/131): ceph-mon-18.2.0-1249.ga6f82997.el8.x86 4.7 MB/s | 4.6 MB 00:00 2023-12-01T01:06:44.402 INFO:teuthology.orchestra.run.smithi137.stdout:(7/131): ceph-common-18.2.0-1249.ga6f82997.el8. 6.6 MB/s | 22 MB 00:03 2023-12-01T01:06:44.515 INFO:teuthology.orchestra.run.smithi077.stdout:(7/131): ceph-common-18.2.0-1249.ga6f82997.el8. 3.1 MB/s | 22 MB 00:07 2023-12-01T01:06:44.969 INFO:teuthology.orchestra.run.smithi137.stdout:(8/131): ceph-base-18.2.0-1249.ga6f82997.el8.x8 1.4 MB/s | 5.4 MB 00:03 2023-12-01T01:06:45.077 INFO:teuthology.orchestra.run.smithi137.stdout:(9/131): ceph-selinux-18.2.0-1249.ga6f82997.el8 225 kB/s | 24 kB 00:00 2023-12-01T01:06:45.274 INFO:teuthology.orchestra.run.smithi077.stdout:(8/131): ceph-mon-18.2.0-1249.ga6f82997.el8.x86 1.2 MB/s | 4.6 MB 00:03 2023-12-01T01:06:45.365 INFO:teuthology.orchestra.run.smithi077.stdout:(9/131): ceph-selinux-18.2.0-1249.ga6f82997.el8 266 kB/s | 24 kB 00:00 2023-12-01T01:06:46.353 INFO:teuthology.orchestra.run.smithi137.stdout:(10/131): ceph-radosgw-18.2.0-1249.ga6f82997.el 6.7 MB/s | 13 MB 00:01 2023-12-01T01:06:46.453 INFO:teuthology.orchestra.run.smithi137.stdout:(11/131): cephfs-mirror-18.2.0-1249.ga6f82997.e 2.5 MB/s | 258 kB 00:00 2023-12-01T01:06:46.528 INFO:teuthology.orchestra.run.smithi137.stdout:(12/131): libcephfs-devel-18.2.0-1249.ga6f82997 412 kB/s | 31 kB 00:00 2023-12-01T01:06:46.803 INFO:teuthology.orchestra.run.smithi137.stdout:(13/131): ceph-osd-18.2.0-1249.ga6f82997.el8.x8 5.6 MB/s | 18 MB 00:03 2023-12-01T01:06:46.829 INFO:teuthology.orchestra.run.smithi137.stdout:(14/131): libcephfs2-18.2.0-1249.ga6f82997.el8. 2.5 MB/s | 774 kB 00:00 2023-12-01T01:06:46.862 INFO:teuthology.orchestra.run.smithi137.stdout:(15/131): libcephsqlite-18.2.0-1249.ga6f82997.e 2.4 MB/s | 141 kB 00:00 2023-12-01T01:06:46.929 INFO:teuthology.orchestra.run.smithi137.stdout:(16/131): librados-devel-18.2.0-1249.ga6f82997. 1.4 MB/s | 139 kB 00:00 2023-12-01T01:06:47.005 INFO:teuthology.orchestra.run.smithi137.stdout:(17/131): libradosstriper1-18.2.0-1249.ga6f8299 3.3 MB/s | 474 kB 00:00 2023-12-01T01:06:47.088 INFO:teuthology.orchestra.run.smithi137.stdout:(18/131): python3-ceph-argparse-18.2.0-1249.ga6 537 kB/s | 45 kB 00:00 2023-12-01T01:06:47.164 INFO:teuthology.orchestra.run.smithi137.stdout:(19/131): python3-ceph-common-18.2.0-1249.ga6f8 1.6 MB/s | 123 kB 00:00 2023-12-01T01:06:47.264 INFO:teuthology.orchestra.run.smithi137.stdout:(20/131): python3-cephfs-18.2.0-1249.ga6f82997. 1.9 MB/s | 196 kB 00:00 2023-12-01T01:06:47.364 INFO:teuthology.orchestra.run.smithi137.stdout:(21/131): python3-rados-18.2.0-1249.ga6f82997.e 3.6 MB/s | 371 kB 00:00 2023-12-01T01:06:47.464 INFO:teuthology.orchestra.run.smithi137.stdout:(22/131): python3-rbd-18.2.0-1249.ga6f82997.el8 3.6 MB/s | 364 kB 00:00 2023-12-01T01:06:47.565 INFO:teuthology.orchestra.run.smithi137.stdout:(23/131): python3-rgw-18.2.0-1249.ga6f82997.el8 1.0 MB/s | 107 kB 00:00 2023-12-01T01:06:47.657 INFO:teuthology.orchestra.run.smithi137.stdout:(24/131): rbd-fuse-18.2.0-1249.ga6f82997.el8.x8 1.0 MB/s | 94 kB 00:00 2023-12-01T01:06:48.090 INFO:teuthology.orchestra.run.smithi137.stdout:(25/131): librgw2-18.2.0-1249.ga6f82997.el8.x86 5.7 MB/s | 6.6 MB 00:01 2023-12-01T01:06:48.182 INFO:teuthology.orchestra.run.smithi137.stdout:(26/131): rbd-nbd-18.2.0-1249.ga6f82997.el8.x86 1.9 MB/s | 181 kB 00:00 2023-12-01T01:06:48.266 INFO:teuthology.orchestra.run.smithi137.stdout:(27/131): ceph-grafana-dashboards-18.2.0-1249.g 277 kB/s | 23 kB 00:00 2023-12-01T01:06:48.341 INFO:teuthology.orchestra.run.smithi137.stdout:(28/131): ceph-mgr-cephadm-18.2.0-1249.ga6f8299 1.7 MB/s | 132 kB 00:00 2023-12-01T01:06:48.475 INFO:teuthology.orchestra.run.smithi137.stdout:(29/131): rbd-mirror-18.2.0-1249.ga6f82997.el8. 4.2 MB/s | 3.4 MB 00:00 2023-12-01T01:06:48.731 INFO:teuthology.orchestra.run.smithi137.stdout:(30/131): ceph-mgr-dashboard-18.2.0-1249.ga6f82 4.5 MB/s | 1.8 MB 00:00 2023-12-01T01:06:48.817 INFO:teuthology.orchestra.run.smithi137.stdout:(31/131): ceph-mgr-modules-core-18.2.0-1249.ga6 2.8 MB/s | 242 kB 00:00 2023-12-01T01:06:48.876 INFO:teuthology.orchestra.run.smithi137.stdout:(32/131): ceph-mgr-rook-18.2.0-1249.ga6f82997.e 859 kB/s | 50 kB 00:00 2023-12-01T01:06:48.934 INFO:teuthology.orchestra.run.smithi137.stdout:(33/131): ceph-prometheus-alerts-18.2.0-1249.ga 250 kB/s | 15 kB 00:00 2023-12-01T01:06:48.941 INFO:teuthology.orchestra.run.smithi077.stdout:(10/131): ceph-radosgw-18.2.0-1249.ga6f82997.el 3.0 MB/s | 13 MB 00:04 2023-12-01T01:06:49.036 INFO:teuthology.orchestra.run.smithi137.stdout:(34/131): ceph-volume-18.2.0-1249.ga6f82997.el8 2.5 MB/s | 262 kB 00:00 2023-12-01T01:06:49.110 INFO:teuthology.orchestra.run.smithi137.stdout:(35/131): cephadm-18.2.0-1249.ga6f82997.el8.noa 2.3 MB/s | 168 kB 00:00 2023-12-01T01:06:49.141 INFO:teuthology.orchestra.run.smithi077.stdout:(11/131): cephfs-mirror-18.2.0-1249.ga6f82997.e 1.3 MB/s | 258 kB 00:00 2023-12-01T01:06:49.177 INFO:teuthology.orchestra.run.smithi137.stdout:(36/131): cephfs-top-18.2.0-1249.ga6f82997.el8. 270 kB/s | 18 kB 00:00 2023-12-01T01:06:49.241 INFO:teuthology.orchestra.run.smithi077.stdout:(12/131): libcephfs-devel-18.2.0-1249.ga6f82997 310 kB/s | 31 kB 00:00 2023-12-01T01:06:49.502 INFO:teuthology.orchestra.run.smithi137.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 942 kB/s | 306 kB 00:00 2023-12-01T01:06:49.552 INFO:teuthology.orchestra.run.smithi137.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 1.2 MB/s | 59 kB 00:00 2023-12-01T01:06:49.592 INFO:teuthology.orchestra.run.smithi077.stdout:(13/131): libcephfs2-18.2.0-1249.ga6f82997.el8. 2.2 MB/s | 774 kB 00:00 2023-12-01T01:06:49.661 INFO:teuthology.orchestra.run.smithi137.stdout:(39/131): ceph-mgr-diskprediction-local-18.2.0- 6.2 MB/s | 7.4 MB 00:01 2023-12-01T01:06:49.683 INFO:teuthology.orchestra.run.smithi077.stdout:(14/131): libcephsqlite-18.2.0-1249.ga6f82997.e 1.5 MB/s | 141 kB 00:00 2023-12-01T01:06:49.695 INFO:teuthology.orchestra.run.smithi137.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 528 kB/s | 75 kB 00:00 2023-12-01T01:06:49.745 INFO:teuthology.orchestra.run.smithi137.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 876 kB/s | 44 kB 00:00 2023-12-01T01:06:49.775 INFO:teuthology.orchestra.run.smithi077.stdout:(15/131): librados-devel-18.2.0-1249.ga6f82997. 1.5 MB/s | 139 kB 00:00 2023-12-01T01:06:49.837 INFO:teuthology.orchestra.run.smithi137.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 2.0 MB/s | 185 kB 00:00 2023-12-01T01:06:49.895 INFO:teuthology.orchestra.run.smithi137.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 568 kB/s | 33 kB 00:00 2023-12-01T01:06:49.951 INFO:teuthology.orchestra.run.smithi077.stdout:(16/131): libradosstriper1-18.2.0-1249.ga6f8299 2.6 MB/s | 474 kB 00:00 2023-12-01T01:06:49.971 INFO:teuthology.orchestra.run.smithi137.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 217 kB/s | 16 kB 00:00 2023-12-01T01:06:49.996 INFO:teuthology.orchestra.run.smithi137.stdout:(45/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.4 MB/s | 476 kB 00:00 2023-12-01T01:06:50.071 INFO:teuthology.orchestra.run.smithi137.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.7 MB/s | 173 kB 00:00 2023-12-01T01:06:50.139 INFO:teuthology.orchestra.run.smithi137.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 1.2 MB/s | 81 kB 00:00 2023-12-01T01:06:50.205 INFO:teuthology.orchestra.run.smithi137.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 167 kB/s | 11 kB 00:00 2023-12-01T01:06:50.255 INFO:teuthology.orchestra.run.smithi137.stdout:(49/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.4 MB/s | 384 kB 00:00 2023-12-01T01:06:50.273 INFO:teuthology.orchestra.run.smithi137.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 263 kB/s | 18 kB 00:00 2023-12-01T01:06:50.340 INFO:teuthology.orchestra.run.smithi137.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 731 kB/s | 49 kB 00:00 2023-12-01T01:06:50.408 INFO:teuthology.orchestra.run.smithi203.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-01T01:06:50.409 INFO:teuthology.orchestra.run.smithi203.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-01T01:06:50.415 INFO:teuthology.orchestra.run.smithi137.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 785 kB/s | 59 kB 00:00 2023-12-01T01:06:50.423 INFO:teuthology.orchestra.run.smithi203.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-12-01T01:06:50.424 INFO:teuthology.orchestra.run.smithi203.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-12-01T01:06:50.425 INFO:teuthology.orchestra.run.smithi203.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-12-01T01:06:50.426 INFO:teuthology.orchestra.run.smithi203.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-12-01T01:06:50.428 INFO:teuthology.orchestra.run.smithi203.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-12-01T01:06:50.431 INFO:teuthology.orchestra.run.smithi203.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-12-01T01:06:50.432 INFO:teuthology.orchestra.run.smithi203.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-12-01T01:06:50.490 INFO:teuthology.orchestra.run.smithi137.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 804 kB/s | 60 kB 00:00 2023-12-01T01:06:50.564 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2023-12-01T01:06:50.574 INFO:teuthology.orchestra.run.smithi137.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 3.3 MB/s | 283 kB 00:00 2023-12-01T01:06:50.576 INFO:teuthology.orchestra.run.smithi203.stdout:=========================================================================================================================== 2023-12-01T01:06:50.576 INFO:teuthology.orchestra.run.smithi203.stdout: Package Arch Version Repository Size 2023-12-01T01:06:50.576 INFO:teuthology.orchestra.run.smithi203.stdout:=========================================================================================================================== 2023-12-01T01:06:50.577 INFO:teuthology.orchestra.run.smithi203.stdout:Installing: 2023-12-01T01:06:50.577 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-12-01T01:06:50.577 INFO:teuthology.orchestra.run.smithi203.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-12-01T01:06:50.577 INFO:teuthology.orchestra.run.smithi203.stdout: ceph x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 6.4 k 2023-12-01T01:06:50.577 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-base x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 5.4 M 2023-12-01T01:06:50.577 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-fuse x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 851 k 2023-12-01T01:06:50.577 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-immutable-object-cache x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 165 k 2023-12-01T01:06:50.577 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 1.6 M 2023-12-01T01:06:50.577 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-cephadm noarch 2:18.2.0-1249.ga6f82997.el8 ceph-noarch 132 k 2023-12-01T01:06:50.577 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-dashboard noarch 2:18.2.0-1249.ga6f82997.el8 ceph-noarch 1.8 M 2023-12-01T01:06:50.577 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.0-1249.ga6f82997.el8 ceph-noarch 7.4 M 2023-12-01T01:06:50.577 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-rook noarch 2:18.2.0-1249.ga6f82997.el8 ceph-noarch 50 k 2023-12-01T01:06:50.578 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-radosgw x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 13 M 2023-12-01T01:06:50.578 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-test x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 47 M 2023-12-01T01:06:50.578 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-volume noarch 2:18.2.0-1249.ga6f82997.el8 ceph-noarch 262 k 2023-12-01T01:06:50.578 INFO:teuthology.orchestra.run.smithi203.stdout: cephadm noarch 2:18.2.0-1249.ga6f82997.el8 ceph-noarch 168 k 2023-12-01T01:06:50.578 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-mirror x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 258 k 2023-12-01T01:06:50.578 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-top noarch 2:18.2.0-1249.ga6f82997.el8 ceph-noarch 18 k 2023-12-01T01:06:50.578 INFO:teuthology.orchestra.run.smithi203.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-12-01T01:06:50.578 INFO:teuthology.orchestra.run.smithi203.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-12-01T01:06:50.578 INFO:teuthology.orchestra.run.smithi203.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-12-01T01:06:50.578 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs-devel x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 31 k 2023-12-01T01:06:50.578 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs2 x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 774 k 2023-12-01T01:06:50.579 INFO:teuthology.orchestra.run.smithi203.stdout: librados-devel x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 139 k 2023-12-01T01:06:50.579 INFO:teuthology.orchestra.run.smithi203.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-12-01T01:06:50.579 INFO:teuthology.orchestra.run.smithi203.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-12-01T01:06:50.579 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-12-01T01:06:50.579 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-12-01T01:06:50.579 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-12-01T01:06:50.579 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-12-01T01:06:50.579 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cephfs x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 196 k 2023-12-01T01:06:50.579 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rados x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 371 k 2023-12-01T01:06:50.580 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rbd x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 364 k 2023-12-01T01:06:50.580 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rgw x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 107 k 2023-12-01T01:06:50.580 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-fuse x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 94 k 2023-12-01T01:06:50.580 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-mirror x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 3.4 M 2023-12-01T01:06:50.580 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-nbd x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 181 k 2023-12-01T01:06:50.580 INFO:teuthology.orchestra.run.smithi203.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-12-01T01:06:50.580 INFO:teuthology.orchestra.run.smithi203.stdout:Upgrading: 2023-12-01T01:06:50.580 INFO:teuthology.orchestra.run.smithi203.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-12-01T01:06:50.580 INFO:teuthology.orchestra.run.smithi203.stdout: librados2 x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 3.7 M 2023-12-01T01:06:50.580 INFO:teuthology.orchestra.run.smithi203.stdout: librbd1 x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 3.6 M 2023-12-01T01:06:50.580 INFO:teuthology.orchestra.run.smithi203.stdout:Installing dependencies: 2023-12-01T01:06:50.580 INFO:teuthology.orchestra.run.smithi203.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-12-01T01:06:50.581 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-common x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 22 M 2023-12-01T01:06:50.581 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-grafana-dashboards noarch 2:18.2.0-1249.ga6f82997.el8 ceph-noarch 23 k 2023-12-01T01:06:50.581 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mds x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 2.3 M 2023-12-01T01:06:50.581 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-modules-core noarch 2:18.2.0-1249.ga6f82997.el8 ceph-noarch 242 k 2023-12-01T01:06:50.581 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mon x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 4.6 M 2023-12-01T01:06:50.581 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-osd x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 18 M 2023-12-01T01:06:50.581 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-prometheus-alerts noarch 2:18.2.0-1249.ga6f82997.el8 ceph-noarch 15 k 2023-12-01T01:06:50.581 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-selinux x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 24 k 2023-12-01T01:06:50.581 INFO:teuthology.orchestra.run.smithi203.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-12-01T01:06:50.581 INFO:teuthology.orchestra.run.smithi203.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-12-01T01:06:50.581 INFO:teuthology.orchestra.run.smithi203.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-12-01T01:06:50.581 INFO:teuthology.orchestra.run.smithi203.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-12-01T01:06:50.581 INFO:teuthology.orchestra.run.smithi203.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-12-01T01:06:50.582 INFO:teuthology.orchestra.run.smithi203.stdout: libcephsqlite x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 141 k 2023-12-01T01:06:50.582 INFO:teuthology.orchestra.run.smithi203.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-12-01T01:06:50.582 INFO:teuthology.orchestra.run.smithi203.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-12-01T01:06:50.582 INFO:teuthology.orchestra.run.smithi203.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-12-01T01:06:50.582 INFO:teuthology.orchestra.run.smithi203.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-12-01T01:06:50.582 INFO:teuthology.orchestra.run.smithi203.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-12-01T01:06:50.582 INFO:teuthology.orchestra.run.smithi203.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-12-01T01:06:50.582 INFO:teuthology.orchestra.run.smithi203.stdout: libradosstriper1 x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 474 k 2023-12-01T01:06:50.582 INFO:teuthology.orchestra.run.smithi203.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-12-01T01:06:50.583 INFO:teuthology.orchestra.run.smithi203.stdout: librgw2 x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 6.6 M 2023-12-01T01:06:50.583 INFO:teuthology.orchestra.run.smithi203.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-12-01T01:06:50.583 INFO:teuthology.orchestra.run.smithi203.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-12-01T01:06:50.583 INFO:teuthology.orchestra.run.smithi203.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-12-01T01:06:50.583 INFO:teuthology.orchestra.run.smithi203.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-12-01T01:06:50.583 INFO:teuthology.orchestra.run.smithi203.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-12-01T01:06:50.583 INFO:teuthology.orchestra.run.smithi203.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-12-01T01:06:50.583 INFO:teuthology.orchestra.run.smithi203.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-12-01T01:06:50.583 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-12-01T01:06:50.584 INFO:teuthology.orchestra.run.smithi203.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-12-01T01:06:50.584 INFO:teuthology.orchestra.run.smithi203.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-12-01T01:06:50.584 INFO:teuthology.orchestra.run.smithi203.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-12-01T01:06:50.584 INFO:teuthology.orchestra.run.smithi203.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-12-01T01:06:50.584 INFO:teuthology.orchestra.run.smithi203.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-12-01T01:06:50.584 INFO:teuthology.orchestra.run.smithi203.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-12-01T01:06:50.584 INFO:teuthology.orchestra.run.smithi203.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-12-01T01:06:50.584 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-12-01T01:06:50.584 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-argparse x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 45 k 2023-12-01T01:06:50.584 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-common x86_64 2:18.2.0-1249.ga6f82997.el8 ceph 123 k 2023-12-01T01:06:50.585 INFO:teuthology.orchestra.run.smithi203.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-12-01T01:06:50.585 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-12-01T01:06:50.585 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-12-01T01:06:50.585 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-12-01T01:06:50.585 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-12-01T01:06:50.585 INFO:teuthology.orchestra.run.smithi203.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-12-01T01:06:50.585 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-12-01T01:06:50.585 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-12-01T01:06:50.585 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-12-01T01:06:50.585 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-12-01T01:06:50.585 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-12-01T01:06:50.586 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-01T01:06:50.586 INFO:teuthology.orchestra.run.smithi203.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-12-01T01:06:50.586 INFO:teuthology.orchestra.run.smithi203.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-12-01T01:06:50.586 INFO:teuthology.orchestra.run.smithi203.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-12-01T01:06:50.586 INFO:teuthology.orchestra.run.smithi203.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-12-01T01:06:50.586 INFO:teuthology.orchestra.run.smithi203.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-12-01T01:06:50.586 INFO:teuthology.orchestra.run.smithi203.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-12-01T01:06:50.586 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-12-01T01:06:50.587 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-12-01T01:06:50.587 INFO:teuthology.orchestra.run.smithi203.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-12-01T01:06:50.587 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-12-01T01:06:50.587 INFO:teuthology.orchestra.run.smithi203.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-12-01T01:06:50.587 INFO:teuthology.orchestra.run.smithi203.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-12-01T01:06:50.587 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-12-01T01:06:50.588 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-12-01T01:06:50.588 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-12-01T01:06:50.588 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-12-01T01:06:50.588 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-12-01T01:06:50.588 INFO:teuthology.orchestra.run.smithi203.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-12-01T01:06:50.588 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-01T01:06:50.588 INFO:teuthology.orchestra.run.smithi203.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-12-01T01:06:50.588 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-12-01T01:06:50.588 INFO:teuthology.orchestra.run.smithi203.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-12-01T01:06:50.589 INFO:teuthology.orchestra.run.smithi203.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-12-01T01:06:50.589 INFO:teuthology.orchestra.run.smithi203.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-12-01T01:06:50.589 INFO:teuthology.orchestra.run.smithi203.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-12-01T01:06:50.589 INFO:teuthology.orchestra.run.smithi203.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-12-01T01:06:50.589 INFO:teuthology.orchestra.run.smithi203.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-12-01T01:06:50.589 INFO:teuthology.orchestra.run.smithi203.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-12-01T01:06:50.589 INFO:teuthology.orchestra.run.smithi203.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-12-01T01:06:50.589 INFO:teuthology.orchestra.run.smithi203.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-12-01T01:06:50.589 INFO:teuthology.orchestra.run.smithi203.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-12-01T01:06:50.590 INFO:teuthology.orchestra.run.smithi203.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-12-01T01:06:50.590 INFO:teuthology.orchestra.run.smithi203.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-12-01T01:06:50.590 INFO:teuthology.orchestra.run.smithi203.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-12-01T01:06:50.590 INFO:teuthology.orchestra.run.smithi203.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-12-01T01:06:50.590 INFO:teuthology.orchestra.run.smithi203.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-12-01T01:06:50.590 INFO:teuthology.orchestra.run.smithi203.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-12-01T01:06:50.590 INFO:teuthology.orchestra.run.smithi203.stdout:Enabling module streams: 2023-12-01T01:06:50.590 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql 10 2023-12-01T01:06:50.590 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-12-01T01:06:50.590 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2023-12-01T01:06:50.590 INFO:teuthology.orchestra.run.smithi203.stdout:=========================================================================================================================== 2023-12-01T01:06:50.591 INFO:teuthology.orchestra.run.smithi203.stdout:Install 128 Packages 2023-12-01T01:06:50.591 INFO:teuthology.orchestra.run.smithi203.stdout:Upgrade 3 Packages 2023-12-01T01:06:50.591 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-12-01T01:06:50.591 INFO:teuthology.orchestra.run.smithi203.stdout:Total download size: 203 M 2023-12-01T01:06:50.591 INFO:teuthology.orchestra.run.smithi203.stdout:Downloading Packages: 2023-12-01T01:06:50.608 INFO:teuthology.orchestra.run.smithi137.stdout:(55/131): python3-kubernetes-11.0.0-6.el8.noarc 3.3 MB/s | 1.2 MB 00:00 2023-12-01T01:06:50.633 INFO:teuthology.orchestra.run.smithi137.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 266 kB/s | 16 kB 00:00 2023-12-01T01:06:50.649 INFO:teuthology.orchestra.run.smithi137.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 805 kB/s | 34 kB 00:00 2023-12-01T01:06:50.692 INFO:teuthology.orchestra.run.smithi137.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 3.3 MB/s | 196 kB 00:00 2023-12-01T01:06:50.717 INFO:teuthology.orchestra.run.smithi137.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 959 kB/s | 64 kB 00:00 2023-12-01T01:06:50.734 INFO:teuthology.orchestra.run.smithi137.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 446 kB/s | 19 kB 00:00 2023-12-01T01:06:50.751 INFO:teuthology.orchestra.run.smithi137.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 715 kB/s | 24 kB 00:00 2023-12-01T01:06:50.768 INFO:teuthology.orchestra.run.smithi137.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 875 kB/s | 29 kB 00:00 2023-12-01T01:06:50.785 INFO:teuthology.orchestra.run.smithi137.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 818 kB/s | 27 kB 00:00 2023-12-01T01:06:50.873 INFO:teuthology.orchestra.run.smithi137.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 2.4 MB/s | 255 kB 00:00 2023-12-01T01:06:50.902 INFO:teuthology.orchestra.run.smithi137.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.1 MB/s | 251 kB 00:00 2023-12-01T01:06:50.927 INFO:teuthology.orchestra.run.smithi137.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 1.1 MB/s | 61 kB 00:00 2023-12-01T01:06:50.953 INFO:teuthology.orchestra.run.smithi137.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.7 MB/s | 85 kB 00:00 2023-12-01T01:06:50.986 INFO:teuthology.orchestra.run.smithi137.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 389 kB/s | 23 kB 00:00 2023-12-01T01:06:51.003 INFO:teuthology.orchestra.run.smithi137.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 4.1 MB/s | 70 kB 00:00 2023-12-01T01:06:51.011 INFO:teuthology.orchestra.run.smithi203.stdout:(1/131): ceph-18.2.0-1249.ga6f82997.el8.x86_64. 23 kB/s | 6.4 kB 00:00 2023-12-01T01:06:51.195 INFO:teuthology.orchestra.run.smithi137.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 244 kB/s | 47 kB 00:00 2023-12-01T01:06:51.237 INFO:teuthology.orchestra.run.smithi137.stdout:(71/131): thrift-0.13.0-2.el8.x86_64.rpm 6.1 MB/s | 1.7 MB 00:00 2023-12-01T01:06:51.338 INFO:teuthology.orchestra.run.smithi137.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 1.4 MB/s | 200 kB 00:00 2023-12-01T01:06:51.405 INFO:teuthology.orchestra.run.smithi137.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 255 kB/s | 43 kB 00:00 2023-12-01T01:06:51.497 INFO:teuthology.orchestra.run.smithi137.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.4 MB/s | 220 kB 00:00 2023-12-01T01:06:51.580 INFO:teuthology.orchestra.run.smithi137.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 275 kB/s | 48 kB 00:00 2023-12-01T01:06:51.647 INFO:teuthology.orchestra.run.smithi137.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 579 kB/s | 87 kB 00:00 2023-12-01T01:06:51.673 INFO:teuthology.orchestra.run.smithi137.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 464 kB/s | 43 kB 00:00 2023-12-01T01:06:51.743 INFO:teuthology.orchestra.run.smithi077.stdout:(17/131): ceph-osd-18.2.0-1249.ga6f82997.el8.x8 1.9 MB/s | 18 MB 00:09 2023-12-01T01:06:51.789 INFO:teuthology.orchestra.run.smithi137.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 768 kB/s | 109 kB 00:00 2023-12-01T01:06:51.815 INFO:teuthology.orchestra.run.smithi137.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.6 MB/s | 229 kB 00:00 2023-12-01T01:06:51.817 INFO:teuthology.orchestra.run.smithi077.stdout:(18/131): librgw2-18.2.0-1249.ga6f82997.el8.x86 3.5 MB/s | 6.6 MB 00:01 2023-12-01T01:06:51.843 INFO:teuthology.orchestra.run.smithi077.stdout:(19/131): python3-ceph-argparse-18.2.0-1249.ga6 446 kB/s | 45 kB 00:00 2023-12-01T01:06:51.886 INFO:teuthology.orchestra.run.smithi203.stdout:(2/131): ceph-fuse-18.2.0-1249.ga6f82997.el8.x8 973 kB/s | 851 kB 00:00 2023-12-01T01:06:51.901 INFO:teuthology.orchestra.run.smithi077.stdout:(20/131): python3-ceph-common-18.2.0-1249.ga6f8 1.4 MB/s | 123 kB 00:00 2023-12-01T01:06:51.965 INFO:teuthology.orchestra.run.smithi137.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.1 MB/s | 560 kB 00:00 2023-12-01T01:06:51.968 INFO:teuthology.orchestra.run.smithi077.stdout:(21/131): python3-cephfs-18.2.0-1249.ga6f82997. 1.5 MB/s | 196 kB 00:00 2023-12-01T01:06:51.990 INFO:teuthology.orchestra.run.smithi137.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 107 kB/s | 19 kB 00:00 2023-12-01T01:06:52.036 INFO:teuthology.orchestra.run.smithi203.stdout:(3/131): ceph-immutable-object-cache-18.2.0-124 1.1 MB/s | 165 kB 00:00 2023-12-01T01:06:52.060 INFO:teuthology.orchestra.run.smithi077.stdout:(22/131): python3-rados-18.2.0-1249.ga6f82997.e 2.3 MB/s | 371 kB 00:00 2023-12-01T01:06:52.152 INFO:teuthology.orchestra.run.smithi077.stdout:(23/131): python3-rbd-18.2.0-1249.ga6f82997.el8 1.9 MB/s | 364 kB 00:00 2023-12-01T01:06:52.178 INFO:teuthology.orchestra.run.smithi077.stdout:(24/131): python3-rgw-18.2.0-1249.ga6f82997.el8 912 kB/s | 107 kB 00:00 2023-12-01T01:06:52.228 INFO:teuthology.orchestra.run.smithi077.stdout:(25/131): rbd-fuse-18.2.0-1249.ga6f82997.el8.x8 1.2 MB/s | 94 kB 00:00 2023-12-01T01:06:52.271 INFO:teuthology.orchestra.run.smithi137.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 507 kB/s | 155 kB 00:00 2023-12-01T01:06:52.308 INFO:teuthology.orchestra.run.smithi137.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 1.7 MB/s | 559 kB 00:00 2023-12-01T01:06:52.378 INFO:teuthology.orchestra.run.smithi077.stdout:(26/131): rbd-nbd-18.2.0-1249.ga6f82997.el8.x86 1.2 MB/s | 181 kB 00:00 2023-12-01T01:06:52.400 INFO:teuthology.orchestra.run.smithi137.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 139 kB/s | 18 kB 00:00 2023-12-01T01:06:52.458 INFO:teuthology.orchestra.run.smithi137.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 3.9 MB/s | 605 kB 00:00 2023-12-01T01:06:52.470 INFO:teuthology.orchestra.run.smithi077.stdout:(27/131): ceph-grafana-dashboards-18.2.0-1249.g 252 kB/s | 23 kB 00:00 2023-12-01T01:06:52.509 INFO:teuthology.orchestra.run.smithi137.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.1 MB/s | 238 kB 00:00 2023-12-01T01:06:52.570 INFO:teuthology.orchestra.run.smithi077.stdout:(28/131): ceph-mgr-cephadm-18.2.0-1249.ga6f8299 1.3 MB/s | 132 kB 00:00 2023-12-01T01:06:52.684 INFO:teuthology.orchestra.run.smithi137.stdout:(87/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 10 MB/s | 2.3 MB 00:00 2023-12-01T01:06:52.709 INFO:teuthology.orchestra.run.smithi137.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 361 kB/s | 72 kB 00:00 2023-12-01T01:06:52.801 INFO:teuthology.orchestra.run.smithi137.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 709 kB/s | 83 kB 00:00 2023-12-01T01:06:52.826 INFO:teuthology.orchestra.run.smithi137.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 512 kB/s | 60 kB 00:00 2023-12-01T01:06:52.950 INFO:teuthology.orchestra.run.smithi137.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 658 kB/s | 98 kB 00:00 2023-12-01T01:06:52.977 INFO:teuthology.orchestra.run.smithi137.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 384 kB/s | 58 kB 00:00 2023-12-01T01:06:53.037 INFO:teuthology.orchestra.run.smithi203.stdout:(4/131): ceph-base-18.2.0-1249.ga6f82997.el8.x8 2.3 MB/s | 5.4 MB 00:02 2023-12-01T01:06:53.044 INFO:teuthology.orchestra.run.smithi137.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 415 kB/s | 39 kB 00:00 2023-12-01T01:06:53.136 INFO:teuthology.orchestra.run.smithi137.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 4.0 MB/s | 644 kB 00:00 2023-12-01T01:06:53.211 INFO:teuthology.orchestra.run.smithi137.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 1.0 MB/s | 171 kB 00:00 2023-12-01T01:06:53.287 INFO:teuthology.orchestra.run.smithi203.stdout:(5/131): ceph-mds-18.2.0-1249.ga6f82997.el8.x86 1.8 MB/s | 2.3 MB 00:01 2023-12-01T01:06:53.378 INFO:teuthology.orchestra.run.smithi137.stdout:(96/131): NetworkManager-initscripts-updown-1.3 568 kB/s | 138 kB 00:00 2023-12-01T01:06:53.454 INFO:teuthology.orchestra.run.smithi203.stdout:(6/131): ceph-mgr-18.2.0-1249.ga6f82997.el8.x86 3.7 MB/s | 1.6 MB 00:00 2023-12-01T01:06:53.462 INFO:teuthology.orchestra.run.smithi077.stdout:(29/131): rbd-mirror-18.2.0-1249.ga6f82997.el8. 2.7 MB/s | 3.4 MB 00:01 2023-12-01T01:06:53.504 INFO:teuthology.orchestra.run.smithi137.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 882 kB/s | 110 kB 00:00 2023-12-01T01:06:53.679 INFO:teuthology.orchestra.run.smithi137.stdout:(98/131): python3-scipy-1.0.0-21.module+el8.5.0 29 MB/s | 14 MB 00:00 2023-12-01T01:06:53.704 INFO:teuthology.orchestra.run.smithi137.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 271 kB/s | 54 kB 00:00 2023-12-01T01:06:53.704 INFO:teuthology.orchestra.run.smithi203.stdout:(7/131): ceph-common-18.2.0-1249.ga6f82997.el8. 7.3 MB/s | 22 MB 00:02 2023-12-01T01:06:53.779 INFO:teuthology.orchestra.run.smithi137.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 4.2 MB/s | 323 kB 00:00 2023-12-01T01:06:53.845 INFO:teuthology.orchestra.run.smithi137.stdout:(101/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.5 MB/s | 259 kB 00:00 2023-12-01T01:06:53.871 INFO:teuthology.orchestra.run.smithi137.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 422 kB/s | 39 kB 00:00 2023-12-01T01:06:53.879 INFO:teuthology.orchestra.run.smithi077.stdout:(30/131): ceph-mgr-dashboard-18.2.0-1249.ga6f82 1.3 MB/s | 1.8 MB 00:01 2023-12-01T01:06:53.905 INFO:teuthology.orchestra.run.smithi137.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 11 MB/s | 688 kB 00:00 2023-12-01T01:06:53.997 INFO:teuthology.orchestra.run.smithi137.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 4.2 MB/s | 538 kB 00:00 2023-12-01T01:06:54.073 INFO:teuthology.orchestra.run.smithi137.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 940 kB/s | 157 kB 00:00 2023-12-01T01:06:54.097 INFO:teuthology.orchestra.run.smithi137.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.8 MB/s | 187 kB 00:00 2023-12-01T01:06:54.182 INFO:teuthology.orchestra.run.smithi137.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 6.4 MB/s | 713 kB 00:00 2023-12-01T01:06:54.188 INFO:teuthology.orchestra.run.smithi077.stdout:(31/131): ceph-mgr-modules-core-18.2.0-1249.ga6 785 kB/s | 242 kB 00:00 2023-12-01T01:06:54.248 INFO:teuthology.orchestra.run.smithi137.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 398 kB/s | 27 kB 00:00 2023-12-01T01:06:54.280 INFO:teuthology.orchestra.run.smithi077.stdout:(32/131): ceph-mgr-rook-18.2.0-1249.ga6f82997.e 547 kB/s | 50 kB 00:00 2023-12-01T01:06:54.332 INFO:teuthology.orchestra.run.smithi137.stdout:(109/131): python3-babel-2.5.1-7.el8.noarch.rpm 20 MB/s | 4.8 MB 00:00 2023-12-01T01:06:54.372 INFO:teuthology.orchestra.run.smithi077.stdout:(33/131): ceph-prometheus-alerts-18.2.0-1249.ga 159 kB/s | 15 kB 00:00 2023-12-01T01:06:54.399 INFO:teuthology.orchestra.run.smithi137.stdout:(110/131): flex-2.6.1-9.el8.x86_64.rpm 2.1 MB/s | 320 kB 00:00 2023-12-01T01:06:54.466 INFO:teuthology.orchestra.run.smithi137.stdout:(111/131): python3-prettytable-0.7.2-14.el8.noa 329 kB/s | 44 kB 00:00 2023-12-01T01:06:54.591 INFO:teuthology.orchestra.run.smithi137.stdout:(112/131): python3-numpy-1.14.3-10.el8.x86_64.r 19 MB/s | 3.7 MB 00:00 2023-12-01T01:06:54.624 INFO:teuthology.orchestra.run.smithi137.stdout:(113/131): python3-werkzeug-0.12.2-4.el8.noarch 2.8 MB/s | 457 kB 00:00 2023-12-01T01:06:54.680 INFO:teuthology.orchestra.run.smithi077.stdout:(34/131): ceph-volume-18.2.0-1249.ga6f82997.el8 849 kB/s | 262 kB 00:00 2023-12-01T01:06:54.740 INFO:teuthology.orchestra.run.smithi137.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 1.5 MB/s | 225 kB 00:00 2023-12-01T01:06:54.767 INFO:teuthology.orchestra.run.smithi137.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 451 kB/s | 64 kB 00:00 2023-12-01T01:06:54.838 INFO:teuthology.orchestra.run.smithi203.stdout:(8/131): ceph-mon-18.2.0-1249.ga6f82997.el8.x86 3.0 MB/s | 4.6 MB 00:01 2023-12-01T01:06:54.892 INFO:teuthology.orchestra.run.smithi137.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 827 kB/s | 126 kB 00:00 2023-12-01T01:06:54.905 INFO:teuthology.orchestra.run.smithi077.stdout:(35/131): cephadm-18.2.0-1249.ga6f82997.el8.noa 748 kB/s | 168 kB 00:00 2023-12-01T01:06:54.951 INFO:teuthology.orchestra.run.smithi137.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 109 kB/s | 20 kB 00:00 2023-12-01T01:06:54.954 INFO:teuthology.orchestra.run.smithi203.stdout:(9/131): ceph-selinux-18.2.0-1249.ga6f82997.el8 209 kB/s | 24 kB 00:00 2023-12-01T01:06:54.997 INFO:teuthology.orchestra.run.smithi077.stdout:(36/131): cephfs-top-18.2.0-1249.ga6f82997.el8. 196 kB/s | 18 kB 00:00 2023-12-01T01:06:55.009 INFO:teuthology.orchestra.run.smithi137.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 887 kB/s | 103 kB 00:00 2023-12-01T01:06:55.101 INFO:teuthology.orchestra.run.smithi137.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 4.6 MB/s | 709 kB 00:00 2023-12-01T01:06:55.218 INFO:teuthology.orchestra.run.smithi137.stdout:(120/131): openblas-threads-0.3.15-3.el8.x86_64 23 MB/s | 4.9 MB 00:00 2023-12-01T01:06:55.244 INFO:teuthology.orchestra.run.smithi137.stdout:(121/131): libpq-13.5-1.el8.x86_64.rpm 1.4 MB/s | 198 kB 00:00 2023-12-01T01:06:55.310 INFO:teuthology.orchestra.run.smithi137.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 3.0 MB/s | 202 kB 00:00 2023-12-01T01:06:55.386 INFO:teuthology.orchestra.run.smithi137.stdout:(123/131): openblas-0.3.15-3.el8.x86_64.rpm 28 MB/s | 4.7 MB 00:00 2023-12-01T01:06:55.445 INFO:teuthology.orchestra.run.smithi137.stdout:(124/131): librdkafka-0.11.4-3.el8.x86_64.rpm 2.6 MB/s | 354 kB 00:00 2023-12-01T01:06:55.528 INFO:teuthology.orchestra.run.smithi137.stdout:(125/131): postgresql-contrib-10.21-2.module+el 9.5 MB/s | 808 kB 00:00 2023-12-01T01:06:55.662 INFO:teuthology.orchestra.run.smithi137.stdout:(126/131): postgresql-server-10.21-2.module+el8 18 MB/s | 5.1 MB 00:00 2023-12-01T01:06:55.721 INFO:teuthology.orchestra.run.smithi137.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 7.8 MB/s | 1.5 MB 00:00 2023-12-01T01:06:55.789 INFO:teuthology.orchestra.run.smithi203.stdout:(10/131): ceph-osd-18.2.0-1249.ga6f82997.el8.x8 7.6 MB/s | 18 MB 00:02 2023-12-01T01:06:55.922 INFO:teuthology.orchestra.run.smithi203.stdout:(11/131): ceph-radosgw-18.2.0-1249.ga6f82997.el 5.9 MB/s | 13 MB 00:02 2023-12-01T01:06:55.947 INFO:teuthology.orchestra.run.smithi203.stdout:(12/131): cephfs-mirror-18.2.0-1249.ga6f82997.e 1.6 MB/s | 258 kB 00:00 2023-12-01T01:06:55.980 INFO:teuthology.orchestra.run.smithi203.stdout:(13/131): libcephfs-devel-18.2.0-1249.ga6f82997 532 kB/s | 31 kB 00:00 2023-12-01T01:06:56.039 INFO:teuthology.orchestra.run.smithi077.stdout:(37/131): ceph-mgr-diskprediction-local-18.2.0- 2.9 MB/s | 7.4 MB 00:02 2023-12-01T01:06:56.081 INFO:teuthology.orchestra.run.smithi203.stdout:(14/131): libcephsqlite-18.2.0-1249.ga6f82997.e 1.4 MB/s | 141 kB 00:00 2023-12-01T01:06:56.115 INFO:teuthology.orchestra.run.smithi203.stdout:(15/131): libcephfs2-18.2.0-1249.ga6f82997.el8. 4.5 MB/s | 774 kB 00:00 2023-12-01T01:06:56.157 INFO:teuthology.orchestra.run.smithi203.stdout:(16/131): librados-devel-18.2.0-1249.ga6f82997. 1.8 MB/s | 139 kB 00:00 2023-12-01T01:06:56.232 INFO:teuthology.orchestra.run.smithi203.stdout:(17/131): libradosstriper1-18.2.0-1249.ga6f8299 4.0 MB/s | 474 kB 00:00 2023-12-01T01:06:56.321 INFO:teuthology.orchestra.run.smithi137.stdout:(128/131): librados2-18.2.0-1249.ga6f82997.el8. 5.6 MB/s | 3.7 MB 00:00 2023-12-01T01:06:56.332 INFO:teuthology.orchestra.run.smithi203.stdout:(18/131): python3-ceph-argparse-18.2.0-1249.ga6 446 kB/s | 45 kB 00:00 2023-12-01T01:06:56.396 INFO:teuthology.orchestra.run.smithi137.stdout:(129/131): librbd1-18.2.0-1249.ga6f82997.el8.x8 5.3 MB/s | 3.6 MB 00:00 2023-12-01T01:06:56.399 INFO:teuthology.orchestra.run.smithi203.stdout:(19/131): python3-ceph-common-18.2.0-1249.ga6f8 1.8 MB/s | 123 kB 00:00 2023-12-01T01:06:56.467 INFO:teuthology.orchestra.run.smithi203.stdout:(20/131): python3-cephfs-18.2.0-1249.ga6f82997. 2.9 MB/s | 196 kB 00:00 2023-12-01T01:06:56.471 INFO:teuthology.orchestra.run.smithi137.stdout:(130/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 5.2 MB/s | 798 kB 00:00 2023-12-01T01:06:56.514 INFO:teuthology.orchestra.run.smithi077.stdout:(38/131): gperftools-libs-2.7-9.el8.x86_64.rpm 202 kB/s | 306 kB 00:01 2023-12-01T01:06:56.550 INFO:teuthology.orchestra.run.smithi203.stdout:(21/131): python3-rados-18.2.0-1249.ga6f82997.e 4.4 MB/s | 371 kB 00:00 2023-12-01T01:06:56.659 INFO:teuthology.orchestra.run.smithi203.stdout:(22/131): python3-rbd-18.2.0-1249.ga6f82997.el8 3.3 MB/s | 364 kB 00:00 2023-12-01T01:06:56.734 INFO:teuthology.orchestra.run.smithi203.stdout:(23/131): python3-rgw-18.2.0-1249.ga6f82997.el8 1.4 MB/s | 107 kB 00:00 2023-12-01T01:06:56.773 INFO:teuthology.orchestra.run.smithi077.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 290 kB/s | 75 kB 00:00 2023-12-01T01:06:56.809 INFO:teuthology.orchestra.run.smithi203.stdout:(24/131): rbd-fuse-18.2.0-1249.ga6f82997.el8.x8 1.2 MB/s | 94 kB 00:00 2023-12-01T01:06:56.910 INFO:teuthology.orchestra.run.smithi203.stdout:(25/131): librgw2-18.2.0-1249.ga6f82997.el8.x86 8.7 MB/s | 6.6 MB 00:00 2023-12-01T01:06:56.968 INFO:teuthology.orchestra.run.smithi203.stdout:(26/131): rbd-nbd-18.2.0-1249.ga6f82997.el8.x86 3.0 MB/s | 181 kB 00:00 2023-12-01T01:06:57.027 INFO:teuthology.orchestra.run.smithi203.stdout:(27/131): ceph-grafana-dashboards-18.2.0-1249.g 395 kB/s | 23 kB 00:00 2023-12-01T01:06:57.082 INFO:teuthology.orchestra.run.smithi077.stdout:(40/131): liboath-2.6.2-3.el8.x86_64.rpm 56 kB/s | 59 kB 00:01 2023-12-01T01:06:57.111 INFO:teuthology.orchestra.run.smithi203.stdout:(28/131): ceph-mgr-cephadm-18.2.0-1249.ga6f8299 1.5 MB/s | 132 kB 00:00 2023-12-01T01:06:57.170 INFO:teuthology.orchestra.run.smithi203.stdout:(29/131): rbd-mirror-18.2.0-1249.ga6f82997.el8. 9.6 MB/s | 3.4 MB 00:00 2023-12-01T01:06:57.241 INFO:teuthology.orchestra.run.smithi077.stdout:(41/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.0 MB/s | 476 kB 00:00 2023-12-01T01:06:57.386 INFO:teuthology.orchestra.run.smithi203.stdout:(30/131): ceph-mgr-dashboard-18.2.0-1249.ga6f82 6.4 MB/s | 1.8 MB 00:00 2023-12-01T01:06:57.462 INFO:teuthology.orchestra.run.smithi203.stdout:(31/131): ceph-mgr-modules-core-18.2.0-1249.ga6 3.2 MB/s | 242 kB 00:00 2023-12-01T01:06:57.500 INFO:teuthology.orchestra.run.smithi077.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 717 kB/s | 185 kB 00:00 2023-12-01T01:06:57.546 INFO:teuthology.orchestra.run.smithi203.stdout:(32/131): ceph-mgr-rook-18.2.0-1249.ga6f82997.e 596 kB/s | 50 kB 00:00 2023-12-01T01:06:57.583 INFO:teuthology.orchestra.run.smithi077.stdout:(43/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 87 kB/s | 44 kB 00:00 2023-12-01T01:06:57.612 INFO:teuthology.orchestra.run.smithi203.stdout:(33/131): ceph-prometheus-alerts-18.2.0-1249.ga 220 kB/s | 15 kB 00:00 2023-12-01T01:06:57.679 INFO:teuthology.orchestra.run.smithi203.stdout:(34/131): ceph-volume-18.2.0-1249.ga6f82997.el8 3.8 MB/s | 262 kB 00:00 2023-12-01T01:06:57.733 INFO:teuthology.orchestra.run.smithi077.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 141 kB/s | 33 kB 00:00 2023-12-01T01:06:57.788 INFO:teuthology.orchestra.run.smithi203.stdout:(35/131): ceph-mgr-diskprediction-local-18.2.0- 12 MB/s | 7.4 MB 00:00 2023-12-01T01:06:57.813 INFO:teuthology.orchestra.run.smithi203.stdout:(36/131): cephadm-18.2.0-1249.ga6f82997.el8.noa 1.2 MB/s | 168 kB 00:00 2023-12-01T01:06:57.842 INFO:teuthology.orchestra.run.smithi077.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 63 kB/s | 16 kB 00:00 2023-12-01T01:06:57.847 INFO:teuthology.orchestra.run.smithi203.stdout:(37/131): cephfs-top-18.2.0-1249.ga6f82997.el8. 307 kB/s | 18 kB 00:00 2023-12-01T01:06:57.983 INFO:teuthology.orchestra.run.smithi077.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 690 kB/s | 173 kB 00:00 2023-12-01T01:06:58.225 INFO:teuthology.orchestra.run.smithi077.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 336 kB/s | 81 kB 00:00 2023-12-01T01:06:58.231 INFO:teuthology.orchestra.run.smithi203.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 153 kB/s | 59 kB 00:00 2023-12-01T01:06:58.448 INFO:teuthology.orchestra.run.smithi203.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 346 kB/s | 75 kB 00:00 2023-12-01T01:06:58.450 INFO:teuthology.orchestra.run.smithi077.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 49 kB/s | 11 kB 00:00 2023-12-01T01:06:58.576 INFO:teuthology.orchestra.run.smithi077.stdout:(49/131): python3-cherrypy-18.4.0-1.el8.noarch. 524 kB/s | 384 kB 00:00 2023-12-01T01:06:58.676 INFO:teuthology.orchestra.run.smithi077.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 78 kB/s | 18 kB 00:00 2023-12-01T01:06:58.832 INFO:teuthology.orchestra.run.smithi203.stdout:(40/131): gperftools-libs-2.7-9.el8.x86_64.rpm 301 kB/s | 306 kB 00:01 2023-12-01T01:06:58.910 INFO:teuthology.orchestra.run.smithi077.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 209 kB/s | 49 kB 00:00 2023-12-01T01:06:59.040 INFO:teuthology.orchestra.run.smithi203.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 209 kB/s | 44 kB 00:00 2023-12-01T01:06:59.152 INFO:teuthology.orchestra.run.smithi077.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 243 kB/s | 59 kB 00:00 2023-12-01T01:06:59.541 INFO:teuthology.orchestra.run.smithi203.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 370 kB/s | 185 kB 00:00 2023-12-01T01:06:59.561 INFO:teuthology.orchestra.run.smithi077.stdout:(53/131): ceph-test-18.2.0-1249.ga6f82997.el8.x 3.3 MB/s | 47 MB 00:14 2023-12-01T01:06:59.583 INFO:teuthology.orchestra.run.smithi203.stdout:(43/131): python3-asyncssh-2.7.0-2.el8.noarch.r 419 kB/s | 476 kB 00:01 2023-12-01T01:06:59.586 INFO:teuthology.orchestra.run.smithi077.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 139 kB/s | 60 kB 00:00 2023-12-01T01:06:59.619 INFO:teuthology.orchestra.run.smithi077.stdout:(55/131): python3-kubernetes-11.0.0-6.el8.noarc 1.1 MB/s | 1.2 MB 00:01 2023-12-01T01:06:59.650 INFO:teuthology.orchestra.run.smithi203.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 303 kB/s | 33 kB 00:00 2023-12-01T01:06:59.692 INFO:teuthology.orchestra.run.smithi203.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 150 kB/s | 16 kB 00:00 2023-12-01T01:06:59.820 INFO:teuthology.orchestra.run.smithi077.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 67 kB/s | 16 kB 00:00 2023-12-01T01:06:59.845 INFO:teuthology.orchestra.run.smithi077.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 149 kB/s | 34 kB 00:00 2023-12-01T01:07:00.075 INFO:teuthology.orchestra.run.smithi203.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 406 kB/s | 173 kB 00:00 2023-12-01T01:07:00.078 INFO:teuthology.orchestra.run.smithi077.stdout:(58/131): python3-rsa-4.9-2.el8.noarch.rpm 275 kB/s | 64 kB 00:00 2023-12-01T01:07:00.104 INFO:teuthology.orchestra.run.smithi077.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 692 kB/s | 196 kB 00:00 2023-12-01T01:07:00.284 INFO:teuthology.orchestra.run.smithi203.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 390 kB/s | 81 kB 00:00 2023-12-01T01:07:00.312 INFO:teuthology.orchestra.run.smithi077.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 80 kB/s | 19 kB 00:00 2023-12-01T01:07:00.337 INFO:teuthology.orchestra.run.smithi077.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 102 kB/s | 24 kB 00:00 2023-12-01T01:07:00.418 INFO:teuthology.orchestra.run.smithi203.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 83 kB/s | 11 kB 00:00 2023-12-01T01:07:00.493 INFO:teuthology.orchestra.run.smithi203.stdout:(49/131): python3-cherrypy-18.4.0-1.el8.noarch. 480 kB/s | 384 kB 00:00 2023-12-01T01:07:00.552 INFO:teuthology.orchestra.run.smithi203.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 132 kB/s | 18 kB 00:00 2023-12-01T01:07:00.563 INFO:teuthology.orchestra.run.smithi077.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 117 kB/s | 29 kB 00:00 2023-12-01T01:07:00.588 INFO:teuthology.orchestra.run.smithi077.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 110 kB/s | 27 kB 00:00 2023-12-01T01:07:00.819 INFO:teuthology.orchestra.run.smithi203.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 183 kB/s | 49 kB 00:00 2023-12-01T01:07:00.855 INFO:teuthology.orchestra.run.smithi077.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 875 kB/s | 255 kB 00:00 2023-12-01T01:07:01.061 INFO:teuthology.orchestra.run.smithi203.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 243 kB/s | 59 kB 00:00 2023-12-01T01:07:01.105 INFO:teuthology.orchestra.run.smithi077.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 245 kB/s | 61 kB 00:00 2023-12-01T01:07:01.130 INFO:teuthology.orchestra.run.smithi077.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 463 kB/s | 251 kB 00:00 2023-12-01T01:07:01.189 INFO:teuthology.orchestra.run.smithi077.stdout:(67/131): python3-pecan-1.3.2-9.el8.noarch.rpm 174 kB/s | 283 kB 00:01 2023-12-01T01:07:01.303 INFO:teuthology.orchestra.run.smithi203.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 249 kB/s | 60 kB 00:00 2023-12-01T01:07:01.367 INFO:teuthology.orchestra.run.smithi077.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 327 kB/s | 85 kB 00:00 2023-12-01T01:07:01.383 INFO:teuthology.orchestra.run.smithi077.stdout:(69/131): python3-zc-lockfile-2.0-2.el8.noarch. 91 kB/s | 23 kB 00:00 2023-12-01T01:07:01.409 INFO:teuthology.orchestra.run.smithi077.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.6 MB/s | 70 kB 00:00 2023-12-01T01:07:01.609 INFO:teuthology.orchestra.run.smithi077.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 566 kB/s | 110 kB 00:00 2023-12-01T01:07:01.701 INFO:teuthology.orchestra.run.smithi077.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 592 kB/s | 54 kB 00:00 2023-12-01T01:07:01.826 INFO:teuthology.orchestra.run.smithi077.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.0 MB/s | 259 kB 00:00 2023-12-01T01:07:01.935 INFO:teuthology.orchestra.run.smithi077.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 2.9 MB/s | 323 kB 00:00 2023-12-01T01:07:02.011 INFO:teuthology.orchestra.run.smithi203.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 399 kB/s | 283 kB 00:00 2023-12-01T01:07:02.110 INFO:teuthology.orchestra.run.smithi077.stdout:(75/131): python3-scipy-1.0.0-21.module+el8.5.0 19 MB/s | 14 MB 00:00 2023-12-01T01:07:02.135 INFO:teuthology.orchestra.run.smithi077.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 194 kB/s | 39 kB 00:00 2023-12-01T01:07:02.153 INFO:teuthology.orchestra.run.smithi203.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 110 kB/s | 16 kB 00:00 2023-12-01T01:07:02.194 INFO:teuthology.orchestra.run.smithi077.stdout:(77/131): thrift-0.13.0-2.el8.x86_64.rpm 1.7 MB/s | 1.7 MB 00:01 2023-12-01T01:07:02.245 INFO:teuthology.orchestra.run.smithi077.stdout:(78/131): bison-3.0.4-10.el8.x86_64.rpm 5.0 MB/s | 688 kB 00:00 2023-12-01T01:07:02.287 INFO:teuthology.orchestra.run.smithi203.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 252 kB/s | 34 kB 00:00 2023-12-01T01:07:02.311 INFO:teuthology.orchestra.run.smithi077.stdout:(79/131): python3-jinja2-2.10.1-3.el8.noarch.rp 3.0 MB/s | 538 kB 00:00 2023-12-01T01:07:02.370 INFO:teuthology.orchestra.run.smithi077.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.5 MB/s | 187 kB 00:00 2023-12-01T01:07:02.429 INFO:teuthology.orchestra.run.smithi077.stdout:(81/131): python3-mako-1.0.6-13.el8.noarch.rpm 670 kB/s | 157 kB 00:00 2023-12-01T01:07:02.462 INFO:teuthology.orchestra.run.smithi077.stdout:(82/131): automake-1.16.1-7.el8.noarch.rpm 4.6 MB/s | 713 kB 00:00 2023-12-01T01:07:02.596 INFO:teuthology.orchestra.run.smithi077.stdout:(83/131): python3-jsonpatch-1.21-2.el8.noarch.r 159 kB/s | 27 kB 00:00 2023-12-01T01:07:02.621 INFO:teuthology.orchestra.run.smithi077.stdout:(84/131): flex-2.6.1-9.el8.x86_64.rpm 2.0 MB/s | 320 kB 00:00 2023-12-01T01:07:02.705 INFO:teuthology.orchestra.run.smithi077.stdout:(85/131): python3-babel-2.5.1-7.el8.noarch.rpm 14 MB/s | 4.8 MB 00:00 2023-12-01T01:07:02.730 INFO:teuthology.orchestra.run.smithi077.stdout:(86/131): python3-prettytable-0.7.2-14.el8.noar 328 kB/s | 44 kB 00:00 2023-12-01T01:07:02.737 INFO:teuthology.orchestra.run.smithi203.stdout:(57/131): python3-routes-2.4.1-12.el8.noarch.rp 436 kB/s | 196 kB 00:00 2023-12-01T01:07:02.814 INFO:teuthology.orchestra.run.smithi077.stdout:(87/131): python3-werkzeug-0.12.2-4.el8.noarch. 4.1 MB/s | 457 kB 00:00 2023-12-01T01:07:02.839 INFO:teuthology.orchestra.run.smithi077.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 2.0 MB/s | 225 kB 00:00 2023-12-01T01:07:02.906 INFO:teuthology.orchestra.run.smithi077.stdout:(89/131): python3-numpy-1.14.3-10.el8.x86_64.rp 13 MB/s | 3.7 MB 00:00 2023-12-01T01:07:02.931 INFO:teuthology.orchestra.run.smithi077.stdout:(90/131): uuid-1.6.2-43.el8.x86_64.rpm 546 kB/s | 64 kB 00:00 2023-12-01T01:07:02.948 INFO:teuthology.orchestra.run.smithi077.stdout:(91/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.1 MB/s | 126 kB 00:00 2023-12-01T01:07:02.995 INFO:teuthology.orchestra.run.smithi203.stdout:(58/131): python3-kubernetes-11.0.0-6.el8.noarc 477 kB/s | 1.2 MB 00:02 2023-12-01T01:07:03.021 INFO:teuthology.orchestra.run.smithi203.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 226 kB/s | 64 kB 00:00 2023-12-01T01:07:03.032 INFO:teuthology.orchestra.run.smithi077.stdout:(92/131): python3-jsonpointer-1.10-11.el8.noarc 159 kB/s | 20 kB 00:00 2023-12-01T01:07:03.074 INFO:teuthology.orchestra.run.smithi077.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 5.5 MB/s | 709 kB 00:00 2023-12-01T01:07:03.119 INFO:teuthology.orchestra.run.smithi077.stdout:(94/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 551 kB/s | 103 kB 00:00 2023-12-01T01:07:03.224 INFO:teuthology.orchestra.run.smithi077.stdout:(95/131): openblas-threads-0.3.15-3.el8.x86_64. 25 MB/s | 4.9 MB 00:00 2023-12-01T01:07:03.250 INFO:teuthology.orchestra.run.smithi077.stdout:(96/131): libpq-13.5-1.el8.x86_64.rpm 1.1 MB/s | 198 kB 00:00 2023-12-01T01:07:03.289 INFO:teuthology.orchestra.run.smithi203.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 64 kB/s | 19 kB 00:00 2023-12-01T01:07:03.314 INFO:teuthology.orchestra.run.smithi203.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 82 kB/s | 24 kB 00:00 2023-12-01T01:07:03.325 INFO:teuthology.orchestra.run.smithi077.stdout:(97/131): jq-1.6-3.el8.x86_64.rpm 2.0 MB/s | 202 kB 00:00 2023-12-01T01:07:03.367 INFO:teuthology.orchestra.run.smithi077.stdout:(98/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.0 MB/s | 354 kB 00:00 2023-12-01T01:07:03.431 INFO:teuthology.orchestra.run.smithi203.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 205 kB/s | 29 kB 00:00 2023-12-01T01:07:03.451 INFO:teuthology.orchestra.run.smithi077.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 14 MB/s | 4.7 MB 00:00 2023-12-01T01:07:03.456 INFO:teuthology.orchestra.run.smithi203.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 193 kB/s | 27 kB 00:00 2023-12-01T01:07:03.518 INFO:teuthology.orchestra.run.smithi077.stdout:(100/131): postgresql-contrib-10.21-2.module+el 5.2 MB/s | 808 kB 00:00 2023-12-01T01:07:03.585 INFO:teuthology.orchestra.run.smithi077.stdout:(101/131): postgresql-server-10.21-2.module+el8 20 MB/s | 5.1 MB 00:00 2023-12-01T01:07:03.635 INFO:teuthology.orchestra.run.smithi077.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 8.2 MB/s | 1.5 MB 00:00 2023-12-01T01:07:03.669 INFO:teuthology.orchestra.run.smithi077.stdout:(103/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 311 kB/s | 47 kB 00:00 2023-12-01T01:07:03.694 INFO:teuthology.orchestra.run.smithi077.stdout:(104/131): libselinux-devel-2.9-5.el8.x86_64.rp 1.8 MB/s | 200 kB 00:00 2023-12-01T01:07:03.727 INFO:teuthology.orchestra.run.smithi077.stdout:(105/131): python3-jwt-1.6.1-2.el8.noarch.rpm 461 kB/s | 43 kB 00:00 2023-12-01T01:07:03.794 INFO:teuthology.orchestra.run.smithi077.stdout:(106/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.7 MB/s | 220 kB 00:00 2023-12-01T01:07:03.845 INFO:teuthology.orchestra.run.smithi077.stdout:(107/131): keyutils-libs-devel-1.5.10-9.el8.x86 319 kB/s | 48 kB 00:00 2023-12-01T01:07:03.907 INFO:teuthology.orchestra.run.smithi203.stdout:(64/131): python3-webob-1.8.5-1.el8.1.noarch.rp 558 kB/s | 251 kB 00:00 2023-12-01T01:07:03.912 INFO:teuthology.orchestra.run.smithi137.stdout:(131/131): ceph-test-18.2.0-1249.ga6f82997.el8. 2.5 MB/s | 47 MB 00:18 2023-12-01T01:07:03.915 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2023-12-01T01:07:03.916 INFO:teuthology.orchestra.run.smithi137.stdout:Total 8.9 MB/s | 203 MB 00:22 2023-12-01T01:07:03.928 INFO:teuthology.orchestra.run.smithi077.stdout:(108/131): libsepol-devel-2.9-3.el8.x86_64.rpm 434 kB/s | 87 kB 00:00 2023-12-01T01:07:03.954 INFO:teuthology.orchestra.run.smithi077.stdout:(109/131): python3-requests-oauthlib-1.0.0-1.el 268 kB/s | 43 kB 00:00 2023-12-01T01:07:03.974 INFO:teuthology.orchestra.run.smithi203.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 471 kB/s | 255 kB 00:00 2023-12-01T01:07:04.012 INFO:teuthology.orchestra.run.smithi077.stdout:(110/131): python3-pycparser-2.14-14.el8.noarch 652 kB/s | 109 kB 00:00 2023-12-01T01:07:04.032 INFO:teuthology.orchestra.run.smithi203.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 489 kB/s | 61 kB 00:00 2023-12-01T01:07:04.087 INFO:teuthology.orchestra.run.smithi077.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 4.1 MB/s | 560 kB 00:00 2023-12-01T01:07:04.129 INFO:teuthology.orchestra.run.smithi077.stdout:(112/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.1 MB/s | 229 kB 00:00 2023-12-01T01:07:04.166 INFO:teuthology.orchestra.run.smithi203.stdout:(67/131): python3-zc-lockfile-2.0-2.el8.noarch. 172 kB/s | 23 kB 00:00 2023-12-01T01:07:04.182 INFO:teuthology.orchestra.run.smithi077.stdout:(113/131): libaio-devel-0.3.112-1.el8.x86_64.rp 112 kB/s | 19 kB 00:00 2023-12-01T01:07:04.210 INFO:teuthology.orchestra.run.smithi203.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 365 kB/s | 85 kB 00:00 2023-12-01T01:07:04.233 INFO:teuthology.orchestra.run.smithi203.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-12-01T01:07:04.247 INFO:teuthology.orchestra.run.smithi077.stdout:(114/131): python3-oauthlib-2.1.0-1.el8.noarch. 974 kB/s | 155 kB 00:00 2023-12-01T01:07:04.280 INFO:teuthology.orchestra.run.smithi077.stdout:(115/131): python3-cryptography-3.2.1-5.el8.x86 3.6 MB/s | 559 kB 00:00 2023-12-01T01:07:04.333 INFO:teuthology.orchestra.run.smithi203.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 468 kB/s | 47 kB 00:00 2023-12-01T01:07:04.381 INFO:teuthology.orchestra.run.smithi077.stdout:(116/131): pcre2-devel-10.32-2.el8.x86_64.rpm 4.4 MB/s | 605 kB 00:00 2023-12-01T01:07:04.459 INFO:teuthology.orchestra.run.smithi203.stdout:(71/131): libselinux-devel-2.9-5.el8.x86_64.rpm 1.6 MB/s | 200 kB 00:00 2023-12-01T01:07:04.581 INFO:teuthology.orchestra.run.smithi077.stdout:(117/131): libverto-devel-0.3.0-5.el8.x86_64.rp 45 kB/s | 18 kB 00:00 2023-12-01T01:07:04.598 INFO:teuthology.orchestra.run.smithi077.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 749 kB/s | 238 kB 00:00 2023-12-01T01:07:04.609 INFO:teuthology.orchestra.run.smithi203.stdout:(72/131): python3-jwt-1.6.1-2.el8.noarch.rpm 284 kB/s | 43 kB 00:00 2023-12-01T01:07:04.665 INFO:teuthology.orchestra.run.smithi077.stdout:(119/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 8.2 MB/s | 2.3 MB 00:00 2023-12-01T01:07:04.690 INFO:teuthology.orchestra.run.smithi077.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 900 kB/s | 83 kB 00:00 2023-12-01T01:07:04.707 INFO:teuthology.orchestra.run.smithi077.stdout:(121/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 575 kB/s | 72 kB 00:00 2023-12-01T01:07:04.727 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2023-12-01T01:07:04.747 INFO:teuthology.orchestra.run.smithi203.stdout:(73/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.6 MB/s | 220 kB 00:00 2023-12-01T01:07:04.816 INFO:teuthology.orchestra.run.smithi077.stdout:(122/131): elfutils-libelf-devel-0.186-1.el8.x8 398 kB/s | 60 kB 00:00 2023-12-01T01:07:04.846 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2023-12-01T01:07:04.846 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2023-12-01T01:07:04.849 INFO:teuthology.orchestra.run.smithi077.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 406 kB/s | 58 kB 00:00 2023-12-01T01:07:04.891 INFO:teuthology.orchestra.run.smithi077.stdout:(124/131): libuuid-devel-2.32.1-35.el8.x86_64.r 488 kB/s | 98 kB 00:00 2023-12-01T01:07:04.917 INFO:teuthology.orchestra.run.smithi077.stdout:(125/131): libcom_err-devel-1.45.6-4.el8.x86_64 383 kB/s | 39 kB 00:00 2023-12-01T01:07:04.959 INFO:teuthology.orchestra.run.smithi203.stdout:(74/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 226 kB/s | 48 kB 00:00 2023-12-01T01:07:04.975 INFO:teuthology.orchestra.run.smithi077.stdout:(126/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 5.0 MB/s | 644 kB 00:00 2023-12-01T01:07:05.059 INFO:teuthology.orchestra.run.smithi077.stdout:(127/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 1.0 MB/s | 171 kB 00:00 2023-12-01T01:07:05.102 INFO:teuthology.orchestra.run.smithi203.stdout:(75/131): libsepol-devel-2.9-3.el8.x86_64.rpm 614 kB/s | 87 kB 00:00 2023-12-01T01:07:05.134 INFO:teuthology.orchestra.run.smithi077.stdout:(128/131): NetworkManager-initscripts-updown-1. 634 kB/s | 138 kB 00:00 2023-12-01T01:07:05.285 INFO:teuthology.orchestra.run.smithi077.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 5.2 MB/s | 798 kB 00:00 2023-12-01T01:07:05.552 INFO:teuthology.orchestra.run.smithi077.stdout:(130/131): librbd1-18.2.0-1249.ga6f82997.el8.x8 7.2 MB/s | 3.6 MB 00:00 2023-12-01T01:07:05.655 INFO:teuthology.orchestra.run.smithi203.stdout:(76/131): ceph-test-18.2.0-1249.ga6f82997.el8.x 4.4 MB/s | 47 MB 00:10 2023-12-01T01:07:05.686 INFO:teuthology.orchestra.run.smithi203.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 73 kB/s | 43 kB 00:00 2023-12-01T01:07:05.844 INFO:teuthology.orchestra.run.smithi203.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 578 kB/s | 109 kB 00:00 2023-12-01T01:07:05.887 INFO:teuthology.orchestra.run.smithi203.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.1 MB/s | 229 kB 00:00 2023-12-01T01:07:05.962 INFO:teuthology.orchestra.run.smithi203.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 4.7 MB/s | 560 kB 00:00 2023-12-01T01:07:05.972 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2023-12-01T01:07:05.976 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2023-12-01T01:07:05.978 INFO:teuthology.orchestra.run.smithi203.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 203 kB/s | 19 kB 00:00 2023-12-01T01:07:06.018 INFO:teuthology.orchestra.run.smithi077.stdout:(131/131): librados2-18.2.0-1249.ga6f82997.el8. 3.5 MB/s | 3.7 MB 00:01 2023-12-01T01:07:06.023 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2023-12-01T01:07:06.023 INFO:teuthology.orchestra.run.smithi077.stdout:Total 7.1 MB/s | 203 MB 00:28 2023-12-01T01:07:06.054 INFO:teuthology.orchestra.run.smithi203.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 1.6 MB/s | 155 kB 00:00 2023-12-01T01:07:06.138 INFO:teuthology.orchestra.run.smithi203.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 3.4 MB/s | 559 kB 00:00 2023-12-01T01:07:06.163 INFO:teuthology.orchestra.run.smithi203.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 164 kB/s | 18 kB 00:00 2023-12-01T01:07:06.272 INFO:teuthology.orchestra.run.smithi203.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 4.4 MB/s | 605 kB 00:00 2023-12-01T01:07:06.297 INFO:teuthology.orchestra.run.smithi203.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 1.7 MB/s | 238 kB 00:00 2023-12-01T01:07:06.414 INFO:teuthology.orchestra.run.smithi203.stdout:(87/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 16 MB/s | 2.3 MB 00:00 2023-12-01T01:07:06.506 INFO:teuthology.orchestra.run.smithi203.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 346 kB/s | 72 kB 00:00 2023-12-01T01:07:06.581 INFO:teuthology.orchestra.run.smithi203.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 495 kB/s | 83 kB 00:00 2023-12-01T01:07:06.623 INFO:teuthology.orchestra.run.smithi203.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 511 kB/s | 60 kB 00:00 2023-12-01T01:07:06.690 INFO:teuthology.orchestra.run.smithi203.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 904 kB/s | 98 kB 00:00 2023-12-01T01:07:06.749 INFO:teuthology.orchestra.run.smithi203.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 460 kB/s | 58 kB 00:00 2023-12-01T01:07:06.791 INFO:teuthology.orchestra.run.smithi203.stdout:(93/131): thrift-0.13.0-2.el8.x86_64.rpm 677 kB/s | 1.7 MB 00:02 2023-12-01T01:07:06.825 INFO:teuthology.orchestra.run.smithi203.stdout:(94/131): libcom_err-devel-1.45.6-4.el8.x86_64. 288 kB/s | 39 kB 00:00 2023-12-01T01:07:06.865 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2023-12-01T01:07:06.900 INFO:teuthology.orchestra.run.smithi203.stdout:(95/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 4.2 MB/s | 644 kB 00:00 2023-12-01T01:07:06.959 INFO:teuthology.orchestra.run.smithi203.stdout:(96/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 1.0 MB/s | 171 kB 00:00 2023-12-01T01:07:06.985 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2023-12-01T01:07:06.985 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2023-12-01T01:07:07.014 INFO:teuthology.orchestra.run.smithi203.stdout:(97/131): NetworkManager-initscripts-updown-1.3 752 kB/s | 138 kB 00:00 2023-12-01T01:07:07.126 INFO:teuthology.orchestra.run.smithi203.stdout:(98/131): python3-pyasn1-modules-0.3.7-6.el8.no 660 kB/s | 110 kB 00:00 2023-12-01T01:07:07.226 INFO:teuthology.orchestra.run.smithi203.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 255 kB/s | 54 kB 00:00 2023-12-01T01:07:07.368 INFO:teuthology.orchestra.run.smithi203.stdout:(100/131): python3-scipy-1.0.0-21.module+el8.5. 29 MB/s | 14 MB 00:00 2023-12-01T01:07:07.394 INFO:teuthology.orchestra.run.smithi203.stdout:(101/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 969 kB/s | 259 kB 00:00 2023-12-01T01:07:07.419 INFO:teuthology.orchestra.run.smithi203.stdout:(102/131): socat-1.7.4.1-1.el8.x86_64.rpm 1.6 MB/s | 323 kB 00:00 2023-12-01T01:07:07.511 INFO:teuthology.orchestra.run.smithi203.stdout:(103/131): python3-markupsafe-0.23-19.el8.x86_6 273 kB/s | 39 kB 00:00 2023-12-01T01:07:07.544 INFO:teuthology.orchestra.run.smithi203.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 4.2 MB/s | 538 kB 00:00 2023-12-01T01:07:07.578 INFO:teuthology.orchestra.run.smithi203.stdout:(105/131): bison-3.0.4-10.el8.x86_64.rpm 3.6 MB/s | 688 kB 00:00 2023-12-01T01:07:07.637 INFO:teuthology.orchestra.run.smithi203.stdout:(106/131): python3-mako-1.0.6-13.el8.noarch.rpm 1.2 MB/s | 157 kB 00:00 2023-12-01T01:07:07.687 INFO:teuthology.orchestra.run.smithi203.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 6.4 MB/s | 713 kB 00:00 2023-12-01T01:07:07.720 INFO:teuthology.orchestra.run.smithi203.stdout:(108/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.0 MB/s | 187 kB 00:00 2023-12-01T01:07:07.865 INFO:teuthology.orchestra.run.smithi203.stdout:(109/131): python3-babel-2.5.1-7.el8.noarch.rpm 21 MB/s | 4.8 MB 00:00 2023-12-01T01:07:07.879 INFO:teuthology.orchestra.run.smithi203.stdout:(110/131): python3-jsonpatch-1.21-2.el8.noarch. 138 kB/s | 27 kB 00:00 2023-12-01T01:07:07.930 INFO:teuthology.orchestra.run.smithi203.stdout:(111/131): flex-2.6.1-9.el8.x86_64.rpm 1.5 MB/s | 320 kB 00:00 2023-12-01T01:07:07.997 INFO:teuthology.orchestra.run.smithi203.stdout:(112/131): python3-prettytable-0.7.2-14.el8.noa 328 kB/s | 44 kB 00:00 2023-12-01T01:07:08.056 INFO:teuthology.orchestra.run.smithi203.stdout:(113/131): python3-werkzeug-0.12.2-4.el8.noarch 3.6 MB/s | 457 kB 00:00 2023-12-01T01:07:08.092 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2023-12-01T01:07:08.096 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2023-12-01T01:07:08.123 INFO:teuthology.orchestra.run.smithi203.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 15 MB/s | 3.7 MB 00:00 2023-12-01T01:07:08.181 INFO:teuthology.orchestra.run.smithi203.stdout:(115/131): python3-numpy-f2py-1.14.3-10.el8.x86 1.2 MB/s | 225 kB 00:00 2023-12-01T01:07:08.215 INFO:teuthology.orchestra.run.smithi203.stdout:(116/131): uuid-1.6.2-43.el8.x86_64.rpm 403 kB/s | 64 kB 00:00 2023-12-01T01:07:08.274 INFO:teuthology.orchestra.run.smithi203.stdout:(117/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 839 kB/s | 126 kB 00:00 2023-12-01T01:07:08.324 INFO:teuthology.orchestra.run.smithi203.stdout:(118/131): python3-jsonpointer-1.10-11.el8.noar 140 kB/s | 20 kB 00:00 2023-12-01T01:07:08.374 INFO:teuthology.orchestra.run.smithi203.stdout:(119/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 650 kB/s | 103 kB 00:00 2023-12-01T01:07:08.416 INFO:teuthology.orchestra.run.smithi203.stdout:(120/131): libtool-2.4.6-25.el8.x86_64.rpm 4.9 MB/s | 709 kB 00:00 2023-12-01T01:07:08.633 INFO:teuthology.orchestra.run.smithi203.stdout:(121/131): openblas-threads-0.3.15-3.el8.x86_64 16 MB/s | 4.9 MB 00:00 2023-12-01T01:07:08.709 INFO:teuthology.orchestra.run.smithi203.stdout:(122/131): libpq-13.5-1.el8.x86_64.rpm 592 kB/s | 198 kB 00:00 2023-12-01T01:07:09.000 INFO:teuthology.orchestra.run.smithi203.stdout:(123/131): jq-1.6-3.el8.x86_64.rpm 551 kB/s | 202 kB 00:00 2023-12-01T01:07:09.034 INFO:teuthology.orchestra.run.smithi203.stdout:(124/131): librdkafka-0.11.4-3.el8.x86_64.rpm 1.1 MB/s | 354 kB 00:00 2023-12-01T01:07:09.168 INFO:teuthology.orchestra.run.smithi203.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 6.3 MB/s | 4.7 MB 00:00 2023-12-01T01:07:09.210 INFO:teuthology.orchestra.run.smithi203.stdout:(126/131): postgresql-contrib-10.21-2.module+el 4.5 MB/s | 808 kB 00:00 2023-12-01T01:07:09.318 INFO:teuthology.orchestra.run.smithi203.stdout:(127/131): postgresql-server-10.21-2.module+el8 16 MB/s | 5.1 MB 00:00 2023-12-01T01:07:09.377 INFO:teuthology.orchestra.run.smithi203.stdout:(128/131): postgresql-10.21-2.module+el8.6.0+15 7.2 MB/s | 1.5 MB 00:00 2023-12-01T01:07:09.577 INFO:teuthology.orchestra.run.smithi203.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 3.9 MB/s | 798 kB 00:00 2023-12-01T01:07:09.794 INFO:teuthology.orchestra.run.smithi203.stdout:(130/131): librbd1-18.2.0-1249.ga6f82997.el8.x8 7.5 MB/s | 3.6 MB 00:00 2023-12-01T01:07:10.003 INFO:teuthology.orchestra.run.smithi203.stdout:(131/131): librados2-18.2.0-1249.ga6f82997.el8. 4.6 MB/s | 3.7 MB 00:00 2023-12-01T01:07:10.006 INFO:teuthology.orchestra.run.smithi203.stdout:-------------------------------------------------------------------------------- 2023-12-01T01:07:10.006 INFO:teuthology.orchestra.run.smithi203.stdout:Total 10 MB/s | 203 MB 00:19 2023-12-01T01:07:10.839 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction check 2023-12-01T01:07:10.959 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction check succeeded. 2023-12-01T01:07:10.960 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction test 2023-12-01T01:07:11.354 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2023-12-01T01:07:11.564 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-12-01T01:07:11.780 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-12-01T01:07:11.967 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-12-01T01:07:12.073 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction test succeeded. 2023-12-01T01:07:12.078 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction 2023-12-01T01:07:12.095 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-12-01T01:07:12.316 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-12-01T01:07:12.716 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-12-01T01:07:12.748 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2023-12-01T01:07:12.917 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-12-01T01:07:12.949 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : librados2-2:18.2.0-1249.ga6f82997.el8.x86_64 5/134 2023-12-01T01:07:13.078 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-12-01T01:07:13.275 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-12-01T01:07:13.343 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librados2-2:18.2.0-1249.ga6f82997.el8.x86_64 5/134 2023-12-01T01:07:13.452 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-12-01T01:07:13.558 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-12-01T01:07:13.566 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : librbd1-2:18.2.0-1249.ga6f82997.el8.x86_64 6/134 2023-12-01T01:07:13.694 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librbd1-2:18.2.0-1249.ga6f82997.el8.x86_64 6/134 2023-12-01T01:07:13.810 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-12-01T01:07:13.907 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephfs2-2:18.2.0-1249.ga6f82997.el8.x86_64 7/134 2023-12-01T01:07:13.999 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libcephfs2-2:18.2.0-1249.ga6f82997.el8.x86_64 7/134 2023-12-01T01:07:14.179 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rados-2:18.2.0-1249.ga6f82997.el8.x86_64 8/134 2023-12-01T01:07:14.207 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-12-01T01:07:14.333 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-12-01T01:07:14.410 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : librados2-2:18.2.0-1249.ga6f82997.el8.x86_64 6/134 2023-12-01T01:07:14.586 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-12-01T01:07:14.802 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librados2-2:18.2.0-1249.ga6f82997.el8.x86_64 6/134 2023-12-01T01:07:14.844 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-12-01T01:07:14.994 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : librbd1-2:18.2.0-1249.ga6f82997.el8.x86_64 7/134 2023-12-01T01:07:15.126 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librbd1-2:18.2.0-1249.ga6f82997.el8.x86_64 7/134 2023-12-01T01:07:15.292 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-12-01T01:07:15.410 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libcephfs2-2:18.2.0-1249.ga6f82997.el8.x86_64 8/134 2023-12-01T01:07:15.507 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libcephfs2-2:18.2.0-1249.ga6f82997.el8.x86_64 8/134 2023-12-01T01:07:15.553 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-12-01T01:07:15.685 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rados-2:18.2.0-1249.ga6f82997.el8.x86_64 9/134 2023-12-01T01:07:15.741 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephsqlite-2:18.2.0-1249.ga6f82997.el8.x86_64 14/134 2023-12-01T01:07:15.831 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libcephsqlite-2:18.2.0-1249.ga6f82997.el8.x86_64 14/134 2023-12-01T01:07:15.922 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-12-01T01:07:15.999 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libradosstriper1-2:18.2.0-1249.ga6f82997.el8.x86 15/134 2023-12-01T01:07:16.132 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libradosstriper1-2:18.2.0-1249.ga6f82997.el8.x86 15/134 2023-12-01T01:07:16.138 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-12-01T01:07:16.352 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-12-01T01:07:16.528 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-12-01T01:07:16.542 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-12-01T01:07:16.722 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-12-01T01:07:16.751 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-12-01T01:07:16.869 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libcephsqlite-2:18.2.0-1249.ga6f82997.el8.x86_64 14/134 2023-12-01T01:07:16.882 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-12-01T01:07:16.988 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libcephsqlite-2:18.2.0-1249.ga6f82997.el8.x86_64 14/134 2023-12-01T01:07:17.035 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-12-01T01:07:17.048 INFO:teuthology.orchestra.run.smithi203.stdout: Preparing : 1/1 2023-12-01T01:07:17.143 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libradosstriper1-2:18.2.0-1249.ga6f82997.el8.x86 15/134 2023-12-01T01:07:17.182 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-12-01T01:07:17.221 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libradosstriper1-2:18.2.0-1249.ga6f82997.el8.x86 15/134 2023-12-01T01:07:17.229 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-12-01T01:07:17.360 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-12-01T01:07:17.413 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-12-01T01:07:17.495 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-12-01T01:07:17.607 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-12-01T01:07:17.651 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-12-01T01:07:17.722 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-12-01T01:07:17.812 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-12-01T01:07:17.940 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-12-01T01:07:17.990 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-12-01T01:07:18.261 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-12-01T01:07:18.342 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-12-01T01:07:18.435 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-12-01T01:07:18.540 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : librados2-2:18.2.0-1249.ga6f82997.el8.x86_64 5/134 2023-12-01T01:07:18.587 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-12-01T01:07:18.790 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-12-01T01:07:18.920 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librados2-2:18.2.0-1249.ga6f82997.el8.x86_64 5/134 2023-12-01T01:07:19.028 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-12-01T01:07:19.098 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : librbd1-2:18.2.0-1249.ga6f82997.el8.x86_64 6/134 2023-12-01T01:07:19.201 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-12-01T01:07:19.228 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librbd1-2:18.2.0-1249.ga6f82997.el8.x86_64 6/134 2023-12-01T01:07:19.406 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephfs2-2:18.2.0-1249.ga6f82997.el8.x86_64 7/134 2023-12-01T01:07:19.466 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-12-01T01:07:19.695 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libcephfs2-2:18.2.0-1249.ga6f82997.el8.x86_64 7/134 2023-12-01T01:07:19.755 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-12-01T01:07:19.888 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-12-01T01:07:19.945 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rados-2:18.2.0-1249.ga6f82997.el8.x86_64 8/134 2023-12-01T01:07:19.983 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-12-01T01:07:20.101 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-12-01T01:07:20.175 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-12-01T01:07:20.216 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-12-01T01:07:20.358 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-12-01T01:07:20.427 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-12-01T01:07:20.430 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ceph-common-2:18.2.0-1249.ga6f82997.el8. 26/134 2023-12-01T01:07:20.561 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-12-01T01:07:20.592 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ceph-argparse-2:18.2.0-1249.ga6f82997.el 27/134 2023-12-01T01:07:20.646 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-12-01T01:07:20.802 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cephfs-2:18.2.0-1249.ga6f82997.el8.x86_6 28/134 2023-12-01T01:07:20.815 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-12-01T01:07:20.965 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-12-01T01:07:21.015 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-12-01T01:07:21.023 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-12-01T01:07:21.213 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-12-01T01:07:21.252 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-12-01T01:07:21.348 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephsqlite-2:18.2.0-1249.ga6f82997.el8.x86_64 14/134 2023-12-01T01:07:21.416 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-ceph-common-2:18.2.0-1249.ga6f82997.el8. 26/134 2023-12-01T01:07:21.449 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libcephsqlite-2:18.2.0-1249.ga6f82997.el8.x86_64 14/134 2023-12-01T01:07:21.570 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-ceph-argparse-2:18.2.0-1249.ga6f82997.el 27/134 2023-12-01T01:07:21.606 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libradosstriper1-2:18.2.0-1249.ga6f82997.el8.x86 15/134 2023-12-01T01:07:21.750 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libradosstriper1-2:18.2.0-1249.ga6f82997.el8.x86 15/134 2023-12-01T01:07:21.951 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-12-01T01:07:22.162 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-12-01T01:07:22.321 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-12-01T01:07:22.489 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-12-01T01:07:22.508 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cephfs-2:18.2.0-1249.ga6f82997.el8.x86_6 28/134 2023-12-01T01:07:22.617 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-12-01T01:07:22.772 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-12-01T01:07:22.793 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-12-01T01:07:22.832 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-12-01T01:07:23.121 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-12-01T01:07:23.377 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-12-01T01:07:23.384 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-12-01T01:07:23.531 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-12-01T01:07:23.576 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-12-01T01:07:23.814 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-12-01T01:07:24.120 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-12-01T01:07:24.133 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-12-01T01:07:24.330 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-12-01T01:07:24.383 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-12-01T01:07:24.609 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-12-01T01:07:24.851 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-12-01T01:07:25.032 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-12-01T01:07:25.036 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-12-01T01:07:25.228 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rbd-2:18.2.0-1249.ga6f82997.el8.x86_64 36/134 2023-12-01T01:07:25.359 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-12-01T01:07:25.412 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librados-devel-2:18.2.0-1249.ga6f82997.el8.x86_6 37/134 2023-12-01T01:07:25.531 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-12-01T01:07:25.563 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-12-01T01:07:25.652 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-12-01T01:07:25.745 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-12-01T01:07:25.745 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-12-01T01:07:25.855 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-12-01T01:07:25.888 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-12-01T01:07:25.957 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-12-01T01:07:25.985 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-12-01T01:07:26.052 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-12-01T01:07:26.153 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-ceph-common-2:18.2.0-1249.ga6f82997.el8. 26/134 2023-12-01T01:07:26.202 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-12-01T01:07:26.256 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rbd-2:18.2.0-1249.ga6f82997.el8.x86_64 36/134 2023-12-01T01:07:26.282 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-ceph-argparse-2:18.2.0-1249.ga6f82997.el 27/134 2023-12-01T01:07:26.442 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-12-01T01:07:26.480 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librados-devel-2:18.2.0-1249.ga6f82997.el8.x86_6 37/134 2023-12-01T01:07:26.501 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cephfs-2:18.2.0-1249.ga6f82997.el8.x86_6 28/134 2023-12-01T01:07:26.622 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-12-01T01:07:26.667 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-12-01T01:07:26.684 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-12-01T01:07:26.697 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-12-01T01:07:26.836 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-12-01T01:07:26.893 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-12-01T01:07:27.020 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-12-01T01:07:27.067 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-12-01T01:07:27.162 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-12-01T01:07:27.257 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-12-01T01:07:27.532 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-12-01T01:07:27.532 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-12-01T01:07:27.598 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-12-01T01:07:27.608 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-12-01T01:07:27.733 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-12-01T01:07:27.955 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-12-01T01:07:27.979 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-12-01T01:07:28.194 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-12-01T01:07:28.245 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-12-01T01:07:28.457 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-12-01T01:07:28.460 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-12-01T01:07:28.553 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-12-01T01:07:28.712 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-12-01T01:07:28.873 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-12-01T01:07:28.971 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-12-01T01:07:29.124 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-12-01T01:07:29.274 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-12-01T01:07:29.350 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-12-01T01:07:29.442 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-12-01T01:07:29.474 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-12-01T01:07:29.585 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-12-01T01:07:29.655 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-12-01T01:07:29.745 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-12-01T01:07:29.845 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-12-01T01:07:29.903 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-12-01T01:07:30.007 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-12-01T01:07:30.016 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-12-01T01:07:30.146 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-12-01T01:07:30.178 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-12-01T01:07:30.252 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-12-01T01:07:30.351 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-12-01T01:07:30.441 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-12-01T01:07:30.501 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-12-01T01:07:30.617 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-12-01T01:07:30.653 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-12-01T01:07:30.754 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-12-01T01:07:30.783 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-12-01T01:07:30.826 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rbd-2:18.2.0-1249.ga6f82997.el8.x86_64 36/134 2023-12-01T01:07:30.966 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-12-01T01:07:30.968 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librados-devel-2:18.2.0-1249.ga6f82997.el8.x86_6 37/134 2023-12-01T01:07:30.997 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-12-01T01:07:31.136 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-12-01T01:07:31.232 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-12-01T01:07:31.263 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-12-01T01:07:31.319 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-12-01T01:07:31.407 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-12-01T01:07:31.412 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-12-01T01:07:31.491 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-12-01T01:07:31.560 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-12-01T01:07:31.609 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-12-01T01:07:31.732 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-12-01T01:07:31.741 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-12-01T01:07:31.797 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-12-01T01:07:31.875 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-12-01T01:07:31.961 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-12-01T01:07:32.055 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-12-01T01:07:32.113 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-12-01T01:07:32.131 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-12-01T01:07:32.194 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-12-01T01:07:32.253 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-12-01T01:07:32.313 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-12-01T01:07:32.383 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-12-01T01:07:32.392 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-12-01T01:07:32.459 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-12-01T01:07:32.517 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-12-01T01:07:32.574 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-12-01T01:07:32.689 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-12-01T01:07:32.744 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-12-01T01:07:32.766 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-12-01T01:07:32.859 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-12-01T01:07:32.935 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-12-01T01:07:32.970 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-12-01T01:07:33.009 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-12-01T01:07:33.098 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-12-01T01:07:33.134 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-12-01T01:07:33.188 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-12-01T01:07:33.224 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-12-01T01:07:33.266 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-12-01T01:07:33.434 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-12-01T01:07:33.473 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-12-01T01:07:33.498 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-12-01T01:07:33.624 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-12-01T01:07:33.715 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-12-01T01:07:33.751 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-12-01T01:07:33.862 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-12-01T01:07:33.933 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-12-01T01:07:34.011 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-12-01T01:07:34.031 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-12-01T01:07:34.140 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-12-01T01:07:34.210 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-12-01T01:07:34.311 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-12-01T01:07:34.372 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-12-01T01:07:34.387 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-12-01T01:07:34.534 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-12-01T01:07:34.561 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-12-01T01:07:34.615 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-12-01T01:07:34.753 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-12-01T01:07:34.850 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-12-01T01:07:34.914 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-12-01T01:07:35.109 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-12-01T01:07:35.141 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-12-01T01:07:35.163 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-12-01T01:07:35.360 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-12-01T01:07:35.378 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-12-01T01:07:35.421 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-12-01T01:07:35.580 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-12-01T01:07:35.617 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-12-01T01:07:35.670 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-12-01T01:07:35.829 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-12-01T01:07:35.857 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-12-01T01:07:35.861 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-12-01T01:07:36.012 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-12-01T01:07:36.049 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-12-01T01:07:36.134 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-12-01T01:07:36.224 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-12-01T01:07:36.245 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-12-01T01:07:36.274 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-12-01T01:07:36.440 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-12-01T01:07:36.449 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-12-01T01:07:36.752 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-12-01T01:07:37.008 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-12-01T01:07:37.233 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-12-01T01:07:37.375 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-12-01T01:07:37.504 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-12-01T01:07:37.544 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-12-01T01:07:37.661 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-12-01T01:07:37.674 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-12-01T01:07:37.811 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-12-01T01:07:37.831 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-12-01T01:07:37.903 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-12-01T01:07:38.105 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-12-01T01:07:38.115 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-12-01T01:07:38.227 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-12-01T01:07:38.318 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-12-01T01:07:38.432 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-12-01T01:07:38.482 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-12-01T01:07:38.545 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-12-01T01:07:38.630 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-12-01T01:07:38.716 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-12-01T01:07:38.748 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-12-01T01:07:38.802 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-12-01T01:07:38.923 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-12-01T01:07:39.031 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-12-01T01:07:39.204 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-12-01T01:07:39.372 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-12-01T01:07:39.501 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-12-01T01:07:39.540 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-12-01T01:07:39.686 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-12-01T01:07:39.704 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-12-01T01:07:39.737 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librgw2-2:18.2.0-1249.ga6f82997.el8.x86_64 92/134 2023-12-01T01:07:39.810 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librgw2-2:18.2.0-1249.ga6f82997.el8.x86_64 92/134 2023-12-01T01:07:39.849 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librgw2-2:18.2.0-1249.ga6f82997.el8.x86_64 92/134 2023-12-01T01:07:39.923 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librgw2-2:18.2.0-1249.ga6f82997.el8.x86_64 92/134 2023-12-01T01:07:39.948 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-12-01T01:07:39.970 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rgw-2:18.2.0-1249.ga6f82997.el8.x86_64 93/134 2023-12-01T01:07:40.091 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rgw-2:18.2.0-1249.ga6f82997.el8.x86_64 93/134 2023-12-01T01:07:40.134 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-12-01T01:07:40.392 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-12-01T01:07:40.585 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-12-01T01:07:40.786 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-12-01T01:07:41.062 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-12-01T01:07:42.165 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-12-01T01:07:42.431 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-12-01T01:07:42.702 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-12-01T01:07:43.037 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-12-01T01:07:43.286 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-12-01T01:07:43.486 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-12-01T01:07:43.613 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-common-2:18.2.0-1249.ga6f82997.el8.x86_64 94/134 2023-12-01T01:07:43.759 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-common-2:18.2.0-1249.ga6f82997.el8.x86_64 94/134 2023-12-01T01:07:43.845 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-common-2:18.2.0-1249.ga6f82997.el8.x86_64 94/134 2023-12-01T01:07:43.999 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-common-2:18.2.0-1249.ga6f82997.el8.x86_64 94/134 2023-12-01T01:07:44.243 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-12-01T01:07:44.394 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librgw2-2:18.2.0-1249.ga6f82997.el8.x86_64 92/134 2023-12-01T01:07:44.434 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-common-2:18.2.0-1249.ga6f82997.el8.x86_64 94/134 2023-12-01T01:07:44.474 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librgw2-2:18.2.0-1249.ga6f82997.el8.x86_64 92/134 2023-12-01T01:07:44.587 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-common-2:18.2.0-1249.ga6f82997.el8.x86_64 94/134 2023-12-01T01:07:44.627 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rgw-2:18.2.0-1249.ga6f82997.el8.x86_64 93/134 2023-12-01T01:07:44.722 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-base-2:18.2.0-1249.ga6f82997.el8.x86_64 95/134 2023-12-01T01:07:44.808 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-base-2:18.2.0-1249.ga6f82997.el8.x86_64 95/134 2023-12-01T01:07:44.879 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-base-2:18.2.0-1249.ga6f82997.el8.x86_64 95/134 2023-12-01T01:07:44.965 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-base-2:18.2.0-1249.ga6f82997.el8.x86_64 95/134 2023-12-01T01:07:44.978 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-selinux-2:18.2.0-1249.ga6f82997.el8.x86_64 96/134 2023-12-01T01:07:45.140 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-selinux-2:18.2.0-1249.ga6f82997.el8.x86_64 96/134 2023-12-01T01:07:48.289 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-common-2:18.2.0-1249.ga6f82997.el8.x86_64 94/134 2023-12-01T01:07:48.544 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-common-2:18.2.0-1249.ga6f82997.el8.x86_64 94/134 2023-12-01T01:07:49.136 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-common-2:18.2.0-1249.ga6f82997.el8.x86_64 94/134 2023-12-01T01:07:49.415 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-base-2:18.2.0-1249.ga6f82997.el8.x86_64 95/134 2023-12-01T01:07:49.485 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-base-2:18.2.0-1249.ga6f82997.el8.x86_64 95/134 2023-12-01T01:07:49.693 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-selinux-2:18.2.0-1249.ga6f82997.el8.x86_64 96/134 2023-12-01T01:08:03.652 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-selinux-2:18.2.0-1249.ga6f82997.el8.x86_64 96/134 2023-12-01T01:08:03.652 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /sys 2023-12-01T01:08:03.652 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /proc 2023-12-01T01:08:03.652 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /dev 2023-12-01T01:08:03.652 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /run 2023-12-01T01:08:03.652 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /mnt 2023-12-01T01:08:03.652 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /var/tmp 2023-12-01T01:08:03.652 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /home 2023-12-01T01:08:03.652 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /tmp 2023-12-01T01:08:03.652 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /dev 2023-12-01T01:08:03.653 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:08:03.909 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-selinux-2:18.2.0-1249.ga6f82997.el8.x86_64 96/134 2023-12-01T01:08:03.909 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /sys 2023-12-01T01:08:03.909 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /proc 2023-12-01T01:08:03.909 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /dev 2023-12-01T01:08:03.909 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /run 2023-12-01T01:08:03.909 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /mnt 2023-12-01T01:08:03.909 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /var/tmp 2023-12-01T01:08:03.909 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /home 2023-12-01T01:08:03.909 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /tmp 2023-12-01T01:08:03.910 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /dev 2023-12-01T01:08:03.910 INFO:teuthology.orchestra.run.smithi137.stdout: 2023-12-01T01:08:04.004 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-volume-2:18.2.0-1249.ga6f82997.el8.noarch 97/134 2023-12-01T01:08:04.307 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-volume-2:18.2.0-1249.ga6f82997.el8.noarch 97/134 2023-12-01T01:08:05.568 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-volume-2:18.2.0-1249.ga6f82997.el8.noarch 97/134 2023-12-01T01:08:05.785 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-osd-2:18.2.0-1249.ga6f82997.el8.x86_64 98/134 2023-12-01T01:08:05.868 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-volume-2:18.2.0-1249.ga6f82997.el8.noarch 97/134 2023-12-01T01:08:06.011 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-osd-2:18.2.0-1249.ga6f82997.el8.x86_64 98/134 2023-12-01T01:08:06.068 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-osd-2:18.2.0-1249.ga6f82997.el8.x86_64 98/134 2023-12-01T01:08:06.196 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mds-2:18.2.0-1249.ga6f82997.el8.x86_64 99/134 2023-12-01T01:08:06.305 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-osd-2:18.2.0-1249.ga6f82997.el8.x86_64 98/134 2023-12-01T01:08:06.508 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mds-2:18.2.0-1249.ga6f82997.el8.x86_64 99/134 2023-12-01T01:08:06.651 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mds-2:18.2.0-1249.ga6f82997.el8.x86_64 99/134 2023-12-01T01:08:06.812 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mon-2:18.2.0-1249.ga6f82997.el8.x86_64 100/134 2023-12-01T01:08:06.830 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mon-2:18.2.0-1249.ga6f82997.el8.x86_64 100/134 2023-12-01T01:08:06.975 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mds-2:18.2.0-1249.ga6f82997.el8.x86_64 99/134 2023-12-01T01:08:07.124 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mon-2:18.2.0-1249.ga6f82997.el8.x86_64 100/134 2023-12-01T01:08:07.141 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mon-2:18.2.0-1249.ga6f82997.el8.x86_64 100/134 2023-12-01T01:08:08.450 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-selinux-2:18.2.0-1249.ga6f82997.el8.x86_64 96/134 2023-12-01T01:08:08.450 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /sys 2023-12-01T01:08:08.450 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /proc 2023-12-01T01:08:08.450 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /dev 2023-12-01T01:08:08.450 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /run 2023-12-01T01:08:08.450 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /mnt 2023-12-01T01:08:08.450 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /var/tmp 2023-12-01T01:08:08.450 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /home 2023-12-01T01:08:08.450 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /tmp 2023-12-01T01:08:08.450 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /dev 2023-12-01T01:08:08.450 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-12-01T01:08:08.508 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: cephadm-2:18.2.0-1249.ga6f82997.el8.noarch 101/134 2023-12-01T01:08:08.695 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : cephadm-2:18.2.0-1249.ga6f82997.el8.noarch 101/134 2023-12-01T01:08:08.773 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-volume-2:18.2.0-1249.ga6f82997.el8.noarch 97/134 2023-12-01T01:08:08.810 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-prometheus-alerts-2:18.2.0-1249.ga6f82997.e 102/134 2023-12-01T01:08:08.897 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: cephadm-2:18.2.0-1249.ga6f82997.el8.noarch 101/134 2023-12-01T01:08:09.020 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-grafana-dashboards-2:18.2.0-1249.ga6f82997. 103/134 2023-12-01T01:08:09.128 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephadm-2:18.2.0-1249.ga6f82997.el8.noarch 101/134 2023-12-01T01:08:09.203 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-cephadm-2:18.2.0-1249.ga6f82997.el8.noa 104/134 2023-12-01T01:08:09.263 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-prometheus-alerts-2:18.2.0-1249.ga6f82997.e 102/134 2023-12-01T01:08:09.439 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.0-1249.ga6f82997.el8.noa 104/134 2023-12-01T01:08:09.490 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-grafana-dashboards-2:18.2.0-1249.ga6f82997. 103/134 2023-12-01T01:08:09.706 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-cephadm-2:18.2.0-1249.ga6f82997.el8.noa 104/134 2023-12-01T01:08:09.726 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-dashboard-2:18.2.0-1249.ga6f82997.el8.n 105/134 2023-12-01T01:08:09.926 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.0-1249.ga6f82997.el8.noa 104/134 2023-12-01T01:08:10.251 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-dashboard-2:18.2.0-1249.ga6f82997.el8.n 105/134 2023-12-01T01:08:10.340 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-volume-2:18.2.0-1249.ga6f82997.el8.noarch 97/134 2023-12-01T01:08:10.520 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-osd-2:18.2.0-1249.ga6f82997.el8.x86_64 98/134 2023-12-01T01:08:10.689 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.0-1249.ga6f82997.el8.n 105/134 2023-12-01T01:08:10.752 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-osd-2:18.2.0-1249.ga6f82997.el8.x86_64 98/134 2023-12-01T01:08:10.861 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.0-1249.ga6f 106/134 2023-12-01T01:08:10.932 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mds-2:18.2.0-1249.ga6f82997.el8.x86_64 99/134 2023-12-01T01:08:10.953 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.0-1249.ga6f 106/134 2023-12-01T01:08:11.215 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.0-1249.ga6f82997.el8.n 105/134 2023-12-01T01:08:11.279 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-modules-core-2:18.2.0-1249.ga6f82997.el 107/134 2023-12-01T01:08:11.385 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mds-2:18.2.0-1249.ga6f82997.el8.x86_64 99/134 2023-12-01T01:08:11.407 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.0-1249.ga6f 106/134 2023-12-01T01:08:11.495 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-2:18.2.0-1249.ga6f82997.el8.x86_64 108/134 2023-12-01T01:08:11.498 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.0-1249.ga6f 106/134 2023-12-01T01:08:11.532 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mon-2:18.2.0-1249.ga6f82997.el8.x86_64 100/134 2023-12-01T01:08:11.532 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-2:18.2.0-1249.ga6f82997.el8.x86_64 108/134 2023-12-01T01:08:11.548 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mon-2:18.2.0-1249.ga6f82997.el8.x86_64 100/134 2023-12-01T01:08:11.695 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-rook-2:18.2.0-1249.ga6f82997.el8.noarch 109/134 2023-12-01T01:08:11.706 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.0-1249.ga6f82997.el8.noarch 109/134 2023-12-01T01:08:11.854 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-modules-core-2:18.2.0-1249.ga6f82997.el 107/134 2023-12-01T01:08:12.093 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-2:18.2.0-1249.ga6f82997.el8.x86_64 108/134 2023-12-01T01:08:12.129 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-2:18.2.0-1249.ga6f82997.el8.x86_64 108/134 2023-12-01T01:08:12.298 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-rook-2:18.2.0-1249.ga6f82997.el8.noarch 109/134 2023-12-01T01:08:12.309 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.0-1249.ga6f82997.el8.noarch 109/134 2023-12-01T01:08:12.951 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-2:18.2.0-1249.ga6f82997.el8.x86_64 110/134 2023-12-01T01:08:13.141 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-radosgw-2:18.2.0-1249.ga6f82997.el8.x86_64 111/134 2023-12-01T01:08:13.183 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-radosgw-2:18.2.0-1249.ga6f82997.el8.x86_64 111/134 2023-12-01T01:08:13.246 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: cephadm-2:18.2.0-1249.ga6f82997.el8.noarch 101/134 2023-12-01T01:08:13.345 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-immutable-object-cache-2:18.2.0-1249.ga6f82 112/134 2023-12-01T01:08:13.395 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.0-1249.ga6f82 112/134 2023-12-01T01:08:13.455 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephadm-2:18.2.0-1249.ga6f82997.el8.noarch 101/134 2023-12-01T01:08:13.537 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : cephfs-mirror-2:18.2.0-1249.ga6f82997.el8.x86_64 113/134 2023-12-01T01:08:13.579 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-prometheus-alerts-2:18.2.0-1249.ga6f82997.e 102/134 2023-12-01T01:08:13.591 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-2:18.2.0-1249.ga6f82997.el8.x86_64 110/134 2023-12-01T01:08:13.771 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-grafana-dashboards-2:18.2.0-1249.ga6f82997. 103/134 2023-12-01T01:08:13.806 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-radosgw-2:18.2.0-1249.ga6f82997.el8.x86_64 111/134 2023-12-01T01:08:13.866 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-radosgw-2:18.2.0-1249.ga6f82997.el8.x86_64 111/134 2023-12-01T01:08:13.882 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: cephfs-mirror-2:18.2.0-1249.ga6f82997.el8.x86_64 113/134 2023-12-01T01:08:13.997 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-cephadm-2:18.2.0-1249.ga6f82997.el8.noa 104/134 2023-12-01T01:08:14.023 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-immutable-object-cache-2:18.2.0-1249.ga6f82 112/134 2023-12-01T01:08:14.045 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : rbd-mirror-2:18.2.0-1249.ga6f82997.el8.x86_64 114/134 2023-12-01T01:08:14.068 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.0-1249.ga6f82 112/134 2023-12-01T01:08:14.218 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.0-1249.ga6f82997.el8.noa 104/134 2023-12-01T01:08:14.264 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephfs-mirror-2:18.2.0-1249.ga6f82997.el8.x86_64 113/134 2023-12-01T01:08:14.519 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-dashboard-2:18.2.0-1249.ga6f82997.el8.n 105/134 2023-12-01T01:08:14.608 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: cephfs-mirror-2:18.2.0-1249.ga6f82997.el8.x86_64 113/134 2023-12-01T01:08:14.781 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-mirror-2:18.2.0-1249.ga6f82997.el8.x86_64 114/134 2023-12-01T01:08:15.480 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.0-1249.ga6f82997.el8.n 105/134 2023-12-01T01:08:15.681 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.0-1249.ga6f 106/134 2023-12-01T01:08:15.772 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.0-1249.ga6f 106/134 2023-12-01T01:08:16.113 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-modules-core-2:18.2.0-1249.ga6f82997.el 107/134 2023-12-01T01:08:16.353 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-2:18.2.0-1249.ga6f82997.el8.x86_64 108/134 2023-12-01T01:08:16.390 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-2:18.2.0-1249.ga6f82997.el8.x86_64 108/134 2023-12-01T01:08:16.549 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-rook-2:18.2.0-1249.ga6f82997.el8.noarch 109/134 2023-12-01T01:08:16.558 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.0-1249.ga6f82997.el8.noarch 109/134 2023-12-01T01:08:17.837 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-2:18.2.0-1249.ga6f82997.el8.x86_64 110/134 2023-12-01T01:08:18.088 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-radosgw-2:18.2.0-1249.ga6f82997.el8.x86_64 111/134 2023-12-01T01:08:18.130 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-radosgw-2:18.2.0-1249.ga6f82997.el8.x86_64 111/134 2023-12-01T01:08:18.280 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-immutable-object-cache-2:18.2.0-1249.ga6f82 112/134 2023-12-01T01:08:18.331 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.0-1249.ga6f82 112/134 2023-12-01T01:08:18.498 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: rbd-mirror-2:18.2.0-1249.ga6f82997.el8.x86_64 114/134 2023-12-01T01:08:18.505 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephfs-mirror-2:18.2.0-1249.ga6f82997.el8.x86_64 113/134 2023-12-01T01:08:18.786 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-test-2:18.2.0-1249.ga6f82997.el8.x86_64 115/134 2023-12-01T01:08:18.850 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: cephfs-mirror-2:18.2.0-1249.ga6f82997.el8.x86_64 113/134 2023-12-01T01:08:18.935 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-12-01T01:08:19.030 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-mirror-2:18.2.0-1249.ga6f82997.el8.x86_64 114/134 2023-12-01T01:08:19.127 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-12-01T01:08:19.220 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: rbd-mirror-2:18.2.0-1249.ga6f82997.el8.x86_64 114/134 2023-12-01T01:08:20.017 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-12-01T01:08:20.131 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-test-2:18.2.0-1249.ga6f82997.el8.x86_64 115/134 2023-12-01T01:08:20.326 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-12-01T01:08:20.445 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-12-01T01:08:20.505 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-fuse-2:18.2.0-1249.ga6f82997.el8.x86_64 119/134 2023-12-01T01:08:20.633 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-fuse-2:18.2.0-1249.ga6f82997.el8.x86_64 117/134 2023-12-01T01:08:20.689 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-12-01T01:08:20.881 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libcephfs-devel-2:18.2.0-1249.ga6f82997.el8.x86_ 121/134 2023-12-01T01:08:20.921 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-12-01T01:08:21.045 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-12-01T01:08:21.105 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-12-01T01:08:21.300 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-12-01T01:08:21.559 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-12-01T01:08:21.758 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephfs-devel-2:18.2.0-1249.ga6f82997.el8.x86_ 121/134 2023-12-01T01:08:21.889 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-12-01T01:08:23.219 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-12-01T01:08:23.468 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-12-01T01:08:23.477 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: rbd-mirror-2:18.2.0-1249.ga6f82997.el8.x86_64 114/134 2023-12-01T01:08:23.486 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-12-01T01:08:23.644 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : cephfs-top-2:18.2.0-1249.ga6f82997.el8.noarch 124/134 2023-12-01T01:08:23.951 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : rbd-fuse-2:18.2.0-1249.ga6f82997.el8.x86_64 125/134 2023-12-01T01:08:24.115 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-12-01T01:08:24.274 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : rbd-nbd-2:18.2.0-1249.ga6f82997.el8.x86_64 126/134 2023-12-01T01:08:24.343 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-test-2:18.2.0-1249.ga6f82997.el8.x86_64 115/134 2023-12-01T01:08:24.405 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-12-01T01:08:24.422 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-12-01T01:08:24.572 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephfs-top-2:18.2.0-1249.ga6f82997.el8.noarch 124/134 2023-12-01T01:08:24.578 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-12-01T01:08:24.644 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-12-01T01:08:24.735 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-12-01T01:08:24.743 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-12-01T01:08:24.840 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-fuse-2:18.2.0-1249.ga6f82997.el8.x86_64 117/134 2023-12-01T01:08:24.862 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-fuse-2:18.2.0-1249.ga6f82997.el8.x86_64 125/134 2023-12-01T01:08:24.940 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-12-01T01:08:25.075 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-12-01T01:08:25.126 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-12-01T01:08:25.194 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-nbd-2:18.2.0-1249.ga6f82997.el8.x86_64 126/134 2023-12-01T01:08:25.204 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-12-01T01:08:25.254 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-12-01T01:08:25.385 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-12-01T01:08:25.397 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-12-01T01:08:25.397 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-12-01T01:08:25.444 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-12-01T01:08:25.520 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-12-01T01:08:25.613 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-12-01T01:08:25.613 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-12-01T01:08:25.650 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-12-01T01:08:25.704 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-12-01T01:08:25.786 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-12-01T01:08:25.849 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephfs-devel-2:18.2.0-1249.ga6f82997.el8.x86_ 121/134 2023-12-01T01:08:25.887 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-12-01T01:08:25.997 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-12-01T01:08:26.012 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-12-01T01:08:26.025 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-12-01T01:08:26.092 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-12-01T01:08:26.204 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-12-01T01:08:26.212 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-12-01T01:08:26.331 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-12-01T01:08:26.332 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-12-01T01:08:26.549 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-12-01T01:08:26.549 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-12-01T01:08:26.594 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-12-01T01:08:26.594 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-2:18.2.0-1249.ga6f82997.el8.x86_64 1/134 2023-12-01T01:08:26.594 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-base-2:18.2.0-1249.ga6f82997.el8.x86_64 2/134 2023-12-01T01:08:26.595 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-common-2:18.2.0-1249.ga6f82997.el8.x86_64 3/134 2023-12-01T01:08:26.595 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-fuse-2:18.2.0-1249.ga6f82997.el8.x86_64 4/134 2023-12-01T01:08:26.595 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-immutable-object-cache-2:18.2.0-1249.ga6f82 5/134 2023-12-01T01:08:26.595 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mds-2:18.2.0-1249.ga6f82997.el8.x86_64 6/134 2023-12-01T01:08:26.595 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-2:18.2.0-1249.ga6f82997.el8.x86_64 7/134 2023-12-01T01:08:26.595 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mon-2:18.2.0-1249.ga6f82997.el8.x86_64 8/134 2023-12-01T01:08:26.595 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-osd-2:18.2.0-1249.ga6f82997.el8.x86_64 9/134 2023-12-01T01:08:26.595 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-radosgw-2:18.2.0-1249.ga6f82997.el8.x86_64 10/134 2023-12-01T01:08:26.595 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-selinux-2:18.2.0-1249.ga6f82997.el8.x86_64 11/134 2023-12-01T01:08:26.595 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-test-2:18.2.0-1249.ga6f82997.el8.x86_64 12/134 2023-12-01T01:08:26.595 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : cephfs-mirror-2:18.2.0-1249.ga6f82997.el8.x86_64 13/134 2023-12-01T01:08:26.595 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libcephfs-devel-2:18.2.0-1249.ga6f82997.el8.x86_ 14/134 2023-12-01T01:08:26.595 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libcephfs2-2:18.2.0-1249.ga6f82997.el8.x86_64 15/134 2023-12-01T01:08:26.596 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libcephsqlite-2:18.2.0-1249.ga6f82997.el8.x86_64 16/134 2023-12-01T01:08:26.596 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librados-devel-2:18.2.0-1249.ga6f82997.el8.x86_6 17/134 2023-12-01T01:08:26.596 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libradosstriper1-2:18.2.0-1249.ga6f82997.el8.x86 18/134 2023-12-01T01:08:26.596 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librgw2-2:18.2.0-1249.ga6f82997.el8.x86_64 19/134 2023-12-01T01:08:26.596 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-ceph-argparse-2:18.2.0-1249.ga6f82997.el 20/134 2023-12-01T01:08:26.596 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-ceph-common-2:18.2.0-1249.ga6f82997.el8. 21/134 2023-12-01T01:08:26.596 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cephfs-2:18.2.0-1249.ga6f82997.el8.x86_6 22/134 2023-12-01T01:08:26.596 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rados-2:18.2.0-1249.ga6f82997.el8.x86_64 23/134 2023-12-01T01:08:26.596 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rbd-2:18.2.0-1249.ga6f82997.el8.x86_64 24/134 2023-12-01T01:08:26.596 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rgw-2:18.2.0-1249.ga6f82997.el8.x86_64 25/134 2023-12-01T01:08:26.596 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : rbd-fuse-2:18.2.0-1249.ga6f82997.el8.x86_64 26/134 2023-12-01T01:08:26.596 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : rbd-mirror-2:18.2.0-1249.ga6f82997.el8.x86_64 27/134 2023-12-01T01:08:26.596 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : rbd-nbd-2:18.2.0-1249.ga6f82997.el8.x86_64 28/134 2023-12-01T01:08:26.597 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-grafana-dashboards-2:18.2.0-1249.ga6f82997. 29/134 2023-12-01T01:08:26.597 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-cephadm-2:18.2.0-1249.ga6f82997.el8.noa 30/134 2023-12-01T01:08:26.606 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-dashboard-2:18.2.0-1249.ga6f82997.el8.n 31/134 2023-12-01T01:08:26.606 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.0-1249.ga6f 32/134 2023-12-01T01:08:26.607 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-modules-core-2:18.2.0-1249.ga6f82997.el 33/134 2023-12-01T01:08:26.607 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-rook-2:18.2.0-1249.ga6f82997.el8.noarch 34/134 2023-12-01T01:08:26.607 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-prometheus-alerts-2:18.2.0-1249.ga6f82997.e 35/134 2023-12-01T01:08:26.607 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-volume-2:18.2.0-1249.ga6f82997.el8.noarch 36/134 2023-12-01T01:08:26.607 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : cephadm-2:18.2.0-1249.ga6f82997.el8.noarch 37/134 2023-12-01T01:08:26.607 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : cephfs-top-2:18.2.0-1249.ga6f82997.el8.noarch 38/134 2023-12-01T01:08:26.607 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-12-01T01:08:26.607 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-12-01T01:08:26.607 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-12-01T01:08:26.607 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-12-01T01:08:26.607 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-12-01T01:08:26.608 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-12-01T01:08:26.608 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-12-01T01:08:26.608 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-12-01T01:08:26.608 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-12-01T01:08:26.608 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-12-01T01:08:26.608 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-12-01T01:08:26.608 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-12-01T01:08:26.608 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-12-01T01:08:26.608 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-12-01T01:08:26.608 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-12-01T01:08:26.608 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-12-01T01:08:26.609 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-12-01T01:08:26.609 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-12-01T01:08:26.609 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-12-01T01:08:26.609 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-12-01T01:08:26.609 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-12-01T01:08:26.609 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-12-01T01:08:26.610 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-12-01T01:08:26.610 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-12-01T01:08:26.610 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-12-01T01:08:26.610 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-12-01T01:08:26.610 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-12-01T01:08:26.610 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-12-01T01:08:26.610 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-12-01T01:08:26.611 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-12-01T01:08:26.611 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-12-01T01:08:26.613 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-12-01T01:08:26.613 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-12-01T01:08:26.613 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-12-01T01:08:26.613 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-12-01T01:08:26.614 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-12-01T01:08:26.614 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-12-01T01:08:26.614 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-12-01T01:08:26.614 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-12-01T01:08:26.614 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-12-01T01:08:26.614 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-12-01T01:08:26.615 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-12-01T01:08:26.615 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-12-01T01:08:26.615 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-12-01T01:08:26.615 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-12-01T01:08:26.615 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-12-01T01:08:26.615 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-12-01T01:08:26.616 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-12-01T01:08:26.616 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-12-01T01:08:26.616 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-12-01T01:08:26.616 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-12-01T01:08:26.616 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-12-01T01:08:26.616 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-12-01T01:08:26.617 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-12-01T01:08:26.617 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-12-01T01:08:26.617 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-12-01T01:08:26.617 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-12-01T01:08:26.617 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-12-01T01:08:26.617 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-12-01T01:08:26.618 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-12-01T01:08:26.618 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-12-01T01:08:26.618 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-12-01T01:08:26.618 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-12-01T01:08:26.618 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-12-01T01:08:26.618 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-12-01T01:08:26.619 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-12-01T01:08:26.619 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-12-01T01:08:26.619 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-12-01T01:08:26.619 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-12-01T01:08:26.619 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-12-01T01:08:26.619 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-12-01T01:08:26.620 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-12-01T01:08:26.620 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-12-01T01:08:26.620 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-12-01T01:08:26.620 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-12-01T01:08:26.620 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-12-01T01:08:26.620 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-12-01T01:08:26.621 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-12-01T01:08:26.621 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-12-01T01:08:26.621 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-12-01T01:08:26.621 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-12-01T01:08:26.621 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-12-01T01:08:26.621 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-12-01T01:08:26.622 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-12-01T01:08:26.622 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-12-01T01:08:26.622 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-12-01T01:08:26.622 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-12-01T01:08:26.622 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-12-01T01:08:26.623 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-12-01T01:08:26.623 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-12-01T01:08:26.623 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librados2-2:18.2.0-1249.ga6f82997.el8.x86_64 129/134 2023-12-01T01:08:26.623 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-12-01T01:08:26.623 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librbd1-2:18.2.0-1249.ga6f82997.el8.x86_64 131/134 2023-12-01T01:08:26.623 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-12-01T01:08:26.624 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-12-01T01:08:26.781 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-12-01T01:08:27.002 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-12-01T01:08:27.517 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-12-01T01:08:27.517 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-2:18.2.0-1249.ga6f82997.el8.x86_64 1/134 2023-12-01T01:08:27.517 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-base-2:18.2.0-1249.ga6f82997.el8.x86_64 2/134 2023-12-01T01:08:27.517 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-common-2:18.2.0-1249.ga6f82997.el8.x86_64 3/134 2023-12-01T01:08:27.518 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-fuse-2:18.2.0-1249.ga6f82997.el8.x86_64 4/134 2023-12-01T01:08:27.518 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-immutable-object-cache-2:18.2.0-1249.ga6f82 5/134 2023-12-01T01:08:27.518 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mds-2:18.2.0-1249.ga6f82997.el8.x86_64 6/134 2023-12-01T01:08:27.518 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-2:18.2.0-1249.ga6f82997.el8.x86_64 7/134 2023-12-01T01:08:27.518 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mon-2:18.2.0-1249.ga6f82997.el8.x86_64 8/134 2023-12-01T01:08:27.518 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-osd-2:18.2.0-1249.ga6f82997.el8.x86_64 9/134 2023-12-01T01:08:27.518 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-radosgw-2:18.2.0-1249.ga6f82997.el8.x86_64 10/134 2023-12-01T01:08:27.518 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-selinux-2:18.2.0-1249.ga6f82997.el8.x86_64 11/134 2023-12-01T01:08:27.518 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-test-2:18.2.0-1249.ga6f82997.el8.x86_64 12/134 2023-12-01T01:08:27.518 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephfs-mirror-2:18.2.0-1249.ga6f82997.el8.x86_64 13/134 2023-12-01T01:08:27.518 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephfs-devel-2:18.2.0-1249.ga6f82997.el8.x86_ 14/134 2023-12-01T01:08:27.518 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephfs2-2:18.2.0-1249.ga6f82997.el8.x86_64 15/134 2023-12-01T01:08:27.518 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephsqlite-2:18.2.0-1249.ga6f82997.el8.x86_64 16/134 2023-12-01T01:08:27.518 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados-devel-2:18.2.0-1249.ga6f82997.el8.x86_6 17/134 2023-12-01T01:08:27.519 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libradosstriper1-2:18.2.0-1249.ga6f82997.el8.x86 18/134 2023-12-01T01:08:27.519 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librgw2-2:18.2.0-1249.ga6f82997.el8.x86_64 19/134 2023-12-01T01:08:27.519 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ceph-argparse-2:18.2.0-1249.ga6f82997.el 20/134 2023-12-01T01:08:27.519 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ceph-common-2:18.2.0-1249.ga6f82997.el8. 21/134 2023-12-01T01:08:27.519 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cephfs-2:18.2.0-1249.ga6f82997.el8.x86_6 22/134 2023-12-01T01:08:27.519 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rados-2:18.2.0-1249.ga6f82997.el8.x86_64 23/134 2023-12-01T01:08:27.519 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rbd-2:18.2.0-1249.ga6f82997.el8.x86_64 24/134 2023-12-01T01:08:27.519 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rgw-2:18.2.0-1249.ga6f82997.el8.x86_64 25/134 2023-12-01T01:08:27.519 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-fuse-2:18.2.0-1249.ga6f82997.el8.x86_64 26/134 2023-12-01T01:08:27.519 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-mirror-2:18.2.0-1249.ga6f82997.el8.x86_64 27/134 2023-12-01T01:08:27.519 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-nbd-2:18.2.0-1249.ga6f82997.el8.x86_64 28/134 2023-12-01T01:08:27.519 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-grafana-dashboards-2:18.2.0-1249.ga6f82997. 29/134 2023-12-01T01:08:27.519 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-cephadm-2:18.2.0-1249.ga6f82997.el8.noa 30/134 2023-12-01T01:08:27.519 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-dashboard-2:18.2.0-1249.ga6f82997.el8.n 31/134 2023-12-01T01:08:27.519 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.0-1249.ga6f 32/134 2023-12-01T01:08:27.529 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-modules-core-2:18.2.0-1249.ga6f82997.el 33/134 2023-12-01T01:08:27.529 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-rook-2:18.2.0-1249.ga6f82997.el8.noarch 34/134 2023-12-01T01:08:27.529 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-prometheus-alerts-2:18.2.0-1249.ga6f82997.e 35/134 2023-12-01T01:08:27.529 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-volume-2:18.2.0-1249.ga6f82997.el8.noarch 36/134 2023-12-01T01:08:27.529 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephadm-2:18.2.0-1249.ga6f82997.el8.noarch 37/134 2023-12-01T01:08:27.529 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephfs-top-2:18.2.0-1249.ga6f82997.el8.noarch 38/134 2023-12-01T01:08:27.529 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-12-01T01:08:27.529 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-12-01T01:08:27.529 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-12-01T01:08:27.529 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-12-01T01:08:27.529 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-12-01T01:08:27.529 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-12-01T01:08:27.530 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-12-01T01:08:27.530 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-12-01T01:08:27.530 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-12-01T01:08:27.530 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-12-01T01:08:27.530 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-12-01T01:08:27.530 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-12-01T01:08:27.530 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-12-01T01:08:27.530 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-12-01T01:08:27.530 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-12-01T01:08:27.530 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-12-01T01:08:27.530 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-12-01T01:08:27.530 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-12-01T01:08:27.530 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-12-01T01:08:27.530 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-12-01T01:08:27.531 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-12-01T01:08:27.531 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-12-01T01:08:27.531 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-12-01T01:08:27.531 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-12-01T01:08:27.531 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-12-01T01:08:27.531 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-12-01T01:08:27.531 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-12-01T01:08:27.532 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-12-01T01:08:27.532 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-12-01T01:08:27.532 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-12-01T01:08:27.532 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-12-01T01:08:27.532 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-12-01T01:08:27.532 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-12-01T01:08:27.532 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-12-01T01:08:27.532 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-12-01T01:08:27.532 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-12-01T01:08:27.533 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-12-01T01:08:27.533 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-12-01T01:08:27.533 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-12-01T01:08:27.533 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-12-01T01:08:27.533 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-12-01T01:08:27.533 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-12-01T01:08:27.533 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-12-01T01:08:27.533 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-12-01T01:08:27.533 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-12-01T01:08:27.533 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-12-01T01:08:27.533 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-12-01T01:08:27.534 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-12-01T01:08:27.534 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-12-01T01:08:27.534 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-12-01T01:08:27.534 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-12-01T01:08:27.534 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-12-01T01:08:27.534 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-12-01T01:08:27.534 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-12-01T01:08:27.534 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-12-01T01:08:27.535 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-12-01T01:08:27.535 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-12-01T01:08:27.535 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-12-01T01:08:27.535 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-12-01T01:08:27.535 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-12-01T01:08:27.535 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-12-01T01:08:27.535 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-12-01T01:08:27.535 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-12-01T01:08:27.535 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-12-01T01:08:27.535 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-12-01T01:08:27.535 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-12-01T01:08:27.535 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-12-01T01:08:27.535 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-12-01T01:08:27.535 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-12-01T01:08:27.536 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-12-01T01:08:27.536 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-12-01T01:08:27.536 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-12-01T01:08:27.536 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-12-01T01:08:27.536 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-12-01T01:08:27.536 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-12-01T01:08:27.536 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-12-01T01:08:27.536 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-12-01T01:08:27.536 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-12-01T01:08:27.536 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-12-01T01:08:27.536 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-12-01T01:08:27.536 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-12-01T01:08:27.536 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-12-01T01:08:27.536 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-12-01T01:08:27.536 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-12-01T01:08:27.537 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-12-01T01:08:27.537 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-12-01T01:08:27.537 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-12-01T01:08:27.537 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-12-01T01:08:27.537 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-12-01T01:08:27.537 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-12-01T01:08:27.537 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados2-2:18.2.0-1249.ga6f82997.el8.x86_64 129/134 2023-12-01T01:08:27.537 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-12-01T01:08:27.537 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librbd1-2:18.2.0-1249.ga6f82997.el8.x86_64 131/134 2023-12-01T01:08:27.537 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-12-01T01:08:27.537 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-12-01T01:08:28.192 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-12-01T01:08:28.470 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-12-01T01:08:28.488 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-12-01T01:08:28.646 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephfs-top-2:18.2.0-1249.ga6f82997.el8.noarch 124/134 2023-12-01T01:08:28.936 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-fuse-2:18.2.0-1249.ga6f82997.el8.x86_64 125/134 2023-12-01T01:08:29.074 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-12-01T01:08:29.075 INFO:teuthology.orchestra.run.smithi077.stdout:Last metadata expiration check: 0:02:04 ago on Fri 01 Dec 2023 01:06:25 AM UTC. 2023-12-01T01:08:29.218 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-nbd-2:18.2.0-1249.ga6f82997.el8.x86_64 126/134 2023-12-01T01:08:29.604 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-12-01T01:08:29.812 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-12-01T01:08:29.897 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-12-01T01:08:30.078 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-12-01T01:08:30.091 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-12-01T01:08:30.133 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-12-01T01:08:30.133 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:02:01 ago on Fri 01 Dec 2023 01:06:29 AM UTC. 2023-12-01T01:08:30.278 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-12-01T01:08:30.286 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-12-01T01:08:30.422 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-12-01T01:08:30.422 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-12-01T01:08:30.631 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-12-01T01:08:30.631 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-12-01T01:08:30.875 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-12-01T01:08:31.068 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-12-01T01:08:31.594 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-12-01T01:08:31.595 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-2:18.2.0-1249.ga6f82997.el8.x86_64 1/134 2023-12-01T01:08:31.595 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-base-2:18.2.0-1249.ga6f82997.el8.x86_64 2/134 2023-12-01T01:08:31.595 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-common-2:18.2.0-1249.ga6f82997.el8.x86_64 3/134 2023-12-01T01:08:31.595 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-fuse-2:18.2.0-1249.ga6f82997.el8.x86_64 4/134 2023-12-01T01:08:31.595 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-immutable-object-cache-2:18.2.0-1249.ga6f82 5/134 2023-12-01T01:08:31.595 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mds-2:18.2.0-1249.ga6f82997.el8.x86_64 6/134 2023-12-01T01:08:31.595 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-2:18.2.0-1249.ga6f82997.el8.x86_64 7/134 2023-12-01T01:08:31.595 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mon-2:18.2.0-1249.ga6f82997.el8.x86_64 8/134 2023-12-01T01:08:31.595 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-osd-2:18.2.0-1249.ga6f82997.el8.x86_64 9/134 2023-12-01T01:08:31.595 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-radosgw-2:18.2.0-1249.ga6f82997.el8.x86_64 10/134 2023-12-01T01:08:31.595 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-selinux-2:18.2.0-1249.ga6f82997.el8.x86_64 11/134 2023-12-01T01:08:31.595 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-test-2:18.2.0-1249.ga6f82997.el8.x86_64 12/134 2023-12-01T01:08:31.596 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephfs-mirror-2:18.2.0-1249.ga6f82997.el8.x86_64 13/134 2023-12-01T01:08:31.596 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephfs-devel-2:18.2.0-1249.ga6f82997.el8.x86_ 14/134 2023-12-01T01:08:31.596 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephfs2-2:18.2.0-1249.ga6f82997.el8.x86_64 15/134 2023-12-01T01:08:31.596 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephsqlite-2:18.2.0-1249.ga6f82997.el8.x86_64 16/134 2023-12-01T01:08:31.596 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados-devel-2:18.2.0-1249.ga6f82997.el8.x86_6 17/134 2023-12-01T01:08:31.596 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libradosstriper1-2:18.2.0-1249.ga6f82997.el8.x86 18/134 2023-12-01T01:08:31.596 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librgw2-2:18.2.0-1249.ga6f82997.el8.x86_64 19/134 2023-12-01T01:08:31.596 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-ceph-argparse-2:18.2.0-1249.ga6f82997.el 20/134 2023-12-01T01:08:31.596 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-ceph-common-2:18.2.0-1249.ga6f82997.el8. 21/134 2023-12-01T01:08:31.596 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cephfs-2:18.2.0-1249.ga6f82997.el8.x86_6 22/134 2023-12-01T01:08:31.596 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rados-2:18.2.0-1249.ga6f82997.el8.x86_64 23/134 2023-12-01T01:08:31.597 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rbd-2:18.2.0-1249.ga6f82997.el8.x86_64 24/134 2023-12-01T01:08:31.597 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rgw-2:18.2.0-1249.ga6f82997.el8.x86_64 25/134 2023-12-01T01:08:31.597 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-fuse-2:18.2.0-1249.ga6f82997.el8.x86_64 26/134 2023-12-01T01:08:31.597 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-mirror-2:18.2.0-1249.ga6f82997.el8.x86_64 27/134 2023-12-01T01:08:31.597 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-nbd-2:18.2.0-1249.ga6f82997.el8.x86_64 28/134 2023-12-01T01:08:31.597 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-grafana-dashboards-2:18.2.0-1249.ga6f82997. 29/134 2023-12-01T01:08:31.597 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-cephadm-2:18.2.0-1249.ga6f82997.el8.noa 30/134 2023-12-01T01:08:31.597 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-dashboard-2:18.2.0-1249.ga6f82997.el8.n 31/134 2023-12-01T01:08:31.597 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.0-1249.ga6f 32/134 2023-12-01T01:08:31.597 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-modules-core-2:18.2.0-1249.ga6f82997.el 33/134 2023-12-01T01:08:31.598 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-rook-2:18.2.0-1249.ga6f82997.el8.noarch 34/134 2023-12-01T01:08:31.598 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-prometheus-alerts-2:18.2.0-1249.ga6f82997.e 35/134 2023-12-01T01:08:31.598 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-volume-2:18.2.0-1249.ga6f82997.el8.noarch 36/134 2023-12-01T01:08:31.598 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephadm-2:18.2.0-1249.ga6f82997.el8.noarch 37/134 2023-12-01T01:08:31.598 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephfs-top-2:18.2.0-1249.ga6f82997.el8.noarch 38/134 2023-12-01T01:08:31.598 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-12-01T01:08:31.598 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-12-01T01:08:31.598 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-12-01T01:08:31.598 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-12-01T01:08:31.599 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-12-01T01:08:31.599 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-12-01T01:08:31.599 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-12-01T01:08:31.599 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-12-01T01:08:31.599 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-12-01T01:08:31.599 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-12-01T01:08:31.599 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-12-01T01:08:31.599 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-12-01T01:08:31.599 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-12-01T01:08:31.599 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-12-01T01:08:31.599 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-12-01T01:08:31.599 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-12-01T01:08:31.600 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-12-01T01:08:31.600 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-12-01T01:08:31.600 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-12-01T01:08:31.600 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-12-01T01:08:31.600 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-12-01T01:08:31.600 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-12-01T01:08:31.600 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-12-01T01:08:31.600 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-12-01T01:08:31.600 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-12-01T01:08:31.600 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-12-01T01:08:31.600 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-12-01T01:08:31.601 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-12-01T01:08:31.601 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-12-01T01:08:31.601 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-12-01T01:08:31.601 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-12-01T01:08:31.601 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-12-01T01:08:31.601 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-12-01T01:08:31.601 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-12-01T01:08:31.601 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-12-01T01:08:31.601 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-12-01T01:08:31.601 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-12-01T01:08:31.601 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-12-01T01:08:31.602 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-12-01T01:08:31.602 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-12-01T01:08:31.602 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-12-01T01:08:31.602 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-12-01T01:08:31.602 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-12-01T01:08:31.602 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-12-01T01:08:31.602 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-12-01T01:08:31.602 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-12-01T01:08:31.602 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-12-01T01:08:31.602 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-12-01T01:08:31.602 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-12-01T01:08:31.603 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-12-01T01:08:31.603 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-12-01T01:08:31.603 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-12-01T01:08:31.604 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-12-01T01:08:31.604 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-12-01T01:08:31.604 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-12-01T01:08:31.604 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-12-01T01:08:31.604 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-12-01T01:08:31.604 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-12-01T01:08:31.605 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-12-01T01:08:31.605 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-12-01T01:08:31.605 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-12-01T01:08:31.605 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-12-01T01:08:31.606 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-12-01T01:08:31.606 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-12-01T01:08:31.606 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-12-01T01:08:31.606 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-12-01T01:08:31.606 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-12-01T01:08:31.606 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-12-01T01:08:31.606 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-12-01T01:08:31.606 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-12-01T01:08:31.606 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-12-01T01:08:31.606 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-12-01T01:08:31.606 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-12-01T01:08:31.606 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-12-01T01:08:31.606 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-12-01T01:08:31.607 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-12-01T01:08:31.607 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-12-01T01:08:31.607 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-12-01T01:08:31.607 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-12-01T01:08:31.607 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-12-01T01:08:31.607 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-12-01T01:08:31.607 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-12-01T01:08:31.608 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-12-01T01:08:31.608 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-12-01T01:08:31.608 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-12-01T01:08:31.608 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-12-01T01:08:31.608 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-12-01T01:08:31.608 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-12-01T01:08:31.608 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-12-01T01:08:31.608 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-12-01T01:08:31.608 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados2-2:18.2.0-1249.ga6f82997.el8.x86_64 129/134 2023-12-01T01:08:31.608 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-12-01T01:08:31.608 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librbd1-2:18.2.0-1249.ga6f82997.el8.x86_64 131/134 2023-12-01T01:08:31.608 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-12-01T01:08:31.608 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-12-01T01:08:33.439 INFO:teuthology.orchestra.run.smithi077.stdout:Installed products updated. 2023-12-01T01:08:33.650 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:08:33.650 INFO:teuthology.orchestra.run.smithi077.stdout:Upgraded: 2023-12-01T01:08:33.650 INFO:teuthology.orchestra.run.smithi077.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-12-01T01:08:33.650 INFO:teuthology.orchestra.run.smithi077.stdout: librados2-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:33.651 INFO:teuthology.orchestra.run.smithi077.stdout: librbd1-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:33.651 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2023-12-01T01:08:33.651 INFO:teuthology.orchestra.run.smithi077.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-12-01T01:08:33.651 INFO:teuthology.orchestra.run.smithi077.stdout: automake-1.16.1-7.el8.noarch 2023-12-01T01:08:33.651 INFO:teuthology.orchestra.run.smithi077.stdout: bison-3.0.4-10.el8.x86_64 2023-12-01T01:08:33.651 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:33.651 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-base-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:33.651 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-common-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:33.651 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-fuse-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:33.651 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-grafana-dashboards-2:18.2.0-1249.ga6f82997.el8.noarch 2023-12-01T01:08:33.651 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-immutable-object-cache-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:33.651 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mds-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:33.651 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:33.651 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-cephadm-2:18.2.0-1249.ga6f82997.el8.noarch 2023-12-01T01:08:33.651 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-dashboard-2:18.2.0-1249.ga6f82997.el8.noarch 2023-12-01T01:08:33.652 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-diskprediction-local-2:18.2.0-1249.ga6f82997.el8.noarch 2023-12-01T01:08:33.652 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-modules-core-2:18.2.0-1249.ga6f82997.el8.noarch 2023-12-01T01:08:33.652 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-rook-2:18.2.0-1249.ga6f82997.el8.noarch 2023-12-01T01:08:33.652 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mon-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:33.652 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-osd-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:33.652 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-prometheus-alerts-2:18.2.0-1249.ga6f82997.el8.noarch 2023-12-01T01:08:33.652 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-radosgw-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:33.652 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-selinux-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:33.652 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-test-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:33.652 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-volume-2:18.2.0-1249.ga6f82997.el8.noarch 2023-12-01T01:08:33.652 INFO:teuthology.orchestra.run.smithi077.stdout: cephadm-2:18.2.0-1249.ga6f82997.el8.noarch 2023-12-01T01:08:33.652 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-mirror-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:33.652 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-top-2:18.2.0-1249.ga6f82997.el8.noarch 2023-12-01T01:08:33.652 INFO:teuthology.orchestra.run.smithi077.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-12-01T01:08:33.653 INFO:teuthology.orchestra.run.smithi077.stdout: flex-2.6.1-9.el8.x86_64 2023-12-01T01:08:33.653 INFO:teuthology.orchestra.run.smithi077.stdout: fuse-2.9.7-15.el8.x86_64 2023-12-01T01:08:33.653 INFO:teuthology.orchestra.run.smithi077.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-12-01T01:08:33.653 INFO:teuthology.orchestra.run.smithi077.stdout: jq-1.6-3.el8.x86_64 2023-12-01T01:08:33.653 INFO:teuthology.orchestra.run.smithi077.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-12-01T01:08:33.653 INFO:teuthology.orchestra.run.smithi077.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-12-01T01:08:33.653 INFO:teuthology.orchestra.run.smithi077.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-12-01T01:08:33.653 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs-devel-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:33.653 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs2-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:33.653 INFO:teuthology.orchestra.run.smithi077.stdout: libcephsqlite-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:33.653 INFO:teuthology.orchestra.run.smithi077.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-12-01T01:08:33.653 INFO:teuthology.orchestra.run.smithi077.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-12-01T01:08:33.653 INFO:teuthology.orchestra.run.smithi077.stdout: liboath-2.6.2-3.el8.x86_64 2023-12-01T01:08:33.653 INFO:teuthology.orchestra.run.smithi077.stdout: libpq-13.5-1.el8.x86_64 2023-12-01T01:08:33.653 INFO:teuthology.orchestra.run.smithi077.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-12-01T01:08:33.654 INFO:teuthology.orchestra.run.smithi077.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-12-01T01:08:33.654 INFO:teuthology.orchestra.run.smithi077.stdout: librados-devel-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:33.654 INFO:teuthology.orchestra.run.smithi077.stdout: libradosstriper1-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:33.654 INFO:teuthology.orchestra.run.smithi077.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-12-01T01:08:33.654 INFO:teuthology.orchestra.run.smithi077.stdout: librgw2-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:33.654 INFO:teuthology.orchestra.run.smithi077.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-12-01T01:08:33.654 INFO:teuthology.orchestra.run.smithi077.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-12-01T01:08:33.654 INFO:teuthology.orchestra.run.smithi077.stdout: libtool-2.4.6-25.el8.x86_64 2023-12-01T01:08:33.654 INFO:teuthology.orchestra.run.smithi077.stdout: libunwind-1.3.1-3.el8.x86_64 2023-12-01T01:08:33.654 INFO:teuthology.orchestra.run.smithi077.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-12-01T01:08:33.654 INFO:teuthology.orchestra.run.smithi077.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-12-01T01:08:33.654 INFO:teuthology.orchestra.run.smithi077.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-12-01T01:08:33.654 INFO:teuthology.orchestra.run.smithi077.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-12-01T01:08:33.654 INFO:teuthology.orchestra.run.smithi077.stdout: openblas-0.3.15-3.el8.x86_64 2023-12-01T01:08:33.654 INFO:teuthology.orchestra.run.smithi077.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-12-01T01:08:33.655 INFO:teuthology.orchestra.run.smithi077.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-12-01T01:08:33.655 INFO:teuthology.orchestra.run.smithi077.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-12-01T01:08:33.655 INFO:teuthology.orchestra.run.smithi077.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-12-01T01:08:33.655 INFO:teuthology.orchestra.run.smithi077.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-12-01T01:08:33.655 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-01T01:08:33.655 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-01T01:08:33.655 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-01T01:08:33.655 INFO:teuthology.orchestra.run.smithi077.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-12-01T01:08:33.655 INFO:teuthology.orchestra.run.smithi077.stdout: python3-babel-2.5.1-7.el8.noarch 2023-12-01T01:08:33.655 INFO:teuthology.orchestra.run.smithi077.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-12-01T01:08:33.655 INFO:teuthology.orchestra.run.smithi077.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-12-01T01:08:33.655 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-12-01T01:08:33.655 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-argparse-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:33.655 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-common-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:33.656 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cephfs-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:33.656 INFO:teuthology.orchestra.run.smithi077.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-12-01T01:08:33.656 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-12-01T01:08:33.656 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-12-01T01:08:33.656 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-12-01T01:08:33.656 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-12-01T01:08:33.656 INFO:teuthology.orchestra.run.smithi077.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-12-01T01:08:33.656 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-6.2-6.el8.noarch 2023-12-01T01:08:33.656 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-12-01T01:08:33.656 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-12-01T01:08:33.656 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-12-01T01:08:33.656 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-12-01T01:08:33.656 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-12-01T01:08:33.656 INFO:teuthology.orchestra.run.smithi077.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-12-01T01:08:33.656 INFO:teuthology.orchestra.run.smithi077.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-12-01T01:08:33.657 INFO:teuthology.orchestra.run.smithi077.stdout: python3-mako-1.0.6-13.el8.noarch 2023-12-01T01:08:33.657 INFO:teuthology.orchestra.run.smithi077.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-12-01T01:08:33.657 INFO:teuthology.orchestra.run.smithi077.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-12-01T01:08:33.657 INFO:teuthology.orchestra.run.smithi077.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-12-01T01:08:33.657 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-12-01T01:08:33.657 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-12-01T01:08:33.657 INFO:teuthology.orchestra.run.smithi077.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-12-01T01:08:33.657 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-12-01T01:08:33.657 INFO:teuthology.orchestra.run.smithi077.stdout: python3-portend-2.6-1.el8.noarch 2023-12-01T01:08:33.657 INFO:teuthology.orchestra.run.smithi077.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-12-01T01:08:33.657 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-12-01T01:08:33.657 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-12-01T01:08:33.657 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-12-01T01:08:33.657 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pycparser-2.14-14.el8.noarch 2023-12-01T01:08:33.657 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pytz-2017.2-9.el8.noarch 2023-12-01T01:08:33.658 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rados-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:33.658 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rbd-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:33.658 INFO:teuthology.orchestra.run.smithi077.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-12-01T01:08:33.658 INFO:teuthology.orchestra.run.smithi077.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-12-01T01:08:33.658 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rgw-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:33.658 INFO:teuthology.orchestra.run.smithi077.stdout: python3-routes-2.4.1-12.el8.noarch 2023-12-01T01:08:33.658 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rsa-4.9-2.el8.noarch 2023-12-01T01:08:33.658 INFO:teuthology.orchestra.run.smithi077.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-12-01T01:08:33.658 INFO:teuthology.orchestra.run.smithi077.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-12-01T01:08:33.659 INFO:teuthology.orchestra.run.smithi077.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-12-01T01:08:33.659 INFO:teuthology.orchestra.run.smithi077.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-12-01T01:08:33.659 INFO:teuthology.orchestra.run.smithi077.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-12-01T01:08:33.659 INFO:teuthology.orchestra.run.smithi077.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-12-01T01:08:33.659 INFO:teuthology.orchestra.run.smithi077.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-12-01T01:08:33.659 INFO:teuthology.orchestra.run.smithi077.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-12-01T01:08:33.659 INFO:teuthology.orchestra.run.smithi077.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-12-01T01:08:33.659 INFO:teuthology.orchestra.run.smithi077.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-12-01T01:08:33.659 INFO:teuthology.orchestra.run.smithi077.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-12-01T01:08:33.659 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-fuse-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:33.659 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-mirror-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:33.659 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-nbd-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:33.659 INFO:teuthology.orchestra.run.smithi077.stdout: socat-1.7.4.1-1.el8.x86_64 2023-12-01T01:08:33.660 INFO:teuthology.orchestra.run.smithi077.stdout: thrift-0.13.0-2.el8.x86_64 2023-12-01T01:08:33.660 INFO:teuthology.orchestra.run.smithi077.stdout: uuid-1.6.2-43.el8.x86_64 2023-12-01T01:08:33.660 INFO:teuthology.orchestra.run.smithi077.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-12-01T01:08:33.660 INFO:teuthology.orchestra.run.smithi077.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-12-01T01:08:33.660 INFO:teuthology.orchestra.run.smithi077.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-12-01T01:08:33.660 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:08:33.660 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2023-12-01T01:08:33.842 DEBUG:teuthology.parallel:result is None 2023-12-01T01:08:34.000 INFO:teuthology.orchestra.run.smithi137.stdout:Installed products updated. 2023-12-01T01:08:34.108 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-12-01T01:08:34.108 INFO:teuthology.orchestra.run.smithi203.stdout:Last metadata expiration check: 0:01:56 ago on Fri 01 Dec 2023 01:06:38 AM UTC. 2023-12-01T01:08:34.213 INFO:teuthology.orchestra.run.smithi137.stdout: 2023-12-01T01:08:34.214 INFO:teuthology.orchestra.run.smithi137.stdout:Upgraded: 2023-12-01T01:08:34.214 INFO:teuthology.orchestra.run.smithi137.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-12-01T01:08:34.214 INFO:teuthology.orchestra.run.smithi137.stdout: librados2-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:34.214 INFO:teuthology.orchestra.run.smithi137.stdout: librbd1-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:34.214 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2023-12-01T01:08:34.214 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-12-01T01:08:34.214 INFO:teuthology.orchestra.run.smithi137.stdout: automake-1.16.1-7.el8.noarch 2023-12-01T01:08:34.214 INFO:teuthology.orchestra.run.smithi137.stdout: bison-3.0.4-10.el8.x86_64 2023-12-01T01:08:34.214 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:34.214 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-base-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:34.214 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-common-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:34.214 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-fuse-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:34.215 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-grafana-dashboards-2:18.2.0-1249.ga6f82997.el8.noarch 2023-12-01T01:08:34.215 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-immutable-object-cache-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:34.215 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mds-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:34.215 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:34.215 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-cephadm-2:18.2.0-1249.ga6f82997.el8.noarch 2023-12-01T01:08:34.215 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-dashboard-2:18.2.0-1249.ga6f82997.el8.noarch 2023-12-01T01:08:34.215 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-diskprediction-local-2:18.2.0-1249.ga6f82997.el8.noarch 2023-12-01T01:08:34.215 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-modules-core-2:18.2.0-1249.ga6f82997.el8.noarch 2023-12-01T01:08:34.215 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-rook-2:18.2.0-1249.ga6f82997.el8.noarch 2023-12-01T01:08:34.215 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mon-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:34.215 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-osd-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:34.215 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-prometheus-alerts-2:18.2.0-1249.ga6f82997.el8.noarch 2023-12-01T01:08:34.215 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-radosgw-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:34.215 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-selinux-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:34.215 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-test-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:34.216 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-volume-2:18.2.0-1249.ga6f82997.el8.noarch 2023-12-01T01:08:34.216 INFO:teuthology.orchestra.run.smithi137.stdout: cephadm-2:18.2.0-1249.ga6f82997.el8.noarch 2023-12-01T01:08:34.216 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-mirror-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:34.216 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-top-2:18.2.0-1249.ga6f82997.el8.noarch 2023-12-01T01:08:34.216 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-12-01T01:08:34.216 INFO:teuthology.orchestra.run.smithi137.stdout: flex-2.6.1-9.el8.x86_64 2023-12-01T01:08:34.216 INFO:teuthology.orchestra.run.smithi137.stdout: fuse-2.9.7-15.el8.x86_64 2023-12-01T01:08:34.216 INFO:teuthology.orchestra.run.smithi137.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-12-01T01:08:34.216 INFO:teuthology.orchestra.run.smithi137.stdout: jq-1.6-3.el8.x86_64 2023-12-01T01:08:34.216 INFO:teuthology.orchestra.run.smithi137.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-12-01T01:08:34.216 INFO:teuthology.orchestra.run.smithi137.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-12-01T01:08:34.216 INFO:teuthology.orchestra.run.smithi137.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-12-01T01:08:34.216 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs-devel-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:34.216 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs2-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:34.217 INFO:teuthology.orchestra.run.smithi137.stdout: libcephsqlite-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:34.217 INFO:teuthology.orchestra.run.smithi137.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-12-01T01:08:34.217 INFO:teuthology.orchestra.run.smithi137.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-12-01T01:08:34.217 INFO:teuthology.orchestra.run.smithi137.stdout: liboath-2.6.2-3.el8.x86_64 2023-12-01T01:08:34.217 INFO:teuthology.orchestra.run.smithi137.stdout: libpq-13.5-1.el8.x86_64 2023-12-01T01:08:34.217 INFO:teuthology.orchestra.run.smithi137.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-12-01T01:08:34.217 INFO:teuthology.orchestra.run.smithi137.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-12-01T01:08:34.217 INFO:teuthology.orchestra.run.smithi137.stdout: librados-devel-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:34.217 INFO:teuthology.orchestra.run.smithi137.stdout: libradosstriper1-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:34.217 INFO:teuthology.orchestra.run.smithi137.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-12-01T01:08:34.217 INFO:teuthology.orchestra.run.smithi137.stdout: librgw2-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:34.217 INFO:teuthology.orchestra.run.smithi137.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-12-01T01:08:34.217 INFO:teuthology.orchestra.run.smithi137.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-12-01T01:08:34.217 INFO:teuthology.orchestra.run.smithi137.stdout: libtool-2.4.6-25.el8.x86_64 2023-12-01T01:08:34.218 INFO:teuthology.orchestra.run.smithi137.stdout: libunwind-1.3.1-3.el8.x86_64 2023-12-01T01:08:34.218 INFO:teuthology.orchestra.run.smithi137.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-12-01T01:08:34.218 INFO:teuthology.orchestra.run.smithi137.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-12-01T01:08:34.218 INFO:teuthology.orchestra.run.smithi137.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-12-01T01:08:34.218 INFO:teuthology.orchestra.run.smithi137.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-12-01T01:08:34.218 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-0.3.15-3.el8.x86_64 2023-12-01T01:08:34.218 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-12-01T01:08:34.218 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-12-01T01:08:34.218 INFO:teuthology.orchestra.run.smithi137.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-12-01T01:08:34.218 INFO:teuthology.orchestra.run.smithi137.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-12-01T01:08:34.218 INFO:teuthology.orchestra.run.smithi137.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-12-01T01:08:34.218 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-01T01:08:34.218 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-01T01:08:34.218 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-01T01:08:34.219 INFO:teuthology.orchestra.run.smithi137.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-12-01T01:08:34.219 INFO:teuthology.orchestra.run.smithi137.stdout: python3-babel-2.5.1-7.el8.noarch 2023-12-01T01:08:34.219 INFO:teuthology.orchestra.run.smithi137.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-12-01T01:08:34.219 INFO:teuthology.orchestra.run.smithi137.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-12-01T01:08:34.219 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-12-01T01:08:34.219 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-argparse-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:34.219 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-common-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:34.219 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cephfs-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:34.219 INFO:teuthology.orchestra.run.smithi137.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-12-01T01:08:34.219 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-12-01T01:08:34.219 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-12-01T01:08:34.219 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-12-01T01:08:34.219 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-12-01T01:08:34.219 INFO:teuthology.orchestra.run.smithi137.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-12-01T01:08:34.220 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-6.2-6.el8.noarch 2023-12-01T01:08:34.220 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-12-01T01:08:34.220 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-12-01T01:08:34.220 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-12-01T01:08:34.220 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-12-01T01:08:34.220 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-12-01T01:08:34.220 INFO:teuthology.orchestra.run.smithi137.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-12-01T01:08:34.220 INFO:teuthology.orchestra.run.smithi137.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-12-01T01:08:34.220 INFO:teuthology.orchestra.run.smithi137.stdout: python3-mako-1.0.6-13.el8.noarch 2023-12-01T01:08:34.220 INFO:teuthology.orchestra.run.smithi137.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-12-01T01:08:34.220 INFO:teuthology.orchestra.run.smithi137.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-12-01T01:08:34.220 INFO:teuthology.orchestra.run.smithi137.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-12-01T01:08:34.220 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-12-01T01:08:34.220 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-12-01T01:08:34.220 INFO:teuthology.orchestra.run.smithi137.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-12-01T01:08:34.221 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-12-01T01:08:34.221 INFO:teuthology.orchestra.run.smithi137.stdout: python3-portend-2.6-1.el8.noarch 2023-12-01T01:08:34.221 INFO:teuthology.orchestra.run.smithi137.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-12-01T01:08:34.221 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-12-01T01:08:34.221 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-12-01T01:08:34.221 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-12-01T01:08:34.221 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pycparser-2.14-14.el8.noarch 2023-12-01T01:08:34.221 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pytz-2017.2-9.el8.noarch 2023-12-01T01:08:34.221 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rados-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:34.221 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rbd-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:34.221 INFO:teuthology.orchestra.run.smithi137.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-12-01T01:08:34.221 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-12-01T01:08:34.221 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rgw-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:34.221 INFO:teuthology.orchestra.run.smithi137.stdout: python3-routes-2.4.1-12.el8.noarch 2023-12-01T01:08:34.222 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rsa-4.9-2.el8.noarch 2023-12-01T01:08:34.222 INFO:teuthology.orchestra.run.smithi137.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-12-01T01:08:34.222 INFO:teuthology.orchestra.run.smithi137.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-12-01T01:08:34.222 INFO:teuthology.orchestra.run.smithi137.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-12-01T01:08:34.222 INFO:teuthology.orchestra.run.smithi137.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-12-01T01:08:34.222 INFO:teuthology.orchestra.run.smithi137.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-12-01T01:08:34.222 INFO:teuthology.orchestra.run.smithi137.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-12-01T01:08:34.222 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-12-01T01:08:34.222 INFO:teuthology.orchestra.run.smithi137.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-12-01T01:08:34.222 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-12-01T01:08:34.222 INFO:teuthology.orchestra.run.smithi137.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-12-01T01:08:34.222 INFO:teuthology.orchestra.run.smithi137.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-12-01T01:08:34.222 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-fuse-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:34.222 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-mirror-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:34.223 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-nbd-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:34.223 INFO:teuthology.orchestra.run.smithi137.stdout: socat-1.7.4.1-1.el8.x86_64 2023-12-01T01:08:34.223 INFO:teuthology.orchestra.run.smithi137.stdout: thrift-0.13.0-2.el8.x86_64 2023-12-01T01:08:34.223 INFO:teuthology.orchestra.run.smithi137.stdout: uuid-1.6.2-43.el8.x86_64 2023-12-01T01:08:34.223 INFO:teuthology.orchestra.run.smithi137.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-12-01T01:08:34.223 INFO:teuthology.orchestra.run.smithi137.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-12-01T01:08:34.223 INFO:teuthology.orchestra.run.smithi137.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-12-01T01:08:34.223 INFO:teuthology.orchestra.run.smithi137.stdout: 2023-12-01T01:08:34.223 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2023-12-01T01:08:34.443 DEBUG:teuthology.parallel:result is None 2023-12-01T01:08:38.052 INFO:teuthology.orchestra.run.smithi203.stdout:Installed products updated. 2023-12-01T01:08:38.266 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-12-01T01:08:38.266 INFO:teuthology.orchestra.run.smithi203.stdout:Upgraded: 2023-12-01T01:08:38.266 INFO:teuthology.orchestra.run.smithi203.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-12-01T01:08:38.266 INFO:teuthology.orchestra.run.smithi203.stdout: librados2-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:38.266 INFO:teuthology.orchestra.run.smithi203.stdout: librbd1-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:38.266 INFO:teuthology.orchestra.run.smithi203.stdout:Installed: 2023-12-01T01:08:38.266 INFO:teuthology.orchestra.run.smithi203.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-12-01T01:08:38.266 INFO:teuthology.orchestra.run.smithi203.stdout: automake-1.16.1-7.el8.noarch 2023-12-01T01:08:38.266 INFO:teuthology.orchestra.run.smithi203.stdout: bison-3.0.4-10.el8.x86_64 2023-12-01T01:08:38.267 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:38.267 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-base-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:38.267 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-common-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:38.267 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-fuse-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:38.267 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-grafana-dashboards-2:18.2.0-1249.ga6f82997.el8.noarch 2023-12-01T01:08:38.267 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-immutable-object-cache-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:38.267 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mds-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:38.267 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:38.267 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-cephadm-2:18.2.0-1249.ga6f82997.el8.noarch 2023-12-01T01:08:38.267 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-dashboard-2:18.2.0-1249.ga6f82997.el8.noarch 2023-12-01T01:08:38.267 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-diskprediction-local-2:18.2.0-1249.ga6f82997.el8.noarch 2023-12-01T01:08:38.267 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-modules-core-2:18.2.0-1249.ga6f82997.el8.noarch 2023-12-01T01:08:38.267 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-rook-2:18.2.0-1249.ga6f82997.el8.noarch 2023-12-01T01:08:38.267 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mon-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:38.268 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-osd-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:38.268 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-prometheus-alerts-2:18.2.0-1249.ga6f82997.el8.noarch 2023-12-01T01:08:38.268 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-radosgw-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:38.268 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-selinux-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:38.268 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-test-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:38.268 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-volume-2:18.2.0-1249.ga6f82997.el8.noarch 2023-12-01T01:08:38.268 INFO:teuthology.orchestra.run.smithi203.stdout: cephadm-2:18.2.0-1249.ga6f82997.el8.noarch 2023-12-01T01:08:38.268 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-mirror-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:38.268 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-top-2:18.2.0-1249.ga6f82997.el8.noarch 2023-12-01T01:08:38.268 INFO:teuthology.orchestra.run.smithi203.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-12-01T01:08:38.268 INFO:teuthology.orchestra.run.smithi203.stdout: flex-2.6.1-9.el8.x86_64 2023-12-01T01:08:38.268 INFO:teuthology.orchestra.run.smithi203.stdout: fuse-2.9.7-15.el8.x86_64 2023-12-01T01:08:38.268 INFO:teuthology.orchestra.run.smithi203.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-12-01T01:08:38.268 INFO:teuthology.orchestra.run.smithi203.stdout: jq-1.6-3.el8.x86_64 2023-12-01T01:08:38.268 INFO:teuthology.orchestra.run.smithi203.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-12-01T01:08:38.269 INFO:teuthology.orchestra.run.smithi203.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-12-01T01:08:38.269 INFO:teuthology.orchestra.run.smithi203.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-12-01T01:08:38.269 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs-devel-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:38.269 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs2-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:38.269 INFO:teuthology.orchestra.run.smithi203.stdout: libcephsqlite-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:38.269 INFO:teuthology.orchestra.run.smithi203.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-12-01T01:08:38.269 INFO:teuthology.orchestra.run.smithi203.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-12-01T01:08:38.269 INFO:teuthology.orchestra.run.smithi203.stdout: liboath-2.6.2-3.el8.x86_64 2023-12-01T01:08:38.269 INFO:teuthology.orchestra.run.smithi203.stdout: libpq-13.5-1.el8.x86_64 2023-12-01T01:08:38.269 INFO:teuthology.orchestra.run.smithi203.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-12-01T01:08:38.269 INFO:teuthology.orchestra.run.smithi203.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-12-01T01:08:38.269 INFO:teuthology.orchestra.run.smithi203.stdout: librados-devel-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:38.270 INFO:teuthology.orchestra.run.smithi203.stdout: libradosstriper1-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:38.270 INFO:teuthology.orchestra.run.smithi203.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-12-01T01:08:38.271 INFO:teuthology.orchestra.run.smithi203.stdout: librgw2-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:38.271 INFO:teuthology.orchestra.run.smithi203.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-12-01T01:08:38.271 INFO:teuthology.orchestra.run.smithi203.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-12-01T01:08:38.271 INFO:teuthology.orchestra.run.smithi203.stdout: libtool-2.4.6-25.el8.x86_64 2023-12-01T01:08:38.271 INFO:teuthology.orchestra.run.smithi203.stdout: libunwind-1.3.1-3.el8.x86_64 2023-12-01T01:08:38.271 INFO:teuthology.orchestra.run.smithi203.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-12-01T01:08:38.271 INFO:teuthology.orchestra.run.smithi203.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-12-01T01:08:38.271 INFO:teuthology.orchestra.run.smithi203.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-12-01T01:08:38.271 INFO:teuthology.orchestra.run.smithi203.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-12-01T01:08:38.271 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-0.3.15-3.el8.x86_64 2023-12-01T01:08:38.271 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-12-01T01:08:38.271 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-12-01T01:08:38.271 INFO:teuthology.orchestra.run.smithi203.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-12-01T01:08:38.271 INFO:teuthology.orchestra.run.smithi203.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-12-01T01:08:38.272 INFO:teuthology.orchestra.run.smithi203.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-12-01T01:08:38.272 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-01T01:08:38.272 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-01T01:08:38.272 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-01T01:08:38.272 INFO:teuthology.orchestra.run.smithi203.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-12-01T01:08:38.272 INFO:teuthology.orchestra.run.smithi203.stdout: python3-babel-2.5.1-7.el8.noarch 2023-12-01T01:08:38.272 INFO:teuthology.orchestra.run.smithi203.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-12-01T01:08:38.272 INFO:teuthology.orchestra.run.smithi203.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-12-01T01:08:38.272 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-12-01T01:08:38.272 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-argparse-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:38.272 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-common-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:38.272 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cephfs-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:38.273 INFO:teuthology.orchestra.run.smithi203.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-12-01T01:08:38.273 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-12-01T01:08:38.273 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-12-01T01:08:38.273 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-12-01T01:08:38.273 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-12-01T01:08:38.273 INFO:teuthology.orchestra.run.smithi203.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-12-01T01:08:38.273 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-6.2-6.el8.noarch 2023-12-01T01:08:38.273 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-12-01T01:08:38.273 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-12-01T01:08:38.273 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-12-01T01:08:38.273 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-12-01T01:08:38.273 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-12-01T01:08:38.273 INFO:teuthology.orchestra.run.smithi203.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-12-01T01:08:38.274 INFO:teuthology.orchestra.run.smithi203.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-12-01T01:08:38.274 INFO:teuthology.orchestra.run.smithi203.stdout: python3-mako-1.0.6-13.el8.noarch 2023-12-01T01:08:38.274 INFO:teuthology.orchestra.run.smithi203.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-12-01T01:08:38.274 INFO:teuthology.orchestra.run.smithi203.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-12-01T01:08:38.274 INFO:teuthology.orchestra.run.smithi203.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-12-01T01:08:38.274 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-12-01T01:08:38.274 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-12-01T01:08:38.274 INFO:teuthology.orchestra.run.smithi203.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-12-01T01:08:38.274 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-12-01T01:08:38.274 INFO:teuthology.orchestra.run.smithi203.stdout: python3-portend-2.6-1.el8.noarch 2023-12-01T01:08:38.274 INFO:teuthology.orchestra.run.smithi203.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-12-01T01:08:38.274 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-12-01T01:08:38.274 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-12-01T01:08:38.275 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-12-01T01:08:38.275 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pycparser-2.14-14.el8.noarch 2023-12-01T01:08:38.275 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pytz-2017.2-9.el8.noarch 2023-12-01T01:08:38.275 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rados-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:38.275 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rbd-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:38.275 INFO:teuthology.orchestra.run.smithi203.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-12-01T01:08:38.275 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-12-01T01:08:38.275 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rgw-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:38.275 INFO:teuthology.orchestra.run.smithi203.stdout: python3-routes-2.4.1-12.el8.noarch 2023-12-01T01:08:38.275 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rsa-4.9-2.el8.noarch 2023-12-01T01:08:38.275 INFO:teuthology.orchestra.run.smithi203.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-12-01T01:08:38.275 INFO:teuthology.orchestra.run.smithi203.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-12-01T01:08:38.275 INFO:teuthology.orchestra.run.smithi203.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-12-01T01:08:38.275 INFO:teuthology.orchestra.run.smithi203.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-12-01T01:08:38.275 INFO:teuthology.orchestra.run.smithi203.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-12-01T01:08:38.276 INFO:teuthology.orchestra.run.smithi203.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-12-01T01:08:38.276 INFO:teuthology.orchestra.run.smithi203.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-12-01T01:08:38.276 INFO:teuthology.orchestra.run.smithi203.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-12-01T01:08:38.276 INFO:teuthology.orchestra.run.smithi203.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-12-01T01:08:38.276 INFO:teuthology.orchestra.run.smithi203.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-12-01T01:08:38.276 INFO:teuthology.orchestra.run.smithi203.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-12-01T01:08:38.276 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-fuse-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:38.276 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-mirror-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:38.276 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-nbd-2:18.2.0-1249.ga6f82997.el8.x86_64 2023-12-01T01:08:38.276 INFO:teuthology.orchestra.run.smithi203.stdout: socat-1.7.4.1-1.el8.x86_64 2023-12-01T01:08:38.276 INFO:teuthology.orchestra.run.smithi203.stdout: thrift-0.13.0-2.el8.x86_64 2023-12-01T01:08:38.276 INFO:teuthology.orchestra.run.smithi203.stdout: uuid-1.6.2-43.el8.x86_64 2023-12-01T01:08:38.276 INFO:teuthology.orchestra.run.smithi203.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-12-01T01:08:38.276 INFO:teuthology.orchestra.run.smithi203.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-12-01T01:08:38.277 INFO:teuthology.orchestra.run.smithi203.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-12-01T01:08:38.277 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-12-01T01:08:38.277 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2023-12-01T01:08:38.637 DEBUG:teuthology.parallel:result is None 2023-12-01T01:08:38.637 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a6f82997ac57b70a895455dfed6256360a1e4c32 2023-12-01T01:08:38.855 DEBUG:teuthology.orchestra.run.smithi077:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-12-01T01:08:38.884 INFO:teuthology.orchestra.run.smithi077.stdout:18.2.0-1249.ga6f82997.el8 2023-12-01T01:08:38.884 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-1249.ga6f82997.el8 2023-12-01T01:08:38.885 INFO:teuthology.task.install:The correct ceph version 18.2.0-1249.ga6f82997 is installed. 2023-12-01T01:08:38.885 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a6f82997ac57b70a895455dfed6256360a1e4c32 2023-12-01T01:08:39.125 DEBUG:teuthology.orchestra.run.smithi137:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-12-01T01:08:39.154 INFO:teuthology.orchestra.run.smithi137.stdout:18.2.0-1249.ga6f82997.el8 2023-12-01T01:08:39.154 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-1249.ga6f82997.el8 2023-12-01T01:08:39.154 INFO:teuthology.task.install:The correct ceph version 18.2.0-1249.ga6f82997 is installed. 2023-12-01T01:08:39.154 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a6f82997ac57b70a895455dfed6256360a1e4c32 2023-12-01T01:08:39.373 DEBUG:teuthology.orchestra.run.smithi203:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-12-01T01:08:39.402 INFO:teuthology.orchestra.run.smithi203.stdout:18.2.0-1249.ga6f82997.el8 2023-12-01T01:08:39.403 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-1249.ga6f82997.el8 2023-12-01T01:08:39.403 INFO:teuthology.task.install:The correct ceph version 18.2.0-1249.ga6f82997 is installed. 2023-12-01T01:08:39.403 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-12-01T01:08:39.403 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-12-01T01:08:39.403 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-01T01:08:39.451 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2023-12-01T01:08:39.451 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-01T01:08:39.499 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-12-01T01:08:39.499 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-01T01:08:39.546 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-12-01T01:08:39.547 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-12-01T01:08:39.547 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/usr/bin/daemon-helper 2023-12-01T01:08:39.577 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-01T01:08:39.647 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2023-12-01T01:08:39.647 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/daemon-helper 2023-12-01T01:08:39.675 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-01T01:08:39.744 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-12-01T01:08:39.745 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/daemon-helper 2023-12-01T01:08:39.777 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-01T01:08:39.848 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-12-01T01:08:39.849 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-12-01T01:08:39.849 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-01T01:08:39.880 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-01T01:08:39.949 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2023-12-01T01:08:39.949 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-01T01:08:39.979 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-01T01:08:40.048 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-12-01T01:08:40.048 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-01T01:08:40.079 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-01T01:08:40.150 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-12-01T01:08:40.150 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-12-01T01:08:40.150 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/usr/bin/stdin-killer 2023-12-01T01:08:40.181 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-01T01:08:40.252 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2023-12-01T01:08:40.252 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/stdin-killer 2023-12-01T01:08:40.282 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-01T01:08:40.351 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-12-01T01:08:40.351 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/stdin-killer 2023-12-01T01:08:40.382 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-01T01:08:40.453 INFO:teuthology.run_tasks:Running task cephadm... 2023-12-01T01:08:40.540 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'a6f82997ac57b70a895455dfed6256360a1e4c32', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2023-12-01T01:08:40.540 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 2023-12-01T01:08:40.540 INFO:tasks.cephadm:Cluster fsid is 2a06e29a-8fe6-11ee-95a2-87774f69a715 2023-12-01T01:08:40.541 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-12-01T01:08:40.541 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.77', 'mon.b': '172.21.15.137', 'mon.c': '172.21.15.203'} 2023-12-01T01:08:40.541 INFO:tasks.cephadm:First mon is mon.a on smithi077 2023-12-01T01:08:40.541 INFO:tasks.cephadm:First mgr is x 2023-12-01T01:08:40.541 INFO:tasks.cephadm:Normalizing hostnames... 2023-12-01T01:08:40.541 DEBUG:teuthology.orchestra.run.smithi077:> sudo hostname $(hostname -s) 2023-12-01T01:08:40.574 DEBUG:teuthology.orchestra.run.smithi137:> sudo hostname $(hostname -s) 2023-12-01T01:08:40.605 DEBUG:teuthology.orchestra.run.smithi203:> sudo hostname $(hostname -s) 2023-12-01T01:08:40.636 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-12-01T01:08:40.636 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a6f82997ac57b70a895455dfed6256360a1e4c32 2023-12-01T01:08:40.779 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'a6f82997ac57b70a895455dfed6256360a1e4c32', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/74585/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.0-1249-ga6f82997', 'node_name': '172.21.4.65+confusa03', 'job_name': 'ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.0-1249.ga6f82997'}, 'url': 'https://3.chacra.ceph.com/r/ceph/reef-release/a6f82997ac57b70a895455dfed6256360a1e4c32/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-11-30 23:20:29.682137', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'reef-release', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/reef-release/a6f82997ac57b70a895455dfed6256360a1e4c32/centos/8/flavors/default/', 'archs': ['x86_64', 'source', 'arm64'], 'distro': 'centos'}] 2023-12-01T01:08:40.912 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref reef-release, sha1 a6f82997ac57b70a895455dfed6256360a1e4c32 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=a6f82997ac57b70a895455dfed6256360a1e4c32 2023-12-01T01:08:40.912 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/reef-release/a6f82997ac57b70a895455dfed6256360a1e4c32/centos/8/x86_64/flavors/default/cephadm 2023-12-01T01:08:40.913 DEBUG:teuthology.orchestra.run.smithi077:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/reef-release/a6f82997ac57b70a895455dfed6256360a1e4c32/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-12-01T01:08:41.285 INFO:teuthology.orchestra.run.smithi077.stdout:-rw-rw-r--. 1 ubuntu ubuntu 690689 Dec 1 01:08 /home/ubuntu/cephtest/cephadm 2023-12-01T01:08:41.285 DEBUG:teuthology.orchestra.run.smithi137:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/reef-release/a6f82997ac57b70a895455dfed6256360a1e4c32/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-12-01T01:08:41.667 INFO:teuthology.orchestra.run.smithi137.stdout:-rw-rw-r--. 1 ubuntu ubuntu 690689 Dec 1 01:08 /home/ubuntu/cephtest/cephadm 2023-12-01T01:08:41.667 DEBUG:teuthology.orchestra.run.smithi203:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/reef-release/a6f82997ac57b70a895455dfed6256360a1e4c32/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-12-01T01:08:42.367 INFO:teuthology.orchestra.run.smithi203.stdout:-rw-rw-r--. 1 ubuntu ubuntu 690689 Dec 1 01:08 /home/ubuntu/cephtest/cephadm 2023-12-01T01:08:42.367 DEBUG:teuthology.orchestra.run.smithi077:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-12-01T01:08:42.389 DEBUG:teuthology.orchestra.run.smithi137:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-12-01T01:08:42.410 DEBUG:teuthology.orchestra.run.smithi203:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-12-01T01:08:42.446 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 on all hosts... 2023-12-01T01:08:42.446 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 pull 2023-12-01T01:08:42.448 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 pull 2023-12-01T01:08:42.452 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 pull 2023-12-01T01:08:47.184 INFO:teuthology.orchestra.run.smithi137.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32... 2023-12-01T01:08:47.291 INFO:teuthology.orchestra.run.smithi077.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32... 2023-12-01T01:08:48.469 INFO:teuthology.orchestra.run.smithi203.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32... 2023-12-01T01:09:20.289 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2023-12-01T01:09:20.289 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_version": "ceph version 18.2.0-1249-ga6f82997 (a6f82997ac57b70a895455dfed6256360a1e4c32) reef (stable)", 2023-12-01T01:09:20.290 INFO:teuthology.orchestra.run.smithi137.stdout: "image_id": "96d837748436087e6a178851a9eca1321e50ebbd34bb41f0a6c21f217e801f91", 2023-12-01T01:09:20.290 INFO:teuthology.orchestra.run.smithi137.stdout: "repo_digests": [ 2023-12-01T01:09:20.290 INFO:teuthology.orchestra.run.smithi137.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce186d5741e90182960931ace897a7b2a2061bc709a9daf86e9971f6fbef16" 2023-12-01T01:09:20.290 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2023-12-01T01:09:20.290 INFO:teuthology.orchestra.run.smithi137.stdout:} 2023-12-01T01:09:22.790 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2023-12-01T01:09:22.790 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_version": "ceph version 18.2.0-1249-ga6f82997 (a6f82997ac57b70a895455dfed6256360a1e4c32) reef (stable)", 2023-12-01T01:09:22.790 INFO:teuthology.orchestra.run.smithi203.stdout: "image_id": "96d837748436087e6a178851a9eca1321e50ebbd34bb41f0a6c21f217e801f91", 2023-12-01T01:09:22.790 INFO:teuthology.orchestra.run.smithi203.stdout: "repo_digests": [ 2023-12-01T01:09:22.790 INFO:teuthology.orchestra.run.smithi203.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce186d5741e90182960931ace897a7b2a2061bc709a9daf86e9971f6fbef16" 2023-12-01T01:09:22.791 INFO:teuthology.orchestra.run.smithi203.stdout: ] 2023-12-01T01:09:22.791 INFO:teuthology.orchestra.run.smithi203.stdout:} 2023-12-01T01:09:22.808 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2023-12-01T01:09:22.809 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_version": "ceph version 18.2.0-1249-ga6f82997 (a6f82997ac57b70a895455dfed6256360a1e4c32) reef (stable)", 2023-12-01T01:09:22.809 INFO:teuthology.orchestra.run.smithi077.stdout: "image_id": "96d837748436087e6a178851a9eca1321e50ebbd34bb41f0a6c21f217e801f91", 2023-12-01T01:09:22.809 INFO:teuthology.orchestra.run.smithi077.stdout: "repo_digests": [ 2023-12-01T01:09:22.809 INFO:teuthology.orchestra.run.smithi077.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7cce186d5741e90182960931ace897a7b2a2061bc709a9daf86e9971f6fbef16" 2023-12-01T01:09:22.809 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2023-12-01T01:09:22.809 INFO:teuthology.orchestra.run.smithi077.stdout:} 2023-12-01T01:09:22.828 DEBUG:teuthology.orchestra.run.smithi077:> sudo mkdir -p /etc/ceph 2023-12-01T01:09:22.858 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /etc/ceph 2023-12-01T01:09:22.894 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /etc/ceph 2023-12-01T01:09:22.924 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 777 /etc/ceph 2023-12-01T01:09:22.951 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 777 /etc/ceph 2023-12-01T01:09:22.982 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 777 /etc/ceph 2023-12-01T01:09:23.008 INFO:tasks.cephadm:Writing seed config... 2023-12-01T01:09:23.009 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-12-01T01:09:23.009 INFO:tasks.cephadm: override: [client] debug client = 20 2023-12-01T01:09:23.009 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-12-01T01:09:23.009 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-12-01T01:09:23.009 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-12-01T01:09:23.009 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-12-01T01:09:23.009 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-12-01T01:09:23.009 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-12-01T01:09:23.010 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-12-01T01:09:23.010 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-12-01T01:09:23.010 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-12-01T01:09:23.010 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-12-01T01:09:23.010 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-12-01T01:09:23.010 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-12-01T01:09:23.010 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-12-01T01:09:23.010 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-12-01T01:09:23.010 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-12-01T01:09:23.010 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2023-12-01T01:09:23.010 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-12-01T01:09:23.010 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-12-01T01:09:23.010 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-12-01T01:09:23.011 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-12-01T01:09:23.011 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-12-01T01:09:23.011 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-12-01T01:09:23.011 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-12-01T01:09:23.011 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-12-01T01:09:23.011 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-12-01T01:09:23.011 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-12-01T01:09:23.011 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2023-12-01T01:09:23.011 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2023-12-01T01:09:23.011 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2023-12-01T01:09:23.011 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-12-01T01:09:23.011 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-12-01T01:09:23.011 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2023-12-01T01:09:23.012 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2023-12-01T01:09:23.012 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-12-01T01:09:23.012 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-12-01T01:09:23.012 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2023-12-01T01:09:23.012 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-12-01T01:09:23.012 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-12-01T01:09:23.012 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-12-01T01:09:23.012 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-12-01T01:09:23.012 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-12-01T01:09:23.012 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-12-01T01:09:23.012 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-12-01T01:09:23.012 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2023-12-01T01:09:23.013 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-12-01T01:09:23.013 DEBUG:teuthology.orchestra.run.smithi077:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-12-01T01:09:23.030 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 2a06e29a-8fe6-11ee-95a2-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-12-01T01:09:23.030 DEBUG:teuthology.orchestra.run.smithi077:mon.a> sudo journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mon.a.service 2023-12-01T01:09:23.073 DEBUG:teuthology.orchestra.run.smithi077:mgr.x> sudo journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mgr.x.service 2023-12-01T01:09:23.098 INFO:journalctl@ceph.mon.a.smithi077.stdout:-- Logs begin at Fri 2023-12-01 00:55:13 UTC. -- 2023-12-01T01:09:23.099 INFO:tasks.cephadm:Bootstrapping... 2023-12-01T01:09:23.099 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 -v bootstrap --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.77 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-12-01T01:09:23.124 INFO:journalctl@ceph.mgr.x.smithi077.stdout:-- Logs begin at Fri 2023-12-01 00:55:13 UTC. -- 2023-12-01T01:09:23.257 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2023-12-01T01:09:23.257 INFO:teuthology.orchestra.run.smithi077.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32', '-v', 'bootstrap', '--fsid', '2a06e29a-8fe6-11ee-95a2-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.77', '--skip-admin-label'] 2023-12-01T01:09:23.402 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stdout 4.1.1 2023-12-01T01:09:23.403 INFO:teuthology.orchestra.run.smithi077.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-12-01T01:09:23.403 INFO:teuthology.orchestra.run.smithi077.stdout:Verifying podman|docker is present... 2023-12-01T01:09:23.549 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stdout 4.1.1 2023-12-01T01:09:23.549 INFO:teuthology.orchestra.run.smithi077.stdout:Verifying lvm2 is present... 2023-12-01T01:09:23.549 INFO:teuthology.orchestra.run.smithi077.stdout:Verifying time synchronization is in place... 2023-12-01T01:09:23.557 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-12-01T01:09:23.557 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-12-01T01:09:23.562 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-12-01T01:09:23.562 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout inactive 2023-12-01T01:09:23.568 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout enabled 2023-12-01T01:09:23.574 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout active 2023-12-01T01:09:23.574 INFO:teuthology.orchestra.run.smithi077.stdout:Unit chronyd.service is enabled and running 2023-12-01T01:09:23.574 INFO:teuthology.orchestra.run.smithi077.stdout:Repeating the final host check... 2023-12-01T01:09:23.716 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stdout 4.1.1 2023-12-01T01:09:23.716 INFO:teuthology.orchestra.run.smithi077.stdout:podman (/bin/podman) version 4.1.1 is present 2023-12-01T01:09:23.717 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl is present 2023-12-01T01:09:23.717 INFO:teuthology.orchestra.run.smithi077.stdout:lvcreate is present 2023-12-01T01:09:23.722 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-12-01T01:09:23.722 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-12-01T01:09:23.728 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-12-01T01:09:23.728 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout inactive 2023-12-01T01:09:23.734 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout enabled 2023-12-01T01:09:23.739 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout active 2023-12-01T01:09:23.740 INFO:teuthology.orchestra.run.smithi077.stdout:Unit chronyd.service is enabled and running 2023-12-01T01:09:23.740 INFO:teuthology.orchestra.run.smithi077.stdout:Host looks OK 2023-12-01T01:09:23.740 INFO:teuthology.orchestra.run.smithi077.stdout:Cluster fsid: 2a06e29a-8fe6-11ee-95a2-87774f69a715 2023-12-01T01:09:23.740 INFO:teuthology.orchestra.run.smithi077.stdout:Acquiring lock 140029088696808 on /run/cephadm/2a06e29a-8fe6-11ee-95a2-87774f69a715.lock 2023-12-01T01:09:23.740 INFO:teuthology.orchestra.run.smithi077.stdout:Lock 140029088696808 acquired on /run/cephadm/2a06e29a-8fe6-11ee-95a2-87774f69a715.lock 2023-12-01T01:09:23.740 INFO:teuthology.orchestra.run.smithi077.stdout:Verifying IP 172.21.15.77 port 3300 ... 2023-12-01T01:09:23.741 INFO:teuthology.orchestra.run.smithi077.stdout:Verifying IP 172.21.15.77 port 6789 ... 2023-12-01T01:09:23.741 INFO:teuthology.orchestra.run.smithi077.stdout:Base mon IP(s) is [172.21.15.77:3300, 172.21.15.77:6789], mon addrv is [v2:172.21.15.77:3300,v1:172.21.15.77:6789] 2023-12-01T01:09:23.744 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp metric 100 2023-12-01T01:09:23.744 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.77 metric 100 2023-12-01T01:09:23.747 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-12-01T01:09:23.747 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2023-12-01T01:09:23.747 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2023-12-01T01:09:23.749 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-12-01T01:09:23.750 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-12-01T01:09:23.750 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-12-01T01:09:23.750 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2023-12-01T01:09:23.750 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout inet6 fe80::579d:dec9:4881:625c/64 scope link noprefixroute 2023-12-01T01:09:23.750 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-12-01T01:09:23.750 INFO:teuthology.orchestra.run.smithi077.stdout:Mon IP `172.21.15.77` is in CIDR network `172.21.0.0/20` 2023-12-01T01:09:23.751 INFO:teuthology.orchestra.run.smithi077.stdout:Mon IP `172.21.15.77` is in CIDR network `172.21.0.0/20` 2023-12-01T01:09:23.751 INFO:teuthology.orchestra.run.smithi077.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-12-01T01:09:23.751 INFO:teuthology.orchestra.run.smithi077.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-12-01T01:09:23.752 INFO:teuthology.orchestra.run.smithi077.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32... 2023-12-01T01:09:24.195 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stdout 96d837748436087e6a178851a9eca1321e50ebbd34bb41f0a6c21f217e801f91 2023-12-01T01:09:24.195 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32... 2023-12-01T01:09:24.195 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stderr Getting image source signatures 2023-12-01T01:09:24.195 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stderr Copying blob sha256:96395add11bdaea6c79b23b622f6a5a532b33bf04ac49dcc9b4900c1b06af734 2023-12-01T01:09:24.195 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stderr Copying blob sha256:d3e436ac93e4b5a6d94c7ac8024df331e66e67275836276615dc044fd3cbf83a 2023-12-01T01:09:24.195 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stderr Copying config sha256:96d837748436087e6a178851a9eca1321e50ebbd34bb41f0a6c21f217e801f91 2023-12-01T01:09:24.196 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stderr Writing manifest to image destination 2023-12-01T01:09:24.196 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stderr Storing signatures 2023-12-01T01:09:24.939 INFO:teuthology.orchestra.run.smithi077.stdout:ceph: stdout ceph version 18.2.0-1249-ga6f82997 (a6f82997ac57b70a895455dfed6256360a1e4c32) reef (stable) 2023-12-01T01:09:24.939 INFO:teuthology.orchestra.run.smithi077.stdout:Ceph version: ceph version 18.2.0-1249-ga6f82997 (a6f82997ac57b70a895455dfed6256360a1e4c32) reef (stable) 2023-12-01T01:09:24.940 INFO:teuthology.orchestra.run.smithi077.stdout:Extracting ceph user uid/gid from container image... 2023-12-01T01:09:25.590 INFO:teuthology.orchestra.run.smithi077.stdout:stat: stdout 167 167 2023-12-01T01:09:25.590 INFO:teuthology.orchestra.run.smithi077.stdout:Creating initial keys... 2023-12-01T01:09:26.334 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph-authtool: stdout AQBFMmllSEjrOBAA/RRLOKt2g1enzpSyTyVQjA== 2023-12-01T01:09:26.980 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph-authtool: stdout AQBGMmllWXNSKBAAQCmc4IGnoNsYc79TOSrT1A== 2023-12-01T01:09:27.727 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph-authtool: stdout AQBHMmllmwz5FBAAgAmM9N+Y4tO43XqpN8WG1Q== 2023-12-01T01:09:27.728 INFO:teuthology.orchestra.run.smithi077.stdout:Creating initial monmap... 2023-12-01T01:09:28.373 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-12-01T01:09:28.373 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-12-01T01:09:28.373 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 2a06e29a-8fe6-11ee-95a2-87774f69a715 2023-12-01T01:09:28.373 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-12-01T01:09:28.373 INFO:teuthology.orchestra.run.smithi077.stdout:monmaptool for a [v2:172.21.15.77:3300,v1:172.21.15.77:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-12-01T01:09:28.373 INFO:teuthology.orchestra.run.smithi077.stdout:setting min_mon_release = pacific 2023-12-01T01:09:28.373 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/monmaptool: set fsid to 2a06e29a-8fe6-11ee-95a2-87774f69a715 2023-12-01T01:09:28.373 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-12-01T01:09:28.373 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:09:28.374 INFO:teuthology.orchestra.run.smithi077.stdout:Creating mon... 2023-12-01T01:09:29.121 INFO:teuthology.orchestra.run.smithi077.stdout:create mon.a on 2023-12-01T01:09:29.242 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-12-01T01:09:29.329 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-12-01T01:09:29.455 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715.target → /etc/systemd/system/ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715.target. 2023-12-01T01:09:29.456 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715.target → /etc/systemd/system/ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715.target. 2023-12-01T01:09:29.633 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mon.a 2023-12-01T01:09:29.633 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mon.a.service: Unit ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mon.a.service not loaded. 2023-12-01T01:09:29.721 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715.target.wants/ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mon.a.service → /etc/systemd/system/ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@.service. 2023-12-01T01:09:29.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:29 smithi077 systemd[1]: Starting Ceph mon.a for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:09:30.224 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-12-01T01:09:30.224 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout disabled 2023-12-01T01:09:30.231 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-12-01T01:09:30.231 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout inactive 2023-12-01T01:09:30.231 INFO:teuthology.orchestra.run.smithi077.stdout:firewalld.service is not enabled 2023-12-01T01:09:30.231 INFO:teuthology.orchestra.run.smithi077.stdout:Not possible to enable service . firewalld.service is not available 2023-12-01T01:09:30.232 INFO:teuthology.orchestra.run.smithi077.stdout:Waiting for mon to start... 2023-12-01T01:09:30.232 INFO:teuthology.orchestra.run.smithi077.stdout:Waiting for mon... 2023-12-01T01:09:30.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:29 smithi077 podman[98329]: 2023-12-01T01:09:30.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:30 smithi077 bash[98329]: 5ce8c2d4ecacd2913e8eb31368e81ea98bbc07d9d5365f3f6a4eb7451aa84d2b 2023-12-01T01:09:30.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:30 smithi077 systemd[1]: Started Ceph mon.a for 2a06e29a-8fe6-11ee-95a2-87774f69a715. 2023-12-01T01:09:30.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:30 smithi077 ceph-mon[98370]: mkfs 2a06e29a-8fe6-11ee-95a2-87774f69a715 2023-12-01T01:09:30.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:30 smithi077 ceph-mon[98370]: mon.a is new leader, mons a in quorum (ranks 0) 2023-12-01T01:09:31.082 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout cluster: 2023-12-01T01:09:31.082 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout id: 2a06e29a-8fe6-11ee-95a2-87774f69a715 2023-12-01T01:09:31.082 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-12-01T01:09:31.083 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2023-12-01T01:09:31.083 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout services: 2023-12-01T01:09:31.083 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.555706s) 2023-12-01T01:09:31.083 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-12-01T01:09:31.083 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-12-01T01:09:31.083 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2023-12-01T01:09:31.083 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout data: 2023-12-01T01:09:31.083 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-12-01T01:09:31.083 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-12-01T01:09:31.083 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-12-01T01:09:31.083 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout pgs: 2023-12-01T01:09:31.083 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2023-12-01T01:09:31.083 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.688+0000 7fa2cb090700 1 Processor -- start 2023-12-01T01:09:31.083 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.689+0000 7fa2cb090700 1 -- start start 2023-12-01T01:09:31.083 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.689+0000 7fa2cb090700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2c41055c0 0x7fa2c41059e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:09:31.083 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.689+0000 7fa2cb090700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c4105fb0 con 0x7fa2c41055c0 2023-12-01T01:09:31.084 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.689+0000 7fa2c8e2c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2c41055c0 0x7fa2c41059e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:09:31.084 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.689+0000 7fa2c8e2c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2c41055c0 0x7fa2c41059e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56134/0 (socket says 172.21.15.77:56134) 2023-12-01T01:09:31.084 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.689+0000 7fa2c8e2c700 1 -- 172.21.15.77:0/1222084881 learned_addr learned my addr 172.21.15.77:0/1222084881 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:09:31.084 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.689+0000 7fa2c8e2c700 1 -- 172.21.15.77:0/1222084881 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c41067c0 con 0x7fa2c41055c0 2023-12-01T01:09:31.084 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.689+0000 7fa2c8e2c700 1 --2- 172.21.15.77:0/1222084881 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2c41055c0 0x7fa2c41059e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa2b80098a0 tx=0x7fa2b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=243e7546f011e400 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:09:31.084 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.690+0000 7fa2c37fe700 1 -- 172.21.15.77:0/1222084881 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa2b8004160 con 0x7fa2c41055c0 2023-12-01T01:09:31.084 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.690+0000 7fa2c37fe700 1 -- 172.21.15.77:0/1222084881 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b80042c0 con 0x7fa2c41055c0 2023-12-01T01:09:31.084 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.690+0000 7fa2c37fe700 1 -- 172.21.15.77:0/1222084881 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa2b80045a0 con 0x7fa2c41055c0 2023-12-01T01:09:31.084 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.690+0000 7fa2cb090700 1 -- 172.21.15.77:0/1222084881 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2c41055c0 msgr2=0x7fa2c41059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:09:31.084 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.690+0000 7fa2cb090700 1 --2- 172.21.15.77:0/1222084881 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2c41055c0 0x7fa2c41059e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa2b80098a0 tx=0x7fa2b8009bb0 comp rx=0 tx=0).stop 2023-12-01T01:09:31.084 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.690+0000 7fa2cb090700 1 -- 172.21.15.77:0/1222084881 shutdown_connections 2023-12-01T01:09:31.084 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.690+0000 7fa2cb090700 1 --2- 172.21.15.77:0/1222084881 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2c41055c0 0x7fa2c41059e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:09:31.084 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.690+0000 7fa2cb090700 1 -- 172.21.15.77:0/1222084881 >> 172.21.15.77:0/1222084881 conn(0x7fa2c4100b80 msgr2=0x7fa2c4102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:09:31.084 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.690+0000 7fa2cb090700 1 -- 172.21.15.77:0/1222084881 shutdown_connections 2023-12-01T01:09:31.084 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.690+0000 7fa2cb090700 1 -- 172.21.15.77:0/1222084881 wait complete. 2023-12-01T01:09:31.085 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.691+0000 7fa2cb090700 1 Processor -- start 2023-12-01T01:09:31.085 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.691+0000 7fa2cb090700 1 -- start start 2023-12-01T01:09:31.085 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.691+0000 7fa2cb090700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2c41055c0 0x7fa2c4198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:09:31.085 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.691+0000 7fa2cb090700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c4198fb0 con 0x7fa2c41055c0 2023-12-01T01:09:31.085 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.691+0000 7fa2c8e2c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2c41055c0 0x7fa2c4198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:09:31.085 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.691+0000 7fa2c8e2c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2c41055c0 0x7fa2c4198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56136/0 (socket says 172.21.15.77:56136) 2023-12-01T01:09:31.085 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.691+0000 7fa2c8e2c700 1 -- 172.21.15.77:0/1603038982 learned_addr learned my addr 172.21.15.77:0/1603038982 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:09:31.085 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.691+0000 7fa2c8e2c700 1 -- 172.21.15.77:0/1603038982 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b8009550 con 0x7fa2c41055c0 2023-12-01T01:09:31.085 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.692+0000 7fa2c8e2c700 1 --2- 172.21.15.77:0/1603038982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2c41055c0 0x7fa2c4198a70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8004000 tx=0x7fa2b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:09:31.085 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.692+0000 7fa2c1ffb700 1 -- 172.21.15.77:0/1603038982 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa2b8011630 con 0x7fa2c41055c0 2023-12-01T01:09:31.085 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.692+0000 7fa2c1ffb700 1 -- 172.21.15.77:0/1603038982 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b8011790 con 0x7fa2c41055c0 2023-12-01T01:09:31.085 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.692+0000 7fa2cb090700 1 -- 172.21.15.77:0/1603038982 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2c41991b0 con 0x7fa2c41055c0 2023-12-01T01:09:31.085 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.692+0000 7fa2c1ffb700 1 -- 172.21.15.77:0/1603038982 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa2b8011a30 con 0x7fa2c41055c0 2023-12-01T01:09:31.085 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.692+0000 7fa2cb090700 1 -- 172.21.15.77:0/1603038982 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c41995d0 con 0x7fa2c41055c0 2023-12-01T01:09:31.086 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.692+0000 7fa2c1ffb700 1 -- 172.21.15.77:0/1603038982 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa2b8022070 con 0x7fa2c41055c0 2023-12-01T01:09:31.086 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.692+0000 7fa2c1ffb700 1 -- 172.21.15.77:0/1603038982 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa2b801c450 con 0x7fa2c41055c0 2023-12-01T01:09:31.086 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.693+0000 7fa2cb090700 1 -- 172.21.15.77:0/1603038982 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2c4192340 con 0x7fa2c41055c0 2023-12-01T01:09:31.086 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.695+0000 7fa2c1ffb700 1 -- 172.21.15.77:0/1603038982 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7fa2b801b450 con 0x7fa2c41055c0 2023-12-01T01:09:31.086 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.763+0000 7fa2cb090700 1 -- 172.21.15.77:0/1603038982 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fa2c402cc70 con 0x7fa2c41055c0 2023-12-01T01:09:31.086 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.764+0000 7fa2c1ffb700 1 -- 172.21.15.77:0/1603038982 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7fa2b801b5f0 con 0x7fa2c41055c0 2023-12-01T01:09:31.086 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.765+0000 7fa2cb090700 1 -- 172.21.15.77:0/1603038982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2c41055c0 msgr2=0x7fa2c4198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:09:31.086 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.765+0000 7fa2cb090700 1 --2- 172.21.15.77:0/1603038982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2c41055c0 0x7fa2c4198a70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8004000 tx=0x7fa2b8004750 comp rx=0 tx=0).stop 2023-12-01T01:09:31.086 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.765+0000 7fa2cb090700 1 -- 172.21.15.77:0/1603038982 shutdown_connections 2023-12-01T01:09:31.086 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.765+0000 7fa2cb090700 1 --2- 172.21.15.77:0/1603038982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2c41055c0 0x7fa2c4198a70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:09:31.086 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.765+0000 7fa2cb090700 1 -- 172.21.15.77:0/1603038982 >> 172.21.15.77:0/1603038982 conn(0x7fa2c4100b80 msgr2=0x7fa2c418cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:09:31.086 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.765+0000 7fa2cb090700 1 -- 172.21.15.77:0/1603038982 shutdown_connections 2023-12-01T01:09:31.086 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:30.765+0000 7fa2cb090700 1 -- 172.21.15.77:0/1603038982 wait complete. 2023-12-01T01:09:31.086 INFO:teuthology.orchestra.run.smithi077.stdout:mon is available 2023-12-01T01:09:31.087 INFO:teuthology.orchestra.run.smithi077.stdout:Assimilating anything we can from ceph.conf... 2023-12-01T01:09:31.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:31 smithi077 ceph-mon[98370]: mon.a is new leader, mons a in quorum (ranks 0) 2023-12-01T01:09:31.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:31 smithi077 ceph-mon[98370]: monmap e1: 1 mons at {a=[v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-01T01:09:31.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:31 smithi077 ceph-mon[98370]: fsmap 2023-12-01T01:09:31.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:31 smithi077 ceph-mon[98370]: osdmap e1: 0 total, 0 up, 0 in 2023-12-01T01:09:31.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:31 smithi077 ceph-mon[98370]: mgrmap e1: no daemons active 2023-12-01T01:09:31.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:31 smithi077 ceph-mon[98370]: from='client.? 172.21.15.77:0/1603038982' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-12-01T01:09:32.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:32 smithi077 ceph-mon[98370]: from='client.? 172.21.15.77:0/1936601978' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-12-01T01:09:32.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:32 smithi077 ceph-mon[98370]: from='client.? 172.21.15.77:0/1936601978' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-12-01T01:09:32.924 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2023-12-01T01:09:32.930 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout [global] 2023-12-01T01:09:32.930 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout fsid = 2a06e29a-8fe6-11ee-95a2-87774f69a715 2023-12-01T01:09:32.931 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.77:3300,v1:172.21.15.77:6789] 2023-12-01T01:09:32.931 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-12-01T01:09:32.931 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-12-01T01:09:32.931 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-12-01T01:09:32.931 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-12-01T01:09:32.931 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2023-12-01T01:09:32.931 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout [mgr] 2023-12-01T01:09:32.931 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-12-01T01:09:32.931 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2023-12-01T01:09:32.931 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout [osd] 2023-12-01T01:09:32.931 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-12-01T01:09:32.931 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-12-01T01:09:32.931 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-12-01T01:09:32.931 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-12-01T01:09:32.932 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-12-01T01:09:32.932 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.534+0000 7f3e31226700 1 Processor -- start 2023-12-01T01:09:32.932 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.534+0000 7f3e31226700 1 -- start start 2023-12-01T01:09:32.932 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.534+0000 7f3e31226700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e2c107170 0x7f3e2c109560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:09:32.932 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.534+0000 7f3e31226700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2c0745c0 con 0x7f3e2c107170 2023-12-01T01:09:32.932 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.534+0000 7f3e2ad9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e2c107170 0x7f3e2c109560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:09:32.932 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.534+0000 7f3e2ad9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e2c107170 0x7f3e2c109560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56138/0 (socket says 172.21.15.77:56138) 2023-12-01T01:09:32.932 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.534+0000 7f3e2ad9d700 1 -- 172.21.15.77:0/3173900192 learned_addr learned my addr 172.21.15.77:0/3173900192 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:09:32.932 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.534+0000 7f3e2ad9d700 1 -- 172.21.15.77:0/3173900192 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e2c109aa0 con 0x7f3e2c107170 2023-12-01T01:09:32.932 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.535+0000 7f3e2ad9d700 1 --2- 172.21.15.77:0/3173900192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e2c107170 0x7f3e2c109560 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3e200098a0 tx=0x7f3e20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5982e770dc471c14 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:09:32.932 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.535+0000 7f3e29d9b700 1 -- 172.21.15.77:0/3173900192 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e20004160 con 0x7f3e2c107170 2023-12-01T01:09:32.932 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.535+0000 7f3e29d9b700 1 -- 172.21.15.77:0/3173900192 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e200042c0 con 0x7f3e2c107170 2023-12-01T01:09:32.932 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.535+0000 7f3e29d9b700 1 -- 172.21.15.77:0/3173900192 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e200045a0 con 0x7f3e2c107170 2023-12-01T01:09:32.932 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.535+0000 7f3e31226700 1 -- 172.21.15.77:0/3173900192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e2c107170 msgr2=0x7f3e2c109560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:09:32.933 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.535+0000 7f3e31226700 1 --2- 172.21.15.77:0/3173900192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e2c107170 0x7f3e2c109560 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3e200098a0 tx=0x7f3e20009bb0 comp rx=0 tx=0).stop 2023-12-01T01:09:32.933 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.535+0000 7f3e31226700 1 -- 172.21.15.77:0/3173900192 shutdown_connections 2023-12-01T01:09:32.933 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.535+0000 7f3e31226700 1 --2- 172.21.15.77:0/3173900192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e2c107170 0x7f3e2c109560 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:09:32.933 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.535+0000 7f3e31226700 1 -- 172.21.15.77:0/3173900192 >> 172.21.15.77:0/3173900192 conn(0x7f3e2c100b80 msgr2=0x7f3e2c102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:09:32.933 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.535+0000 7f3e31226700 1 -- 172.21.15.77:0/3173900192 shutdown_connections 2023-12-01T01:09:32.933 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.535+0000 7f3e31226700 1 -- 172.21.15.77:0/3173900192 wait complete. 2023-12-01T01:09:32.933 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.536+0000 7f3e31226700 1 Processor -- start 2023-12-01T01:09:32.933 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.536+0000 7f3e31226700 1 -- start start 2023-12-01T01:09:32.933 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.537+0000 7f3e31226700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e2c107170 0x7f3e2c199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:09:32.933 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.537+0000 7f3e31226700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2c19a0c0 con 0x7f3e2c107170 2023-12-01T01:09:32.933 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.537+0000 7f3e2ad9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e2c107170 0x7f3e2c199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:09:32.933 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.537+0000 7f3e2ad9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e2c107170 0x7f3e2c199b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56140/0 (socket says 172.21.15.77:56140) 2023-12-01T01:09:32.933 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.537+0000 7f3e2ad9d700 1 -- 172.21.15.77:0/1936601978 learned_addr learned my addr 172.21.15.77:0/1936601978 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:09:32.933 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.537+0000 7f3e2ad9d700 1 -- 172.21.15.77:0/1936601978 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e20009550 con 0x7f3e2c107170 2023-12-01T01:09:32.934 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.537+0000 7f3e2ad9d700 1 --2- 172.21.15.77:0/1936601978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e2c107170 0x7f3e2c199b80 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c073a00 tx=0x7f3e20004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:09:32.934 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.537+0000 7f3e13fff700 1 -- 172.21.15.77:0/1936601978 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e20011630 con 0x7f3e2c107170 2023-12-01T01:09:32.934 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.537+0000 7f3e13fff700 1 -- 172.21.15.77:0/1936601978 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e20011790 con 0x7f3e2c107170 2023-12-01T01:09:32.934 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.537+0000 7f3e31226700 1 -- 172.21.15.77:0/1936601978 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e2c19a2c0 con 0x7f3e2c107170 2023-12-01T01:09:32.934 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.537+0000 7f3e13fff700 1 -- 172.21.15.77:0/1936601978 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e20011a30 con 0x7f3e2c107170 2023-12-01T01:09:32.934 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.537+0000 7f3e31226700 1 -- 172.21.15.77:0/1936601978 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e2c19a6e0 con 0x7f3e2c107170 2023-12-01T01:09:32.934 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.538+0000 7f3e13fff700 1 -- 172.21.15.77:0/1936601978 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f3e20022070 con 0x7f3e2c107170 2023-12-01T01:09:32.934 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.538+0000 7f3e13fff700 1 -- 172.21.15.77:0/1936601978 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3e2001c450 con 0x7f3e2c107170 2023-12-01T01:09:32.934 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.538+0000 7f3e31226700 1 -- 172.21.15.77:0/1936601978 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e2c1939c0 con 0x7f3e2c107170 2023-12-01T01:09:32.934 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.541+0000 7f3e13fff700 1 -- 172.21.15.77:0/1936601978 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f3e2001b450 con 0x7f3e2c107170 2023-12-01T01:09:32.934 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.606+0000 7f3e31226700 1 -- 172.21.15.77:0/1936601978 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f3e2c0623c0 con 0x7f3e2c107170 2023-12-01T01:09:32.934 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.610+0000 7f3e13fff700 1 -- 172.21.15.77:0/1936601978 <== mon.0 v2:172.21.15.77:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3e2001b5f0 con 0x7f3e2c107170 2023-12-01T01:09:32.934 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.610+0000 7f3e13fff700 1 -- 172.21.15.77:0/1936601978 <== mon.0 v2:172.21.15.77:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f3e20024630 con 0x7f3e2c107170 2023-12-01T01:09:32.934 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.611+0000 7f3e31226700 1 -- 172.21.15.77:0/1936601978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e2c107170 msgr2=0x7f3e2c199b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:09:32.935 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.611+0000 7f3e31226700 1 --2- 172.21.15.77:0/1936601978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e2c107170 0x7f3e2c199b80 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c073a00 tx=0x7f3e20004750 comp rx=0 tx=0).stop 2023-12-01T01:09:32.935 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.611+0000 7f3e31226700 1 -- 172.21.15.77:0/1936601978 shutdown_connections 2023-12-01T01:09:32.935 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.611+0000 7f3e31226700 1 --2- 172.21.15.77:0/1936601978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e2c107170 0x7f3e2c199b80 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:09:32.935 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.611+0000 7f3e31226700 1 -- 172.21.15.77:0/1936601978 >> 172.21.15.77:0/1936601978 conn(0x7f3e2c100b80 msgr2=0x7f3e2c102500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:09:32.935 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.611+0000 7f3e31226700 1 -- 172.21.15.77:0/1936601978 shutdown_connections 2023-12-01T01:09:32.935 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:31.611+0000 7f3e31226700 1 -- 172.21.15.77:0/1936601978 wait complete. 2023-12-01T01:09:32.935 INFO:teuthology.orchestra.run.smithi077.stdout:Generating new minimal ceph.conf... 2023-12-01T01:09:35.087 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.573+0000 7f3424d3b700 1 Processor -- start 2023-12-01T01:09:35.087 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.573+0000 7f3424d3b700 1 -- start start 2023-12-01T01:09:35.087 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.573+0000 7f3424d3b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3420105aa0 0x7f3420105ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:09:35.087 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.573+0000 7f3424d3b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3420106490 con 0x7f3420105aa0 2023-12-01T01:09:35.087 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.573+0000 7f341e59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3420105aa0 0x7f3420105ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:09:35.087 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.573+0000 7f341e59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3420105aa0 0x7f3420105ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56142/0 (socket says 172.21.15.77:56142) 2023-12-01T01:09:35.087 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.573+0000 7f341e59c700 1 -- 172.21.15.77:0/223391310 learned_addr learned my addr 172.21.15.77:0/223391310 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:09:35.087 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.574+0000 7f341e59c700 1 -- 172.21.15.77:0/223391310 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34201067c0 con 0x7f3420105aa0 2023-12-01T01:09:35.087 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.574+0000 7f341e59c700 1 --2- 172.21.15.77:0/223391310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3420105aa0 0x7f3420105ec0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f341000bb40 tx=0x7f341000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba87e1fbf3ee135e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:09:35.088 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.574+0000 7f341d59a700 1 -- 172.21.15.77:0/223391310 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f341000dbe0 con 0x7f3420105aa0 2023-12-01T01:09:35.088 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.574+0000 7f341d59a700 1 -- 172.21.15.77:0/223391310 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f34100044d0 con 0x7f3420105aa0 2023-12-01T01:09:35.088 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.574+0000 7f341d59a700 1 -- 172.21.15.77:0/223391310 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3410010470 con 0x7f3420105aa0 2023-12-01T01:09:35.088 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.575+0000 7f3424d3b700 1 -- 172.21.15.77:0/223391310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3420105aa0 msgr2=0x7f3420105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:09:35.088 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.575+0000 7f3424d3b700 1 --2- 172.21.15.77:0/223391310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3420105aa0 0x7f3420105ec0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f341000bb40 tx=0x7f341000be50 comp rx=0 tx=0).stop 2023-12-01T01:09:35.088 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.575+0000 7f3424d3b700 1 -- 172.21.15.77:0/223391310 shutdown_connections 2023-12-01T01:09:35.088 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.575+0000 7f3424d3b700 1 --2- 172.21.15.77:0/223391310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3420105aa0 0x7f3420105ec0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:09:35.088 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.575+0000 7f3424d3b700 1 -- 172.21.15.77:0/223391310 >> 172.21.15.77:0/223391310 conn(0x7f3420100300 msgr2=0x7f3420102720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:09:35.088 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.575+0000 7f3424d3b700 1 -- 172.21.15.77:0/223391310 shutdown_connections 2023-12-01T01:09:35.088 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.575+0000 7f3424d3b700 1 -- 172.21.15.77:0/223391310 wait complete. 2023-12-01T01:09:35.089 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.575+0000 7f3424d3b700 1 Processor -- start 2023-12-01T01:09:35.089 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.576+0000 7f3424d3b700 1 -- start start 2023-12-01T01:09:35.089 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.576+0000 7f3424d3b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3420105aa0 0x7f342019b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:09:35.089 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.576+0000 7f3424d3b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342019bbf0 con 0x7f3420105aa0 2023-12-01T01:09:35.089 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.576+0000 7f341e59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3420105aa0 0x7f342019b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:09:35.089 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.576+0000 7f341e59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3420105aa0 0x7f342019b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56144/0 (socket says 172.21.15.77:56144) 2023-12-01T01:09:35.089 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.576+0000 7f341e59c700 1 -- 172.21.15.77:0/2554745418 learned_addr learned my addr 172.21.15.77:0/2554745418 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:09:35.089 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.576+0000 7f341e59c700 1 -- 172.21.15.77:0/2554745418 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341000b7f0 con 0x7f3420105aa0 2023-12-01T01:09:35.089 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.576+0000 7f341e59c700 1 --2- 172.21.15.77:0/2554745418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3420105aa0 0x7f342019b6b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3410003a80 tx=0x7f3410003df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:09:35.089 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.576+0000 7f340f7fe700 1 -- 172.21.15.77:0/2554745418 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f34100106e0 con 0x7f3420105aa0 2023-12-01T01:09:35.089 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.576+0000 7f340f7fe700 1 -- 172.21.15.77:0/2554745418 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3410010ce0 con 0x7f3420105aa0 2023-12-01T01:09:35.089 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.576+0000 7f3424d3b700 1 -- 172.21.15.77:0/2554745418 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f342019bdf0 con 0x7f3420105aa0 2023-12-01T01:09:35.089 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.576+0000 7f340f7fe700 1 -- 172.21.15.77:0/2554745418 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3410021c60 con 0x7f3420105aa0 2023-12-01T01:09:35.089 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.576+0000 7f3424d3b700 1 -- 172.21.15.77:0/2554745418 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34201982e0 con 0x7f3420105aa0 2023-12-01T01:09:35.089 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.578+0000 7f340f7fe700 1 -- 172.21.15.77:0/2554745418 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f341002a3f0 con 0x7f3420105aa0 2023-12-01T01:09:35.090 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.578+0000 7f340f7fe700 1 -- 172.21.15.77:0/2554745418 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f34100193f0 con 0x7f3420105aa0 2023-12-01T01:09:35.090 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.578+0000 7f3424d3b700 1 -- 172.21.15.77:0/2554745418 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f342019bf80 con 0x7f3420105aa0 2023-12-01T01:09:35.090 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.580+0000 7f340f7fe700 1 -- 172.21.15.77:0/2554745418 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f341002f060 con 0x7f3420105aa0 2023-12-01T01:09:35.090 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.650+0000 7f3424d3b700 1 -- 172.21.15.77:0/2554745418 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f34200623c0 con 0x7f3420105aa0 2023-12-01T01:09:35.090 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.650+0000 7f340f7fe700 1 -- 172.21.15.77:0/2554745418 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f3410021410 con 0x7f3420105aa0 2023-12-01T01:09:35.090 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.651+0000 7f3424d3b700 1 -- 172.21.15.77:0/2554745418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3420105aa0 msgr2=0x7f342019b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:09:35.090 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.651+0000 7f3424d3b700 1 --2- 172.21.15.77:0/2554745418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3420105aa0 0x7f342019b6b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3410003a80 tx=0x7f3410003df0 comp rx=0 tx=0).stop 2023-12-01T01:09:35.090 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.651+0000 7f3424d3b700 1 -- 172.21.15.77:0/2554745418 shutdown_connections 2023-12-01T01:09:35.090 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.651+0000 7f3424d3b700 1 --2- 172.21.15.77:0/2554745418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3420105aa0 0x7f342019b6b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:09:35.090 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.651+0000 7f3424d3b700 1 -- 172.21.15.77:0/2554745418 >> 172.21.15.77:0/2554745418 conn(0x7f3420100300 msgr2=0x7f342018ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:09:35.090 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.651+0000 7f3424d3b700 1 -- 172.21.15.77:0/2554745418 shutdown_connections 2023-12-01T01:09:35.090 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:34.651+0000 7f3424d3b700 1 -- 172.21.15.77:0/2554745418 wait complete. 2023-12-01T01:09:35.090 INFO:teuthology.orchestra.run.smithi077.stdout:Restarting the monitor... 2023-12-01T01:09:35.091 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:34 smithi077 ceph-mon[98370]: from='client.? 172.21.15.77:0/2554745418' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:09:35.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:35 smithi077 systemd[1]: Stopping Ceph mon.a for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:09:35.717 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:35 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mon-a[98345]: 2023-12-01T01:09:35.406+0000 7fbe0b096700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-12-01T01:09:35.717 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:35 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mon-a[98345]: 2023-12-01T01:09:35.406+0000 7fbe0b096700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-12-01T01:09:36.055 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:35 smithi077 bash[98698]: ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mon-a 2023-12-01T01:09:36.313 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 systemd[1]: ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mon.a.service: Succeeded. 2023-12-01T01:09:36.313 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 systemd[1]: Stopped Ceph mon.a for 2a06e29a-8fe6-11ee-95a2-87774f69a715. 2023-12-01T01:09:36.313 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 systemd[1]: Starting Ceph mon.a for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:09:36.539 INFO:teuthology.orchestra.run.smithi077.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-12-01T01:09:36.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 podman[98827]: 2023-12-01T01:09:36.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 bash[98827]: b7510927ab445e77855cb80bbd2767ebd858162cb4791131cbdec922cfbd174f 2023-12-01T01:09:36.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: set uid:gid to 167:167 (ceph:ceph) 2023-12-01T01:09:36.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: ceph version 18.2.0-1249-ga6f82997 (a6f82997ac57b70a895455dfed6256360a1e4c32) reef (stable), process ceph-mon, pid 7 2023-12-01T01:09:36.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: pidfile_write: ignore empty --pid-file 2023-12-01T01:09:36.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: load: jerasure load: lrc 2023-12-01T01:09:36.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: RocksDB version: 7.9.2 2023-12-01T01:09:36.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Git sha 0 2023-12-01T01:09:36.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Compile date 2023-11-30 20:12:29 2023-12-01T01:09:36.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: DB SUMMARY 2023-12-01T01:09:36.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: DB Session ID: HGUFLH0XVL4I1X5ZEK65 2023-12-01T01:09:36.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: CURRENT file: CURRENT 2023-12-01T01:09:36.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: IDENTITY file: IDENTITY 2023-12-01T01:09:36.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-12-01T01:09:36.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-12-01T01:09:36.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 100232 ; 2023-12-01T01:09:36.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.error_if_exists: 0 2023-12-01T01:09:36.656 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.create_if_missing: 0 2023-12-01T01:09:36.656 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.paranoid_checks: 1 2023-12-01T01:09:36.656 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.flush_verify_memtable_count: 1 2023-12-01T01:09:36.656 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-12-01T01:09:36.656 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-12-01T01:09:36.656 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.env: 0x5579431ee720 2023-12-01T01:09:36.656 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.fs: PosixFileSystem 2023-12-01T01:09:36.656 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.info_log: 0x55794658b440 2023-12-01T01:09:36.656 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.max_file_opening_threads: 16 2023-12-01T01:09:36.656 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.statistics: (nil) 2023-12-01T01:09:36.656 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.use_fsync: 0 2023-12-01T01:09:36.656 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.max_log_file_size: 0 2023-12-01T01:09:36.656 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-12-01T01:09:36.656 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.log_file_time_to_roll: 0 2023-12-01T01:09:36.656 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.keep_log_file_num: 1000 2023-12-01T01:09:36.657 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.recycle_log_file_num: 0 2023-12-01T01:09:36.657 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.allow_fallocate: 1 2023-12-01T01:09:36.657 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.allow_mmap_reads: 0 2023-12-01T01:09:36.657 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.allow_mmap_writes: 0 2023-12-01T01:09:36.657 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.use_direct_reads: 0 2023-12-01T01:09:36.657 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-12-01T01:09:36.657 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.create_missing_column_families: 0 2023-12-01T01:09:36.657 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.db_log_dir: 2023-12-01T01:09:36.657 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.wal_dir: 2023-12-01T01:09:36.657 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.table_cache_numshardbits: 6 2023-12-01T01:09:36.657 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.WAL_ttl_seconds: 0 2023-12-01T01:09:36.657 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.WAL_size_limit_MB: 0 2023-12-01T01:09:36.657 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-12-01T01:09:36.657 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-12-01T01:09:36.657 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.is_fd_close_on_exec: 1 2023-12-01T01:09:36.658 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.advise_random_on_open: 1 2023-12-01T01:09:36.658 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.db_write_buffer_size: 0 2023-12-01T01:09:36.658 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.write_buffer_manager: 0x55794581c320 2023-12-01T01:09:36.658 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-12-01T01:09:36.658 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-12-01T01:09:36.658 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.use_adaptive_mutex: 0 2023-12-01T01:09:36.658 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.rate_limiter: (nil) 2023-12-01T01:09:36.658 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-12-01T01:09:36.658 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.wal_recovery_mode: 2 2023-12-01T01:09:36.658 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.enable_thread_tracking: 0 2023-12-01T01:09:36.658 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.enable_pipelined_write: 0 2023-12-01T01:09:36.659 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.unordered_write: 0 2023-12-01T01:09:36.659 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-12-01T01:09:36.659 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-12-01T01:09:36.659 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-12-01T01:09:36.659 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-12-01T01:09:36.659 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.row_cache: None 2023-12-01T01:09:36.659 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.wal_filter: None 2023-12-01T01:09:36.659 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-12-01T01:09:36.659 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.allow_ingest_behind: 0 2023-12-01T01:09:36.660 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.two_write_queues: 0 2023-12-01T01:09:36.660 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.manual_wal_flush: 0 2023-12-01T01:09:36.660 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.wal_compression: 0 2023-12-01T01:09:36.660 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.atomic_flush: 0 2023-12-01T01:09:36.660 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-12-01T01:09:36.660 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.persist_stats_to_disk: 0 2023-12-01T01:09:36.660 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.write_dbid_to_manifest: 0 2023-12-01T01:09:36.660 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.log_readahead_size: 0 2023-12-01T01:09:36.660 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-12-01T01:09:36.660 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.best_efforts_recovery: 0 2023-12-01T01:09:36.660 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-12-01T01:09:36.660 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-12-01T01:09:36.660 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.allow_data_in_errors: 0 2023-12-01T01:09:36.662 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.db_host_id: __hostname__ 2023-12-01T01:09:36.662 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.enforce_single_del_contracts: true 2023-12-01T01:09:36.662 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.max_background_jobs: 2 2023-12-01T01:09:36.662 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.max_background_compactions: -1 2023-12-01T01:09:36.662 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.max_subcompactions: 1 2023-12-01T01:09:36.662 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-12-01T01:09:36.662 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-12-01T01:09:36.662 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.delayed_write_rate : 16777216 2023-12-01T01:09:36.662 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.max_total_wal_size: 0 2023-12-01T01:09:36.662 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-12-01T01:09:36.663 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.stats_dump_period_sec: 600 2023-12-01T01:09:36.663 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.stats_persist_period_sec: 600 2023-12-01T01:09:36.663 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-12-01T01:09:36.663 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.max_open_files: -1 2023-12-01T01:09:36.663 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.bytes_per_sync: 0 2023-12-01T01:09:36.663 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.wal_bytes_per_sync: 0 2023-12-01T01:09:36.663 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.strict_bytes_per_sync: 0 2023-12-01T01:09:36.663 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.compaction_readahead_size: 0 2023-12-01T01:09:36.663 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.max_background_flushes: -1 2023-12-01T01:09:36.663 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Compression algorithms supported: 2023-12-01T01:09:36.663 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-12-01T01:09:36.663 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: kZSTD supported: 0 2023-12-01T01:09:36.663 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: kXpressCompression supported: 0 2023-12-01T01:09:36.663 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: kLZ4HCCompression supported: 1 2023-12-01T01:09:36.663 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: kZlibCompression supported: 1 2023-12-01T01:09:36.664 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: kSnappyCompression supported: 1 2023-12-01T01:09:36.664 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: kLZ4Compression supported: 1 2023-12-01T01:09:36.664 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: kBZip2Compression supported: 0 2023-12-01T01:09:36.664 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Fast CRC32 supported: Supported on x86 2023-12-01T01:09:36.664 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: DMutex implementation: pthread_mutex_t 2023-12-01T01:09:36.664 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-12-01T01:09:36.664 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-12-01T01:09:36.664 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-12-01T01:09:36.664 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.merge_operator: 2023-12-01T01:09:36.664 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.compaction_filter: None 2023-12-01T01:09:36.664 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.compaction_filter_factory: None 2023-12-01T01:09:36.664 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.sst_partitioner_factory: None 2023-12-01T01:09:36.664 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.memtable_factory: SkipListFactory 2023-12-01T01:09:36.665 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.table_factory: BlockBasedTable 2023-12-01T01:09:36.665 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55794658b540) 2023-12-01T01:09:36.665 INFO:journalctl@ceph.mon.a.smithi077.stdout: cache_index_and_filter_blocks: 1 2023-12-01T01:09:36.665 INFO:journalctl@ceph.mon.a.smithi077.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-12-01T01:09:36.665 INFO:journalctl@ceph.mon.a.smithi077.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-12-01T01:09:36.665 INFO:journalctl@ceph.mon.a.smithi077.stdout: pin_top_level_index_and_filter: 1 2023-12-01T01:09:36.665 INFO:journalctl@ceph.mon.a.smithi077.stdout: index_type: 0 2023-12-01T01:09:36.665 INFO:journalctl@ceph.mon.a.smithi077.stdout: data_block_index_type: 0 2023-12-01T01:09:36.665 INFO:journalctl@ceph.mon.a.smithi077.stdout: index_shortening: 1 2023-12-01T01:09:36.665 INFO:journalctl@ceph.mon.a.smithi077.stdout: data_block_hash_table_util_ratio: 0.750000 2023-12-01T01:09:36.665 INFO:journalctl@ceph.mon.a.smithi077.stdout: checksum: 4 2023-12-01T01:09:36.665 INFO:journalctl@ceph.mon.a.smithi077.stdout: no_block_cache: 0 2023-12-01T01:09:36.665 INFO:journalctl@ceph.mon.a.smithi077.stdout: block_cache: 0x55794589d350 2023-12-01T01:09:36.665 INFO:journalctl@ceph.mon.a.smithi077.stdout: block_cache_name: BinnedLRUCache 2023-12-01T01:09:36.666 INFO:journalctl@ceph.mon.a.smithi077.stdout: block_cache_options: 2023-12-01T01:09:36.666 INFO:journalctl@ceph.mon.a.smithi077.stdout: capacity : 536870912 2023-12-01T01:09:36.666 INFO:journalctl@ceph.mon.a.smithi077.stdout: num_shard_bits : 4 2023-12-01T01:09:36.666 INFO:journalctl@ceph.mon.a.smithi077.stdout: strict_capacity_limit : 0 2023-12-01T01:09:36.666 INFO:journalctl@ceph.mon.a.smithi077.stdout: high_pri_pool_ratio: 0.000 2023-12-01T01:09:36.666 INFO:journalctl@ceph.mon.a.smithi077.stdout: block_cache_compressed: (nil) 2023-12-01T01:09:36.666 INFO:journalctl@ceph.mon.a.smithi077.stdout: persistent_cache: (nil) 2023-12-01T01:09:36.666 INFO:journalctl@ceph.mon.a.smithi077.stdout: block_size: 4096 2023-12-01T01:09:36.666 INFO:journalctl@ceph.mon.a.smithi077.stdout: block_size_deviation: 10 2023-12-01T01:09:36.666 INFO:journalctl@ceph.mon.a.smithi077.stdout: block_restart_interval: 16 2023-12-01T01:09:36.666 INFO:journalctl@ceph.mon.a.smithi077.stdout: index_block_restart_interval: 1 2023-12-01T01:09:36.666 INFO:journalctl@ceph.mon.a.smithi077.stdout: metadata_block_size: 4096 2023-12-01T01:09:36.666 INFO:journalctl@ceph.mon.a.smithi077.stdout: partition_filters: 0 2023-12-01T01:09:36.667 INFO:journalctl@ceph.mon.a.smithi077.stdout: use_delta_encoding: 1 2023-12-01T01:09:36.667 INFO:journalctl@ceph.mon.a.smithi077.stdout: filter_policy: bloomfilter 2023-12-01T01:09:36.667 INFO:journalctl@ceph.mon.a.smithi077.stdout: whole_key_filtering: 1 2023-12-01T01:09:36.667 INFO:journalctl@ceph.mon.a.smithi077.stdout: verify_compression: 0 2023-12-01T01:09:36.667 INFO:journalctl@ceph.mon.a.smithi077.stdout: read_amp_bytes_per_bit: 0 2023-12-01T01:09:36.667 INFO:journalctl@ceph.mon.a.smithi077.stdout: format_version: 5 2023-12-01T01:09:36.667 INFO:journalctl@ceph.mon.a.smithi077.stdout: enable_index_compression: 1 2023-12-01T01:09:36.667 INFO:journalctl@ceph.mon.a.smithi077.stdout: block_align: 0 2023-12-01T01:09:36.667 INFO:journalctl@ceph.mon.a.smithi077.stdout: max_auto_readahead_size: 262144 2023-12-01T01:09:36.667 INFO:journalctl@ceph.mon.a.smithi077.stdout: prepopulate_block_cache: 0 2023-12-01T01:09:36.667 INFO:journalctl@ceph.mon.a.smithi077.stdout: initial_auto_readahead_size: 8192 2023-12-01T01:09:36.667 INFO:journalctl@ceph.mon.a.smithi077.stdout: num_file_reads_for_auto_readahead: 2 2023-12-01T01:09:36.667 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.write_buffer_size: 33554432 2023-12-01T01:09:36.667 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.max_write_buffer_number: 2 2023-12-01T01:09:36.667 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.compression: NoCompression 2023-12-01T01:09:36.667 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.bottommost_compression: Disabled 2023-12-01T01:09:36.668 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.prefix_extractor: nullptr 2023-12-01T01:09:36.668 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-12-01T01:09:36.668 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.num_levels: 7 2023-12-01T01:09:36.668 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-12-01T01:09:36.668 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-12-01T01:09:36.668 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-12-01T01:09:36.668 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-12-01T01:09:36.668 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-12-01T01:09:36.668 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-12-01T01:09:36.668 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-12-01T01:09:36.668 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-12-01T01:09:36.668 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-12-01T01:09:36.668 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-12-01T01:09:36.668 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-12-01T01:09:36.669 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-12-01T01:09:36.669 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.compression_opts.window_bits: -14 2023-12-01T01:09:36.669 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.compression_opts.level: 32767 2023-12-01T01:09:36.669 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.compression_opts.strategy: 0 2023-12-01T01:09:36.669 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-12-01T01:09:36.669 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-12-01T01:09:36.669 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-12-01T01:09:36.669 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-12-01T01:09:36.670 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.compression_opts.enabled: false 2023-12-01T01:09:36.670 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-12-01T01:09:36.670 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-12-01T01:09:36.670 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-12-01T01:09:36.670 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-12-01T01:09:36.670 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.target_file_size_base: 67108864 2023-12-01T01:09:36.670 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.target_file_size_multiplier: 1 2023-12-01T01:09:36.670 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-12-01T01:09:36.670 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-12-01T01:09:36.670 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-12-01T01:09:36.670 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-12-01T01:09:36.670 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-12-01T01:09:36.670 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-12-01T01:09:36.671 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-12-01T01:09:36.671 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-12-01T01:09:36.671 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-12-01T01:09:36.671 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-12-01T01:09:36.671 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-12-01T01:09:36.671 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-12-01T01:09:36.671 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-12-01T01:09:36.671 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.arena_block_size: 1048576 2023-12-01T01:09:36.671 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-12-01T01:09:36.671 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-12-01T01:09:36.671 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.disable_auto_compactions: 0 2023-12-01T01:09:36.671 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-12-01T01:09:36.671 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-12-01T01:09:36.672 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-12-01T01:09:36.672 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-12-01T01:09:36.672 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-12-01T01:09:36.672 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-12-01T01:09:36.672 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-12-01T01:09:36.672 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-12-01T01:09:36.672 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-12-01T01:09:36.672 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-12-01T01:09:36.672 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.table_properties_collectors: 2023-12-01T01:09:36.672 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.inplace_update_support: 0 2023-12-01T01:09:36.672 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.inplace_update_num_locks: 10000 2023-12-01T01:09:36.672 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-12-01T01:09:36.672 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-12-01T01:09:36.672 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.memtable_huge_page_size: 0 2023-12-01T01:09:36.672 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.bloom_locality: 0 2023-12-01T01:09:36.673 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.max_successive_merges: 0 2023-12-01T01:09:36.673 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.optimize_filters_for_hits: 0 2023-12-01T01:09:36.673 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.paranoid_file_checks: 0 2023-12-01T01:09:36.673 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.force_consistency_checks: 1 2023-12-01T01:09:36.673 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.report_bg_io_stats: 0 2023-12-01T01:09:36.673 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.ttl: 2592000 2023-12-01T01:09:36.673 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.periodic_compaction_seconds: 0 2023-12-01T01:09:36.673 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-12-01T01:09:36.673 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-12-01T01:09:36.673 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.enable_blob_files: false 2023-12-01T01:09:36.673 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.min_blob_size: 0 2023-12-01T01:09:36.673 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.blob_file_size: 268435456 2023-12-01T01:09:36.673 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.blob_compression_type: NoCompression 2023-12-01T01:09:36.673 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.enable_blob_garbage_collection: false 2023-12-01T01:09:36.673 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-12-01T01:09:36.674 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-12-01T01:09:36.674 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-12-01T01:09:36.674 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.blob_file_starting_level: 0 2023-12-01T01:09:36.674 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-12-01T01:09:36.674 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-12-01T01:09:36.674 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-12-01T01:09:36.674 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 54b39629-e90a-4621-919e-8105dcd7e6a4 2023-12-01T01:09:36.674 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: EVENT_LOG_v1 {"time_micros": 1701392976486123, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-12-01T01:09:36.674 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-12-01T01:09:36.674 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: EVENT_LOG_v1 {"time_micros": 1701392976487139, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 95147, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 327, "table_properties": {"data_size": 93138, "index_size": 291, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13896, "raw_average_key_size": 49, "raw_value_size": 85566, "raw_average_value_size": 307, "num_data_blocks": 11, "num_entries": 278, "num_filter_entries": 278, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1701392976, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "54b39629-e90a-4621-919e-8105dcd7e6a4", "db_session_id": "HGUFLH0XVL4I1X5ZEK65", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-12-01T01:09:36.674 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: EVENT_LOG_v1 {"time_micros": 1701392976487211, "job": 1, "event": "recovery_finished"} 2023-12-01T01:09:36.674 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-12-01T01:09:36.674 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-12-01T01:09:36.675 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55794593a000 2023-12-01T01:09:36.675 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: DB pointer 0x557945926000 2023-12-01T01:09:36.675 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-12-01T01:09:36.675 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-12-01T01:09:36.675 INFO:journalctl@ceph.mon.a.smithi077.stdout: ** DB Stats ** 2023-12-01T01:09:36.675 INFO:journalctl@ceph.mon.a.smithi077.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-01T01:09:36.675 INFO:journalctl@ceph.mon.a.smithi077.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-12-01T01:09:36.675 INFO:journalctl@ceph.mon.a.smithi077.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-01T01:09:36.675 INFO:journalctl@ceph.mon.a.smithi077.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-01T01:09:36.675 INFO:journalctl@ceph.mon.a.smithi077.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-12-01T01:09:36.675 INFO:journalctl@ceph.mon.a.smithi077.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-01T01:09:36.675 INFO:journalctl@ceph.mon.a.smithi077.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-01T01:09:36.675 INFO:journalctl@ceph.mon.a.smithi077.stdout: 2023-12-01T01:09:36.675 INFO:journalctl@ceph.mon.a.smithi077.stdout: ** Compaction Stats [default] ** 2023-12-01T01:09:36.676 INFO:journalctl@ceph.mon.a.smithi077.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-01T01:09:36.676 INFO:journalctl@ceph.mon.a.smithi077.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-12-01T01:09:36.676 INFO:journalctl@ceph.mon.a.smithi077.stdout: L0 2/0 94.75 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 139.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-01T01:09:36.676 INFO:journalctl@ceph.mon.a.smithi077.stdout: Sum 2/0 94.75 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 139.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-01T01:09:36.676 INFO:journalctl@ceph.mon.a.smithi077.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 139.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-01T01:09:36.676 INFO:journalctl@ceph.mon.a.smithi077.stdout: 2023-12-01T01:09:36.676 INFO:journalctl@ceph.mon.a.smithi077.stdout: ** Compaction Stats [default] ** 2023-12-01T01:09:36.676 INFO:journalctl@ceph.mon.a.smithi077.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-01T01:09:36.676 INFO:journalctl@ceph.mon.a.smithi077.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-12-01T01:09:36.676 INFO:journalctl@ceph.mon.a.smithi077.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 139.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-01T01:09:36.676 INFO:journalctl@ceph.mon.a.smithi077.stdout: 2023-12-01T01:09:36.676 INFO:journalctl@ceph.mon.a.smithi077.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-12-01T01:09:36.676 INFO:journalctl@ceph.mon.a.smithi077.stdout: 2023-12-01T01:09:36.676 INFO:journalctl@ceph.mon.a.smithi077.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-01T01:09:36.676 INFO:journalctl@ceph.mon.a.smithi077.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-12-01T01:09:36.677 INFO:journalctl@ceph.mon.a.smithi077.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-12-01T01:09:36.677 INFO:journalctl@ceph.mon.a.smithi077.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-12-01T01:09:36.677 INFO:journalctl@ceph.mon.a.smithi077.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-12-01T01:09:36.677 INFO:journalctl@ceph.mon.a.smithi077.stdout: AddFile(Keys): cumulative 0, interval 0 2023-12-01T01:09:36.677 INFO:journalctl@ceph.mon.a.smithi077.stdout: Cumulative compaction: 0.00 GB write, 30.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-01T01:09:36.677 INFO:journalctl@ceph.mon.a.smithi077.stdout: Interval compaction: 0.00 GB write, 30.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-01T01:09:36.677 INFO:journalctl@ceph.mon.a.smithi077.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-12-01T01:09:36.677 INFO:journalctl@ceph.mon.a.smithi077.stdout: Block cache BinnedLRUCache@0x55794589d350#7 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2023-12-01T01:09:36.677 INFO:journalctl@ceph.mon.a.smithi077.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2023-12-01T01:09:36.677 INFO:journalctl@ceph.mon.a.smithi077.stdout: 2023-12-01T01:09:36.677 INFO:journalctl@ceph.mon.a.smithi077.stdout: ** File Read Latency Histogram By Level [default] ** 2023-12-01T01:09:36.677 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: starting mon.a rank 0 at public addrs [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] at bind addrs [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 2023-12-01T01:09:36.677 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: mon.a@-1(???) e1 preinit fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 2023-12-01T01:09:36.677 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: mon.a@-1(???).mds e1 new map 2023-12-01T01:09:36.677 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: mon.a@-1(???).mds e1 print_map 2023-12-01T01:09:36.678 INFO:journalctl@ceph.mon.a.smithi077.stdout: e1 2023-12-01T01:09:36.678 INFO:journalctl@ceph.mon.a.smithi077.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-12-01T01:09:36.678 INFO:journalctl@ceph.mon.a.smithi077.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-12-01T01:09:36.678 INFO:journalctl@ceph.mon.a.smithi077.stdout: legacy client fscid: -1 2023-12-01T01:09:36.678 INFO:journalctl@ceph.mon.a.smithi077.stdout: 2023-12-01T01:09:36.678 INFO:journalctl@ceph.mon.a.smithi077.stdout: No filesystems configured 2023-12-01T01:09:36.678 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-12-01T01:09:36.678 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-12-01T01:09:36.678 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-12-01T01:09:36.678 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-12-01T01:09:36.678 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-12-01T01:09:36.678 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: expand_channel_meta expand map: {default=false} 2023-12-01T01:09:36.678 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: expand_channel_meta from 'false' to 'false' 2023-12-01T01:09:36.678 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: expand_channel_meta expanded map: {default=false} 2023-12-01T01:09:36.679 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: expand_channel_meta expand map: {default=info} 2023-12-01T01:09:36.679 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: expand_channel_meta from 'info' to 'info' 2023-12-01T01:09:36.679 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: expand_channel_meta expanded map: {default=info} 2023-12-01T01:09:36.679 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: expand_channel_meta expand map: {default=daemon} 2023-12-01T01:09:36.679 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: expand_channel_meta from 'daemon' to 'daemon' 2023-12-01T01:09:36.679 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: expand_channel_meta expanded map: {default=daemon} 2023-12-01T01:09:36.679 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: expand_channel_meta expand map: {default=debug} 2023-12-01T01:09:36.679 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: expand_channel_meta from 'debug' to 'debug' 2023-12-01T01:09:36.679 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: expand_channel_meta expanded map: {default=debug} 2023-12-01T01:09:36.679 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 systemd[1]: Started Ceph mon.a for 2a06e29a-8fe6-11ee-95a2-87774f69a715. 2023-12-01T01:09:36.679 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: mon.a is new leader, mons a in quorum (ranks 0) 2023-12-01T01:09:36.679 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: monmap e1: 1 mons at {a=[v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-01T01:09:36.680 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: fsmap 2023-12-01T01:09:36.680 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: osdmap e1: 0 total, 0 up, 0 in 2023-12-01T01:09:36.680 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:36 smithi077 ceph-mon[98867]: mgrmap e1: no daemons active 2023-12-01T01:09:37.386 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.009+0000 7fd395ad1700 1 Processor -- start 2023-12-01T01:09:37.387 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.010+0000 7fd395ad1700 1 -- start start 2023-12-01T01:09:37.387 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.010+0000 7fd395ad1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3901055c0 0x7fd3901059e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:09:37.387 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.010+0000 7fd395ad1700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd390105fb0 con 0x7fd3901055c0 2023-12-01T01:09:37.387 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.010+0000 7fd38f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3901055c0 0x7fd3901059e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:09:37.387 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.010+0000 7fd38f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3901055c0 0x7fd3901059e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56146/0 (socket says 172.21.15.77:56146) 2023-12-01T01:09:37.387 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.010+0000 7fd38f7fe700 1 -- 172.21.15.77:0/2144626874 learned_addr learned my addr 172.21.15.77:0/2144626874 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:09:37.387 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.011+0000 7fd38f7fe700 1 -- 172.21.15.77:0/2144626874 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3901067c0 con 0x7fd3901055c0 2023-12-01T01:09:37.387 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.011+0000 7fd38f7fe700 1 --2- 172.21.15.77:0/2144626874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3901055c0 0x7fd3901059e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd3780098a0 tx=0x7fd378009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc55436ab249731 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:09:37.387 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.011+0000 7fd38e7fc700 1 -- 172.21.15.77:0/2144626874 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd378004160 con 0x7fd3901055c0 2023-12-01T01:09:37.387 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.011+0000 7fd38e7fc700 1 -- 172.21.15.77:0/2144626874 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd3780042c0 con 0x7fd3901055c0 2023-12-01T01:09:37.387 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.011+0000 7fd38e7fc700 1 -- 172.21.15.77:0/2144626874 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd378011a00 con 0x7fd3901055c0 2023-12-01T01:09:37.387 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.011+0000 7fd395ad1700 1 -- 172.21.15.77:0/2144626874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3901055c0 msgr2=0x7fd3901059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:09:37.387 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.011+0000 7fd395ad1700 1 --2- 172.21.15.77:0/2144626874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3901055c0 0x7fd3901059e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd3780098a0 tx=0x7fd378009bb0 comp rx=0 tx=0).stop 2023-12-01T01:09:37.388 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.011+0000 7fd395ad1700 1 -- 172.21.15.77:0/2144626874 shutdown_connections 2023-12-01T01:09:37.388 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.011+0000 7fd395ad1700 1 --2- 172.21.15.77:0/2144626874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3901055c0 0x7fd3901059e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:09:37.388 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.011+0000 7fd395ad1700 1 -- 172.21.15.77:0/2144626874 >> 172.21.15.77:0/2144626874 conn(0x7fd390100b80 msgr2=0x7fd390102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:09:37.388 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.012+0000 7fd395ad1700 1 -- 172.21.15.77:0/2144626874 shutdown_connections 2023-12-01T01:09:37.388 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.012+0000 7fd395ad1700 1 -- 172.21.15.77:0/2144626874 wait complete. 2023-12-01T01:09:37.388 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.012+0000 7fd395ad1700 1 Processor -- start 2023-12-01T01:09:37.388 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.013+0000 7fd395ad1700 1 -- start start 2023-12-01T01:09:37.388 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.013+0000 7fd395ad1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3901055c0 0x7fd390199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:09:37.388 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.013+0000 7fd395ad1700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3901997c0 con 0x7fd3901055c0 2023-12-01T01:09:37.388 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.013+0000 7fd38f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3901055c0 0x7fd390199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:09:37.388 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.013+0000 7fd38f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3901055c0 0x7fd390199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56148/0 (socket says 172.21.15.77:56148) 2023-12-01T01:09:37.388 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.013+0000 7fd38f7fe700 1 -- 172.21.15.77:0/4027265968 learned_addr learned my addr 172.21.15.77:0/4027265968 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:09:37.388 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.013+0000 7fd38f7fe700 1 -- 172.21.15.77:0/4027265968 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd378009550 con 0x7fd3901055c0 2023-12-01T01:09:37.388 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.013+0000 7fd38f7fe700 1 --2- 172.21.15.77:0/4027265968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3901055c0 0x7fd390199280 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd378006af0 tx=0x7fd378011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:09:37.389 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.013+0000 7fd38cff9700 1 -- 172.21.15.77:0/4027265968 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd378003a40 con 0x7fd3901055c0 2023-12-01T01:09:37.389 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.013+0000 7fd38cff9700 1 -- 172.21.15.77:0/4027265968 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd378003ba0 con 0x7fd3901055c0 2023-12-01T01:09:37.389 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.013+0000 7fd38cff9700 1 -- 172.21.15.77:0/4027265968 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd37801a700 con 0x7fd3901055c0 2023-12-01T01:09:37.389 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.013+0000 7fd395ad1700 1 -- 172.21.15.77:0/4027265968 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3901999c0 con 0x7fd3901055c0 2023-12-01T01:09:37.389 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.014+0000 7fd395ad1700 1 -- 172.21.15.77:0/4027265968 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd390199de0 con 0x7fd3901055c0 2023-12-01T01:09:37.389 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.014+0000 7fd38cff9700 1 -- 172.21.15.77:0/4027265968 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd37801ac30 con 0x7fd3901055c0 2023-12-01T01:09:37.389 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.014+0000 7fd38cff9700 1 -- 172.21.15.77:0/4027265968 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd378019700 con 0x7fd3901055c0 2023-12-01T01:09:37.389 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.014+0000 7fd395ad1700 1 -- 172.21.15.77:0/4027265968 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3900623c0 con 0x7fd3901055c0 2023-12-01T01:09:37.389 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.017+0000 7fd38cff9700 1 -- 172.21.15.77:0/4027265968 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7fd37802de40 con 0x7fd3901055c0 2023-12-01T01:09:37.389 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.084+0000 7fd395ad1700 1 -- 172.21.15.77:0/4027265968 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fd3901099c0 con 0x7fd3901055c0 2023-12-01T01:09:37.389 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.086+0000 7fd38cff9700 1 -- 172.21.15.77:0/4027265968 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fd378024ad0 con 0x7fd3901055c0 2023-12-01T01:09:37.389 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.087+0000 7fd395ad1700 1 -- 172.21.15.77:0/4027265968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3901055c0 msgr2=0x7fd390199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:09:37.389 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.087+0000 7fd395ad1700 1 --2- 172.21.15.77:0/4027265968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3901055c0 0x7fd390199280 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd378006af0 tx=0x7fd378011ad0 comp rx=0 tx=0).stop 2023-12-01T01:09:37.389 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.087+0000 7fd395ad1700 1 -- 172.21.15.77:0/4027265968 shutdown_connections 2023-12-01T01:09:37.389 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.087+0000 7fd395ad1700 1 --2- 172.21.15.77:0/4027265968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3901055c0 0x7fd390199280 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:09:37.390 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.087+0000 7fd395ad1700 1 -- 172.21.15.77:0/4027265968 >> 172.21.15.77:0/4027265968 conn(0x7fd390100b80 msgr2=0x7fd390102270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:09:37.390 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.087+0000 7fd395ad1700 1 -- 172.21.15.77:0/4027265968 shutdown_connections 2023-12-01T01:09:37.390 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:37.087+0000 7fd395ad1700 1 -- 172.21.15.77:0/4027265968 wait complete. 2023-12-01T01:09:37.390 INFO:teuthology.orchestra.run.smithi077.stdout:Wrote config to /etc/ceph/ceph.conf 2023-12-01T01:09:37.426 INFO:teuthology.orchestra.run.smithi077.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-12-01T01:09:37.426 INFO:teuthology.orchestra.run.smithi077.stdout:Creating mgr... 2023-12-01T01:09:37.426 INFO:teuthology.orchestra.run.smithi077.stdout:Verifying port 0.0.0.0:9283 ... 2023-12-01T01:09:37.427 INFO:teuthology.orchestra.run.smithi077.stdout:Verifying port 0.0.0.0:8765 ... 2023-12-01T01:09:37.628 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mgr.x 2023-12-01T01:09:37.628 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mgr.x.service: Unit ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mgr.x.service not loaded. 2023-12-01T01:09:37.716 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715.target.wants/ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@.service. 2023-12-01T01:09:38.012 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:37 smithi077 systemd[1]: Starting Ceph mgr.x for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:09:38.309 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:38 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/4027265968' entity='client.admin' 2023-12-01T01:09:38.309 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:38 smithi077 podman[99101]: 2023-12-01T01:09:38.309 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:38 smithi077 bash[99101]: c7e874ec0c681ebae83ea368249ecbb8f1176e73b3be346a96f94312ec32c61e 2023-12-01T01:09:38.320 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-12-01T01:09:38.321 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout disabled 2023-12-01T01:09:38.328 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-12-01T01:09:38.328 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout inactive 2023-12-01T01:09:38.328 INFO:teuthology.orchestra.run.smithi077.stdout:firewalld.service is not enabled 2023-12-01T01:09:38.328 INFO:teuthology.orchestra.run.smithi077.stdout:Not possible to enable service . firewalld.service is not available 2023-12-01T01:09:38.336 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-12-01T01:09:38.336 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout disabled 2023-12-01T01:09:38.345 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-12-01T01:09:38.345 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout inactive 2023-12-01T01:09:38.345 INFO:teuthology.orchestra.run.smithi077.stdout:firewalld.service is not enabled 2023-12-01T01:09:38.345 INFO:teuthology.orchestra.run.smithi077.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-12-01T01:09:38.345 INFO:teuthology.orchestra.run.smithi077.stdout:Waiting for mgr to start... 2023-12-01T01:09:38.345 INFO:teuthology.orchestra.run.smithi077.stdout:Waiting for mgr... 2023-12-01T01:09:38.581 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:38 smithi077 systemd[1]: Started Ceph mgr.x for 2a06e29a-8fe6-11ee-95a2-87774f69a715. 2023-12-01T01:09:38.581 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:38 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:38.360+0000 7fdb5293f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T01:09:38.581 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:38 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:38.442+0000 7fdb5293f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T01:09:38.581 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:38 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:38.533+0000 7fdb5293f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T01:09:38.904 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:38 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:38.633+0000 7fdb5293f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T01:09:38.904 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:38 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:38.809+0000 7fdb5293f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T01:09:39.290 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:39 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:39.195+0000 7fdb5293f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T01:09:39.291 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:39 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/4047591579' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-12-01T01:09:39.292 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2023-12-01T01:09:39.295 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout { 2023-12-01T01:09:39.295 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "fsid": "2a06e29a-8fe6-11ee-95a2-87774f69a715", 2023-12-01T01:09:39.295 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "health": { 2023-12-01T01:09:39.296 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-12-01T01:09:39.296 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-12-01T01:09:39.296 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-12-01T01:09:39.296 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2023-12-01T01:09:39.296 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-12-01T01:09:39.296 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-12-01T01:09:39.296 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 0 2023-12-01T01:09:39.296 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2023-12-01T01:09:39.296 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-12-01T01:09:39.297 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "a" 2023-12-01T01:09:39.297 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2023-12-01T01:09:39.297 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2023-12-01T01:09:39.297 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "monmap": { 2023-12-01T01:09:39.297 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-01T01:09:39.297 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-12-01T01:09:39.297 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-12-01T01:09:39.297 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2023-12-01T01:09:39.298 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-12-01T01:09:39.298 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-01T01:09:39.298 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-12-01T01:09:39.298 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-12-01T01:09:39.298 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-12-01T01:09:39.305 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-12-01T01:09:39.305 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-12-01T01:09:39.306 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-12-01T01:09:39.306 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2023-12-01T01:09:39.306 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-12-01T01:09:39.306 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-12-01T01:09:39.306 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-12-01T01:09:39.306 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-12-01T01:09:39.306 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-12-01T01:09:39.306 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-12-01T01:09:39.306 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-12-01T01:09:39.306 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-12-01T01:09:39.306 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-12-01T01:09:39.306 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2023-12-01T01:09:39.306 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-12-01T01:09:39.306 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-01T01:09:39.307 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-12-01T01:09:39.307 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-12-01T01:09:39.307 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2023-12-01T01:09:39.307 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-12-01T01:09:39.307 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "available": false, 2023-12-01T01:09:39.307 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-12-01T01:09:39.307 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "modules": [ 2023-12-01T01:09:39.307 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "iostat", 2023-12-01T01:09:39.307 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "nfs", 2023-12-01T01:09:39.307 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "restful" 2023-12-01T01:09:39.307 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2023-12-01T01:09:39.307 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-01T01:09:39.308 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2023-12-01T01:09:39.308 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-12-01T01:09:39.308 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-01T01:09:39.308 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "modified": "2023-12-01T01:09:30.211018+0000", 2023-12-01T01:09:39.308 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-01T01:09:39.308 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2023-12-01T01:09:39.308 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-12-01T01:09:39.308 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout } 2023-12-01T01:09:39.308 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.862+0000 7f054b909700 1 Processor -- start 2023-12-01T01:09:39.308 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.863+0000 7f054b909700 1 -- start start 2023-12-01T01:09:39.308 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.863+0000 7f054b909700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0544107190 0x7f0544109580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:09:39.308 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.863+0000 7f054b909700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05440745a0 con 0x7f0544107190 2023-12-01T01:09:39.308 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.863+0000 7f05496a5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0544107190 0x7f0544109580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:09:39.308 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.863+0000 7f05496a5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0544107190 0x7f0544109580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56154/0 (socket says 172.21.15.77:56154) 2023-12-01T01:09:39.309 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.863+0000 7f05496a5700 1 -- 172.21.15.77:0/314336786 learned_addr learned my addr 172.21.15.77:0/314336786 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:09:39.309 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.863+0000 7f05496a5700 1 -- 172.21.15.77:0/314336786 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0544109ac0 con 0x7f0544107190 2023-12-01T01:09:39.309 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.864+0000 7f05496a5700 1 --2- 172.21.15.77:0/314336786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0544107190 0x7f0544109580 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f05400098a0 tx=0x7f0540009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7fe0ec45101ee49 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:09:39.309 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.864+0000 7f053bfff700 1 -- 172.21.15.77:0/314336786 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0540004160 con 0x7f0544107190 2023-12-01T01:09:39.309 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.864+0000 7f053bfff700 1 -- 172.21.15.77:0/314336786 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f05400042c0 con 0x7f0544107190 2023-12-01T01:09:39.309 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.864+0000 7f053bfff700 1 -- 172.21.15.77:0/314336786 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0540011a00 con 0x7f0544107190 2023-12-01T01:09:39.309 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.864+0000 7f054b909700 1 -- 172.21.15.77:0/314336786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0544107190 msgr2=0x7f0544109580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:09:39.309 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.864+0000 7f054b909700 1 --2- 172.21.15.77:0/314336786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0544107190 0x7f0544109580 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f05400098a0 tx=0x7f0540009bb0 comp rx=0 tx=0).stop 2023-12-01T01:09:39.309 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.864+0000 7f054b909700 1 -- 172.21.15.77:0/314336786 shutdown_connections 2023-12-01T01:09:39.309 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.864+0000 7f054b909700 1 --2- 172.21.15.77:0/314336786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0544107190 0x7f0544109580 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:09:39.309 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.864+0000 7f054b909700 1 -- 172.21.15.77:0/314336786 >> 172.21.15.77:0/314336786 conn(0x7f0544100b80 msgr2=0x7f0544102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:09:39.309 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.865+0000 7f054b909700 1 -- 172.21.15.77:0/314336786 shutdown_connections 2023-12-01T01:09:39.309 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.865+0000 7f054b909700 1 -- 172.21.15.77:0/314336786 wait complete. 2023-12-01T01:09:39.309 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.865+0000 7f054b909700 1 Processor -- start 2023-12-01T01:09:39.309 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.865+0000 7f054b909700 1 -- start start 2023-12-01T01:09:39.310 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.865+0000 7f054b909700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0544107190 0x7f0544106af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:09:39.310 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.865+0000 7f054b909700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0544107030 con 0x7f0544107190 2023-12-01T01:09:39.310 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.865+0000 7f05496a5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0544107190 0x7f0544106af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:09:39.310 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.866+0000 7f05496a5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0544107190 0x7f0544106af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56156/0 (socket says 172.21.15.77:56156) 2023-12-01T01:09:39.310 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.866+0000 7f05496a5700 1 -- 172.21.15.77:0/4047591579 learned_addr learned my addr 172.21.15.77:0/4047591579 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:09:39.310 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.866+0000 7f05496a5700 1 -- 172.21.15.77:0/4047591579 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0540009550 con 0x7f0544107190 2023-12-01T01:09:39.310 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.866+0000 7f05496a5700 1 --2- 172.21.15.77:0/4047591579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0544107190 0x7f0544106af0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0540009fb0 tx=0x7f0540011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:09:39.310 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.866+0000 7f053a7fc700 1 -- 172.21.15.77:0/4047591579 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0540003930 con 0x7f0544107190 2023-12-01T01:09:39.310 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.866+0000 7f054b909700 1 -- 172.21.15.77:0/4047591579 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0544105140 con 0x7f0544107190 2023-12-01T01:09:39.310 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.866+0000 7f053a7fc700 1 -- 172.21.15.77:0/4047591579 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0540003a90 con 0x7f0544107190 2023-12-01T01:09:39.310 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.866+0000 7f053a7fc700 1 -- 172.21.15.77:0/4047591579 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0540019610 con 0x7f0544107190 2023-12-01T01:09:39.310 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.866+0000 7f054b909700 1 -- 172.21.15.77:0/4047591579 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0544105560 con 0x7f0544107190 2023-12-01T01:09:39.310 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.867+0000 7f053a7fc700 1 -- 172.21.15.77:0/4047591579 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f0540019bf0 con 0x7f0544107190 2023-12-01T01:09:39.311 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.867+0000 7f053a7fc700 1 -- 172.21.15.77:0/4047591579 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f05400243f0 con 0x7f0544107190 2023-12-01T01:09:39.311 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.867+0000 7f054b909700 1 -- 172.21.15.77:0/4047591579 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f054410a860 con 0x7f0544107190 2023-12-01T01:09:39.311 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.869+0000 7f053a7fc700 1 -- 172.21.15.77:0/4047591579 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f0540003c00 con 0x7f0544107190 2023-12-01T01:09:39.311 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.942+0000 7f054b909700 1 -- 172.21.15.77:0/4047591579 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f054410aa80 con 0x7f0544107190 2023-12-01T01:09:39.311 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.942+0000 7f053a7fc700 1 -- 172.21.15.77:0/4047591579 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f054001a9d0 con 0x7f0544107190 2023-12-01T01:09:39.311 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.944+0000 7f054b909700 1 -- 172.21.15.77:0/4047591579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0544107190 msgr2=0x7f0544106af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:09:39.311 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.944+0000 7f054b909700 1 --2- 172.21.15.77:0/4047591579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0544107190 0x7f0544106af0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0540009fb0 tx=0x7f0540011ad0 comp rx=0 tx=0).stop 2023-12-01T01:09:39.311 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.944+0000 7f054b909700 1 -- 172.21.15.77:0/4047591579 shutdown_connections 2023-12-01T01:09:39.311 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.944+0000 7f054b909700 1 --2- 172.21.15.77:0/4047591579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0544107190 0x7f0544106af0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:09:39.311 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.944+0000 7f054b909700 1 -- 172.21.15.77:0/4047591579 >> 172.21.15.77:0/4047591579 conn(0x7f0544100b80 msgr2=0x7f0544101950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:09:39.311 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.944+0000 7f054b909700 1 -- 172.21.15.77:0/4047591579 shutdown_connections 2023-12-01T01:09:39.311 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:38.944+0000 7f054b909700 1 -- 172.21.15.77:0/4047591579 wait complete. 2023-12-01T01:09:39.311 INFO:teuthology.orchestra.run.smithi077.stdout:mgr not available, waiting (1/15)... 2023-12-01T01:09:39.654 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:39 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:39.331+0000 7fdb5293f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T01:09:39.654 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:39 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:39.418+0000 7fdb5293f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T01:09:39.654 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:39 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:39.512+0000 7fdb5293f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T01:09:40.404 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:39 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:39.933+0000 7fdb5293f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T01:09:40.904 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:40 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:40.591+0000 7fdb5293f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T01:09:40.904 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:40 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:40.672+0000 7fdb5293f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T01:09:41.721 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:41 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:41.441+0000 7fdb5293f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:09:41.722 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:41 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:41.541+0000 7fdb5293f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T01:09:41.981 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:41 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:41.720+0000 7fdb5293f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:09:42.238 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:42 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:42.013+0000 7fdb5293f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T01:09:42.244 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2023-12-01T01:09:42.246 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout { 2023-12-01T01:09:42.246 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "fsid": "2a06e29a-8fe6-11ee-95a2-87774f69a715", 2023-12-01T01:09:42.246 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "health": { 2023-12-01T01:09:42.246 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-12-01T01:09:42.247 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-12-01T01:09:42.247 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-12-01T01:09:42.247 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2023-12-01T01:09:42.247 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-12-01T01:09:42.247 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-12-01T01:09:42.247 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 0 2023-12-01T01:09:42.247 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2023-12-01T01:09:42.247 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-12-01T01:09:42.247 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "a" 2023-12-01T01:09:42.247 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2023-12-01T01:09:42.247 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2023-12-01T01:09:42.247 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "monmap": { 2023-12-01T01:09:42.247 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-01T01:09:42.247 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-12-01T01:09:42.248 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-12-01T01:09:42.248 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2023-12-01T01:09:42.248 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-12-01T01:09:42.248 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-01T01:09:42.248 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-12-01T01:09:42.248 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-12-01T01:09:42.248 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-12-01T01:09:42.255 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-12-01T01:09:42.255 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-12-01T01:09:42.256 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-12-01T01:09:42.256 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2023-12-01T01:09:42.256 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-12-01T01:09:42.256 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-12-01T01:09:42.256 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-12-01T01:09:42.256 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-12-01T01:09:42.256 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-12-01T01:09:42.256 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-12-01T01:09:42.256 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-12-01T01:09:42.256 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-12-01T01:09:42.256 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-12-01T01:09:42.256 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2023-12-01T01:09:42.256 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-12-01T01:09:42.256 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-01T01:09:42.257 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-12-01T01:09:42.257 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-12-01T01:09:42.257 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2023-12-01T01:09:42.257 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-12-01T01:09:42.257 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "available": false, 2023-12-01T01:09:42.257 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-12-01T01:09:42.257 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "modules": [ 2023-12-01T01:09:42.257 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "iostat", 2023-12-01T01:09:42.257 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "nfs", 2023-12-01T01:09:42.257 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "restful" 2023-12-01T01:09:42.257 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2023-12-01T01:09:42.257 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-01T01:09:42.257 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2023-12-01T01:09:42.257 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-12-01T01:09:42.257 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-01T01:09:42.257 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "modified": "2023-12-01T01:09:30.211018+0000", 2023-12-01T01:09:42.258 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-01T01:09:42.258 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2023-12-01T01:09:42.258 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-12-01T01:09:42.258 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout } 2023-12-01T01:09:42.258 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.775+0000 7fc4cc328700 1 Processor -- start 2023-12-01T01:09:42.258 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.775+0000 7fc4cc328700 1 -- start start 2023-12-01T01:09:42.258 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.775+0000 7fc4cc328700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4c41055c0 0x7fc4c41059e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:09:42.258 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.775+0000 7fc4cc328700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c4105fb0 con 0x7fc4c41055c0 2023-12-01T01:09:42.258 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.775+0000 7fc4ca0c4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4c41055c0 0x7fc4c41059e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:09:42.258 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.775+0000 7fc4ca0c4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4c41055c0 0x7fc4c41059e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56158/0 (socket says 172.21.15.77:56158) 2023-12-01T01:09:42.258 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.775+0000 7fc4ca0c4700 1 -- 172.21.15.77:0/2505970470 learned_addr learned my addr 172.21.15.77:0/2505970470 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:09:42.258 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.776+0000 7fc4ca0c4700 1 -- 172.21.15.77:0/2505970470 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c41067c0 con 0x7fc4c41055c0 2023-12-01T01:09:42.258 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.776+0000 7fc4ca0c4700 1 --2- 172.21.15.77:0/2505970470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4c41055c0 0x7fc4c41059e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc4c00098a0 tx=0x7fc4c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1201db05a79d7436 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:09:42.258 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.776+0000 7fc4c90c2700 1 -- 172.21.15.77:0/2505970470 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4c0004160 con 0x7fc4c41055c0 2023-12-01T01:09:42.258 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.776+0000 7fc4c90c2700 1 -- 172.21.15.77:0/2505970470 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc4c00042c0 con 0x7fc4c41055c0 2023-12-01T01:09:42.259 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.776+0000 7fc4c90c2700 1 -- 172.21.15.77:0/2505970470 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4c0011a00 con 0x7fc4c41055c0 2023-12-01T01:09:42.259 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.777+0000 7fc4cc328700 1 -- 172.21.15.77:0/2505970470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4c41055c0 msgr2=0x7fc4c41059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:09:42.259 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.777+0000 7fc4cc328700 1 --2- 172.21.15.77:0/2505970470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4c41055c0 0x7fc4c41059e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc4c00098a0 tx=0x7fc4c0009bb0 comp rx=0 tx=0).stop 2023-12-01T01:09:42.259 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.777+0000 7fc4cc328700 1 -- 172.21.15.77:0/2505970470 shutdown_connections 2023-12-01T01:09:42.259 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.777+0000 7fc4cc328700 1 --2- 172.21.15.77:0/2505970470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4c41055c0 0x7fc4c41059e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:09:42.259 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.777+0000 7fc4cc328700 1 -- 172.21.15.77:0/2505970470 >> 172.21.15.77:0/2505970470 conn(0x7fc4c4100b80 msgr2=0x7fc4c4102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:09:42.259 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.777+0000 7fc4cc328700 1 -- 172.21.15.77:0/2505970470 shutdown_connections 2023-12-01T01:09:42.259 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.777+0000 7fc4cc328700 1 -- 172.21.15.77:0/2505970470 wait complete. 2023-12-01T01:09:42.259 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.778+0000 7fc4cc328700 1 Processor -- start 2023-12-01T01:09:42.259 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.778+0000 7fc4cc328700 1 -- start start 2023-12-01T01:09:42.259 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.778+0000 7fc4cc328700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4c41055c0 0x7fc4c4199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:09:42.259 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.778+0000 7fc4cc328700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c41997c0 con 0x7fc4c41055c0 2023-12-01T01:09:42.259 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.778+0000 7fc4ca0c4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4c41055c0 0x7fc4c4199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:09:42.259 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.778+0000 7fc4ca0c4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4c41055c0 0x7fc4c4199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56160/0 (socket says 172.21.15.77:56160) 2023-12-01T01:09:42.260 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.778+0000 7fc4ca0c4700 1 -- 172.21.15.77:0/1802889038 learned_addr learned my addr 172.21.15.77:0/1802889038 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:09:42.260 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.778+0000 7fc4ca0c4700 1 -- 172.21.15.77:0/1802889038 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c0009550 con 0x7fc4c41055c0 2023-12-01T01:09:42.260 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.778+0000 7fc4ca0c4700 1 --2- 172.21.15.77:0/1802889038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4c41055c0 0x7fc4c4199280 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0006af0 tx=0x7fc4c0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:09:42.260 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.779+0000 7fc4bb7fe700 1 -- 172.21.15.77:0/1802889038 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4c0003a40 con 0x7fc4c41055c0 2023-12-01T01:09:42.260 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.779+0000 7fc4bb7fe700 1 -- 172.21.15.77:0/1802889038 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc4c0003ba0 con 0x7fc4c41055c0 2023-12-01T01:09:42.260 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.779+0000 7fc4cc328700 1 -- 172.21.15.77:0/1802889038 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4c41999c0 con 0x7fc4c41055c0 2023-12-01T01:09:42.260 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.779+0000 7fc4bb7fe700 1 -- 172.21.15.77:0/1802889038 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4c001a700 con 0x7fc4c41055c0 2023-12-01T01:09:42.260 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.779+0000 7fc4cc328700 1 -- 172.21.15.77:0/1802889038 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c4199de0 con 0x7fc4c41055c0 2023-12-01T01:09:42.260 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.779+0000 7fc4bb7fe700 1 -- 172.21.15.77:0/1802889038 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc4c001ac30 con 0x7fc4c41055c0 2023-12-01T01:09:42.260 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.779+0000 7fc4bb7fe700 1 -- 172.21.15.77:0/1802889038 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc4c0019700 con 0x7fc4c41055c0 2023-12-01T01:09:42.260 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.780+0000 7fc4cc328700 1 -- 172.21.15.77:0/1802889038 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4c40623c0 con 0x7fc4c41055c0 2023-12-01T01:09:42.260 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.782+0000 7fc4bb7fe700 1 -- 172.21.15.77:0/1802889038 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7fc4c002de40 con 0x7fc4c41055c0 2023-12-01T01:09:42.260 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.854+0000 7fc4cc328700 1 -- 172.21.15.77:0/1802889038 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fc4c41099c0 con 0x7fc4c41055c0 2023-12-01T01:09:42.260 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.855+0000 7fc4bb7fe700 1 -- 172.21.15.77:0/1802889038 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fc4c0024ad0 con 0x7fc4c41055c0 2023-12-01T01:09:42.261 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.856+0000 7fc4cc328700 1 -- 172.21.15.77:0/1802889038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4c41055c0 msgr2=0x7fc4c4199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:09:42.261 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.856+0000 7fc4cc328700 1 --2- 172.21.15.77:0/1802889038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4c41055c0 0x7fc4c4199280 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0006af0 tx=0x7fc4c0011ad0 comp rx=0 tx=0).stop 2023-12-01T01:09:42.261 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.856+0000 7fc4cc328700 1 -- 172.21.15.77:0/1802889038 shutdown_connections 2023-12-01T01:09:42.261 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.856+0000 7fc4cc328700 1 --2- 172.21.15.77:0/1802889038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4c41055c0 0x7fc4c4199280 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:09:42.261 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.856+0000 7fc4cc328700 1 -- 172.21.15.77:0/1802889038 >> 172.21.15.77:0/1802889038 conn(0x7fc4c4100b80 msgr2=0x7fc4c4102270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:09:42.261 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.856+0000 7fc4cc328700 1 -- 172.21.15.77:0/1802889038 shutdown_connections 2023-12-01T01:09:42.261 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:41.856+0000 7fc4cc328700 1 -- 172.21.15.77:0/1802889038 wait complete. 2023-12-01T01:09:42.261 INFO:teuthology.orchestra.run.smithi077.stdout:mgr not available, waiting (2/15)... 2023-12-01T01:09:42.494 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:42 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:42.236+0000 7fdb5293f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T01:09:42.494 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:42 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:42.386+0000 7fdb5293f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T01:09:42.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:42 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/1802889038' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-12-01T01:09:42.904 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:42 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:42.646+0000 7fdb5293f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T01:09:43.651 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:43 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:43.332+0000 7fdb5293f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T01:09:43.652 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:43 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:43.461+0000 7fdb5293f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T01:09:43.904 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:43 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:43.650+0000 7fdb5293f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T01:09:43.904 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:43 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:43.762+0000 7fdb5293f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T01:09:44.404 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:43 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:43.974+0000 7fdb5293f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T01:09:45.154 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:44 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:44.853+0000 7fdb5293f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T01:09:45.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:44 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/1764437806' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-12-01T01:09:45.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:44 smithi077 ceph-mon[98867]: Activating manager daemon x 2023-12-01T01:09:45.698 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2023-12-01T01:09:45.698 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout { 2023-12-01T01:09:45.699 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "fsid": "2a06e29a-8fe6-11ee-95a2-87774f69a715", 2023-12-01T01:09:45.699 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "health": { 2023-12-01T01:09:45.699 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-12-01T01:09:45.699 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-12-01T01:09:45.699 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-12-01T01:09:45.699 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2023-12-01T01:09:45.699 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-12-01T01:09:45.699 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-12-01T01:09:45.699 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 0 2023-12-01T01:09:45.699 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2023-12-01T01:09:45.699 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-12-01T01:09:45.699 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "a" 2023-12-01T01:09:45.699 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2023-12-01T01:09:45.700 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2023-12-01T01:09:45.700 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "monmap": { 2023-12-01T01:09:45.700 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-01T01:09:45.700 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-12-01T01:09:45.700 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-12-01T01:09:45.700 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2023-12-01T01:09:45.700 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-12-01T01:09:45.700 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-01T01:09:45.700 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-12-01T01:09:45.700 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-12-01T01:09:45.700 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-12-01T01:09:45.709 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-12-01T01:09:45.709 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-12-01T01:09:45.709 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-12-01T01:09:45.709 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2023-12-01T01:09:45.709 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-12-01T01:09:45.710 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-12-01T01:09:45.710 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-12-01T01:09:45.710 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-12-01T01:09:45.710 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-12-01T01:09:45.710 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-12-01T01:09:45.710 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-12-01T01:09:45.710 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-12-01T01:09:45.710 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-12-01T01:09:45.710 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2023-12-01T01:09:45.710 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-12-01T01:09:45.710 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-01T01:09:45.710 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-12-01T01:09:45.710 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-12-01T01:09:45.711 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2023-12-01T01:09:45.711 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-12-01T01:09:45.711 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "available": false, 2023-12-01T01:09:45.711 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-12-01T01:09:45.711 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "modules": [ 2023-12-01T01:09:45.711 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "iostat", 2023-12-01T01:09:45.711 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "nfs", 2023-12-01T01:09:45.711 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "restful" 2023-12-01T01:09:45.711 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2023-12-01T01:09:45.711 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-01T01:09:45.711 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2023-12-01T01:09:45.711 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-12-01T01:09:45.711 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-01T01:09:45.711 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "modified": "2023-12-01T01:09:30.211018+0000", 2023-12-01T01:09:45.712 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-01T01:09:45.712 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2023-12-01T01:09:45.712 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-12-01T01:09:45.712 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout } 2023-12-01T01:09:45.712 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.740+0000 7fc8c579e700 1 Processor -- start 2023-12-01T01:09:45.712 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.740+0000 7fc8c579e700 1 -- start start 2023-12-01T01:09:45.712 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.740+0000 7fc8c579e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8c01058b0 0x7fc8c0105cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:09:45.712 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.740+0000 7fc8c579e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c01062a0 con 0x7fc8c01058b0 2023-12-01T01:09:45.712 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.741+0000 7fc8bffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8c01058b0 0x7fc8c0105cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:09:45.712 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.741+0000 7fc8bffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8c01058b0 0x7fc8c0105cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56162/0 (socket says 172.21.15.77:56162) 2023-12-01T01:09:45.712 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.741+0000 7fc8bffff700 1 -- 172.21.15.77:0/166896714 learned_addr learned my addr 172.21.15.77:0/166896714 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:09:45.712 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.741+0000 7fc8bffff700 1 -- 172.21.15.77:0/166896714 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c01065d0 con 0x7fc8c01058b0 2023-12-01T01:09:45.712 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.741+0000 7fc8bffff700 1 --2- 172.21.15.77:0/166896714 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8c01058b0 0x7fc8c0105cd0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc8b00098a0 tx=0x7fc8b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd1c59962c3163f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:09:45.713 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.741+0000 7fc8beffd700 1 -- 172.21.15.77:0/166896714 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8b0004160 con 0x7fc8c01058b0 2023-12-01T01:09:45.713 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.741+0000 7fc8beffd700 1 -- 172.21.15.77:0/166896714 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc8b00042c0 con 0x7fc8c01058b0 2023-12-01T01:09:45.713 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.741+0000 7fc8beffd700 1 -- 172.21.15.77:0/166896714 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8b0011a00 con 0x7fc8c01058b0 2023-12-01T01:09:45.713 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.742+0000 7fc8c579e700 1 -- 172.21.15.77:0/166896714 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8c01058b0 msgr2=0x7fc8c0105cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:09:45.713 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.742+0000 7fc8c579e700 1 --2- 172.21.15.77:0/166896714 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8c01058b0 0x7fc8c0105cd0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc8b00098a0 tx=0x7fc8b0009bb0 comp rx=0 tx=0).stop 2023-12-01T01:09:45.713 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.742+0000 7fc8c579e700 1 -- 172.21.15.77:0/166896714 shutdown_connections 2023-12-01T01:09:45.713 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.742+0000 7fc8c579e700 1 --2- 172.21.15.77:0/166896714 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8c01058b0 0x7fc8c0105cd0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:09:45.713 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.742+0000 7fc8c579e700 1 -- 172.21.15.77:0/166896714 >> 172.21.15.77:0/166896714 conn(0x7fc8c0100110 msgr2=0x7fc8c0102530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:09:45.713 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.742+0000 7fc8c579e700 1 -- 172.21.15.77:0/166896714 shutdown_connections 2023-12-01T01:09:45.713 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.742+0000 7fc8c579e700 1 -- 172.21.15.77:0/166896714 wait complete. 2023-12-01T01:09:45.713 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.743+0000 7fc8c579e700 1 Processor -- start 2023-12-01T01:09:45.713 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.743+0000 7fc8c579e700 1 -- start start 2023-12-01T01:09:45.713 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.743+0000 7fc8c579e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8c01058b0 0x7fc8c019a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:09:45.714 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.743+0000 7fc8c579e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c019a690 con 0x7fc8c01058b0 2023-12-01T01:09:45.714 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.743+0000 7fc8bffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8c01058b0 0x7fc8c019a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:09:45.714 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.743+0000 7fc8bffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8c01058b0 0x7fc8c019a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56164/0 (socket says 172.21.15.77:56164) 2023-12-01T01:09:45.714 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.743+0000 7fc8bffff700 1 -- 172.21.15.77:0/1764437806 learned_addr learned my addr 172.21.15.77:0/1764437806 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:09:45.714 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.743+0000 7fc8bffff700 1 -- 172.21.15.77:0/1764437806 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b0009550 con 0x7fc8c01058b0 2023-12-01T01:09:45.714 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.744+0000 7fc8bffff700 1 --2- 172.21.15.77:0/1764437806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8c01058b0 0x7fc8c019a150 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0009fb0 tx=0x7fc8b0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:09:45.714 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.744+0000 7fc8bd7fa700 1 -- 172.21.15.77:0/1764437806 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8b0003930 con 0x7fc8c01058b0 2023-12-01T01:09:45.714 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.744+0000 7fc8bd7fa700 1 -- 172.21.15.77:0/1764437806 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc8b0003a90 con 0x7fc8c01058b0 2023-12-01T01:09:45.714 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.744+0000 7fc8c579e700 1 -- 172.21.15.77:0/1764437806 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8c019a890 con 0x7fc8c01058b0 2023-12-01T01:09:45.714 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.744+0000 7fc8bd7fa700 1 -- 172.21.15.77:0/1764437806 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8b0019610 con 0x7fc8c01058b0 2023-12-01T01:09:45.714 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.744+0000 7fc8c579e700 1 -- 172.21.15.77:0/1764437806 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8c019acb0 con 0x7fc8c01058b0 2023-12-01T01:09:45.714 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.744+0000 7fc8bd7fa700 1 -- 172.21.15.77:0/1764437806 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc8b0019bf0 con 0x7fc8c01058b0 2023-12-01T01:09:45.714 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.744+0000 7fc8bd7fa700 1 -- 172.21.15.77:0/1764437806 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc8b00243f0 con 0x7fc8c01058b0 2023-12-01T01:09:45.714 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.745+0000 7fc8c579e700 1 -- 172.21.15.77:0/1764437806 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8c00623c0 con 0x7fc8c01058b0 2023-12-01T01:09:45.715 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.747+0000 7fc8bd7fa700 1 -- 172.21.15.77:0/1764437806 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7fc8b0003c00 con 0x7fc8c01058b0 2023-12-01T01:09:45.715 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.819+0000 7fc8c579e700 1 -- 172.21.15.77:0/1764437806 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fc8c01097d0 con 0x7fc8c01058b0 2023-12-01T01:09:45.715 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.819+0000 7fc8bd7fa700 1 -- 172.21.15.77:0/1764437806 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fc8b001a9d0 con 0x7fc8c01058b0 2023-12-01T01:09:45.715 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.820+0000 7fc8c579e700 1 -- 172.21.15.77:0/1764437806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8c01058b0 msgr2=0x7fc8c019a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:09:45.715 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.820+0000 7fc8c579e700 1 --2- 172.21.15.77:0/1764437806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8c01058b0 0x7fc8c019a150 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0009fb0 tx=0x7fc8b0011ad0 comp rx=0 tx=0).stop 2023-12-01T01:09:45.715 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.820+0000 7fc8c579e700 1 -- 172.21.15.77:0/1764437806 shutdown_connections 2023-12-01T01:09:45.715 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.820+0000 7fc8c579e700 1 --2- 172.21.15.77:0/1764437806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8c01058b0 0x7fc8c019a150 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:09:45.715 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.820+0000 7fc8c579e700 1 -- 172.21.15.77:0/1764437806 >> 172.21.15.77:0/1764437806 conn(0x7fc8c0100110 msgr2=0x7fc8c0104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:09:45.715 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.820+0000 7fc8c579e700 1 -- 172.21.15.77:0/1764437806 shutdown_connections 2023-12-01T01:09:45.715 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:44.820+0000 7fc8c579e700 1 -- 172.21.15.77:0/1764437806 wait complete. 2023-12-01T01:09:45.715 INFO:teuthology.orchestra.run.smithi077.stdout:mgr not available, waiting (3/15)... 2023-12-01T01:09:46.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:45 smithi077 ceph-mon[98867]: mgrmap e2: x(active, starting, since 0.00308366s) 2023-12-01T01:09:46.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:45 smithi077 ceph-mon[98867]: from='mgr.14100 172.21.15.77:0/1035126431' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-01T01:09:46.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:45 smithi077 ceph-mon[98867]: from='mgr.14100 172.21.15.77:0/1035126431' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-01T01:09:46.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:45 smithi077 ceph-mon[98867]: from='mgr.14100 172.21.15.77:0/1035126431' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-01T01:09:46.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:45 smithi077 ceph-mon[98867]: from='mgr.14100 172.21.15.77:0/1035126431' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-01T01:09:46.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:45 smithi077 ceph-mon[98867]: from='mgr.14100 172.21.15.77:0/1035126431' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-01T01:09:46.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:45 smithi077 ceph-mon[98867]: Manager daemon x is now available 2023-12-01T01:09:46.155 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:45 smithi077 ceph-mon[98867]: from='mgr.14100 172.21.15.77:0/1035126431' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-12-01T01:09:46.155 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:45 smithi077 ceph-mon[98867]: from='mgr.14100 172.21.15.77:0/1035126431' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-12-01T01:09:46.155 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:45 smithi077 ceph-mon[98867]: from='mgr.14100 172.21.15.77:0/1035126431' entity='mgr.x' 2023-12-01T01:09:46.155 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:45 smithi077 ceph-mon[98867]: from='mgr.14100 172.21.15.77:0/1035126431' entity='mgr.x' 2023-12-01T01:09:46.155 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:45 smithi077 ceph-mon[98867]: from='mgr.14100 172.21.15.77:0/1035126431' entity='mgr.x' 2023-12-01T01:09:47.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:46 smithi077 ceph-mon[98867]: mgrmap e3: x(active, since 1.01567s) 2023-12-01T01:09:48.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:48 smithi077 ceph-mon[98867]: mgrmap e4: x(active, since 2s) 2023-12-01T01:09:51.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:51 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/3371157903' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-12-01T01:09:53.775 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2023-12-01T01:09:53.775 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout { 2023-12-01T01:09:53.775 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "fsid": "2a06e29a-8fe6-11ee-95a2-87774f69a715", 2023-12-01T01:09:53.775 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "health": { 2023-12-01T01:09:53.775 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-12-01T01:09:53.775 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-12-01T01:09:53.775 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-12-01T01:09:53.775 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2023-12-01T01:09:53.775 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-12-01T01:09:53.775 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-12-01T01:09:53.776 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 0 2023-12-01T01:09:53.776 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2023-12-01T01:09:53.776 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-12-01T01:09:53.776 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "a" 2023-12-01T01:09:53.776 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2023-12-01T01:09:53.776 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2023-12-01T01:09:53.776 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "monmap": { 2023-12-01T01:09:53.776 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-01T01:09:53.776 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-12-01T01:09:53.776 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-12-01T01:09:53.776 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2023-12-01T01:09:53.776 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-12-01T01:09:53.776 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-01T01:09:53.776 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-12-01T01:09:53.776 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-12-01T01:09:53.777 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-12-01T01:09:53.777 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-12-01T01:09:53.777 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-12-01T01:09:53.777 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-12-01T01:09:53.777 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2023-12-01T01:09:53.777 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-12-01T01:09:53.777 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-12-01T01:09:53.777 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-12-01T01:09:53.777 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-12-01T01:09:53.777 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-12-01T01:09:53.777 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-12-01T01:09:53.777 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-12-01T01:09:53.777 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-12-01T01:09:53.777 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-12-01T01:09:53.778 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2023-12-01T01:09:53.778 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-12-01T01:09:53.778 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-01T01:09:53.778 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-12-01T01:09:53.778 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-12-01T01:09:53.778 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2023-12-01T01:09:53.778 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-12-01T01:09:53.778 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "available": true, 2023-12-01T01:09:53.778 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-12-01T01:09:53.778 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "modules": [ 2023-12-01T01:09:53.778 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "iostat", 2023-12-01T01:09:53.778 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "nfs", 2023-12-01T01:09:53.778 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "restful" 2023-12-01T01:09:53.778 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2023-12-01T01:09:53.779 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-01T01:09:53.779 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2023-12-01T01:09:53.779 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-12-01T01:09:53.779 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-01T01:09:53.779 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "modified": "2023-12-01T01:09:30.211018+0000", 2023-12-01T01:09:53.779 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-01T01:09:53.779 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2023-12-01T01:09:53.779 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-12-01T01:09:53.779 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout } 2023-12-01T01:09:53.779 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:50.755+0000 7f2e691e0700 1 Processor -- start 2023-12-01T01:09:53.779 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:50.755+0000 7f2e691e0700 1 -- start start 2023-12-01T01:09:53.779 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:50.756+0000 7f2e691e0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e641055c0 0x7f2e641059e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:09:53.779 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:50.756+0000 7f2e691e0700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e64105fb0 con 0x7f2e641055c0 2023-12-01T01:09:53.779 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:50.756+0000 7f2e62d9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e641055c0 0x7f2e641059e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:09:53.779 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:50.756+0000 7f2e62d9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e641055c0 0x7f2e641059e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56192/0 (socket says 172.21.15.77:56192) 2023-12-01T01:09:53.780 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:50.756+0000 7f2e62d9d700 1 -- 172.21.15.77:0/439217323 learned_addr learned my addr 172.21.15.77:0/439217323 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:09:53.780 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:50.756+0000 7f2e62d9d700 1 -- 172.21.15.77:0/439217323 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e641067c0 con 0x7f2e641055c0 2023-12-01T01:09:53.780 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:50.756+0000 7f2e62d9d700 1 --2- 172.21.15.77:0/439217323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e641055c0 0x7f2e641059e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2e580098a0 tx=0x7f2e58009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eccc6a7d0d98842a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:09:53.780 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:50.756+0000 7f2e61d9b700 1 -- 172.21.15.77:0/439217323 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2e58004160 con 0x7f2e641055c0 2023-12-01T01:09:53.780 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:50.756+0000 7f2e61d9b700 1 -- 172.21.15.77:0/439217323 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2e580042c0 con 0x7f2e641055c0 2023-12-01T01:09:53.780 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:50.756+0000 7f2e61d9b700 1 -- 172.21.15.77:0/439217323 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2e58011a00 con 0x7f2e641055c0 2023-12-01T01:09:53.780 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:50.757+0000 7f2e691e0700 1 -- 172.21.15.77:0/439217323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e641055c0 msgr2=0x7f2e641059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:09:53.780 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:50.757+0000 7f2e691e0700 1 --2- 172.21.15.77:0/439217323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e641055c0 0x7f2e641059e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2e580098a0 tx=0x7f2e58009bb0 comp rx=0 tx=0).stop 2023-12-01T01:09:53.780 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:50.757+0000 7f2e691e0700 1 -- 172.21.15.77:0/439217323 shutdown_connections 2023-12-01T01:09:53.780 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:50.757+0000 7f2e691e0700 1 --2- 172.21.15.77:0/439217323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e641055c0 0x7f2e641059e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:09:53.780 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:50.757+0000 7f2e691e0700 1 -- 172.21.15.77:0/439217323 >> 172.21.15.77:0/439217323 conn(0x7f2e64100b80 msgr2=0x7f2e64102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:09:53.780 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:50.757+0000 7f2e691e0700 1 -- 172.21.15.77:0/439217323 shutdown_connections 2023-12-01T01:09:53.780 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:50.757+0000 7f2e691e0700 1 -- 172.21.15.77:0/439217323 wait complete. 2023-12-01T01:09:53.780 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:50.758+0000 7f2e691e0700 1 Processor -- start 2023-12-01T01:09:53.781 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:50.758+0000 7f2e691e0700 1 -- start start 2023-12-01T01:09:53.781 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:50.758+0000 7f2e691e0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e641055c0 0x7f2e64199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:09:53.781 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:50.758+0000 7f2e691e0700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e641997c0 con 0x7f2e641055c0 2023-12-01T01:09:53.781 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:50.758+0000 7f2e62d9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e641055c0 0x7f2e64199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:09:53.781 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:50.758+0000 7f2e62d9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e641055c0 0x7f2e64199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56194/0 (socket says 172.21.15.77:56194) 2023-12-01T01:09:53.781 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:50.758+0000 7f2e62d9d700 1 -- 172.21.15.77:0/3371157903 learned_addr learned my addr 172.21.15.77:0/3371157903 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:09:53.781 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:50.758+0000 7f2e62d9d700 1 -- 172.21.15.77:0/3371157903 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e58009550 con 0x7f2e641055c0 2023-12-01T01:09:53.781 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:50.759+0000 7f2e62d9d700 1 --2- 172.21.15.77:0/3371157903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e641055c0 0x7f2e64199280 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2e58003eb0 tx=0x7f2e58003980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:09:53.781 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:50.759+0000 7f2e4bfff700 1 -- 172.21.15.77:0/3371157903 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2e58003ba0 con 0x7f2e641055c0 2023-12-01T01:09:53.781 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:50.759+0000 7f2e4bfff700 1 -- 172.21.15.77:0/3371157903 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2e58020070 con 0x7f2e641055c0 2023-12-01T01:09:53.781 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:50.759+0000 7f2e691e0700 1 -- 172.21.15.77:0/3371157903 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e641999c0 con 0x7f2e641055c0 2023-12-01T01:09:53.781 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:50.759+0000 7f2e4bfff700 1 -- 172.21.15.77:0/3371157903 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2e5801c5e0 con 0x7f2e641055c0 2023-12-01T01:09:53.781 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:50.759+0000 7f2e691e0700 1 -- 172.21.15.77:0/3371157903 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e64199de0 con 0x7f2e641055c0 2023-12-01T01:09:53.782 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:50.760+0000 7f2e4bfff700 1 -- 172.21.15.77:0/3371157903 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 45257+0+0 (secure 0 0 0) 0x7f2e58022030 con 0x7f2e641055c0 2023-12-01T01:09:53.782 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:50.760+0000 7f2e4bfff700 1 --2- 172.21.15.77:0/3371157903 >> [v2:172.21.15.77:6800/1661788026,v1:172.21.15.77:6801/1661788026] conn(0x7f2e4c0382b0 0x7f2e4c03a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:09:53.782 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:50.760+0000 7f2e4bfff700 1 -- 172.21.15.77:0/3371157903 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f2e5804e4b0 con 0x7f2e641055c0 2023-12-01T01:09:53.782 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:50.760+0000 7f2e6259c700 1 --2- 172.21.15.77:0/3371157903 >> [v2:172.21.15.77:6800/1661788026,v1:172.21.15.77:6801/1661788026] conn(0x7f2e4c0382b0 0x7f2e4c03a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:09:53.782 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:50.760+0000 7f2e691e0700 1 -- 172.21.15.77:0/3371157903 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e640623c0 con 0x7f2e641055c0 2023-12-01T01:09:53.782 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:50.761+0000 7f2e6259c700 1 --2- 172.21.15.77:0/3371157903 >> [v2:172.21.15.77:6800/1661788026,v1:172.21.15.77:6801/1661788026] conn(0x7f2e4c0382b0 0x7f2e4c03a770 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2e54006fd0 tx=0x7f2e54006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:09:53.782 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:50.765+0000 7f2e4bfff700 1 -- 172.21.15.77:0/3371157903 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2e5802e440 con 0x7f2e641055c0 2023-12-01T01:09:53.782 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:51.040+0000 7f2e691e0700 1 -- 172.21.15.77:0/3371157903 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f2e641099c0 con 0x7f2e641055c0 2023-12-01T01:09:53.782 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:51.041+0000 7f2e4bfff700 1 -- 172.21.15.77:0/3371157903 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f2e5802db90 con 0x7f2e641055c0 2023-12-01T01:09:53.782 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:51.043+0000 7f2e691e0700 1 -- 172.21.15.77:0/3371157903 >> [v2:172.21.15.77:6800/1661788026,v1:172.21.15.77:6801/1661788026] conn(0x7f2e4c0382b0 msgr2=0x7f2e4c03a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:09:53.782 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:51.043+0000 7f2e691e0700 1 --2- 172.21.15.77:0/3371157903 >> [v2:172.21.15.77:6800/1661788026,v1:172.21.15.77:6801/1661788026] conn(0x7f2e4c0382b0 0x7f2e4c03a770 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2e54006fd0 tx=0x7f2e54006e40 comp rx=0 tx=0).stop 2023-12-01T01:09:53.782 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:51.043+0000 7f2e691e0700 1 -- 172.21.15.77:0/3371157903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e641055c0 msgr2=0x7f2e64199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:09:53.782 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:51.043+0000 7f2e691e0700 1 --2- 172.21.15.77:0/3371157903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e641055c0 0x7f2e64199280 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2e58003eb0 tx=0x7f2e58003980 comp rx=0 tx=0).stop 2023-12-01T01:09:53.783 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:51.043+0000 7f2e691e0700 1 -- 172.21.15.77:0/3371157903 shutdown_connections 2023-12-01T01:09:53.783 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:51.043+0000 7f2e691e0700 1 --2- 172.21.15.77:0/3371157903 >> [v2:172.21.15.77:6800/1661788026,v1:172.21.15.77:6801/1661788026] conn(0x7f2e4c0382b0 0x7f2e4c03a770 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:09:53.783 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:51.043+0000 7f2e691e0700 1 --2- 172.21.15.77:0/3371157903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2e641055c0 0x7f2e64199280 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:09:53.783 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:51.043+0000 7f2e691e0700 1 -- 172.21.15.77:0/3371157903 >> 172.21.15.77:0/3371157903 conn(0x7f2e64100b80 msgr2=0x7f2e64102270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:09:53.783 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:51.043+0000 7f2e691e0700 1 -- 172.21.15.77:0/3371157903 shutdown_connections 2023-12-01T01:09:53.783 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:51.043+0000 7f2e691e0700 1 -- 172.21.15.77:0/3371157903 wait complete. 2023-12-01T01:09:53.783 INFO:teuthology.orchestra.run.smithi077.stdout:mgr is available 2023-12-01T01:09:54.798 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:54 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/502632388' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-12-01T01:09:54.824 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2023-12-01T01:09:54.824 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout [global] 2023-12-01T01:09:54.824 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout fsid = 2a06e29a-8fe6-11ee-95a2-87774f69a715 2023-12-01T01:09:54.824 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-12-01T01:09:54.824 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-12-01T01:09:54.824 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-12-01T01:09:54.824 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-12-01T01:09:54.824 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2023-12-01T01:09:54.825 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout [mgr] 2023-12-01T01:09:54.825 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-12-01T01:09:54.825 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2023-12-01T01:09:54.825 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout [osd] 2023-12-01T01:09:54.825 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-12-01T01:09:54.825 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-12-01T01:09:54.825 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-12-01T01:09:54.825 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-12-01T01:09:54.825 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-12-01T01:09:54.825 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.227+0000 7fe3e5e27700 1 Processor -- start 2023-12-01T01:09:54.825 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.227+0000 7fe3e5e27700 1 -- start start 2023-12-01T01:09:54.825 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.227+0000 7fe3e5e27700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3e01066a0 0x7fe3e0106ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:09:54.825 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.227+0000 7fe3e5e27700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e0107090 con 0x7fe3e01066a0 2023-12-01T01:09:54.825 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.227+0000 7fe3df7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3e01066a0 0x7fe3e0106ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:09:54.825 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.227+0000 7fe3df7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3e01066a0 0x7fe3e0106ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56198/0 (socket says 172.21.15.77:56198) 2023-12-01T01:09:54.826 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.227+0000 7fe3df7fe700 1 -- 172.21.15.77:0/124885338 learned_addr learned my addr 172.21.15.77:0/124885338 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:09:54.826 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.227+0000 7fe3df7fe700 1 -- 172.21.15.77:0/124885338 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e01078a0 con 0x7fe3e01066a0 2023-12-01T01:09:54.826 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.228+0000 7fe3df7fe700 1 --2- 172.21.15.77:0/124885338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3e01066a0 0x7fe3e0106ac0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe3d00098a0 tx=0x7fe3d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=619c03368cbf7d9f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:09:54.826 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.228+0000 7fe3de7fc700 1 -- 172.21.15.77:0/124885338 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3d0004160 con 0x7fe3e01066a0 2023-12-01T01:09:54.826 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.228+0000 7fe3de7fc700 1 -- 172.21.15.77:0/124885338 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe3d00042c0 con 0x7fe3e01066a0 2023-12-01T01:09:54.826 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.228+0000 7fe3de7fc700 1 -- 172.21.15.77:0/124885338 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3d0011a00 con 0x7fe3e01066a0 2023-12-01T01:09:54.826 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.228+0000 7fe3e5e27700 1 -- 172.21.15.77:0/124885338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3e01066a0 msgr2=0x7fe3e0106ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:09:54.826 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.228+0000 7fe3e5e27700 1 --2- 172.21.15.77:0/124885338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3e01066a0 0x7fe3e0106ac0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe3d00098a0 tx=0x7fe3d0009bb0 comp rx=0 tx=0).stop 2023-12-01T01:09:54.826 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.228+0000 7fe3e5e27700 1 -- 172.21.15.77:0/124885338 shutdown_connections 2023-12-01T01:09:54.826 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.228+0000 7fe3e5e27700 1 --2- 172.21.15.77:0/124885338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3e01066a0 0x7fe3e0106ac0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:09:54.826 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.228+0000 7fe3e5e27700 1 -- 172.21.15.77:0/124885338 >> 172.21.15.77:0/124885338 conn(0x7fe3e0101c40 msgr2=0x7fe3e0104080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:09:54.826 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.228+0000 7fe3e5e27700 1 -- 172.21.15.77:0/124885338 shutdown_connections 2023-12-01T01:09:54.826 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.228+0000 7fe3e5e27700 1 -- 172.21.15.77:0/124885338 wait complete. 2023-12-01T01:09:54.827 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.229+0000 7fe3e5e27700 1 Processor -- start 2023-12-01T01:09:54.827 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.229+0000 7fe3e5e27700 1 -- start start 2023-12-01T01:09:54.827 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.229+0000 7fe3e5e27700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3e01066a0 0x7fe3e019a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:09:54.827 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.229+0000 7fe3e5e27700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e019a910 con 0x7fe3e01066a0 2023-12-01T01:09:54.827 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.229+0000 7fe3df7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3e01066a0 0x7fe3e019a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:09:54.827 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.229+0000 7fe3df7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3e01066a0 0x7fe3e019a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56200/0 (socket says 172.21.15.77:56200) 2023-12-01T01:09:54.827 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.229+0000 7fe3df7fe700 1 -- 172.21.15.77:0/502632388 learned_addr learned my addr 172.21.15.77:0/502632388 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:09:54.827 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.230+0000 7fe3df7fe700 1 -- 172.21.15.77:0/502632388 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d0009550 con 0x7fe3e01066a0 2023-12-01T01:09:54.827 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.230+0000 7fe3df7fe700 1 --2- 172.21.15.77:0/502632388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3e01066a0 0x7fe3e019a3d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0009fb0 tx=0x7fe3d0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:09:54.827 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.230+0000 7fe3dcff9700 1 -- 172.21.15.77:0/502632388 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3d0003a40 con 0x7fe3e01066a0 2023-12-01T01:09:54.827 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.230+0000 7fe3dcff9700 1 -- 172.21.15.77:0/502632388 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe3d0003ba0 con 0x7fe3e01066a0 2023-12-01T01:09:54.827 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.230+0000 7fe3e5e27700 1 -- 172.21.15.77:0/502632388 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3e019ab10 con 0x7fe3e01066a0 2023-12-01T01:09:54.827 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.230+0000 7fe3dcff9700 1 -- 172.21.15.77:0/502632388 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3d001a750 con 0x7fe3e01066a0 2023-12-01T01:09:54.827 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.230+0000 7fe3e5e27700 1 -- 172.21.15.77:0/502632388 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e019af30 con 0x7fe3e01066a0 2023-12-01T01:09:54.828 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.231+0000 7fe3dcff9700 1 -- 172.21.15.77:0/502632388 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 45257+0+0 (secure 0 0 0) 0x7fe3d001a980 con 0x7fe3e01066a0 2023-12-01T01:09:54.828 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.231+0000 7fe3dcff9700 1 --2- 172.21.15.77:0/502632388 >> [v2:172.21.15.77:6800/1661788026,v1:172.21.15.77:6801/1661788026] conn(0x7fe3c80382b0 0x7fe3c803a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:09:54.828 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.231+0000 7fe3dcff9700 1 -- 172.21.15.77:0/502632388 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe3d004bda0 con 0x7fe3e01066a0 2023-12-01T01:09:54.828 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.231+0000 7fe3e5e27700 1 -- 172.21.15.77:0/502632388 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3e00623c0 con 0x7fe3e01066a0 2023-12-01T01:09:54.828 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.231+0000 7fe3deffd700 1 --2- 172.21.15.77:0/502632388 >> [v2:172.21.15.77:6800/1661788026,v1:172.21.15.77:6801/1661788026] conn(0x7fe3c80382b0 0x7fe3c803a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:09:54.828 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.231+0000 7fe3deffd700 1 --2- 172.21.15.77:0/502632388 >> [v2:172.21.15.77:6800/1661788026,v1:172.21.15.77:6801/1661788026] conn(0x7fe3c80382b0 0x7fe3c803a770 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4006fd0 tx=0x7fe3d4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:09:54.828 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.236+0000 7fe3dcff9700 1 -- 172.21.15.77:0/502632388 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe3d002d3f0 con 0x7fe3e01066a0 2023-12-01T01:09:54.828 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.421+0000 7fe3e5e27700 1 -- 172.21.15.77:0/502632388 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fe3e019dd90 con 0x7fe3e01066a0 2023-12-01T01:09:54.828 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.422+0000 7fe3dcff9700 1 -- 172.21.15.77:0/502632388 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fe3d0049020 con 0x7fe3e01066a0 2023-12-01T01:09:54.828 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.424+0000 7fe3e5e27700 1 -- 172.21.15.77:0/502632388 >> [v2:172.21.15.77:6800/1661788026,v1:172.21.15.77:6801/1661788026] conn(0x7fe3c80382b0 msgr2=0x7fe3c803a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:09:54.828 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.424+0000 7fe3e5e27700 1 --2- 172.21.15.77:0/502632388 >> [v2:172.21.15.77:6800/1661788026,v1:172.21.15.77:6801/1661788026] conn(0x7fe3c80382b0 0x7fe3c803a770 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4006fd0 tx=0x7fe3d4006e40 comp rx=0 tx=0).stop 2023-12-01T01:09:54.828 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.424+0000 7fe3e5e27700 1 -- 172.21.15.77:0/502632388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3e01066a0 msgr2=0x7fe3e019a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:09:54.828 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.424+0000 7fe3e5e27700 1 --2- 172.21.15.77:0/502632388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3e01066a0 0x7fe3e019a3d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0009fb0 tx=0x7fe3d0011ad0 comp rx=0 tx=0).stop 2023-12-01T01:09:54.829 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.424+0000 7fe3e5e27700 1 -- 172.21.15.77:0/502632388 shutdown_connections 2023-12-01T01:09:54.829 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.424+0000 7fe3e5e27700 1 --2- 172.21.15.77:0/502632388 >> [v2:172.21.15.77:6800/1661788026,v1:172.21.15.77:6801/1661788026] conn(0x7fe3c80382b0 0x7fe3c803a770 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:09:54.829 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.424+0000 7fe3e5e27700 1 --2- 172.21.15.77:0/502632388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3e01066a0 0x7fe3e019a3d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:09:54.829 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.424+0000 7fe3e5e27700 1 -- 172.21.15.77:0/502632388 >> 172.21.15.77:0/502632388 conn(0x7fe3e0101c40 msgr2=0x7fe3e0103370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:09:54.829 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.424+0000 7fe3e5e27700 1 -- 172.21.15.77:0/502632388 shutdown_connections 2023-12-01T01:09:54.829 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:54.424+0000 7fe3e5e27700 1 -- 172.21.15.77:0/502632388 wait complete. 2023-12-01T01:09:54.829 INFO:teuthology.orchestra.run.smithi077.stdout:Enabling cephadm module... 2023-12-01T01:09:55.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:55 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/2059700114' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-12-01T01:09:55.904 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:55 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: ignoring --setuser ceph since I am not root 2023-12-01T01:09:55.904 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:55 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: ignoring --setgroup ceph since I am not root 2023-12-01T01:09:55.904 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:55 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:55.732+0000 7f8f69958200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T01:09:55.904 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:55 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:55.813+0000 7f8f69958200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T01:09:55.966 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.264+0000 7fd325ca5700 1 Processor -- start 2023-12-01T01:09:55.966 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.264+0000 7fd325ca5700 1 -- start start 2023-12-01T01:09:55.966 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.264+0000 7fd325ca5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd320104f60 0x7fd320105380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:09:55.966 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.264+0000 7fd325ca5700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3200745c0 con 0x7fd320104f60 2023-12-01T01:09:55.966 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.264+0000 7fd31f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd320104f60 0x7fd320105380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:09:55.966 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.264+0000 7fd31f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd320104f60 0x7fd320105380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56204/0 (socket says 172.21.15.77:56204) 2023-12-01T01:09:55.966 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.264+0000 7fd31f7fe700 1 -- 172.21.15.77:0/2398091144 learned_addr learned my addr 172.21.15.77:0/2398091144 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:09:55.966 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.265+0000 7fd31f7fe700 1 -- 172.21.15.77:0/2398091144 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3201058c0 con 0x7fd320104f60 2023-12-01T01:09:55.966 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.265+0000 7fd31f7fe700 1 --2- 172.21.15.77:0/2398091144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd320104f60 0x7fd320105380 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd3140098a0 tx=0x7fd314009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba470cd2341c389d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:09:55.966 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.265+0000 7fd31e7fc700 1 -- 172.21.15.77:0/2398091144 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd314004160 con 0x7fd320104f60 2023-12-01T01:09:55.966 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.265+0000 7fd31e7fc700 1 -- 172.21.15.77:0/2398091144 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd3140042c0 con 0x7fd320104f60 2023-12-01T01:09:55.967 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.265+0000 7fd31e7fc700 1 -- 172.21.15.77:0/2398091144 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd314011a00 con 0x7fd320104f60 2023-12-01T01:09:55.967 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.265+0000 7fd325ca5700 1 -- 172.21.15.77:0/2398091144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd320104f60 msgr2=0x7fd320105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:09:55.967 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.265+0000 7fd325ca5700 1 --2- 172.21.15.77:0/2398091144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd320104f60 0x7fd320105380 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd3140098a0 tx=0x7fd314009bb0 comp rx=0 tx=0).stop 2023-12-01T01:09:55.967 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.266+0000 7fd325ca5700 1 -- 172.21.15.77:0/2398091144 shutdown_connections 2023-12-01T01:09:55.967 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.266+0000 7fd325ca5700 1 --2- 172.21.15.77:0/2398091144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd320104f60 0x7fd320105380 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:09:55.967 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.266+0000 7fd325ca5700 1 -- 172.21.15.77:0/2398091144 >> 172.21.15.77:0/2398091144 conn(0x7fd320100b80 msgr2=0x7fd320102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:09:55.967 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.266+0000 7fd325ca5700 1 -- 172.21.15.77:0/2398091144 shutdown_connections 2023-12-01T01:09:55.967 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.266+0000 7fd325ca5700 1 -- 172.21.15.77:0/2398091144 wait complete. 2023-12-01T01:09:55.967 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.266+0000 7fd325ca5700 1 Processor -- start 2023-12-01T01:09:55.967 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.266+0000 7fd325ca5700 1 -- start start 2023-12-01T01:09:55.967 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.267+0000 7fd325ca5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd320104f60 0x7fd32019c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:09:55.967 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.267+0000 7fd325ca5700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32019caa0 con 0x7fd320104f60 2023-12-01T01:09:55.967 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.267+0000 7fd31f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd320104f60 0x7fd32019c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:09:55.968 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.267+0000 7fd31f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd320104f60 0x7fd32019c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56206/0 (socket says 172.21.15.77:56206) 2023-12-01T01:09:55.968 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.267+0000 7fd31f7fe700 1 -- 172.21.15.77:0/2059700114 learned_addr learned my addr 172.21.15.77:0/2059700114 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:09:55.968 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.267+0000 7fd31f7fe700 1 -- 172.21.15.77:0/2059700114 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd314009550 con 0x7fd320104f60 2023-12-01T01:09:55.968 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.267+0000 7fd31f7fe700 1 --2- 172.21.15.77:0/2059700114 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd320104f60 0x7fd32019c560 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd314006af0 tx=0x7fd314011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:09:55.968 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.267+0000 7fd31cff9700 1 -- 172.21.15.77:0/2059700114 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd314003a40 con 0x7fd320104f60 2023-12-01T01:09:55.968 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.267+0000 7fd31cff9700 1 -- 172.21.15.77:0/2059700114 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd314003ba0 con 0x7fd320104f60 2023-12-01T01:09:55.968 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.267+0000 7fd325ca5700 1 -- 172.21.15.77:0/2059700114 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd32019cca0 con 0x7fd320104f60 2023-12-01T01:09:55.968 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.267+0000 7fd31cff9700 1 -- 172.21.15.77:0/2059700114 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd31401a6e0 con 0x7fd320104f60 2023-12-01T01:09:55.968 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.267+0000 7fd325ca5700 1 -- 172.21.15.77:0/2059700114 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd32019d0c0 con 0x7fd320104f60 2023-12-01T01:09:55.968 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.268+0000 7fd31cff9700 1 -- 172.21.15.77:0/2059700114 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) v1 ==== 45257+0+0 (secure 0 0 0) 0x7fd31401a860 con 0x7fd320104f60 2023-12-01T01:09:55.969 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.268+0000 7fd325ca5700 1 -- 172.21.15.77:0/2059700114 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd320195ea0 con 0x7fd320104f60 2023-12-01T01:09:55.969 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.268+0000 7fd31cff9700 1 --2- 172.21.15.77:0/2059700114 >> [v2:172.21.15.77:6800/1661788026,v1:172.21.15.77:6801/1661788026] conn(0x7fd308038300 0x7fd30803a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:09:55.969 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.268+0000 7fd31cff9700 1 -- 172.21.15.77:0/2059700114 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd31404c0b0 con 0x7fd320104f60 2023-12-01T01:09:55.969 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.269+0000 7fd31effd700 1 --2- 172.21.15.77:0/2059700114 >> [v2:172.21.15.77:6800/1661788026,v1:172.21.15.77:6801/1661788026] conn(0x7fd308038300 0x7fd30803a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:09:55.969 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.269+0000 7fd31effd700 1 --2- 172.21.15.77:0/2059700114 >> [v2:172.21.15.77:6800/1661788026,v1:172.21.15.77:6801/1661788026] conn(0x7fd308038300 0x7fd30803a7c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd310006fd0 tx=0x7fd310006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:09:55.969 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.273+0000 7fd31cff9700 1 -- 172.21.15.77:0/2059700114 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd31401ab10 con 0x7fd320104f60 2023-12-01T01:09:55.969 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.512+0000 7fd325ca5700 1 -- 172.21.15.77:0/2059700114 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fd3200623f0 con 0x7fd320104f60 2023-12-01T01:09:55.969 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.565+0000 7fd31cff9700 1 -- 172.21.15.77:0/2059700114 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mgrmap(e 5) v1 ==== 45268+0+0 (secure 0 0 0) 0x7fd31402ad60 con 0x7fd320104f60 2023-12-01T01:09:55.969 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.567+0000 7fd31cff9700 1 -- 172.21.15.77:0/2059700114 <== mon.0 v2:172.21.15.77:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fd314018350 con 0x7fd320104f60 2023-12-01T01:09:55.969 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.569+0000 7fd325ca5700 1 -- 172.21.15.77:0/2059700114 >> [v2:172.21.15.77:6800/1661788026,v1:172.21.15.77:6801/1661788026] conn(0x7fd308038300 msgr2=0x7fd30803a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:09:55.969 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.569+0000 7fd325ca5700 1 --2- 172.21.15.77:0/2059700114 >> [v2:172.21.15.77:6800/1661788026,v1:172.21.15.77:6801/1661788026] conn(0x7fd308038300 0x7fd30803a7c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd310006fd0 tx=0x7fd310006e40 comp rx=0 tx=0).stop 2023-12-01T01:09:55.969 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.569+0000 7fd325ca5700 1 -- 172.21.15.77:0/2059700114 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd320104f60 msgr2=0x7fd32019c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:09:55.969 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.569+0000 7fd325ca5700 1 --2- 172.21.15.77:0/2059700114 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd320104f60 0x7fd32019c560 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd314006af0 tx=0x7fd314011ad0 comp rx=0 tx=0).stop 2023-12-01T01:09:55.969 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.569+0000 7fd325ca5700 1 -- 172.21.15.77:0/2059700114 shutdown_connections 2023-12-01T01:09:55.970 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.569+0000 7fd325ca5700 1 --2- 172.21.15.77:0/2059700114 >> [v2:172.21.15.77:6800/1661788026,v1:172.21.15.77:6801/1661788026] conn(0x7fd308038300 0x7fd30803a7c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:09:55.970 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.569+0000 7fd325ca5700 1 --2- 172.21.15.77:0/2059700114 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd320104f60 0x7fd32019c560 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:09:55.970 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.570+0000 7fd325ca5700 1 -- 172.21.15.77:0/2059700114 >> 172.21.15.77:0/2059700114 conn(0x7fd320100b80 msgr2=0x7fd320190b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:09:55.970 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.570+0000 7fd325ca5700 1 -- 172.21.15.77:0/2059700114 shutdown_connections 2023-12-01T01:09:55.970 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:55.570+0000 7fd325ca5700 1 -- 172.21.15.77:0/2059700114 wait complete. 2023-12-01T01:09:56.170 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:55 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:55.904+0000 7f8f69958200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T01:09:56.170 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:56 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:56.002+0000 7f8f69958200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T01:09:56.557 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:56 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:56.179+0000 7f8f69958200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T01:09:56.855 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:56 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/2059700114' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-12-01T01:09:56.855 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:56 smithi077 ceph-mon[98867]: mgrmap e5: x(active, since 10s) 2023-12-01T01:09:56.855 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:56 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:56.555+0000 7f8f69958200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T01:09:56.855 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:56 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:56.693+0000 7f8f69958200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T01:09:56.856 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:56 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:56.789+0000 7f8f69958200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T01:09:57.106 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout { 2023-12-01T01:09:57.106 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-12-01T01:09:57.106 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "available": true, 2023-12-01T01:09:57.106 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-12-01T01:09:57.106 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-12-01T01:09:57.106 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout } 2023-12-01T01:09:57.106 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.447+0000 7feb90c4a700 1 Processor -- start 2023-12-01T01:09:57.107 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.447+0000 7feb90c4a700 1 -- start start 2023-12-01T01:09:57.107 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.448+0000 7feb90c4a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb8c107130 0x7feb8c109520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:09:57.107 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.448+0000 7feb90c4a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c074600 con 0x7feb8c107130 2023-12-01T01:09:57.107 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.448+0000 7feb8a59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb8c107130 0x7feb8c109520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:09:57.107 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.448+0000 7feb8a59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb8c107130 0x7feb8c109520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56220/0 (socket says 172.21.15.77:56220) 2023-12-01T01:09:57.107 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.448+0000 7feb8a59c700 1 -- 172.21.15.77:0/309317298 learned_addr learned my addr 172.21.15.77:0/309317298 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:09:57.107 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.448+0000 7feb8a59c700 1 -- 172.21.15.77:0/309317298 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb8c109a60 con 0x7feb8c107130 2023-12-01T01:09:57.107 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.448+0000 7feb8a59c700 1 --2- 172.21.15.77:0/309317298 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb8c107130 0x7feb8c109520 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7feb7c0098a0 tx=0x7feb7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7cb751b9dbfa5c1 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:09:57.107 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.449+0000 7feb8959a700 1 -- 172.21.15.77:0/309317298 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feb7c004160 con 0x7feb8c107130 2023-12-01T01:09:57.107 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.449+0000 7feb8959a700 1 -- 172.21.15.77:0/309317298 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7feb7c0042c0 con 0x7feb8c107130 2023-12-01T01:09:57.107 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.449+0000 7feb8959a700 1 -- 172.21.15.77:0/309317298 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feb7c011a00 con 0x7feb8c107130 2023-12-01T01:09:57.107 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.449+0000 7feb90c4a700 1 -- 172.21.15.77:0/309317298 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb8c107130 msgr2=0x7feb8c109520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:09:57.107 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.449+0000 7feb90c4a700 1 --2- 172.21.15.77:0/309317298 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb8c107130 0x7feb8c109520 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7feb7c0098a0 tx=0x7feb7c009bb0 comp rx=0 tx=0).stop 2023-12-01T01:09:57.107 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.449+0000 7feb90c4a700 1 -- 172.21.15.77:0/309317298 shutdown_connections 2023-12-01T01:09:57.107 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.449+0000 7feb90c4a700 1 --2- 172.21.15.77:0/309317298 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb8c107130 0x7feb8c109520 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:09:57.108 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:56 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:56.887+0000 7f8f69958200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T01:09:57.109 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.449+0000 7feb90c4a700 1 -- 172.21.15.77:0/309317298 >> 172.21.15.77:0/309317298 conn(0x7feb8c100b80 msgr2=0x7feb8c102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:09:57.110 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.449+0000 7feb90c4a700 1 -- 172.21.15.77:0/309317298 shutdown_connections 2023-12-01T01:09:57.110 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.449+0000 7feb90c4a700 1 -- 172.21.15.77:0/309317298 wait complete. 2023-12-01T01:09:57.110 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.450+0000 7feb90c4a700 1 Processor -- start 2023-12-01T01:09:57.110 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.450+0000 7feb90c4a700 1 -- start start 2023-12-01T01:09:57.110 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.450+0000 7feb90c4a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb8c107130 0x7feb8c106a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:09:57.110 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.450+0000 7feb90c4a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c106f80 con 0x7feb8c107130 2023-12-01T01:09:57.110 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.450+0000 7feb8a59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb8c107130 0x7feb8c106a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:09:57.110 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.450+0000 7feb8a59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb8c107130 0x7feb8c106a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56222/0 (socket says 172.21.15.77:56222) 2023-12-01T01:09:57.110 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.450+0000 7feb8a59c700 1 -- 172.21.15.77:0/2227611844 learned_addr learned my addr 172.21.15.77:0/2227611844 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:09:57.110 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.451+0000 7feb8a59c700 1 -- 172.21.15.77:0/2227611844 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb7c009550 con 0x7feb8c107130 2023-12-01T01:09:57.110 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.451+0000 7feb8a59c700 1 --2- 172.21.15.77:0/2227611844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb8c107130 0x7feb8c106a40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7feb7c006af0 tx=0x7feb7c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:09:57.110 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.451+0000 7feb7b7fe700 1 -- 172.21.15.77:0/2227611844 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feb7c003a40 con 0x7feb8c107130 2023-12-01T01:09:57.110 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.451+0000 7feb7b7fe700 1 -- 172.21.15.77:0/2227611844 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7feb7c003ba0 con 0x7feb8c107130 2023-12-01T01:09:57.110 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.451+0000 7feb90c4a700 1 -- 172.21.15.77:0/2227611844 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb8c1050e0 con 0x7feb8c107130 2023-12-01T01:09:57.111 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.451+0000 7feb7b7fe700 1 -- 172.21.15.77:0/2227611844 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feb7c01a6e0 con 0x7feb8c107130 2023-12-01T01:09:57.111 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.451+0000 7feb90c4a700 1 -- 172.21.15.77:0/2227611844 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb8c105560 con 0x7feb8c107130 2023-12-01T01:09:57.111 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.452+0000 7feb90c4a700 1 -- 172.21.15.77:0/2227611844 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb8c10a860 con 0x7feb8c107130 2023-12-01T01:09:57.111 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.452+0000 7feb7b7fe700 1 -- 172.21.15.77:0/2227611844 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 5) v1 ==== 45268+0+0 (secure 0 0 0) 0x7feb7c024a60 con 0x7feb8c107130 2023-12-01T01:09:57.111 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.452+0000 7feb7b7fe700 1 --2- 172.21.15.77:0/2227611844 >> [v2:172.21.15.77:6800/1661788026,v1:172.21.15.77:6801/1661788026] conn(0x7feb74038300 0x7feb7403a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:09:57.111 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.452+0000 7feb7b7fe700 1 -- 172.21.15.77:0/2227611844 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7feb7c04c120 con 0x7feb8c107130 2023-12-01T01:09:57.111 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.452+0000 7feb89d9b700 1 -- 172.21.15.77:0/2227611844 >> [v2:172.21.15.77:6800/1661788026,v1:172.21.15.77:6801/1661788026] conn(0x7feb74038300 msgr2=0x7feb7403a7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/1661788026 2023-12-01T01:09:57.111 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.452+0000 7feb89d9b700 1 --2- 172.21.15.77:0/2227611844 >> [v2:172.21.15.77:6800/1661788026,v1:172.21.15.77:6801/1661788026] conn(0x7feb74038300 0x7feb7403a7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:09:57.111 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.457+0000 7feb7b7fe700 1 -- 172.21.15.77:0/2227611844 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7feb7c024430 con 0x7feb8c107130 2023-12-01T01:09:57.111 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.653+0000 7feb89d9b700 1 -- 172.21.15.77:0/2227611844 >> [v2:172.21.15.77:6800/1661788026,v1:172.21.15.77:6801/1661788026] conn(0x7feb74038300 msgr2=0x7feb7403a7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/1661788026 2023-12-01T01:09:57.111 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.653+0000 7feb89d9b700 1 --2- 172.21.15.77:0/2227611844 >> [v2:172.21.15.77:6800/1661788026,v1:172.21.15.77:6801/1661788026] conn(0x7feb74038300 0x7feb7403a7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:09:57.111 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.704+0000 7feb90c4a700 1 -- 172.21.15.77:0/2227611844 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7feb8c1a03d0 con 0x7feb8c107130 2023-12-01T01:09:57.111 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.705+0000 7feb7b7fe700 1 -- 172.21.15.77:0/2227611844 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7feb7c018350 con 0x7feb8c107130 2023-12-01T01:09:57.111 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.708+0000 7feb90c4a700 1 -- 172.21.15.77:0/2227611844 >> [v2:172.21.15.77:6800/1661788026,v1:172.21.15.77:6801/1661788026] conn(0x7feb74038300 msgr2=0x7feb7403a7c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:09:57.111 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.708+0000 7feb90c4a700 1 --2- 172.21.15.77:0/2227611844 >> [v2:172.21.15.77:6800/1661788026,v1:172.21.15.77:6801/1661788026] conn(0x7feb74038300 0x7feb7403a7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:09:57.111 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.708+0000 7feb90c4a700 1 -- 172.21.15.77:0/2227611844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb8c107130 msgr2=0x7feb8c106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:09:57.112 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.708+0000 7feb90c4a700 1 --2- 172.21.15.77:0/2227611844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb8c107130 0x7feb8c106a40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7feb7c006af0 tx=0x7feb7c011ad0 comp rx=0 tx=0).stop 2023-12-01T01:09:57.112 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.708+0000 7feb90c4a700 1 -- 172.21.15.77:0/2227611844 shutdown_connections 2023-12-01T01:09:57.112 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.708+0000 7feb90c4a700 1 --2- 172.21.15.77:0/2227611844 >> [v2:172.21.15.77:6800/1661788026,v1:172.21.15.77:6801/1661788026] conn(0x7feb74038300 0x7feb7403a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:09:57.112 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.708+0000 7feb90c4a700 1 --2- 172.21.15.77:0/2227611844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb8c107130 0x7feb8c106a40 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:09:57.112 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.708+0000 7feb90c4a700 1 -- 172.21.15.77:0/2227611844 >> 172.21.15.77:0/2227611844 conn(0x7feb8c100b80 msgr2=0x7feb8c102220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:09:57.112 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.708+0000 7feb90c4a700 1 -- 172.21.15.77:0/2227611844 shutdown_connections 2023-12-01T01:09:57.112 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:56.709+0000 7feb90c4a700 1 -- 172.21.15.77:0/2227611844 wait complete. 2023-12-01T01:09:57.112 INFO:teuthology.orchestra.run.smithi077.stdout:Waiting for the mgr to restart... 2023-12-01T01:09:57.112 INFO:teuthology.orchestra.run.smithi077.stdout:Waiting for mgr epoch 5... 2023-12-01T01:09:57.404 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:57 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:57.323+0000 7f8f69958200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T01:09:57.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:09:57 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/2227611844' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-12-01T01:09:58.404 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:57 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:57.985+0000 7f8f69958200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T01:09:58.404 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:58 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:58.066+0000 7f8f69958200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T01:09:59.090 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:58 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:58.827+0000 7f8f69958200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:09:59.090 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:58 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:58.917+0000 7f8f69958200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T01:09:59.365 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:59 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:59.088+0000 7f8f69958200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:09:59.654 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:59 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:59.364+0000 7f8f69958200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T01:09:59.654 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:59 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:59.587+0000 7f8f69958200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T01:09:59.995 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:59 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:59.733+0000 7f8f69958200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T01:10:00.404 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:09:59 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:09:59.994+0000 7f8f69958200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T01:10:01.040 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:00 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:10:00.714+0000 7f8f69958200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T01:10:01.040 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:00 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:10:00.845+0000 7f8f69958200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T01:10:01.369 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:01 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:10:01.038+0000 7f8f69958200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T01:10:01.369 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:01 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:10:01.151+0000 7f8f69958200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T01:10:01.654 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:01 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:10:01.367+0000 7f8f69958200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T01:10:02.496 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:02 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:10:02.218+0000 7f8f69958200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T01:10:02.909 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:02 smithi077 ceph-mon[98867]: Active manager daemon x restarted 2023-12-01T01:10:02.909 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:02 smithi077 ceph-mon[98867]: Activating manager daemon x 2023-12-01T01:10:02.909 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:02 smithi077 ceph-mon[98867]: osdmap e2: 0 total, 0 up, 0 in 2023-12-01T01:10:02.909 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:02 smithi077 ceph-mon[98867]: mgrmap e6: x(active, starting, since 0.00298901s) 2023-12-01T01:10:02.910 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:02 smithi077 ceph-mon[98867]: from='mgr.14120 172.21.15.77:0/775606565' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-01T01:10:02.910 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:02 smithi077 ceph-mon[98867]: from='mgr.14120 172.21.15.77:0/775606565' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-01T01:10:02.910 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:02 smithi077 ceph-mon[98867]: from='mgr.14120 172.21.15.77:0/775606565' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-01T01:10:02.910 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:02 smithi077 ceph-mon[98867]: from='mgr.14120 172.21.15.77:0/775606565' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-01T01:10:02.910 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:02 smithi077 ceph-mon[98867]: from='mgr.14120 172.21.15.77:0/775606565' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-01T01:10:02.910 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:02 smithi077 ceph-mon[98867]: Manager daemon x is now available 2023-12-01T01:10:02.910 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:02 smithi077 ceph-mon[98867]: from='mgr.14120 172.21.15.77:0/775606565' entity='mgr.x' 2023-12-01T01:10:02.911 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:02 smithi077 ceph-mon[98867]: from='mgr.14120 172.21.15.77:0/775606565' entity='mgr.x' 2023-12-01T01:10:02.911 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:02 smithi077 ceph-mon[98867]: from='mgr.14120 172.21.15.77:0/775606565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:10:02.911 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:02 smithi077 ceph-mon[98867]: from='mgr.14120 172.21.15.77:0/775606565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:10:02.911 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:02 smithi077 ceph-mon[98867]: from='mgr.14120 172.21.15.77:0/775606565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:10:02.911 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:02 smithi077 ceph-mon[98867]: from='mgr.14120 172.21.15.77:0/775606565' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-12-01T01:10:02.911 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:02 smithi077 ceph-mon[98867]: from='mgr.14120 172.21.15.77:0/775606565' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-12-01T01:10:03.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:03 smithi077 ceph-mon[98867]: Found migration_current of "None". Setting to last migration. 2023-12-01T01:10:03.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:03 smithi077 ceph-mon[98867]: mgrmap e7: x(active, since 1.00503s) 2023-12-01T01:10:03.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:03 smithi077 ceph-mon[98867]: from='mgr.14120 172.21.15.77:0/775606565' entity='mgr.x' 2023-12-01T01:10:03.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:03 smithi077 ceph-mon[98867]: from='mgr.14120 172.21.15.77:0/775606565' entity='mgr.x' 2023-12-01T01:10:03.668 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout { 2023-12-01T01:10:03.669 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-12-01T01:10:03.669 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "initialized": true 2023-12-01T01:10:03.669 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout } 2023-12-01T01:10:03.669 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:57.548+0000 7f729fc06700 1 Processor -- start 2023-12-01T01:10:03.669 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:57.549+0000 7f729fc06700 1 -- start start 2023-12-01T01:10:03.669 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:57.549+0000 7f729fc06700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7298107170 0x7f7298109560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:03.669 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:57.549+0000 7f729fc06700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72980745c0 con 0x7f7298107170 2023-12-01T01:10:03.669 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:57.549+0000 7f729d9a2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7298107170 0x7f7298109560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:03.669 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:57.549+0000 7f729d9a2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7298107170 0x7f7298109560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56230/0 (socket says 172.21.15.77:56230) 2023-12-01T01:10:03.669 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:57.549+0000 7f729d9a2700 1 -- 172.21.15.77:0/1596637869 learned_addr learned my addr 172.21.15.77:0/1596637869 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:03.669 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:57.549+0000 7f729d9a2700 1 -- 172.21.15.77:0/1596637869 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7298109aa0 con 0x7f7298107170 2023-12-01T01:10:03.669 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:57.550+0000 7f729d9a2700 1 --2- 172.21.15.77:0/1596637869 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7298107170 0x7f7298109560 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f72940098a0 tx=0x7f7294009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d01c5e4bac774a48 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:03.669 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:57.550+0000 7f729c9a0700 1 -- 172.21.15.77:0/1596637869 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7294004160 con 0x7f7298107170 2023-12-01T01:10:03.670 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:57.550+0000 7f729c9a0700 1 -- 172.21.15.77:0/1596637869 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f72940042c0 con 0x7f7298107170 2023-12-01T01:10:03.670 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:57.550+0000 7f729c9a0700 1 -- 172.21.15.77:0/1596637869 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7294011a00 con 0x7f7298107170 2023-12-01T01:10:03.670 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:57.550+0000 7f729fc06700 1 -- 172.21.15.77:0/1596637869 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7298107170 msgr2=0x7f7298109560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:03.670 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:57.550+0000 7f729fc06700 1 --2- 172.21.15.77:0/1596637869 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7298107170 0x7f7298109560 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f72940098a0 tx=0x7f7294009bb0 comp rx=0 tx=0).stop 2023-12-01T01:10:03.670 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:57.550+0000 7f729fc06700 1 -- 172.21.15.77:0/1596637869 shutdown_connections 2023-12-01T01:10:03.670 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:57.550+0000 7f729fc06700 1 --2- 172.21.15.77:0/1596637869 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7298107170 0x7f7298109560 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:03.670 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:57.550+0000 7f729fc06700 1 -- 172.21.15.77:0/1596637869 >> 172.21.15.77:0/1596637869 conn(0x7f7298100b80 msgr2=0x7f7298102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:03.670 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:57.551+0000 7f729fc06700 1 -- 172.21.15.77:0/1596637869 shutdown_connections 2023-12-01T01:10:03.670 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:57.551+0000 7f729fc06700 1 -- 172.21.15.77:0/1596637869 wait complete. 2023-12-01T01:10:03.670 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:57.551+0000 7f729fc06700 1 Processor -- start 2023-12-01T01:10:03.670 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:57.551+0000 7f729fc06700 1 -- start start 2023-12-01T01:10:03.670 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:57.551+0000 7f729fc06700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7298107170 0x7f729819c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:03.670 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:57.551+0000 7f729fc06700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729819ca60 con 0x7f7298107170 2023-12-01T01:10:03.670 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:57.551+0000 7f729d9a2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7298107170 0x7f729819c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:03.671 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:57.551+0000 7f729d9a2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7298107170 0x7f729819c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56232/0 (socket says 172.21.15.77:56232) 2023-12-01T01:10:03.671 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:57.551+0000 7f729d9a2700 1 -- 172.21.15.77:0/2507660229 learned_addr learned my addr 172.21.15.77:0/2507660229 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:03.671 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:57.552+0000 7f729d9a2700 1 -- 172.21.15.77:0/2507660229 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7294009550 con 0x7f7298107170 2023-12-01T01:10:03.671 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:57.552+0000 7f729d9a2700 1 --2- 172.21.15.77:0/2507660229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7298107170 0x7f729819c520 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7294006af0 tx=0x7f7294011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:03.671 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:57.552+0000 7f728effd700 1 -- 172.21.15.77:0/2507660229 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7294003a40 con 0x7f7298107170 2023-12-01T01:10:03.671 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:57.552+0000 7f728effd700 1 -- 172.21.15.77:0/2507660229 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7294003ba0 con 0x7f7298107170 2023-12-01T01:10:03.671 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:57.552+0000 7f729fc06700 1 -- 172.21.15.77:0/2507660229 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f729819cc60 con 0x7f7298107170 2023-12-01T01:10:03.671 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:57.552+0000 7f728effd700 1 -- 172.21.15.77:0/2507660229 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f729401a6e0 con 0x7f7298107170 2023-12-01T01:10:03.671 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:57.552+0000 7f729fc06700 1 -- 172.21.15.77:0/2507660229 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f729819d080 con 0x7f7298107170 2023-12-01T01:10:03.671 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:57.553+0000 7f728effd700 1 -- 172.21.15.77:0/2507660229 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 5) v1 ==== 45268+0+0 (secure 0 0 0) 0x7f729401a860 con 0x7f7298107170 2023-12-01T01:10:03.671 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:57.553+0000 7f728effd700 1 --2- 172.21.15.77:0/2507660229 >> [v2:172.21.15.77:6800/1661788026,v1:172.21.15.77:6801/1661788026] conn(0x7f7284038350 0x7f728403a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:03.671 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:57.553+0000 7f729fc06700 1 -- 172.21.15.77:0/2507660229 --> [v2:172.21.15.77:6800/1661788026,v1:172.21.15.77:6801/1661788026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f72980623c0 con 0x7f7284038350 2023-12-01T01:10:03.671 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:57.553+0000 7f728effd700 1 -- 172.21.15.77:0/2507660229 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f729404c0a0 con 0x7f7298107170 2023-12-01T01:10:03.671 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:57.553+0000 7f729d1a1700 1 -- 172.21.15.77:0/2507660229 >> [v2:172.21.15.77:6800/1661788026,v1:172.21.15.77:6801/1661788026] conn(0x7f7284038350 msgr2=0x7f728403a810 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/1661788026 2023-12-01T01:10:03.672 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:57.553+0000 7f729d1a1700 1 --2- 172.21.15.77:0/2507660229 >> [v2:172.21.15.77:6800/1661788026,v1:172.21.15.77:6801/1661788026] conn(0x7f7284038350 0x7f728403a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:10:03.672 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:57.754+0000 7f729d1a1700 1 -- 172.21.15.77:0/2507660229 >> [v2:172.21.15.77:6800/1661788026,v1:172.21.15.77:6801/1661788026] conn(0x7f7284038350 msgr2=0x7f728403a810 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/1661788026 2023-12-01T01:10:03.672 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:57.754+0000 7f729d1a1700 1 --2- 172.21.15.77:0/2507660229 >> [v2:172.21.15.77:6800/1661788026,v1:172.21.15.77:6801/1661788026] conn(0x7f7284038350 0x7f728403a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:10:03.672 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:58.155+0000 7f729d1a1700 1 -- 172.21.15.77:0/2507660229 >> [v2:172.21.15.77:6800/1661788026,v1:172.21.15.77:6801/1661788026] conn(0x7f7284038350 msgr2=0x7f728403a810 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/1661788026 2023-12-01T01:10:03.672 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:58.155+0000 7f729d1a1700 1 --2- 172.21.15.77:0/2507660229 >> [v2:172.21.15.77:6800/1661788026,v1:172.21.15.77:6801/1661788026] conn(0x7f7284038350 0x7f728403a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-01T01:10:03.672 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:58.956+0000 7f729d1a1700 1 -- 172.21.15.77:0/2507660229 >> [v2:172.21.15.77:6800/1661788026,v1:172.21.15.77:6801/1661788026] conn(0x7f7284038350 msgr2=0x7f728403a810 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/1661788026 2023-12-01T01:10:03.672 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:09:58.956+0000 7f729d1a1700 1 --2- 172.21.15.77:0/2507660229 >> [v2:172.21.15.77:6800/1661788026,v1:172.21.15.77:6801/1661788026] conn(0x7f7284038350 0x7f728403a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-01T01:10:03.672 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:00.557+0000 7f729d1a1700 1 -- 172.21.15.77:0/2507660229 >> [v2:172.21.15.77:6800/1661788026,v1:172.21.15.77:6801/1661788026] conn(0x7f7284038350 msgr2=0x7f728403a810 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/1661788026 2023-12-01T01:10:03.672 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:00.557+0000 7f729d1a1700 1 --2- 172.21.15.77:0/2507660229 >> [v2:172.21.15.77:6800/1661788026,v1:172.21.15.77:6801/1661788026] conn(0x7f7284038350 0x7f728403a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-01T01:10:03.672 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:02.233+0000 7f728effd700 1 -- 172.21.15.77:0/2507660229 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mgrmap(e 6) v1 ==== 45035+0+0 (secure 0 0 0) 0x7f729401acf0 con 0x7f7298107170 2023-12-01T01:10:03.672 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:02.234+0000 7f728effd700 1 -- 172.21.15.77:0/2507660229 >> [v2:172.21.15.77:6800/1661788026,v1:172.21.15.77:6801/1661788026] conn(0x7f7284038350 msgr2=0x7f728403a810 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:10:03.672 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:02.234+0000 7f728effd700 1 --2- 172.21.15.77:0/2507660229 >> [v2:172.21.15.77:6800/1661788026,v1:172.21.15.77:6801/1661788026] conn(0x7f7284038350 0x7f728403a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:03.672 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:03.235+0000 7f728effd700 1 -- 172.21.15.77:0/2507660229 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mgrmap(e 7) v1 ==== 45162+0+0 (secure 0 0 0) 0x7f729404dbd0 con 0x7f7298107170 2023-12-01T01:10:03.672 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:03.235+0000 7f728effd700 1 --2- 172.21.15.77:0/2507660229 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7f728403ba80 0x7f728403de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:03.673 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:03.235+0000 7f728effd700 1 -- 172.21.15.77:0/2507660229 --> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f72980623c0 con 0x7f728403ba80 2023-12-01T01:10:03.673 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:03.235+0000 7f729d1a1700 1 --2- 172.21.15.77:0/2507660229 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7f728403ba80 0x7f728403de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:03.673 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:03.236+0000 7f729d1a1700 1 --2- 172.21.15.77:0/2507660229 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7f728403ba80 0x7f728403de70 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7288003a10 tx=0x7f7288009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:03.673 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:03.236+0000 7f728effd700 1 -- 172.21.15.77:0/2507660229 <== mgr.14120 v2:172.21.15.77:6800/1456621747 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f72980623c0 con 0x7f728403ba80 2023-12-01T01:10:03.673 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:03.243+0000 7f729fc06700 1 -- 172.21.15.77:0/2507660229 --> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f729819d860 con 0x7f728403ba80 2023-12-01T01:10:03.673 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:03.243+0000 7f728effd700 1 -- 172.21.15.77:0/2507660229 <== mgr.14120 v2:172.21.15.77:6800/1456621747 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f729819d860 con 0x7f728403ba80 2023-12-01T01:10:03.673 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:03.243+0000 7f729fc06700 1 -- 172.21.15.77:0/2507660229 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7f728403ba80 msgr2=0x7f728403de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:03.673 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:03.243+0000 7f729fc06700 1 --2- 172.21.15.77:0/2507660229 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7f728403ba80 0x7f728403de70 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7288003a10 tx=0x7f7288009290 comp rx=0 tx=0).stop 2023-12-01T01:10:03.673 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:03.243+0000 7f729fc06700 1 -- 172.21.15.77:0/2507660229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7298107170 msgr2=0x7f729819c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:03.673 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:03.243+0000 7f729fc06700 1 --2- 172.21.15.77:0/2507660229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7298107170 0x7f729819c520 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7294006af0 tx=0x7f7294011ad0 comp rx=0 tx=0).stop 2023-12-01T01:10:03.673 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:03.243+0000 7f729fc06700 1 -- 172.21.15.77:0/2507660229 shutdown_connections 2023-12-01T01:10:03.673 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:03.243+0000 7f729fc06700 1 --2- 172.21.15.77:0/2507660229 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7f728403ba80 0x7f728403de70 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:03.673 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:03.243+0000 7f729fc06700 1 --2- 172.21.15.77:0/2507660229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7298107170 0x7f729819c520 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:03.673 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:03.243+0000 7f729fc06700 1 --2- 172.21.15.77:0/2507660229 >> [v2:172.21.15.77:6800/1661788026,v1:172.21.15.77:6801/1661788026] conn(0x7f7284038350 0x7f728403a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:03.674 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:03.243+0000 7f729fc06700 1 -- 172.21.15.77:0/2507660229 >> 172.21.15.77:0/2507660229 conn(0x7f7298100b80 msgr2=0x7f7298101940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:03.674 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:03.243+0000 7f729fc06700 1 -- 172.21.15.77:0/2507660229 shutdown_connections 2023-12-01T01:10:03.674 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:03.243+0000 7f729fc06700 1 -- 172.21.15.77:0/2507660229 wait complete. 2023-12-01T01:10:03.674 INFO:teuthology.orchestra.run.smithi077.stdout:mgr epoch 5 is available 2023-12-01T01:10:03.674 INFO:teuthology.orchestra.run.smithi077.stdout:Setting orchestrator backend to cephadm... 2023-12-01T01:10:04.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:04 smithi077 ceph-mon[98867]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-12-01T01:10:04.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:04 smithi077 ceph-mon[98867]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-12-01T01:10:04.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:04 smithi077 ceph-mon[98867]: [01/Dec/2023:01:10:03] ENGINE Bus STARTING 2023-12-01T01:10:04.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:04 smithi077 ceph-mon[98867]: from='mgr.14120 172.21.15.77:0/775606565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:10:04.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:04 smithi077 ceph-mon[98867]: from='mgr.14120 172.21.15.77:0/775606565' entity='mgr.x' 2023-12-01T01:10:04.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:04 smithi077 ceph-mon[98867]: from='mgr.14120 172.21.15.77:0/775606565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:10:04.712 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.122+0000 7f30dc109700 1 Processor -- start 2023-12-01T01:10:04.712 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.122+0000 7f30dc109700 1 -- start start 2023-12-01T01:10:04.712 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.122+0000 7f30dc109700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30d4107170 0x7f30d4109560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:04.712 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.122+0000 7f30dc109700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d40745c0 con 0x7f30d4107170 2023-12-01T01:10:04.713 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.122+0000 7f30d9ea5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30d4107170 0x7f30d4109560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:04.713 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.122+0000 7f30d9ea5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30d4107170 0x7f30d4109560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56284/0 (socket says 172.21.15.77:56284) 2023-12-01T01:10:04.713 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.122+0000 7f30d9ea5700 1 -- 172.21.15.77:0/1051043766 learned_addr learned my addr 172.21.15.77:0/1051043766 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:04.713 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.123+0000 7f30d9ea5700 1 -- 172.21.15.77:0/1051043766 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d4109aa0 con 0x7f30d4107170 2023-12-01T01:10:04.713 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.123+0000 7f30d9ea5700 1 --2- 172.21.15.77:0/1051043766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30d4107170 0x7f30d4109560 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f30d00098a0 tx=0x7f30d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55991eea99164daf server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:04.713 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.123+0000 7f30d8ea3700 1 -- 172.21.15.77:0/1051043766 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30d0004160 con 0x7f30d4107170 2023-12-01T01:10:04.713 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.123+0000 7f30d8ea3700 1 -- 172.21.15.77:0/1051043766 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f30d00042c0 con 0x7f30d4107170 2023-12-01T01:10:04.713 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.123+0000 7f30d8ea3700 1 -- 172.21.15.77:0/1051043766 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30d0011a00 con 0x7f30d4107170 2023-12-01T01:10:04.713 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.123+0000 7f30dc109700 1 -- 172.21.15.77:0/1051043766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30d4107170 msgr2=0x7f30d4109560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:04.713 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.123+0000 7f30dc109700 1 --2- 172.21.15.77:0/1051043766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30d4107170 0x7f30d4109560 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f30d00098a0 tx=0x7f30d0009bb0 comp rx=0 tx=0).stop 2023-12-01T01:10:04.713 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.123+0000 7f30dc109700 1 -- 172.21.15.77:0/1051043766 shutdown_connections 2023-12-01T01:10:04.713 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.123+0000 7f30dc109700 1 --2- 172.21.15.77:0/1051043766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30d4107170 0x7f30d4109560 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:04.713 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.123+0000 7f30dc109700 1 -- 172.21.15.77:0/1051043766 >> 172.21.15.77:0/1051043766 conn(0x7f30d4100b80 msgr2=0x7f30d4102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:04.714 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.123+0000 7f30dc109700 1 -- 172.21.15.77:0/1051043766 shutdown_connections 2023-12-01T01:10:04.714 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.123+0000 7f30dc109700 1 -- 172.21.15.77:0/1051043766 wait complete. 2023-12-01T01:10:04.714 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.124+0000 7f30dc109700 1 Processor -- start 2023-12-01T01:10:04.714 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.124+0000 7f30dc109700 1 -- start start 2023-12-01T01:10:04.714 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.124+0000 7f30dc109700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30d4107170 0x7f30d419a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:04.714 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.124+0000 7f30dc109700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d419a890 con 0x7f30d4107170 2023-12-01T01:10:04.714 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.124+0000 7f30d9ea5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30d4107170 0x7f30d419a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:04.714 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.124+0000 7f30d9ea5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30d4107170 0x7f30d419a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56286/0 (socket says 172.21.15.77:56286) 2023-12-01T01:10:04.714 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.124+0000 7f30d9ea5700 1 -- 172.21.15.77:0/890129854 learned_addr learned my addr 172.21.15.77:0/890129854 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:04.715 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.124+0000 7f30d9ea5700 1 -- 172.21.15.77:0/890129854 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d0009550 con 0x7f30d4107170 2023-12-01T01:10:04.715 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.125+0000 7f30d9ea5700 1 --2- 172.21.15.77:0/890129854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30d4107170 0x7f30d419a350 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f30d0009fb0 tx=0x7f30d0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:04.715 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.125+0000 7f30caffd700 1 -- 172.21.15.77:0/890129854 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30d0003930 con 0x7f30d4107170 2023-12-01T01:10:04.715 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.125+0000 7f30caffd700 1 -- 172.21.15.77:0/890129854 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f30d0003a90 con 0x7f30d4107170 2023-12-01T01:10:04.715 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.125+0000 7f30caffd700 1 -- 172.21.15.77:0/890129854 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30d0019610 con 0x7f30d4107170 2023-12-01T01:10:04.715 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.125+0000 7f30dc109700 1 -- 172.21.15.77:0/890129854 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d419aa90 con 0x7f30d4107170 2023-12-01T01:10:04.715 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.125+0000 7f30dc109700 1 -- 172.21.15.77:0/890129854 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d419aeb0 con 0x7f30d4107170 2023-12-01T01:10:04.715 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.126+0000 7f30caffd700 1 -- 172.21.15.77:0/890129854 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 7) v1 ==== 45162+0+0 (secure 0 0 0) 0x7f30d001a430 con 0x7f30d4107170 2023-12-01T01:10:04.716 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.126+0000 7f30dc109700 1 -- 172.21.15.77:0/890129854 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30d40623c0 con 0x7f30d4107170 2023-12-01T01:10:04.716 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.126+0000 7f30caffd700 1 --2- 172.21.15.77:0/890129854 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7f30c00381e0 0x7f30c003a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:04.716 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.126+0000 7f30caffd700 1 -- 172.21.15.77:0/890129854 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f30d004c100 con 0x7f30d4107170 2023-12-01T01:10:04.716 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.126+0000 7f30d96a4700 1 --2- 172.21.15.77:0/890129854 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7f30c00381e0 0x7f30c003a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:04.716 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.126+0000 7f30d96a4700 1 --2- 172.21.15.77:0/890129854 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7f30c00381e0 0x7f30c003a6a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f30c4006fd0 tx=0x7f30c4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:04.716 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.131+0000 7f30caffd700 1 -- 172.21.15.77:0/890129854 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f30d0023b80 con 0x7f30d4107170 2023-12-01T01:10:04.716 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.333+0000 7f30dc109700 1 -- 172.21.15.77:0/890129854 --> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f30d4102e50 con 0x7f30c00381e0 2023-12-01T01:10:04.716 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.339+0000 7f30caffd700 1 -- 172.21.15.77:0/890129854 <== mgr.14120 v2:172.21.15.77:6800/1456621747 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f30d4102e50 con 0x7f30c00381e0 2023-12-01T01:10:04.716 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.341+0000 7f30dc109700 1 -- 172.21.15.77:0/890129854 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7f30c00381e0 msgr2=0x7f30c003a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:04.716 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.341+0000 7f30dc109700 1 --2- 172.21.15.77:0/890129854 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7f30c00381e0 0x7f30c003a6a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f30c4006fd0 tx=0x7f30c4006e40 comp rx=0 tx=0).stop 2023-12-01T01:10:04.716 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.341+0000 7f30dc109700 1 -- 172.21.15.77:0/890129854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30d4107170 msgr2=0x7f30d419a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:04.716 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.341+0000 7f30dc109700 1 --2- 172.21.15.77:0/890129854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30d4107170 0x7f30d419a350 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f30d0009fb0 tx=0x7f30d0011ad0 comp rx=0 tx=0).stop 2023-12-01T01:10:04.716 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.341+0000 7f30dc109700 1 -- 172.21.15.77:0/890129854 shutdown_connections 2023-12-01T01:10:04.716 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.341+0000 7f30dc109700 1 --2- 172.21.15.77:0/890129854 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7f30c00381e0 0x7f30c003a6a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:04.717 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.341+0000 7f30dc109700 1 --2- 172.21.15.77:0/890129854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30d4107170 0x7f30d419a350 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:04.717 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.341+0000 7f30dc109700 1 -- 172.21.15.77:0/890129854 >> 172.21.15.77:0/890129854 conn(0x7f30d4100b80 msgr2=0x7f30d4102230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:04.717 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.341+0000 7f30dc109700 1 -- 172.21.15.77:0/890129854 shutdown_connections 2023-12-01T01:10:04.717 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:04.341+0000 7f30dc109700 1 -- 172.21.15.77:0/890129854 wait complete. 2023-12-01T01:10:05.655 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout value unchanged 2023-12-01T01:10:05.655 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.138+0000 7fd83781e700 1 Processor -- start 2023-12-01T01:10:05.655 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.138+0000 7fd83781e700 1 -- start start 2023-12-01T01:10:05.655 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.138+0000 7fd83781e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8301055c0 0x7fd8301059e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:05.655 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.138+0000 7fd83781e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd830105fb0 con 0x7fd8301055c0 2023-12-01T01:10:05.655 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.138+0000 7fd8355ba700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8301055c0 0x7fd8301059e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:05.656 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.138+0000 7fd8355ba700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8301055c0 0x7fd8301059e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56290/0 (socket says 172.21.15.77:56290) 2023-12-01T01:10:05.656 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.138+0000 7fd8355ba700 1 -- 172.21.15.77:0/4194407350 learned_addr learned my addr 172.21.15.77:0/4194407350 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:05.656 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.139+0000 7fd8355ba700 1 -- 172.21.15.77:0/4194407350 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8301067c0 con 0x7fd8301055c0 2023-12-01T01:10:05.656 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.139+0000 7fd8355ba700 1 --2- 172.21.15.77:0/4194407350 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8301055c0 0x7fd8301059e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd8240098a0 tx=0x7fd824009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdfdd182ed97fca7 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:05.656 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.139+0000 7fd823fff700 1 -- 172.21.15.77:0/4194407350 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd824004160 con 0x7fd8301055c0 2023-12-01T01:10:05.656 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.139+0000 7fd823fff700 1 -- 172.21.15.77:0/4194407350 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd8240042c0 con 0x7fd8301055c0 2023-12-01T01:10:05.656 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.139+0000 7fd823fff700 1 -- 172.21.15.77:0/4194407350 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd824011a00 con 0x7fd8301055c0 2023-12-01T01:10:05.656 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.139+0000 7fd83781e700 1 -- 172.21.15.77:0/4194407350 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8301055c0 msgr2=0x7fd8301059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:05.656 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.139+0000 7fd83781e700 1 --2- 172.21.15.77:0/4194407350 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8301055c0 0x7fd8301059e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd8240098a0 tx=0x7fd824009bb0 comp rx=0 tx=0).stop 2023-12-01T01:10:05.656 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.139+0000 7fd83781e700 1 -- 172.21.15.77:0/4194407350 shutdown_connections 2023-12-01T01:10:05.656 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.139+0000 7fd83781e700 1 --2- 172.21.15.77:0/4194407350 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8301055c0 0x7fd8301059e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:05.656 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.139+0000 7fd83781e700 1 -- 172.21.15.77:0/4194407350 >> 172.21.15.77:0/4194407350 conn(0x7fd830100b80 msgr2=0x7fd830102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:05.656 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.140+0000 7fd83781e700 1 -- 172.21.15.77:0/4194407350 shutdown_connections 2023-12-01T01:10:05.657 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.140+0000 7fd83781e700 1 -- 172.21.15.77:0/4194407350 wait complete. 2023-12-01T01:10:05.657 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.140+0000 7fd83781e700 1 Processor -- start 2023-12-01T01:10:05.657 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.140+0000 7fd83781e700 1 -- start start 2023-12-01T01:10:05.657 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.140+0000 7fd83781e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8301055c0 0x7fd830199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:05.657 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.140+0000 7fd83781e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8301997c0 con 0x7fd8301055c0 2023-12-01T01:10:05.657 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.140+0000 7fd8355ba700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8301055c0 0x7fd830199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:05.657 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.140+0000 7fd8355ba700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8301055c0 0x7fd830199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56292/0 (socket says 172.21.15.77:56292) 2023-12-01T01:10:05.657 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.140+0000 7fd8355ba700 1 -- 172.21.15.77:0/52141661 learned_addr learned my addr 172.21.15.77:0/52141661 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:05.657 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.141+0000 7fd8355ba700 1 -- 172.21.15.77:0/52141661 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd824009550 con 0x7fd8301055c0 2023-12-01T01:10:05.657 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.141+0000 7fd8355ba700 1 --2- 172.21.15.77:0/52141661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8301055c0 0x7fd830199280 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd824006af0 tx=0x7fd824011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:05.657 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.141+0000 7fd8227fc700 1 -- 172.21.15.77:0/52141661 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd824003a40 con 0x7fd8301055c0 2023-12-01T01:10:05.657 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.141+0000 7fd8227fc700 1 -- 172.21.15.77:0/52141661 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd824003ba0 con 0x7fd8301055c0 2023-12-01T01:10:05.657 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.141+0000 7fd8227fc700 1 -- 172.21.15.77:0/52141661 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd82401a700 con 0x7fd8301055c0 2023-12-01T01:10:05.657 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.141+0000 7fd83781e700 1 -- 172.21.15.77:0/52141661 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8301999c0 con 0x7fd8301055c0 2023-12-01T01:10:05.658 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.141+0000 7fd83781e700 1 -- 172.21.15.77:0/52141661 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd830199de0 con 0x7fd8301055c0 2023-12-01T01:10:05.658 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.142+0000 7fd83781e700 1 -- 172.21.15.77:0/52141661 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8300623c0 con 0x7fd8301055c0 2023-12-01T01:10:05.658 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.142+0000 7fd8227fc700 1 -- 172.21.15.77:0/52141661 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) v1 ==== 45268+0+0 (secure 0 0 0) 0x7fd824024ad0 con 0x7fd8301055c0 2023-12-01T01:10:05.658 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.142+0000 7fd8227fc700 1 --2- 172.21.15.77:0/52141661 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7fd81c038350 0x7fd81c03a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:05.658 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.142+0000 7fd8227fc700 1 -- 172.21.15.77:0/52141661 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd82404c710 con 0x7fd8301055c0 2023-12-01T01:10:05.658 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.143+0000 7fd834db9700 1 --2- 172.21.15.77:0/52141661 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7fd81c038350 0x7fd81c03a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:05.658 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.143+0000 7fd834db9700 1 --2- 172.21.15.77:0/52141661 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7fd81c038350 0x7fd81c03a810 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd82c006fd0 tx=0x7fd82c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:05.658 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.147+0000 7fd8227fc700 1 -- 172.21.15.77:0/52141661 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd824024430 con 0x7fd8301055c0 2023-12-01T01:10:05.658 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.334+0000 7fd83781e700 1 -- 172.21.15.77:0/52141661 --> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fd830102e90 con 0x7fd81c038350 2023-12-01T01:10:05.658 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.334+0000 7fd8227fc700 1 -- 172.21.15.77:0/52141661 <== mgr.14120 v2:172.21.15.77:6800/1456621747 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fd830102e90 con 0x7fd81c038350 2023-12-01T01:10:05.658 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.336+0000 7fd83781e700 1 -- 172.21.15.77:0/52141661 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7fd81c038350 msgr2=0x7fd81c03a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:05.658 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.336+0000 7fd83781e700 1 --2- 172.21.15.77:0/52141661 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7fd81c038350 0x7fd81c03a810 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd82c006fd0 tx=0x7fd82c006e40 comp rx=0 tx=0).stop 2023-12-01T01:10:05.658 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.336+0000 7fd83781e700 1 -- 172.21.15.77:0/52141661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8301055c0 msgr2=0x7fd830199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:05.659 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.336+0000 7fd83781e700 1 --2- 172.21.15.77:0/52141661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8301055c0 0x7fd830199280 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd824006af0 tx=0x7fd824011ad0 comp rx=0 tx=0).stop 2023-12-01T01:10:05.659 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.336+0000 7fd83781e700 1 -- 172.21.15.77:0/52141661 shutdown_connections 2023-12-01T01:10:05.659 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.336+0000 7fd83781e700 1 --2- 172.21.15.77:0/52141661 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7fd81c038350 0x7fd81c03a810 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:05.659 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.336+0000 7fd83781e700 1 --2- 172.21.15.77:0/52141661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8301055c0 0x7fd830199280 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:05.659 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.336+0000 7fd83781e700 1 -- 172.21.15.77:0/52141661 >> 172.21.15.77:0/52141661 conn(0x7fd830100b80 msgr2=0x7fd830102270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:05.659 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.336+0000 7fd83781e700 1 -- 172.21.15.77:0/52141661 shutdown_connections 2023-12-01T01:10:05.659 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:05.336+0000 7fd83781e700 1 -- 172.21.15.77:0/52141661 wait complete. 2023-12-01T01:10:05.659 INFO:teuthology.orchestra.run.smithi077.stdout:Generating ssh key... 2023-12-01T01:10:05.659 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:05 smithi077 ceph-mon[98867]: [01/Dec/2023:01:10:03] ENGINE Serving on http://172.21.15.77:8765 2023-12-01T01:10:05.659 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:05 smithi077 ceph-mon[98867]: [01/Dec/2023:01:10:03] ENGINE Serving on https://172.21.15.77:7150 2023-12-01T01:10:05.659 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:05 smithi077 ceph-mon[98867]: [01/Dec/2023:01:10:03] ENGINE Bus STARTED 2023-12-01T01:10:05.659 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:05 smithi077 ceph-mon[98867]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:10:05.660 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:05 smithi077 ceph-mon[98867]: mgrmap e8: x(active, since 2s) 2023-12-01T01:10:06.819 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:06 smithi077 ceph-mon[98867]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:10:06.820 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:06 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: Generating public/private rsa key pair. 2023-12-01T01:10:06.820 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:06 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: Your identification has been saved in /tmp/tmpjkdq31kd/key. 2023-12-01T01:10:06.820 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:06 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: Your public key has been saved in /tmp/tmpjkdq31kd/key.pub. 2023-12-01T01:10:06.820 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:06 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: The key fingerprint is: 2023-12-01T01:10:06.820 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:06 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: SHA256:v6An9p5YWNQ0IN2YJoIa1gfSGwuyNsnaYy4fO0yfh44 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715 2023-12-01T01:10:06.820 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:06 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: The key's randomart image is: 2023-12-01T01:10:06.820 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:06 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: +---[RSA 3072]----+ 2023-12-01T01:10:06.820 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:06 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: | .oo. ...=o | 2023-12-01T01:10:06.820 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:06 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: |ooo+....=o.. | 2023-12-01T01:10:06.820 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:06 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: |++o =. o. . | 2023-12-01T01:10:06.820 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:06 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: |o= o . | 2023-12-01T01:10:06.820 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:06 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: |o.. S | 2023-12-01T01:10:06.821 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:06 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: |. = o . | 2023-12-01T01:10:06.821 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:06 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: | =.o o. o . | 2023-12-01T01:10:06.821 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:06 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: |. +++ ++.o . | 2023-12-01T01:10:06.821 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:06 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: | oEo.oo=+ . | 2023-12-01T01:10:06.821 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:06 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: +----[SHA256]-----+ 2023-12-01T01:10:06.994 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.103+0000 7ff886347700 1 Processor -- start 2023-12-01T01:10:07.000 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.103+0000 7ff886347700 1 -- start start 2023-12-01T01:10:07.000 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.103+0000 7ff886347700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff880106730 0x7ff880106b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:07.000 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.103+0000 7ff886347700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff880107120 con 0x7ff880106730 2023-12-01T01:10:07.001 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.103+0000 7ff87ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff880106730 0x7ff880106b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:07.001 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.103+0000 7ff87ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff880106730 0x7ff880106b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56296/0 (socket says 172.21.15.77:56296) 2023-12-01T01:10:07.001 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.103+0000 7ff87ffff700 1 -- 172.21.15.77:0/1129400270 learned_addr learned my addr 172.21.15.77:0/1129400270 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:07.001 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.103+0000 7ff87ffff700 1 -- 172.21.15.77:0/1129400270 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff880107930 con 0x7ff880106730 2023-12-01T01:10:07.001 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.104+0000 7ff87ffff700 1 --2- 172.21.15.77:0/1129400270 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff880106730 0x7ff880106b50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff8700098a0 tx=0x7ff870009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5f5214aa8e81a94 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:07.001 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.104+0000 7ff87effd700 1 -- 172.21.15.77:0/1129400270 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff870004160 con 0x7ff880106730 2023-12-01T01:10:07.001 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.104+0000 7ff87effd700 1 -- 172.21.15.77:0/1129400270 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff8700042c0 con 0x7ff880106730 2023-12-01T01:10:07.001 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.104+0000 7ff87effd700 1 -- 172.21.15.77:0/1129400270 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff870011a00 con 0x7ff880106730 2023-12-01T01:10:07.001 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.104+0000 7ff886347700 1 -- 172.21.15.77:0/1129400270 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff880106730 msgr2=0x7ff880106b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:07.001 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.104+0000 7ff886347700 1 --2- 172.21.15.77:0/1129400270 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff880106730 0x7ff880106b50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff8700098a0 tx=0x7ff870009bb0 comp rx=0 tx=0).stop 2023-12-01T01:10:07.001 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.104+0000 7ff886347700 1 -- 172.21.15.77:0/1129400270 shutdown_connections 2023-12-01T01:10:07.001 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.104+0000 7ff886347700 1 --2- 172.21.15.77:0/1129400270 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff880106730 0x7ff880106b50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:07.001 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.104+0000 7ff886347700 1 -- 172.21.15.77:0/1129400270 >> 172.21.15.77:0/1129400270 conn(0x7ff880101cd0 msgr2=0x7ff880104110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:07.002 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.104+0000 7ff886347700 1 -- 172.21.15.77:0/1129400270 shutdown_connections 2023-12-01T01:10:07.002 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.104+0000 7ff886347700 1 -- 172.21.15.77:0/1129400270 wait complete. 2023-12-01T01:10:07.002 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.105+0000 7ff886347700 1 Processor -- start 2023-12-01T01:10:07.002 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.105+0000 7ff886347700 1 -- start start 2023-12-01T01:10:07.002 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.106+0000 7ff886347700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff880106730 0x7ff88019a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:07.002 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.106+0000 7ff886347700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88019a8c0 con 0x7ff880106730 2023-12-01T01:10:07.002 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.106+0000 7ff87ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff880106730 0x7ff88019a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:07.002 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.106+0000 7ff87ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff880106730 0x7ff88019a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56298/0 (socket says 172.21.15.77:56298) 2023-12-01T01:10:07.002 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.106+0000 7ff87ffff700 1 -- 172.21.15.77:0/2935342980 learned_addr learned my addr 172.21.15.77:0/2935342980 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:07.002 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.106+0000 7ff87ffff700 1 -- 172.21.15.77:0/2935342980 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff870009550 con 0x7ff880106730 2023-12-01T01:10:07.002 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.106+0000 7ff87ffff700 1 --2- 172.21.15.77:0/2935342980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff880106730 0x7ff88019a380 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff870006af0 tx=0x7ff870011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:07.002 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.106+0000 7ff87d7fa700 1 -- 172.21.15.77:0/2935342980 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff870003a40 con 0x7ff880106730 2023-12-01T01:10:07.002 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.106+0000 7ff87d7fa700 1 -- 172.21.15.77:0/2935342980 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff870003ba0 con 0x7ff880106730 2023-12-01T01:10:07.002 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.106+0000 7ff886347700 1 -- 172.21.15.77:0/2935342980 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff88019aac0 con 0x7ff880106730 2023-12-01T01:10:07.003 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.106+0000 7ff87d7fa700 1 -- 172.21.15.77:0/2935342980 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff87001a700 con 0x7ff880106730 2023-12-01T01:10:07.003 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.106+0000 7ff886347700 1 -- 172.21.15.77:0/2935342980 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff88019aee0 con 0x7ff880106730 2023-12-01T01:10:07.003 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.107+0000 7ff87d7fa700 1 -- 172.21.15.77:0/2935342980 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) v1 ==== 45268+0+0 (secure 0 0 0) 0x7ff87001a930 con 0x7ff880106730 2023-12-01T01:10:07.003 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.107+0000 7ff886347700 1 -- 172.21.15.77:0/2935342980 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8800623c0 con 0x7ff880106730 2023-12-01T01:10:07.003 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.107+0000 7ff87d7fa700 1 --2- 172.21.15.77:0/2935342980 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7ff868038350 0x7ff86803a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:07.003 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.107+0000 7ff87d7fa700 1 -- 172.21.15.77:0/2935342980 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff87004c310 con 0x7ff880106730 2023-12-01T01:10:07.003 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.108+0000 7ff87f7fe700 1 --2- 172.21.15.77:0/2935342980 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7ff868038350 0x7ff86803a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:07.003 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.108+0000 7ff87f7fe700 1 --2- 172.21.15.77:0/2935342980 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7ff868038350 0x7ff86803a810 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff874006fd0 tx=0x7ff874006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:07.003 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.112+0000 7ff87d7fa700 1 -- 172.21.15.77:0/2935342980 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff870018b80 con 0x7ff880106730 2023-12-01T01:10:07.003 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.301+0000 7ff886347700 1 -- 172.21.15.77:0/2935342980 --> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7ff880103190 con 0x7ff868038350 2023-12-01T01:10:07.003 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.664+0000 7ff87d7fa700 1 -- 172.21.15.77:0/2935342980 <== mgr.14120 v2:172.21.15.77:6800/1456621747 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff880103190 con 0x7ff868038350 2023-12-01T01:10:07.003 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.666+0000 7ff886347700 1 -- 172.21.15.77:0/2935342980 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7ff868038350 msgr2=0x7ff86803a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:07.003 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.666+0000 7ff886347700 1 --2- 172.21.15.77:0/2935342980 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7ff868038350 0x7ff86803a810 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff874006fd0 tx=0x7ff874006e40 comp rx=0 tx=0).stop 2023-12-01T01:10:07.004 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.666+0000 7ff886347700 1 -- 172.21.15.77:0/2935342980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff880106730 msgr2=0x7ff88019a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:07.004 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.666+0000 7ff886347700 1 --2- 172.21.15.77:0/2935342980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff880106730 0x7ff88019a380 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff870006af0 tx=0x7ff870011ad0 comp rx=0 tx=0).stop 2023-12-01T01:10:07.004 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.667+0000 7ff886347700 1 -- 172.21.15.77:0/2935342980 shutdown_connections 2023-12-01T01:10:07.004 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.667+0000 7ff886347700 1 --2- 172.21.15.77:0/2935342980 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7ff868038350 0x7ff86803a810 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:07.004 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.667+0000 7ff886347700 1 --2- 172.21.15.77:0/2935342980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff880106730 0x7ff88019a380 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:07.004 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.667+0000 7ff886347700 1 -- 172.21.15.77:0/2935342980 >> 172.21.15.77:0/2935342980 conn(0x7ff880101cd0 msgr2=0x7ff880102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:07.004 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.667+0000 7ff886347700 1 -- 172.21.15.77:0/2935342980 shutdown_connections 2023-12-01T01:10:07.004 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:06.667+0000 7ff886347700 1 -- 172.21.15.77:0/2935342980 wait complete. 2023-12-01T01:10:07.819 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:07 smithi077 ceph-mon[98867]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:10:07.819 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:07 smithi077 ceph-mon[98867]: Generating ssh key... 2023-12-01T01:10:07.819 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:07 smithi077 ceph-mon[98867]: from='mgr.14120 172.21.15.77:0/775606565' entity='mgr.x' 2023-12-01T01:10:07.820 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:07 smithi077 ceph-mon[98867]: from='mgr.14120 172.21.15.77:0/775606565' entity='mgr.x' 2023-12-01T01:10:08.043 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC013SX4LJ7LQ6G7zvtIEvaTmfjMzkecGP59RsGSW39+s7geeA2D6DrXMxwH9zSgfGhwYCRGUewSxvrp2kAUs/Y1tVMbYbTsEryBD4/k213hpx+26Cwxf65YyoKvN6N0NGXJQsFnz3exJC8/Om0RDReWgHcmdm1z3pT/Iq4fiFfQqizsnTTNuxiSVCRl6xS20+VcnN1ZyU/VF0kvtoQb2ipvvTQ0Noh4zAUudhXWLUeidjlECK7eDg51DKb5YjLBiry4PU1BLn9p6SQpzfZwdaNRySM/YrcZUZBpr+mDXwFXjtcnS1W68tdSuFJHMAIJqBrNVVYeJK9WN4v+ibXhruLQgPOEsc6N/U0lQamftIhWA5b6/4Iqmnsnh3elsXZ0BO6mCjZvE+jRpc3RwXfr2sJdP8GHtjpkKxSvMZspJce/tkRPoFm2V3l5xgi7OITGTVD90PVtT7FdxxemugcZM3tymoWJYga0QR/vHnW1tdDGYoDeac660KpMtkmy4Rppz8= ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715 2023-12-01T01:10:08.043 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.467+0000 7f0ac6b5f700 1 Processor -- start 2023-12-01T01:10:08.044 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.467+0000 7f0ac6b5f700 1 -- start start 2023-12-01T01:10:08.044 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.467+0000 7f0ac6b5f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ac0105620 0x7f0ac0105a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:08.044 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.467+0000 7f0ac6b5f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac0106010 con 0x7f0ac0105620 2023-12-01T01:10:08.044 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.467+0000 7f0ac48fb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ac0105620 0x7f0ac0105a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:08.044 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.467+0000 7f0ac48fb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ac0105620 0x7f0ac0105a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56302/0 (socket says 172.21.15.77:56302) 2023-12-01T01:10:08.044 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.467+0000 7f0ac48fb700 1 -- 172.21.15.77:0/1934178811 learned_addr learned my addr 172.21.15.77:0/1934178811 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:08.044 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.467+0000 7f0ac48fb700 1 -- 172.21.15.77:0/1934178811 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac0106820 con 0x7f0ac0105620 2023-12-01T01:10:08.044 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.467+0000 7f0ac48fb700 1 --2- 172.21.15.77:0/1934178811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ac0105620 0x7f0ac0105a40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0ab40098a0 tx=0x7f0ab4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b10889708ae8720 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:08.044 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.468+0000 7f0abf7fe700 1 -- 172.21.15.77:0/1934178811 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ab4004160 con 0x7f0ac0105620 2023-12-01T01:10:08.044 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.468+0000 7f0abf7fe700 1 -- 172.21.15.77:0/1934178811 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0ab40042c0 con 0x7f0ac0105620 2023-12-01T01:10:08.044 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.468+0000 7f0abf7fe700 1 -- 172.21.15.77:0/1934178811 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ab4011a00 con 0x7f0ac0105620 2023-12-01T01:10:08.044 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.468+0000 7f0ac6b5f700 1 -- 172.21.15.77:0/1934178811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ac0105620 msgr2=0x7f0ac0105a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:08.044 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.468+0000 7f0ac6b5f700 1 --2- 172.21.15.77:0/1934178811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ac0105620 0x7f0ac0105a40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0ab40098a0 tx=0x7f0ab4009bb0 comp rx=0 tx=0).stop 2023-12-01T01:10:08.044 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.468+0000 7f0ac6b5f700 1 -- 172.21.15.77:0/1934178811 shutdown_connections 2023-12-01T01:10:08.045 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.468+0000 7f0ac6b5f700 1 --2- 172.21.15.77:0/1934178811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ac0105620 0x7f0ac0105a40 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:08.045 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.468+0000 7f0ac6b5f700 1 -- 172.21.15.77:0/1934178811 >> 172.21.15.77:0/1934178811 conn(0x7f0ac0100bc0 msgr2=0x7f0ac0103000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:08.045 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.468+0000 7f0ac6b5f700 1 -- 172.21.15.77:0/1934178811 shutdown_connections 2023-12-01T01:10:08.045 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.468+0000 7f0ac6b5f700 1 -- 172.21.15.77:0/1934178811 wait complete. 2023-12-01T01:10:08.045 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.469+0000 7f0ac6b5f700 1 Processor -- start 2023-12-01T01:10:08.045 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.469+0000 7f0ac6b5f700 1 -- start start 2023-12-01T01:10:08.045 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.469+0000 7f0ac6b5f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ac0105620 0x7f0ac019a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:08.045 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.469+0000 7f0ac6b5f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac019a8a0 con 0x7f0ac0105620 2023-12-01T01:10:08.045 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.469+0000 7f0ac48fb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ac0105620 0x7f0ac019a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:08.045 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.469+0000 7f0ac48fb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ac0105620 0x7f0ac019a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56304/0 (socket says 172.21.15.77:56304) 2023-12-01T01:10:08.045 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.469+0000 7f0ac48fb700 1 -- 172.21.15.77:0/727745401 learned_addr learned my addr 172.21.15.77:0/727745401 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:08.045 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.469+0000 7f0ac48fb700 1 -- 172.21.15.77:0/727745401 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab4009550 con 0x7f0ac0105620 2023-12-01T01:10:08.045 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.469+0000 7f0ac48fb700 1 --2- 172.21.15.77:0/727745401 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ac0105620 0x7f0ac019a360 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4009fb0 tx=0x7f0ab4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:08.045 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.470+0000 7f0abdffb700 1 -- 172.21.15.77:0/727745401 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ab4003930 con 0x7f0ac0105620 2023-12-01T01:10:08.045 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.470+0000 7f0abdffb700 1 -- 172.21.15.77:0/727745401 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0ab4003a90 con 0x7f0ac0105620 2023-12-01T01:10:08.046 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.470+0000 7f0ac6b5f700 1 -- 172.21.15.77:0/727745401 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac019aaa0 con 0x7f0ac0105620 2023-12-01T01:10:08.046 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.470+0000 7f0abdffb700 1 -- 172.21.15.77:0/727745401 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ab40195e0 con 0x7f0ac0105620 2023-12-01T01:10:08.046 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.470+0000 7f0ac6b5f700 1 -- 172.21.15.77:0/727745401 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac019aec0 con 0x7f0ac0105620 2023-12-01T01:10:08.046 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.471+0000 7f0abdffb700 1 -- 172.21.15.77:0/727745401 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) v1 ==== 45268+0+0 (secure 0 0 0) 0x7f0ab4019850 con 0x7f0ac0105620 2023-12-01T01:10:08.046 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.471+0000 7f0ac6b5f700 1 -- 172.21.15.77:0/727745401 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ac00623c0 con 0x7f0ac0105620 2023-12-01T01:10:08.046 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.471+0000 7f0abdffb700 1 --2- 172.21.15.77:0/727745401 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7f0ab0038300 0x7f0ab003a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:08.046 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.471+0000 7f0abdffb700 1 -- 172.21.15.77:0/727745401 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0ab404c430 con 0x7f0ac0105620 2023-12-01T01:10:08.046 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.471+0000 7f0abffff700 1 --2- 172.21.15.77:0/727745401 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7f0ab0038300 0x7f0ab003a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:08.046 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.472+0000 7f0abffff700 1 --2- 172.21.15.77:0/727745401 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7f0ab0038300 0x7f0ab003a7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0aac006fd0 tx=0x7f0aac006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:08.046 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.476+0000 7f0abdffb700 1 -- 172.21.15.77:0/727745401 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0ab402b340 con 0x7f0ac0105620 2023-12-01T01:10:08.046 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.665+0000 7f0ac6b5f700 1 -- 172.21.15.77:0/727745401 --> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ac0102d20 con 0x7f0ab0038300 2023-12-01T01:10:08.046 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.666+0000 7f0abdffb700 1 -- 172.21.15.77:0/727745401 <== mgr.14120 v2:172.21.15.77:6800/1456621747 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f0ac0102d20 con 0x7f0ab0038300 2023-12-01T01:10:08.046 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.668+0000 7f0ac6b5f700 1 -- 172.21.15.77:0/727745401 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7f0ab0038300 msgr2=0x7f0ab003a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:08.046 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.668+0000 7f0ac6b5f700 1 --2- 172.21.15.77:0/727745401 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7f0ab0038300 0x7f0ab003a7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0aac006fd0 tx=0x7f0aac006e40 comp rx=0 tx=0).stop 2023-12-01T01:10:08.047 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.668+0000 7f0ac6b5f700 1 -- 172.21.15.77:0/727745401 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ac0105620 msgr2=0x7f0ac019a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:08.047 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.668+0000 7f0ac6b5f700 1 --2- 172.21.15.77:0/727745401 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ac0105620 0x7f0ac019a360 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4009fb0 tx=0x7f0ab4011ad0 comp rx=0 tx=0).stop 2023-12-01T01:10:08.047 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.668+0000 7f0ac6b5f700 1 -- 172.21.15.77:0/727745401 shutdown_connections 2023-12-01T01:10:08.047 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.668+0000 7f0ac6b5f700 1 --2- 172.21.15.77:0/727745401 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7f0ab0038300 0x7f0ab003a7c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:08.047 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.668+0000 7f0ac6b5f700 1 --2- 172.21.15.77:0/727745401 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ac0105620 0x7f0ac019a360 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:08.047 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.668+0000 7f0ac6b5f700 1 -- 172.21.15.77:0/727745401 >> 172.21.15.77:0/727745401 conn(0x7f0ac0100bc0 msgr2=0x7f0ac00798b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:08.047 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.669+0000 7f0ac6b5f700 1 -- 172.21.15.77:0/727745401 shutdown_connections 2023-12-01T01:10:08.047 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:07.669+0000 7f0ac6b5f700 1 -- 172.21.15.77:0/727745401 wait complete. 2023-12-01T01:10:08.047 INFO:teuthology.orchestra.run.smithi077.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-12-01T01:10:08.047 INFO:teuthology.orchestra.run.smithi077.stdout:Adding key to root@localhost authorized_keys... 2023-12-01T01:10:08.047 INFO:teuthology.orchestra.run.smithi077.stdout:Adding host smithi077... 2023-12-01T01:10:08.941 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:08 smithi077 ceph-mon[98867]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:10:09.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:09 smithi077 ceph-mon[98867]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi077", "addr": "172.21.15.77", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:10:09.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:09 smithi077 ceph-mon[98867]: Deploying cephadm binary to smithi077 2023-12-01T01:10:11.391 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout Added host 'smithi077' with addr '172.21.15.77' 2023-12-01T01:10:11.391 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:08.506+0000 7f99cf6c6700 1 Processor -- start 2023-12-01T01:10:11.391 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:08.506+0000 7f99cf6c6700 1 -- start start 2023-12-01T01:10:11.391 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:08.506+0000 7f99cf6c6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99c8105aa0 0x7f99c8105ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:11.391 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:08.506+0000 7f99cf6c6700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c8106490 con 0x7f99c8105aa0 2023-12-01T01:10:11.391 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:08.506+0000 7f99cd462700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99c8105aa0 0x7f99c8105ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:11.392 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:08.506+0000 7f99cd462700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99c8105aa0 0x7f99c8105ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56308/0 (socket says 172.21.15.77:56308) 2023-12-01T01:10:11.392 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:08.506+0000 7f99cd462700 1 -- 172.21.15.77:0/4129033236 learned_addr learned my addr 172.21.15.77:0/4129033236 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:11.392 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:08.507+0000 7f99cd462700 1 -- 172.21.15.77:0/4129033236 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c81067c0 con 0x7f99c8105aa0 2023-12-01T01:10:11.392 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:08.507+0000 7f99cd462700 1 --2- 172.21.15.77:0/4129033236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99c8105aa0 0x7f99c8105ec0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f99b80098a0 tx=0x7f99b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8233a6de11542745 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:11.392 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:08.507+0000 7f99bffff700 1 -- 172.21.15.77:0/4129033236 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99b8004160 con 0x7f99c8105aa0 2023-12-01T01:10:11.392 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:08.507+0000 7f99bffff700 1 -- 172.21.15.77:0/4129033236 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f99b80042c0 con 0x7f99c8105aa0 2023-12-01T01:10:11.392 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:08.507+0000 7f99bffff700 1 -- 172.21.15.77:0/4129033236 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99b8011a00 con 0x7f99c8105aa0 2023-12-01T01:10:11.392 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:08.507+0000 7f99cf6c6700 1 -- 172.21.15.77:0/4129033236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99c8105aa0 msgr2=0x7f99c8105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:11.392 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:08.507+0000 7f99cf6c6700 1 --2- 172.21.15.77:0/4129033236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99c8105aa0 0x7f99c8105ec0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f99b80098a0 tx=0x7f99b8009bb0 comp rx=0 tx=0).stop 2023-12-01T01:10:11.392 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:08.508+0000 7f99cf6c6700 1 -- 172.21.15.77:0/4129033236 shutdown_connections 2023-12-01T01:10:11.392 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:08.508+0000 7f99cf6c6700 1 --2- 172.21.15.77:0/4129033236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99c8105aa0 0x7f99c8105ec0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:11.393 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:08.508+0000 7f99cf6c6700 1 -- 172.21.15.77:0/4129033236 >> 172.21.15.77:0/4129033236 conn(0x7f99c8100300 msgr2=0x7f99c8102720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:11.393 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:08.508+0000 7f99cf6c6700 1 -- 172.21.15.77:0/4129033236 shutdown_connections 2023-12-01T01:10:11.393 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:08.508+0000 7f99cf6c6700 1 -- 172.21.15.77:0/4129033236 wait complete. 2023-12-01T01:10:11.393 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:08.508+0000 7f99cf6c6700 1 Processor -- start 2023-12-01T01:10:11.393 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:08.508+0000 7f99cf6c6700 1 -- start start 2023-12-01T01:10:11.393 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:08.508+0000 7f99cf6c6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99c8105aa0 0x7f99c8105160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:11.393 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:08.508+0000 7f99cf6c6700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c819d080 con 0x7f99c8105aa0 2023-12-01T01:10:11.393 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:08.508+0000 7f99cd462700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99c8105aa0 0x7f99c8105160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:11.393 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:08.508+0000 7f99cd462700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99c8105aa0 0x7f99c8105160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56310/0 (socket says 172.21.15.77:56310) 2023-12-01T01:10:11.393 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:08.508+0000 7f99cd462700 1 -- 172.21.15.77:0/708931880 learned_addr learned my addr 172.21.15.77:0/708931880 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:11.393 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:08.509+0000 7f99cd462700 1 -- 172.21.15.77:0/708931880 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b8009550 con 0x7f99c8105aa0 2023-12-01T01:10:11.393 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:08.509+0000 7f99cd462700 1 --2- 172.21.15.77:0/708931880 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99c8105aa0 0x7f99c8105160 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f99b8009fb0 tx=0x7f99b8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:11.393 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:08.509+0000 7f99be7fc700 1 -- 172.21.15.77:0/708931880 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99b8003930 con 0x7f99c8105aa0 2023-12-01T01:10:11.393 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:08.509+0000 7f99cf6c6700 1 -- 172.21.15.77:0/708931880 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c819d1c0 con 0x7f99c8105aa0 2023-12-01T01:10:11.394 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:08.509+0000 7f99be7fc700 1 -- 172.21.15.77:0/708931880 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f99b8003a90 con 0x7f99c8105aa0 2023-12-01T01:10:11.394 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:08.509+0000 7f99be7fc700 1 -- 172.21.15.77:0/708931880 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99b8019610 con 0x7f99c8105aa0 2023-12-01T01:10:11.394 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:08.509+0000 7f99cf6c6700 1 -- 172.21.15.77:0/708931880 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c819d5e0 con 0x7f99c8105aa0 2023-12-01T01:10:11.394 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:08.510+0000 7f99be7fc700 1 -- 172.21.15.77:0/708931880 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) v1 ==== 45268+0+0 (secure 0 0 0) 0x7f99b80198d0 con 0x7f99c8105aa0 2023-12-01T01:10:11.394 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:08.510+0000 7f99cf6c6700 1 -- 172.21.15.77:0/708931880 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99c80623c0 con 0x7f99c8105aa0 2023-12-01T01:10:11.394 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:08.510+0000 7f99be7fc700 1 --2- 172.21.15.77:0/708931880 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7f99b40382b0 0x7f99b403a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:11.394 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:08.510+0000 7f99be7fc700 1 -- 172.21.15.77:0/708931880 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f99b804c480 con 0x7f99c8105aa0 2023-12-01T01:10:11.394 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:08.510+0000 7f99ccc61700 1 --2- 172.21.15.77:0/708931880 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7f99b40382b0 0x7f99b403a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:11.394 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:08.510+0000 7f99ccc61700 1 --2- 172.21.15.77:0/708931880 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7f99b40382b0 0x7f99b403a770 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f99c4006fd0 tx=0x7f99c4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:11.394 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:08.515+0000 7f99be7fc700 1 -- 172.21.15.77:0/708931880 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f99b802b340 con 0x7f99c8105aa0 2023-12-01T01:10:11.394 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:08.707+0000 7f99cf6c6700 1 -- 172.21.15.77:0/708931880 --> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi077", "addr": "172.21.15.77", "target": ["mon-mgr", ""]}) v1 -- 0x7f99c81099c0 con 0x7f99b40382b0 2023-12-01T01:10:11.394 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:10.975+0000 7f99be7fc700 1 -- 172.21.15.77:0/708931880 <== mgr.14120 v2:172.21.15.77:6800/1456621747 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f99c81099c0 con 0x7f99b40382b0 2023-12-01T01:10:11.394 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:10.978+0000 7f99cf6c6700 1 -- 172.21.15.77:0/708931880 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7f99b40382b0 msgr2=0x7f99b403a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:11.394 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:10.978+0000 7f99cf6c6700 1 --2- 172.21.15.77:0/708931880 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7f99b40382b0 0x7f99b403a770 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f99c4006fd0 tx=0x7f99c4006e40 comp rx=0 tx=0).stop 2023-12-01T01:10:11.395 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:10.978+0000 7f99cf6c6700 1 -- 172.21.15.77:0/708931880 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99c8105aa0 msgr2=0x7f99c8105160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:11.395 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:10.978+0000 7f99cf6c6700 1 --2- 172.21.15.77:0/708931880 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99c8105aa0 0x7f99c8105160 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f99b8009fb0 tx=0x7f99b8011ad0 comp rx=0 tx=0).stop 2023-12-01T01:10:11.395 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:10.978+0000 7f99cf6c6700 1 -- 172.21.15.77:0/708931880 shutdown_connections 2023-12-01T01:10:11.395 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:10.978+0000 7f99cf6c6700 1 --2- 172.21.15.77:0/708931880 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7f99b40382b0 0x7f99b403a770 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:11.395 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:10.978+0000 7f99cf6c6700 1 --2- 172.21.15.77:0/708931880 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99c8105aa0 0x7f99c8105160 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:11.395 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:10.978+0000 7f99cf6c6700 1 -- 172.21.15.77:0/708931880 >> 172.21.15.77:0/708931880 conn(0x7f99c8100300 msgr2=0x7f99c81019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:11.395 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:10.978+0000 7f99cf6c6700 1 -- 172.21.15.77:0/708931880 shutdown_connections 2023-12-01T01:10:11.395 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:10.978+0000 7f99cf6c6700 1 -- 172.21.15.77:0/708931880 wait complete. 2023-12-01T01:10:11.395 INFO:teuthology.orchestra.run.smithi077.stdout:Deploying unmanaged mon service... 2023-12-01T01:10:12.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:11 smithi077 ceph-mon[98867]: from='mgr.14120 172.21.15.77:0/775606565' entity='mgr.x' 2023-12-01T01:10:12.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:11 smithi077 ceph-mon[98867]: Added host smithi077 2023-12-01T01:10:12.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:11 smithi077 ceph-mon[98867]: from='mgr.14120 172.21.15.77:0/775606565' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:10:12.736 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-12-01T01:10:12.736 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:11.911+0000 7fe25b270700 1 Processor -- start 2023-12-01T01:10:12.736 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:11.911+0000 7fe25b270700 1 -- start start 2023-12-01T01:10:12.736 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:11.911+0000 7fe25b270700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe254104f60 0x7fe254105380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:12.737 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:11.911+0000 7fe25b270700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2540745c0 con 0x7fe254104f60 2023-12-01T01:10:12.737 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:11.912+0000 7fe25900c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe254104f60 0x7fe254105380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:12.737 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:11.912+0000 7fe25900c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe254104f60 0x7fe254105380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56334/0 (socket says 172.21.15.77:56334) 2023-12-01T01:10:12.737 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:11.912+0000 7fe25900c700 1 -- 172.21.15.77:0/3668492294 learned_addr learned my addr 172.21.15.77:0/3668492294 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:12.737 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:11.912+0000 7fe25900c700 1 -- 172.21.15.77:0/3668492294 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2541058c0 con 0x7fe254104f60 2023-12-01T01:10:12.737 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:11.912+0000 7fe25900c700 1 --2- 172.21.15.77:0/3668492294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe254104f60 0x7fe254105380 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe2500098a0 tx=0x7fe250009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a59f53b267eb262 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:12.737 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:11.912+0000 7fe24bfff700 1 -- 172.21.15.77:0/3668492294 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe250004160 con 0x7fe254104f60 2023-12-01T01:10:12.737 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:11.912+0000 7fe24bfff700 1 -- 172.21.15.77:0/3668492294 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe2500042c0 con 0x7fe254104f60 2023-12-01T01:10:12.737 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:11.912+0000 7fe24bfff700 1 -- 172.21.15.77:0/3668492294 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe250011a00 con 0x7fe254104f60 2023-12-01T01:10:12.737 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:11.913+0000 7fe25b270700 1 -- 172.21.15.77:0/3668492294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe254104f60 msgr2=0x7fe254105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:12.737 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:11.913+0000 7fe25b270700 1 --2- 172.21.15.77:0/3668492294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe254104f60 0x7fe254105380 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe2500098a0 tx=0x7fe250009bb0 comp rx=0 tx=0).stop 2023-12-01T01:10:12.737 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:11.913+0000 7fe25b270700 1 -- 172.21.15.77:0/3668492294 shutdown_connections 2023-12-01T01:10:12.737 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:11.913+0000 7fe25b270700 1 --2- 172.21.15.77:0/3668492294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe254104f60 0x7fe254105380 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:12.737 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:11.913+0000 7fe25b270700 1 -- 172.21.15.77:0/3668492294 >> 172.21.15.77:0/3668492294 conn(0x7fe254100b80 msgr2=0x7fe254102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:12.738 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:11.913+0000 7fe25b270700 1 -- 172.21.15.77:0/3668492294 shutdown_connections 2023-12-01T01:10:12.738 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:11.913+0000 7fe25b270700 1 -- 172.21.15.77:0/3668492294 wait complete. 2023-12-01T01:10:12.738 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:11.914+0000 7fe25b270700 1 Processor -- start 2023-12-01T01:10:12.738 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:11.914+0000 7fe25b270700 1 -- start start 2023-12-01T01:10:12.738 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:11.914+0000 7fe25b270700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe254104f60 0x7fe25419c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:12.738 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:11.914+0000 7fe25b270700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25419caa0 con 0x7fe254104f60 2023-12-01T01:10:12.738 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:11.914+0000 7fe25900c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe254104f60 0x7fe25419c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:12.738 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:11.914+0000 7fe25900c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe254104f60 0x7fe25419c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56336/0 (socket says 172.21.15.77:56336) 2023-12-01T01:10:12.738 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:11.914+0000 7fe25900c700 1 -- 172.21.15.77:0/3244956441 learned_addr learned my addr 172.21.15.77:0/3244956441 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:12.738 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:11.914+0000 7fe25900c700 1 -- 172.21.15.77:0/3244956441 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe250009550 con 0x7fe254104f60 2023-12-01T01:10:12.738 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:11.914+0000 7fe25900c700 1 --2- 172.21.15.77:0/3244956441 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe254104f60 0x7fe25419c560 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe250009fb0 tx=0x7fe250011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:12.738 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:11.915+0000 7fe24a7fc700 1 -- 172.21.15.77:0/3244956441 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe250003930 con 0x7fe254104f60 2023-12-01T01:10:12.738 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:11.915+0000 7fe24a7fc700 1 -- 172.21.15.77:0/3244956441 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe250003a90 con 0x7fe254104f60 2023-12-01T01:10:12.738 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:11.915+0000 7fe25b270700 1 -- 172.21.15.77:0/3244956441 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe25419cca0 con 0x7fe254104f60 2023-12-01T01:10:12.739 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:11.915+0000 7fe24a7fc700 1 -- 172.21.15.77:0/3244956441 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe250019610 con 0x7fe254104f60 2023-12-01T01:10:12.739 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:11.915+0000 7fe25b270700 1 -- 172.21.15.77:0/3244956441 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe25419d0c0 con 0x7fe254104f60 2023-12-01T01:10:12.739 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:11.915+0000 7fe24a7fc700 1 -- 172.21.15.77:0/3244956441 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) v1 ==== 45268+0+0 (secure 0 0 0) 0x7fe25001a9d0 con 0x7fe254104f60 2023-12-01T01:10:12.739 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:11.916+0000 7fe25b270700 1 -- 172.21.15.77:0/3244956441 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe254195ea0 con 0x7fe254104f60 2023-12-01T01:10:12.739 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:11.916+0000 7fe24a7fc700 1 --2- 172.21.15.77:0/3244956441 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7fe240038300 0x7fe24003a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:12.739 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:11.916+0000 7fe24a7fc700 1 -- 172.21.15.77:0/3244956441 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe25004c3a0 con 0x7fe254104f60 2023-12-01T01:10:12.739 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:11.916+0000 7fe25880b700 1 --2- 172.21.15.77:0/3244956441 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7fe240038300 0x7fe24003a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:12.739 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:11.916+0000 7fe25880b700 1 --2- 172.21.15.77:0/3244956441 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7fe240038300 0x7fe24003a7c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe244006fd0 tx=0x7fe244006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:12.739 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:11.921+0000 7fe24a7fc700 1 -- 172.21.15.77:0/3244956441 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe250003c00 con 0x7fe254104f60 2023-12-01T01:10:12.739 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:12.137+0000 7fe25b270700 1 -- 172.21.15.77:0/3244956441 --> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fe2540611d0 con 0x7fe240038300 2023-12-01T01:10:12.739 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:12.144+0000 7fe24a7fc700 1 -- 172.21.15.77:0/3244956441 <== mgr.14120 v2:172.21.15.77:6800/1456621747 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fe2540611d0 con 0x7fe240038300 2023-12-01T01:10:12.739 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:12.147+0000 7fe25b270700 1 -- 172.21.15.77:0/3244956441 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7fe240038300 msgr2=0x7fe24003a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:12.739 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:12.147+0000 7fe25b270700 1 --2- 172.21.15.77:0/3244956441 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7fe240038300 0x7fe24003a7c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe244006fd0 tx=0x7fe244006e40 comp rx=0 tx=0).stop 2023-12-01T01:10:12.739 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:12.147+0000 7fe25b270700 1 -- 172.21.15.77:0/3244956441 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe254104f60 msgr2=0x7fe25419c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:12.740 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:12.147+0000 7fe25b270700 1 --2- 172.21.15.77:0/3244956441 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe254104f60 0x7fe25419c560 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe250009fb0 tx=0x7fe250011ad0 comp rx=0 tx=0).stop 2023-12-01T01:10:12.740 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:12.147+0000 7fe25b270700 1 -- 172.21.15.77:0/3244956441 shutdown_connections 2023-12-01T01:10:12.740 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:12.147+0000 7fe25b270700 1 --2- 172.21.15.77:0/3244956441 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7fe240038300 0x7fe24003a7c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:12.740 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:12.147+0000 7fe25b270700 1 --2- 172.21.15.77:0/3244956441 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe254104f60 0x7fe25419c560 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:12.740 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:12.147+0000 7fe25b270700 1 -- 172.21.15.77:0/3244956441 >> 172.21.15.77:0/3244956441 conn(0x7fe254100b80 msgr2=0x7fe254190b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:12.740 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:12.147+0000 7fe25b270700 1 -- 172.21.15.77:0/3244956441 shutdown_connections 2023-12-01T01:10:12.740 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:12.147+0000 7fe25b270700 1 -- 172.21.15.77:0/3244956441 wait complete. 2023-12-01T01:10:12.740 INFO:teuthology.orchestra.run.smithi077.stdout:Deploying unmanaged mgr service... 2023-12-01T01:10:13.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:13 smithi077 ceph-mon[98867]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:10:13.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:13 smithi077 ceph-mon[98867]: Saving service mon spec with placement count:5 2023-12-01T01:10:13.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:13 smithi077 ceph-mon[98867]: from='mgr.14120 172.21.15.77:0/775606565' entity='mgr.x' 2023-12-01T01:10:13.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:13 smithi077 ceph-mon[98867]: from='mgr.14120 172.21.15.77:0/775606565' entity='mgr.x' 2023-12-01T01:10:13.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:13 smithi077 ceph-mon[98867]: from='mgr.14120 172.21.15.77:0/775606565' entity='mgr.x' 2023-12-01T01:10:13.793 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-12-01T01:10:13.794 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.211+0000 7faea7d44700 1 Processor -- start 2023-12-01T01:10:13.794 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.212+0000 7faea7d44700 1 -- start start 2023-12-01T01:10:13.794 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.212+0000 7faea7d44700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faea01055c0 0x7faea01059e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:13.794 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.212+0000 7faea7d44700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea0105fb0 con 0x7faea01055c0 2023-12-01T01:10:13.794 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.212+0000 7faea5ae0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faea01055c0 0x7faea01059e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:13.794 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.212+0000 7faea5ae0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faea01055c0 0x7faea01059e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56340/0 (socket says 172.21.15.77:56340) 2023-12-01T01:10:13.794 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.212+0000 7faea5ae0700 1 -- 172.21.15.77:0/1233110804 learned_addr learned my addr 172.21.15.77:0/1233110804 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:13.794 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.212+0000 7faea5ae0700 1 -- 172.21.15.77:0/1233110804 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea01067c0 con 0x7faea01055c0 2023-12-01T01:10:13.794 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.212+0000 7faea5ae0700 1 --2- 172.21.15.77:0/1233110804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faea01055c0 0x7faea01059e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fae9c0098a0 tx=0x7fae9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc7cb9a0ef0e6dc8 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:13.794 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.213+0000 7faea4ade700 1 -- 172.21.15.77:0/1233110804 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae9c004160 con 0x7faea01055c0 2023-12-01T01:10:13.794 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.213+0000 7faea4ade700 1 -- 172.21.15.77:0/1233110804 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae9c0042c0 con 0x7faea01055c0 2023-12-01T01:10:13.794 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.213+0000 7faea4ade700 1 -- 172.21.15.77:0/1233110804 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae9c0119b0 con 0x7faea01055c0 2023-12-01T01:10:13.794 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.213+0000 7faea7d44700 1 -- 172.21.15.77:0/1233110804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faea01055c0 msgr2=0x7faea01059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:13.794 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.213+0000 7faea7d44700 1 --2- 172.21.15.77:0/1233110804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faea01055c0 0x7faea01059e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fae9c0098a0 tx=0x7fae9c009bb0 comp rx=0 tx=0).stop 2023-12-01T01:10:13.795 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.213+0000 7faea7d44700 1 -- 172.21.15.77:0/1233110804 shutdown_connections 2023-12-01T01:10:13.795 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.213+0000 7faea7d44700 1 --2- 172.21.15.77:0/1233110804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faea01055c0 0x7faea01059e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:13.795 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.213+0000 7faea7d44700 1 -- 172.21.15.77:0/1233110804 >> 172.21.15.77:0/1233110804 conn(0x7faea0100b80 msgr2=0x7faea0102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:13.795 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.213+0000 7faea7d44700 1 -- 172.21.15.77:0/1233110804 shutdown_connections 2023-12-01T01:10:13.795 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.213+0000 7faea7d44700 1 -- 172.21.15.77:0/1233110804 wait complete. 2023-12-01T01:10:13.795 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.214+0000 7faea7d44700 1 Processor -- start 2023-12-01T01:10:13.795 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.214+0000 7faea7d44700 1 -- start start 2023-12-01T01:10:13.795 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.215+0000 7faea7d44700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faea01055c0 0x7faea00742a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:13.795 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.215+0000 7faea7d44700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea019d0d0 con 0x7faea01055c0 2023-12-01T01:10:13.795 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.215+0000 7faea5ae0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faea01055c0 0x7faea00742a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:13.795 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.215+0000 7faea5ae0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faea01055c0 0x7faea00742a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56342/0 (socket says 172.21.15.77:56342) 2023-12-01T01:10:13.795 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.215+0000 7faea5ae0700 1 -- 172.21.15.77:0/1819747211 learned_addr learned my addr 172.21.15.77:0/1819747211 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:13.796 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.215+0000 7faea5ae0700 1 -- 172.21.15.77:0/1819747211 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae9c009550 con 0x7faea01055c0 2023-12-01T01:10:13.796 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.215+0000 7faea5ae0700 1 --2- 172.21.15.77:0/1819747211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faea01055c0 0x7faea00742a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fae9c003e60 tx=0x7fae9c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:13.796 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.215+0000 7fae96ffd700 1 -- 172.21.15.77:0/1819747211 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae9c003970 con 0x7faea01055c0 2023-12-01T01:10:13.796 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.215+0000 7fae96ffd700 1 -- 172.21.15.77:0/1819747211 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae9c003ad0 con 0x7faea01055c0 2023-12-01T01:10:13.796 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.215+0000 7faea7d44700 1 -- 172.21.15.77:0/1819747211 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faea019d210 con 0x7faea01055c0 2023-12-01T01:10:13.796 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.215+0000 7fae96ffd700 1 -- 172.21.15.77:0/1819747211 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae9c0195e0 con 0x7faea01055c0 2023-12-01T01:10:13.796 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.215+0000 7faea7d44700 1 -- 172.21.15.77:0/1819747211 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faea019d630 con 0x7faea01055c0 2023-12-01T01:10:13.796 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.216+0000 7fae96ffd700 1 -- 172.21.15.77:0/1819747211 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) v1 ==== 45268+0+0 (secure 0 0 0) 0x7fae9c019890 con 0x7faea01055c0 2023-12-01T01:10:13.796 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.216+0000 7fae96ffd700 1 --2- 172.21.15.77:0/1819747211 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7fae8c038300 0x7fae8c03a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:13.796 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.216+0000 7faea7d44700 1 -- 172.21.15.77:0/1819747211 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faea01099f0 con 0x7faea01055c0 2023-12-01T01:10:13.796 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.216+0000 7fae96ffd700 1 -- 172.21.15.77:0/1819747211 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fae9c04c2c0 con 0x7faea01055c0 2023-12-01T01:10:13.796 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.217+0000 7faea52df700 1 --2- 172.21.15.77:0/1819747211 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7fae8c038300 0x7fae8c03a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:13.796 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.217+0000 7faea52df700 1 --2- 172.21.15.77:0/1819747211 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7fae8c038300 0x7fae8c03a7c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fae90006fd0 tx=0x7fae90006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:13.797 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.221+0000 7fae96ffd700 1 -- 172.21.15.77:0/1819747211 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fae9c019b90 con 0x7faea01055c0 2023-12-01T01:10:13.797 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.444+0000 7faea7d44700 1 -- 172.21.15.77:0/1819747211 --> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7faea0102e90 con 0x7fae8c038300 2023-12-01T01:10:13.797 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.451+0000 7fae96ffd700 1 -- 172.21.15.77:0/1819747211 <== mgr.14120 v2:172.21.15.77:6800/1456621747 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7faea0102e90 con 0x7fae8c038300 2023-12-01T01:10:13.797 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.454+0000 7faea7d44700 1 -- 172.21.15.77:0/1819747211 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7fae8c038300 msgr2=0x7fae8c03a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:13.797 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.454+0000 7faea7d44700 1 --2- 172.21.15.77:0/1819747211 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7fae8c038300 0x7fae8c03a7c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fae90006fd0 tx=0x7fae90006e40 comp rx=0 tx=0).stop 2023-12-01T01:10:13.797 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.454+0000 7faea7d44700 1 -- 172.21.15.77:0/1819747211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faea01055c0 msgr2=0x7faea00742a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:13.797 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.454+0000 7faea7d44700 1 --2- 172.21.15.77:0/1819747211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faea01055c0 0x7faea00742a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fae9c003e60 tx=0x7fae9c011aa0 comp rx=0 tx=0).stop 2023-12-01T01:10:13.797 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.454+0000 7faea7d44700 1 -- 172.21.15.77:0/1819747211 shutdown_connections 2023-12-01T01:10:13.797 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.454+0000 7faea7d44700 1 --2- 172.21.15.77:0/1819747211 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7fae8c038300 0x7fae8c03a7c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:13.797 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.454+0000 7faea7d44700 1 --2- 172.21.15.77:0/1819747211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faea01055c0 0x7faea00742a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:13.797 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.454+0000 7faea7d44700 1 -- 172.21.15.77:0/1819747211 >> 172.21.15.77:0/1819747211 conn(0x7faea0100b80 msgr2=0x7faea0102270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:13.797 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.454+0000 7faea7d44700 1 -- 172.21.15.77:0/1819747211 shutdown_connections 2023-12-01T01:10:13.797 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:13.454+0000 7faea7d44700 1 -- 172.21.15.77:0/1819747211 wait complete. 2023-12-01T01:10:14.837 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.237+0000 7fc946901700 1 Processor -- start 2023-12-01T01:10:14.837 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.237+0000 7fc946901700 1 -- start start 2023-12-01T01:10:14.837 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.237+0000 7fc946901700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc940104d20 0x7fc940105140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:14.837 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.237+0000 7fc946901700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc940105710 con 0x7fc940104d20 2023-12-01T01:10:14.837 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.238+0000 7fc93ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc940104d20 0x7fc940105140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:14.837 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.238+0000 7fc93ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc940104d20 0x7fc940105140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56346/0 (socket says 172.21.15.77:56346) 2023-12-01T01:10:14.837 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.238+0000 7fc93ffff700 1 -- 172.21.15.77:0/1111629665 learned_addr learned my addr 172.21.15.77:0/1111629665 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:14.837 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.238+0000 7fc93ffff700 1 -- 172.21.15.77:0/1111629665 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc940105f20 con 0x7fc940104d20 2023-12-01T01:10:14.838 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.238+0000 7fc93ffff700 1 --2- 172.21.15.77:0/1111629665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc940104d20 0x7fc940105140 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc93400aa40 tx=0x7fc93400ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d1220be4dffa419 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:14.838 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.238+0000 7fc93effd700 1 -- 172.21.15.77:0/1111629665 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc934004160 con 0x7fc940104d20 2023-12-01T01:10:14.838 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.238+0000 7fc93effd700 1 -- 172.21.15.77:0/1111629665 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc93400cdb0 con 0x7fc940104d20 2023-12-01T01:10:14.838 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.238+0000 7fc93effd700 1 -- 172.21.15.77:0/1111629665 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc934003c30 con 0x7fc940104d20 2023-12-01T01:10:14.838 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.239+0000 7fc946901700 1 -- 172.21.15.77:0/1111629665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc940104d20 msgr2=0x7fc940105140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:14.838 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.239+0000 7fc946901700 1 --2- 172.21.15.77:0/1111629665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc940104d20 0x7fc940105140 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc93400aa40 tx=0x7fc93400ad50 comp rx=0 tx=0).stop 2023-12-01T01:10:14.838 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.239+0000 7fc946901700 1 -- 172.21.15.77:0/1111629665 shutdown_connections 2023-12-01T01:10:14.838 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.239+0000 7fc946901700 1 --2- 172.21.15.77:0/1111629665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc940104d20 0x7fc940105140 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:14.838 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.239+0000 7fc946901700 1 -- 172.21.15.77:0/1111629665 >> 172.21.15.77:0/1111629665 conn(0x7fc940100300 msgr2=0x7fc940102720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:14.838 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.239+0000 7fc946901700 1 -- 172.21.15.77:0/1111629665 shutdown_connections 2023-12-01T01:10:14.838 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.239+0000 7fc946901700 1 -- 172.21.15.77:0/1111629665 wait complete. 2023-12-01T01:10:14.839 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.239+0000 7fc946901700 1 Processor -- start 2023-12-01T01:10:14.839 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.239+0000 7fc946901700 1 -- start start 2023-12-01T01:10:14.839 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.240+0000 7fc946901700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc940104d20 0x7fc94019a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:14.839 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.240+0000 7fc946901700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94019a8c0 con 0x7fc940104d20 2023-12-01T01:10:14.839 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.240+0000 7fc93ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc940104d20 0x7fc94019a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:14.839 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.240+0000 7fc93ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc940104d20 0x7fc94019a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56348/0 (socket says 172.21.15.77:56348) 2023-12-01T01:10:14.839 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.240+0000 7fc93ffff700 1 -- 172.21.15.77:0/686749740 learned_addr learned my addr 172.21.15.77:0/686749740 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:14.839 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.240+0000 7fc93ffff700 1 -- 172.21.15.77:0/686749740 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc93400a6f0 con 0x7fc940104d20 2023-12-01T01:10:14.839 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.240+0000 7fc93ffff700 1 --2- 172.21.15.77:0/686749740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc940104d20 0x7fc94019a380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc9340046e0 tx=0x7fc934004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:14.839 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.240+0000 7fc93d7fa700 1 -- 172.21.15.77:0/686749740 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc934010520 con 0x7fc940104d20 2023-12-01T01:10:14.839 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.240+0000 7fc93d7fa700 1 -- 172.21.15.77:0/686749740 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc934010b40 con 0x7fc940104d20 2023-12-01T01:10:14.839 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.240+0000 7fc946901700 1 -- 172.21.15.77:0/686749740 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc94019aac0 con 0x7fc940104d20 2023-12-01T01:10:14.839 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.240+0000 7fc93d7fa700 1 -- 172.21.15.77:0/686749740 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc934021aa0 con 0x7fc940104d20 2023-12-01T01:10:14.840 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.241+0000 7fc946901700 1 -- 172.21.15.77:0/686749740 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc94019aee0 con 0x7fc940104d20 2023-12-01T01:10:14.840 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.242+0000 7fc946901700 1 -- 172.21.15.77:0/686749740 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc940193c80 con 0x7fc940104d20 2023-12-01T01:10:14.840 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.242+0000 7fc93d7fa700 1 -- 172.21.15.77:0/686749740 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) v1 ==== 45268+0+0 (secure 0 0 0) 0x7fc934010cb0 con 0x7fc940104d20 2023-12-01T01:10:14.840 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.242+0000 7fc93d7fa700 1 --2- 172.21.15.77:0/686749740 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7fc928038350 0x7fc92803a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:14.840 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.242+0000 7fc93d7fa700 1 -- 172.21.15.77:0/686749740 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc93404c8e0 con 0x7fc940104d20 2023-12-01T01:10:14.840 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.243+0000 7fc93f7fe700 1 --2- 172.21.15.77:0/686749740 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7fc928038350 0x7fc92803a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:14.840 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.243+0000 7fc93f7fe700 1 --2- 172.21.15.77:0/686749740 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7fc928038350 0x7fc92803a810 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc930006fd0 tx=0x7fc930006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:14.840 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.247+0000 7fc93d7fa700 1 -- 172.21.15.77:0/686749740 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc9340213f0 con 0x7fc940104d20 2023-12-01T01:10:14.840 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.448+0000 7fc946901700 1 -- 172.21.15.77:0/686749740 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fc94002cef0 con 0x7fc940104d20 2023-12-01T01:10:14.840 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.452+0000 7fc93d7fa700 1 -- 172.21.15.77:0/686749740 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7fc934026030 con 0x7fc940104d20 2023-12-01T01:10:14.840 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.454+0000 7fc946901700 1 -- 172.21.15.77:0/686749740 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7fc928038350 msgr2=0x7fc92803a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:14.840 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.454+0000 7fc946901700 1 --2- 172.21.15.77:0/686749740 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7fc928038350 0x7fc92803a810 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc930006fd0 tx=0x7fc930006e40 comp rx=0 tx=0).stop 2023-12-01T01:10:14.841 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.454+0000 7fc946901700 1 -- 172.21.15.77:0/686749740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc940104d20 msgr2=0x7fc94019a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:14.841 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.454+0000 7fc946901700 1 --2- 172.21.15.77:0/686749740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc940104d20 0x7fc94019a380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc9340046e0 tx=0x7fc934004180 comp rx=0 tx=0).stop 2023-12-01T01:10:14.841 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.454+0000 7fc946901700 1 -- 172.21.15.77:0/686749740 shutdown_connections 2023-12-01T01:10:14.841 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.454+0000 7fc946901700 1 --2- 172.21.15.77:0/686749740 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7fc928038350 0x7fc92803a810 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:14.841 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.454+0000 7fc946901700 1 --2- 172.21.15.77:0/686749740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc940104d20 0x7fc94019a380 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:14.841 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.454+0000 7fc946901700 1 -- 172.21.15.77:0/686749740 >> 172.21.15.77:0/686749740 conn(0x7fc940100300 msgr2=0x7fc94018c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:14.841 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.454+0000 7fc946901700 1 -- 172.21.15.77:0/686749740 shutdown_connections 2023-12-01T01:10:14.841 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:14.454+0000 7fc946901700 1 -- 172.21.15.77:0/686749740 wait complete. 2023-12-01T01:10:14.842 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:14 smithi077 ceph-mon[98867]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:10:14.842 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:14 smithi077 ceph-mon[98867]: Saving service mgr spec with placement count:2 2023-12-01T01:10:14.842 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:14 smithi077 ceph-mon[98867]: from='mgr.14120 172.21.15.77:0/775606565' entity='mgr.x' 2023-12-01T01:10:15.635 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:15 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/686749740' entity='client.admin' 2023-12-01T01:10:15.635 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:15 smithi077 ceph-mon[98867]: from='mgr.14120 172.21.15.77:0/775606565' entity='mgr.x' 2023-12-01T01:10:15.879 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.289+0000 7ff49ebfe700 1 Processor -- start 2023-12-01T01:10:15.879 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.289+0000 7ff49ebfe700 1 -- start start 2023-12-01T01:10:15.879 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.290+0000 7ff49ebfe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff49807adc0 0x7ff498079220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:15.879 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.290+0000 7ff49ebfe700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4980797f0 con 0x7ff49807adc0 2023-12-01T01:10:15.879 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.290+0000 7ff49c99a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff49807adc0 0x7ff498079220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:15.880 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.290+0000 7ff49c99a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff49807adc0 0x7ff498079220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56352/0 (socket says 172.21.15.77:56352) 2023-12-01T01:10:15.880 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.290+0000 7ff49c99a700 1 -- 172.21.15.77:0/2730842266 learned_addr learned my addr 172.21.15.77:0/2730842266 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:15.880 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.290+0000 7ff49c99a700 1 -- 172.21.15.77:0/2730842266 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff498079930 con 0x7ff49807adc0 2023-12-01T01:10:15.880 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.290+0000 7ff49c99a700 1 --2- 172.21.15.77:0/2730842266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff49807adc0 0x7ff498079220 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff48c0098a0 tx=0x7ff48c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a51caaab65a18cfb server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:15.880 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.290+0000 7ff4977fe700 1 -- 172.21.15.77:0/2730842266 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff48c004160 con 0x7ff49807adc0 2023-12-01T01:10:15.880 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.290+0000 7ff4977fe700 1 -- 172.21.15.77:0/2730842266 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff48c0042c0 con 0x7ff49807adc0 2023-12-01T01:10:15.880 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.290+0000 7ff4977fe700 1 -- 172.21.15.77:0/2730842266 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff48c0119b0 con 0x7ff49807adc0 2023-12-01T01:10:15.880 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.291+0000 7ff49ebfe700 1 -- 172.21.15.77:0/2730842266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff49807adc0 msgr2=0x7ff498079220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:15.880 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.291+0000 7ff49ebfe700 1 --2- 172.21.15.77:0/2730842266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff49807adc0 0x7ff498079220 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff48c0098a0 tx=0x7ff48c009bb0 comp rx=0 tx=0).stop 2023-12-01T01:10:15.880 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.291+0000 7ff49ebfe700 1 -- 172.21.15.77:0/2730842266 shutdown_connections 2023-12-01T01:10:15.880 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.291+0000 7ff49ebfe700 1 --2- 172.21.15.77:0/2730842266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff49807adc0 0x7ff498079220 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:15.880 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.291+0000 7ff49ebfe700 1 -- 172.21.15.77:0/2730842266 >> 172.21.15.77:0/2730842266 conn(0x7ff498101cd0 msgr2=0x7ff498104110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:15.880 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.291+0000 7ff49ebfe700 1 -- 172.21.15.77:0/2730842266 shutdown_connections 2023-12-01T01:10:15.880 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.291+0000 7ff49ebfe700 1 -- 172.21.15.77:0/2730842266 wait complete. 2023-12-01T01:10:15.881 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.292+0000 7ff49ebfe700 1 Processor -- start 2023-12-01T01:10:15.881 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.292+0000 7ff49ebfe700 1 -- start start 2023-12-01T01:10:15.881 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.292+0000 7ff49ebfe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff49807adc0 0x7ff49819c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:15.881 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.292+0000 7ff49ebfe700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49819cae0 con 0x7ff49807adc0 2023-12-01T01:10:15.881 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.292+0000 7ff49c99a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff49807adc0 0x7ff49819c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:15.881 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.292+0000 7ff49c99a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff49807adc0 0x7ff49819c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56354/0 (socket says 172.21.15.77:56354) 2023-12-01T01:10:15.881 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.292+0000 7ff49c99a700 1 -- 172.21.15.77:0/2094886142 learned_addr learned my addr 172.21.15.77:0/2094886142 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:15.881 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.293+0000 7ff49c99a700 1 -- 172.21.15.77:0/2094886142 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff48c009550 con 0x7ff49807adc0 2023-12-01T01:10:15.881 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.293+0000 7ff49c99a700 1 --2- 172.21.15.77:0/2094886142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff49807adc0 0x7ff49819c5a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff48c000c00 tx=0x7ff48c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:15.881 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.293+0000 7ff495ffb700 1 -- 172.21.15.77:0/2094886142 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff48c0039f0 con 0x7ff49807adc0 2023-12-01T01:10:15.881 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.293+0000 7ff495ffb700 1 -- 172.21.15.77:0/2094886142 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff48c003b50 con 0x7ff49807adc0 2023-12-01T01:10:15.881 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.293+0000 7ff49ebfe700 1 -- 172.21.15.77:0/2094886142 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff49819cce0 con 0x7ff49807adc0 2023-12-01T01:10:15.881 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.293+0000 7ff495ffb700 1 -- 172.21.15.77:0/2094886142 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff48c01a780 con 0x7ff49807adc0 2023-12-01T01:10:15.881 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.293+0000 7ff49ebfe700 1 -- 172.21.15.77:0/2094886142 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff49819d100 con 0x7ff49807adc0 2023-12-01T01:10:15.881 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.294+0000 7ff495ffb700 1 -- 172.21.15.77:0/2094886142 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) v1 ==== 45268+0+0 (secure 0 0 0) 0x7ff48c01aa40 con 0x7ff49807adc0 2023-12-01T01:10:15.882 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.294+0000 7ff495ffb700 1 --2- 172.21.15.77:0/2094886142 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7ff488038300 0x7ff48803a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:15.882 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.294+0000 7ff495ffb700 1 -- 172.21.15.77:0/2094886142 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff48c04c1e0 con 0x7ff49807adc0 2023-12-01T01:10:15.882 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.294+0000 7ff49ebfe700 1 -- 172.21.15.77:0/2094886142 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4980623c0 con 0x7ff49807adc0 2023-12-01T01:10:15.882 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.294+0000 7ff497fff700 1 --2- 172.21.15.77:0/2094886142 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7ff488038300 0x7ff48803a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:15.882 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.294+0000 7ff497fff700 1 --2- 172.21.15.77:0/2094886142 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7ff488038300 0x7ff48803a7c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff484006fd0 tx=0x7ff484006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:15.882 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.299+0000 7ff495ffb700 1 -- 172.21.15.77:0/2094886142 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff48c01acf0 con 0x7ff49807adc0 2023-12-01T01:10:15.882 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.502+0000 7ff49ebfe700 1 -- 172.21.15.77:0/2094886142 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7ff49819fdd0 con 0x7ff49807adc0 2023-12-01T01:10:15.882 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.508+0000 7ff495ffb700 1 -- 172.21.15.77:0/2094886142 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7ff48c0172b0 con 0x7ff49807adc0 2023-12-01T01:10:15.882 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.511+0000 7ff49ebfe700 1 -- 172.21.15.77:0/2094886142 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7ff488038300 msgr2=0x7ff48803a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:15.882 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.511+0000 7ff49ebfe700 1 --2- 172.21.15.77:0/2094886142 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7ff488038300 0x7ff48803a7c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff484006fd0 tx=0x7ff484006e40 comp rx=0 tx=0).stop 2023-12-01T01:10:15.882 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.511+0000 7ff49ebfe700 1 -- 172.21.15.77:0/2094886142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff49807adc0 msgr2=0x7ff49819c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:15.882 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.511+0000 7ff49ebfe700 1 --2- 172.21.15.77:0/2094886142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff49807adc0 0x7ff49819c5a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff48c000c00 tx=0x7ff48c011a30 comp rx=0 tx=0).stop 2023-12-01T01:10:15.882 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.511+0000 7ff49ebfe700 1 -- 172.21.15.77:0/2094886142 shutdown_connections 2023-12-01T01:10:15.883 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.511+0000 7ff49ebfe700 1 --2- 172.21.15.77:0/2094886142 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7ff488038300 0x7ff48803a7c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:15.883 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.511+0000 7ff49ebfe700 1 --2- 172.21.15.77:0/2094886142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff49807adc0 0x7ff49819c5a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:15.883 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.511+0000 7ff49ebfe700 1 -- 172.21.15.77:0/2094886142 >> 172.21.15.77:0/2094886142 conn(0x7ff498101cd0 msgr2=0x7ff498102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:15.883 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.511+0000 7ff49ebfe700 1 -- 172.21.15.77:0/2094886142 shutdown_connections 2023-12-01T01:10:15.883 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:15.512+0000 7ff49ebfe700 1 -- 172.21.15.77:0/2094886142 wait complete. 2023-12-01T01:10:15.883 INFO:teuthology.orchestra.run.smithi077.stdout:Enabling the dashboard module... 2023-12-01T01:10:16.783 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:16 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/2094886142' entity='client.admin' 2023-12-01T01:10:16.783 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:16 smithi077 ceph-mon[98867]: from='mgr.14120 172.21.15.77:0/775606565' entity='mgr.x' 2023-12-01T01:10:17.776 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:17 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/1214844095' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-12-01T01:10:17.776 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:17 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: ignoring --setuser ceph since I am not root 2023-12-01T01:10:17.776 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:17 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: ignoring --setgroup ceph since I am not root 2023-12-01T01:10:17.776 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:17 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:10:17.694+0000 7f748acf4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T01:10:17.776 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:17 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:10:17.774+0000 7f748acf4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T01:10:17.934 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:16.375+0000 7f2809d16700 1 Processor -- start 2023-12-01T01:10:17.934 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:16.375+0000 7f2809d16700 1 -- start start 2023-12-01T01:10:17.934 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:16.375+0000 7f2809d16700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28041070e0 0x7f28041094d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:17.934 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:16.375+0000 7f2809d16700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28040745c0 con 0x7f28041070e0 2023-12-01T01:10:17.934 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:16.375+0000 7f28037fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28041070e0 0x7f28041094d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:17.934 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:16.375+0000 7f28037fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28041070e0 0x7f28041094d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56358/0 (socket says 172.21.15.77:56358) 2023-12-01T01:10:17.934 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:16.375+0000 7f28037fe700 1 -- 172.21.15.77:0/1505785815 learned_addr learned my addr 172.21.15.77:0/1505785815 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:17.935 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:16.375+0000 7f28037fe700 1 -- 172.21.15.77:0/1505785815 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2804109a10 con 0x7f28041070e0 2023-12-01T01:10:17.935 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:16.376+0000 7f28037fe700 1 --2- 172.21.15.77:0/1505785815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28041070e0 0x7f28041094d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f27f40098a0 tx=0x7f27f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2a8b769baa3ab37 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:17.935 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:16.376+0000 7f28027fc700 1 -- 172.21.15.77:0/1505785815 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27f4004160 con 0x7f28041070e0 2023-12-01T01:10:17.935 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:16.376+0000 7f28027fc700 1 -- 172.21.15.77:0/1505785815 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27f40042c0 con 0x7f28041070e0 2023-12-01T01:10:17.935 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:16.376+0000 7f28027fc700 1 -- 172.21.15.77:0/1505785815 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27f40119b0 con 0x7f28041070e0 2023-12-01T01:10:17.935 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:16.376+0000 7f2809d16700 1 -- 172.21.15.77:0/1505785815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28041070e0 msgr2=0x7f28041094d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:17.935 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:16.376+0000 7f2809d16700 1 --2- 172.21.15.77:0/1505785815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28041070e0 0x7f28041094d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f27f40098a0 tx=0x7f27f4009bb0 comp rx=0 tx=0).stop 2023-12-01T01:10:17.935 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:16.376+0000 7f2809d16700 1 -- 172.21.15.77:0/1505785815 shutdown_connections 2023-12-01T01:10:17.935 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:16.376+0000 7f2809d16700 1 --2- 172.21.15.77:0/1505785815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28041070e0 0x7f28041094d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:17.935 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:16.376+0000 7f2809d16700 1 -- 172.21.15.77:0/1505785815 >> 172.21.15.77:0/1505785815 conn(0x7f2804100ab0 msgr2=0x7f2804102f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:17.935 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:16.376+0000 7f2809d16700 1 -- 172.21.15.77:0/1505785815 shutdown_connections 2023-12-01T01:10:17.935 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:16.376+0000 7f2809d16700 1 -- 172.21.15.77:0/1505785815 wait complete. 2023-12-01T01:10:17.935 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:16.377+0000 7f2809d16700 1 Processor -- start 2023-12-01T01:10:17.936 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:16.377+0000 7f2809d16700 1 -- start start 2023-12-01T01:10:17.936 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:16.377+0000 7f2809d16700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28041070e0 0x7f280419a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:17.936 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:16.377+0000 7f2809d16700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280419a7d0 con 0x7f28041070e0 2023-12-01T01:10:17.936 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:16.377+0000 7f28037fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28041070e0 0x7f280419a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:17.936 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:16.377+0000 7f28037fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28041070e0 0x7f280419a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56360/0 (socket says 172.21.15.77:56360) 2023-12-01T01:10:17.936 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:16.377+0000 7f28037fe700 1 -- 172.21.15.77:0/1214844095 learned_addr learned my addr 172.21.15.77:0/1214844095 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:17.936 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:16.378+0000 7f28037fe700 1 -- 172.21.15.77:0/1214844095 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f4009550 con 0x7f28041070e0 2023-12-01T01:10:17.936 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:16.378+0000 7f28037fe700 1 --2- 172.21.15.77:0/1214844095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28041070e0 0x7f280419a290 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f27f4011980 tx=0x7f27f4011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:17.936 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:16.378+0000 7f2800ff9700 1 -- 172.21.15.77:0/1214844095 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27f4003a60 con 0x7f28041070e0 2023-12-01T01:10:17.936 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:16.378+0000 7f2800ff9700 1 -- 172.21.15.77:0/1214844095 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27f4003bc0 con 0x7f28041070e0 2023-12-01T01:10:17.936 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:16.378+0000 7f2809d16700 1 -- 172.21.15.77:0/1214844095 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f280419a9d0 con 0x7f28041070e0 2023-12-01T01:10:17.936 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:16.378+0000 7f2800ff9700 1 -- 172.21.15.77:0/1214844095 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27f401a670 con 0x7f28041070e0 2023-12-01T01:10:17.936 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:16.378+0000 7f2809d16700 1 -- 172.21.15.77:0/1214844095 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f280419ae70 con 0x7f28041070e0 2023-12-01T01:10:17.937 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:16.379+0000 7f2800ff9700 1 -- 172.21.15.77:0/1214844095 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) v1 ==== 45268+0+0 (secure 0 0 0) 0x7f27f401a930 con 0x7f28041070e0 2023-12-01T01:10:17.937 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:16.379+0000 7f2800ff9700 1 --2- 172.21.15.77:0/1214844095 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7f27ec038300 0x7f27ec03a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:17.937 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:16.379+0000 7f2809d16700 1 -- 172.21.15.77:0/1214844095 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f280410a7f0 con 0x7f28041070e0 2023-12-01T01:10:17.937 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:16.379+0000 7f2800ff9700 1 -- 172.21.15.77:0/1214844095 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f27f404c410 con 0x7f28041070e0 2023-12-01T01:10:17.937 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:16.379+0000 7f2802ffd700 1 --2- 172.21.15.77:0/1214844095 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7f27ec038300 0x7f27ec03a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:17.937 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:16.379+0000 7f2802ffd700 1 --2- 172.21.15.77:0/1214844095 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7f27ec038300 0x7f27ec03a7c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f27f8006fd0 tx=0x7f27f8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:17.937 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:16.384+0000 7f2800ff9700 1 -- 172.21.15.77:0/1214844095 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f27f401abe0 con 0x7f28041070e0 2023-12-01T01:10:17.937 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:16.645+0000 7f2809d16700 1 -- 172.21.15.77:0/1214844095 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f280410aa10 con 0x7f28041070e0 2023-12-01T01:10:17.937 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:17.517+0000 7f2800ff9700 1 -- 172.21.15.77:0/1214844095 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mgrmap(e 9) v1 ==== 45281+0+0 (secure 0 0 0) 0x7f27f402d910 con 0x7f28041070e0 2023-12-01T01:10:17.937 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:17.518+0000 7f2800ff9700 1 -- 172.21.15.77:0/1214844095 <== mon.0 v2:172.21.15.77:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f27f4018db0 con 0x7f28041070e0 2023-12-01T01:10:17.937 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:17.521+0000 7f2809d16700 1 -- 172.21.15.77:0/1214844095 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7f27ec038300 msgr2=0x7f27ec03a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:17.937 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:17.521+0000 7f2809d16700 1 --2- 172.21.15.77:0/1214844095 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7f27ec038300 0x7f27ec03a7c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f27f8006fd0 tx=0x7f27f8006e40 comp rx=0 tx=0).stop 2023-12-01T01:10:17.937 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:17.521+0000 7f2809d16700 1 -- 172.21.15.77:0/1214844095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28041070e0 msgr2=0x7f280419a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:17.938 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:17.521+0000 7f2809d16700 1 --2- 172.21.15.77:0/1214844095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28041070e0 0x7f280419a290 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f27f4011980 tx=0x7f27f4011a60 comp rx=0 tx=0).stop 2023-12-01T01:10:17.938 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:17.521+0000 7f2809d16700 1 -- 172.21.15.77:0/1214844095 shutdown_connections 2023-12-01T01:10:17.938 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:17.521+0000 7f2809d16700 1 --2- 172.21.15.77:0/1214844095 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7f27ec038300 0x7f27ec03a7c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:17.938 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:17.521+0000 7f2809d16700 1 --2- 172.21.15.77:0/1214844095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28041070e0 0x7f280419a290 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:17.938 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:17.521+0000 7f2809d16700 1 -- 172.21.15.77:0/1214844095 >> 172.21.15.77:0/1214844095 conn(0x7f2804100ab0 msgr2=0x7f2804101850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:17.938 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:17.521+0000 7f2809d16700 1 -- 172.21.15.77:0/1214844095 shutdown_connections 2023-12-01T01:10:17.938 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:17.521+0000 7f2809d16700 1 -- 172.21.15.77:0/1214844095 wait complete. 2023-12-01T01:10:18.138 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:17 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:10:17.865+0000 7f748acf4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T01:10:18.138 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:17 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:10:17.965+0000 7f748acf4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T01:10:18.404 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:18 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:10:18.142+0000 7f748acf4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T01:10:18.843 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:18 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/1214844095' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-12-01T01:10:18.844 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:18 smithi077 ceph-mon[98867]: mgrmap e9: x(active, since 15s) 2023-12-01T01:10:18.844 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:18 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:10:18.531+0000 7f748acf4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T01:10:18.844 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:18 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:10:18.663+0000 7f748acf4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T01:10:18.844 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:18 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:10:18.756+0000 7f748acf4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T01:10:19.076 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout { 2023-12-01T01:10:19.076 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-12-01T01:10:19.076 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "available": true, 2023-12-01T01:10:19.076 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-12-01T01:10:19.076 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-12-01T01:10:19.076 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout } 2023-12-01T01:10:19.077 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.412+0000 7f615d81d700 1 Processor -- start 2023-12-01T01:10:19.077 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.412+0000 7f615d81d700 1 -- start start 2023-12-01T01:10:19.077 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.413+0000 7f615d81d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6158107210 0x7f6158109600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:19.077 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.413+0000 7f615d81d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6158079e40 con 0x7f6158107210 2023-12-01T01:10:19.077 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.413+0000 7f6156ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6158107210 0x7f6158109600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:19.077 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.413+0000 7f6156ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6158107210 0x7f6158109600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56374/0 (socket says 172.21.15.77:56374) 2023-12-01T01:10:19.077 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.413+0000 7f6156ffd700 1 -- 172.21.15.77:0/1467225929 learned_addr learned my addr 172.21.15.77:0/1467225929 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:19.077 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.413+0000 7f6156ffd700 1 -- 172.21.15.77:0/1467225929 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6158109b40 con 0x7f6158107210 2023-12-01T01:10:19.077 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.413+0000 7f6156ffd700 1 --2- 172.21.15.77:0/1467225929 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6158107210 0x7f6158109600 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f61400098a0 tx=0x7f6140009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3afb292b92d4af0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:19.077 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.413+0000 7f6155ffb700 1 -- 172.21.15.77:0/1467225929 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6140004160 con 0x7f6158107210 2023-12-01T01:10:19.077 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.413+0000 7f6155ffb700 1 -- 172.21.15.77:0/1467225929 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61400042c0 con 0x7f6158107210 2023-12-01T01:10:19.077 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.413+0000 7f6155ffb700 1 -- 172.21.15.77:0/1467225929 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f61400119b0 con 0x7f6158107210 2023-12-01T01:10:19.077 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.414+0000 7f615d81d700 1 -- 172.21.15.77:0/1467225929 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6158107210 msgr2=0x7f6158109600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:19.077 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.414+0000 7f615d81d700 1 --2- 172.21.15.77:0/1467225929 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6158107210 0x7f6158109600 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f61400098a0 tx=0x7f6140009bb0 comp rx=0 tx=0).stop 2023-12-01T01:10:19.077 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.414+0000 7f615d81d700 1 -- 172.21.15.77:0/1467225929 shutdown_connections 2023-12-01T01:10:19.078 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.414+0000 7f615d81d700 1 --2- 172.21.15.77:0/1467225929 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6158107210 0x7f6158109600 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:19.078 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.414+0000 7f615d81d700 1 -- 172.21.15.77:0/1467225929 >> 172.21.15.77:0/1467225929 conn(0x7f6158100bc0 msgr2=0x7f6158103000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:19.078 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.414+0000 7f615d81d700 1 -- 172.21.15.77:0/1467225929 shutdown_connections 2023-12-01T01:10:19.078 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.414+0000 7f615d81d700 1 -- 172.21.15.77:0/1467225929 wait complete. 2023-12-01T01:10:19.078 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.415+0000 7f615d81d700 1 Processor -- start 2023-12-01T01:10:19.078 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.415+0000 7f615d81d700 1 -- start start 2023-12-01T01:10:19.078 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.415+0000 7f615d81d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6158107210 0x7f615819a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:19.078 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.415+0000 7f615d81d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f615819a860 con 0x7f6158107210 2023-12-01T01:10:19.078 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.415+0000 7f6156ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6158107210 0x7f615819a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:19.078 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.416+0000 7f6156ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6158107210 0x7f615819a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56376/0 (socket says 172.21.15.77:56376) 2023-12-01T01:10:19.078 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.416+0000 7f6156ffd700 1 -- 172.21.15.77:0/1849681309 learned_addr learned my addr 172.21.15.77:0/1849681309 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:19.078 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.416+0000 7f6156ffd700 1 -- 172.21.15.77:0/1849681309 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6140009550 con 0x7f6158107210 2023-12-01T01:10:19.078 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.416+0000 7f6156ffd700 1 --2- 172.21.15.77:0/1849681309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6158107210 0x7f615819a320 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6140003e60 tx=0x7f6140011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:19.079 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.416+0000 7f615c81b700 1 -- 172.21.15.77:0/1849681309 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6140003970 con 0x7f6158107210 2023-12-01T01:10:19.079 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.416+0000 7f615c81b700 1 -- 172.21.15.77:0/1849681309 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6140003ad0 con 0x7f6158107210 2023-12-01T01:10:19.079 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.416+0000 7f615c81b700 1 -- 172.21.15.77:0/1849681309 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f61400195e0 con 0x7f6158107210 2023-12-01T01:10:19.079 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.416+0000 7f615d81d700 1 -- 172.21.15.77:0/1849681309 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f615819aa60 con 0x7f6158107210 2023-12-01T01:10:19.079 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.416+0000 7f615d81d700 1 -- 172.21.15.77:0/1849681309 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f615819ae80 con 0x7f6158107210 2023-12-01T01:10:19.079 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.417+0000 7f615c81b700 1 -- 172.21.15.77:0/1849681309 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 9) v1 ==== 45281+0+0 (secure 0 0 0) 0x7f61400198a0 con 0x7f6158107210 2023-12-01T01:10:19.080 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.417+0000 7f615c81b700 1 --2- 172.21.15.77:0/1849681309 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7f6144038300 0x7f614403a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:19.080 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.417+0000 7f615c81b700 1 -- 172.21.15.77:0/1849681309 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f614004c3e0 con 0x7f6158107210 2023-12-01T01:10:19.080 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.417+0000 7f61567fc700 1 -- 172.21.15.77:0/1849681309 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7f6144038300 msgr2=0x7f614403a7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/1456621747 2023-12-01T01:10:19.081 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.417+0000 7f61567fc700 1 --2- 172.21.15.77:0/1849681309 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7f6144038300 0x7f614403a7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:10:19.081 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.418+0000 7f615d81d700 1 -- 172.21.15.77:0/1849681309 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f615810a8d0 con 0x7f6158107210 2023-12-01T01:10:19.081 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.427+0000 7f615c81b700 1 -- 172.21.15.77:0/1849681309 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f614002b340 con 0x7f6158107210 2023-12-01T01:10:19.081 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.618+0000 7f61567fc700 1 -- 172.21.15.77:0/1849681309 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7f6144038300 msgr2=0x7f614403a7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/1456621747 2023-12-01T01:10:19.081 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.618+0000 7f61567fc700 1 --2- 172.21.15.77:0/1849681309 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7f6144038300 0x7f614403a7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:10:19.081 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.683+0000 7f615d81d700 1 -- 172.21.15.77:0/1849681309 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f615819dce0 con 0x7f6158107210 2023-12-01T01:10:19.081 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.683+0000 7f615c81b700 1 -- 172.21.15.77:0/1849681309 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f6140018b40 con 0x7f6158107210 2023-12-01T01:10:19.081 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.685+0000 7f615d81d700 1 -- 172.21.15.77:0/1849681309 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7f6144038300 msgr2=0x7f614403a7c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:10:19.081 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.685+0000 7f615d81d700 1 --2- 172.21.15.77:0/1849681309 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7f6144038300 0x7f614403a7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:19.081 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.685+0000 7f615d81d700 1 -- 172.21.15.77:0/1849681309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6158107210 msgr2=0x7f615819a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:19.081 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.685+0000 7f615d81d700 1 --2- 172.21.15.77:0/1849681309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6158107210 0x7f615819a320 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6140003e60 tx=0x7f6140011aa0 comp rx=0 tx=0).stop 2023-12-01T01:10:19.081 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.685+0000 7f615d81d700 1 -- 172.21.15.77:0/1849681309 shutdown_connections 2023-12-01T01:10:19.081 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.685+0000 7f615d81d700 1 --2- 172.21.15.77:0/1849681309 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7f6144038300 0x7f614403a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:19.081 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.685+0000 7f615d81d700 1 --2- 172.21.15.77:0/1849681309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6158107210 0x7f615819a320 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:19.082 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.685+0000 7f615d81d700 1 -- 172.21.15.77:0/1849681309 >> 172.21.15.77:0/1849681309 conn(0x7f6158100bc0 msgr2=0x7f6158101930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:19.082 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.686+0000 7f615d81d700 1 -- 172.21.15.77:0/1849681309 shutdown_connections 2023-12-01T01:10:19.082 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:18.686+0000 7f615d81d700 1 -- 172.21.15.77:0/1849681309 wait complete. 2023-12-01T01:10:19.082 INFO:teuthology.orchestra.run.smithi077.stdout:Waiting for the mgr to restart... 2023-12-01T01:10:19.082 INFO:teuthology.orchestra.run.smithi077.stdout:Waiting for mgr epoch 9... 2023-12-01T01:10:19.154 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:18 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:10:18.856+0000 7f748acf4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T01:10:19.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:19 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/1849681309' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-12-01T01:10:19.654 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:19 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:10:19.290+0000 7f748acf4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T01:10:20.404 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:19 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:10:19.958+0000 7f748acf4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T01:10:20.404 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:20 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:10:20.039+0000 7f748acf4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T01:10:21.072 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:20 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:10:20.807+0000 7f748acf4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:10:21.072 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:20 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:10:20.899+0000 7f748acf4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T01:10:21.352 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:21 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:10:21.071+0000 7f748acf4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:10:21.654 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:21 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:10:21.350+0000 7f748acf4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T01:10:21.654 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:21 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:10:21.573+0000 7f748acf4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T01:10:21.982 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:21 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:10:21.720+0000 7f748acf4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T01:10:22.404 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:21 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:10:21.981+0000 7f748acf4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T01:10:22.998 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:22 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:10:22.675+0000 7f748acf4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T01:10:22.998 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:22 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:10:22.805+0000 7f748acf4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T01:10:23.328 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:22 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:10:22.996+0000 7f748acf4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T01:10:23.328 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:23 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:10:23.110+0000 7f748acf4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T01:10:23.654 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:23 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:10:23.327+0000 7f748acf4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T01:10:24.654 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:10:24 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:10:24.177+0000 7f748acf4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T01:10:24.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:24 smithi077 ceph-mon[98867]: Active manager daemon x restarted 2023-12-01T01:10:24.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:24 smithi077 ceph-mon[98867]: Activating manager daemon x 2023-12-01T01:10:24.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:24 smithi077 ceph-mon[98867]: osdmap e3: 0 total, 0 up, 0 in 2023-12-01T01:10:24.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:24 smithi077 ceph-mon[98867]: mgrmap e10: x(active, starting, since 0.0035144s) 2023-12-01T01:10:24.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:24 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-01T01:10:24.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:24 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-01T01:10:24.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:24 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-01T01:10:24.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:24 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-01T01:10:24.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:24 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-01T01:10:24.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:24 smithi077 ceph-mon[98867]: Manager daemon x is now available 2023-12-01T01:10:24.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:24 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:10:24.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:24 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:10:25.492 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:25 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-12-01T01:10:25.492 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:25 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-12-01T01:10:25.492 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:25 smithi077 ceph-mon[98867]: [01/Dec/2023:01:10:24] ENGINE Bus STARTING 2023-12-01T01:10:25.492 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:25 smithi077 ceph-mon[98867]: [01/Dec/2023:01:10:24] ENGINE Serving on https://172.21.15.77:7150 2023-12-01T01:10:25.492 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:25 smithi077 ceph-mon[98867]: mgrmap e11: x(active, since 1.00553s) 2023-12-01T01:10:25.534 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout { 2023-12-01T01:10:25.534 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-12-01T01:10:25.534 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "initialized": true 2023-12-01T01:10:25.534 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout } 2023-12-01T01:10:25.534 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:19.558+0000 7ff0f408d700 1 Processor -- start 2023-12-01T01:10:25.534 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:19.559+0000 7ff0f408d700 1 -- start start 2023-12-01T01:10:25.534 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:19.559+0000 7ff0f408d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0ec107170 0x7ff0ec109560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:25.534 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:19.559+0000 7ff0f408d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ec0745c0 con 0x7ff0ec107170 2023-12-01T01:10:25.534 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:19.559+0000 7ff0f1e29700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0ec107170 0x7ff0ec109560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:25.535 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:19.559+0000 7ff0f1e29700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0ec107170 0x7ff0ec109560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56384/0 (socket says 172.21.15.77:56384) 2023-12-01T01:10:25.535 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:19.559+0000 7ff0f1e29700 1 -- 172.21.15.77:0/1914416958 learned_addr learned my addr 172.21.15.77:0/1914416958 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:25.535 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:19.559+0000 7ff0f1e29700 1 -- 172.21.15.77:0/1914416958 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ec109aa0 con 0x7ff0ec107170 2023-12-01T01:10:25.535 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:19.559+0000 7ff0f1e29700 1 --2- 172.21.15.77:0/1914416958 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0ec107170 0x7ff0ec109560 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff0e80098a0 tx=0x7ff0e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208a051d0883f60f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:25.535 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:19.560+0000 7ff0f0e27700 1 -- 172.21.15.77:0/1914416958 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff0e8004160 con 0x7ff0ec107170 2023-12-01T01:10:25.535 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:19.560+0000 7ff0f0e27700 1 -- 172.21.15.77:0/1914416958 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0e80042c0 con 0x7ff0ec107170 2023-12-01T01:10:25.535 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:19.560+0000 7ff0f0e27700 1 -- 172.21.15.77:0/1914416958 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff0e80119b0 con 0x7ff0ec107170 2023-12-01T01:10:25.535 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:19.560+0000 7ff0f408d700 1 -- 172.21.15.77:0/1914416958 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0ec107170 msgr2=0x7ff0ec109560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:25.536 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:19.560+0000 7ff0f408d700 1 --2- 172.21.15.77:0/1914416958 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0ec107170 0x7ff0ec109560 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff0e80098a0 tx=0x7ff0e8009bb0 comp rx=0 tx=0).stop 2023-12-01T01:10:25.536 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:19.560+0000 7ff0f408d700 1 -- 172.21.15.77:0/1914416958 shutdown_connections 2023-12-01T01:10:25.536 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:19.560+0000 7ff0f408d700 1 --2- 172.21.15.77:0/1914416958 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0ec107170 0x7ff0ec109560 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:25.536 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:19.560+0000 7ff0f408d700 1 -- 172.21.15.77:0/1914416958 >> 172.21.15.77:0/1914416958 conn(0x7ff0ec100b80 msgr2=0x7ff0ec102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:25.536 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:19.560+0000 7ff0f408d700 1 -- 172.21.15.77:0/1914416958 shutdown_connections 2023-12-01T01:10:25.536 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:19.560+0000 7ff0f408d700 1 -- 172.21.15.77:0/1914416958 wait complete. 2023-12-01T01:10:25.536 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:19.561+0000 7ff0f408d700 1 Processor -- start 2023-12-01T01:10:25.536 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:19.561+0000 7ff0f408d700 1 -- start start 2023-12-01T01:10:25.536 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:19.561+0000 7ff0f408d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0ec107170 0x7ff0ec19a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:25.536 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:19.561+0000 7ff0f408d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ec19a8a0 con 0x7ff0ec107170 2023-12-01T01:10:25.537 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:19.561+0000 7ff0f1e29700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0ec107170 0x7ff0ec19a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:25.537 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:19.561+0000 7ff0f1e29700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0ec107170 0x7ff0ec19a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56386/0 (socket says 172.21.15.77:56386) 2023-12-01T01:10:25.537 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:19.561+0000 7ff0f1e29700 1 -- 172.21.15.77:0/1095060900 learned_addr learned my addr 172.21.15.77:0/1095060900 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:25.537 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:19.561+0000 7ff0f1e29700 1 -- 172.21.15.77:0/1095060900 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e8009550 con 0x7ff0ec107170 2023-12-01T01:10:25.537 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:19.562+0000 7ff0f1e29700 1 --2- 172.21.15.77:0/1095060900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0ec107170 0x7ff0ec19a360 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8003e60 tx=0x7ff0e8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:25.537 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:19.562+0000 7ff0e2ffd700 1 -- 172.21.15.77:0/1095060900 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff0e8003970 con 0x7ff0ec107170 2023-12-01T01:10:25.537 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:19.562+0000 7ff0e2ffd700 1 -- 172.21.15.77:0/1095060900 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0e8003ad0 con 0x7ff0ec107170 2023-12-01T01:10:25.537 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:19.562+0000 7ff0e2ffd700 1 -- 172.21.15.77:0/1095060900 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff0e80195e0 con 0x7ff0ec107170 2023-12-01T01:10:25.537 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:19.562+0000 7ff0f408d700 1 -- 172.21.15.77:0/1095060900 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0ec19aaa0 con 0x7ff0ec107170 2023-12-01T01:10:25.537 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:19.562+0000 7ff0f408d700 1 -- 172.21.15.77:0/1095060900 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0ec19aec0 con 0x7ff0ec107170 2023-12-01T01:10:25.537 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:19.563+0000 7ff0e2ffd700 1 -- 172.21.15.77:0/1095060900 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 9) v1 ==== 45281+0+0 (secure 0 0 0) 0x7ff0e8019850 con 0x7ff0ec107170 2023-12-01T01:10:25.537 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:19.563+0000 7ff0e2ffd700 1 --2- 172.21.15.77:0/1095060900 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7ff0d8038350 0x7ff0d803a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:25.537 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:19.563+0000 7ff0e2ffd700 1 -- 172.21.15.77:0/1095060900 --> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff0d803af20 con 0x7ff0d8038350 2023-12-01T01:10:25.537 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:19.563+0000 7ff0f1628700 1 -- 172.21.15.77:0/1095060900 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7ff0d8038350 msgr2=0x7ff0d803a810 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/1456621747 2023-12-01T01:10:25.538 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:19.563+0000 7ff0e2ffd700 1 -- 172.21.15.77:0/1095060900 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff0e804c620 con 0x7ff0ec107170 2023-12-01T01:10:25.538 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:19.563+0000 7ff0f1628700 1 --2- 172.21.15.77:0/1095060900 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7ff0d8038350 0x7ff0d803a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:10:25.538 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:19.764+0000 7ff0f1628700 1 -- 172.21.15.77:0/1095060900 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7ff0d8038350 msgr2=0x7ff0d803a810 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/1456621747 2023-12-01T01:10:25.538 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:19.764+0000 7ff0f1628700 1 --2- 172.21.15.77:0/1095060900 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7ff0d8038350 0x7ff0d803a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-01T01:10:25.538 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:20.165+0000 7ff0f1628700 1 -- 172.21.15.77:0/1095060900 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7ff0d8038350 msgr2=0x7ff0d803a810 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/1456621747 2023-12-01T01:10:25.538 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:20.165+0000 7ff0f1628700 1 --2- 172.21.15.77:0/1095060900 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7ff0d8038350 0x7ff0d803a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-01T01:10:25.538 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:20.966+0000 7ff0f1628700 1 -- 172.21.15.77:0/1095060900 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7ff0d8038350 msgr2=0x7ff0d803a810 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/1456621747 2023-12-01T01:10:25.538 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:20.966+0000 7ff0f1628700 1 --2- 172.21.15.77:0/1095060900 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7ff0d8038350 0x7ff0d803a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-01T01:10:25.538 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:22.567+0000 7ff0f1628700 1 -- 172.21.15.77:0/1095060900 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7ff0d8038350 msgr2=0x7ff0d803a810 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/1456621747 2023-12-01T01:10:25.538 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:22.567+0000 7ff0f1628700 1 --2- 172.21.15.77:0/1095060900 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7ff0d8038350 0x7ff0d803a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-01T01:10:25.538 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:24.193+0000 7ff0e2ffd700 1 -- 172.21.15.77:0/1095060900 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mgrmap(e 10) v1 ==== 45048+0+0 (secure 0 0 0) 0x7ff0e802d9b0 con 0x7ff0ec107170 2023-12-01T01:10:25.538 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:24.193+0000 7ff0e2ffd700 1 -- 172.21.15.77:0/1095060900 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7ff0d8038350 msgr2=0x7ff0d803a810 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:10:25.539 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:24.193+0000 7ff0e2ffd700 1 --2- 172.21.15.77:0/1095060900 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7ff0d8038350 0x7ff0d803a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:25.539 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:25.194+0000 7ff0e2ffd700 1 -- 172.21.15.77:0/1095060900 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mgrmap(e 11) v1 ==== 45175+0+0 (secure 0 0 0) 0x7ff0e804dfd0 con 0x7ff0ec107170 2023-12-01T01:10:25.539 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:25.194+0000 7ff0e2ffd700 1 --2- 172.21.15.77:0/1095060900 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff0d803bdc0 0x7ff0d803e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:25.539 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:25.194+0000 7ff0e2ffd700 1 -- 172.21.15.77:0/1095060900 --> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff0d803af20 con 0x7ff0d803bdc0 2023-12-01T01:10:25.539 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:25.194+0000 7ff0f1628700 1 --2- 172.21.15.77:0/1095060900 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff0d803bdc0 0x7ff0d803e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:25.539 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:25.195+0000 7ff0f1628700 1 --2- 172.21.15.77:0/1095060900 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff0d803bdc0 0x7ff0d803e1b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc003a10 tx=0x7ff0dc009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:25.539 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:25.195+0000 7ff0e2ffd700 1 -- 172.21.15.77:0/1095060900 <== mgr.14152 v2:172.21.15.77:6800/3838941679 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7ff0d803af20 con 0x7ff0d803bdc0 2023-12-01T01:10:25.539 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:25.202+0000 7ff0f408d700 1 -- 172.21.15.77:0/1095060900 --> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7ff0ec19b6a0 con 0x7ff0d803bdc0 2023-12-01T01:10:25.539 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:25.202+0000 7ff0e2ffd700 1 -- 172.21.15.77:0/1095060900 <== mgr.14152 v2:172.21.15.77:6800/3838941679 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7ff0ec19b6a0 con 0x7ff0d803bdc0 2023-12-01T01:10:25.539 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:25.202+0000 7ff0f408d700 1 -- 172.21.15.77:0/1095060900 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff0d803bdc0 msgr2=0x7ff0d803e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:25.539 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:25.202+0000 7ff0f408d700 1 --2- 172.21.15.77:0/1095060900 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff0d803bdc0 0x7ff0d803e1b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc003a10 tx=0x7ff0dc009270 comp rx=0 tx=0).stop 2023-12-01T01:10:25.539 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:25.202+0000 7ff0f408d700 1 -- 172.21.15.77:0/1095060900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0ec107170 msgr2=0x7ff0ec19a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:25.539 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:25.202+0000 7ff0f408d700 1 --2- 172.21.15.77:0/1095060900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0ec107170 0x7ff0ec19a360 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8003e60 tx=0x7ff0e8011aa0 comp rx=0 tx=0).stop 2023-12-01T01:10:25.539 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:25.202+0000 7ff0f408d700 1 -- 172.21.15.77:0/1095060900 shutdown_connections 2023-12-01T01:10:25.540 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:25.202+0000 7ff0f408d700 1 --2- 172.21.15.77:0/1095060900 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff0d803bdc0 0x7ff0d803e1b0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:25.540 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:25.202+0000 7ff0f408d700 1 --2- 172.21.15.77:0/1095060900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0ec107170 0x7ff0ec19a360 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:25.540 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:25.202+0000 7ff0f408d700 1 --2- 172.21.15.77:0/1095060900 >> [v2:172.21.15.77:6800/1456621747,v1:172.21.15.77:6801/1456621747] conn(0x7ff0d8038350 0x7ff0d803a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:25.540 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:25.202+0000 7ff0f408d700 1 -- 172.21.15.77:0/1095060900 >> 172.21.15.77:0/1095060900 conn(0x7ff0ec100b80 msgr2=0x7ff0ec102250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:25.540 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:25.202+0000 7ff0f408d700 1 -- 172.21.15.77:0/1095060900 shutdown_connections 2023-12-01T01:10:25.540 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:25.202+0000 7ff0f408d700 1 -- 172.21.15.77:0/1095060900 wait complete. 2023-12-01T01:10:25.540 INFO:teuthology.orchestra.run.smithi077.stdout:mgr epoch 9 is available 2023-12-01T01:10:25.540 INFO:teuthology.orchestra.run.smithi077.stdout:Generating a dashboard self-signed certificate... 2023-12-01T01:10:26.542 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:26 smithi077 ceph-mon[98867]: [01/Dec/2023:01:10:24] ENGINE Serving on http://172.21.15.77:8765 2023-12-01T01:10:26.542 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:26 smithi077 ceph-mon[98867]: [01/Dec/2023:01:10:24] ENGINE Bus STARTED 2023-12-01T01:10:26.542 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:26 smithi077 ceph-mon[98867]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-12-01T01:10:26.543 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:26 smithi077 ceph-mon[98867]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-12-01T01:10:26.543 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:26 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:10:26.777 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-12-01T01:10:26.777 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.141+0000 7fd82f5a3700 1 Processor -- start 2023-12-01T01:10:26.777 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.141+0000 7fd82f5a3700 1 -- start start 2023-12-01T01:10:26.777 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.141+0000 7fd82f5a3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd828106730 0x7fd828106b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:26.777 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.141+0000 7fd82f5a3700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd828107120 con 0x7fd828106730 2023-12-01T01:10:26.777 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.141+0000 7fd82d33f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd828106730 0x7fd828106b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:26.777 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.142+0000 7fd82d33f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd828106730 0x7fd828106b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56440/0 (socket says 172.21.15.77:56440) 2023-12-01T01:10:26.777 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.142+0000 7fd82d33f700 1 -- 172.21.15.77:0/2337160279 learned_addr learned my addr 172.21.15.77:0/2337160279 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:26.777 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.142+0000 7fd82d33f700 1 -- 172.21.15.77:0/2337160279 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd828107930 con 0x7fd828106730 2023-12-01T01:10:26.777 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.142+0000 7fd82d33f700 1 --2- 172.21.15.77:0/2337160279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd828106730 0x7fd828106b50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd8240098a0 tx=0x7fd824009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6677a93ada6f9ab server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:26.778 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.142+0000 7fd81ffff700 1 -- 172.21.15.77:0/2337160279 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd824004160 con 0x7fd828106730 2023-12-01T01:10:26.778 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.142+0000 7fd81ffff700 1 -- 172.21.15.77:0/2337160279 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8240042c0 con 0x7fd828106730 2023-12-01T01:10:26.778 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.142+0000 7fd81ffff700 1 -- 172.21.15.77:0/2337160279 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd8240119b0 con 0x7fd828106730 2023-12-01T01:10:26.778 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.143+0000 7fd82f5a3700 1 -- 172.21.15.77:0/2337160279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd828106730 msgr2=0x7fd828106b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:26.778 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.143+0000 7fd82f5a3700 1 --2- 172.21.15.77:0/2337160279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd828106730 0x7fd828106b50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd8240098a0 tx=0x7fd824009bb0 comp rx=0 tx=0).stop 2023-12-01T01:10:26.778 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.143+0000 7fd82f5a3700 1 -- 172.21.15.77:0/2337160279 shutdown_connections 2023-12-01T01:10:26.778 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.143+0000 7fd82f5a3700 1 --2- 172.21.15.77:0/2337160279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd828106730 0x7fd828106b50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:26.778 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.143+0000 7fd82f5a3700 1 -- 172.21.15.77:0/2337160279 >> 172.21.15.77:0/2337160279 conn(0x7fd828101cd0 msgr2=0x7fd828104110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:26.778 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.143+0000 7fd82f5a3700 1 -- 172.21.15.77:0/2337160279 shutdown_connections 2023-12-01T01:10:26.778 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.143+0000 7fd82f5a3700 1 -- 172.21.15.77:0/2337160279 wait complete. 2023-12-01T01:10:26.778 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.144+0000 7fd82f5a3700 1 Processor -- start 2023-12-01T01:10:26.778 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.144+0000 7fd82f5a3700 1 -- start start 2023-12-01T01:10:26.778 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.144+0000 7fd82f5a3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd828106730 0x7fd82807ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:26.778 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.144+0000 7fd82f5a3700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82807b090 con 0x7fd828106730 2023-12-01T01:10:26.778 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.144+0000 7fd82d33f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd828106730 0x7fd82807ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:26.779 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.144+0000 7fd82d33f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd828106730 0x7fd82807ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56442/0 (socket says 172.21.15.77:56442) 2023-12-01T01:10:26.779 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.144+0000 7fd82d33f700 1 -- 172.21.15.77:0/1171545963 learned_addr learned my addr 172.21.15.77:0/1171545963 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:26.779 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.145+0000 7fd82d33f700 1 -- 172.21.15.77:0/1171545963 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd824009550 con 0x7fd828106730 2023-12-01T01:10:26.779 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.145+0000 7fd82d33f700 1 --2- 172.21.15.77:0/1171545963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd828106730 0x7fd82807ab50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd824003e60 tx=0x7fd824011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:26.779 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.145+0000 7fd81e7fc700 1 -- 172.21.15.77:0/1171545963 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd824003970 con 0x7fd828106730 2023-12-01T01:10:26.779 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.145+0000 7fd81e7fc700 1 -- 172.21.15.77:0/1171545963 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd824003ad0 con 0x7fd828106730 2023-12-01T01:10:26.779 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.145+0000 7fd82f5a3700 1 -- 172.21.15.77:0/1171545963 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8280791a0 con 0x7fd828106730 2023-12-01T01:10:26.779 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.145+0000 7fd81e7fc700 1 -- 172.21.15.77:0/1171545963 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd8240195e0 con 0x7fd828106730 2023-12-01T01:10:26.779 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.145+0000 7fd82f5a3700 1 -- 172.21.15.77:0/1171545963 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8280795c0 con 0x7fd828106730 2023-12-01T01:10:26.779 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.146+0000 7fd82f5a3700 1 -- 172.21.15.77:0/1171545963 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8280623c0 con 0x7fd828106730 2023-12-01T01:10:26.779 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.146+0000 7fd81e7fc700 1 -- 172.21.15.77:0/1171545963 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 11) v1 ==== 45175+0+0 (secure 0 0 0) 0x7fd824019870 con 0x7fd828106730 2023-12-01T01:10:26.779 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.146+0000 7fd81e7fc700 1 --2- 172.21.15.77:0/1171545963 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd814038230 0x7fd81403a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:26.779 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.146+0000 7fd81e7fc700 1 -- 172.21.15.77:0/1171545963 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd82404c250 con 0x7fd828106730 2023-12-01T01:10:26.779 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.146+0000 7fd82cb3e700 1 --2- 172.21.15.77:0/1171545963 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd814038230 0x7fd81403a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:26.780 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.147+0000 7fd82cb3e700 1 --2- 172.21.15.77:0/1171545963 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd814038230 0x7fd81403a6f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd818006fd0 tx=0x7fd818006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:26.780 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.151+0000 7fd81e7fc700 1 -- 172.21.15.77:0/1171545963 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd824003c40 con 0x7fd828106730 2023-12-01T01:10:26.780 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.354+0000 7fd82f5a3700 1 -- 172.21.15.77:0/1171545963 --> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fd828103190 con 0x7fd814038230 2023-12-01T01:10:26.780 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.432+0000 7fd81e7fc700 1 -- 172.21.15.77:0/1171545963 <== mgr.14152 v2:172.21.15.77:6800/3838941679 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fd828103190 con 0x7fd814038230 2023-12-01T01:10:26.780 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.434+0000 7fd82f5a3700 1 -- 172.21.15.77:0/1171545963 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd814038230 msgr2=0x7fd81403a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:26.780 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.434+0000 7fd82f5a3700 1 --2- 172.21.15.77:0/1171545963 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd814038230 0x7fd81403a6f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd818006fd0 tx=0x7fd818006e40 comp rx=0 tx=0).stop 2023-12-01T01:10:26.780 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.434+0000 7fd82f5a3700 1 -- 172.21.15.77:0/1171545963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd828106730 msgr2=0x7fd82807ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:26.780 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.434+0000 7fd82f5a3700 1 --2- 172.21.15.77:0/1171545963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd828106730 0x7fd82807ab50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd824003e60 tx=0x7fd824011aa0 comp rx=0 tx=0).stop 2023-12-01T01:10:26.780 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.434+0000 7fd82f5a3700 1 -- 172.21.15.77:0/1171545963 shutdown_connections 2023-12-01T01:10:26.780 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.434+0000 7fd82f5a3700 1 --2- 172.21.15.77:0/1171545963 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd814038230 0x7fd81403a6f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:26.780 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.434+0000 7fd82f5a3700 1 --2- 172.21.15.77:0/1171545963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd828106730 0x7fd82807ab50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:26.780 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.434+0000 7fd82f5a3700 1 -- 172.21.15.77:0/1171545963 >> 172.21.15.77:0/1171545963 conn(0x7fd828101cd0 msgr2=0x7fd828102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:26.780 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.434+0000 7fd82f5a3700 1 -- 172.21.15.77:0/1171545963 shutdown_connections 2023-12-01T01:10:26.780 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:26.435+0000 7fd82f5a3700 1 -- 172.21.15.77:0/1171545963 wait complete. 2023-12-01T01:10:26.780 INFO:teuthology.orchestra.run.smithi077.stdout:Creating initial admin user... 2023-12-01T01:10:27.721 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:27 smithi077 ceph-mon[98867]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:10:27.721 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:27 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:10:27.721 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:27 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:10:27.721 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:27 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:10:27.721 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:27 smithi077 ceph-mon[98867]: mgrmap e12: x(active, since 2s) 2023-12-01T01:10:28.222 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$pNdDonj3cN5dEd6KeXEige82AYzDq6KlufqQGDCnirMTVP7G721O2", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1701393027, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-12-01T01:10:28.222 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.234+0000 7f884ae51700 1 Processor -- start 2023-12-01T01:10:28.222 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.234+0000 7f884ae51700 1 -- start start 2023-12-01T01:10:28.222 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.234+0000 7f884ae51700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8844106ba0 0x7f8844106fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:28.222 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.234+0000 7f884ae51700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8844107590 con 0x7f8844106ba0 2023-12-01T01:10:28.222 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.235+0000 7f8848bed700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8844106ba0 0x7f8844106fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:28.222 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.235+0000 7f8848bed700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8844106ba0 0x7f8844106fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56446/0 (socket says 172.21.15.77:56446) 2023-12-01T01:10:28.222 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.235+0000 7f8848bed700 1 -- 172.21.15.77:0/3481935809 learned_addr learned my addr 172.21.15.77:0/3481935809 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:28.222 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.235+0000 7f8848bed700 1 -- 172.21.15.77:0/3481935809 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88441076d0 con 0x7f8844106ba0 2023-12-01T01:10:28.222 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.235+0000 7f8848bed700 1 --2- 172.21.15.77:0/3481935809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8844106ba0 0x7f8844106fc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f88380098a0 tx=0x7f8838009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fb291aa4a2a2604 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:28.222 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.235+0000 7f88437fe700 1 -- 172.21.15.77:0/3481935809 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8838004160 con 0x7f8844106ba0 2023-12-01T01:10:28.223 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.235+0000 7f88437fe700 1 -- 172.21.15.77:0/3481935809 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88380042c0 con 0x7f8844106ba0 2023-12-01T01:10:28.223 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.235+0000 7f88437fe700 1 -- 172.21.15.77:0/3481935809 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f88380119b0 con 0x7f8844106ba0 2023-12-01T01:10:28.223 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.236+0000 7f884ae51700 1 -- 172.21.15.77:0/3481935809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8844106ba0 msgr2=0x7f8844106fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:28.223 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.236+0000 7f884ae51700 1 --2- 172.21.15.77:0/3481935809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8844106ba0 0x7f8844106fc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f88380098a0 tx=0x7f8838009bb0 comp rx=0 tx=0).stop 2023-12-01T01:10:28.223 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.236+0000 7f884ae51700 1 -- 172.21.15.77:0/3481935809 shutdown_connections 2023-12-01T01:10:28.223 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.236+0000 7f884ae51700 1 --2- 172.21.15.77:0/3481935809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8844106ba0 0x7f8844106fc0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:28.223 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.236+0000 7f884ae51700 1 -- 172.21.15.77:0/3481935809 >> 172.21.15.77:0/3481935809 conn(0x7f8844100300 msgr2=0x7f8844102720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:28.223 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.236+0000 7f884ae51700 1 -- 172.21.15.77:0/3481935809 shutdown_connections 2023-12-01T01:10:28.223 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.236+0000 7f884ae51700 1 -- 172.21.15.77:0/3481935809 wait complete. 2023-12-01T01:10:28.223 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.236+0000 7f884ae51700 1 Processor -- start 2023-12-01T01:10:28.223 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.236+0000 7f884ae51700 1 -- start start 2023-12-01T01:10:28.223 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.237+0000 7f884ae51700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8844106ba0 0x7f8844190760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:28.223 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.237+0000 7f884ae51700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8844190ca0 con 0x7f8844106ba0 2023-12-01T01:10:28.223 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.237+0000 7f8848bed700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8844106ba0 0x7f8844190760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:28.224 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.237+0000 7f8848bed700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8844106ba0 0x7f8844190760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56448/0 (socket says 172.21.15.77:56448) 2023-12-01T01:10:28.224 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.237+0000 7f8848bed700 1 -- 172.21.15.77:0/1716800197 learned_addr learned my addr 172.21.15.77:0/1716800197 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:28.224 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.237+0000 7f8848bed700 1 -- 172.21.15.77:0/1716800197 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8838009550 con 0x7f8844106ba0 2023-12-01T01:10:28.224 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.237+0000 7f8848bed700 1 --2- 172.21.15.77:0/1716800197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8844106ba0 0x7f8844190760 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8838000c00 tx=0x7f8838011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:28.224 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.237+0000 7f8841ffb700 1 -- 172.21.15.77:0/1716800197 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f88380039f0 con 0x7f8844106ba0 2023-12-01T01:10:28.224 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.237+0000 7f8841ffb700 1 -- 172.21.15.77:0/1716800197 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8838003b50 con 0x7f8844106ba0 2023-12-01T01:10:28.224 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.237+0000 7f884ae51700 1 -- 172.21.15.77:0/1716800197 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f884418edb0 con 0x7f8844106ba0 2023-12-01T01:10:28.224 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.237+0000 7f8841ffb700 1 -- 172.21.15.77:0/1716800197 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f883801a780 con 0x7f8844106ba0 2023-12-01T01:10:28.224 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.237+0000 7f884ae51700 1 -- 172.21.15.77:0/1716800197 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f884418f1d0 con 0x7f8844106ba0 2023-12-01T01:10:28.224 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.238+0000 7f8841ffb700 1 -- 172.21.15.77:0/1716800197 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 12) v1 ==== 45281+0+0 (secure 0 0 0) 0x7f883801a9f0 con 0x7f8844106ba0 2023-12-01T01:10:28.224 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.238+0000 7f8841ffb700 1 --2- 172.21.15.77:0/1716800197 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f88340383a0 0x7f883403a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:28.224 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.238+0000 7f8841ffb700 1 -- 172.21.15.77:0/1716800197 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f883804c120 con 0x7f8844106ba0 2023-12-01T01:10:28.224 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.238+0000 7f8843fff700 1 --2- 172.21.15.77:0/1716800197 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f88340383a0 0x7f883403a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:28.224 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.238+0000 7f884ae51700 1 -- 172.21.15.77:0/1716800197 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88441076d0 con 0x7f8844106ba0 2023-12-01T01:10:28.225 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.239+0000 7f8843fff700 1 --2- 172.21.15.77:0/1716800197 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f88340383a0 0x7f883403a860 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8830006fd0 tx=0x7f8830006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:28.225 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.243+0000 7f8841ffb700 1 -- 172.21.15.77:0/1716800197 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f88441076d0 con 0x7f8844106ba0 2023-12-01T01:10:28.225 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.465+0000 7f884ae51700 1 -- 172.21.15.77:0/1716800197 --> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f88441076d0 con 0x7f88340383a0 2023-12-01T01:10:28.225 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.692+0000 7f8841ffb700 1 -- 172.21.15.77:0/1716800197 <== mgr.14152 v2:172.21.15.77:6800/3838941679 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f88441076d0 con 0x7f88340383a0 2023-12-01T01:10:28.225 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.694+0000 7f884ae51700 1 -- 172.21.15.77:0/1716800197 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f88340383a0 msgr2=0x7f883403a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:28.225 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.694+0000 7f884ae51700 1 --2- 172.21.15.77:0/1716800197 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f88340383a0 0x7f883403a860 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8830006fd0 tx=0x7f8830006e40 comp rx=0 tx=0).stop 2023-12-01T01:10:28.225 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.694+0000 7f884ae51700 1 -- 172.21.15.77:0/1716800197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8844106ba0 msgr2=0x7f8844190760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:28.225 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.694+0000 7f884ae51700 1 --2- 172.21.15.77:0/1716800197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8844106ba0 0x7f8844190760 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8838000c00 tx=0x7f8838011a30 comp rx=0 tx=0).stop 2023-12-01T01:10:28.225 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.694+0000 7f884ae51700 1 -- 172.21.15.77:0/1716800197 shutdown_connections 2023-12-01T01:10:28.225 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.694+0000 7f884ae51700 1 --2- 172.21.15.77:0/1716800197 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f88340383a0 0x7f883403a860 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:28.225 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.694+0000 7f884ae51700 1 --2- 172.21.15.77:0/1716800197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8844106ba0 0x7f8844190760 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:28.225 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.694+0000 7f884ae51700 1 -- 172.21.15.77:0/1716800197 >> 172.21.15.77:0/1716800197 conn(0x7f8844100300 msgr2=0x7f8844105010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:28.225 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.695+0000 7f884ae51700 1 -- 172.21.15.77:0/1716800197 shutdown_connections 2023-12-01T01:10:28.225 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:27.695+0000 7f884ae51700 1 -- 172.21.15.77:0/1716800197 wait complete. 2023-12-01T01:10:28.226 INFO:teuthology.orchestra.run.smithi077.stdout:Fetching dashboard port number... 2023-12-01T01:10:28.782 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:28 smithi077 ceph-mon[98867]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:10:28.782 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:28 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:10:29.415 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 8443 2023-12-01T01:10:29.415 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:28.818+0000 7f0b3b8d1700 1 Processor -- start 2023-12-01T01:10:29.415 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:28.818+0000 7f0b3b8d1700 1 -- start start 2023-12-01T01:10:29.415 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:28.819+0000 7f0b3b8d1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b34104fc0 0x7f0b341053e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:29.415 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:28.819+0000 7f0b3b8d1700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b34079e80 con 0x7f0b34104fc0 2023-12-01T01:10:29.415 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:28.819+0000 7f0b3966d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b34104fc0 0x7f0b341053e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:29.415 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:28.819+0000 7f0b3966d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b34104fc0 0x7f0b341053e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56452/0 (socket says 172.21.15.77:56452) 2023-12-01T01:10:29.415 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:28.819+0000 7f0b3966d700 1 -- 172.21.15.77:0/766371566 learned_addr learned my addr 172.21.15.77:0/766371566 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:29.415 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:28.819+0000 7f0b3966d700 1 -- 172.21.15.77:0/766371566 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b34105920 con 0x7f0b34104fc0 2023-12-01T01:10:29.415 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:28.819+0000 7f0b3966d700 1 --2- 172.21.15.77:0/766371566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b34104fc0 0x7f0b341053e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0b240098a0 tx=0x7f0b24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d322550b997ac6d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:29.416 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:28.819+0000 7f0b2bfff700 1 -- 172.21.15.77:0/766371566 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b24004160 con 0x7f0b34104fc0 2023-12-01T01:10:29.416 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:28.819+0000 7f0b2bfff700 1 -- 172.21.15.77:0/766371566 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b240042c0 con 0x7f0b34104fc0 2023-12-01T01:10:29.416 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:28.819+0000 7f0b2bfff700 1 -- 172.21.15.77:0/766371566 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b240119b0 con 0x7f0b34104fc0 2023-12-01T01:10:29.416 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:28.820+0000 7f0b3b8d1700 1 -- 172.21.15.77:0/766371566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b34104fc0 msgr2=0x7f0b341053e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:29.416 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:28.820+0000 7f0b3b8d1700 1 --2- 172.21.15.77:0/766371566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b34104fc0 0x7f0b341053e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0b240098a0 tx=0x7f0b24009bb0 comp rx=0 tx=0).stop 2023-12-01T01:10:29.416 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:28.820+0000 7f0b3b8d1700 1 -- 172.21.15.77:0/766371566 shutdown_connections 2023-12-01T01:10:29.416 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:28.820+0000 7f0b3b8d1700 1 --2- 172.21.15.77:0/766371566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b34104fc0 0x7f0b341053e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:29.416 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:28.820+0000 7f0b3b8d1700 1 -- 172.21.15.77:0/766371566 >> 172.21.15.77:0/766371566 conn(0x7f0b34100bc0 msgr2=0x7f0b34103000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:29.416 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:28.820+0000 7f0b3b8d1700 1 -- 172.21.15.77:0/766371566 shutdown_connections 2023-12-01T01:10:29.416 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:28.820+0000 7f0b3b8d1700 1 -- 172.21.15.77:0/766371566 wait complete. 2023-12-01T01:10:29.416 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:28.821+0000 7f0b3b8d1700 1 Processor -- start 2023-12-01T01:10:29.416 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:28.821+0000 7f0b3b8d1700 1 -- start start 2023-12-01T01:10:29.416 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:28.821+0000 7f0b3b8d1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b34104fc0 0x7f0b3419c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:29.416 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:28.821+0000 7f0b3b8d1700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3419ca90 con 0x7f0b34104fc0 2023-12-01T01:10:29.416 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:28.821+0000 7f0b3966d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b34104fc0 0x7f0b3419c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:29.417 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:28.821+0000 7f0b3966d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b34104fc0 0x7f0b3419c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56454/0 (socket says 172.21.15.77:56454) 2023-12-01T01:10:29.417 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:28.821+0000 7f0b3966d700 1 -- 172.21.15.77:0/532778397 learned_addr learned my addr 172.21.15.77:0/532778397 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:29.417 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:28.822+0000 7f0b3966d700 1 -- 172.21.15.77:0/532778397 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b24009550 con 0x7f0b34104fc0 2023-12-01T01:10:29.417 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:28.822+0000 7f0b3966d700 1 --2- 172.21.15.77:0/532778397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b34104fc0 0x7f0b3419c550 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0b24003e60 tx=0x7f0b24011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:29.417 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:28.822+0000 7f0b2a7fc700 1 -- 172.21.15.77:0/532778397 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b24003970 con 0x7f0b34104fc0 2023-12-01T01:10:29.417 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:28.822+0000 7f0b3b8d1700 1 -- 172.21.15.77:0/532778397 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b3419cc90 con 0x7f0b34104fc0 2023-12-01T01:10:29.417 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:28.822+0000 7f0b2a7fc700 1 -- 172.21.15.77:0/532778397 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b24003ad0 con 0x7f0b34104fc0 2023-12-01T01:10:29.417 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:28.822+0000 7f0b2a7fc700 1 -- 172.21.15.77:0/532778397 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b240195e0 con 0x7f0b34104fc0 2023-12-01T01:10:29.417 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:28.822+0000 7f0b3b8d1700 1 -- 172.21.15.77:0/532778397 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b3419d0b0 con 0x7f0b34104fc0 2023-12-01T01:10:29.417 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:28.823+0000 7f0b3b8d1700 1 -- 172.21.15.77:0/532778397 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b34195e70 con 0x7f0b34104fc0 2023-12-01T01:10:29.417 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:28.823+0000 7f0b2a7fc700 1 -- 172.21.15.77:0/532778397 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 12) v1 ==== 45281+0+0 (secure 0 0 0) 0x7f0b240198c0 con 0x7f0b34104fc0 2023-12-01T01:10:29.417 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:28.823+0000 7f0b2a7fc700 1 --2- 172.21.15.77:0/532778397 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f0b20038300 0x7f0b2003a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:29.417 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:28.823+0000 7f0b2a7fc700 1 -- 172.21.15.77:0/532778397 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0b2404c490 con 0x7f0b34104fc0 2023-12-01T01:10:29.417 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:28.824+0000 7f0b38e6c700 1 --2- 172.21.15.77:0/532778397 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f0b20038300 0x7f0b2003a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:29.418 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:28.824+0000 7f0b38e6c700 1 --2- 172.21.15.77:0/532778397 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f0b20038300 0x7f0b2003a7c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0b30006fd0 tx=0x7f0b30006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:29.418 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:28.828+0000 7f0b2a7fc700 1 -- 172.21.15.77:0/532778397 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0b24003c40 con 0x7f0b34104fc0 2023-12-01T01:10:29.418 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.016+0000 7f0b3b8d1700 1 -- 172.21.15.77:0/532778397 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f0b3402cc70 con 0x7f0b34104fc0 2023-12-01T01:10:29.418 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.016+0000 7f0b2a7fc700 1 -- 172.21.15.77:0/532778397 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f0b2400b3e0 con 0x7f0b34104fc0 2023-12-01T01:10:29.418 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.018+0000 7f0b3b8d1700 1 -- 172.21.15.77:0/532778397 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f0b20038300 msgr2=0x7f0b2003a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:29.418 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.018+0000 7f0b3b8d1700 1 --2- 172.21.15.77:0/532778397 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f0b20038300 0x7f0b2003a7c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0b30006fd0 tx=0x7f0b30006e40 comp rx=0 tx=0).stop 2023-12-01T01:10:29.418 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.018+0000 7f0b3b8d1700 1 -- 172.21.15.77:0/532778397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b34104fc0 msgr2=0x7f0b3419c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:29.418 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.018+0000 7f0b3b8d1700 1 --2- 172.21.15.77:0/532778397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b34104fc0 0x7f0b3419c550 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0b24003e60 tx=0x7f0b24011aa0 comp rx=0 tx=0).stop 2023-12-01T01:10:29.418 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.018+0000 7f0b3b8d1700 1 -- 172.21.15.77:0/532778397 shutdown_connections 2023-12-01T01:10:29.418 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.018+0000 7f0b3b8d1700 1 --2- 172.21.15.77:0/532778397 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f0b20038300 0x7f0b2003a7c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:29.418 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.018+0000 7f0b3b8d1700 1 --2- 172.21.15.77:0/532778397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b34104fc0 0x7f0b3419c550 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:29.418 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.018+0000 7f0b3b8d1700 1 -- 172.21.15.77:0/532778397 >> 172.21.15.77:0/532778397 conn(0x7f0b34100bc0 msgr2=0x7f0b34190b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:29.418 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.018+0000 7f0b3b8d1700 1 -- 172.21.15.77:0/532778397 shutdown_connections 2023-12-01T01:10:29.418 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.018+0000 7f0b3b8d1700 1 -- 172.21.15.77:0/532778397 wait complete. 2023-12-01T01:10:29.421 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-12-01T01:10:29.421 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout disabled 2023-12-01T01:10:29.429 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-12-01T01:10:29.429 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout inactive 2023-12-01T01:10:29.429 INFO:teuthology.orchestra.run.smithi077.stdout:firewalld.service is not enabled 2023-12-01T01:10:29.429 INFO:teuthology.orchestra.run.smithi077.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-12-01T01:10:29.431 INFO:teuthology.orchestra.run.smithi077.stdout:Ceph Dashboard is now available at: 2023-12-01T01:10:29.431 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:10:29.431 INFO:teuthology.orchestra.run.smithi077.stdout: URL: https://smithi077.front.sepia.ceph.com:8443/ 2023-12-01T01:10:29.431 INFO:teuthology.orchestra.run.smithi077.stdout: User: admin 2023-12-01T01:10:29.431 INFO:teuthology.orchestra.run.smithi077.stdout: Password: slzzqrw3vq 2023-12-01T01:10:29.431 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:10:29.431 INFO:teuthology.orchestra.run.smithi077.stdout:Saving cluster configuration to /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/config directory 2023-12-01T01:10:29.431 INFO:teuthology.orchestra.run.smithi077.stdout:Enabling autotune for osd_memory_target 2023-12-01T01:10:29.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:29 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/532778397' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-12-01T01:10:30.378 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.866+0000 7fa7e8b03700 1 Processor -- start 2023-12-01T01:10:30.378 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.867+0000 7fa7e8b03700 1 -- start start 2023-12-01T01:10:30.378 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.867+0000 7fa7e8b03700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7e41055c0 0x7fa7e41059e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:30.378 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.867+0000 7fa7e8b03700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e4105fb0 con 0x7fa7e41055c0 2023-12-01T01:10:30.378 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.867+0000 7fa7e259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7e41055c0 0x7fa7e41059e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:30.378 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.867+0000 7fa7e259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7e41055c0 0x7fa7e41059e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56462/0 (socket says 172.21.15.77:56462) 2023-12-01T01:10:30.378 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.867+0000 7fa7e259c700 1 -- 172.21.15.77:0/699783071 learned_addr learned my addr 172.21.15.77:0/699783071 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:30.378 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.867+0000 7fa7e259c700 1 -- 172.21.15.77:0/699783071 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e41067c0 con 0x7fa7e41055c0 2023-12-01T01:10:30.379 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.867+0000 7fa7e259c700 1 --2- 172.21.15.77:0/699783071 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7e41055c0 0x7fa7e41059e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa7d80098a0 tx=0x7fa7d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25458783da72200a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:30.379 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.868+0000 7fa7e159a700 1 -- 172.21.15.77:0/699783071 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa7d8004160 con 0x7fa7e41055c0 2023-12-01T01:10:30.379 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.868+0000 7fa7e159a700 1 -- 172.21.15.77:0/699783071 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7d80042c0 con 0x7fa7e41055c0 2023-12-01T01:10:30.379 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.868+0000 7fa7e159a700 1 -- 172.21.15.77:0/699783071 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa7d80119b0 con 0x7fa7e41055c0 2023-12-01T01:10:30.379 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.868+0000 7fa7e8b03700 1 -- 172.21.15.77:0/699783071 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7e41055c0 msgr2=0x7fa7e41059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:30.379 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.868+0000 7fa7e8b03700 1 --2- 172.21.15.77:0/699783071 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7e41055c0 0x7fa7e41059e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa7d80098a0 tx=0x7fa7d8009bb0 comp rx=0 tx=0).stop 2023-12-01T01:10:30.379 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.868+0000 7fa7e8b03700 1 -- 172.21.15.77:0/699783071 shutdown_connections 2023-12-01T01:10:30.379 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.868+0000 7fa7e8b03700 1 --2- 172.21.15.77:0/699783071 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7e41055c0 0x7fa7e41059e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:30.379 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.868+0000 7fa7e8b03700 1 -- 172.21.15.77:0/699783071 >> 172.21.15.77:0/699783071 conn(0x7fa7e4100b80 msgr2=0x7fa7e4102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:30.379 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.868+0000 7fa7e8b03700 1 -- 172.21.15.77:0/699783071 shutdown_connections 2023-12-01T01:10:30.379 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.868+0000 7fa7e8b03700 1 -- 172.21.15.77:0/699783071 wait complete. 2023-12-01T01:10:30.379 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.869+0000 7fa7e8b03700 1 Processor -- start 2023-12-01T01:10:30.379 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.869+0000 7fa7e8b03700 1 -- start start 2023-12-01T01:10:30.379 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.869+0000 7fa7e8b03700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7e41055c0 0x7fa7e419a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:30.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.869+0000 7fa7e8b03700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e419a900 con 0x7fa7e41055c0 2023-12-01T01:10:30.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.869+0000 7fa7e259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7e41055c0 0x7fa7e419a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:30.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.869+0000 7fa7e259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7e41055c0 0x7fa7e419a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56464/0 (socket says 172.21.15.77:56464) 2023-12-01T01:10:30.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.869+0000 7fa7e259c700 1 -- 172.21.15.77:0/2306353399 learned_addr learned my addr 172.21.15.77:0/2306353399 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:30.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.870+0000 7fa7e259c700 1 -- 172.21.15.77:0/2306353399 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d8009550 con 0x7fa7e41055c0 2023-12-01T01:10:30.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.870+0000 7fa7e259c700 1 --2- 172.21.15.77:0/2306353399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7e41055c0 0x7fa7e419a3c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8000c00 tx=0x7fa7d8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:30.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.870+0000 7fa7d37fe700 1 -- 172.21.15.77:0/2306353399 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa7d80039f0 con 0x7fa7e41055c0 2023-12-01T01:10:30.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.870+0000 7fa7d37fe700 1 -- 172.21.15.77:0/2306353399 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7d8003b50 con 0x7fa7e41055c0 2023-12-01T01:10:30.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.870+0000 7fa7e8b03700 1 -- 172.21.15.77:0/2306353399 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e419ab00 con 0x7fa7e41055c0 2023-12-01T01:10:30.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.870+0000 7fa7d37fe700 1 -- 172.21.15.77:0/2306353399 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa7d801a780 con 0x7fa7e41055c0 2023-12-01T01:10:30.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.870+0000 7fa7e8b03700 1 -- 172.21.15.77:0/2306353399 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e419af20 con 0x7fa7e41055c0 2023-12-01T01:10:30.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.871+0000 7fa7e8b03700 1 -- 172.21.15.77:0/2306353399 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7e40623c0 con 0x7fa7e41055c0 2023-12-01T01:10:30.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.871+0000 7fa7d37fe700 1 -- 172.21.15.77:0/2306353399 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 12) v1 ==== 45281+0+0 (secure 0 0 0) 0x7fa7d8024440 con 0x7fa7e41055c0 2023-12-01T01:10:30.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.871+0000 7fa7d37fe700 1 --2- 172.21.15.77:0/2306353399 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa7cc0383a0 0x7fa7cc03a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:30.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.871+0000 7fa7d37fe700 1 -- 172.21.15.77:0/2306353399 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa7d804c2f0 con 0x7fa7e41055c0 2023-12-01T01:10:30.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.871+0000 7fa7e1d9b700 1 --2- 172.21.15.77:0/2306353399 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa7cc0383a0 0x7fa7cc03a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:30.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.872+0000 7fa7e1d9b700 1 --2- 172.21.15.77:0/2306353399 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa7cc0383a0 0x7fa7cc03a860 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4006fd0 tx=0x7fa7d4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:30.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:29.876+0000 7fa7d37fe700 1 -- 172.21.15.77:0/2306353399 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa7d8018d90 con 0x7fa7e41055c0 2023-12-01T01:10:30.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.065+0000 7fa7e8b03700 1 -- 172.21.15.77:0/2306353399 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fa7e419dda0 con 0x7fa7e41055c0 2023-12-01T01:10:30.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.065+0000 7fa7d37fe700 1 -- 172.21.15.77:0/2306353399 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fa7d8018350 con 0x7fa7e41055c0 2023-12-01T01:10:30.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.067+0000 7fa7e8b03700 1 -- 172.21.15.77:0/2306353399 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa7cc0383a0 msgr2=0x7fa7cc03a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:30.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.067+0000 7fa7e8b03700 1 --2- 172.21.15.77:0/2306353399 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa7cc0383a0 0x7fa7cc03a860 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4006fd0 tx=0x7fa7d4006e40 comp rx=0 tx=0).stop 2023-12-01T01:10:30.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.067+0000 7fa7e8b03700 1 -- 172.21.15.77:0/2306353399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7e41055c0 msgr2=0x7fa7e419a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:30.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.067+0000 7fa7e8b03700 1 --2- 172.21.15.77:0/2306353399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7e41055c0 0x7fa7e419a3c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8000c00 tx=0x7fa7d8011a30 comp rx=0 tx=0).stop 2023-12-01T01:10:30.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.067+0000 7fa7e8b03700 1 -- 172.21.15.77:0/2306353399 shutdown_connections 2023-12-01T01:10:30.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.067+0000 7fa7e8b03700 1 --2- 172.21.15.77:0/2306353399 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa7cc0383a0 0x7fa7cc03a860 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:30.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.067+0000 7fa7e8b03700 1 --2- 172.21.15.77:0/2306353399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7e41055c0 0x7fa7e419a3c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:30.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.067+0000 7fa7e8b03700 1 -- 172.21.15.77:0/2306353399 >> 172.21.15.77:0/2306353399 conn(0x7fa7e4100b80 msgr2=0x7fa7e4102270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:30.382 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.067+0000 7fa7e8b03700 1 -- 172.21.15.77:0/2306353399 shutdown_connections 2023-12-01T01:10:30.382 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.067+0000 7fa7e8b03700 1 -- 172.21.15.77:0/2306353399 wait complete. 2023-12-01T01:10:31.521 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.836+0000 7f8a741e5700 1 Processor -- start 2023-12-01T01:10:31.521 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.836+0000 7f8a741e5700 1 -- start start 2023-12-01T01:10:31.522 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.836+0000 7f8a741e5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a6c106730 0x7f8a6c106b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:31.522 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.836+0000 7f8a741e5700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6c107120 con 0x7f8a6c106730 2023-12-01T01:10:31.522 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.836+0000 7f8a71f81700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a6c106730 0x7f8a6c106b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:31.522 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.836+0000 7f8a71f81700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a6c106730 0x7f8a6c106b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56468/0 (socket says 172.21.15.77:56468) 2023-12-01T01:10:31.522 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.836+0000 7f8a71f81700 1 -- 172.21.15.77:0/3364715450 learned_addr learned my addr 172.21.15.77:0/3364715450 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:31.522 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.837+0000 7f8a71f81700 1 -- 172.21.15.77:0/3364715450 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a6c107930 con 0x7f8a6c106730 2023-12-01T01:10:31.522 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.837+0000 7f8a71f81700 1 --2- 172.21.15.77:0/3364715450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a6c106730 0x7f8a6c106b50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8a680098a0 tx=0x7f8a68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d444be6a0952b2dd server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:31.522 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.837+0000 7f8a70f7f700 1 -- 172.21.15.77:0/3364715450 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a68004160 con 0x7f8a6c106730 2023-12-01T01:10:31.522 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.837+0000 7f8a70f7f700 1 -- 172.21.15.77:0/3364715450 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a680042c0 con 0x7f8a6c106730 2023-12-01T01:10:31.522 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.837+0000 7f8a70f7f700 1 -- 172.21.15.77:0/3364715450 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a680119b0 con 0x7f8a6c106730 2023-12-01T01:10:31.522 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.837+0000 7f8a741e5700 1 -- 172.21.15.77:0/3364715450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a6c106730 msgr2=0x7f8a6c106b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:31.522 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.837+0000 7f8a741e5700 1 --2- 172.21.15.77:0/3364715450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a6c106730 0x7f8a6c106b50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8a680098a0 tx=0x7f8a68009bb0 comp rx=0 tx=0).stop 2023-12-01T01:10:31.522 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.837+0000 7f8a741e5700 1 -- 172.21.15.77:0/3364715450 shutdown_connections 2023-12-01T01:10:31.522 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.837+0000 7f8a741e5700 1 --2- 172.21.15.77:0/3364715450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a6c106730 0x7f8a6c106b50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:31.523 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.837+0000 7f8a741e5700 1 -- 172.21.15.77:0/3364715450 >> 172.21.15.77:0/3364715450 conn(0x7f8a6c101cd0 msgr2=0x7f8a6c104110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:31.523 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.838+0000 7f8a741e5700 1 -- 172.21.15.77:0/3364715450 shutdown_connections 2023-12-01T01:10:31.523 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.838+0000 7f8a741e5700 1 -- 172.21.15.77:0/3364715450 wait complete. 2023-12-01T01:10:31.523 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.838+0000 7f8a741e5700 1 Processor -- start 2023-12-01T01:10:31.523 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.838+0000 7f8a741e5700 1 -- start start 2023-12-01T01:10:31.523 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.838+0000 7f8a741e5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a6c106730 0x7f8a6c19a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:31.523 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.838+0000 7f8a741e5700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6c19a8d0 con 0x7f8a6c106730 2023-12-01T01:10:31.523 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.838+0000 7f8a71f81700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a6c106730 0x7f8a6c19a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:31.523 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.838+0000 7f8a71f81700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a6c106730 0x7f8a6c19a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56470/0 (socket says 172.21.15.77:56470) 2023-12-01T01:10:31.523 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.838+0000 7f8a71f81700 1 -- 172.21.15.77:0/3536629451 learned_addr learned my addr 172.21.15.77:0/3536629451 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:31.523 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.839+0000 7f8a71f81700 1 -- 172.21.15.77:0/3536629451 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a68009550 con 0x7f8a6c106730 2023-12-01T01:10:31.523 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.839+0000 7f8a71f81700 1 --2- 172.21.15.77:0/3536629451 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a6c106730 0x7f8a6c19a390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8a68000c00 tx=0x7f8a68011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:31.523 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.839+0000 7f8a62ffd700 1 -- 172.21.15.77:0/3536629451 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a680039f0 con 0x7f8a6c106730 2023-12-01T01:10:31.523 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.839+0000 7f8a62ffd700 1 -- 172.21.15.77:0/3536629451 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a68003b50 con 0x7f8a6c106730 2023-12-01T01:10:31.524 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.839+0000 7f8a62ffd700 1 -- 172.21.15.77:0/3536629451 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a6801a780 con 0x7f8a6c106730 2023-12-01T01:10:31.524 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.839+0000 7f8a741e5700 1 -- 172.21.15.77:0/3536629451 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a6c19aad0 con 0x7f8a6c106730 2023-12-01T01:10:31.524 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.839+0000 7f8a741e5700 1 -- 172.21.15.77:0/3536629451 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a6c19aef0 con 0x7f8a6c106730 2023-12-01T01:10:31.524 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.840+0000 7f8a62ffd700 1 -- 172.21.15.77:0/3536629451 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) v1 ==== 45324+0+0 (secure 0 0 0) 0x7f8a68024440 con 0x7f8a6c106730 2023-12-01T01:10:31.524 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.840+0000 7f8a741e5700 1 -- 172.21.15.77:0/3536629451 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a6c0623c0 con 0x7f8a6c106730 2023-12-01T01:10:31.524 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.840+0000 7f8a62ffd700 1 --2- 172.21.15.77:0/3536629451 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8a58038440 0x7f8a5803a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:31.524 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.840+0000 7f8a62ffd700 1 -- 172.21.15.77:0/3536629451 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8a6804c390 con 0x7f8a6c106730 2023-12-01T01:10:31.524 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.840+0000 7f8a71780700 1 --2- 172.21.15.77:0/3536629451 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8a58038440 0x7f8a5803a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:31.524 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.841+0000 7f8a71780700 1 --2- 172.21.15.77:0/3536629451 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8a58038440 0x7f8a5803a900 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c006fd0 tx=0x7f8a5c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:31.524 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:30.845+0000 7f8a62ffd700 1 -- 172.21.15.77:0/3536629451 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8a68018db0 con 0x7f8a6c106730 2023-12-01T01:10:31.524 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:31.135+0000 7f8a741e5700 1 -- 172.21.15.77:0/3536629451 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f8a6c19dd50 con 0x7f8a6c106730 2023-12-01T01:10:31.524 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:31.137+0000 7f8a62ffd700 1 -- 172.21.15.77:0/3536629451 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f8a68018350 con 0x7f8a6c106730 2023-12-01T01:10:31.524 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:31.139+0000 7f8a741e5700 1 -- 172.21.15.77:0/3536629451 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8a58038440 msgr2=0x7f8a5803a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:31.524 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:31.139+0000 7f8a741e5700 1 --2- 172.21.15.77:0/3536629451 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8a58038440 0x7f8a5803a900 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c006fd0 tx=0x7f8a5c006e40 comp rx=0 tx=0).stop 2023-12-01T01:10:31.524 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:31.139+0000 7f8a741e5700 1 -- 172.21.15.77:0/3536629451 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a6c106730 msgr2=0x7f8a6c19a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:31.525 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:31.139+0000 7f8a741e5700 1 --2- 172.21.15.77:0/3536629451 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a6c106730 0x7f8a6c19a390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8a68000c00 tx=0x7f8a68011a30 comp rx=0 tx=0).stop 2023-12-01T01:10:31.525 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:31.139+0000 7f8a741e5700 1 -- 172.21.15.77:0/3536629451 shutdown_connections 2023-12-01T01:10:31.525 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:31.139+0000 7f8a741e5700 1 --2- 172.21.15.77:0/3536629451 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8a58038440 0x7f8a5803a900 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:31.525 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:31.139+0000 7f8a741e5700 1 --2- 172.21.15.77:0/3536629451 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8a6c106730 0x7f8a6c19a390 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:31.525 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:31.139+0000 7f8a741e5700 1 -- 172.21.15.77:0/3536629451 >> 172.21.15.77:0/3536629451 conn(0x7f8a6c101cd0 msgr2=0x7f8a6c103380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:31.525 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:31.139+0000 7f8a741e5700 1 -- 172.21.15.77:0/3536629451 shutdown_connections 2023-12-01T01:10:31.525 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2023-12-01T01:10:31.139+0000 7f8a741e5700 1 -- 172.21.15.77:0/3536629451 wait complete. 2023-12-01T01:10:31.525 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-12-01T01:10:31.525 INFO:teuthology.orchestra.run.smithi077.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-12-01T01:10:31.525 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:10:31.525 INFO:teuthology.orchestra.run.smithi077.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-12-01T01:10:31.525 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:10:31.525 INFO:teuthology.orchestra.run.smithi077.stdout:Or, if you are only running a single cluster on this host: 2023-12-01T01:10:31.525 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:10:31.525 INFO:teuthology.orchestra.run.smithi077.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-12-01T01:10:31.525 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:10:31.526 INFO:teuthology.orchestra.run.smithi077.stdout:Please consider enabling telemetry to help improve Ceph: 2023-12-01T01:10:31.526 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:10:31.526 INFO:teuthology.orchestra.run.smithi077.stdout: ceph telemetry on 2023-12-01T01:10:31.526 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:10:31.526 INFO:teuthology.orchestra.run.smithi077.stdout:For more information see: 2023-12-01T01:10:31.526 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:10:31.526 INFO:teuthology.orchestra.run.smithi077.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-12-01T01:10:31.526 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:10:31.526 INFO:teuthology.orchestra.run.smithi077.stdout:Bootstrap complete. 2023-12-01T01:10:31.556 INFO:tasks.cephadm:Fetching config... 2023-12-01T01:10:31.556 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-12-01T01:10:31.556 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-12-01T01:10:31.573 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-12-01T01:10:31.573 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-12-01T01:10:31.573 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-12-01T01:10:31.631 INFO:tasks.cephadm:Fetching mon keyring... 2023-12-01T01:10:31.631 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-12-01T01:10:31.631 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/keyring of=/dev/stdout 2023-12-01T01:10:31.696 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:31 smithi077 ceph-mon[98867]: mgrmap e13: x(active, since 6s) 2023-12-01T01:10:31.696 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:31 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/3536629451' entity='client.admin' 2023-12-01T01:10:31.700 INFO:tasks.cephadm:Fetching pub ssh key... 2023-12-01T01:10:31.700 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-12-01T01:10:31.700 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-12-01T01:10:31.757 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-12-01T01:10:31.757 DEBUG:teuthology.orchestra.run.smithi077:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC013SX4LJ7LQ6G7zvtIEvaTmfjMzkecGP59RsGSW39+s7geeA2D6DrXMxwH9zSgfGhwYCRGUewSxvrp2kAUs/Y1tVMbYbTsEryBD4/k213hpx+26Cwxf65YyoKvN6N0NGXJQsFnz3exJC8/Om0RDReWgHcmdm1z3pT/Iq4fiFfQqizsnTTNuxiSVCRl6xS20+VcnN1ZyU/VF0kvtoQb2ipvvTQ0Noh4zAUudhXWLUeidjlECK7eDg51DKb5YjLBiry4PU1BLn9p6SQpzfZwdaNRySM/YrcZUZBpr+mDXwFXjtcnS1W68tdSuFJHMAIJqBrNVVYeJK9WN4v+ibXhruLQgPOEsc6N/U0lQamftIhWA5b6/4Iqmnsnh3elsXZ0BO6mCjZvE+jRpc3RwXfr2sJdP8GHtjpkKxSvMZspJce/tkRPoFm2V3l5xgi7OITGTVD90PVtT7FdxxemugcZM3tymoWJYga0QR/vHnW1tdDGYoDeac660KpMtkmy4Rppz8= ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-12-01T01:10:31.835 INFO:teuthology.orchestra.run.smithi077.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC013SX4LJ7LQ6G7zvtIEvaTmfjMzkecGP59RsGSW39+s7geeA2D6DrXMxwH9zSgfGhwYCRGUewSxvrp2kAUs/Y1tVMbYbTsEryBD4/k213hpx+26Cwxf65YyoKvN6N0NGXJQsFnz3exJC8/Om0RDReWgHcmdm1z3pT/Iq4fiFfQqizsnTTNuxiSVCRl6xS20+VcnN1ZyU/VF0kvtoQb2ipvvTQ0Noh4zAUudhXWLUeidjlECK7eDg51DKb5YjLBiry4PU1BLn9p6SQpzfZwdaNRySM/YrcZUZBpr+mDXwFXjtcnS1W68tdSuFJHMAIJqBrNVVYeJK9WN4v+ibXhruLQgPOEsc6N/U0lQamftIhWA5b6/4Iqmnsnh3elsXZ0BO6mCjZvE+jRpc3RwXfr2sJdP8GHtjpkKxSvMZspJce/tkRPoFm2V3l5xgi7OITGTVD90PVtT7FdxxemugcZM3tymoWJYga0QR/vHnW1tdDGYoDeac660KpMtkmy4Rppz8= ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715 2023-12-01T01:10:31.856 DEBUG:teuthology.orchestra.run.smithi137:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC013SX4LJ7LQ6G7zvtIEvaTmfjMzkecGP59RsGSW39+s7geeA2D6DrXMxwH9zSgfGhwYCRGUewSxvrp2kAUs/Y1tVMbYbTsEryBD4/k213hpx+26Cwxf65YyoKvN6N0NGXJQsFnz3exJC8/Om0RDReWgHcmdm1z3pT/Iq4fiFfQqizsnTTNuxiSVCRl6xS20+VcnN1ZyU/VF0kvtoQb2ipvvTQ0Noh4zAUudhXWLUeidjlECK7eDg51DKb5YjLBiry4PU1BLn9p6SQpzfZwdaNRySM/YrcZUZBpr+mDXwFXjtcnS1W68tdSuFJHMAIJqBrNVVYeJK9WN4v+ibXhruLQgPOEsc6N/U0lQamftIhWA5b6/4Iqmnsnh3elsXZ0BO6mCjZvE+jRpc3RwXfr2sJdP8GHtjpkKxSvMZspJce/tkRPoFm2V3l5xgi7OITGTVD90PVtT7FdxxemugcZM3tymoWJYga0QR/vHnW1tdDGYoDeac660KpMtkmy4Rppz8= ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-12-01T01:10:31.905 INFO:teuthology.orchestra.run.smithi137.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC013SX4LJ7LQ6G7zvtIEvaTmfjMzkecGP59RsGSW39+s7geeA2D6DrXMxwH9zSgfGhwYCRGUewSxvrp2kAUs/Y1tVMbYbTsEryBD4/k213hpx+26Cwxf65YyoKvN6N0NGXJQsFnz3exJC8/Om0RDReWgHcmdm1z3pT/Iq4fiFfQqizsnTTNuxiSVCRl6xS20+VcnN1ZyU/VF0kvtoQb2ipvvTQ0Noh4zAUudhXWLUeidjlECK7eDg51DKb5YjLBiry4PU1BLn9p6SQpzfZwdaNRySM/YrcZUZBpr+mDXwFXjtcnS1W68tdSuFJHMAIJqBrNVVYeJK9WN4v+ibXhruLQgPOEsc6N/U0lQamftIhWA5b6/4Iqmnsnh3elsXZ0BO6mCjZvE+jRpc3RwXfr2sJdP8GHtjpkKxSvMZspJce/tkRPoFm2V3l5xgi7OITGTVD90PVtT7FdxxemugcZM3tymoWJYga0QR/vHnW1tdDGYoDeac660KpMtkmy4Rppz8= ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715 2023-12-01T01:10:31.918 DEBUG:teuthology.orchestra.run.smithi203:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC013SX4LJ7LQ6G7zvtIEvaTmfjMzkecGP59RsGSW39+s7geeA2D6DrXMxwH9zSgfGhwYCRGUewSxvrp2kAUs/Y1tVMbYbTsEryBD4/k213hpx+26Cwxf65YyoKvN6N0NGXJQsFnz3exJC8/Om0RDReWgHcmdm1z3pT/Iq4fiFfQqizsnTTNuxiSVCRl6xS20+VcnN1ZyU/VF0kvtoQb2ipvvTQ0Noh4zAUudhXWLUeidjlECK7eDg51DKb5YjLBiry4PU1BLn9p6SQpzfZwdaNRySM/YrcZUZBpr+mDXwFXjtcnS1W68tdSuFJHMAIJqBrNVVYeJK9WN4v+ibXhruLQgPOEsc6N/U0lQamftIhWA5b6/4Iqmnsnh3elsXZ0BO6mCjZvE+jRpc3RwXfr2sJdP8GHtjpkKxSvMZspJce/tkRPoFm2V3l5xgi7OITGTVD90PVtT7FdxxemugcZM3tymoWJYga0QR/vHnW1tdDGYoDeac660KpMtkmy4Rppz8= ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-12-01T01:10:31.967 INFO:teuthology.orchestra.run.smithi203.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC013SX4LJ7LQ6G7zvtIEvaTmfjMzkecGP59RsGSW39+s7geeA2D6DrXMxwH9zSgfGhwYCRGUewSxvrp2kAUs/Y1tVMbYbTsEryBD4/k213hpx+26Cwxf65YyoKvN6N0NGXJQsFnz3exJC8/Om0RDReWgHcmdm1z3pT/Iq4fiFfQqizsnTTNuxiSVCRl6xS20+VcnN1ZyU/VF0kvtoQb2ipvvTQ0Noh4zAUudhXWLUeidjlECK7eDg51DKb5YjLBiry4PU1BLn9p6SQpzfZwdaNRySM/YrcZUZBpr+mDXwFXjtcnS1W68tdSuFJHMAIJqBrNVVYeJK9WN4v+ibXhruLQgPOEsc6N/U0lQamftIhWA5b6/4Iqmnsnh3elsXZ0BO6mCjZvE+jRpc3RwXfr2sJdP8GHtjpkKxSvMZspJce/tkRPoFm2V3l5xgi7OITGTVD90PVtT7FdxxemugcZM3tymoWJYga0QR/vHnW1tdDGYoDeac660KpMtkmy4Rppz8= ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715 2023-12-01T01:10:31.980 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-12-01T01:10:32.535 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:10:33.993 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:33.990+0000 7fb756934700 1 -- 172.21.15.77:0/3072330120 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb750101310 msgr2=0x7fb750103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:33.994 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:33.990+0000 7fb756934700 1 --2- 172.21.15.77:0/3072330120 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb750101310 0x7fb750103700 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb740009910 tx=0x7fb740009c20 comp rx=0 tx=0).stop 2023-12-01T01:10:33.994 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:33.991+0000 7fb756934700 1 -- 172.21.15.77:0/3072330120 shutdown_connections 2023-12-01T01:10:33.994 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:33.991+0000 7fb756934700 1 --2- 172.21.15.77:0/3072330120 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb750101310 0x7fb750103700 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:33.994 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:33.991+0000 7fb756934700 1 -- 172.21.15.77:0/3072330120 >> 172.21.15.77:0/3072330120 conn(0x7fb7500fad20 msgr2=0x7fb7500fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:33.994 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:33.991+0000 7fb756934700 1 -- 172.21.15.77:0/3072330120 shutdown_connections 2023-12-01T01:10:33.994 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:33.991+0000 7fb756934700 1 -- 172.21.15.77:0/3072330120 wait complete. 2023-12-01T01:10:33.994 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:33.991+0000 7fb756934700 1 Processor -- start 2023-12-01T01:10:33.994 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:33.991+0000 7fb756934700 1 -- start start 2023-12-01T01:10:33.994 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:33.991+0000 7fb756934700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb750101310 0x7fb750100c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:33.994 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:33.991+0000 7fb756934700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7501011b0 con 0x7fb750101310 2023-12-01T01:10:33.994 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:33.992+0000 7fb74ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb750101310 0x7fb750100c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:33.994 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:33.992+0000 7fb74ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb750101310 0x7fb750100c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56476/0 (socket says 172.21.15.77:56476) 2023-12-01T01:10:33.995 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:33.992+0000 7fb74ffff700 1 -- 172.21.15.77:0/1408860690 learned_addr learned my addr 172.21.15.77:0/1408860690 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:33.995 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:33.992+0000 7fb74ffff700 1 -- 172.21.15.77:0/1408860690 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7400095f0 con 0x7fb750101310 2023-12-01T01:10:33.995 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:33.992+0000 7fb74ffff700 1 --2- 172.21.15.77:0/1408860690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb750101310 0x7fb750100c70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb740005f50 tx=0x7fb7400057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:33.995 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:33.992+0000 7fb74d7fa700 1 -- 172.21.15.77:0/1408860690 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb74001d070 con 0x7fb750101310 2023-12-01T01:10:33.995 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:33.992+0000 7fb74d7fa700 1 -- 172.21.15.77:0/1408860690 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb740022470 con 0x7fb750101310 2023-12-01T01:10:33.995 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:33.992+0000 7fb756934700 1 -- 172.21.15.77:0/1408860690 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7500ff2c0 con 0x7fb750101310 2023-12-01T01:10:33.995 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:33.992+0000 7fb74d7fa700 1 -- 172.21.15.77:0/1408860690 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb74000f5f0 con 0x7fb750101310 2023-12-01T01:10:33.995 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:33.992+0000 7fb756934700 1 -- 172.21.15.77:0/1408860690 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7500ff760 con 0x7fb750101310 2023-12-01T01:10:33.996 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:33.993+0000 7fb74d7fa700 1 -- 172.21.15.77:0/1408860690 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) v1 ==== 45324+0+0 (secure 0 0 0) 0x7fb74000f860 con 0x7fb750101310 2023-12-01T01:10:33.996 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:33.993+0000 7fb756934700 1 -- 172.21.15.77:0/1408860690 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7500623c0 con 0x7fb750101310 2023-12-01T01:10:33.996 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:33.993+0000 7fb74d7fa700 1 --2- 172.21.15.77:0/1408860690 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fb7380383f0 0x7fb73803a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:33.996 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:33.993+0000 7fb74d7fa700 1 -- 172.21.15.77:0/1408860690 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb74004c490 con 0x7fb750101310 2023-12-01T01:10:33.996 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:33.993+0000 7fb74f7fe700 1 --2- 172.21.15.77:0/1408860690 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fb7380383f0 0x7fb73803a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:33.996 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:33.994+0000 7fb74f7fe700 1 --2- 172.21.15.77:0/1408860690 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fb7380383f0 0x7fb73803a8b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb744006fd0 tx=0x7fb744006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:33.999 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:33.998+0000 7fb74d7fa700 1 -- 172.21.15.77:0/1408860690 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb74002c960 con 0x7fb750101310 2023-12-01T01:10:34.210 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:34.208+0000 7fb756934700 1 -- 172.21.15.77:0/1408860690 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fb75018f250 con 0x7fb750101310 2023-12-01T01:10:34.213 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:34.212+0000 7fb74d7fa700 1 -- 172.21.15.77:0/1408860690 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fb740027030 con 0x7fb750101310 2023-12-01T01:10:34.215 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:34.214+0000 7fb756934700 1 -- 172.21.15.77:0/1408860690 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fb7380383f0 msgr2=0x7fb73803a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:34.215 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:34.214+0000 7fb756934700 1 --2- 172.21.15.77:0/1408860690 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fb7380383f0 0x7fb73803a8b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb744006fd0 tx=0x7fb744006e40 comp rx=0 tx=0).stop 2023-12-01T01:10:34.215 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:34.214+0000 7fb756934700 1 -- 172.21.15.77:0/1408860690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb750101310 msgr2=0x7fb750100c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:34.216 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:34.214+0000 7fb756934700 1 --2- 172.21.15.77:0/1408860690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb750101310 0x7fb750100c70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb740005f50 tx=0x7fb7400057c0 comp rx=0 tx=0).stop 2023-12-01T01:10:34.216 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:34.214+0000 7fb756934700 1 -- 172.21.15.77:0/1408860690 shutdown_connections 2023-12-01T01:10:34.216 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:34.214+0000 7fb756934700 1 --2- 172.21.15.77:0/1408860690 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fb7380383f0 0x7fb73803a8b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:34.216 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:34.214+0000 7fb756934700 1 --2- 172.21.15.77:0/1408860690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb750101310 0x7fb750100c70 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:34.216 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:34.214+0000 7fb756934700 1 -- 172.21.15.77:0/1408860690 >> 172.21.15.77:0/1408860690 conn(0x7fb7500fad20 msgr2=0x7fb7500fba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:34.216 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:34.214+0000 7fb756934700 1 -- 172.21.15.77:0/1408860690 shutdown_connections 2023-12-01T01:10:34.216 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:34.214+0000 7fb756934700 1 -- 172.21.15.77:0/1408860690 wait complete. 2023-12-01T01:10:34.583 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-12-01T01:10:34.583 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-12-01T01:10:35.451 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:10:35.638 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:35 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/1408860690' entity='client.admin' 2023-12-01T01:10:36.632 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:36.629+0000 7f1fef06f700 1 -- 172.21.15.77:0/4081780361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1fe8100e10 msgr2=0x7f1fe8101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:36.632 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:36.629+0000 7f1fef06f700 1 --2- 172.21.15.77:0/4081780361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1fe8100e10 0x7f1fe8101230 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc009910 tx=0x7f1fdc009c20 comp rx=0 tx=0).stop 2023-12-01T01:10:36.632 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:36.629+0000 7f1fef06f700 1 -- 172.21.15.77:0/4081780361 shutdown_connections 2023-12-01T01:10:36.632 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:36.629+0000 7f1fef06f700 1 --2- 172.21.15.77:0/4081780361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1fe8100e10 0x7f1fe8101230 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:36.633 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:36.629+0000 7f1fef06f700 1 -- 172.21.15.77:0/4081780361 >> 172.21.15.77:0/4081780361 conn(0x7f1fe80fa490 msgr2=0x7f1fe80fc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:36.633 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:36.629+0000 7f1fef06f700 1 -- 172.21.15.77:0/4081780361 shutdown_connections 2023-12-01T01:10:36.633 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:36.629+0000 7f1fef06f700 1 -- 172.21.15.77:0/4081780361 wait complete. 2023-12-01T01:10:36.633 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:36.630+0000 7f1fef06f700 1 Processor -- start 2023-12-01T01:10:36.633 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:36.630+0000 7f1fef06f700 1 -- start start 2023-12-01T01:10:36.633 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:36.630+0000 7f1fef06f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1fe8100e10 0x7f1fe8197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:36.633 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:36.630+0000 7f1fef06f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe81980b0 con 0x7f1fe8100e10 2023-12-01T01:10:36.633 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:36.630+0000 7f1fece0b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1fe8100e10 0x7f1fe8197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:36.633 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:36.630+0000 7f1fece0b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1fe8100e10 0x7f1fe8197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56482/0 (socket says 172.21.15.77:56482) 2023-12-01T01:10:36.633 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:36.630+0000 7f1fece0b700 1 -- 172.21.15.77:0/3678808904 learned_addr learned my addr 172.21.15.77:0/3678808904 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:36.633 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:36.630+0000 7f1fece0b700 1 -- 172.21.15.77:0/3678808904 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fdc0095f0 con 0x7f1fe8100e10 2023-12-01T01:10:36.633 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:36.631+0000 7f1fece0b700 1 --2- 172.21.15.77:0/3678808904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1fe8100e10 0x7f1fe8197b70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc004f40 tx=0x7f1fdc0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:36.633 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:36.631+0000 7f1fe5ffb700 1 -- 172.21.15.77:0/3678808904 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1fdc01d070 con 0x7f1fe8100e10 2023-12-01T01:10:36.633 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:36.631+0000 7f1fe5ffb700 1 -- 172.21.15.77:0/3678808904 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fdc0044e0 con 0x7f1fe8100e10 2023-12-01T01:10:36.634 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:36.631+0000 7f1fef06f700 1 -- 172.21.15.77:0/3678808904 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe81982b0 con 0x7f1fe8100e10 2023-12-01T01:10:36.634 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:36.631+0000 7f1fe5ffb700 1 -- 172.21.15.77:0/3678808904 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1fdc003a20 con 0x7f1fe8100e10 2023-12-01T01:10:36.634 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:36.631+0000 7f1fef06f700 1 -- 172.21.15.77:0/3678808904 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe81986f0 con 0x7f1fe8100e10 2023-12-01T01:10:36.634 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:36.632+0000 7f1fe5ffb700 1 -- 172.21.15.77:0/3678808904 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) v1 ==== 45324+0+0 (secure 0 0 0) 0x7f1fdc003c50 con 0x7f1fe8100e10 2023-12-01T01:10:36.635 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:36.632+0000 7f1fef06f700 1 -- 172.21.15.77:0/3678808904 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fe8191760 con 0x7f1fe8100e10 2023-12-01T01:10:36.635 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:36.632+0000 7f1fe5ffb700 1 --2- 172.21.15.77:0/3678808904 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f1fd80383f0 0x7f1fd803a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:36.635 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:36.632+0000 7f1fe5ffb700 1 -- 172.21.15.77:0/3678808904 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1fdc04c540 con 0x7f1fe8100e10 2023-12-01T01:10:36.635 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:36.632+0000 7f1fe7fff700 1 --2- 172.21.15.77:0/3678808904 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f1fd80383f0 0x7f1fd803a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:36.635 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:36.632+0000 7f1fe7fff700 1 --2- 172.21.15.77:0/3678808904 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f1fd80383f0 0x7f1fd803a8b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4006fd0 tx=0x7f1fd4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:36.638 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:36.637+0000 7f1fe5ffb700 1 -- 172.21.15.77:0/3678808904 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1fdc02a950 con 0x7f1fe8100e10 2023-12-01T01:10:36.827 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:36.825+0000 7f1fef06f700 1 -- 172.21.15.77:0/3678808904 --> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f1fe8061260 con 0x7f1fd80383f0 2023-12-01T01:10:36.830 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:36.829+0000 7f1fe5ffb700 1 -- 172.21.15.77:0/3678808904 <== mgr.14152 v2:172.21.15.77:6800/3838941679 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1fe8061260 con 0x7f1fd80383f0 2023-12-01T01:10:36.832 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:36.831+0000 7f1fef06f700 1 -- 172.21.15.77:0/3678808904 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f1fd80383f0 msgr2=0x7f1fd803a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:36.832 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:36.831+0000 7f1fef06f700 1 --2- 172.21.15.77:0/3678808904 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f1fd80383f0 0x7f1fd803a8b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4006fd0 tx=0x7f1fd4006e40 comp rx=0 tx=0).stop 2023-12-01T01:10:36.832 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:36.831+0000 7f1fef06f700 1 -- 172.21.15.77:0/3678808904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1fe8100e10 msgr2=0x7f1fe8197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:36.832 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:36.831+0000 7f1fef06f700 1 --2- 172.21.15.77:0/3678808904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1fe8100e10 0x7f1fe8197b70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc004f40 tx=0x7f1fdc0057c0 comp rx=0 tx=0).stop 2023-12-01T01:10:36.833 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:36.831+0000 7f1fef06f700 1 -- 172.21.15.77:0/3678808904 shutdown_connections 2023-12-01T01:10:36.833 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:36.831+0000 7f1fef06f700 1 --2- 172.21.15.77:0/3678808904 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f1fd80383f0 0x7f1fd803a8b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:36.833 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:36.831+0000 7f1fef06f700 1 --2- 172.21.15.77:0/3678808904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1fe8100e10 0x7f1fe8197b70 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:36.833 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:36.831+0000 7f1fef06f700 1 -- 172.21.15.77:0/3678808904 >> 172.21.15.77:0/3678808904 conn(0x7f1fe80fa490 msgr2=0x7f1fe80ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:36.833 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:36.831+0000 7f1fef06f700 1 -- 172.21.15.77:0/3678808904 shutdown_connections 2023-12-01T01:10:36.833 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:36.831+0000 7f1fef06f700 1 -- 172.21.15.77:0/3678808904 wait complete. 2023-12-01T01:10:36.855 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:36 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:10:36.855 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:36 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:10:36.855 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:36 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi077", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:10:36.855 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:36 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:10:36.855 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:36 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:10:36.855 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:36 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:10:37.298 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi137 2023-12-01T01:10:37.298 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2023-12-01T01:10:37.298 DEBUG:teuthology.orchestra.run.smithi137:> dd of=/etc/ceph/ceph.conf 2023-12-01T01:10:37.318 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2023-12-01T01:10:37.319 DEBUG:teuthology.orchestra.run.smithi137:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-01T01:10:37.378 INFO:tasks.cephadm:Adding host smithi137 to orchestrator... 2023-12-01T01:10:37.378 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph orch host add smithi137 2023-12-01T01:10:37.932 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:10:38.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:37 smithi077 ceph-mon[98867]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:10:38.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:37 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:10:38.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:37 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:10:38.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:37 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:10:38.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:37 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:10:38.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:37 smithi077 ceph-mon[98867]: Updating smithi077:/etc/ceph/ceph.conf 2023-12-01T01:10:38.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:37 smithi077 ceph-mon[98867]: Updating smithi077:/var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-01T01:10:38.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:37 smithi077 ceph-mon[98867]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2023-12-01T01:10:38.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:37 smithi077 ceph-mon[98867]: Updating smithi077:/var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-12-01T01:10:38.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:37 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:10:38.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:37 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:10:38.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:37 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:10:39.113 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:39.110+0000 7f3b85665700 1 -- 172.21.15.77:0/2795541023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b80106820 msgr2=0x7f3b80106c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:39.113 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:39.110+0000 7f3b85665700 1 --2- 172.21.15.77:0/2795541023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b80106820 0x7f3b80106c00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3b74009910 tx=0x7f3b74009c20 comp rx=0 tx=0).stop 2023-12-01T01:10:39.113 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:39.110+0000 7f3b85665700 1 -- 172.21.15.77:0/2795541023 shutdown_connections 2023-12-01T01:10:39.113 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:39.110+0000 7f3b85665700 1 --2- 172.21.15.77:0/2795541023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b80106820 0x7f3b80106c00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:39.113 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:39.110+0000 7f3b85665700 1 -- 172.21.15.77:0/2795541023 >> 172.21.15.77:0/2795541023 conn(0x7f3b800fc350 msgr2=0x7f3b800fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:39.113 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:39.110+0000 7f3b85665700 1 -- 172.21.15.77:0/2795541023 shutdown_connections 2023-12-01T01:10:39.113 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:39.110+0000 7f3b85665700 1 -- 172.21.15.77:0/2795541023 wait complete. 2023-12-01T01:10:39.113 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:39.110+0000 7f3b85665700 1 Processor -- start 2023-12-01T01:10:39.113 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:39.111+0000 7f3b85665700 1 -- start start 2023-12-01T01:10:39.113 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:39.111+0000 7f3b85665700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b80106820 0x7f3b8019bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:39.114 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:39.111+0000 7f3b85665700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8019c000 con 0x7f3b80106820 2023-12-01T01:10:39.114 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:39.111+0000 7f3b7effd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b80106820 0x7f3b8019bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:39.114 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:39.111+0000 7f3b7effd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b80106820 0x7f3b8019bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56488/0 (socket says 172.21.15.77:56488) 2023-12-01T01:10:39.114 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:39.111+0000 7f3b7effd700 1 -- 172.21.15.77:0/2900624352 learned_addr learned my addr 172.21.15.77:0/2900624352 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:39.114 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:39.111+0000 7f3b7effd700 1 -- 172.21.15.77:0/2900624352 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b740095f0 con 0x7f3b80106820 2023-12-01T01:10:39.114 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:39.111+0000 7f3b7effd700 1 --2- 172.21.15.77:0/2900624352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b80106820 0x7f3b8019bac0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3b74004f40 tx=0x7f3b74005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:39.114 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:39.111+0000 7f3b67fff700 1 -- 172.21.15.77:0/2900624352 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3b7401d070 con 0x7f3b80106820 2023-12-01T01:10:39.114 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:39.112+0000 7f3b67fff700 1 -- 172.21.15.77:0/2900624352 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b74004370 con 0x7f3b80106820 2023-12-01T01:10:39.114 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:39.112+0000 7f3b85665700 1 -- 172.21.15.77:0/2900624352 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b8019c260 con 0x7f3b80106820 2023-12-01T01:10:39.114 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:39.112+0000 7f3b67fff700 1 -- 172.21.15.77:0/2900624352 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3b74003970 con 0x7f3b80106820 2023-12-01T01:10:39.114 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:39.112+0000 7f3b85665700 1 -- 172.21.15.77:0/2900624352 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b80195b40 con 0x7f3b80106820 2023-12-01T01:10:39.115 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:39.112+0000 7f3b67fff700 1 -- 172.21.15.77:0/2900624352 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) v1 ==== 45324+0+0 (secure 0 0 0) 0x7f3b74003be0 con 0x7f3b80106820 2023-12-01T01:10:39.115 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:39.112+0000 7f3b67fff700 1 --2- 172.21.15.77:0/2900624352 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f3b680383f0 0x7f3b6803a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:39.115 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:39.112+0000 7f3b85665700 1 -- 172.21.15.77:0/2900624352 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b800623c0 con 0x7f3b80106820 2023-12-01T01:10:39.115 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:39.112+0000 7f3b67fff700 1 -- 172.21.15.77:0/2900624352 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3b7404bf60 con 0x7f3b80106820 2023-12-01T01:10:39.115 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:39.113+0000 7f3b7e7fc700 1 --2- 172.21.15.77:0/2900624352 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f3b680383f0 0x7f3b6803a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:39.115 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:39.113+0000 7f3b7e7fc700 1 --2- 172.21.15.77:0/2900624352 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f3b680383f0 0x7f3b6803a8b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3b70006fd0 tx=0x7f3b70006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:39.118 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:39.117+0000 7f3b67fff700 1 -- 172.21.15.77:0/2900624352 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3b74026070 con 0x7f3b80106820 2023-12-01T01:10:39.311 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:39.309+0000 7f3b85665700 1 -- 172.21.15.77:0/2900624352 --> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi137", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b80195d30 con 0x7f3b680383f0 2023-12-01T01:10:39.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:39 smithi077 ceph-mon[98867]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi137", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:10:40.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:40 smithi077 ceph-mon[98867]: Deploying cephadm binary to smithi137 2023-12-01T01:10:41.588 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:41.587+0000 7f3b67fff700 1 -- 172.21.15.77:0/2900624352 <== mgr.14152 v2:172.21.15.77:6800/3838941679 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f3b80195d30 con 0x7f3b680383f0 2023-12-01T01:10:41.589 INFO:teuthology.orchestra.run.smithi077.stdout:Added host 'smithi137' with addr '172.21.15.137' 2023-12-01T01:10:41.590 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:41.589+0000 7f3b85665700 1 -- 172.21.15.77:0/2900624352 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f3b680383f0 msgr2=0x7f3b6803a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:41.590 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:41.589+0000 7f3b85665700 1 --2- 172.21.15.77:0/2900624352 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f3b680383f0 0x7f3b6803a8b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3b70006fd0 tx=0x7f3b70006e40 comp rx=0 tx=0).stop 2023-12-01T01:10:41.590 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:41.589+0000 7f3b85665700 1 -- 172.21.15.77:0/2900624352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b80106820 msgr2=0x7f3b8019bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:41.591 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:41.589+0000 7f3b85665700 1 --2- 172.21.15.77:0/2900624352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b80106820 0x7f3b8019bac0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3b74004f40 tx=0x7f3b74005020 comp rx=0 tx=0).stop 2023-12-01T01:10:41.591 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:41.589+0000 7f3b85665700 1 -- 172.21.15.77:0/2900624352 shutdown_connections 2023-12-01T01:10:41.591 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:41.589+0000 7f3b85665700 1 --2- 172.21.15.77:0/2900624352 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f3b680383f0 0x7f3b6803a8b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:41.591 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:41.589+0000 7f3b85665700 1 --2- 172.21.15.77:0/2900624352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b80106820 0x7f3b8019bac0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:41.591 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:41.589+0000 7f3b85665700 1 -- 172.21.15.77:0/2900624352 >> 172.21.15.77:0/2900624352 conn(0x7f3b800fc350 msgr2=0x7f3b80108cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:41.591 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:41.589+0000 7f3b85665700 1 -- 172.21.15.77:0/2900624352 shutdown_connections 2023-12-01T01:10:41.591 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:41.589+0000 7f3b85665700 1 -- 172.21.15.77:0/2900624352 wait complete. 2023-12-01T01:10:41.992 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph orch host ls --format=json 2023-12-01T01:10:42.543 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:10:42.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:42 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:10:42.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:42 smithi077 ceph-mon[98867]: Added host smithi137 2023-12-01T01:10:42.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:42 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:10:42.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:42 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:10:43.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:43.770+0000 7f9dd73a2700 1 -- 172.21.15.77:0/4038619210 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9dd0105710 msgr2=0x7f9dd0105af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:43.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:43.770+0000 7f9dd73a2700 1 --2- 172.21.15.77:0/4038619210 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9dd0105710 0x7f9dd0105af0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc009910 tx=0x7f9dcc009c20 comp rx=0 tx=0).stop 2023-12-01T01:10:43.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:43.770+0000 7f9dd73a2700 1 -- 172.21.15.77:0/4038619210 shutdown_connections 2023-12-01T01:10:43.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:43.770+0000 7f9dd73a2700 1 --2- 172.21.15.77:0/4038619210 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9dd0105710 0x7f9dd0105af0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:43.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:43.770+0000 7f9dd73a2700 1 -- 172.21.15.77:0/4038619210 >> 172.21.15.77:0/4038619210 conn(0x7f9dd00fb220 msgr2=0x7f9dd00fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:43.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:43.770+0000 7f9dd73a2700 1 -- 172.21.15.77:0/4038619210 shutdown_connections 2023-12-01T01:10:43.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:43.770+0000 7f9dd73a2700 1 -- 172.21.15.77:0/4038619210 wait complete. 2023-12-01T01:10:43.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:43.771+0000 7f9dd73a2700 1 Processor -- start 2023-12-01T01:10:43.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:43.771+0000 7f9dd73a2700 1 -- start start 2023-12-01T01:10:43.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:43.771+0000 7f9dd73a2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9dd0105710 0x7f9dd019a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:43.774 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:43.771+0000 7f9dd73a2700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd019aec0 con 0x7f9dd0105710 2023-12-01T01:10:43.774 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:43.771+0000 7f9dd513e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9dd0105710 0x7f9dd019a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:43.774 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:43.771+0000 7f9dd513e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9dd0105710 0x7f9dd019a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56514/0 (socket says 172.21.15.77:56514) 2023-12-01T01:10:43.774 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:43.771+0000 7f9dd513e700 1 -- 172.21.15.77:0/2278524670 learned_addr learned my addr 172.21.15.77:0/2278524670 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:43.774 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:43.771+0000 7f9dd513e700 1 -- 172.21.15.77:0/2278524670 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dcc0095f0 con 0x7f9dd0105710 2023-12-01T01:10:43.774 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:43.771+0000 7f9dd513e700 1 --2- 172.21.15.77:0/2278524670 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9dd0105710 0x7f9dd019a980 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc004f40 tx=0x7f9dcc005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:43.774 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:43.772+0000 7f9dc27fc700 1 -- 172.21.15.77:0/2278524670 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9dcc01d070 con 0x7f9dd0105710 2023-12-01T01:10:43.774 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:43.772+0000 7f9dc27fc700 1 -- 172.21.15.77:0/2278524670 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9dcc004370 con 0x7f9dd0105710 2023-12-01T01:10:43.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:43.772+0000 7f9dd73a2700 1 -- 172.21.15.77:0/2278524670 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dd019b120 con 0x7f9dd0105710 2023-12-01T01:10:43.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:43.772+0000 7f9dc27fc700 1 -- 172.21.15.77:0/2278524670 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9dcc003970 con 0x7f9dd0105710 2023-12-01T01:10:43.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:43.772+0000 7f9dd73a2700 1 -- 172.21.15.77:0/2278524670 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dd0194a00 con 0x7f9dd0105710 2023-12-01T01:10:43.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:43.772+0000 7f9dc27fc700 1 -- 172.21.15.77:0/2278524670 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) v1 ==== 45324+0+0 (secure 0 0 0) 0x7f9dcc003be0 con 0x7f9dd0105710 2023-12-01T01:10:43.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:43.773+0000 7f9dd73a2700 1 -- 172.21.15.77:0/2278524670 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dd00623c0 con 0x7f9dd0105710 2023-12-01T01:10:43.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:43.773+0000 7f9dc27fc700 1 --2- 172.21.15.77:0/2278524670 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f9dbc0383f0 0x7f9dbc03a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:43.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:43.773+0000 7f9dc27fc700 1 -- 172.21.15.77:0/2278524670 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9dcc04bf90 con 0x7f9dd0105710 2023-12-01T01:10:43.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:43.773+0000 7f9dd493d700 1 --2- 172.21.15.77:0/2278524670 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f9dbc0383f0 0x7f9dbc03a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:43.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:43.773+0000 7f9dd493d700 1 --2- 172.21.15.77:0/2278524670 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f9dbc0383f0 0x7f9dbc03a8b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4006fd0 tx=0x7f9dc4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:43.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:43.778+0000 7f9dc27fc700 1 -- 172.21.15.77:0/2278524670 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9dcc02a590 con 0x7f9dd0105710 2023-12-01T01:10:43.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:43 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:10:43.972 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:43.970+0000 7f9dd73a2700 1 -- 172.21.15.77:0/2278524670 --> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9dd0195430 con 0x7f9dbc0383f0 2023-12-01T01:10:43.973 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:43.972+0000 7f9dc27fc700 1 -- 172.21.15.77:0/2278524670 <== mgr.14152 v2:172.21.15.77:6800/3838941679 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7f9dd0195430 con 0x7f9dbc0383f0 2023-12-01T01:10:43.973 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:10:43.973 INFO:teuthology.orchestra.run.smithi077.stdout:[{"addr": "172.21.15.77", "hostname": "smithi077", "labels": [], "status": ""}, {"addr": "172.21.15.137", "hostname": "smithi137", "labels": [], "status": ""}] 2023-12-01T01:10:43.975 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:43.974+0000 7f9dd73a2700 1 -- 172.21.15.77:0/2278524670 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f9dbc0383f0 msgr2=0x7f9dbc03a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:43.975 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:43.974+0000 7f9dd73a2700 1 --2- 172.21.15.77:0/2278524670 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f9dbc0383f0 0x7f9dbc03a8b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4006fd0 tx=0x7f9dc4006e40 comp rx=0 tx=0).stop 2023-12-01T01:10:43.975 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:43.974+0000 7f9dd73a2700 1 -- 172.21.15.77:0/2278524670 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9dd0105710 msgr2=0x7f9dd019a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:43.975 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:43.974+0000 7f9dd73a2700 1 --2- 172.21.15.77:0/2278524670 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9dd0105710 0x7f9dd019a980 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc004f40 tx=0x7f9dcc005020 comp rx=0 tx=0).stop 2023-12-01T01:10:43.975 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:43.974+0000 7f9dd73a2700 1 -- 172.21.15.77:0/2278524670 shutdown_connections 2023-12-01T01:10:43.975 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:43.974+0000 7f9dd73a2700 1 --2- 172.21.15.77:0/2278524670 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f9dbc0383f0 0x7f9dbc03a8b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:43.975 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:43.974+0000 7f9dd73a2700 1 --2- 172.21.15.77:0/2278524670 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9dd0105710 0x7f9dd019a980 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:43.976 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:43.974+0000 7f9dd73a2700 1 -- 172.21.15.77:0/2278524670 >> 172.21.15.77:0/2278524670 conn(0x7f9dd00fb220 msgr2=0x7f9dd0107c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:43.976 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:43.974+0000 7f9dd73a2700 1 -- 172.21.15.77:0/2278524670 shutdown_connections 2023-12-01T01:10:43.976 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:43.974+0000 7f9dd73a2700 1 -- 172.21.15.77:0/2278524670 wait complete. 2023-12-01T01:10:44.381 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi203 2023-12-01T01:10:44.381 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-12-01T01:10:44.382 DEBUG:teuthology.orchestra.run.smithi203:> dd of=/etc/ceph/ceph.conf 2023-12-01T01:10:44.403 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-12-01T01:10:44.403 DEBUG:teuthology.orchestra.run.smithi203:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-01T01:10:44.462 INFO:tasks.cephadm:Adding host smithi203 to orchestrator... 2023-12-01T01:10:44.462 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph orch host add smithi203 2023-12-01T01:10:45.010 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:10:45.264 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:44 smithi077 ceph-mon[98867]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-01T01:10:45.264 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:44 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:10:45.265 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:44 smithi077 ceph-mon[98867]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:10:46.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:46.208+0000 7f1dc4742700 1 -- 172.21.15.77:0/1733465156 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1dbc105700 msgr2=0x7f1dbc105ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:46.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:46.208+0000 7f1dc4742700 1 --2- 172.21.15.77:0/1733465156 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1dbc105700 0x7f1dbc105ae0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1dac009910 tx=0x7f1dac009c20 comp rx=0 tx=0).stop 2023-12-01T01:10:46.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:46.208+0000 7f1dc4742700 1 -- 172.21.15.77:0/1733465156 shutdown_connections 2023-12-01T01:10:46.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:46.208+0000 7f1dc4742700 1 --2- 172.21.15.77:0/1733465156 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1dbc105700 0x7f1dbc105ae0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:46.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:46.208+0000 7f1dc4742700 1 -- 172.21.15.77:0/1733465156 >> 172.21.15.77:0/1733465156 conn(0x7f1dbc0fb210 msgr2=0x7f1dbc0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:46.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:46.208+0000 7f1dc4742700 1 -- 172.21.15.77:0/1733465156 shutdown_connections 2023-12-01T01:10:46.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:46.208+0000 7f1dc4742700 1 -- 172.21.15.77:0/1733465156 wait complete. 2023-12-01T01:10:46.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:46.209+0000 7f1dc4742700 1 Processor -- start 2023-12-01T01:10:46.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:46.209+0000 7f1dc4742700 1 -- start start 2023-12-01T01:10:46.215 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:46.209+0000 7f1dc4742700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1dbc105700 0x7f1dbc196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:46.215 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:46.209+0000 7f1dc4742700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dbc197080 con 0x7f1dbc105700 2023-12-01T01:10:46.215 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:46.209+0000 7f1dc24de700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1dbc105700 0x7f1dbc196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:46.215 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:46.209+0000 7f1dc24de700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1dbc105700 0x7f1dbc196b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56520/0 (socket says 172.21.15.77:56520) 2023-12-01T01:10:46.215 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:46.209+0000 7f1dc24de700 1 -- 172.21.15.77:0/3058030211 learned_addr learned my addr 172.21.15.77:0/3058030211 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:46.215 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:46.209+0000 7f1dc24de700 1 -- 172.21.15.77:0/3058030211 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dac0095f0 con 0x7f1dbc105700 2023-12-01T01:10:46.215 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:46.209+0000 7f1dc24de700 1 --2- 172.21.15.77:0/3058030211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1dbc105700 0x7f1dbc196b40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1dac006010 tx=0x7f1dac004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:46.215 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:46.209+0000 7f1db37fe700 1 -- 172.21.15.77:0/3058030211 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1dac01d070 con 0x7f1dbc105700 2023-12-01T01:10:46.215 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:46.209+0000 7f1db37fe700 1 -- 172.21.15.77:0/3058030211 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1dac00bbb0 con 0x7f1dbc105700 2023-12-01T01:10:46.215 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:46.209+0000 7f1dc4742700 1 -- 172.21.15.77:0/3058030211 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dbc1972e0 con 0x7f1dbc105700 2023-12-01T01:10:46.215 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:46.209+0000 7f1db37fe700 1 -- 172.21.15.77:0/3058030211 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1dac003930 con 0x7f1dbc105700 2023-12-01T01:10:46.215 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:46.210+0000 7f1dc4742700 1 -- 172.21.15.77:0/3058030211 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dbc197760 con 0x7f1dbc105700 2023-12-01T01:10:46.215 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:46.210+0000 7f1db37fe700 1 -- 172.21.15.77:0/3058030211 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) v1 ==== 45324+0+0 (secure 0 0 0) 0x7f1dac003b70 con 0x7f1dbc105700 2023-12-01T01:10:46.215 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:46.210+0000 7f1db37fe700 1 --2- 172.21.15.77:0/3058030211 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f1da80383a0 0x7f1da803a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:46.216 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:46.210+0000 7f1db37fe700 1 -- 172.21.15.77:0/3058030211 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1dac04bfc0 con 0x7f1dbc105700 2023-12-01T01:10:46.216 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:46.211+0000 7f1dc1cdd700 1 --2- 172.21.15.77:0/3058030211 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f1da80383a0 0x7f1da803a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:46.216 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:46.211+0000 7f1dc4742700 1 -- 172.21.15.77:0/3058030211 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dbc1979a0 con 0x7f1dbc105700 2023-12-01T01:10:46.216 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:46.211+0000 7f1dc1cdd700 1 --2- 172.21.15.77:0/3058030211 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f1da80383a0 0x7f1da803a860 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1db8006fd0 tx=0x7f1db8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:46.216 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:46.215+0000 7f1db37fe700 1 -- 172.21.15.77:0/3058030211 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1dac026020 con 0x7f1dbc105700 2023-12-01T01:10:46.410 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:46.409+0000 7f1dc4742700 1 -- 172.21.15.77:0/3058030211 --> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi203", "target": ["mon-mgr", ""]}) v1 -- 0x7f1dbc068e70 con 0x7f1da80383a0 2023-12-01T01:10:47.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:47 smithi077 ceph-mon[98867]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:10:47.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:47 smithi077 ceph-mon[98867]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi203", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:10:47.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:47 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:10:47.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:47 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:10:47.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:47 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:10:47.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:47 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:10:47.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:47 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi137", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:10:47.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:47 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:10:47.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:47 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:10:48.732 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:48.730+0000 7f1db37fe700 1 -- 172.21.15.77:0/3058030211 <== mgr.14152 v2:172.21.15.77:6800/3838941679 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f1dbc068e70 con 0x7f1da80383a0 2023-12-01T01:10:48.732 INFO:teuthology.orchestra.run.smithi077.stdout:Added host 'smithi203' with addr '172.21.15.203' 2023-12-01T01:10:48.734 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:48.732+0000 7f1dc4742700 1 -- 172.21.15.77:0/3058030211 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f1da80383a0 msgr2=0x7f1da803a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:48.734 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:48.732+0000 7f1dc4742700 1 --2- 172.21.15.77:0/3058030211 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f1da80383a0 0x7f1da803a860 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1db8006fd0 tx=0x7f1db8006e40 comp rx=0 tx=0).stop 2023-12-01T01:10:48.734 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:48.732+0000 7f1dc4742700 1 -- 172.21.15.77:0/3058030211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1dbc105700 msgr2=0x7f1dbc196b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:48.734 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:48.732+0000 7f1dc4742700 1 --2- 172.21.15.77:0/3058030211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1dbc105700 0x7f1dbc196b40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1dac006010 tx=0x7f1dac004ff0 comp rx=0 tx=0).stop 2023-12-01T01:10:48.734 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:48.732+0000 7f1dc4742700 1 -- 172.21.15.77:0/3058030211 shutdown_connections 2023-12-01T01:10:48.734 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:48.732+0000 7f1dc4742700 1 --2- 172.21.15.77:0/3058030211 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f1da80383a0 0x7f1da803a860 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:48.734 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:48.732+0000 7f1dc4742700 1 --2- 172.21.15.77:0/3058030211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1dbc105700 0x7f1dbc196b40 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:48.734 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:48.732+0000 7f1dc4742700 1 -- 172.21.15.77:0/3058030211 >> 172.21.15.77:0/3058030211 conn(0x7f1dbc0fb210 msgr2=0x7f1dbc107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:48.735 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:48.733+0000 7f1dc4742700 1 -- 172.21.15.77:0/3058030211 shutdown_connections 2023-12-01T01:10:48.735 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:48.733+0000 7f1dc4742700 1 -- 172.21.15.77:0/3058030211 wait complete. 2023-12-01T01:10:48.762 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:48 smithi077 ceph-mon[98867]: Updating smithi137:/etc/ceph/ceph.conf 2023-12-01T01:10:48.762 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:48 smithi077 ceph-mon[98867]: Deploying cephadm binary to smithi203 2023-12-01T01:10:48.762 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:48 smithi077 ceph-mon[98867]: Updating smithi137:/var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-01T01:10:48.762 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:48 smithi077 ceph-mon[98867]: Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2023-12-01T01:10:48.762 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:48 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:10:48.762 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:48 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:10:48.762 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:48 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:10:49.158 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph orch host ls --format=json 2023-12-01T01:10:49.708 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:10:49.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:49 smithi077 ceph-mon[98867]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:10:49.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:49 smithi077 ceph-mon[98867]: Updating smithi137:/var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-12-01T01:10:49.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:49 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:10:49.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:49 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:10:49.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:49 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:10:50.904 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:50.901+0000 7f8cbf615700 1 -- 172.21.15.77:0/489046721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8cb8105780 msgr2=0x7f8cb8105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:50.904 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:50.901+0000 7f8cbf615700 1 --2- 172.21.15.77:0/489046721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8cb8105780 0x7f8cb8105b60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8cac009910 tx=0x7f8cac009c20 comp rx=0 tx=0).stop 2023-12-01T01:10:50.904 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:50.901+0000 7f8cbf615700 1 -- 172.21.15.77:0/489046721 shutdown_connections 2023-12-01T01:10:50.904 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:50.901+0000 7f8cbf615700 1 --2- 172.21.15.77:0/489046721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8cb8105780 0x7f8cb8105b60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:50.904 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:50.901+0000 7f8cbf615700 1 -- 172.21.15.77:0/489046721 >> 172.21.15.77:0/489046721 conn(0x7f8cb8075290 msgr2=0x7f8cb80756a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:50.904 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:50.901+0000 7f8cbf615700 1 -- 172.21.15.77:0/489046721 shutdown_connections 2023-12-01T01:10:50.904 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:50.901+0000 7f8cbf615700 1 -- 172.21.15.77:0/489046721 wait complete. 2023-12-01T01:10:50.904 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:50.902+0000 7f8cbf615700 1 Processor -- start 2023-12-01T01:10:50.905 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:50.902+0000 7f8cbf615700 1 -- start start 2023-12-01T01:10:50.905 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:50.902+0000 7f8cbf615700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8cb8105780 0x7f8cb80fc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:50.905 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:50.902+0000 7f8cbf615700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb80fe760 con 0x7f8cb8105780 2023-12-01T01:10:50.905 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:50.902+0000 7f8cbd3b1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8cb8105780 0x7f8cb80fc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:50.905 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:50.902+0000 7f8cbd3b1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8cb8105780 0x7f8cb80fc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56546/0 (socket says 172.21.15.77:56546) 2023-12-01T01:10:50.905 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:50.902+0000 7f8cbd3b1700 1 -- 172.21.15.77:0/835761181 learned_addr learned my addr 172.21.15.77:0/835761181 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:50.905 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:50.902+0000 7f8cbd3b1700 1 -- 172.21.15.77:0/835761181 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cac0095f0 con 0x7f8cb8105780 2023-12-01T01:10:50.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:50 smithi077 ceph-mon[98867]: Added host smithi203 2023-12-01T01:10:50.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:50 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:10:50.907 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:50.902+0000 7f8cbd3b1700 1 --2- 172.21.15.77:0/835761181 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8cb8105780 0x7f8cb80fc7e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8cac005850 tx=0x7f8cac004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:50.907 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:50.903+0000 7f8caa7fc700 1 -- 172.21.15.77:0/835761181 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8cac01d070 con 0x7f8cb8105780 2023-12-01T01:10:50.907 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:50.903+0000 7f8caa7fc700 1 -- 172.21.15.77:0/835761181 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8cac004370 con 0x7f8cb8105780 2023-12-01T01:10:50.907 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:50.903+0000 7f8caa7fc700 1 -- 172.21.15.77:0/835761181 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8cac003ab0 con 0x7f8cb8105780 2023-12-01T01:10:50.907 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:50.903+0000 7f8cbf615700 1 -- 172.21.15.77:0/835761181 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cb80fcd20 con 0x7f8cb8105780 2023-12-01T01:10:50.907 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:50.903+0000 7f8cbf615700 1 -- 172.21.15.77:0/835761181 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cb80fd1a0 con 0x7f8cb8105780 2023-12-01T01:10:50.907 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:50.904+0000 7f8caa7fc700 1 -- 172.21.15.77:0/835761181 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) v1 ==== 45324+0+0 (secure 0 0 0) 0x7f8cac003d20 con 0x7f8cb8105780 2023-12-01T01:10:50.907 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:50.904+0000 7f8caa7fc700 1 --2- 172.21.15.77:0/835761181 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8ca40383f0 0x7f8ca403a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:50.907 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:50.904+0000 7f8caa7fc700 1 -- 172.21.15.77:0/835761181 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8cac04c0e0 con 0x7f8cb8105780 2023-12-01T01:10:50.907 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:50.904+0000 7f8cbcbb0700 1 --2- 172.21.15.77:0/835761181 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8ca40383f0 0x7f8ca403a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:50.907 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:50.904+0000 7f8cbf615700 1 -- 172.21.15.77:0/835761181 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cb80623c0 con 0x7f8cb8105780 2023-12-01T01:10:50.908 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:50.904+0000 7f8cbcbb0700 1 --2- 172.21.15.77:0/835761181 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8ca40383f0 0x7f8ca403a8b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4006fd0 tx=0x7f8cb4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:50.910 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:50.909+0000 7f8caa7fc700 1 -- 172.21.15.77:0/835761181 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8cac026070 con 0x7f8cb8105780 2023-12-01T01:10:51.103 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:51.102+0000 7f8cbf615700 1 -- 172.21.15.77:0/835761181 --> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8cb80fd480 con 0x7f8ca40383f0 2023-12-01T01:10:51.104 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:51.103+0000 7f8caa7fc700 1 -- 172.21.15.77:0/835761181 <== mgr.14152 v2:172.21.15.77:6800/3838941679 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+240 (secure 0 0 0) 0x7f8cb80fd480 con 0x7f8ca40383f0 2023-12-01T01:10:51.104 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:10:51.104 INFO:teuthology.orchestra.run.smithi077.stdout:[{"addr": "172.21.15.77", "hostname": "smithi077", "labels": [], "status": ""}, {"addr": "172.21.15.137", "hostname": "smithi137", "labels": [], "status": ""}, {"addr": "172.21.15.203", "hostname": "smithi203", "labels": [], "status": ""}] 2023-12-01T01:10:51.106 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:51.105+0000 7f8cbf615700 1 -- 172.21.15.77:0/835761181 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8ca40383f0 msgr2=0x7f8ca403a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:51.106 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:51.105+0000 7f8cbf615700 1 --2- 172.21.15.77:0/835761181 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8ca40383f0 0x7f8ca403a8b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4006fd0 tx=0x7f8cb4006e40 comp rx=0 tx=0).stop 2023-12-01T01:10:51.106 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:51.105+0000 7f8cbf615700 1 -- 172.21.15.77:0/835761181 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8cb8105780 msgr2=0x7f8cb80fc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:51.106 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:51.105+0000 7f8cbf615700 1 --2- 172.21.15.77:0/835761181 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8cb8105780 0x7f8cb80fc7e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8cac005850 tx=0x7f8cac004ff0 comp rx=0 tx=0).stop 2023-12-01T01:10:51.106 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:51.105+0000 7f8cbf615700 1 -- 172.21.15.77:0/835761181 shutdown_connections 2023-12-01T01:10:51.106 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:51.105+0000 7f8cbf615700 1 --2- 172.21.15.77:0/835761181 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8ca40383f0 0x7f8ca403a8b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:51.107 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:51.105+0000 7f8cbf615700 1 --2- 172.21.15.77:0/835761181 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8cb8105780 0x7f8cb80fc7e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:51.107 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:51.105+0000 7f8cbf615700 1 -- 172.21.15.77:0/835761181 >> 172.21.15.77:0/835761181 conn(0x7f8cb8075290 msgr2=0x7f8cb8067f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:51.107 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:51.105+0000 7f8cbf615700 1 -- 172.21.15.77:0/835761181 shutdown_connections 2023-12-01T01:10:51.107 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:51.105+0000 7f8cbf615700 1 -- 172.21.15.77:0/835761181 wait complete. 2023-12-01T01:10:51.460 INFO:tasks.cephadm:Setting crush tunables to default 2023-12-01T01:10:51.460 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph osd crush tunables default 2023-12-01T01:10:51.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:51 smithi077 ceph-mon[98867]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:10:51.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:51 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:10:52.010 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:10:52.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:52 smithi077 ceph-mon[98867]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-01T01:10:53.192 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:53.190+0000 7f6fc85d1700 1 -- 172.21.15.77:0/2867837207 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fc01046e0 msgr2=0x7f6fc0104ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:53.192 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:53.190+0000 7f6fc85d1700 1 --2- 172.21.15.77:0/2867837207 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fc01046e0 0x7f6fc0104ac0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0009910 tx=0x7f6fb0009c20 comp rx=0 tx=0).stop 2023-12-01T01:10:53.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:53.190+0000 7f6fc85d1700 1 -- 172.21.15.77:0/2867837207 shutdown_connections 2023-12-01T01:10:53.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:53.190+0000 7f6fc85d1700 1 --2- 172.21.15.77:0/2867837207 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fc01046e0 0x7f6fc0104ac0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:53.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:53.190+0000 7f6fc85d1700 1 -- 172.21.15.77:0/2867837207 >> 172.21.15.77:0/2867837207 conn(0x7f6fc0075e40 msgr2=0x7f6fc0076250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:53.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:53.190+0000 7f6fc85d1700 1 -- 172.21.15.77:0/2867837207 shutdown_connections 2023-12-01T01:10:53.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:53.190+0000 7f6fc85d1700 1 -- 172.21.15.77:0/2867837207 wait complete. 2023-12-01T01:10:53.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:53.191+0000 7f6fc85d1700 1 Processor -- start 2023-12-01T01:10:53.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:53.191+0000 7f6fc85d1700 1 -- start start 2023-12-01T01:10:53.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:53.191+0000 7f6fc85d1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fc01046e0 0x7f6fc019a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:53.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:53.191+0000 7f6fc85d1700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc019ae90 con 0x7f6fc01046e0 2023-12-01T01:10:53.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:53.192+0000 7f6fc636d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fc01046e0 0x7f6fc019a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:53.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:53.192+0000 7f6fc636d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fc01046e0 0x7f6fc019a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56552/0 (socket says 172.21.15.77:56552) 2023-12-01T01:10:53.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:53.192+0000 7f6fc636d700 1 -- 172.21.15.77:0/3192192902 learned_addr learned my addr 172.21.15.77:0/3192192902 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:10:53.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:53.192+0000 7f6fc636d700 1 -- 172.21.15.77:0/3192192902 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb00095f0 con 0x7f6fc01046e0 2023-12-01T01:10:53.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:53.192+0000 7f6fc636d700 1 --2- 172.21.15.77:0/3192192902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fc01046e0 0x7f6fc019a950 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0004f40 tx=0x7f6fb0005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:53.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:53.192+0000 7f6fb77fe700 1 -- 172.21.15.77:0/3192192902 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6fb001d070 con 0x7f6fc01046e0 2023-12-01T01:10:53.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:53.192+0000 7f6fb77fe700 1 -- 172.21.15.77:0/3192192902 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fb0004370 con 0x7f6fc01046e0 2023-12-01T01:10:53.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:53.192+0000 7f6fb77fe700 1 -- 172.21.15.77:0/3192192902 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6fb00039b0 con 0x7f6fc01046e0 2023-12-01T01:10:53.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:53.192+0000 7f6fc85d1700 1 -- 172.21.15.77:0/3192192902 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fc019b0f0 con 0x7f6fc01046e0 2023-12-01T01:10:53.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:53.192+0000 7f6fc85d1700 1 -- 172.21.15.77:0/3192192902 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fc0194940 con 0x7f6fc01046e0 2023-12-01T01:10:53.196 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:53.193+0000 7f6fb77fe700 1 -- 172.21.15.77:0/3192192902 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) v1 ==== 45324+0+0 (secure 0 0 0) 0x7f6fb0003c70 con 0x7f6fc01046e0 2023-12-01T01:10:53.196 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:53.193+0000 7f6fb77fe700 1 --2- 172.21.15.77:0/3192192902 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f6fac0383a0 0x7f6fac03a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:53.196 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:53.193+0000 7f6fb77fe700 1 -- 172.21.15.77:0/3192192902 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6fb004beb0 con 0x7f6fc01046e0 2023-12-01T01:10:53.196 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:53.193+0000 7f6fc85d1700 1 -- 172.21.15.77:0/3192192902 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fc0105140 con 0x7f6fc01046e0 2023-12-01T01:10:53.196 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:53.193+0000 7f6fc5b6c700 1 --2- 172.21.15.77:0/3192192902 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f6fac0383a0 0x7f6fac03a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:53.196 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:53.194+0000 7f6fc5b6c700 1 --2- 172.21.15.77:0/3192192902 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f6fac0383a0 0x7f6fac03a860 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc006fd0 tx=0x7f6fbc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:53.199 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:53.198+0000 7f6fb77fe700 1 -- 172.21.15.77:0/3192192902 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6fc0105140 con 0x7f6fc01046e0 2023-12-01T01:10:53.394 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:53.393+0000 7f6fc85d1700 1 -- 172.21.15.77:0/3192192902 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f6fc0105140 con 0x7f6fc01046e0 2023-12-01T01:10:53.518 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:53.517+0000 7f6fb77fe700 1 -- 172.21.15.77:0/3192192902 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f6fc0105140 con 0x7f6fc01046e0 2023-12-01T01:10:53.520 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:53.519+0000 7f6fc85d1700 1 -- 172.21.15.77:0/3192192902 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f6fac0383a0 msgr2=0x7f6fac03a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:53.521 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:53.519+0000 7f6fc85d1700 1 --2- 172.21.15.77:0/3192192902 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f6fac0383a0 0x7f6fac03a860 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc006fd0 tx=0x7f6fbc006e40 comp rx=0 tx=0).stop 2023-12-01T01:10:53.521 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:53.519+0000 7f6fc85d1700 1 -- 172.21.15.77:0/3192192902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fc01046e0 msgr2=0x7f6fc019a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:53.521 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:53.519+0000 7f6fc85d1700 1 --2- 172.21.15.77:0/3192192902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fc01046e0 0x7f6fc019a950 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0004f40 tx=0x7f6fb0005020 comp rx=0 tx=0).stop 2023-12-01T01:10:53.521 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:53.519+0000 7f6fc85d1700 1 -- 172.21.15.77:0/3192192902 shutdown_connections 2023-12-01T01:10:53.521 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:53.519+0000 7f6fc85d1700 1 --2- 172.21.15.77:0/3192192902 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f6fac0383a0 0x7f6fac03a860 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:53.521 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:53.519+0000 7f6fc85d1700 1 --2- 172.21.15.77:0/3192192902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fc01046e0 0x7f6fc019a950 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:53.521 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:53.519+0000 7f6fc85d1700 1 -- 172.21.15.77:0/3192192902 >> 172.21.15.77:0/3192192902 conn(0x7f6fc0075e40 msgr2=0x7f6fc00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:53.521 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:53.519+0000 7f6fc85d1700 1 -- 172.21.15.77:0/3192192902 shutdown_connections 2023-12-01T01:10:53.521 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:10:53.519+0000 7f6fc85d1700 1 -- 172.21.15.77:0/3192192902 wait complete. 2023-12-01T01:10:53.522 INFO:teuthology.orchestra.run.smithi077.stderr:adjusted tunables profile to default 2023-12-01T01:10:53.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:53 smithi077 ceph-mon[98867]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:10:53.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:53 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/3192192902' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-12-01T01:10:53.966 INFO:tasks.cephadm:Adding mon.a on smithi077 2023-12-01T01:10:53.966 INFO:tasks.cephadm:Adding mon.b on smithi137 2023-12-01T01:10:53.966 INFO:tasks.cephadm:Adding mon.c on smithi203 2023-12-01T01:10:53.966 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph orch apply mon '3;smithi077:172.21.15.77=a;smithi137:172.21.15.137=b;smithi203:172.21.15.203=c' 2023-12-01T01:10:54.571 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-01T01:10:54.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:54 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/3192192902' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-12-01T01:10:54.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:54 smithi077 ceph-mon[98867]: osdmap e4: 0 total, 0 up, 0 in 2023-12-01T01:10:54.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:54 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:10:54.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:54 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:10:54.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:54 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:10:54.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:54 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:10:54.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:54 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi203", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:10:54.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:54 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:10:54.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:54 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:10:55.882 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:55.878+0000 7f1e15ecd700 1 -- 172.21.15.203:0/3118628466 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e10105780 msgr2=0x7f1e10105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:55.882 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:55.878+0000 7f1e15ecd700 1 --2- 172.21.15.203:0/3118628466 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e10105780 0x7f1e10105b60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1df8009910 tx=0x7f1df8009c20 comp rx=0 tx=0).stop 2023-12-01T01:10:55.882 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:55.878+0000 7f1e15ecd700 1 -- 172.21.15.203:0/3118628466 shutdown_connections 2023-12-01T01:10:55.882 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:55.878+0000 7f1e15ecd700 1 --2- 172.21.15.203:0/3118628466 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e10105780 0x7f1e10105b60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:55.882 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:55.878+0000 7f1e15ecd700 1 -- 172.21.15.203:0/3118628466 >> 172.21.15.203:0/3118628466 conn(0x7f1e10075290 msgr2=0x7f1e100756a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:55.882 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:55.878+0000 7f1e15ecd700 1 -- 172.21.15.203:0/3118628466 shutdown_connections 2023-12-01T01:10:55.882 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:55.878+0000 7f1e15ecd700 1 -- 172.21.15.203:0/3118628466 wait complete. 2023-12-01T01:10:55.883 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:55.878+0000 7f1e15ecd700 1 Processor -- start 2023-12-01T01:10:55.883 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:55.878+0000 7f1e15ecd700 1 -- start start 2023-12-01T01:10:55.883 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:55.879+0000 7f1e15ecd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e10105780 0x7f1e1019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:55.883 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:55.879+0000 7f1e15ecd700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1019af40 con 0x7f1e10105780 2023-12-01T01:10:55.883 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:55.879+0000 7f1e0f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e10105780 0x7f1e1019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:55.883 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:55.879+0000 7f1e0f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e10105780 0x7f1e1019aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.203:58834/0 (socket says 172.21.15.203:58834) 2023-12-01T01:10:55.883 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:55.879+0000 7f1e0f7fe700 1 -- 172.21.15.203:0/1270700992 learned_addr learned my addr 172.21.15.203:0/1270700992 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-12-01T01:10:55.883 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:55.879+0000 7f1e0f7fe700 1 -- 172.21.15.203:0/1270700992 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df80095f0 con 0x7f1e10105780 2023-12-01T01:10:55.883 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:55.880+0000 7f1e0f7fe700 1 --2- 172.21.15.203:0/1270700992 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e10105780 0x7f1e1019aa00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1df8005850 tx=0x7f1df8004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:55.883 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:55.880+0000 7f1e0cff9700 1 -- 172.21.15.203:0/1270700992 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1df801d070 con 0x7f1e10105780 2023-12-01T01:10:55.883 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:55.880+0000 7f1e0cff9700 1 -- 172.21.15.203:0/1270700992 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1df8004370 con 0x7f1e10105780 2023-12-01T01:10:55.884 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:55.880+0000 7f1e15ecd700 1 -- 172.21.15.203:0/1270700992 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e1019b140 con 0x7f1e10105780 2023-12-01T01:10:55.884 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:55.880+0000 7f1e0cff9700 1 -- 172.21.15.203:0/1270700992 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1df8003ab0 con 0x7f1e10105780 2023-12-01T01:10:55.884 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:55.880+0000 7f1e15ecd700 1 -- 172.21.15.203:0/1270700992 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e1019b560 con 0x7f1e10105780 2023-12-01T01:10:55.884 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:55.881+0000 7f1e0cff9700 1 -- 172.21.15.203:0/1270700992 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) v1 ==== 45324+0+0 (secure 0 0 0) 0x7f1df800b6f0 con 0x7f1e10105780 2023-12-01T01:10:55.885 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:55.881+0000 7f1e15ecd700 1 -- 172.21.15.203:0/1270700992 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e100623c0 con 0x7f1e10105780 2023-12-01T01:10:55.885 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:55.881+0000 7f1e0cff9700 1 --2- 172.21.15.203:0/1270700992 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f1dfc0383a0 0x7f1dfc03a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:55.885 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:55.881+0000 7f1e0cff9700 1 -- 172.21.15.203:0/1270700992 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1df804c090 con 0x7f1e10105780 2023-12-01T01:10:55.885 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:55.881+0000 7f1e0effd700 1 --2- 172.21.15.203:0/1270700992 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f1dfc0383a0 0x7f1dfc03a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:55.885 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:55.882+0000 7f1e0effd700 1 --2- 172.21.15.203:0/1270700992 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f1dfc0383a0 0x7f1dfc03a860 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1e00006fd0 tx=0x7f1e00006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:55.888 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:55.886+0000 7f1e0cff9700 1 -- 172.21.15.203:0/1270700992 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1df802a590 con 0x7f1e10105780 2023-12-01T01:10:55.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:55 smithi077 ceph-mon[98867]: Updating smithi203:/etc/ceph/ceph.conf 2023-12-01T01:10:55.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:55 smithi077 ceph-mon[98867]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:10:55.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:55 smithi077 ceph-mon[98867]: Updating smithi203:/var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-01T01:10:55.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:55 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:10:55.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:55 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:10:55.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:55 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:10:56.085 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:56.083+0000 7f1e15ecd700 1 -- 172.21.15.203:0/1270700992 --> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi077:172.21.15.77=a;smithi137:172.21.15.137=b;smithi203:172.21.15.203=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e10103640 con 0x7f1dfc0383a0 2023-12-01T01:10:56.091 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:56.090+0000 7f1e0cff9700 1 -- 172.21.15.203:0/1270700992 <== mgr.14152 v2:172.21.15.77:6800/3838941679 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f1e10103640 con 0x7f1dfc0383a0 2023-12-01T01:10:56.091 INFO:teuthology.orchestra.run.smithi203.stdout:Scheduled mon update... 2023-12-01T01:10:56.093 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:56.091+0000 7f1e15ecd700 1 -- 172.21.15.203:0/1270700992 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f1dfc0383a0 msgr2=0x7f1dfc03a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:56.093 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:56.091+0000 7f1e15ecd700 1 --2- 172.21.15.203:0/1270700992 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f1dfc0383a0 0x7f1dfc03a860 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1e00006fd0 tx=0x7f1e00006e40 comp rx=0 tx=0).stop 2023-12-01T01:10:56.093 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:56.091+0000 7f1e15ecd700 1 -- 172.21.15.203:0/1270700992 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e10105780 msgr2=0x7f1e1019aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:56.094 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:56.091+0000 7f1e15ecd700 1 --2- 172.21.15.203:0/1270700992 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e10105780 0x7f1e1019aa00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1df8005850 tx=0x7f1df8004ff0 comp rx=0 tx=0).stop 2023-12-01T01:10:56.094 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:56.091+0000 7f1e15ecd700 1 -- 172.21.15.203:0/1270700992 shutdown_connections 2023-12-01T01:10:56.094 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:56.091+0000 7f1e15ecd700 1 --2- 172.21.15.203:0/1270700992 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f1dfc0383a0 0x7f1dfc03a860 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:56.094 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:56.091+0000 7f1e15ecd700 1 --2- 172.21.15.203:0/1270700992 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e10105780 0x7f1e1019aa00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:56.094 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:56.091+0000 7f1e15ecd700 1 -- 172.21.15.203:0/1270700992 >> 172.21.15.203:0/1270700992 conn(0x7f1e10075290 msgr2=0x7f1e100fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:56.094 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:56.091+0000 7f1e15ecd700 1 -- 172.21.15.203:0/1270700992 shutdown_connections 2023-12-01T01:10:56.094 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:56.091+0000 7f1e15ecd700 1 -- 172.21.15.203:0/1270700992 wait complete. 2023-12-01T01:10:56.528 DEBUG:teuthology.orchestra.run.smithi137:mon.b> sudo journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mon.b.service 2023-12-01T01:10:56.531 DEBUG:teuthology.orchestra.run.smithi203:mon.c> sudo journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mon.c.service 2023-12-01T01:10:56.533 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-12-01T01:10:56.533 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-01T01:10:56.563 INFO:journalctl@ceph.mon.c.smithi203.stdout:-- Logs begin at Fri 2023-12-01 00:55:29 UTC. -- 2023-12-01T01:10:56.567 INFO:journalctl@ceph.mon.b.smithi137.stdout:-- Logs begin at Fri 2023-12-01 00:55:04 UTC. -- 2023-12-01T01:10:56.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:56 smithi077 ceph-mon[98867]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2023-12-01T01:10:56.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:56 smithi077 ceph-mon[98867]: Updating smithi203:/var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-12-01T01:10:56.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:56 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:10:56.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:56 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:10:56.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:56 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:10:56.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:56 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:10:56.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:56 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:10:56.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:56 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-01T01:10:56.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:56 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:10:57.470 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-01T01:10:57.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:57 smithi077 ceph-mon[98867]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi077:172.21.15.77=a;smithi137:172.21.15.137=b;smithi203:172.21.15.203=c", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:10:57.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:57 smithi077 ceph-mon[98867]: Saving service mon spec with placement smithi077:172.21.15.77=a;smithi137:172.21.15.137=b;smithi203:172.21.15.203=c;count:3 2023-12-01T01:10:57.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:57 smithi077 ceph-mon[98867]: Deploying daemon mon.c on smithi203 2023-12-01T01:10:57.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:57 smithi077 ceph-mon[98867]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:10:59.374 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:59.370+0000 7fe696f71700 1 -- 172.21.15.203:0/924351468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe690106820 msgr2=0x7fe690106c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:59.374 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:59.370+0000 7fe696f71700 1 --2- 172.21.15.203:0/924351468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe690106820 0x7fe690106c00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe684009910 tx=0x7fe684009c20 comp rx=0 tx=0).stop 2023-12-01T01:10:59.374 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:59.370+0000 7fe696f71700 1 -- 172.21.15.203:0/924351468 shutdown_connections 2023-12-01T01:10:59.374 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:59.370+0000 7fe696f71700 1 --2- 172.21.15.203:0/924351468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe690106820 0x7fe690106c00 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:59.374 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:59.370+0000 7fe696f71700 1 -- 172.21.15.203:0/924351468 >> 172.21.15.203:0/924351468 conn(0x7fe690075290 msgr2=0x7fe6900756a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:59.374 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:59.372+0000 7fe696f71700 1 -- 172.21.15.203:0/924351468 shutdown_connections 2023-12-01T01:10:59.374 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:59.372+0000 7fe696f71700 1 -- 172.21.15.203:0/924351468 wait complete. 2023-12-01T01:10:59.376 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:59.372+0000 7fe696f71700 1 Processor -- start 2023-12-01T01:10:59.376 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:59.372+0000 7fe696f71700 1 -- start start 2023-12-01T01:10:59.376 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:59.373+0000 7fe696f71700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe690106820 0x7fe6900ff800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:59.376 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:59.373+0000 7fe696f71700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe690101780 con 0x7fe690106820 2023-12-01T01:10:59.376 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:59.373+0000 7fe694d0d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe690106820 0x7fe6900ff800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:59.376 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:59.373+0000 7fe694d0d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe690106820 0x7fe6900ff800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.203:58840/0 (socket says 172.21.15.203:58840) 2023-12-01T01:10:59.376 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:59.373+0000 7fe694d0d700 1 -- 172.21.15.203:0/94460557 learned_addr learned my addr 172.21.15.203:0/94460557 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-12-01T01:10:59.376 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:59.373+0000 7fe694d0d700 1 -- 172.21.15.203:0/94460557 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6840095f0 con 0x7fe690106820 2023-12-01T01:10:59.377 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:59.373+0000 7fe694d0d700 1 --2- 172.21.15.203:0/94460557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe690106820 0x7fe6900ff800 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe684004f40 tx=0x7fe684005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:59.377 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:59.374+0000 7fe68dffb700 1 -- 172.21.15.203:0/94460557 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe68401d070 con 0x7fe690106820 2023-12-01T01:10:59.377 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:59.374+0000 7fe68dffb700 1 -- 172.21.15.203:0/94460557 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe684004370 con 0x7fe690106820 2023-12-01T01:10:59.377 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:59.374+0000 7fe696f71700 1 -- 172.21.15.203:0/94460557 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6900ffd40 con 0x7fe690106820 2023-12-01T01:10:59.377 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:59.374+0000 7fe68dffb700 1 -- 172.21.15.203:0/94460557 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe6840039b0 con 0x7fe690106820 2023-12-01T01:10:59.377 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:59.374+0000 7fe696f71700 1 -- 172.21.15.203:0/94460557 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6901001c0 con 0x7fe690106820 2023-12-01T01:10:59.378 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:59.375+0000 7fe68dffb700 1 -- 172.21.15.203:0/94460557 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) v1 ==== 45324+0+0 (secure 0 0 0) 0x7fe684003c70 con 0x7fe690106820 2023-12-01T01:10:59.378 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:59.375+0000 7fe68dffb700 1 --2- 172.21.15.203:0/94460557 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fe6800383a0 0x7fe68003a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:10:59.378 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:59.375+0000 7fe68dffb700 1 -- 172.21.15.203:0/94460557 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe68404c100 con 0x7fe690106820 2023-12-01T01:10:59.378 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:59.375+0000 7fe696f71700 1 -- 172.21.15.203:0/94460557 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe690105ad0 con 0x7fe690106820 2023-12-01T01:10:59.378 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:59.375+0000 7fe68ffff700 1 --2- 172.21.15.203:0/94460557 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fe6800383a0 0x7fe68003a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:10:59.378 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:59.376+0000 7fe68ffff700 1 --2- 172.21.15.203:0/94460557 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fe6800383a0 0x7fe68003a860 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe67c006fd0 tx=0x7fe67c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:10:59.382 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:59.380+0000 7fe68dffb700 1 -- 172.21.15.203:0/94460557 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe684026070 con 0x7fe690106820 2023-12-01T01:10:59.660 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:59.658+0000 7fe696f71700 1 -- 172.21.15.203:0/94460557 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe690100e10 con 0x7fe690106820 2023-12-01T01:10:59.660 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:59.658+0000 7fe68dffb700 1 -- 172.21.15.203:0/94460557 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+736 (secure 0 0 0) 0x7fe68402b440 con 0x7fe690106820 2023-12-01T01:10:59.661 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-12-01T01:10:59.661 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":1,"fsid":"2a06e29a-8fe6-11ee-95a2-87774f69a715","modified":"2023-12-01T01:09:28.081844Z","created":"2023-12-01T01:09:28.081844Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:3300","nonce":0},{"type":"v1","addr":"172.21.15.77:6789","nonce":0}]},"addr":"172.21.15.77:6789/0","public_addr":"172.21.15.77:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-01T01:10:59.662 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:59.660+0000 7fe696f71700 1 -- 172.21.15.203:0/94460557 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fe6800383a0 msgr2=0x7fe68003a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:59.662 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:59.660+0000 7fe696f71700 1 --2- 172.21.15.203:0/94460557 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fe6800383a0 0x7fe68003a860 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe67c006fd0 tx=0x7fe67c006e40 comp rx=0 tx=0).stop 2023-12-01T01:10:59.662 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:59.660+0000 7fe696f71700 1 -- 172.21.15.203:0/94460557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe690106820 msgr2=0x7fe6900ff800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:10:59.662 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:59.660+0000 7fe696f71700 1 --2- 172.21.15.203:0/94460557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe690106820 0x7fe6900ff800 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe684004f40 tx=0x7fe684005020 comp rx=0 tx=0).stop 2023-12-01T01:10:59.663 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:59.660+0000 7fe696f71700 1 -- 172.21.15.203:0/94460557 shutdown_connections 2023-12-01T01:10:59.663 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:59.660+0000 7fe696f71700 1 --2- 172.21.15.203:0/94460557 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fe6800383a0 0x7fe68003a860 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:59.663 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:59.660+0000 7fe696f71700 1 --2- 172.21.15.203:0/94460557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe690106820 0x7fe6900ff800 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:10:59.663 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:59.660+0000 7fe696f71700 1 -- 172.21.15.203:0/94460557 >> 172.21.15.203:0/94460557 conn(0x7fe690075290 msgr2=0x7fe6900fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:10:59.663 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:59.661+0000 7fe696f71700 1 -- 172.21.15.203:0/94460557 shutdown_connections 2023-12-01T01:10:59.663 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:10:59.661+0000 7fe696f71700 1 -- 172.21.15.203:0/94460557 wait complete. 2023-12-01T01:10:59.663 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 1 2023-12-01T01:10:59.862 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 podman[103512]: 2023-12-01T01:10:59.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:10:59 smithi077 ceph-mon[98867]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:00.129 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 bash[103512]: 9cc0caf6465778fff05b24727386a0bb64aaef14978f64eb9021be9a0249c0f7 2023-12-01T01:11:00.129 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: set uid:gid to 167:167 (ceph:ceph) 2023-12-01T01:11:00.129 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: ceph version 18.2.0-1249-ga6f82997 (a6f82997ac57b70a895455dfed6256360a1e4c32) reef (stable), process ceph-mon, pid 7 2023-12-01T01:11:00.130 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: pidfile_write: ignore empty --pid-file 2023-12-01T01:11:00.130 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: load: jerasure load: lrc 2023-12-01T01:11:00.130 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: RocksDB version: 7.9.2 2023-12-01T01:11:00.130 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Git sha 0 2023-12-01T01:11:00.130 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Compile date 2023-11-30 20:12:29 2023-12-01T01:11:00.130 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: DB SUMMARY 2023-12-01T01:11:00.130 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: DB Session ID: TVH27E146KQYL6466KC0 2023-12-01T01:11:00.130 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: CURRENT file: CURRENT 2023-12-01T01:11:00.130 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: IDENTITY file: IDENTITY 2023-12-01T01:11:00.130 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-12-01T01:11:00.130 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2023-12-01T01:11:00.130 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2023-12-01T01:11:00.130 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.error_if_exists: 0 2023-12-01T01:11:00.130 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.create_if_missing: 0 2023-12-01T01:11:00.131 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.paranoid_checks: 1 2023-12-01T01:11:00.131 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.flush_verify_memtable_count: 1 2023-12-01T01:11:00.131 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-12-01T01:11:00.131 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-12-01T01:11:00.131 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.env: 0x55ac953ed720 2023-12-01T01:11:00.131 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.fs: PosixFileSystem 2023-12-01T01:11:00.131 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.info_log: 0x55ac96eb9440 2023-12-01T01:11:00.131 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.max_file_opening_threads: 16 2023-12-01T01:11:00.131 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.statistics: (nil) 2023-12-01T01:11:00.131 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.use_fsync: 0 2023-12-01T01:11:00.131 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.max_log_file_size: 0 2023-12-01T01:11:00.131 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-12-01T01:11:00.131 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.log_file_time_to_roll: 0 2023-12-01T01:11:00.131 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.keep_log_file_num: 1000 2023-12-01T01:11:00.131 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.recycle_log_file_num: 0 2023-12-01T01:11:00.132 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.allow_fallocate: 1 2023-12-01T01:11:00.132 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.allow_mmap_reads: 0 2023-12-01T01:11:00.132 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.allow_mmap_writes: 0 2023-12-01T01:11:00.132 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.use_direct_reads: 0 2023-12-01T01:11:00.132 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-12-01T01:11:00.132 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.create_missing_column_families: 0 2023-12-01T01:11:00.132 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.db_log_dir: 2023-12-01T01:11:00.132 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.wal_dir: 2023-12-01T01:11:00.132 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.table_cache_numshardbits: 6 2023-12-01T01:11:00.132 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.WAL_ttl_seconds: 0 2023-12-01T01:11:00.132 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.WAL_size_limit_MB: 0 2023-12-01T01:11:00.132 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-12-01T01:11:00.132 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-12-01T01:11:00.132 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.is_fd_close_on_exec: 1 2023-12-01T01:11:00.132 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.advise_random_on_open: 1 2023-12-01T01:11:00.133 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.db_write_buffer_size: 0 2023-12-01T01:11:00.133 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.write_buffer_manager: 0x55ac9614a320 2023-12-01T01:11:00.133 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-12-01T01:11:00.133 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-12-01T01:11:00.133 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.use_adaptive_mutex: 0 2023-12-01T01:11:00.133 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.rate_limiter: (nil) 2023-12-01T01:11:00.133 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-12-01T01:11:00.133 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.wal_recovery_mode: 2 2023-12-01T01:11:00.133 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.enable_thread_tracking: 0 2023-12-01T01:11:00.133 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.enable_pipelined_write: 0 2023-12-01T01:11:00.133 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.unordered_write: 0 2023-12-01T01:11:00.133 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-12-01T01:11:00.133 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-12-01T01:11:00.133 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-12-01T01:11:00.134 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-12-01T01:11:00.134 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.row_cache: None 2023-12-01T01:11:00.134 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.wal_filter: None 2023-12-01T01:11:00.134 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-12-01T01:11:00.134 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.allow_ingest_behind: 0 2023-12-01T01:11:00.134 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.two_write_queues: 0 2023-12-01T01:11:00.134 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.manual_wal_flush: 0 2023-12-01T01:11:00.134 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.wal_compression: 0 2023-12-01T01:11:00.134 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.atomic_flush: 0 2023-12-01T01:11:00.134 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-12-01T01:11:00.134 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.persist_stats_to_disk: 0 2023-12-01T01:11:00.134 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.write_dbid_to_manifest: 0 2023-12-01T01:11:00.134 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.log_readahead_size: 0 2023-12-01T01:11:00.134 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-12-01T01:11:00.135 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.best_efforts_recovery: 0 2023-12-01T01:11:00.135 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-12-01T01:11:00.135 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-12-01T01:11:00.135 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.allow_data_in_errors: 0 2023-12-01T01:11:00.135 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.db_host_id: __hostname__ 2023-12-01T01:11:00.135 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.enforce_single_del_contracts: true 2023-12-01T01:11:00.135 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.max_background_jobs: 2 2023-12-01T01:11:00.135 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.max_background_compactions: -1 2023-12-01T01:11:00.135 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.max_subcompactions: 1 2023-12-01T01:11:00.135 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-12-01T01:11:00.135 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-12-01T01:11:00.135 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.delayed_write_rate : 16777216 2023-12-01T01:11:00.135 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.max_total_wal_size: 0 2023-12-01T01:11:00.135 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-12-01T01:11:00.135 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.stats_dump_period_sec: 600 2023-12-01T01:11:00.136 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.stats_persist_period_sec: 600 2023-12-01T01:11:00.136 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-12-01T01:11:00.136 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.max_open_files: -1 2023-12-01T01:11:00.136 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.bytes_per_sync: 0 2023-12-01T01:11:00.136 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.wal_bytes_per_sync: 0 2023-12-01T01:11:00.136 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.strict_bytes_per_sync: 0 2023-12-01T01:11:00.136 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.compaction_readahead_size: 0 2023-12-01T01:11:00.136 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.max_background_flushes: -1 2023-12-01T01:11:00.136 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Compression algorithms supported: 2023-12-01T01:11:00.136 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-12-01T01:11:00.136 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: kZSTD supported: 0 2023-12-01T01:11:00.136 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: kXpressCompression supported: 0 2023-12-01T01:11:00.136 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: kLZ4HCCompression supported: 1 2023-12-01T01:11:00.136 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: kZlibCompression supported: 1 2023-12-01T01:11:00.136 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: kSnappyCompression supported: 1 2023-12-01T01:11:00.137 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: kLZ4Compression supported: 1 2023-12-01T01:11:00.137 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: kBZip2Compression supported: 0 2023-12-01T01:11:00.137 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Fast CRC32 supported: Supported on x86 2023-12-01T01:11:00.137 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: DMutex implementation: pthread_mutex_t 2023-12-01T01:11:00.137 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2023-12-01T01:11:00.137 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-12-01T01:11:00.137 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-12-01T01:11:00.137 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.merge_operator: 2023-12-01T01:11:00.137 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.compaction_filter: None 2023-12-01T01:11:00.137 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.compaction_filter_factory: None 2023-12-01T01:11:00.137 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.sst_partitioner_factory: None 2023-12-01T01:11:00.137 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.memtable_factory: SkipListFactory 2023-12-01T01:11:00.137 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.table_factory: BlockBasedTable 2023-12-01T01:11:00.137 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ac96eb9540) 2023-12-01T01:11:00.138 INFO:journalctl@ceph.mon.c.smithi203.stdout: cache_index_and_filter_blocks: 1 2023-12-01T01:11:00.138 INFO:journalctl@ceph.mon.c.smithi203.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-12-01T01:11:00.138 INFO:journalctl@ceph.mon.c.smithi203.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-12-01T01:11:00.138 INFO:journalctl@ceph.mon.c.smithi203.stdout: pin_top_level_index_and_filter: 1 2023-12-01T01:11:00.138 INFO:journalctl@ceph.mon.c.smithi203.stdout: index_type: 0 2023-12-01T01:11:00.138 INFO:journalctl@ceph.mon.c.smithi203.stdout: data_block_index_type: 0 2023-12-01T01:11:00.138 INFO:journalctl@ceph.mon.c.smithi203.stdout: index_shortening: 1 2023-12-01T01:11:00.138 INFO:journalctl@ceph.mon.c.smithi203.stdout: data_block_hash_table_util_ratio: 0.750000 2023-12-01T01:11:00.138 INFO:journalctl@ceph.mon.c.smithi203.stdout: checksum: 4 2023-12-01T01:11:00.138 INFO:journalctl@ceph.mon.c.smithi203.stdout: no_block_cache: 0 2023-12-01T01:11:00.138 INFO:journalctl@ceph.mon.c.smithi203.stdout: block_cache: 0x55ac961cb1f0 2023-12-01T01:11:00.138 INFO:journalctl@ceph.mon.c.smithi203.stdout: block_cache_name: BinnedLRUCache 2023-12-01T01:11:00.138 INFO:journalctl@ceph.mon.c.smithi203.stdout: block_cache_options: 2023-12-01T01:11:00.138 INFO:journalctl@ceph.mon.c.smithi203.stdout: capacity : 536870912 2023-12-01T01:11:00.138 INFO:journalctl@ceph.mon.c.smithi203.stdout: num_shard_bits : 4 2023-12-01T01:11:00.139 INFO:journalctl@ceph.mon.c.smithi203.stdout: strict_capacity_limit : 0 2023-12-01T01:11:00.139 INFO:journalctl@ceph.mon.c.smithi203.stdout: high_pri_pool_ratio: 0.000 2023-12-01T01:11:00.139 INFO:journalctl@ceph.mon.c.smithi203.stdout: block_cache_compressed: (nil) 2023-12-01T01:11:00.139 INFO:journalctl@ceph.mon.c.smithi203.stdout: persistent_cache: (nil) 2023-12-01T01:11:00.139 INFO:journalctl@ceph.mon.c.smithi203.stdout: block_size: 4096 2023-12-01T01:11:00.139 INFO:journalctl@ceph.mon.c.smithi203.stdout: block_size_deviation: 10 2023-12-01T01:11:00.139 INFO:journalctl@ceph.mon.c.smithi203.stdout: block_restart_interval: 16 2023-12-01T01:11:00.139 INFO:journalctl@ceph.mon.c.smithi203.stdout: index_block_restart_interval: 1 2023-12-01T01:11:00.139 INFO:journalctl@ceph.mon.c.smithi203.stdout: metadata_block_size: 4096 2023-12-01T01:11:00.139 INFO:journalctl@ceph.mon.c.smithi203.stdout: partition_filters: 0 2023-12-01T01:11:00.139 INFO:journalctl@ceph.mon.c.smithi203.stdout: use_delta_encoding: 1 2023-12-01T01:11:00.139 INFO:journalctl@ceph.mon.c.smithi203.stdout: filter_policy: bloomfilter 2023-12-01T01:11:00.139 INFO:journalctl@ceph.mon.c.smithi203.stdout: whole_key_filtering: 1 2023-12-01T01:11:00.139 INFO:journalctl@ceph.mon.c.smithi203.stdout: verify_compression: 0 2023-12-01T01:11:00.139 INFO:journalctl@ceph.mon.c.smithi203.stdout: read_amp_bytes_per_bit: 0 2023-12-01T01:11:00.140 INFO:journalctl@ceph.mon.c.smithi203.stdout: format_version: 5 2023-12-01T01:11:00.140 INFO:journalctl@ceph.mon.c.smithi203.stdout: enable_index_compression: 1 2023-12-01T01:11:00.140 INFO:journalctl@ceph.mon.c.smithi203.stdout: block_align: 0 2023-12-01T01:11:00.140 INFO:journalctl@ceph.mon.c.smithi203.stdout: max_auto_readahead_size: 262144 2023-12-01T01:11:00.140 INFO:journalctl@ceph.mon.c.smithi203.stdout: prepopulate_block_cache: 0 2023-12-01T01:11:00.140 INFO:journalctl@ceph.mon.c.smithi203.stdout: initial_auto_readahead_size: 8192 2023-12-01T01:11:00.140 INFO:journalctl@ceph.mon.c.smithi203.stdout: num_file_reads_for_auto_readahead: 2 2023-12-01T01:11:00.140 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.write_buffer_size: 33554432 2023-12-01T01:11:00.140 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.max_write_buffer_number: 2 2023-12-01T01:11:00.140 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.compression: NoCompression 2023-12-01T01:11:00.140 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.bottommost_compression: Disabled 2023-12-01T01:11:00.140 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.prefix_extractor: nullptr 2023-12-01T01:11:00.140 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-12-01T01:11:00.140 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.num_levels: 7 2023-12-01T01:11:00.140 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-12-01T01:11:00.141 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-12-01T01:11:00.141 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-12-01T01:11:00.141 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-12-01T01:11:00.141 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-12-01T01:11:00.141 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-12-01T01:11:00.142 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-12-01T01:11:00.143 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-12-01T01:11:00.143 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-12-01T01:11:00.143 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-12-01T01:11:00.143 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-12-01T01:11:00.143 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-12-01T01:11:00.143 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.compression_opts.window_bits: -14 2023-12-01T01:11:00.143 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.compression_opts.level: 32767 2023-12-01T01:11:00.143 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.compression_opts.strategy: 0 2023-12-01T01:11:00.143 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-12-01T01:11:00.143 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-12-01T01:11:00.143 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-12-01T01:11:00.143 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-12-01T01:11:00.143 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.compression_opts.enabled: false 2023-12-01T01:11:00.143 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-12-01T01:11:00.144 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-12-01T01:11:00.144 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-12-01T01:11:00.144 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-12-01T01:11:00.144 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.target_file_size_base: 67108864 2023-12-01T01:11:00.144 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.target_file_size_multiplier: 1 2023-12-01T01:11:00.144 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-12-01T01:11:00.144 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-12-01T01:11:00.144 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-12-01T01:11:00.144 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-12-01T01:11:00.144 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-12-01T01:11:00.144 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-12-01T01:11:00.144 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-12-01T01:11:00.144 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-12-01T01:11:00.144 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-12-01T01:11:00.144 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-12-01T01:11:00.144 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-12-01T01:11:00.145 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-12-01T01:11:00.145 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-12-01T01:11:00.145 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.arena_block_size: 1048576 2023-12-01T01:11:00.145 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-12-01T01:11:00.145 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-12-01T01:11:00.145 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.disable_auto_compactions: 0 2023-12-01T01:11:00.145 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-12-01T01:11:00.145 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-12-01T01:11:00.145 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-12-01T01:11:00.145 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-12-01T01:11:00.145 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-12-01T01:11:00.145 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-12-01T01:11:00.145 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-12-01T01:11:00.145 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-12-01T01:11:00.145 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-12-01T01:11:00.146 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-12-01T01:11:00.146 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.table_properties_collectors: 2023-12-01T01:11:00.146 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.inplace_update_support: 0 2023-12-01T01:11:00.146 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.inplace_update_num_locks: 10000 2023-12-01T01:11:00.146 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-12-01T01:11:00.146 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-12-01T01:11:00.146 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.memtable_huge_page_size: 0 2023-12-01T01:11:00.146 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.bloom_locality: 0 2023-12-01T01:11:00.146 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.max_successive_merges: 0 2023-12-01T01:11:00.146 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.optimize_filters_for_hits: 0 2023-12-01T01:11:00.146 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.paranoid_file_checks: 0 2023-12-01T01:11:00.146 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.force_consistency_checks: 1 2023-12-01T01:11:00.146 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.report_bg_io_stats: 0 2023-12-01T01:11:00.146 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.ttl: 2592000 2023-12-01T01:11:00.146 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.periodic_compaction_seconds: 0 2023-12-01T01:11:00.147 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-12-01T01:11:00.147 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-12-01T01:11:00.147 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.enable_blob_files: false 2023-12-01T01:11:00.147 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.min_blob_size: 0 2023-12-01T01:11:00.147 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.blob_file_size: 268435456 2023-12-01T01:11:00.147 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.blob_compression_type: NoCompression 2023-12-01T01:11:00.147 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.enable_blob_garbage_collection: false 2023-12-01T01:11:00.147 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-12-01T01:11:00.147 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-12-01T01:11:00.147 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-12-01T01:11:00.147 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.blob_file_starting_level: 0 2023-12-01T01:11:00.147 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-12-01T01:11:00.147 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-12-01T01:11:00.147 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-12-01T01:11:00.148 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: f5ed3f9c-a538-4b29-852e-d82f4d7aa1f1 2023-12-01T01:11:00.148 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: EVENT_LOG_v1 {"time_micros": 1701393059868388, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-12-01T01:11:00.148 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-12-01T01:11:00.148 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: EVENT_LOG_v1 {"time_micros": 1701393059868807, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1701393059, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "f5ed3f9c-a538-4b29-852e-d82f4d7aa1f1", "db_session_id": "TVH27E146KQYL6466KC0", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-12-01T01:11:00.148 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: EVENT_LOG_v1 {"time_micros": 1701393059868875, "job": 1, "event": "recovery_finished"} 2023-12-01T01:11:00.148 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-12-01T01:11:00.148 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-12-01T01:11:00.148 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ac96268000 2023-12-01T01:11:00.148 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: DB pointer 0x55ac96254000 2023-12-01T01:11:00.148 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-12-01T01:11:00.148 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-12-01T01:11:00.148 INFO:journalctl@ceph.mon.c.smithi203.stdout: ** DB Stats ** 2023-12-01T01:11:00.148 INFO:journalctl@ceph.mon.c.smithi203.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-01T01:11:00.148 INFO:journalctl@ceph.mon.c.smithi203.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-12-01T01:11:00.148 INFO:journalctl@ceph.mon.c.smithi203.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-01T01:11:00.149 INFO:journalctl@ceph.mon.c.smithi203.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-01T01:11:00.149 INFO:journalctl@ceph.mon.c.smithi203.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-12-01T01:11:00.149 INFO:journalctl@ceph.mon.c.smithi203.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-01T01:11:00.149 INFO:journalctl@ceph.mon.c.smithi203.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-01T01:11:00.149 INFO:journalctl@ceph.mon.c.smithi203.stdout: 2023-12-01T01:11:00.149 INFO:journalctl@ceph.mon.c.smithi203.stdout: ** Compaction Stats [default] ** 2023-12-01T01:11:00.149 INFO:journalctl@ceph.mon.c.smithi203.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-01T01:11:00.149 INFO:journalctl@ceph.mon.c.smithi203.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-12-01T01:11:00.149 INFO:journalctl@ceph.mon.c.smithi203.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-12-01T01:11:00.149 INFO:journalctl@ceph.mon.c.smithi203.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-12-01T01:11:00.149 INFO:journalctl@ceph.mon.c.smithi203.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-12-01T01:11:00.149 INFO:journalctl@ceph.mon.c.smithi203.stdout: 2023-12-01T01:11:00.149 INFO:journalctl@ceph.mon.c.smithi203.stdout: ** Compaction Stats [default] ** 2023-12-01T01:11:00.149 INFO:journalctl@ceph.mon.c.smithi203.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-01T01:11:00.149 INFO:journalctl@ceph.mon.c.smithi203.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-12-01T01:11:00.149 INFO:journalctl@ceph.mon.c.smithi203.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-12-01T01:11:00.150 INFO:journalctl@ceph.mon.c.smithi203.stdout: 2023-12-01T01:11:00.150 INFO:journalctl@ceph.mon.c.smithi203.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-12-01T01:11:00.150 INFO:journalctl@ceph.mon.c.smithi203.stdout: 2023-12-01T01:11:00.150 INFO:journalctl@ceph.mon.c.smithi203.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-01T01:11:00.150 INFO:journalctl@ceph.mon.c.smithi203.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-12-01T01:11:00.150 INFO:journalctl@ceph.mon.c.smithi203.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-12-01T01:11:00.150 INFO:journalctl@ceph.mon.c.smithi203.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-12-01T01:11:00.150 INFO:journalctl@ceph.mon.c.smithi203.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-12-01T01:11:00.150 INFO:journalctl@ceph.mon.c.smithi203.stdout: AddFile(Keys): cumulative 0, interval 0 2023-12-01T01:11:00.150 INFO:journalctl@ceph.mon.c.smithi203.stdout: Cumulative compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-01T01:11:00.150 INFO:journalctl@ceph.mon.c.smithi203.stdout: Interval compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-01T01:11:00.150 INFO:journalctl@ceph.mon.c.smithi203.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-12-01T01:11:00.150 INFO:journalctl@ceph.mon.c.smithi203.stdout: Block cache BinnedLRUCache@0x55ac961cb1f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-12-01T01:11:00.150 INFO:journalctl@ceph.mon.c.smithi203.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-12-01T01:11:00.150 INFO:journalctl@ceph.mon.c.smithi203.stdout: 2023-12-01T01:11:00.151 INFO:journalctl@ceph.mon.c.smithi203.stdout: ** File Read Latency Histogram By Level [default] ** 2023-12-01T01:11:00.151 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: mon.c does not exist in monmap, will attempt to join an existing cluster 2023-12-01T01:11:00.151 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: using public_addr v2:172.21.15.203:0/0 -> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] 2023-12-01T01:11:00.151 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: starting mon.c rank -1 at public addrs [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] at bind addrs [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 2023-12-01T01:11:00.151 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: mon.c@-1(???) e0 preinit fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 2023-12-01T01:11:00.151 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: mon.c@-1(synchronizing).mds e1 new map 2023-12-01T01:11:00.151 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: mon.c@-1(synchronizing).mds e1 print_map 2023-12-01T01:11:00.151 INFO:journalctl@ceph.mon.c.smithi203.stdout: e1 2023-12-01T01:11:00.151 INFO:journalctl@ceph.mon.c.smithi203.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-12-01T01:11:00.151 INFO:journalctl@ceph.mon.c.smithi203.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-12-01T01:11:00.151 INFO:journalctl@ceph.mon.c.smithi203.stdout: legacy client fscid: -1 2023-12-01T01:11:00.151 INFO:journalctl@ceph.mon.c.smithi203.stdout: 2023-12-01T01:11:00.151 INFO:journalctl@ceph.mon.c.smithi203.stdout: No filesystems configured 2023-12-01T01:11:00.151 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-12-01T01:11:00.151 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-12-01T01:11:00.152 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-12-01T01:11:00.152 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-12-01T01:11:00.152 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-12-01T01:11:00.152 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-12-01T01:11:00.152 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-12-01T01:11:00.152 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-01T01:11:00.152 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-01T01:11:00.152 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-01T01:11:00.152 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:00.152 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: Updating smithi137:/var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-12-01T01:11:00.152 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:00.152 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:11:00.152 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:00.152 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: Added host smithi203 2023-12-01T01:11:00.152 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:00.153 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:00.153 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:00.153 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-01T01:11:00.153 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:00.153 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/3192192902' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-12-01T01:11:00.153 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/3192192902' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-12-01T01:11:00.153 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: osdmap e4: 0 total, 0 up, 0 in 2023-12-01T01:11:00.153 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:00.153 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:00.153 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:00.153 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:00.153 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi203", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:11:00.153 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:00.153 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:11:00.153 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: Updating smithi203:/etc/ceph/ceph.conf 2023-12-01T01:11:00.154 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:00.154 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: Updating smithi203:/var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-01T01:11:00.154 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:00.154 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:00.154 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:00.154 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2023-12-01T01:11:00.154 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: Updating smithi203:/var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-12-01T01:11:00.154 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:00.154 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:11:00.154 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:00.154 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:11:00.154 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:00.154 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-01T01:11:00.154 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:00.154 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi077:172.21.15.77=a;smithi137:172.21.15.137=b;smithi203:172.21.15.203=c", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:11:00.155 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: Saving service mon spec with placement smithi077:172.21.15.77=a;smithi137:172.21.15.137=b;smithi203:172.21.15.203=c;count:3 2023-12-01T01:11:00.155 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: Deploying daemon mon.c on smithi203 2023-12-01T01:11:00.155 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:00.155 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:00.155 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-12-01T01:11:00.155 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: expand_channel_meta expand map: {default=false} 2023-12-01T01:11:00.155 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: expand_channel_meta from 'false' to 'false' 2023-12-01T01:11:00.155 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: expand_channel_meta expanded map: {default=false} 2023-12-01T01:11:00.155 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: expand_channel_meta expand map: {default=info} 2023-12-01T01:11:00.155 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: expand_channel_meta from 'info' to 'info' 2023-12-01T01:11:00.155 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: expand_channel_meta expanded map: {default=info} 2023-12-01T01:11:00.155 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: expand_channel_meta expand map: {default=daemon} 2023-12-01T01:11:00.155 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: expand_channel_meta from 'daemon' to 'daemon' 2023-12-01T01:11:00.155 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: expand_channel_meta expanded map: {default=daemon} 2023-12-01T01:11:00.155 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: expand_channel_meta expand map: {default=debug} 2023-12-01T01:11:00.156 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: expand_channel_meta from 'debug' to 'debug' 2023-12-01T01:11:00.156 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: expand_channel_meta expanded map: {default=debug} 2023-12-01T01:11:00.156 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 ceph-mon[103581]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-12-01T01:11:00.156 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:10:59 smithi203 systemd[1]: Started Ceph mon.c for 2a06e29a-8fe6-11ee-95a2-87774f69a715. 2023-12-01T01:11:01.142 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-12-01T01:11:01.143 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-01T01:11:01.704 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.c/config 2023-12-01T01:11:05.262 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:05 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-01T01:11:05.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:05 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-01T01:11:05.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:05 smithi203 ceph-mon[103581]: mon.a calling monitor election 2023-12-01T01:11:05.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:05 smithi203 ceph-mon[103581]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:05.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:05 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-01T01:11:05.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:05 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-01T01:11:05.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:05 smithi203 ceph-mon[103581]: mon.c calling monitor election 2023-12-01T01:11:05.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:05 smithi203 ceph-mon[103581]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:05.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:05 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-01T01:11:05.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:05 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-01T01:11:05.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:05 smithi203 ceph-mon[103581]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:05.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:05 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-01T01:11:05.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:05 smithi203 ceph-mon[103581]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-12-01T01:11:05.264 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:05 smithi203 ceph-mon[103581]: monmap e2: 2 mons at {a=[v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0],c=[v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-01T01:11:05.264 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:05 smithi203 ceph-mon[103581]: fsmap 2023-12-01T01:11:05.264 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:05 smithi203 ceph-mon[103581]: osdmap e4: 0 total, 0 up, 0 in 2023-12-01T01:11:05.264 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:05 smithi203 ceph-mon[103581]: mgrmap e13: x(active, since 40s) 2023-12-01T01:11:05.264 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:05 smithi203 ceph-mon[103581]: overall HEALTH_OK 2023-12-01T01:11:05.264 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:05 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:05.264 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:05 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:05.264 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:05 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:05.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:05 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-01T01:11:05.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:05 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-01T01:11:05.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:05 smithi077 ceph-mon[98867]: mon.a calling monitor election 2023-12-01T01:11:05.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:05 smithi077 ceph-mon[98867]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:05.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:05 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-01T01:11:05.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:05 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-01T01:11:05.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:05 smithi077 ceph-mon[98867]: mon.c calling monitor election 2023-12-01T01:11:05.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:05 smithi077 ceph-mon[98867]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:05.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:05 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-01T01:11:05.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:05 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-01T01:11:05.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:05 smithi077 ceph-mon[98867]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:05.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:05 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-01T01:11:05.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:05 smithi077 ceph-mon[98867]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-12-01T01:11:05.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:05 smithi077 ceph-mon[98867]: monmap e2: 2 mons at {a=[v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0],c=[v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-01T01:11:05.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:05 smithi077 ceph-mon[98867]: fsmap 2023-12-01T01:11:05.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:05 smithi077 ceph-mon[98867]: osdmap e4: 0 total, 0 up, 0 in 2023-12-01T01:11:05.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:05 smithi077 ceph-mon[98867]: mgrmap e13: x(active, since 40s) 2023-12-01T01:11:05.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:05 smithi077 ceph-mon[98867]: overall HEALTH_OK 2023-12-01T01:11:05.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:05 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:05.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:05 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:05.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:05 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:05.950 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:05.948+0000 7f3edede3700 1 -- 172.21.15.203:0/2749700129 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ec80035c0 msgr2=0x7f3ec8005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:11:05.950 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:05.948+0000 7f3edede3700 1 --2- 172.21.15.203:0/2749700129 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ec80035c0 0x7f3ec8005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8076780 tx=0x7f3ecc00feb0 comp rx=0 tx=0).stop 2023-12-01T01:11:05.950 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:05.948+0000 7f3edede3700 1 -- 172.21.15.203:0/2749700129 shutdown_connections 2023-12-01T01:11:05.950 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:05.948+0000 7f3edede3700 1 --2- 172.21.15.203:0/2749700129 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ec80035c0 0x7f3ec8005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:11:05.950 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:05.948+0000 7f3edede3700 1 -- 172.21.15.203:0/2749700129 >> 172.21.15.203:0/2749700129 conn(0x7f3ed8075340 msgr2=0x7f3ed8075750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:11:05.950 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:05.948+0000 7f3edede3700 1 -- 172.21.15.203:0/2749700129 shutdown_connections 2023-12-01T01:11:05.950 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:05.948+0000 7f3edede3700 1 -- 172.21.15.203:0/2749700129 wait complete. 2023-12-01T01:11:05.952 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:05.949+0000 7f3edede3700 1 Processor -- start 2023-12-01T01:11:05.952 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:05.949+0000 7f3edede3700 1 -- start start 2023-12-01T01:11:05.952 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:05.949+0000 7f3edede3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ec80035c0 0x7f3ed8196cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:11:05.952 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:05.949+0000 7f3edede3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ed8197210 0x7f3ed8197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:11:05.952 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:05.949+0000 7f3edede3700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed819fd40 con 0x7f3ed8197210 2023-12-01T01:11:05.952 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:05.949+0000 7f3edede3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed819fe80 con 0x7f3ec80035c0 2023-12-01T01:11:05.952 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:05.949+0000 7f3edcb7f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ec80035c0 0x7f3ed8196cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:11:05.953 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:05.949+0000 7f3edcb7f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ec80035c0 0x7f3ed8196cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:59132/0 (socket says 172.21.15.203:59132) 2023-12-01T01:11:05.953 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:05.949+0000 7f3edcb7f700 1 -- 172.21.15.203:0/3438359456 learned_addr learned my addr 172.21.15.203:0/3438359456 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-12-01T01:11:05.953 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:05.949+0000 7f3edcb7f700 1 -- 172.21.15.203:0/3438359456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ec80035c0 msgr2=0x7f3ed8196cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-12-01T01:11:05.953 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:05.949+0000 7f3edcb7f700 1 -- 172.21.15.203:0/3438359456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ec80035c0 msgr2=0x7f3ed8196cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-12-01T01:11:05.953 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:05.949+0000 7f3edcb7f700 1 --2- 172.21.15.203:0/3438359456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ec80035c0 0x7f3ed8196cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-01T01:11:05.953 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:05.949+0000 7f3edcb7f700 1 --2- 172.21.15.203:0/3438359456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ec80035c0 0x7f3ed8196cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:11:05.953 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:05.949+0000 7f3ed7fff700 1 --2- 172.21.15.203:0/3438359456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ed8197210 0x7f3ed8197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:11:05.953 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:05.950+0000 7f3ed7fff700 1 -- 172.21.15.203:0/3438359456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ec80035c0 msgr2=0x7f3ed8196cd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:11:05.953 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:05.950+0000 7f3ed7fff700 1 --2- 172.21.15.203:0/3438359456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ec80035c0 0x7f3ed8196cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:11:05.953 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:05.950+0000 7f3ed7fff700 1 -- 172.21.15.203:0/3438359456 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ecc00f900 con 0x7f3ed8197210 2023-12-01T01:11:05.953 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:05.950+0000 7f3ed7fff700 1 --2- 172.21.15.203:0/3438359456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ed8197210 0x7f3ed8197690 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3ec400b4d0 tx=0x7f3ec400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:11:05.953 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:05.950+0000 7f3ed5ffb700 1 -- 172.21.15.203:0/3438359456 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f3ec4011630 con 0x7f3ed8197210 2023-12-01T01:11:05.953 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:05.950+0000 7f3ed5ffb700 1 -- 172.21.15.203:0/3438359456 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ec4011c50 con 0x7f3ed8197210 2023-12-01T01:11:05.954 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:05.950+0000 7f3edede3700 1 -- 172.21.15.203:0/3438359456 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ed81a0080 con 0x7f3ed8197210 2023-12-01T01:11:05.954 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:05.950+0000 7f3ed5ffb700 1 -- 172.21.15.203:0/3438359456 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f3ec4010d90 con 0x7f3ed8197210 2023-12-01T01:11:05.954 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:05.951+0000 7f3edede3700 1 -- 172.21.15.203:0/3438359456 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ed81a05a0 con 0x7f3ed8197210 2023-12-01T01:11:05.955 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:05.951+0000 7f3ed5ffb700 1 -- 172.21.15.203:0/3438359456 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) v1 ==== 45324+0+0 (secure 0 0 0) 0x7f3ec401a400 con 0x7f3ed8197210 2023-12-01T01:11:05.955 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:05.952+0000 7f3ed5ffb700 1 --2- 172.21.15.203:0/3438359456 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f3ebc0381d0 0x7f3ebc03a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:11:05.955 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:05.952+0000 7f3ed5ffb700 1 -- 172.21.15.203:0/3438359456 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3ec404a840 con 0x7f3ed8197210 2023-12-01T01:11:05.955 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:05.952+0000 7f3edcb7f700 1 --2- 172.21.15.203:0/3438359456 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f3ebc0381d0 0x7f3ebc03a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:11:05.955 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:05.952+0000 7f3edede3700 1 -- 172.21.15.203:0/3438359456 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ed80623c0 con 0x7f3ed8197210 2023-12-01T01:11:05.955 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:05.952+0000 7f3edcb7f700 1 --2- 172.21.15.203:0/3438359456 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f3ebc0381d0 0x7f3ebc03a690 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc011510 tx=0x7f3ecc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:11:05.959 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:05.958+0000 7f3ed5ffb700 1 -- 172.21.15.203:0/3438359456 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3ec40178b0 con 0x7f3ed8197210 2023-12-01T01:11:06.226 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:06.224+0000 7f3edede3700 1 -- 172.21.15.203:0/3438359456 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3ed81a0840 con 0x7f3ed8197210 2023-12-01T01:11:06.226 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:06.224+0000 7f3ed5ffb700 1 -- 172.21.15.203:0/3438359456 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1006 (secure 0 0 0) 0x7f3ec403d020 con 0x7f3ed8197210 2023-12-01T01:11:06.227 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-12-01T01:11:06.227 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":2,"fsid":"2a06e29a-8fe6-11ee-95a2-87774f69a715","modified":"2023-12-01T01:10:59.900664Z","created":"2023-12-01T01:09:28.081844Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:3300","nonce":0},{"type":"v1","addr":"172.21.15.77:6789","nonce":0}]},"addr":"172.21.15.77:6789/0","public_addr":"172.21.15.77:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:3300","nonce":0},{"type":"v1","addr":"172.21.15.203:6789","nonce":0}]},"addr":"172.21.15.203:6789/0","public_addr":"172.21.15.203:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-12-01T01:11:06.229 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:06.226+0000 7f3edede3700 1 -- 172.21.15.203:0/3438359456 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f3ebc0381d0 msgr2=0x7f3ebc03a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:11:06.229 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:06.226+0000 7f3edede3700 1 --2- 172.21.15.203:0/3438359456 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f3ebc0381d0 0x7f3ebc03a690 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc011510 tx=0x7f3ecc005960 comp rx=0 tx=0).stop 2023-12-01T01:11:06.229 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:06.226+0000 7f3edede3700 1 -- 172.21.15.203:0/3438359456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ed8197210 msgr2=0x7f3ed8197690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:11:06.229 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:06.226+0000 7f3edede3700 1 --2- 172.21.15.203:0/3438359456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ed8197210 0x7f3ed8197690 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3ec400b4d0 tx=0x7f3ec400b890 comp rx=0 tx=0).stop 2023-12-01T01:11:06.230 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:06.226+0000 7f3edede3700 1 -- 172.21.15.203:0/3438359456 shutdown_connections 2023-12-01T01:11:06.230 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:06.226+0000 7f3edede3700 1 --2- 172.21.15.203:0/3438359456 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f3ebc0381d0 0x7f3ebc03a690 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:11:06.230 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:06.226+0000 7f3edede3700 1 --2- 172.21.15.203:0/3438359456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ec80035c0 0x7f3ed8196cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:11:06.230 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:06.226+0000 7f3edede3700 1 --2- 172.21.15.203:0/3438359456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ed8197210 0x7f3ed8197690 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:11:06.230 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:06.227+0000 7f3edede3700 1 -- 172.21.15.203:0/3438359456 >> 172.21.15.203:0/3438359456 conn(0x7f3ed8075340 msgr2=0x7f3ed818dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:11:06.230 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:06.227+0000 7f3edede3700 1 -- 172.21.15.203:0/3438359456 shutdown_connections 2023-12-01T01:11:06.230 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:06.227+0000 7f3edede3700 1 -- 172.21.15.203:0/3438359456 wait complete. 2023-12-01T01:11:06.230 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 2 2023-12-01T01:11:06.262 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:06 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-01T01:11:06.262 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:06 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:06.262 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:06 smithi203 ceph-mon[103581]: Deploying daemon mon.b on smithi137 2023-12-01T01:11:06.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:06 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-01T01:11:06.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:06 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-01T01:11:06.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:06 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:06.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:06 smithi077 ceph-mon[98867]: Deploying daemon mon.b on smithi137 2023-12-01T01:11:06.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:06 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-01T01:11:07.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:07 smithi077 ceph-mon[98867]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:07.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:07 smithi077 ceph-mon[98867]: from='client.? 172.21.15.203:0/3438359456' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-01T01:11:07.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:07 smithi203 ceph-mon[103581]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:07.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:07 smithi203 ceph-mon[103581]: from='client.? 172.21.15.203:0/3438359456' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-01T01:11:07.665 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-12-01T01:11:07.665 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-12-01T01:11:08.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:07 smithi137 systemd[1]: Starting Ceph mon.b for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:11:08.230 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.c/config 2023-12-01T01:11:08.334 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 podman[103272]: 2023-12-01T01:11:08.335 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: set uid:gid to 167:167 (ceph:ceph) 2023-12-01T01:11:08.335 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: ceph version 18.2.0-1249-ga6f82997 (a6f82997ac57b70a895455dfed6256360a1e4c32) reef (stable), process ceph-mon, pid 7 2023-12-01T01:11:08.335 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: pidfile_write: ignore empty --pid-file 2023-12-01T01:11:08.335 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: load: jerasure load: lrc 2023-12-01T01:11:08.335 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: RocksDB version: 7.9.2 2023-12-01T01:11:08.335 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Git sha 0 2023-12-01T01:11:08.335 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Compile date 2023-11-30 20:12:29 2023-12-01T01:11:08.335 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: DB SUMMARY 2023-12-01T01:11:08.335 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: DB Session ID: WBOWP37ALSVJT64RFY27 2023-12-01T01:11:08.335 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: CURRENT file: CURRENT 2023-12-01T01:11:08.335 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: IDENTITY file: IDENTITY 2023-12-01T01:11:08.335 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-12-01T01:11:08.335 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-12-01T01:11:08.335 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-12-01T01:11:08.335 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.error_if_exists: 0 2023-12-01T01:11:08.336 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.create_if_missing: 0 2023-12-01T01:11:08.336 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.paranoid_checks: 1 2023-12-01T01:11:08.336 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.flush_verify_memtable_count: 1 2023-12-01T01:11:08.336 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-12-01T01:11:08.336 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-12-01T01:11:08.336 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.env: 0x564546187720 2023-12-01T01:11:08.336 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.fs: PosixFileSystem 2023-12-01T01:11:08.336 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.info_log: 0x5645490a3440 2023-12-01T01:11:08.336 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.max_file_opening_threads: 16 2023-12-01T01:11:08.336 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.statistics: (nil) 2023-12-01T01:11:08.336 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.use_fsync: 0 2023-12-01T01:11:08.336 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.max_log_file_size: 0 2023-12-01T01:11:08.336 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-12-01T01:11:08.336 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.log_file_time_to_roll: 0 2023-12-01T01:11:08.336 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.keep_log_file_num: 1000 2023-12-01T01:11:08.337 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.recycle_log_file_num: 0 2023-12-01T01:11:08.337 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.allow_fallocate: 1 2023-12-01T01:11:08.337 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.allow_mmap_reads: 0 2023-12-01T01:11:08.337 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.allow_mmap_writes: 0 2023-12-01T01:11:08.337 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.use_direct_reads: 0 2023-12-01T01:11:08.337 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-12-01T01:11:08.337 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.create_missing_column_families: 0 2023-12-01T01:11:08.337 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.db_log_dir: 2023-12-01T01:11:08.337 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.wal_dir: 2023-12-01T01:11:08.337 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.table_cache_numshardbits: 6 2023-12-01T01:11:08.337 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.WAL_ttl_seconds: 0 2023-12-01T01:11:08.337 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.WAL_size_limit_MB: 0 2023-12-01T01:11:08.337 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-12-01T01:11:08.337 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-12-01T01:11:08.337 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.is_fd_close_on_exec: 1 2023-12-01T01:11:08.337 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.advise_random_on_open: 1 2023-12-01T01:11:08.338 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.db_write_buffer_size: 0 2023-12-01T01:11:08.338 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.write_buffer_manager: 0x564548334320 2023-12-01T01:11:08.338 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-12-01T01:11:08.338 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-12-01T01:11:08.338 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.use_adaptive_mutex: 0 2023-12-01T01:11:08.338 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.rate_limiter: (nil) 2023-12-01T01:11:08.338 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-12-01T01:11:08.338 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.wal_recovery_mode: 2 2023-12-01T01:11:08.338 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.enable_thread_tracking: 0 2023-12-01T01:11:08.338 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.enable_pipelined_write: 0 2023-12-01T01:11:08.338 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.unordered_write: 0 2023-12-01T01:11:08.338 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-12-01T01:11:08.338 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-12-01T01:11:08.338 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-12-01T01:11:08.339 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-12-01T01:11:08.339 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.row_cache: None 2023-12-01T01:11:08.339 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.wal_filter: None 2023-12-01T01:11:08.339 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-12-01T01:11:08.339 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.allow_ingest_behind: 0 2023-12-01T01:11:08.339 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.two_write_queues: 0 2023-12-01T01:11:08.339 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.manual_wal_flush: 0 2023-12-01T01:11:08.339 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.wal_compression: 0 2023-12-01T01:11:08.339 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.atomic_flush: 0 2023-12-01T01:11:08.339 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-12-01T01:11:08.339 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.persist_stats_to_disk: 0 2023-12-01T01:11:08.339 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.write_dbid_to_manifest: 0 2023-12-01T01:11:08.339 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.log_readahead_size: 0 2023-12-01T01:11:08.339 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-12-01T01:11:08.339 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.best_efforts_recovery: 0 2023-12-01T01:11:08.339 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-12-01T01:11:08.340 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-12-01T01:11:08.340 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.allow_data_in_errors: 0 2023-12-01T01:11:08.340 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.db_host_id: __hostname__ 2023-12-01T01:11:08.340 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.enforce_single_del_contracts: true 2023-12-01T01:11:08.340 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.max_background_jobs: 2 2023-12-01T01:11:08.340 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.max_background_compactions: -1 2023-12-01T01:11:08.340 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.max_subcompactions: 1 2023-12-01T01:11:08.340 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-12-01T01:11:08.340 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-12-01T01:11:08.340 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.delayed_write_rate : 16777216 2023-12-01T01:11:08.340 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.max_total_wal_size: 0 2023-12-01T01:11:08.340 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-12-01T01:11:08.340 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.stats_dump_period_sec: 600 2023-12-01T01:11:08.340 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.stats_persist_period_sec: 600 2023-12-01T01:11:08.340 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-12-01T01:11:08.341 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.max_open_files: -1 2023-12-01T01:11:08.341 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.bytes_per_sync: 0 2023-12-01T01:11:08.341 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.wal_bytes_per_sync: 0 2023-12-01T01:11:08.341 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.strict_bytes_per_sync: 0 2023-12-01T01:11:08.341 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.compaction_readahead_size: 0 2023-12-01T01:11:08.341 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.max_background_flushes: -1 2023-12-01T01:11:08.341 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Compression algorithms supported: 2023-12-01T01:11:08.341 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-12-01T01:11:08.341 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: kZSTD supported: 0 2023-12-01T01:11:08.341 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: kXpressCompression supported: 0 2023-12-01T01:11:08.341 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: kLZ4HCCompression supported: 1 2023-12-01T01:11:08.341 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: kZlibCompression supported: 1 2023-12-01T01:11:08.341 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: kSnappyCompression supported: 1 2023-12-01T01:11:08.341 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: kLZ4Compression supported: 1 2023-12-01T01:11:08.341 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: kBZip2Compression supported: 0 2023-12-01T01:11:08.341 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Fast CRC32 supported: Supported on x86 2023-12-01T01:11:08.342 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: DMutex implementation: pthread_mutex_t 2023-12-01T01:11:08.342 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-12-01T01:11:08.342 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-12-01T01:11:08.342 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-12-01T01:11:08.342 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.merge_operator: 2023-12-01T01:11:08.342 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.compaction_filter: None 2023-12-01T01:11:08.342 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.compaction_filter_factory: None 2023-12-01T01:11:08.342 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.sst_partitioner_factory: None 2023-12-01T01:11:08.342 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.memtable_factory: SkipListFactory 2023-12-01T01:11:08.342 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.table_factory: BlockBasedTable 2023-12-01T01:11:08.342 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5645490a3540) 2023-12-01T01:11:08.342 INFO:journalctl@ceph.mon.b.smithi137.stdout: cache_index_and_filter_blocks: 1 2023-12-01T01:11:08.342 INFO:journalctl@ceph.mon.b.smithi137.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-12-01T01:11:08.342 INFO:journalctl@ceph.mon.b.smithi137.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-12-01T01:11:08.343 INFO:journalctl@ceph.mon.b.smithi137.stdout: pin_top_level_index_and_filter: 1 2023-12-01T01:11:08.343 INFO:journalctl@ceph.mon.b.smithi137.stdout: index_type: 0 2023-12-01T01:11:08.343 INFO:journalctl@ceph.mon.b.smithi137.stdout: data_block_index_type: 0 2023-12-01T01:11:08.343 INFO:journalctl@ceph.mon.b.smithi137.stdout: index_shortening: 1 2023-12-01T01:11:08.343 INFO:journalctl@ceph.mon.b.smithi137.stdout: data_block_hash_table_util_ratio: 0.750000 2023-12-01T01:11:08.343 INFO:journalctl@ceph.mon.b.smithi137.stdout: checksum: 4 2023-12-01T01:11:08.343 INFO:journalctl@ceph.mon.b.smithi137.stdout: no_block_cache: 0 2023-12-01T01:11:08.343 INFO:journalctl@ceph.mon.b.smithi137.stdout: block_cache: 0x5645483b5350 2023-12-01T01:11:08.343 INFO:journalctl@ceph.mon.b.smithi137.stdout: block_cache_name: BinnedLRUCache 2023-12-01T01:11:08.343 INFO:journalctl@ceph.mon.b.smithi137.stdout: block_cache_options: 2023-12-01T01:11:08.343 INFO:journalctl@ceph.mon.b.smithi137.stdout: capacity : 536870912 2023-12-01T01:11:08.343 INFO:journalctl@ceph.mon.b.smithi137.stdout: num_shard_bits : 4 2023-12-01T01:11:08.343 INFO:journalctl@ceph.mon.b.smithi137.stdout: strict_capacity_limit : 0 2023-12-01T01:11:08.343 INFO:journalctl@ceph.mon.b.smithi137.stdout: high_pri_pool_ratio: 0.000 2023-12-01T01:11:08.343 INFO:journalctl@ceph.mon.b.smithi137.stdout: block_cache_compressed: (nil) 2023-12-01T01:11:08.344 INFO:journalctl@ceph.mon.b.smithi137.stdout: persistent_cache: (nil) 2023-12-01T01:11:08.344 INFO:journalctl@ceph.mon.b.smithi137.stdout: block_size: 4096 2023-12-01T01:11:08.344 INFO:journalctl@ceph.mon.b.smithi137.stdout: block_size_deviation: 10 2023-12-01T01:11:08.344 INFO:journalctl@ceph.mon.b.smithi137.stdout: block_restart_interval: 16 2023-12-01T01:11:08.344 INFO:journalctl@ceph.mon.b.smithi137.stdout: index_block_restart_interval: 1 2023-12-01T01:11:08.344 INFO:journalctl@ceph.mon.b.smithi137.stdout: metadata_block_size: 4096 2023-12-01T01:11:08.344 INFO:journalctl@ceph.mon.b.smithi137.stdout: partition_filters: 0 2023-12-01T01:11:08.344 INFO:journalctl@ceph.mon.b.smithi137.stdout: use_delta_encoding: 1 2023-12-01T01:11:08.344 INFO:journalctl@ceph.mon.b.smithi137.stdout: filter_policy: bloomfilter 2023-12-01T01:11:08.344 INFO:journalctl@ceph.mon.b.smithi137.stdout: whole_key_filtering: 1 2023-12-01T01:11:08.344 INFO:journalctl@ceph.mon.b.smithi137.stdout: verify_compression: 0 2023-12-01T01:11:08.344 INFO:journalctl@ceph.mon.b.smithi137.stdout: read_amp_bytes_per_bit: 0 2023-12-01T01:11:08.344 INFO:journalctl@ceph.mon.b.smithi137.stdout: format_version: 5 2023-12-01T01:11:08.344 INFO:journalctl@ceph.mon.b.smithi137.stdout: enable_index_compression: 1 2023-12-01T01:11:08.344 INFO:journalctl@ceph.mon.b.smithi137.stdout: block_align: 0 2023-12-01T01:11:08.344 INFO:journalctl@ceph.mon.b.smithi137.stdout: max_auto_readahead_size: 262144 2023-12-01T01:11:08.345 INFO:journalctl@ceph.mon.b.smithi137.stdout: prepopulate_block_cache: 0 2023-12-01T01:11:08.345 INFO:journalctl@ceph.mon.b.smithi137.stdout: initial_auto_readahead_size: 8192 2023-12-01T01:11:08.345 INFO:journalctl@ceph.mon.b.smithi137.stdout: num_file_reads_for_auto_readahead: 2 2023-12-01T01:11:08.345 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.write_buffer_size: 33554432 2023-12-01T01:11:08.345 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.max_write_buffer_number: 2 2023-12-01T01:11:08.345 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.compression: NoCompression 2023-12-01T01:11:08.345 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.bottommost_compression: Disabled 2023-12-01T01:11:08.345 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.prefix_extractor: nullptr 2023-12-01T01:11:08.345 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-12-01T01:11:08.345 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.num_levels: 7 2023-12-01T01:11:08.345 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-12-01T01:11:08.345 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-12-01T01:11:08.345 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-12-01T01:11:08.345 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-12-01T01:11:08.345 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-12-01T01:11:08.346 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-12-01T01:11:08.346 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-12-01T01:11:08.347 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-12-01T01:11:08.347 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-12-01T01:11:08.347 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-12-01T01:11:08.347 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-12-01T01:11:08.347 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-12-01T01:11:08.347 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.compression_opts.window_bits: -14 2023-12-01T01:11:08.347 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.compression_opts.level: 32767 2023-12-01T01:11:08.347 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.compression_opts.strategy: 0 2023-12-01T01:11:08.347 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-12-01T01:11:08.347 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-12-01T01:11:08.347 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-12-01T01:11:08.348 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-12-01T01:11:08.348 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.compression_opts.enabled: false 2023-12-01T01:11:08.348 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-12-01T01:11:08.348 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-12-01T01:11:08.348 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-12-01T01:11:08.348 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-12-01T01:11:08.348 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.target_file_size_base: 67108864 2023-12-01T01:11:08.348 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.target_file_size_multiplier: 1 2023-12-01T01:11:08.348 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-12-01T01:11:08.348 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-12-01T01:11:08.348 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-12-01T01:11:08.348 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-12-01T01:11:08.348 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-12-01T01:11:08.349 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-12-01T01:11:08.349 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-12-01T01:11:08.349 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-12-01T01:11:08.349 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-12-01T01:11:08.349 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-12-01T01:11:08.349 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-12-01T01:11:08.349 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-12-01T01:11:08.349 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-12-01T01:11:08.349 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.arena_block_size: 1048576 2023-12-01T01:11:08.349 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-12-01T01:11:08.349 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-12-01T01:11:08.349 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.disable_auto_compactions: 0 2023-12-01T01:11:08.349 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-12-01T01:11:08.349 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-12-01T01:11:08.349 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-12-01T01:11:08.350 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-12-01T01:11:08.350 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-12-01T01:11:08.350 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-12-01T01:11:08.350 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-12-01T01:11:08.350 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-12-01T01:11:08.350 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-12-01T01:11:08.350 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-12-01T01:11:08.350 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.table_properties_collectors: 2023-12-01T01:11:08.350 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.inplace_update_support: 0 2023-12-01T01:11:08.350 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.inplace_update_num_locks: 10000 2023-12-01T01:11:08.350 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-12-01T01:11:08.350 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-12-01T01:11:08.350 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.memtable_huge_page_size: 0 2023-12-01T01:11:08.350 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.bloom_locality: 0 2023-12-01T01:11:08.350 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.max_successive_merges: 0 2023-12-01T01:11:08.351 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.optimize_filters_for_hits: 0 2023-12-01T01:11:08.351 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.paranoid_file_checks: 0 2023-12-01T01:11:08.351 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.force_consistency_checks: 1 2023-12-01T01:11:08.351 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.report_bg_io_stats: 0 2023-12-01T01:11:08.351 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.ttl: 2592000 2023-12-01T01:11:08.351 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.periodic_compaction_seconds: 0 2023-12-01T01:11:08.351 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-12-01T01:11:08.351 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-12-01T01:11:08.351 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.enable_blob_files: false 2023-12-01T01:11:08.351 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.min_blob_size: 0 2023-12-01T01:11:08.351 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.blob_file_size: 268435456 2023-12-01T01:11:08.351 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.blob_compression_type: NoCompression 2023-12-01T01:11:08.351 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.enable_blob_garbage_collection: false 2023-12-01T01:11:08.351 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-12-01T01:11:08.351 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-12-01T01:11:08.352 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-12-01T01:11:08.352 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.blob_file_starting_level: 0 2023-12-01T01:11:08.352 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-12-01T01:11:08.352 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-12-01T01:11:08.352 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-12-01T01:11:08.352 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 78122e57-7147-4eab-9168-cd525bcd0bd3 2023-12-01T01:11:08.352 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: EVENT_LOG_v1 {"time_micros": 1701393068266985, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-12-01T01:11:08.352 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-12-01T01:11:08.352 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: EVENT_LOG_v1 {"time_micros": 1701393068267544, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1701393068, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "78122e57-7147-4eab-9168-cd525bcd0bd3", "db_session_id": "WBOWP37ALSVJT64RFY27", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-12-01T01:11:08.352 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: EVENT_LOG_v1 {"time_micros": 1701393068267635, "job": 1, "event": "recovery_finished"} 2023-12-01T01:11:08.352 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-12-01T01:11:08.352 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-12-01T01:11:08.352 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x564548452000 2023-12-01T01:11:08.352 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: DB pointer 0x56454843e000 2023-12-01T01:11:08.352 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-12-01T01:11:08.353 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-12-01T01:11:08.353 INFO:journalctl@ceph.mon.b.smithi137.stdout: ** DB Stats ** 2023-12-01T01:11:08.353 INFO:journalctl@ceph.mon.b.smithi137.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-01T01:11:08.353 INFO:journalctl@ceph.mon.b.smithi137.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-12-01T01:11:08.353 INFO:journalctl@ceph.mon.b.smithi137.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-01T01:11:08.353 INFO:journalctl@ceph.mon.b.smithi137.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-01T01:11:08.353 INFO:journalctl@ceph.mon.b.smithi137.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-12-01T01:11:08.353 INFO:journalctl@ceph.mon.b.smithi137.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-01T01:11:08.353 INFO:journalctl@ceph.mon.b.smithi137.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-01T01:11:08.353 INFO:journalctl@ceph.mon.b.smithi137.stdout: 2023-12-01T01:11:08.353 INFO:journalctl@ceph.mon.b.smithi137.stdout: ** Compaction Stats [default] ** 2023-12-01T01:11:08.353 INFO:journalctl@ceph.mon.b.smithi137.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-01T01:11:08.353 INFO:journalctl@ceph.mon.b.smithi137.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-12-01T01:11:08.353 INFO:journalctl@ceph.mon.b.smithi137.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-01T01:11:08.353 INFO:journalctl@ceph.mon.b.smithi137.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-01T01:11:08.353 INFO:journalctl@ceph.mon.b.smithi137.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-01T01:11:08.354 INFO:journalctl@ceph.mon.b.smithi137.stdout: 2023-12-01T01:11:08.354 INFO:journalctl@ceph.mon.b.smithi137.stdout: ** Compaction Stats [default] ** 2023-12-01T01:11:08.354 INFO:journalctl@ceph.mon.b.smithi137.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-01T01:11:08.354 INFO:journalctl@ceph.mon.b.smithi137.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-12-01T01:11:08.354 INFO:journalctl@ceph.mon.b.smithi137.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-01T01:11:08.354 INFO:journalctl@ceph.mon.b.smithi137.stdout: 2023-12-01T01:11:08.354 INFO:journalctl@ceph.mon.b.smithi137.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-12-01T01:11:08.354 INFO:journalctl@ceph.mon.b.smithi137.stdout: 2023-12-01T01:11:08.354 INFO:journalctl@ceph.mon.b.smithi137.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-01T01:11:08.354 INFO:journalctl@ceph.mon.b.smithi137.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-12-01T01:11:08.354 INFO:journalctl@ceph.mon.b.smithi137.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-12-01T01:11:08.354 INFO:journalctl@ceph.mon.b.smithi137.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-12-01T01:11:08.354 INFO:journalctl@ceph.mon.b.smithi137.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-12-01T01:11:08.354 INFO:journalctl@ceph.mon.b.smithi137.stdout: AddFile(Keys): cumulative 0, interval 0 2023-12-01T01:11:08.354 INFO:journalctl@ceph.mon.b.smithi137.stdout: Cumulative compaction: 0.00 GB write, 0.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-01T01:11:08.355 INFO:journalctl@ceph.mon.b.smithi137.stdout: Interval compaction: 0.00 GB write, 0.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-01T01:11:08.355 INFO:journalctl@ceph.mon.b.smithi137.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-12-01T01:11:08.355 INFO:journalctl@ceph.mon.b.smithi137.stdout: Block cache BinnedLRUCache@0x5645483b5350#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-12-01T01:11:08.355 INFO:journalctl@ceph.mon.b.smithi137.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-12-01T01:11:08.355 INFO:journalctl@ceph.mon.b.smithi137.stdout: 2023-12-01T01:11:08.355 INFO:journalctl@ceph.mon.b.smithi137.stdout: ** File Read Latency Histogram By Level [default] ** 2023-12-01T01:11:08.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-12-01T01:11:08.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: using public_addr v2:172.21.15.137:0/0 -> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] 2023-12-01T01:11:08.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: starting mon.b rank -1 at public addrs [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] at bind addrs [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 2023-12-01T01:11:08.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: mon.b@-1(???) e0 preinit fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 2023-12-01T01:11:08.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: mon.b@-1(synchronizing).mds e1 new map 2023-12-01T01:11:08.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: mon.b@-1(synchronizing).mds e1 print_map 2023-12-01T01:11:08.355 INFO:journalctl@ceph.mon.b.smithi137.stdout: e1 2023-12-01T01:11:08.355 INFO:journalctl@ceph.mon.b.smithi137.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-12-01T01:11:08.356 INFO:journalctl@ceph.mon.b.smithi137.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-12-01T01:11:08.356 INFO:journalctl@ceph.mon.b.smithi137.stdout: legacy client fscid: -1 2023-12-01T01:11:08.356 INFO:journalctl@ceph.mon.b.smithi137.stdout: 2023-12-01T01:11:08.356 INFO:journalctl@ceph.mon.b.smithi137.stdout: No filesystems configured 2023-12-01T01:11:08.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-12-01T01:11:08.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-12-01T01:11:08.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-12-01T01:11:08.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-12-01T01:11:08.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-12-01T01:11:08.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-12-01T01:11:08.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-12-01T01:11:08.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-01T01:11:08.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-01T01:11:08.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-01T01:11:08.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:08.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: Updating smithi137:/var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-12-01T01:11:08.357 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:08.357 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:11:08.357 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:08.357 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: Added host smithi203 2023-12-01T01:11:08.357 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:08.357 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:08.357 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:08.357 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-01T01:11:08.357 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:08.357 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/3192192902' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-12-01T01:11:08.357 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/3192192902' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-12-01T01:11:08.357 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: osdmap e4: 0 total, 0 up, 0 in 2023-12-01T01:11:08.357 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:08.357 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:08.357 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:08.358 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:08.358 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi203", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:11:08.358 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:08.358 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:11:08.358 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: Updating smithi203:/etc/ceph/ceph.conf 2023-12-01T01:11:08.358 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:08.358 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: Updating smithi203:/var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-01T01:11:08.358 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:08.358 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:08.358 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:08.358 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2023-12-01T01:11:08.358 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: Updating smithi203:/var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-12-01T01:11:08.358 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:08.358 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:11:08.358 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:08.359 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:11:08.359 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:08.359 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-01T01:11:08.359 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:08.359 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi077:172.21.15.77=a;smithi137:172.21.15.137=b;smithi203:172.21.15.203=c", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:11:08.359 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: Saving service mon spec with placement smithi077:172.21.15.77=a;smithi137:172.21.15.137=b;smithi203:172.21.15.203=c;count:3 2023-12-01T01:11:08.359 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: Deploying daemon mon.c on smithi203 2023-12-01T01:11:08.359 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:08.359 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:08.359 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-01T01:11:08.359 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-01T01:11:08.359 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: mon.a calling monitor election 2023-12-01T01:11:08.359 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 bash[103272]: 2db06a6075f82302bd94d7d26f4b7559ceb5ccb811efba0efc93c430af86a234 2023-12-01T01:11:08.359 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:08.359 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-01T01:11:08.360 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-01T01:11:08.360 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: mon.c calling monitor election 2023-12-01T01:11:08.360 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:08.360 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-01T01:11:08.360 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-01T01:11:08.360 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:08.360 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-01T01:11:08.360 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-12-01T01:11:08.360 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: monmap e2: 2 mons at {a=[v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0],c=[v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-01T01:11:08.360 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: fsmap 2023-12-01T01:11:08.360 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: osdmap e4: 0 total, 0 up, 0 in 2023-12-01T01:11:08.360 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: mgrmap e13: x(active, since 40s) 2023-12-01T01:11:08.360 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: overall HEALTH_OK 2023-12-01T01:11:08.360 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:08.360 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:08.361 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:08.361 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-01T01:11:08.361 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:08.361 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: Deploying daemon mon.b on smithi137 2023-12-01T01:11:08.361 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-01T01:11:08.361 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:08.361 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: from='client.? 172.21.15.203:0/3438359456' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-01T01:11:08.361 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-12-01T01:11:08.361 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 ceph-mon[103312]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-12-01T01:11:08.361 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:08 smithi137 systemd[1]: Started Ceph mon.b for 2a06e29a-8fe6-11ee-95a2-87774f69a715. 2023-12-01T01:11:13.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:13 smithi077 ceph-mon[98867]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:13.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:13 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-01T01:11:13.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:13 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-01T01:11:13.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:13 smithi077 ceph-mon[98867]: mon.a calling monitor election 2023-12-01T01:11:13.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:13 smithi077 ceph-mon[98867]: mon.c calling monitor election 2023-12-01T01:11:13.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:13 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-01T01:11:13.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:13 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-01T01:11:13.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:13 smithi077 ceph-mon[98867]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:13.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:13 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-01T01:11:13.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:13 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-01T01:11:13.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:13 smithi077 ceph-mon[98867]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:13.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:13 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-01T01:11:13.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:13 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-01T01:11:13.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:13 smithi077 ceph-mon[98867]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-12-01T01:11:13.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:13 smithi077 ceph-mon[98867]: monmap e3: 3 mons at {a=[v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0],b=[v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0],c=[v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-01T01:11:13.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:13 smithi077 ceph-mon[98867]: fsmap 2023-12-01T01:11:13.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:13 smithi077 ceph-mon[98867]: osdmap e4: 0 total, 0 up, 0 in 2023-12-01T01:11:13.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:13 smithi077 ceph-mon[98867]: mgrmap e13: x(active, since 49s) 2023-12-01T01:11:13.656 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:13 smithi077 ceph-mon[98867]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-12-01T01:11:13.656 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:13 smithi077 ceph-mon[98867]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-12-01T01:11:13.656 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:13 smithi077 ceph-mon[98867]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-12-01T01:11:13.656 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:13 smithi077 ceph-mon[98867]: mon.b (rank 2) addr [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] is down (out of quorum) 2023-12-01T01:11:13.656 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:13 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:13.656 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:13 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:13.656 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:13 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:13.656 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:13 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:13.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:13 smithi203 ceph-mon[103581]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:13.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:13 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-01T01:11:13.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:13 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-01T01:11:13.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:13 smithi203 ceph-mon[103581]: mon.a calling monitor election 2023-12-01T01:11:13.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:13 smithi203 ceph-mon[103581]: mon.c calling monitor election 2023-12-01T01:11:13.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:13 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-01T01:11:13.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:13 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-01T01:11:13.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:13 smithi203 ceph-mon[103581]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:13.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:13 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-01T01:11:13.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:13 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-01T01:11:13.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:13 smithi203 ceph-mon[103581]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:13.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:13 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-01T01:11:13.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:13 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-01T01:11:13.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:13 smithi203 ceph-mon[103581]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-12-01T01:11:13.764 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:13 smithi203 ceph-mon[103581]: monmap e3: 3 mons at {a=[v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0],b=[v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0],c=[v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-01T01:11:13.764 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:13 smithi203 ceph-mon[103581]: fsmap 2023-12-01T01:11:13.764 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:13 smithi203 ceph-mon[103581]: osdmap e4: 0 total, 0 up, 0 in 2023-12-01T01:11:13.764 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:13 smithi203 ceph-mon[103581]: mgrmap e13: x(active, since 49s) 2023-12-01T01:11:13.764 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:13 smithi203 ceph-mon[103581]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-12-01T01:11:13.764 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:13 smithi203 ceph-mon[103581]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-12-01T01:11:13.764 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:13 smithi203 ceph-mon[103581]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-12-01T01:11:13.764 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:13 smithi203 ceph-mon[103581]: mon.b (rank 2) addr [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] is down (out of quorum) 2023-12-01T01:11:13.764 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:13 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:13.764 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:13 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:13.764 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:13 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:13.765 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:13 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:13.954 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:13.953+0000 7fb1da3fc700 1 -- 172.21.15.203:0/1763544480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1bc0035c0 msgr2=0x7fb1bc005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:11:13.954 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:13.953+0000 7fb1da3fc700 1 --2- 172.21.15.203:0/1763544480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1bc0035c0 0x7fb1bc005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4076780 tx=0x7fb1c400fe60 comp rx=0 tx=0).stop 2023-12-01T01:11:13.955 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:13.953+0000 7fb1da3fc700 1 -- 172.21.15.203:0/1763544480 shutdown_connections 2023-12-01T01:11:13.955 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:13.953+0000 7fb1da3fc700 1 --2- 172.21.15.203:0/1763544480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1bc0035c0 0x7fb1bc005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:11:13.955 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:13.953+0000 7fb1da3fc700 1 -- 172.21.15.203:0/1763544480 >> 172.21.15.203:0/1763544480 conn(0x7fb1d4075340 msgr2=0x7fb1d4075750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:11:13.955 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:13.953+0000 7fb1da3fc700 1 -- 172.21.15.203:0/1763544480 shutdown_connections 2023-12-01T01:11:13.955 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:13.953+0000 7fb1da3fc700 1 -- 172.21.15.203:0/1763544480 wait complete. 2023-12-01T01:11:13.957 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:13.954+0000 7fb1da3fc700 1 Processor -- start 2023-12-01T01:11:13.957 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:13.954+0000 7fb1da3fc700 1 -- start start 2023-12-01T01:11:13.957 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:13.954+0000 7fb1da3fc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1bc0035c0 0x7fb1d40ff9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:11:13.957 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:13.954+0000 7fb1da3fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1d40fff00 0x7fb1d4100380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:11:13.957 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:13.954+0000 7fb1da3fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1d41a20b0 0x7fb1d41a44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:11:13.957 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:13.954+0000 7fb1da3fc700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d41a4af0 con 0x7fb1d41a20b0 2023-12-01T01:11:13.957 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:13.954+0000 7fb1da3fc700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d41a4c60 con 0x7fb1d40fff00 2023-12-01T01:11:13.957 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:13.954+0000 7fb1da3fc700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d41a4e20 con 0x7fb1bc0035c0 2023-12-01T01:11:13.957 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:13.955+0000 7fb1d3fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1bc0035c0 0x7fb1d40ff9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:11:13.958 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:13.955+0000 7fb1d3fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1bc0035c0 0x7fb1d40ff9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:59158/0 (socket says 172.21.15.203:59158) 2023-12-01T01:11:13.958 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:13.955+0000 7fb1d3fff700 1 -- 172.21.15.203:0/411738954 learned_addr learned my addr 172.21.15.203:0/411738954 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-12-01T01:11:13.958 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:13.955+0000 7fb1d3fff700 1 -- 172.21.15.203:0/411738954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1bc0035c0 msgr2=0x7fb1d40ff9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-12-01T01:11:13.958 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:13.955+0000 7fb1d3fff700 1 -- 172.21.15.203:0/411738954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1bc0035c0 msgr2=0x7fb1d40ff9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-12-01T01:11:13.958 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:13.955+0000 7fb1d3fff700 1 --2- 172.21.15.203:0/411738954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1bc0035c0 0x7fb1d40ff9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-01T01:11:13.958 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:13.955+0000 7fb1d3fff700 1 --2- 172.21.15.203:0/411738954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1bc0035c0 0x7fb1d40ff9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-01T01:11:13.958 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:13.955+0000 7fb1d8999700 1 --2- 172.21.15.203:0/411738954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1d41a20b0 0x7fb1d41a44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:11:13.958 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:13.955+0000 7fb1d37fe700 1 --2- 172.21.15.203:0/411738954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1d40fff00 0x7fb1d4100380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:11:13.958 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:13.955+0000 7fb1d8999700 1 -- 172.21.15.203:0/411738954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1bc0035c0 msgr2=0x7fb1d40ff9c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-01T01:11:13.958 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:13.955+0000 7fb1d8999700 1 --2- 172.21.15.203:0/411738954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1bc0035c0 0x7fb1d40ff9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:11:13.958 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:13.955+0000 7fb1d8999700 1 -- 172.21.15.203:0/411738954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1d40fff00 msgr2=0x7fb1d4100380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:11:13.958 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:13.955+0000 7fb1d8999700 1 --2- 172.21.15.203:0/411738954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1d40fff00 0x7fb1d4100380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:11:13.958 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:13.955+0000 7fb1d8999700 1 -- 172.21.15.203:0/411738954 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c400f8b0 con 0x7fb1d41a20b0 2023-12-01T01:11:13.959 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:13.955+0000 7fb1d8999700 1 --2- 172.21.15.203:0/411738954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1d41a20b0 0x7fb1d41a44a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc00bd50 tx=0x7fb1cc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:11:13.959 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:13.955+0000 7fb1d17fa700 1 -- 172.21.15.203:0/411738954 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc00cd10 con 0x7fb1d41a20b0 2023-12-01T01:11:13.959 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:13.956+0000 7fb1d17fa700 1 -- 172.21.15.203:0/411738954 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1cc00ce70 con 0x7fb1d41a20b0 2023-12-01T01:11:13.959 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:13.956+0000 7fb1da3fc700 1 -- 172.21.15.203:0/411738954 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1d41a5100 con 0x7fb1d41a20b0 2023-12-01T01:11:13.959 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:13.956+0000 7fb1d17fa700 1 -- 172.21.15.203:0/411738954 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc012840 con 0x7fb1d41a20b0 2023-12-01T01:11:13.959 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:13.956+0000 7fb1da3fc700 1 -- 172.21.15.203:0/411738954 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1d41a55b0 con 0x7fb1d41a20b0 2023-12-01T01:11:13.960 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:13.956+0000 7fb1d17fa700 1 -- 172.21.15.203:0/411738954 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) v1 ==== 45324+0+0 (secure 0 0 0) 0x7fb1cc0129c0 con 0x7fb1d41a20b0 2023-12-01T01:11:13.960 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:13.956+0000 7fb1da3fc700 1 -- 172.21.15.203:0/411738954 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1d40623c0 con 0x7fb1d41a20b0 2023-12-01T01:11:13.960 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:13.957+0000 7fb1d17fa700 1 --2- 172.21.15.203:0/411738954 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fb1c0038870 0x7fb1c003ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:11:13.960 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:13.957+0000 7fb1d17fa700 1 -- 172.21.15.203:0/411738954 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb1cc04a800 con 0x7fb1d41a20b0 2023-12-01T01:11:13.960 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:13.957+0000 7fb1d3fff700 1 --2- 172.21.15.203:0/411738954 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fb1c0038870 0x7fb1c003ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:11:13.960 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:13.957+0000 7fb1d3fff700 1 --2- 172.21.15.203:0/411738954 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fb1c0038870 0x7fb1c003ad30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4015040 tx=0x7fb1c4001e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:11:13.963 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:13.962+0000 7fb1d17fa700 1 -- 172.21.15.203:0/411738954 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb1cc01d3b0 con 0x7fb1d41a20b0 2023-12-01T01:11:14.243 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:14.242+0000 7fb1da3fc700 1 -- 172.21.15.203:0/411738954 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb1d4100ef0 con 0x7fb1d41a20b0 2023-12-01T01:11:14.244 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:14.243+0000 7fb1d17fa700 1 -- 172.21.15.203:0/411738954 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1274 (secure 0 0 0) 0x7fb1cc0112a0 con 0x7fb1d41a20b0 2023-12-01T01:11:14.244 INFO:teuthology.orchestra.run.smithi203.stdout: 2023-12-01T01:11:14.244 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":3,"fsid":"2a06e29a-8fe6-11ee-95a2-87774f69a715","modified":"2023-12-01T01:11:08.297674Z","created":"2023-12-01T01:09:28.081844Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:3300","nonce":0},{"type":"v1","addr":"172.21.15.77:6789","nonce":0}]},"addr":"172.21.15.77:6789/0","public_addr":"172.21.15.77:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:3300","nonce":0},{"type":"v1","addr":"172.21.15.203:6789","nonce":0}]},"addr":"172.21.15.203:6789/0","public_addr":"172.21.15.203:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:3300","nonce":0},{"type":"v1","addr":"172.21.15.137:6789","nonce":0}]},"addr":"172.21.15.137:6789/0","public_addr":"172.21.15.137:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-12-01T01:11:14.246 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:14.245+0000 7fb1da3fc700 1 -- 172.21.15.203:0/411738954 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fb1c0038870 msgr2=0x7fb1c003ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:11:14.246 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:14.245+0000 7fb1da3fc700 1 --2- 172.21.15.203:0/411738954 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fb1c0038870 0x7fb1c003ad30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4015040 tx=0x7fb1c4001e50 comp rx=0 tx=0).stop 2023-12-01T01:11:14.247 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:14.245+0000 7fb1da3fc700 1 -- 172.21.15.203:0/411738954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1d41a20b0 msgr2=0x7fb1d41a44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:11:14.247 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:14.245+0000 7fb1da3fc700 1 --2- 172.21.15.203:0/411738954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1d41a20b0 0x7fb1d41a44a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc00bd50 tx=0x7fb1cc00be30 comp rx=0 tx=0).stop 2023-12-01T01:11:14.247 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:14.245+0000 7fb1da3fc700 1 -- 172.21.15.203:0/411738954 shutdown_connections 2023-12-01T01:11:14.247 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:14.245+0000 7fb1da3fc700 1 --2- 172.21.15.203:0/411738954 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fb1c0038870 0x7fb1c003ad30 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:11:14.247 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:14.245+0000 7fb1da3fc700 1 --2- 172.21.15.203:0/411738954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1d41a20b0 0x7fb1d41a44a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:11:14.247 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:14.245+0000 7fb1da3fc700 1 --2- 172.21.15.203:0/411738954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1bc0035c0 0x7fb1d40ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:11:14.247 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:14.245+0000 7fb1da3fc700 1 --2- 172.21.15.203:0/411738954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1d40fff00 0x7fb1d4100380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:11:14.247 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:14.245+0000 7fb1da3fc700 1 -- 172.21.15.203:0/411738954 >> 172.21.15.203:0/411738954 conn(0x7fb1d4075340 msgr2=0x7fb1d418edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:11:14.247 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:14.246+0000 7fb1da3fc700 1 -- 172.21.15.203:0/411738954 shutdown_connections 2023-12-01T01:11:14.247 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:14.246+0000 7fb1da3fc700 1 -- 172.21.15.203:0/411738954 wait complete. 2023-12-01T01:11:14.247 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 3 2023-12-01T01:11:14.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:14 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:11:14.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:14 smithi077 ceph-mon[98867]: from='client.? 172.21.15.203:0/411738954' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-01T01:11:14.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:14 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:14.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:14 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-01T01:11:14.685 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:14 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:11:14.685 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:14 smithi203 ceph-mon[103581]: from='client.? 172.21.15.203:0/411738954' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-01T01:11:14.685 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:14 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:14.685 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:14 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-01T01:11:14.695 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-12-01T01:11:14.696 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph config generate-minimal-conf 2023-12-01T01:11:15.268 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:11:15.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:15 smithi137 ceph-mon[103312]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:15.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:15 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-01T01:11:15.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:15 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-01T01:11:15.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:15 smithi137 ceph-mon[103312]: mon.a calling monitor election 2023-12-01T01:11:15.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:15 smithi137 ceph-mon[103312]: mon.c calling monitor election 2023-12-01T01:11:15.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:15 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-01T01:11:15.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:15 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-01T01:11:15.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:15 smithi137 ceph-mon[103312]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:15.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:15 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-01T01:11:15.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:15 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-01T01:11:15.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:15 smithi137 ceph-mon[103312]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:15.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:15 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-01T01:11:15.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:15 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-01T01:11:15.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:15 smithi137 ceph-mon[103312]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-12-01T01:11:15.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:15 smithi137 ceph-mon[103312]: monmap e3: 3 mons at {a=[v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0],b=[v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0],c=[v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-01T01:11:15.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:15 smithi137 ceph-mon[103312]: fsmap 2023-12-01T01:11:15.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:15 smithi137 ceph-mon[103312]: osdmap e4: 0 total, 0 up, 0 in 2023-12-01T01:11:15.781 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:15 smithi137 ceph-mon[103312]: mgrmap e13: x(active, since 49s) 2023-12-01T01:11:15.781 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:15 smithi137 ceph-mon[103312]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-12-01T01:11:15.781 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:15 smithi137 ceph-mon[103312]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-12-01T01:11:15.781 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:15 smithi137 ceph-mon[103312]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-12-01T01:11:15.781 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:15 smithi137 ceph-mon[103312]: mon.b (rank 2) addr [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] is down (out of quorum) 2023-12-01T01:11:15.781 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:15 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:15.781 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:15 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:15.781 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:15 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:15.781 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:15 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:15.781 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:15 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:11:15.781 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:15 smithi137 ceph-mon[103312]: from='client.? 172.21.15.203:0/411738954' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-01T01:11:15.781 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:15 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:15.781 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:15 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-01T01:11:16.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:16 smithi077 ceph-mon[98867]: mon.b calling monitor election 2023-12-01T01:11:16.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:16 smithi077 ceph-mon[98867]: Updating smithi077:/etc/ceph/ceph.conf 2023-12-01T01:11:16.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:16 smithi077 ceph-mon[98867]: Updating smithi137:/etc/ceph/ceph.conf 2023-12-01T01:11:16.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:16 smithi077 ceph-mon[98867]: Updating smithi203:/etc/ceph/ceph.conf 2023-12-01T01:11:16.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:16 smithi077 ceph-mon[98867]: mon.b calling monitor election 2023-12-01T01:11:16.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:16 smithi077 ceph-mon[98867]: mon.c calling monitor election 2023-12-01T01:11:16.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:16 smithi077 ceph-mon[98867]: mon.a calling monitor election 2023-12-01T01:11:16.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:16 smithi077 ceph-mon[98867]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-12-01T01:11:16.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:16 smithi077 ceph-mon[98867]: Updating smithi077:/var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-01T01:11:16.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:16 smithi077 ceph-mon[98867]: Updating smithi137:/var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-01T01:11:16.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:16 smithi077 ceph-mon[98867]: monmap e3: 3 mons at {a=[v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0],b=[v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0],c=[v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-01T01:11:16.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:16 smithi077 ceph-mon[98867]: Updating smithi203:/var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-01T01:11:16.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:16 smithi077 ceph-mon[98867]: fsmap 2023-12-01T01:11:16.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:16 smithi077 ceph-mon[98867]: osdmap e4: 0 total, 0 up, 0 in 2023-12-01T01:11:16.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:16 smithi077 ceph-mon[98867]: mgrmap e13: x(active, since 51s) 2023-12-01T01:11:16.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:16 smithi077 ceph-mon[98867]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-12-01T01:11:16.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:16 smithi077 ceph-mon[98867]: Cluster is now healthy 2023-12-01T01:11:16.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:16 smithi077 ceph-mon[98867]: overall HEALTH_OK 2023-12-01T01:11:16.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:16 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:16.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:16 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:16.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:16 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:16.406 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:16 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:16.406 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:16 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:16.406 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:16 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:16.406 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:16 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:16.406 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:16 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:16.406 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:16 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:16.406 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:16 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:16.406 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:16 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:16.406 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:16 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-01T01:11:16.406 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:16 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-01T01:11:16.406 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:16 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:16.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:16 smithi203 ceph-mon[103581]: mon.b calling monitor election 2023-12-01T01:11:16.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:16 smithi203 ceph-mon[103581]: Updating smithi077:/etc/ceph/ceph.conf 2023-12-01T01:11:16.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:16 smithi203 ceph-mon[103581]: Updating smithi137:/etc/ceph/ceph.conf 2023-12-01T01:11:16.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:16 smithi203 ceph-mon[103581]: Updating smithi203:/etc/ceph/ceph.conf 2023-12-01T01:11:16.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:16 smithi203 ceph-mon[103581]: mon.b calling monitor election 2023-12-01T01:11:16.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:16 smithi203 ceph-mon[103581]: mon.c calling monitor election 2023-12-01T01:11:16.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:16 smithi203 ceph-mon[103581]: mon.a calling monitor election 2023-12-01T01:11:16.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:16 smithi203 ceph-mon[103581]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-12-01T01:11:16.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:16 smithi203 ceph-mon[103581]: Updating smithi077:/var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-01T01:11:16.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:16 smithi203 ceph-mon[103581]: Updating smithi137:/var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-01T01:11:16.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:16 smithi203 ceph-mon[103581]: monmap e3: 3 mons at {a=[v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0],b=[v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0],c=[v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-01T01:11:16.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:16 smithi203 ceph-mon[103581]: Updating smithi203:/var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-01T01:11:16.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:16 smithi203 ceph-mon[103581]: fsmap 2023-12-01T01:11:16.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:16 smithi203 ceph-mon[103581]: osdmap e4: 0 total, 0 up, 0 in 2023-12-01T01:11:16.514 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:16 smithi203 ceph-mon[103581]: mgrmap e13: x(active, since 51s) 2023-12-01T01:11:16.514 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:16 smithi203 ceph-mon[103581]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-12-01T01:11:16.514 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:16 smithi203 ceph-mon[103581]: Cluster is now healthy 2023-12-01T01:11:16.514 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:16 smithi203 ceph-mon[103581]: overall HEALTH_OK 2023-12-01T01:11:16.514 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:16 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:16.514 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:16 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:16.514 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:16 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:16.514 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:16 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:16.514 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:16 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:16.514 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:16 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:16.514 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:16 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:16.514 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:16 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:16.514 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:16 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:16.514 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:16 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:16.515 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:16 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:16.515 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:16 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-01T01:11:16.515 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:16 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-01T01:11:16.515 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:16 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:16.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:16 smithi137 ceph-mon[103312]: mon.b calling monitor election 2023-12-01T01:11:16.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:16 smithi137 ceph-mon[103312]: Updating smithi077:/etc/ceph/ceph.conf 2023-12-01T01:11:16.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:16 smithi137 ceph-mon[103312]: Updating smithi137:/etc/ceph/ceph.conf 2023-12-01T01:11:16.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:16 smithi137 ceph-mon[103312]: Updating smithi203:/etc/ceph/ceph.conf 2023-12-01T01:11:16.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:16 smithi137 ceph-mon[103312]: mon.b calling monitor election 2023-12-01T01:11:16.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:16 smithi137 ceph-mon[103312]: mon.c calling monitor election 2023-12-01T01:11:16.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:16 smithi137 ceph-mon[103312]: mon.a calling monitor election 2023-12-01T01:11:16.530 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:16 smithi137 ceph-mon[103312]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-12-01T01:11:16.530 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:16 smithi137 ceph-mon[103312]: Updating smithi077:/var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-01T01:11:16.530 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:16 smithi137 ceph-mon[103312]: Updating smithi137:/var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-01T01:11:16.530 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:16 smithi137 ceph-mon[103312]: monmap e3: 3 mons at {a=[v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0],b=[v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0],c=[v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-01T01:11:16.530 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:16 smithi137 ceph-mon[103312]: Updating smithi203:/var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/config/ceph.conf 2023-12-01T01:11:16.530 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:16 smithi137 ceph-mon[103312]: fsmap 2023-12-01T01:11:16.530 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:16 smithi137 ceph-mon[103312]: osdmap e4: 0 total, 0 up, 0 in 2023-12-01T01:11:16.530 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:16 smithi137 ceph-mon[103312]: mgrmap e13: x(active, since 51s) 2023-12-01T01:11:16.530 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:16 smithi137 ceph-mon[103312]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-12-01T01:11:16.530 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:16 smithi137 ceph-mon[103312]: Cluster is now healthy 2023-12-01T01:11:16.530 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:16 smithi137 ceph-mon[103312]: overall HEALTH_OK 2023-12-01T01:11:16.530 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:16 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:16.530 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:16 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:16.530 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:16 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:16.530 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:16 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:16.531 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:16 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:16.531 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:16 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:16.531 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:16 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:16.531 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:16 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:16.531 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:16 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:16.531 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:16 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:16.531 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:16 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:16.531 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:16 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-01T01:11:16.531 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:16 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-01T01:11:16.531 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:16 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:16.641 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.639+0000 7fa4a5a86700 1 -- 172.21.15.77:0/1863172557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4a0105700 msgr2=0x7fa4a0105ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:11:16.641 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.639+0000 7fa4a5a86700 1 --2- 172.21.15.77:0/1863172557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4a0105700 0x7fa4a0105ae0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa490009910 tx=0x7fa490009c20 comp rx=0 tx=0).stop 2023-12-01T01:11:16.641 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.639+0000 7fa4a5a86700 1 -- 172.21.15.77:0/1863172557 shutdown_connections 2023-12-01T01:11:16.641 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.639+0000 7fa4a5a86700 1 --2- 172.21.15.77:0/1863172557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4a0105700 0x7fa4a0105ae0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:11:16.641 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.639+0000 7fa4a5a86700 1 -- 172.21.15.77:0/1863172557 >> 172.21.15.77:0/1863172557 conn(0x7fa4a00fb210 msgr2=0x7fa4a00fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:11:16.641 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.639+0000 7fa4a5a86700 1 -- 172.21.15.77:0/1863172557 shutdown_connections 2023-12-01T01:11:16.641 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.639+0000 7fa4a5a86700 1 -- 172.21.15.77:0/1863172557 wait complete. 2023-12-01T01:11:16.641 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.640+0000 7fa4a5a86700 1 Processor -- start 2023-12-01T01:11:16.642 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.640+0000 7fa4a5a86700 1 -- start start 2023-12-01T01:11:16.642 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.640+0000 7fa4a5a86700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4a0105700 0x7fa4a0198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:11:16.643 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.640+0000 7fa4a5a86700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4a0198560 0x7fa4a019c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:11:16.643 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.640+0000 7fa4a5a86700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4a019cf10 0x7fa4a019d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:11:16.643 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.640+0000 7fa4a5a86700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a0198c50 con 0x7fa4a0198560 2023-12-01T01:11:16.643 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.640+0000 7fa4a5a86700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a01a4c80 con 0x7fa4a0105700 2023-12-01T01:11:16.643 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.640+0000 7fa4a5a86700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a01a4dc0 con 0x7fa4a019cf10 2023-12-01T01:11:16.643 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.640+0000 7fa49effd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4a0198560 0x7fa4a019c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:11:16.643 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.640+0000 7fa49effd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4a0198560 0x7fa4a019c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56560/0 (socket says 172.21.15.77:56560) 2023-12-01T01:11:16.643 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.640+0000 7fa49effd700 1 -- 172.21.15.77:0/4086337280 learned_addr learned my addr 172.21.15.77:0/4086337280 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:11:16.643 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.640+0000 7fa49effd700 1 -- 172.21.15.77:0/4086337280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4a019cf10 msgr2=0x7fa4a019d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:11:16.643 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.640+0000 7fa49effd700 1 --2- 172.21.15.77:0/4086337280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4a019cf10 0x7fa4a019d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:11:16.643 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.640+0000 7fa49effd700 1 -- 172.21.15.77:0/4086337280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4a0105700 msgr2=0x7fa4a0198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:11:16.643 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.640+0000 7fa49effd700 1 --2- 172.21.15.77:0/4086337280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4a0105700 0x7fa4a0198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:11:16.643 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.641+0000 7fa49effd700 1 -- 172.21.15.77:0/4086337280 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4900095f0 con 0x7fa4a0198560 2023-12-01T01:11:16.643 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.641+0000 7fa49effd700 1 --2- 172.21.15.77:0/4086337280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4a0198560 0x7fa4a019c9d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa49400b780 tx=0x7fa49400bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:11:16.644 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.641+0000 7fa49cff9700 1 -- 172.21.15.77:0/4086337280 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa494004230 con 0x7fa4a0198560 2023-12-01T01:11:16.644 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.641+0000 7fa49cff9700 1 -- 172.21.15.77:0/4086337280 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa494004390 con 0x7fa4a0198560 2023-12-01T01:11:16.644 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.641+0000 7fa49cff9700 1 -- 172.21.15.77:0/4086337280 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa494011b30 con 0x7fa4a0198560 2023-12-01T01:11:16.644 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.641+0000 7fa4a5a86700 1 -- 172.21.15.77:0/4086337280 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4a01a50a0 con 0x7fa4a0198560 2023-12-01T01:11:16.644 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.641+0000 7fa4a5a86700 1 -- 172.21.15.77:0/4086337280 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4a01a55a0 con 0x7fa4a0198560 2023-12-01T01:11:16.645 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.642+0000 7fa49cff9700 1 -- 172.21.15.77:0/4086337280 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) v1 ==== 45324+0+0 (secure 0 0 0) 0x7fa494011cd0 con 0x7fa4a0198560 2023-12-01T01:11:16.645 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.642+0000 7fa4a5a86700 1 -- 172.21.15.77:0/4086337280 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4a00623c0 con 0x7fa4a0198560 2023-12-01T01:11:16.645 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.642+0000 7fa49cff9700 1 --2- 172.21.15.77:0/4086337280 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa488038820 0x7fa48803ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:11:16.645 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.642+0000 7fa49cff9700 1 -- 172.21.15.77:0/4086337280 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa49404a920 con 0x7fa4a0198560 2023-12-01T01:11:16.645 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.642+0000 7fa49f7fe700 1 --2- 172.21.15.77:0/4086337280 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa488038820 0x7fa48803ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:11:16.645 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.643+0000 7fa49f7fe700 1 --2- 172.21.15.77:0/4086337280 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa488038820 0x7fa48803ace0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa490005e00 tx=0x7fa490005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:11:16.649 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.647+0000 7fa49cff9700 1 -- 172.21.15.77:0/4086337280 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa49401a400 con 0x7fa4a0198560 2023-12-01T01:11:16.846 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.845+0000 7fa4a5a86700 1 -- 172.21.15.77:0/4086337280 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fa4a0199380 con 0x7fa4a0198560 2023-12-01T01:11:16.846 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.845+0000 7fa49cff9700 1 -- 172.21.15.77:0/4086337280 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+275 (secure 0 0 0) 0x7fa49400f2e0 con 0x7fa4a0198560 2023-12-01T01:11:16.846 INFO:teuthology.orchestra.run.smithi077.stdout:# minimal ceph.conf for 2a06e29a-8fe6-11ee-95a2-87774f69a715 2023-12-01T01:11:16.847 INFO:teuthology.orchestra.run.smithi077.stdout:[global] 2023-12-01T01:11:16.847 INFO:teuthology.orchestra.run.smithi077.stdout: fsid = 2a06e29a-8fe6-11ee-95a2-87774f69a715 2023-12-01T01:11:16.847 INFO:teuthology.orchestra.run.smithi077.stdout: mon_host = [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] 2023-12-01T01:11:16.848 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.847+0000 7fa4a5a86700 1 -- 172.21.15.77:0/4086337280 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa488038820 msgr2=0x7fa48803ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:11:16.848 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.847+0000 7fa4a5a86700 1 --2- 172.21.15.77:0/4086337280 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa488038820 0x7fa48803ace0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa490005e00 tx=0x7fa490005fd0 comp rx=0 tx=0).stop 2023-12-01T01:11:16.848 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.847+0000 7fa4a5a86700 1 -- 172.21.15.77:0/4086337280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4a0198560 msgr2=0x7fa4a019c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:11:16.848 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.847+0000 7fa4a5a86700 1 --2- 172.21.15.77:0/4086337280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4a0198560 0x7fa4a019c9d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa49400b780 tx=0x7fa49400bb40 comp rx=0 tx=0).stop 2023-12-01T01:11:16.849 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.847+0000 7fa4a5a86700 1 -- 172.21.15.77:0/4086337280 shutdown_connections 2023-12-01T01:11:16.849 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.847+0000 7fa4a5a86700 1 --2- 172.21.15.77:0/4086337280 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa488038820 0x7fa48803ace0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:11:16.849 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.847+0000 7fa4a5a86700 1 --2- 172.21.15.77:0/4086337280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4a019cf10 0x7fa4a019d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:11:16.849 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.847+0000 7fa4a5a86700 1 --2- 172.21.15.77:0/4086337280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4a0105700 0x7fa4a0198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:11:16.849 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.847+0000 7fa4a5a86700 1 --2- 172.21.15.77:0/4086337280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4a0198560 0x7fa4a019c9d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:11:16.849 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.847+0000 7fa4a5a86700 1 -- 172.21.15.77:0/4086337280 >> 172.21.15.77:0/4086337280 conn(0x7fa4a00fb210 msgr2=0x7fa4a0068820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:11:16.849 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.847+0000 7fa4a5a86700 1 -- 172.21.15.77:0/4086337280 shutdown_connections 2023-12-01T01:11:16.849 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:16.847+0000 7fa4a5a86700 1 -- 172.21.15.77:0/4086337280 wait complete. 2023-12-01T01:11:17.314 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:17 smithi077 ceph-mon[98867]: Reconfiguring mon.a (unknown last config time)... 2023-12-01T01:11:17.314 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:17 smithi077 ceph-mon[98867]: Reconfiguring daemon mon.a on smithi077 2023-12-01T01:11:17.314 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:17 smithi077 ceph-mon[98867]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:17.314 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:17 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-01T01:11:17.315 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:17 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/4086337280' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:17.315 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-12-01T01:11:17.316 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-12-01T01:11:17.316 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/ceph/ceph.conf 2023-12-01T01:11:17.346 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-12-01T01:11:17.346 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-01T01:11:17.412 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2023-12-01T01:11:17.412 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/ceph/ceph.conf 2023-12-01T01:11:17.438 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:17 smithi137 ceph-mon[103312]: Reconfiguring mon.a (unknown last config time)... 2023-12-01T01:11:17.438 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:17 smithi137 ceph-mon[103312]: Reconfiguring daemon mon.a on smithi077 2023-12-01T01:11:17.438 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:17 smithi137 ceph-mon[103312]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:17.438 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:17 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-01T01:11:17.439 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:17 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/4086337280' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:17.443 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2023-12-01T01:11:17.443 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-01T01:11:17.511 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-12-01T01:11:17.511 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/ceph/ceph.conf 2023-12-01T01:11:17.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:17 smithi203 ceph-mon[103581]: Reconfiguring mon.a (unknown last config time)... 2023-12-01T01:11:17.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:17 smithi203 ceph-mon[103581]: Reconfiguring daemon mon.a on smithi077 2023-12-01T01:11:17.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:17 smithi203 ceph-mon[103581]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:17.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:17 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-01T01:11:17.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:17 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/4086337280' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:17.547 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-12-01T01:11:17.547 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-01T01:11:17.618 INFO:tasks.cephadm:Adding mgr.x on smithi077 2023-12-01T01:11:17.619 INFO:tasks.cephadm:Adding mgr.y on smithi137 2023-12-01T01:11:17.619 INFO:tasks.cephadm:Adding mgr.z on smithi203 2023-12-01T01:11:17.619 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph orch apply mgr '3;smithi077=x;smithi137=y;smithi203=z' 2023-12-01T01:11:18.211 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.c/config 2023-12-01T01:11:18.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:18 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:18.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:18 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:18.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:18 smithi137 ceph-mon[103312]: Reconfiguring mon.b (monmap changed)... 2023-12-01T01:11:18.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:18 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-01T01:11:18.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:18 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-01T01:11:18.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:18 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:18.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:18 smithi137 ceph-mon[103312]: Reconfiguring daemon mon.b on smithi137 2023-12-01T01:11:18.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:18 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:18.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:18 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:18.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:18 smithi077 ceph-mon[98867]: Reconfiguring mon.b (monmap changed)... 2023-12-01T01:11:18.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:18 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-01T01:11:18.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:18 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-01T01:11:18.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:18 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:18.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:18 smithi077 ceph-mon[98867]: Reconfiguring daemon mon.b on smithi137 2023-12-01T01:11:18.450 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:18 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:18.450 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:18 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:18.450 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:18 smithi203 ceph-mon[103581]: Reconfiguring mon.b (monmap changed)... 2023-12-01T01:11:18.451 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:18 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-01T01:11:18.451 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:18 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-01T01:11:18.451 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:18 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:18.451 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:18 smithi203 ceph-mon[103581]: Reconfiguring daemon mon.b on smithi137 2023-12-01T01:11:19.404 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:19 smithi203 ceph-mon[103581]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:19.404 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:19 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:19.404 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:19 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:19.404 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:19 smithi203 ceph-mon[103581]: Reconfiguring mon.c (monmap changed)... 2023-12-01T01:11:19.404 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:19 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-01T01:11:19.405 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:19 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-01T01:11:19.405 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:19 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:19.405 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:19 smithi203 ceph-mon[103581]: Reconfiguring daemon mon.c on smithi203 2023-12-01T01:11:19.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:19 smithi077 ceph-mon[98867]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:19.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:19 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:19.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:19 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:19.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:19 smithi077 ceph-mon[98867]: Reconfiguring mon.c (monmap changed)... 2023-12-01T01:11:19.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:19 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-01T01:11:19.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:19 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-01T01:11:19.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:19 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:19.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:19 smithi077 ceph-mon[98867]: Reconfiguring daemon mon.c on smithi203 2023-12-01T01:11:19.707 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.706+0000 7f6800b38700 1 -- 172.21.15.203:0/780714700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67fc100f30 msgr2=0x7f67fc101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:11:19.707 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.706+0000 7f6800b38700 1 --2- 172.21.15.203:0/780714700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67fc100f30 0x7f67fc101350 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f67ec009910 tx=0x7f67ec009c20 comp rx=0 tx=0).stop 2023-12-01T01:11:19.707 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.706+0000 7f6800b38700 1 -- 172.21.15.203:0/780714700 shutdown_connections 2023-12-01T01:11:19.707 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.706+0000 7f6800b38700 1 --2- 172.21.15.203:0/780714700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67fc100f30 0x7f67fc101350 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:11:19.707 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.706+0000 7f6800b38700 1 -- 172.21.15.203:0/780714700 >> 172.21.15.203:0/780714700 conn(0x7f67fc0fa5d0 msgr2=0x7f67fc0fca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:11:19.708 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.706+0000 7f6800b38700 1 -- 172.21.15.203:0/780714700 shutdown_connections 2023-12-01T01:11:19.708 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.706+0000 7f6800b38700 1 -- 172.21.15.203:0/780714700 wait complete. 2023-12-01T01:11:19.709 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.707+0000 7f6800b38700 1 Processor -- start 2023-12-01T01:11:19.709 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.707+0000 7f6800b38700 1 -- start start 2023-12-01T01:11:19.709 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.707+0000 7f6800b38700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67fc100f30 0x7f67fc197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:11:19.709 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.707+0000 7f6800b38700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67fc198380 0x7f67fc19d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:11:19.709 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.707+0000 7f6800b38700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67fc198800 0x7f67fc19f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:11:19.710 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.707+0000 7f6800b38700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc1a0120 con 0x7f67fc198380 2023-12-01T01:11:19.710 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.707+0000 7f6800b38700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc1a0290 con 0x7f67fc198800 2023-12-01T01:11:19.710 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.707+0000 7f6800b38700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc1a0450 con 0x7f67fc100f30 2023-12-01T01:11:19.710 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.707+0000 7f67fa59c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67fc100f30 0x7f67fc197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:11:19.710 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.707+0000 7f67fa59c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67fc100f30 0x7f67fc197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:59168/0 (socket says 172.21.15.203:59168) 2023-12-01T01:11:19.710 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.707+0000 7f67fa59c700 1 -- 172.21.15.203:0/4085086801 learned_addr learned my addr 172.21.15.203:0/4085086801 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-12-01T01:11:19.710 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.708+0000 7f67f9d9b700 1 --2- 172.21.15.203:0/4085086801 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67fc198380 0x7f67fc19d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:11:19.710 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.708+0000 7f67fad9d700 1 --2- 172.21.15.203:0/4085086801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67fc198800 0x7f67fc19f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:11:19.710 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.708+0000 7f67fa59c700 1 -- 172.21.15.203:0/4085086801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67fc198800 msgr2=0x7f67fc19f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:11:19.710 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.708+0000 7f67fa59c700 1 --2- 172.21.15.203:0/4085086801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67fc198800 0x7f67fc19f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:11:19.710 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.708+0000 7f67fa59c700 1 -- 172.21.15.203:0/4085086801 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67fc198380 msgr2=0x7f67fc19d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:11:19.710 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.708+0000 7f67fa59c700 1 --2- 172.21.15.203:0/4085086801 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67fc198380 0x7f67fc19d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:11:19.710 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.708+0000 7f67fa59c700 1 -- 172.21.15.203:0/4085086801 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ec0095f0 con 0x7f67fc100f30 2023-12-01T01:11:19.710 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.708+0000 7f67f9d9b700 1 --2- 172.21.15.203:0/4085086801 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67fc198380 0x7f67fc19d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:11:19.711 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.708+0000 7f67fa59c700 1 --2- 172.21.15.203:0/4085086801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67fc100f30 0x7f67fc197e40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f67ec004cb0 tx=0x7f67ec004d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:11:19.711 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.708+0000 7f67eb7fe700 1 -- 172.21.15.203:0/4085086801 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ec01d070 con 0x7f67fc100f30 2023-12-01T01:11:19.711 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.708+0000 7f67eb7fe700 1 -- 172.21.15.203:0/4085086801 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67ec00be20 con 0x7f67fc100f30 2023-12-01T01:11:19.711 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.708+0000 7f6800b38700 1 -- 172.21.15.203:0/4085086801 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67fc1a06d0 con 0x7f67fc100f30 2023-12-01T01:11:19.711 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.708+0000 7f67eb7fe700 1 -- 172.21.15.203:0/4085086801 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ec00f460 con 0x7f67fc100f30 2023-12-01T01:11:19.711 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.708+0000 7f6800b38700 1 -- 172.21.15.203:0/4085086801 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67fc1a0bf0 con 0x7f67fc100f30 2023-12-01T01:11:19.712 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.709+0000 7f67eb7fe700 1 -- 172.21.15.203:0/4085086801 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 13) v1 ==== 45324+0+0 (secure 0 0 0) 0x7f67ec00f5c0 con 0x7f67fc100f30 2023-12-01T01:11:19.712 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.709+0000 7f67eb7fe700 1 --2- 172.21.15.203:0/4085086801 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f67e4038820 0x7f67e403ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:11:19.712 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.709+0000 7f6800b38700 1 -- 172.21.15.203:0/4085086801 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67fc105660 con 0x7f67fc100f30 2023-12-01T01:11:19.712 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.709+0000 7f67eb7fe700 1 -- 172.21.15.203:0/4085086801 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f67ec04c3d0 con 0x7f67fc100f30 2023-12-01T01:11:19.712 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.709+0000 7f67f9d9b700 1 --2- 172.21.15.203:0/4085086801 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f67e4038820 0x7f67e403ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:11:19.712 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.710+0000 7f67f9d9b700 1 --2- 172.21.15.203:0/4085086801 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f67e4038820 0x7f67e403ace0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f67f0009910 tx=0x7f67f0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:11:19.715 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.714+0000 7f67eb7fe700 1 -- 172.21.15.203:0/4085086801 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f67ec020550 con 0x7f67fc100f30 2023-12-01T01:11:19.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:19 smithi137 ceph-mon[103312]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:19.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:19 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:19.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:19 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:19.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:19 smithi137 ceph-mon[103312]: Reconfiguring mon.c (monmap changed)... 2023-12-01T01:11:19.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:19 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-01T01:11:19.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:19 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-01T01:11:19.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:19 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:19.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:19 smithi137 ceph-mon[103312]: Reconfiguring daemon mon.c on smithi203 2023-12-01T01:11:19.918 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.917+0000 7f6800b38700 1 -- 172.21.15.203:0/4085086801 --> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi077=x;smithi137=y;smithi203=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f67fc1073f0 con 0x7f67e4038820 2023-12-01T01:11:19.924 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.924+0000 7f67eb7fe700 1 -- 172.21.15.203:0/4085086801 <== mgr.14152 v2:172.21.15.77:6800/3838941679 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f67fc1073f0 con 0x7f67e4038820 2023-12-01T01:11:19.925 INFO:teuthology.orchestra.run.smithi203.stdout:Scheduled mgr update... 2023-12-01T01:11:19.927 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.926+0000 7f6800b38700 1 -- 172.21.15.203:0/4085086801 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f67e4038820 msgr2=0x7f67e403ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:11:19.927 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.926+0000 7f6800b38700 1 --2- 172.21.15.203:0/4085086801 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f67e4038820 0x7f67e403ace0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f67f0009910 tx=0x7f67f0008040 comp rx=0 tx=0).stop 2023-12-01T01:11:19.927 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.926+0000 7f6800b38700 1 -- 172.21.15.203:0/4085086801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67fc100f30 msgr2=0x7f67fc197e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:11:19.927 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.926+0000 7f6800b38700 1 --2- 172.21.15.203:0/4085086801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67fc100f30 0x7f67fc197e40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f67ec004cb0 tx=0x7f67ec004d90 comp rx=0 tx=0).stop 2023-12-01T01:11:19.927 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.926+0000 7f6800b38700 1 -- 172.21.15.203:0/4085086801 shutdown_connections 2023-12-01T01:11:19.927 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.926+0000 7f6800b38700 1 --2- 172.21.15.203:0/4085086801 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f67e4038820 0x7f67e403ace0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:11:19.927 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.926+0000 7f6800b38700 1 --2- 172.21.15.203:0/4085086801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67fc198800 0x7f67fc19f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:11:19.927 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.926+0000 7f6800b38700 1 --2- 172.21.15.203:0/4085086801 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67fc100f30 0x7f67fc197e40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:11:19.927 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.926+0000 7f6800b38700 1 --2- 172.21.15.203:0/4085086801 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67fc198380 0x7f67fc19d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:11:19.927 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.926+0000 7f6800b38700 1 -- 172.21.15.203:0/4085086801 >> 172.21.15.203:0/4085086801 conn(0x7f67fc0fa5d0 msgr2=0x7f67fc0fca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:11:19.927 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.926+0000 7f6800b38700 1 -- 172.21.15.203:0/4085086801 shutdown_connections 2023-12-01T01:11:19.927 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:11:19.926+0000 7f6800b38700 1 -- 172.21.15.203:0/4085086801 wait complete. 2023-12-01T01:11:20.453 DEBUG:teuthology.orchestra.run.smithi137:mgr.y> sudo journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mgr.y.service 2023-12-01T01:11:20.456 DEBUG:teuthology.orchestra.run.smithi203:mgr.z> sudo journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mgr.z.service 2023-12-01T01:11:20.458 INFO:tasks.cephadm:Deploying OSDs... 2023-12-01T01:11:20.458 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-12-01T01:11:20.458 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/scratch_devs of=/dev/stdout 2023-12-01T01:11:20.479 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-01T01:11:20.480 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_1 2023-12-01T01:11:20.489 INFO:journalctl@ceph.mgr.z.smithi203.stdout:-- Logs begin at Fri 2023-12-01 00:55:29 UTC. -- 2023-12-01T01:11:20.491 INFO:journalctl@ceph.mgr.y.smithi137.stdout:-- Logs begin at Fri 2023-12-01 00:55:04 UTC. -- 2023-12-01T01:11:20.541 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-01T01:11:20.541 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-01T01:11:20.541 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 6h/6d Inode: 391493 Links: 1 2023-12-01T01:11:20.541 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-01T01:11:20.541 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2023-12-01T01:11:20.541 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2023-12-01 01:10:34.633496953 +0000 2023-12-01T01:11:20.541 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2023-12-01 01:04:46.183766563 +0000 2023-12-01T01:11:20.542 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2023-12-01 01:04:46.183766563 +0000 2023-12-01T01:11:20.542 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: - 2023-12-01T01:11:20.542 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-01T01:11:20.611 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2023-12-01T01:11:20.611 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2023-12-01T01:11:20.611 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000261653 s, 2.0 MB/s 2023-12-01T01:11:20.612 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-01T01:11:20.673 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_2 2023-12-01T01:11:20.732 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-01T01:11:20.732 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-01T01:11:20.732 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 6h/6d Inode: 384643 Links: 1 2023-12-01T01:11:20.732 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-01T01:11:20.732 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2023-12-01T01:11:20.732 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2023-12-01 01:10:34.673496024 +0000 2023-12-01T01:11:20.732 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2023-12-01 01:04:46.691754819 +0000 2023-12-01T01:11:20.732 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2023-12-01 01:04:46.691754819 +0000 2023-12-01T01:11:20.732 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: - 2023-12-01T01:11:20.733 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-01T01:11:20.798 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2023-12-01T01:11:20.798 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2023-12-01T01:11:20.799 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000255455 s, 2.0 MB/s 2023-12-01T01:11:20.800 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-01T01:11:20.858 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_3 2023-12-01T01:11:20.919 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-01T01:11:20.919 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-01T01:11:20.920 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 6h/6d Inode: 394490 Links: 1 2023-12-01T01:11:20.920 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-01T01:11:20.920 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2023-12-01T01:11:20.920 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2023-12-01 01:10:34.711495140 +0000 2023-12-01T01:11:20.920 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2023-12-01 01:04:47.202743005 +0000 2023-12-01T01:11:20.920 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2023-12-01 01:04:47.202743005 +0000 2023-12-01T01:11:20.920 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: - 2023-12-01T01:11:20.920 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-01T01:11:20.986 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2023-12-01T01:11:20.986 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2023-12-01T01:11:20.986 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000238664 s, 2.1 MB/s 2023-12-01T01:11:20.987 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-01T01:11:21.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:20 smithi203 ceph-mon[103581]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi077=x;smithi137=y;smithi203=z", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:11:21.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:20 smithi203 ceph-mon[103581]: Saving service mgr spec with placement smithi077=x;smithi137=y;smithi203=z;count:3 2023-12-01T01:11:21.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:20 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:21.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:20 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:21.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:20 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:21.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:20 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:11:21.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:20 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:21.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:20 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:11:21.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:20 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:21.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:20 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-01T01:11:21.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:20 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-01T01:11:21.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:20 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-01T01:11:21.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:20 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:21.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:20 smithi203 ceph-mon[103581]: Deploying daemon mgr.z on smithi203 2023-12-01T01:11:21.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:20 smithi203 ceph-mon[103581]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:21.045 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_4 2023-12-01T01:11:21.102 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-01T01:11:21.102 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-01T01:11:21.103 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 6h/6d Inode: 393214 Links: 1 2023-12-01T01:11:21.103 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-01T01:11:21.103 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2023-12-01T01:11:21.103 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2023-12-01 01:10:34.753494164 +0000 2023-12-01T01:11:21.103 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2023-12-01 01:04:47.711731237 +0000 2023-12-01T01:11:21.103 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2023-12-01 01:04:47.711731237 +0000 2023-12-01T01:11:21.103 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: - 2023-12-01T01:11:21.103 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-01T01:11:21.168 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2023-12-01T01:11:21.168 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2023-12-01T01:11:21.168 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000217998 s, 2.3 MB/s 2023-12-01T01:11:21.169 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-01T01:11:21.227 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2023-12-01T01:11:21.227 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/scratch_devs of=/dev/stdout 2023-12-01T01:11:21.247 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-01T01:11:21.248 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_1 2023-12-01T01:11:21.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:20 smithi137 ceph-mon[103312]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi077=x;smithi137=y;smithi203=z", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:11:21.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:20 smithi137 ceph-mon[103312]: Saving service mgr spec with placement smithi077=x;smithi137=y;smithi203=z;count:3 2023-12-01T01:11:21.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:20 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:21.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:20 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:21.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:20 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:21.280 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:20 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:11:21.280 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:20 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:21.280 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:20 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:11:21.280 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:20 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:21.280 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:20 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-01T01:11:21.280 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:20 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-01T01:11:21.280 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:20 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-01T01:11:21.280 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:20 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:21.280 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:20 smithi137 ceph-mon[103312]: Deploying daemon mgr.z on smithi203 2023-12-01T01:11:21.280 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:20 smithi137 ceph-mon[103312]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:21.300 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-01T01:11:21.300 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-01T01:11:21.300 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 6h/6d Inode: 391505 Links: 1 2023-12-01T01:11:21.300 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-01T01:11:21.300 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2023-12-01T01:11:21.300 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2023-12-01 01:11:07.701392108 +0000 2023-12-01T01:11:21.300 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2023-12-01 01:05:03.918985146 +0000 2023-12-01T01:11:21.300 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2023-12-01 01:05:03.918985146 +0000 2023-12-01T01:11:21.300 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: - 2023-12-01T01:11:21.301 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-01T01:11:21.369 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2023-12-01T01:11:21.369 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2023-12-01T01:11:21.369 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000259437 s, 2.0 MB/s 2023-12-01T01:11:21.370 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-01T01:11:21.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:20 smithi077 ceph-mon[98867]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi077=x;smithi137=y;smithi203=z", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:11:21.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:20 smithi077 ceph-mon[98867]: Saving service mgr spec with placement smithi077=x;smithi137=y;smithi203=z;count:3 2023-12-01T01:11:21.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:20 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:21.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:20 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:21.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:20 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:21.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:20 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:11:21.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:20 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:21.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:20 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:11:21.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:20 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:21.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:20 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-01T01:11:21.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:20 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-01T01:11:21.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:20 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-01T01:11:21.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:20 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:21.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:20 smithi077 ceph-mon[98867]: Deploying daemon mgr.z on smithi203 2023-12-01T01:11:21.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:20 smithi077 ceph-mon[98867]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:21.429 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_2 2023-12-01T01:11:21.488 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-01T01:11:21.488 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-01T01:11:21.488 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 6h/6d Inode: 391593 Links: 1 2023-12-01T01:11:21.488 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-01T01:11:21.488 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2023-12-01T01:11:21.488 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2023-12-01 01:11:07.701392108 +0000 2023-12-01T01:11:21.488 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2023-12-01 01:05:04.465975514 +0000 2023-12-01T01:11:21.488 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2023-12-01 01:05:04.465975514 +0000 2023-12-01T01:11:21.488 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: - 2023-12-01T01:11:21.489 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-01T01:11:21.556 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2023-12-01T01:11:21.556 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2023-12-01T01:11:21.557 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000230698 s, 2.2 MB/s 2023-12-01T01:11:21.558 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-01T01:11:21.616 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_3 2023-12-01T01:11:21.675 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-01T01:11:21.675 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-01T01:11:21.675 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 6h/6d Inode: 393315 Links: 1 2023-12-01T01:11:21.676 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-01T01:11:21.676 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2023-12-01T01:11:21.676 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2023-12-01 01:11:07.701392108 +0000 2023-12-01T01:11:21.676 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2023-12-01 01:05:04.997966147 +0000 2023-12-01T01:11:21.676 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2023-12-01 01:05:04.997966147 +0000 2023-12-01T01:11:21.676 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: - 2023-12-01T01:11:21.676 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-01T01:11:21.741 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2023-12-01T01:11:21.741 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2023-12-01T01:11:21.741 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000241816 s, 2.1 MB/s 2023-12-01T01:11:21.742 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-01T01:11:21.762 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Dec 01 01:11:21 smithi203 systemd[1]: Starting Ceph mgr.z for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:11:21.801 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_4 2023-12-01T01:11:21.858 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-01T01:11:21.858 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-01T01:11:21.859 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 6h/6d Inode: 389607 Links: 1 2023-12-01T01:11:21.859 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-01T01:11:21.859 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2023-12-01T01:11:21.859 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2023-12-01 01:11:07.701392108 +0000 2023-12-01T01:11:21.859 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2023-12-01 01:05:05.564956163 +0000 2023-12-01T01:11:21.859 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2023-12-01 01:05:05.564956163 +0000 2023-12-01T01:11:21.859 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: - 2023-12-01T01:11:21.859 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-01T01:11:21.924 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2023-12-01T01:11:21.924 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2023-12-01T01:11:21.924 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000208888 s, 2.5 MB/s 2023-12-01T01:11:21.925 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-01T01:11:21.983 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2023-12-01T01:11:21.983 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/scratch_devs of=/dev/stdout 2023-12-01T01:11:22.001 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-01T01:11:22.001 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_1 2023-12-01T01:11:22.060 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-01T01:11:22.060 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-01T01:11:22.060 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 6h/6d Inode: 386459 Links: 1 2023-12-01T01:11:22.060 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-01T01:11:22.060 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2023-12-01T01:11:22.060 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2023-12-01 01:11:21.638362951 +0000 2023-12-01T01:11:22.060 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2023-12-01 01:05:15.920035694 +0000 2023-12-01T01:11:22.060 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2023-12-01 01:05:15.920035694 +0000 2023-12-01T01:11:22.060 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: - 2023-12-01T01:11:22.060 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-01T01:11:22.130 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2023-12-01T01:11:22.130 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2023-12-01T01:11:22.130 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000249083 s, 2.1 MB/s 2023-12-01T01:11:22.131 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-01T01:11:22.190 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_2 2023-12-01T01:11:22.247 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-01T01:11:22.247 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-01T01:11:22.247 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 6h/6d Inode: 382976 Links: 1 2023-12-01T01:11:22.247 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-01T01:11:22.247 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2023-12-01T01:11:22.247 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2023-12-01 01:11:21.638362951 +0000 2023-12-01T01:11:22.247 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2023-12-01 01:05:16.464022424 +0000 2023-12-01T01:11:22.248 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2023-12-01 01:05:16.464022424 +0000 2023-12-01T01:11:22.248 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: - 2023-12-01T01:11:22.248 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-01T01:11:22.262 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Dec 01 01:11:22 smithi203 podman[105413]: 2023-12-01T01:11:22.288 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2023-12-01T01:11:22.288 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2023-12-01T01:11:22.289 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000225119 s, 2.3 MB/s 2023-12-01T01:11:22.290 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-01T01:11:22.348 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_3 2023-12-01T01:11:22.405 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-01T01:11:22.405 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-01T01:11:22.405 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 6h/6d Inode: 393257 Links: 1 2023-12-01T01:11:22.406 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-01T01:11:22.406 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2023-12-01T01:11:22.406 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2023-12-01 01:11:21.638362951 +0000 2023-12-01T01:11:22.406 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2023-12-01 01:05:16.995009471 +0000 2023-12-01T01:11:22.406 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2023-12-01 01:05:16.995009471 +0000 2023-12-01T01:11:22.406 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: - 2023-12-01T01:11:22.406 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-01T01:11:22.472 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2023-12-01T01:11:22.472 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2023-12-01T01:11:22.472 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000224035 s, 2.3 MB/s 2023-12-01T01:11:22.473 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-01T01:11:22.533 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_4 2023-12-01T01:11:22.581 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Dec 01 01:11:22 smithi203 bash[105413]: f7dd0598a3fb8e07fa22ef7054e8fff420bce4db5eaf080fb5f585e880c7e79a 2023-12-01T01:11:22.581 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Dec 01 01:11:22 smithi203 ceph-mgr[105553]: -- 172.21.15.203:0/2602907782 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e993ac31e0 con 0x55e993a99000 2023-12-01T01:11:22.581 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Dec 01 01:11:22 smithi203 systemd[1]: Started Ceph mgr.z for 2a06e29a-8fe6-11ee-95a2-87774f69a715. 2023-12-01T01:11:22.582 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Dec 01 01:11:22 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-z[105467]: 2023-12-01T01:11:22.400+0000 7f2bef3a4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T01:11:22.582 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Dec 01 01:11:22 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-z[105467]: 2023-12-01T01:11:22.487+0000 7f2bef3a4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T01:11:22.582 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Dec 01 01:11:22 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-z[105467]: 2023-12-01T01:11:22.580+0000 7f2bef3a4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T01:11:22.591 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-01T01:11:22.591 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-01T01:11:22.591 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 6h/6d Inode: 395390 Links: 1 2023-12-01T01:11:22.591 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-01T01:11:22.591 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2023-12-01T01:11:22.591 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2023-12-01 01:11:21.638362951 +0000 2023-12-01T01:11:22.591 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2023-12-01 01:05:17.511996860 +0000 2023-12-01T01:11:22.591 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2023-12-01 01:05:17.511996860 +0000 2023-12-01T01:11:22.591 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: - 2023-12-01T01:11:22.592 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-01T01:11:22.657 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2023-12-01T01:11:22.657 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2023-12-01T01:11:22.657 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000220123 s, 2.3 MB/s 2023-12-01T01:11:22.658 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-01T01:11:22.716 INFO:tasks.cephadm:Deploying osd.0 on smithi077 with /dev/vg_nvme/lv_4... 2023-12-01T01:11:22.716 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-12-01T01:11:22.855 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Dec 01 01:11:22 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-z[105467]: 2023-12-01T01:11:22.680+0000 7f2bef3a4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T01:11:23.222 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Dec 01 01:11:22 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-z[105467]: 2023-12-01T01:11:22.854+0000 7f2bef3a4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T01:11:23.304 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:11:23.512 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Dec 01 01:11:23 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-z[105467]: 2023-12-01T01:11:23.221+0000 7f2bef3a4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T01:11:23.512 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Dec 01 01:11:23 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-z[105467]: 2023-12-01T01:11:23.352+0000 7f2bef3a4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T01:11:23.512 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Dec 01 01:11:23 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-z[105467]: 2023-12-01T01:11:23.442+0000 7f2bef3a4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T01:11:23.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:23 smithi203 ceph-mon[103581]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:23.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:23 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:23.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:23 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:23.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:23 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:23.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:23 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-01T01:11:23.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:23 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-01T01:11:23.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:23 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-01T01:11:23.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:23 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:23.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:23 smithi203 ceph-mon[103581]: Deploying daemon mgr.y on smithi137 2023-12-01T01:11:23.725 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:23 smithi137 ceph-mon[103312]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:23.725 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:23 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:23.725 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:23 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:23.725 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:23 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:23.725 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:23 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-01T01:11:23.725 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:23 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-01T01:11:23.725 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:23 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-01T01:11:23.725 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:23 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:23.725 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:23 smithi137 ceph-mon[103312]: Deploying daemon mgr.y on smithi137 2023-12-01T01:11:23.761 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:23 smithi077 ceph-mon[98867]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:23.761 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:23 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:23.761 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:23 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:23.761 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:23 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:23.761 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:23 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-01T01:11:23.761 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:23 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-01T01:11:23.761 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:23 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-01T01:11:23.761 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:23 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:23.761 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:23 smithi077 ceph-mon[98867]: Deploying daemon mgr.y on smithi137 2023-12-01T01:11:23.963 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Dec 01 01:11:23 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-z[105467]: 2023-12-01T01:11:23.538+0000 7f2bef3a4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T01:11:24.029 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Dec 01 01:11:23 smithi137 systemd[1]: Starting Ceph mgr.y for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:11:24.262 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Dec 01 01:11:23 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-z[105467]: 2023-12-01T01:11:23.961+0000 7f2bef3a4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T01:11:24.529 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Dec 01 01:11:24 smithi137 podman[104765]: 2023-12-01T01:11:24.529 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Dec 01 01:11:24 smithi137 bash[104765]: 17f5ee33ffc9219c06c667d508a4a801af8b4f4fe7f86627a4f7791f88174cf3 2023-12-01T01:11:24.529 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Dec 01 01:11:24 smithi137 ceph-mgr[104805]: -- 172.21.15.137:0/734457186 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5595a76711e0 con 0x5595a7646c00 2023-12-01T01:11:24.691 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-12-01T01:11:24.691 INFO:teuthology.orchestra.run.smithi077.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-12-01T01:11:24.691 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10+0 records in 2023-12-01T01:11:24.691 INFO:teuthology.orchestra.run.smithi077.stderr:10+0 records out 2023-12-01T01:11:24.691 INFO:teuthology.orchestra.run.smithi077.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114746 s, 914 MB/s 2023-12-01T01:11:24.691 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping successful for: 2023-12-01T01:11:24.790 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Dec 01 01:11:24 smithi137 systemd[1]: Started Ceph mgr.y for 2a06e29a-8fe6-11ee-95a2-87774f69a715. 2023-12-01T01:11:24.790 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Dec 01 01:11:24 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-y[104781]: 2023-12-01T01:11:24.616+0000 7fd00b252200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-01T01:11:24.791 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Dec 01 01:11:24 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-y[104781]: 2023-12-01T01:11:24.696+0000 7fd00b252200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-01T01:11:24.791 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Dec 01 01:11:24 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-y[104781]: 2023-12-01T01:11:24.790+0000 7fd00b252200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-01T01:11:25.012 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Dec 01 01:11:24 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-z[105467]: 2023-12-01T01:11:24.637+0000 7f2bef3a4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T01:11:25.012 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Dec 01 01:11:24 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-z[105467]: 2023-12-01T01:11:24.719+0000 7f2bef3a4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T01:11:25.067 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Dec 01 01:11:24 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-y[104781]: 2023-12-01T01:11:24.890+0000 7fd00b252200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-01T01:11:25.067 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Dec 01 01:11:25 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-y[104781]: 2023-12-01T01:11:25.066+0000 7fd00b252200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-01T01:11:25.238 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi077:vg_nvme/lv_4 2023-12-01T01:11:25.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:25 smithi077 ceph-mon[98867]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:25.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:25 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:25.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:25 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:25.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:25 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:25.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:25 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:25.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:25 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:11:25.762 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Dec 01 01:11:25 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-z[105467]: 2023-12-01T01:11:25.544+0000 7f2bef3a4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:11:25.762 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Dec 01 01:11:25 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-z[105467]: 2023-12-01T01:11:25.636+0000 7f2bef3a4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T01:11:25.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:25 smithi203 ceph-mon[103581]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:25.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:25 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:25.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:25 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:25.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:25 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:25.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:25 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:25.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:25 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:11:25.778 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Dec 01 01:11:25 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-y[104781]: 2023-12-01T01:11:25.449+0000 7fd00b252200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-01T01:11:25.778 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Dec 01 01:11:25 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-y[104781]: 2023-12-01T01:11:25.591+0000 7fd00b252200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-01T01:11:25.779 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Dec 01 01:11:25 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-y[104781]: 2023-12-01T01:11:25.683+0000 7fd00b252200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-01T01:11:25.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:25 smithi137 ceph-mon[103312]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:25.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:25 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:25.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:25 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:25.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:25 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:25.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:25 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:25.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:25 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:11:25.788 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:11:26.116 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Dec 01 01:11:25 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-z[105467]: 2023-12-01T01:11:25.820+0000 7f2bef3a4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:11:26.227 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Dec 01 01:11:25 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-y[104781]: 2023-12-01T01:11:25.781+0000 7fd00b252200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-01T01:11:26.504 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Dec 01 01:11:26 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-z[105467]: 2023-12-01T01:11:26.115+0000 7f2bef3a4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T01:11:26.504 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Dec 01 01:11:26 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-z[105467]: 2023-12-01T01:11:26.349+0000 7f2bef3a4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T01:11:26.528 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Dec 01 01:11:26 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-y[104781]: 2023-12-01T01:11:26.226+0000 7fd00b252200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-01T01:11:26.762 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Dec 01 01:11:26 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-z[105467]: 2023-12-01T01:11:26.503+0000 7f2bef3a4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T01:11:26.999 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:26.998+0000 7f570aeee700 1 -- 172.21.15.77:0/3522683374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f57041018f0 msgr2=0x7f57041081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:11:26.999 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:26.998+0000 7f570aeee700 1 --2- 172.21.15.77:0/3522683374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f57041018f0 0x7f57041081a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f56fc00b0d0 tx=0x7f56fc00b3e0 comp rx=0 tx=0).stop 2023-12-01T01:11:26.999 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:26.998+0000 7f570aeee700 1 -- 172.21.15.77:0/3522683374 shutdown_connections 2023-12-01T01:11:27.000 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:26.998+0000 7f570aeee700 1 --2- 172.21.15.77:0/3522683374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f57041018f0 0x7f57041081a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:11:27.000 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:26.998+0000 7f570aeee700 1 --2- 172.21.15.77:0/3522683374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57040ffd50 0x7f5704100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:11:27.000 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:26.998+0000 7f570aeee700 1 --2- 172.21.15.77:0/3522683374 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5704100f50 0x7f57041013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:11:27.000 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:26.998+0000 7f570aeee700 1 -- 172.21.15.77:0/3522683374 >> 172.21.15.77:0/3522683374 conn(0x7f57040fb310 msgr2=0x7f57040fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:11:27.000 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:26.998+0000 7f570aeee700 1 -- 172.21.15.77:0/3522683374 shutdown_connections 2023-12-01T01:11:27.000 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:26.998+0000 7f570aeee700 1 -- 172.21.15.77:0/3522683374 wait complete. 2023-12-01T01:11:27.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:26.998+0000 7f570aeee700 1 Processor -- start 2023-12-01T01:11:27.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:26.999+0000 7f570aeee700 1 -- start start 2023-12-01T01:11:27.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:26.999+0000 7f570aeee700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f57040ffd50 0x7f5704195ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:11:27.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:26.999+0000 7f570aeee700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5704100f50 0x7f5704196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:11:27.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:26.999+0000 7f570aeee700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57041018f0 0x7f570419d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:11:27.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:26.999+0000 7f570aeee700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570419d670 con 0x7f57040ffd50 2023-12-01T01:11:27.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:26.999+0000 7f570aeee700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570419d7b0 con 0x7f5704100f50 2023-12-01T01:11:27.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:26.999+0000 7f570aeee700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570419d940 con 0x7f57041018f0 2023-12-01T01:11:27.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:26.999+0000 7f5708c8a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f57040ffd50 0x7f5704195ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:11:27.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:26.999+0000 7f5703fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5704100f50 0x7f5704196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:11:27.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:26.999+0000 7f5708c8a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f57040ffd50 0x7f5704195ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56570/0 (socket says 172.21.15.77:56570) 2023-12-01T01:11:27.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:26.999+0000 7f5708c8a700 1 -- 172.21.15.77:0/480783428 learned_addr learned my addr 172.21.15.77:0/480783428 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:11:27.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:26.999+0000 7f570948b700 1 --2- 172.21.15.77:0/480783428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57041018f0 0x7f570419d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:11:27.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:26.999+0000 7f5703fff700 1 -- 172.21.15.77:0/480783428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57041018f0 msgr2=0x7f570419d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:11:27.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:26.999+0000 7f5703fff700 1 --2- 172.21.15.77:0/480783428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57041018f0 0x7f570419d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:11:27.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:26.999+0000 7f5703fff700 1 -- 172.21.15.77:0/480783428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f57040ffd50 msgr2=0x7f5704195ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:11:27.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:26.999+0000 7f5703fff700 1 --2- 172.21.15.77:0/480783428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f57040ffd50 0x7f5704195ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:11:27.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:26.999+0000 7f5703fff700 1 -- 172.21.15.77:0/480783428 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56fc009d00 con 0x7f5704100f50 2023-12-01T01:11:27.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:26.999+0000 7f5708c8a700 1 --2- 172.21.15.77:0/480783428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f57040ffd50 0x7f5704195ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:11:27.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:26.999+0000 7f5703fff700 1 --2- 172.21.15.77:0/480783428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5704100f50 0x7f5704196020 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f56f000d950 tx=0x7f56f000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:11:27.004 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:27.000+0000 7f5701ffb700 1 -- 172.21.15.77:0/480783428 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f000f890 con 0x7f5704100f50 2023-12-01T01:11:27.004 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:27.000+0000 7f5701ffb700 1 -- 172.21.15.77:0/480783428 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56f0010460 con 0x7f5704100f50 2023-12-01T01:11:27.004 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:27.000+0000 7f570aeee700 1 -- 172.21.15.77:0/480783428 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f570419dc20 con 0x7f5704100f50 2023-12-01T01:11:27.004 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:27.000+0000 7f5701ffb700 1 -- 172.21.15.77:0/480783428 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f000e5f0 con 0x7f5704100f50 2023-12-01T01:11:27.004 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:27.000+0000 7f570aeee700 1 -- 172.21.15.77:0/480783428 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f570419e120 con 0x7f5704100f50 2023-12-01T01:11:27.004 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:27.001+0000 7f5701ffb700 1 -- 172.21.15.77:0/480783428 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) v1 ==== 45324+0+0 (secure 0 0 0) 0x7f56f000e750 con 0x7f5704100f50 2023-12-01T01:11:27.004 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:27.001+0000 7f5701ffb700 1 --2- 172.21.15.77:0/480783428 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f56f40385d0 0x7f56f403aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:11:27.004 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:27.001+0000 7f5701ffb700 1 -- 172.21.15.77:0/480783428 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f56f004a9e0 con 0x7f5704100f50 2023-12-01T01:11:27.004 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:27.001+0000 7f570aeee700 1 -- 172.21.15.77:0/480783428 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57041072a0 con 0x7f5704100f50 2023-12-01T01:11:27.004 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:27.001+0000 7f5708c8a700 1 --2- 172.21.15.77:0/480783428 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f56f40385d0 0x7f56f403aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:11:27.005 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:27.001+0000 7f5708c8a700 1 --2- 172.21.15.77:0/480783428 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f56f40385d0 0x7f56f403aa90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f56f8007fd0 tx=0x7f56f8007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:11:27.007 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:27.006+0000 7f5701ffb700 1 -- 172.21.15.77:0/480783428 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f56f0016070 con 0x7f5704100f50 2023-12-01T01:11:27.246 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:27.216+0000 7f570aeee700 1 -- 172.21.15.77:0/480783428 --> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f5704061290 con 0x7f56f40385d0 2023-12-01T01:11:27.262 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Dec 01 01:11:26 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-z[105467]: 2023-12-01T01:11:26.766+0000 7f2bef3a4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T01:11:27.274 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Dec 01 01:11:26 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-y[104781]: 2023-12-01T01:11:26.901+0000 7fd00b252200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-01T01:11:27.274 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Dec 01 01:11:26 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-y[104781]: 2023-12-01T01:11:26.982+0000 7fd00b252200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-01T01:11:27.351 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:27 smithi077 ceph-mon[98867]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:27.351 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:27 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:27.351 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:27 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:27.351 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:27 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:27.352 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:27 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:27.352 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:27 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:27.352 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:27 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:11:27.352 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:27 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:27.352 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:27 smithi077 ceph-mon[98867]: Reconfiguring mgr.x (unknown last config time)... 2023-12-01T01:11:27.352 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:27 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-01T01:11:27.352 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:27 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-01T01:11:27.352 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:27 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:27.352 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:27 smithi077 ceph-mon[98867]: Reconfiguring daemon mgr.x on smithi077 2023-12-01T01:11:27.352 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:27 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-01T01:11:27.352 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:27 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-01T01:11:27.352 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:27 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:27.530 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:27 smithi137 ceph-mon[103312]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:27.530 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:27 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:27.530 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:27 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:27.530 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:27 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:27.530 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:27 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:27.530 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:27 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:27.530 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:27 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:11:27.530 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:27 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:27.531 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:27 smithi137 ceph-mon[103312]: Reconfiguring mgr.x (unknown last config time)... 2023-12-01T01:11:27.531 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:27 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-01T01:11:27.531 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:27 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-01T01:11:27.531 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:27 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:27.531 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:27 smithi137 ceph-mon[103312]: Reconfiguring daemon mgr.x on smithi077 2023-12-01T01:11:27.531 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:27 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-01T01:11:27.531 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:27 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-01T01:11:27.531 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:27 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:27.608 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Dec 01 01:11:27 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-z[105467]: 2023-12-01T01:11:27.474+0000 7f2bef3a4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T01:11:27.608 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:27 smithi203 ceph-mon[103581]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:27.608 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:27 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:27.609 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:27 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:27.609 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:27 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:27.609 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:27 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:27.609 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:27 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:27.609 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:27 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:11:27.609 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:27 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:27.609 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:27 smithi203 ceph-mon[103581]: Reconfiguring mgr.x (unknown last config time)... 2023-12-01T01:11:27.609 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:27 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-01T01:11:27.609 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:27 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-01T01:11:27.609 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:27 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:27.609 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:27 smithi203 ceph-mon[103581]: Reconfiguring daemon mgr.x on smithi077 2023-12-01T01:11:27.609 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:27 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-01T01:11:27.609 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:27 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-01T01:11:27.609 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:27 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:27.915 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Dec 01 01:11:27 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-z[105467]: 2023-12-01T01:11:27.607+0000 7f2bef3a4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T01:11:27.915 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Dec 01 01:11:27 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-z[105467]: 2023-12-01T01:11:27.800+0000 7f2bef3a4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T01:11:28.013 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Dec 01 01:11:27 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-y[104781]: 2023-12-01T01:11:27.751+0000 7fd00b252200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:11:28.013 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Dec 01 01:11:27 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-y[104781]: 2023-12-01T01:11:27.842+0000 7fd00b252200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-01T01:11:28.262 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Dec 01 01:11:27 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-z[105467]: 2023-12-01T01:11:27.914+0000 7f2bef3a4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T01:11:28.262 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Dec 01 01:11:28 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-z[105467]: 2023-12-01T01:11:28.134+0000 7f2bef3a4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T01:11:28.275 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Dec 01 01:11:28 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-y[104781]: 2023-12-01T01:11:28.013+0000 7fd00b252200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-01T01:11:28.368 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:28 smithi077 ceph-mon[98867]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:11:28.368 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:28 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:28.368 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:28 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:28.368 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:28 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:11:28.368 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:28 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:28.368 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:28 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:11:28.368 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:28 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:28.529 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Dec 01 01:11:28 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-y[104781]: 2023-12-01T01:11:28.295+0000 7fd00b252200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-01T01:11:28.529 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Dec 01 01:11:28 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-y[104781]: 2023-12-01T01:11:28.521+0000 7fd00b252200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-01T01:11:28.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:28 smithi137 ceph-mon[103312]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:11:28.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:28 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:28.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:28 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:28.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:28 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:11:28.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:28 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:28.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:28 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:11:28.530 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:28 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:28.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:28 smithi203 ceph-mon[103581]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:11:28.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:28 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:28.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:28 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:28.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:28 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:11:28.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:28 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:28.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:28 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:11:28.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:28 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:28.929 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Dec 01 01:11:28 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-y[104781]: 2023-12-01T01:11:28.669+0000 7fd00b252200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-01T01:11:29.262 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Dec 01 01:11:29 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-z[105467]: 2023-12-01T01:11:29.001+0000 7f2bef3a4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T01:11:29.278 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Dec 01 01:11:28 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-y[104781]: 2023-12-01T01:11:28.929+0000 7fd00b252200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-01T01:11:29.282 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:29.280+0000 7f5701ffb700 1 -- 172.21.15.77:0/480783428 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mgrmap(e 14) v1 ==== 90169+0+0 (secure 0 0 0) 0x7f56f00105d0 con 0x7f5704100f50 2023-12-01T01:11:29.632 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:29 smithi137 ceph-mon[103312]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:29.633 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:29 smithi137 ceph-mon[103312]: Standby manager daemon z started 2023-12-01T01:11:29.633 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:29 smithi137 ceph-mon[103312]: from='mgr.? 172.21.15.203:0/370192160' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-12-01T01:11:29.633 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:29 smithi137 ceph-mon[103312]: from='mgr.? 172.21.15.203:0/370192160' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-01T01:11:29.633 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:29 smithi137 ceph-mon[103312]: from='mgr.? 172.21.15.203:0/370192160' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-12-01T01:11:29.633 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:29 smithi137 ceph-mon[103312]: from='mgr.? 172.21.15.203:0/370192160' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-01T01:11:29.633 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:29 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:29.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:29 smithi077 ceph-mon[98867]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:29.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:29 smithi077 ceph-mon[98867]: Standby manager daemon z started 2023-12-01T01:11:29.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:29 smithi077 ceph-mon[98867]: from='mgr.? 172.21.15.203:0/370192160' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-12-01T01:11:29.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:29 smithi077 ceph-mon[98867]: from='mgr.? 172.21.15.203:0/370192160' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-01T01:11:29.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:29 smithi077 ceph-mon[98867]: from='mgr.? 172.21.15.203:0/370192160' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-12-01T01:11:29.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:29 smithi077 ceph-mon[98867]: from='mgr.? 172.21.15.203:0/370192160' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-01T01:11:29.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:29 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:29.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:29 smithi203 ceph-mon[103581]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:29.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:29 smithi203 ceph-mon[103581]: Standby manager daemon z started 2023-12-01T01:11:29.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:29 smithi203 ceph-mon[103581]: from='mgr.? 172.21.15.203:0/370192160' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-12-01T01:11:29.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:29 smithi203 ceph-mon[103581]: from='mgr.? 172.21.15.203:0/370192160' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-01T01:11:29.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:29 smithi203 ceph-mon[103581]: from='mgr.? 172.21.15.203:0/370192160' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-12-01T01:11:29.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:29 smithi203 ceph-mon[103581]: from='mgr.? 172.21.15.203:0/370192160' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-01T01:11:29.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:29 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:29.954 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Dec 01 01:11:29 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-y[104781]: 2023-12-01T01:11:29.632+0000 7fd00b252200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-01T01:11:29.954 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Dec 01 01:11:29 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-y[104781]: 2023-12-01T01:11:29.761+0000 7fd00b252200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-01T01:11:30.278 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Dec 01 01:11:29 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-y[104781]: 2023-12-01T01:11:29.952+0000 7fd00b252200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-01T01:11:30.279 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Dec 01 01:11:30 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-y[104781]: 2023-12-01T01:11:30.066+0000 7fd00b252200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-01T01:11:30.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:30 smithi077 ceph-mon[98867]: mgrmap e14: x(active, since 65s), standbys: z 2023-12-01T01:11:30.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:30 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-12-01T01:11:30.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:30 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/2587287304' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc651e62-a103-4ab8-a76d-d3708ea6aeec"}]: dispatch 2023-12-01T01:11:30.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:30 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/2587287304' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc651e62-a103-4ab8-a76d-d3708ea6aeec"}]': finished 2023-12-01T01:11:30.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:30 smithi077 ceph-mon[98867]: osdmap e5: 1 total, 0 up, 1 in 2023-12-01T01:11:30.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:30 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-01T01:11:30.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:30 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/4047743585' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-01T01:11:30.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:30 smithi203 ceph-mon[103581]: mgrmap e14: x(active, since 65s), standbys: z 2023-12-01T01:11:30.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:30 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-12-01T01:11:30.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:30 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/2587287304' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc651e62-a103-4ab8-a76d-d3708ea6aeec"}]: dispatch 2023-12-01T01:11:30.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:30 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/2587287304' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc651e62-a103-4ab8-a76d-d3708ea6aeec"}]': finished 2023-12-01T01:11:30.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:30 smithi203 ceph-mon[103581]: osdmap e5: 1 total, 0 up, 1 in 2023-12-01T01:11:30.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:30 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-01T01:11:30.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:30 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/4047743585' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-01T01:11:30.779 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Dec 01 01:11:30 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-y[104781]: 2023-12-01T01:11:30.284+0000 7fd00b252200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-01T01:11:30.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:30 smithi137 ceph-mon[103312]: mgrmap e14: x(active, since 65s), standbys: z 2023-12-01T01:11:30.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:30 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-12-01T01:11:30.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:30 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/2587287304' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc651e62-a103-4ab8-a76d-d3708ea6aeec"}]: dispatch 2023-12-01T01:11:30.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:30 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/2587287304' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc651e62-a103-4ab8-a76d-d3708ea6aeec"}]': finished 2023-12-01T01:11:30.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:30 smithi137 ceph-mon[103312]: osdmap e5: 1 total, 0 up, 1 in 2023-12-01T01:11:30.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:30 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-01T01:11:30.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:30 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/4047743585' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-01T01:11:31.529 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Dec 01 01:11:31 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-y[104781]: 2023-12-01T01:11:31.142+0000 7fd00b252200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-01T01:11:31.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:31 smithi137 ceph-mon[103312]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:31.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:31 smithi137 ceph-mon[103312]: Standby manager daemon y started 2023-12-01T01:11:31.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:31 smithi137 ceph-mon[103312]: from='mgr.? 172.21.15.137:0/236081960' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-12-01T01:11:31.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:31 smithi137 ceph-mon[103312]: from='mgr.? 172.21.15.137:0/236081960' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-01T01:11:31.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:31 smithi137 ceph-mon[103312]: from='mgr.? 172.21.15.137:0/236081960' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-12-01T01:11:31.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:31 smithi137 ceph-mon[103312]: from='mgr.? 172.21.15.137:0/236081960' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-01T01:11:31.644 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:31.642+0000 7f5701ffb700 1 -- 172.21.15.77:0/480783428 <== mon.2 v2:172.21.15.137:3300/0 8 ==== mgrmap(e 15) v1 ==== 135014+0+0 (secure 0 0 0) 0x7f56f0019360 con 0x7f5704100f50 2023-12-01T01:11:31.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:31 smithi077 ceph-mon[98867]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:31.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:31 smithi077 ceph-mon[98867]: Standby manager daemon y started 2023-12-01T01:11:31.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:31 smithi077 ceph-mon[98867]: from='mgr.? 172.21.15.137:0/236081960' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-12-01T01:11:31.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:31 smithi077 ceph-mon[98867]: from='mgr.? 172.21.15.137:0/236081960' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-01T01:11:31.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:31 smithi077 ceph-mon[98867]: from='mgr.? 172.21.15.137:0/236081960' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-12-01T01:11:31.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:31 smithi077 ceph-mon[98867]: from='mgr.? 172.21.15.137:0/236081960' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-01T01:11:31.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:31 smithi203 ceph-mon[103581]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:31.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:31 smithi203 ceph-mon[103581]: Standby manager daemon y started 2023-12-01T01:11:31.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:31 smithi203 ceph-mon[103581]: from='mgr.? 172.21.15.137:0/236081960' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-12-01T01:11:31.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:31 smithi203 ceph-mon[103581]: from='mgr.? 172.21.15.137:0/236081960' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-01T01:11:31.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:31 smithi203 ceph-mon[103581]: from='mgr.? 172.21.15.137:0/236081960' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-12-01T01:11:31.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:31 smithi203 ceph-mon[103581]: from='mgr.? 172.21.15.137:0/236081960' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-01T01:11:32.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:32 smithi077 ceph-mon[98867]: mgrmap e15: x(active, since 67s), standbys: y, z 2023-12-01T01:11:32.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:32 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-01T01:11:32.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:32 smithi077 ceph-mon[98867]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:33.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:32 smithi203 ceph-mon[103581]: mgrmap e15: x(active, since 67s), standbys: y, z 2023-12-01T01:11:33.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:32 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-01T01:11:33.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:32 smithi203 ceph-mon[103581]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:33.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:32 smithi137 ceph-mon[103312]: mgrmap e15: x(active, since 67s), standbys: y, z 2023-12-01T01:11:33.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:32 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-01T01:11:33.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:32 smithi137 ceph-mon[103312]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:35.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:35 smithi077 ceph-mon[98867]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:35.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:35 smithi203 ceph-mon[103581]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:35.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:35 smithi137 ceph-mon[103312]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:37.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:37 smithi137 ceph-mon[103312]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:37.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:37 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-12-01T01:11:37.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:37 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:37.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:37 smithi137 ceph-mon[103312]: Deploying daemon osd.0 on smithi077 2023-12-01T01:11:37.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:37 smithi077 ceph-mon[98867]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:37.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:37 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-12-01T01:11:37.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:37 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:37.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:37 smithi077 ceph-mon[98867]: Deploying daemon osd.0 on smithi077 2023-12-01T01:11:37.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:37 smithi203 ceph-mon[103581]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:37.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:37 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-12-01T01:11:37.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:37 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:37.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:37 smithi203 ceph-mon[103581]: Deploying daemon osd.0 on smithi077 2023-12-01T01:11:39.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:39 smithi077 ceph-mon[98867]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:39.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:39 smithi137 ceph-mon[103312]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:39.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:39 smithi203 ceph-mon[103581]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:41.533 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:41 smithi077 ceph-mon[98867]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:41.533 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:41 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:41.533 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:41 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:41.533 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:41 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:11:41.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:41 smithi203 ceph-mon[103581]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:41.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:41 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:41.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:41 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:41.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:41 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:11:41.778 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:41 smithi137 ceph-mon[103312]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:41.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:41 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:41.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:41 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:41.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:41 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:11:43.068 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:43.065+0000 7f5701ffb700 1 -- 172.21.15.77:0/480783428 <== mgr.14152 v2:172.21.15.77:6800/3838941679 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5704061290 con 0x7f56f40385d0 2023-12-01T01:11:43.068 INFO:teuthology.orchestra.run.smithi077.stdout:Created osd(s) 0 on host 'smithi077' 2023-12-01T01:11:43.070 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:43.068+0000 7f570aeee700 1 -- 172.21.15.77:0/480783428 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f56f40385d0 msgr2=0x7f56f403aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:11:43.070 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:43.068+0000 7f570aeee700 1 --2- 172.21.15.77:0/480783428 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f56f40385d0 0x7f56f403aa90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f56f8007fd0 tx=0x7f56f8007480 comp rx=0 tx=0).stop 2023-12-01T01:11:43.070 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:43.068+0000 7f570aeee700 1 -- 172.21.15.77:0/480783428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5704100f50 msgr2=0x7f5704196020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:11:43.070 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:43.068+0000 7f570aeee700 1 --2- 172.21.15.77:0/480783428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5704100f50 0x7f5704196020 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f56f000d950 tx=0x7f56f000dc60 comp rx=0 tx=0).stop 2023-12-01T01:11:43.070 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:43.068+0000 7f570aeee700 1 -- 172.21.15.77:0/480783428 shutdown_connections 2023-12-01T01:11:43.070 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:43.068+0000 7f570aeee700 1 --2- 172.21.15.77:0/480783428 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f56f40385d0 0x7f56f403aa90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:11:43.071 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:43.068+0000 7f570aeee700 1 --2- 172.21.15.77:0/480783428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57041018f0 0x7f570419d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:11:43.071 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:43.068+0000 7f570aeee700 1 --2- 172.21.15.77:0/480783428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f57040ffd50 0x7f5704195ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:11:43.071 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:43.068+0000 7f570aeee700 1 --2- 172.21.15.77:0/480783428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5704100f50 0x7f5704196020 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:11:43.071 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:43.068+0000 7f570aeee700 1 -- 172.21.15.77:0/480783428 >> 172.21.15.77:0/480783428 conn(0x7f57040fb310 msgr2=0x7f57041043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:11:43.071 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:43.068+0000 7f570aeee700 1 -- 172.21.15.77:0/480783428 shutdown_connections 2023-12-01T01:11:43.071 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:11:43.068+0000 7f570aeee700 1 -- 172.21.15.77:0/480783428 wait complete. 2023-12-01T01:11:43.481 DEBUG:teuthology.orchestra.run.smithi077:osd.0> sudo journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@osd.0.service 2023-12-01T01:11:43.483 INFO:tasks.cephadm:Deploying osd.1 on smithi137 with /dev/vg_nvme/lv_4... 2023-12-01T01:11:43.483 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-12-01T01:11:43.513 INFO:journalctl@ceph.osd.0.smithi077.stdout:-- Logs begin at Fri 2023-12-01 00:55:13 UTC. -- 2023-12-01T01:11:43.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:43 smithi203 ceph-mon[103581]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:43.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:43 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:43.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:43 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:43.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:43 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:43.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:43 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:11:43.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:43 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:43.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:43 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:43.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:43 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:43.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:43 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:11:43.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:43 smithi137 ceph-mon[103312]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:43.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:43 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:43.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:43 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:43.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:43 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:43.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:43 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:11:43.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:43 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:43.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:43 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:43.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:43 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:43.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:43 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:11:43.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:43 smithi077 ceph-mon[98867]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:43.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:43 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:43.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:43 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:43.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:43 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:43.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:43 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:11:43.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:43 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:43.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:43 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:43.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:43 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:43.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:43 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:11:44.043 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.b/config 2023-12-01T01:11:44.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:44 smithi137 ceph-mon[103312]: from='osd.0 [v2:172.21.15.77:6802/2244799094,v1:172.21.15.77:6803/2244799094]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-12-01T01:11:44.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:44 smithi077 ceph-mon[98867]: from='osd.0 [v2:172.21.15.77:6802/2244799094,v1:172.21.15.77:6803/2244799094]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-12-01T01:11:44.654 INFO:journalctl@ceph.osd.0.smithi077.stdout:Dec 01 01:11:44 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-0[117495]: 2023-12-01T01:11:44.390+0000 7fdca152a640 -1 osd.0 0 log_to_monitors true 2023-12-01T01:11:44.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:44 smithi203 ceph-mon[103581]: from='osd.0 [v2:172.21.15.77:6802/2244799094,v1:172.21.15.77:6803/2244799094]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-12-01T01:11:45.552 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-12-01T01:11:45.552 INFO:teuthology.orchestra.run.smithi137.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-12-01T01:11:45.552 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10+0 records in 2023-12-01T01:11:45.552 INFO:teuthology.orchestra.run.smithi137.stderr:10+0 records out 2023-12-01T01:11:45.552 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110505 s, 949 MB/s 2023-12-01T01:11:45.552 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping successful for: 2023-12-01T01:11:45.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:45 smithi203 ceph-mon[103581]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:45.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:45 smithi203 ceph-mon[103581]: from='osd.0 [v2:172.21.15.77:6802/2244799094,v1:172.21.15.77:6803/2244799094]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-12-01T01:11:45.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:45 smithi203 ceph-mon[103581]: osdmap e6: 1 total, 0 up, 1 in 2023-12-01T01:11:45.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:45 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-01T01:11:45.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:45 smithi203 ceph-mon[103581]: from='osd.0 [v2:172.21.15.77:6802/2244799094,v1:172.21.15.77:6803/2244799094]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2023-12-01T01:11:45.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:45 smithi137 ceph-mon[103312]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:45.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:45 smithi137 ceph-mon[103312]: from='osd.0 [v2:172.21.15.77:6802/2244799094,v1:172.21.15.77:6803/2244799094]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-12-01T01:11:45.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:45 smithi137 ceph-mon[103312]: osdmap e6: 1 total, 0 up, 1 in 2023-12-01T01:11:45.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:45 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-01T01:11:45.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:45 smithi137 ceph-mon[103312]: from='osd.0 [v2:172.21.15.77:6802/2244799094,v1:172.21.15.77:6803/2244799094]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2023-12-01T01:11:45.782 INFO:journalctl@ceph.osd.0.smithi077.stdout:Dec 01 01:11:45 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-0[117495]: 2023-12-01T01:11:45.479+0000 7fdc963a0700 -1 osd.0 0 waiting for initial osdmap 2023-12-01T01:11:45.783 INFO:journalctl@ceph.osd.0.smithi077.stdout:Dec 01 01:11:45 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-0[117495]: 2023-12-01T01:11:45.486+0000 7fdc91193700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-01T01:11:45.783 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:45 smithi077 ceph-mon[98867]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:45.783 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:45 smithi077 ceph-mon[98867]: from='osd.0 [v2:172.21.15.77:6802/2244799094,v1:172.21.15.77:6803/2244799094]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-12-01T01:11:45.783 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:45 smithi077 ceph-mon[98867]: osdmap e6: 1 total, 0 up, 1 in 2023-12-01T01:11:45.783 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:45 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-01T01:11:45.783 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:45 smithi077 ceph-mon[98867]: from='osd.0 [v2:172.21.15.77:6802/2244799094,v1:172.21.15.77:6803/2244799094]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2023-12-01T01:11:45.997 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi137:vg_nvme/lv_4 2023-12-01T01:11:46.556 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.b/config 2023-12-01T01:11:46.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:46 smithi203 ceph-mon[103581]: from='osd.0 [v2:172.21.15.77:6802/2244799094,v1:172.21.15.77:6803/2244799094]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2023-12-01T01:11:46.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:46 smithi203 ceph-mon[103581]: osdmap e7: 1 total, 0 up, 1 in 2023-12-01T01:11:46.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:46 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-01T01:11:46.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:46 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-01T01:11:46.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:46 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:46.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:46 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:46.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:46 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:11:46.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:46 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:46.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:46 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:46.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:46 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:11:46.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:46 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:46.769 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:46 smithi137 ceph-mon[103312]: from='osd.0 [v2:172.21.15.77:6802/2244799094,v1:172.21.15.77:6803/2244799094]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2023-12-01T01:11:46.769 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:46 smithi137 ceph-mon[103312]: osdmap e7: 1 total, 0 up, 1 in 2023-12-01T01:11:46.769 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:46 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-01T01:11:46.769 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:46 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-01T01:11:46.769 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:46 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:46.769 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:46 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:46.769 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:46 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:11:46.769 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:46 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:46.769 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:46 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:46.769 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:46 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:11:46.769 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:46 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:46.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:46 smithi077 ceph-mon[98867]: from='osd.0 [v2:172.21.15.77:6802/2244799094,v1:172.21.15.77:6803/2244799094]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2023-12-01T01:11:46.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:46 smithi077 ceph-mon[98867]: osdmap e7: 1 total, 0 up, 1 in 2023-12-01T01:11:46.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:46 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-01T01:11:46.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:46 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-01T01:11:46.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:46 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:46.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:46 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:46.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:46 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:11:46.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:46 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:46.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:46 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:46.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:46 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:11:46.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:46 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:11:47.713 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:11:47.711+0000 7f3228bb6700 1 -- 172.21.15.137:0/3869130722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32241018f0 msgr2=0x7f32241081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:11:47.713 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:11:47.711+0000 7f3228bb6700 1 --2- 172.21.15.137:0/3869130722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32241018f0 0x7f32241081a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f321c00b0d0 tx=0x7f321c00b3e0 comp rx=0 tx=0).stop 2023-12-01T01:11:47.713 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:11:47.711+0000 7f3228bb6700 1 -- 172.21.15.137:0/3869130722 shutdown_connections 2023-12-01T01:11:47.713 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:11:47.711+0000 7f3228bb6700 1 --2- 172.21.15.137:0/3869130722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32241018f0 0x7f32241081a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:11:47.713 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:11:47.711+0000 7f3228bb6700 1 --2- 172.21.15.137:0/3869130722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f32240ffd50 0x7f3224100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:11:47.713 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:11:47.712+0000 7f3228bb6700 1 --2- 172.21.15.137:0/3869130722 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3224100f50 0x7f32241013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:11:47.713 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:11:47.712+0000 7f3228bb6700 1 -- 172.21.15.137:0/3869130722 >> 172.21.15.137:0/3869130722 conn(0x7f32240fb310 msgr2=0x7f32240fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:11:47.714 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:11:47.712+0000 7f3228bb6700 1 -- 172.21.15.137:0/3869130722 shutdown_connections 2023-12-01T01:11:47.714 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:11:47.712+0000 7f3228bb6700 1 -- 172.21.15.137:0/3869130722 wait complete. 2023-12-01T01:11:47.715 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:11:47.712+0000 7f3228bb6700 1 Processor -- start 2023-12-01T01:11:47.715 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:11:47.712+0000 7f3228bb6700 1 -- start start 2023-12-01T01:11:47.715 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:11:47.712+0000 7f3228bb6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f32240ffd50 0x7f3224069cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:11:47.716 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:11:47.712+0000 7f3228bb6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3224100f50 0x7f322419b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:11:47.716 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:11:47.712+0000 7f3228bb6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32241018f0 0x7f322419e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:11:47.716 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:11:47.712+0000 7f3228bb6700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322419e7a0 con 0x7f32240ffd50 2023-12-01T01:11:47.716 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:11:47.712+0000 7f3228bb6700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322419e910 con 0x7f3224100f50 2023-12-01T01:11:47.716 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:11:47.712+0000 7f3228bb6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322419ead0 con 0x7f32241018f0 2023-12-01T01:11:47.716 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:11:47.713+0000 7f3221d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3224100f50 0x7f322419b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:11:47.716 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:11:47.713+0000 7f3221d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3224100f50 0x7f322419b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37204/0 (socket says 172.21.15.137:37204) 2023-12-01T01:11:47.716 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:11:47.713+0000 7f3221d9b700 1 -- 172.21.15.137:0/3220909775 learned_addr learned my addr 172.21.15.137:0/3220909775 (peer_addr_for_me v2:172.21.15.137:0/0) 2023-12-01T01:11:47.716 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:11:47.713+0000 7f322259c700 1 --2- 172.21.15.137:0/3220909775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f32240ffd50 0x7f3224069cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:11:47.716 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:11:47.713+0000 7f3222d9d700 1 --2- 172.21.15.137:0/3220909775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32241018f0 0x7f322419e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:11:47.716 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:11:47.713+0000 7f3221d9b700 1 -- 172.21.15.137:0/3220909775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32241018f0 msgr2=0x7f322419e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:11:47.716 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:11:47.713+0000 7f3221d9b700 1 --2- 172.21.15.137:0/3220909775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32241018f0 0x7f322419e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:11:47.716 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:11:47.713+0000 7f3221d9b700 1 -- 172.21.15.137:0/3220909775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f32240ffd50 msgr2=0x7f3224069cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:11:47.716 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:11:47.713+0000 7f3221d9b700 1 --2- 172.21.15.137:0/3220909775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f32240ffd50 0x7f3224069cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:11:47.717 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:11:47.713+0000 7f3221d9b700 1 -- 172.21.15.137:0/3220909775 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f321c009d00 con 0x7f3224100f50 2023-12-01T01:11:47.717 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:11:47.713+0000 7f3221d9b700 1 --2- 172.21.15.137:0/3220909775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3224100f50 0x7f322419b9b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f321400d950 tx=0x7f321400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:11:47.717 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:11:47.713+0000 7f32137fe700 1 -- 172.21.15.137:0/3220909775 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321400f890 con 0x7f3224100f50 2023-12-01T01:11:47.717 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:11:47.713+0000 7f3228bb6700 1 -- 172.21.15.137:0/3220909775 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f322419edb0 con 0x7f3224100f50 2023-12-01T01:11:47.717 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:11:47.713+0000 7f32137fe700 1 -- 172.21.15.137:0/3220909775 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3214010460 con 0x7f3224100f50 2023-12-01T01:11:47.717 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:11:47.713+0000 7f32137fe700 1 -- 172.21.15.137:0/3220909775 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321400e610 con 0x7f3224100f50 2023-12-01T01:11:47.717 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:11:47.713+0000 7f3228bb6700 1 -- 172.21.15.137:0/3220909775 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f322419f2a0 con 0x7f3224100f50 2023-12-01T01:11:47.718 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:11:47.714+0000 7f3228bb6700 1 -- 172.21.15.137:0/3220909775 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3224107290 con 0x7f3224100f50 2023-12-01T01:11:47.718 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:11:47.715+0000 7f32137fe700 1 -- 172.21.15.137:0/3220909775 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 15) v1 ==== 135014+0+0 (secure 0 0 0) 0x7f32140108d0 con 0x7f3224100f50 2023-12-01T01:11:47.718 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:11:47.716+0000 7f32137fe700 1 --2- 172.21.15.137:0/3220909775 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f320c0a0490 0x7f320c0a2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:11:47.718 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:11:47.716+0000 7f32137fe700 1 -- 172.21.15.137:0/3220909775 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f32140ca010 con 0x7f3224100f50 2023-12-01T01:11:47.718 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:11:47.716+0000 7f322259c700 1 --2- 172.21.15.137:0/3220909775 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f320c0a0490 0x7f320c0a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:11:47.718 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:11:47.717+0000 7f322259c700 1 --2- 172.21.15.137:0/3220909775 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f320c0a0490 0x7f320c0a2950 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3224100db0 tx=0x7f3218009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:11:47.722 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:11:47.720+0000 7f32137fe700 1 -- 172.21.15.137:0/3220909775 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3214095620 con 0x7f3224100f50 2023-12-01T01:11:47.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:47 smithi203 ceph-mon[103581]: purged_snaps scrub starts 2023-12-01T01:11:47.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:47 smithi203 ceph-mon[103581]: purged_snaps scrub ok 2023-12-01T01:11:47.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:47 smithi203 ceph-mon[103581]: Detected new or changed devices on smithi077 2023-12-01T01:11:47.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:47 smithi203 ceph-mon[103581]: Adjusting osd_memory_target on smithi077 to 17168M 2023-12-01T01:11:47.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:47 smithi203 ceph-mon[103581]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:47.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:47 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-01T01:11:47.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:47 smithi203 ceph-mon[103581]: osd.0 [v2:172.21.15.77:6802/2244799094,v1:172.21.15.77:6803/2244799094] boot 2023-12-01T01:11:47.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:47 smithi203 ceph-mon[103581]: osdmap e8: 1 total, 1 up, 1 in 2023-12-01T01:11:47.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:47 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-01T01:11:47.778 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:47 smithi137 ceph-mon[103312]: purged_snaps scrub starts 2023-12-01T01:11:47.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:47 smithi137 ceph-mon[103312]: purged_snaps scrub ok 2023-12-01T01:11:47.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:47 smithi137 ceph-mon[103312]: Detected new or changed devices on smithi077 2023-12-01T01:11:47.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:47 smithi137 ceph-mon[103312]: Adjusting osd_memory_target on smithi077 to 17168M 2023-12-01T01:11:47.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:47 smithi137 ceph-mon[103312]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:47.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:47 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-01T01:11:47.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:47 smithi137 ceph-mon[103312]: osd.0 [v2:172.21.15.77:6802/2244799094,v1:172.21.15.77:6803/2244799094] boot 2023-12-01T01:11:47.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:47 smithi137 ceph-mon[103312]: osdmap e8: 1 total, 1 up, 1 in 2023-12-01T01:11:47.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:47 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-01T01:11:47.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:47 smithi077 ceph-mon[98867]: purged_snaps scrub starts 2023-12-01T01:11:47.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:47 smithi077 ceph-mon[98867]: purged_snaps scrub ok 2023-12-01T01:11:47.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:47 smithi077 ceph-mon[98867]: Detected new or changed devices on smithi077 2023-12-01T01:11:47.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:47 smithi077 ceph-mon[98867]: Adjusting osd_memory_target on smithi077 to 17168M 2023-12-01T01:11:47.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:47 smithi077 ceph-mon[98867]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-01T01:11:47.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:47 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-01T01:11:47.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:47 smithi077 ceph-mon[98867]: osd.0 [v2:172.21.15.77:6802/2244799094,v1:172.21.15.77:6803/2244799094] boot 2023-12-01T01:11:47.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:47 smithi077 ceph-mon[98867]: osdmap e8: 1 total, 1 up, 1 in 2023-12-01T01:11:47.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:47 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-01T01:11:47.921 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:11:47.919+0000 7f3228bb6700 1 -- 172.21.15.137:0/3220909775 --> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f32240611d0 con 0x7f320c0a0490 2023-12-01T01:11:48.748 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:48 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-01T01:11:48.749 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:48 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-01T01:11:48.749 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:48 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:48.749 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:48 smithi137 ceph-mon[103312]: osdmap e9: 1 total, 1 up, 1 in 2023-12-01T01:11:48.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:48 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-01T01:11:48.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:48 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-01T01:11:48.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:48 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:48.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:48 smithi203 ceph-mon[103581]: osdmap e9: 1 total, 1 up, 1 in 2023-12-01T01:11:48.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:48 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-01T01:11:48.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:48 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-01T01:11:48.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:48 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:48.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:48 smithi077 ceph-mon[98867]: osdmap e9: 1 total, 1 up, 1 in 2023-12-01T01:11:49.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:49 smithi203 ceph-mon[103581]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:11:49.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:49 smithi203 ceph-mon[103581]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-01T01:11:49.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:49 smithi137 ceph-mon[103312]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:11:49.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:49 smithi137 ceph-mon[103312]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-01T01:11:49.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:49 smithi077 ceph-mon[98867]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:11:49.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:49 smithi077 ceph-mon[98867]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-01T01:11:50.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:50 smithi203 ceph-mon[103581]: from='client.? 172.21.15.137:0/1482903968' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0adf8c54-49ec-49a0-8bf1-af1f20e60729"}]: dispatch 2023-12-01T01:11:50.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:50 smithi203 ceph-mon[103581]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0adf8c54-49ec-49a0-8bf1-af1f20e60729"}]: dispatch 2023-12-01T01:11:50.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:50 smithi203 ceph-mon[103581]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0adf8c54-49ec-49a0-8bf1-af1f20e60729"}]': finished 2023-12-01T01:11:50.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:50 smithi203 ceph-mon[103581]: osdmap e10: 2 total, 1 up, 2 in 2023-12-01T01:11:50.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:50 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-01T01:11:50.778 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:50 smithi137 ceph-mon[103312]: from='client.? 172.21.15.137:0/1482903968' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0adf8c54-49ec-49a0-8bf1-af1f20e60729"}]: dispatch 2023-12-01T01:11:50.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:50 smithi137 ceph-mon[103312]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0adf8c54-49ec-49a0-8bf1-af1f20e60729"}]: dispatch 2023-12-01T01:11:50.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:50 smithi137 ceph-mon[103312]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0adf8c54-49ec-49a0-8bf1-af1f20e60729"}]': finished 2023-12-01T01:11:50.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:50 smithi137 ceph-mon[103312]: osdmap e10: 2 total, 1 up, 2 in 2023-12-01T01:11:50.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:50 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-01T01:11:50.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:50 smithi077 ceph-mon[98867]: from='client.? 172.21.15.137:0/1482903968' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0adf8c54-49ec-49a0-8bf1-af1f20e60729"}]: dispatch 2023-12-01T01:11:50.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:50 smithi077 ceph-mon[98867]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0adf8c54-49ec-49a0-8bf1-af1f20e60729"}]: dispatch 2023-12-01T01:11:50.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:50 smithi077 ceph-mon[98867]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0adf8c54-49ec-49a0-8bf1-af1f20e60729"}]': finished 2023-12-01T01:11:50.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:50 smithi077 ceph-mon[98867]: osdmap e10: 2 total, 1 up, 2 in 2023-12-01T01:11:50.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:50 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-01T01:11:51.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:51 smithi203 ceph-mon[103581]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-01T01:11:51.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:51 smithi203 ceph-mon[103581]: from='client.? 172.21.15.137:0/561958498' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-01T01:11:51.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:51 smithi137 ceph-mon[103312]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-01T01:11:51.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:51 smithi137 ceph-mon[103312]: from='client.? 172.21.15.137:0/561958498' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-01T01:11:51.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:51 smithi077 ceph-mon[98867]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-01T01:11:51.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:51 smithi077 ceph-mon[98867]: from='client.? 172.21.15.137:0/561958498' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-01T01:11:53.778 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:53 smithi137 ceph-mon[103312]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-01T01:11:53.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:53 smithi077 ceph-mon[98867]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-01T01:11:54.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:53 smithi203 ceph-mon[103581]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-01T01:11:55.621 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:55 smithi137 ceph-mon[103312]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-01T01:11:55.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:55 smithi077 ceph-mon[98867]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-01T01:11:56.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:55 smithi203 ceph-mon[103581]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-01T01:11:57.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:57 smithi137 ceph-mon[103312]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-01T01:11:57.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:57 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-12-01T01:11:57.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:57 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:57.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:57 smithi077 ceph-mon[98867]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-01T01:11:57.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:57 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-12-01T01:11:57.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:57 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:58.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:57 smithi203 ceph-mon[103581]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-01T01:11:58.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:57 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-12-01T01:11:58.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:57 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:11:58.709 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:58 smithi137 ceph-mon[103312]: Deploying daemon osd.1 on smithi137 2023-12-01T01:11:58.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:58 smithi077 ceph-mon[98867]: Deploying daemon osd.1 on smithi137 2023-12-01T01:11:59.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:58 smithi203 ceph-mon[103581]: Deploying daemon osd.1 on smithi137 2023-12-01T01:11:59.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:11:59 smithi137 ceph-mon[103312]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-01T01:11:59.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:11:59 smithi077 ceph-mon[98867]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-01T01:12:00.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:11:59 smithi203 ceph-mon[103581]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-01T01:12:01.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:01 smithi077 ceph-mon[98867]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-01T01:12:01.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:01 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:01.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:01 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:01.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:01 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:12:02.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:01 smithi203 ceph-mon[103581]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-01T01:12:02.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:01 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:02.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:01 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:02.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:01 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:12:02.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:01 smithi137 ceph-mon[103312]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-01T01:12:02.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:01 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:02.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:01 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:02.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:01 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:12:03.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:03 smithi137 ceph-mon[103312]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-01T01:12:03.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:03 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:03.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:03 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:03.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:03 smithi077 ceph-mon[98867]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-01T01:12:03.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:03 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:03.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:03 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:04.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:03 smithi203 ceph-mon[103581]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-01T01:12:04.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:03 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:04.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:03 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:04.381 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:04.380+0000 7f32137fe700 1 -- 172.21.15.137:0/3220909775 <== mgr.14152 v2:172.21.15.77:6800/3838941679 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f32240611d0 con 0x7f320c0a0490 2023-12-01T01:12:04.382 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 1 on host 'smithi137' 2023-12-01T01:12:04.383 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:04.382+0000 7f3228bb6700 1 -- 172.21.15.137:0/3220909775 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f320c0a0490 msgr2=0x7f320c0a2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:12:04.384 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:04.382+0000 7f3228bb6700 1 --2- 172.21.15.137:0/3220909775 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f320c0a0490 0x7f320c0a2950 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3224100db0 tx=0x7f3218009190 comp rx=0 tx=0).stop 2023-12-01T01:12:04.384 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:04.382+0000 7f3228bb6700 1 -- 172.21.15.137:0/3220909775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3224100f50 msgr2=0x7f322419b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:12:04.384 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:04.382+0000 7f3228bb6700 1 --2- 172.21.15.137:0/3220909775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3224100f50 0x7f322419b9b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f321400d950 tx=0x7f321400dc60 comp rx=0 tx=0).stop 2023-12-01T01:12:04.384 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:04.382+0000 7f3228bb6700 1 -- 172.21.15.137:0/3220909775 shutdown_connections 2023-12-01T01:12:04.384 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:04.382+0000 7f3228bb6700 1 --2- 172.21.15.137:0/3220909775 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f320c0a0490 0x7f320c0a2950 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:12:04.384 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:04.382+0000 7f3228bb6700 1 --2- 172.21.15.137:0/3220909775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32241018f0 0x7f322419e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:12:04.384 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:04.382+0000 7f3228bb6700 1 --2- 172.21.15.137:0/3220909775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f32240ffd50 0x7f3224069cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:12:04.384 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:04.382+0000 7f3228bb6700 1 --2- 172.21.15.137:0/3220909775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3224100f50 0x7f322419b9b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:12:04.384 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:04.382+0000 7f3228bb6700 1 -- 172.21.15.137:0/3220909775 >> 172.21.15.137:0/3220909775 conn(0x7f32240fb310 msgr2=0x7f32241043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:12:04.384 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:04.382+0000 7f3228bb6700 1 -- 172.21.15.137:0/3220909775 shutdown_connections 2023-12-01T01:12:04.384 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:04.382+0000 7f3228bb6700 1 -- 172.21.15.137:0/3220909775 wait complete. 2023-12-01T01:12:04.835 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:04 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:12:04.836 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:04 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:12:04.836 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:04 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:04.836 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:04 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:04.836 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:04 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:04.836 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:04 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:12:04.837 DEBUG:teuthology.orchestra.run.smithi137:osd.1> sudo journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@osd.1.service 2023-12-01T01:12:04.839 INFO:tasks.cephadm:Deploying osd.2 on smithi203 with /dev/vg_nvme/lv_4... 2023-12-01T01:12:04.839 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-12-01T01:12:04.868 INFO:journalctl@ceph.osd.1.smithi137.stdout:-- Logs begin at Fri 2023-12-01 00:55:04 UTC. -- 2023-12-01T01:12:04.871 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:04 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:12:04.872 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:04 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:12:04.872 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:04 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:04.872 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:04 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:04.872 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:04 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:04.872 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:04 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:12:04.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:04 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:12:04.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:04 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:12:04.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:04 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:04.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:04 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:04.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:04 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:04.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:04 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:12:05.401 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.c/config 2023-12-01T01:12:05.529 INFO:journalctl@ceph.osd.1.smithi137.stdout:Dec 01 01:12:05 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-1[108281]: 2023-12-01T01:12:05.251+0000 7fc623853640 -1 osd.1 0 log_to_monitors true 2023-12-01T01:12:05.843 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:05 smithi203 ceph-mon[103581]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-01T01:12:05.843 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:05 smithi203 ceph-mon[103581]: from='osd.1 [v2:172.21.15.137:6800/224194890,v1:172.21.15.137:6801/224194890]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-01T01:12:05.843 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:05 smithi203 ceph-mon[103581]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-01T01:12:05.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:05 smithi077 ceph-mon[98867]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-01T01:12:05.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:05 smithi077 ceph-mon[98867]: from='osd.1 [v2:172.21.15.137:6800/224194890,v1:172.21.15.137:6801/224194890]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-01T01:12:05.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:05 smithi077 ceph-mon[98867]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-01T01:12:05.909 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:05 smithi137 ceph-mon[103312]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-01T01:12:05.909 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:05 smithi137 ceph-mon[103312]: from='osd.1 [v2:172.21.15.137:6800/224194890,v1:172.21.15.137:6801/224194890]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-01T01:12:05.910 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:05 smithi137 ceph-mon[103312]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-01T01:12:07.017 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-12-01T01:12:07.017 INFO:teuthology.orchestra.run.smithi203.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-12-01T01:12:07.017 INFO:teuthology.orchestra.run.smithi203.stderr: stderr: 10+0 records in 2023-12-01T01:12:07.017 INFO:teuthology.orchestra.run.smithi203.stderr:10+0 records out 2023-12-01T01:12:07.017 INFO:teuthology.orchestra.run.smithi203.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0110507 s, 949 MB/s 2023-12-01T01:12:07.017 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping successful for: 2023-12-01T01:12:07.253 INFO:journalctl@ceph.osd.1.smithi137.stdout:Dec 01 01:12:06 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-1[108281]: 2023-12-01T01:12:06.970+0000 7fc6186c9700 -1 osd.1 0 waiting for initial osdmap 2023-12-01T01:12:07.253 INFO:journalctl@ceph.osd.1.smithi137.stdout:Dec 01 01:12:06 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-1[108281]: 2023-12-01T01:12:06.974+0000 7fc6114b8700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-01T01:12:07.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:06 smithi137 ceph-mon[103312]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-12-01T01:12:07.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:06 smithi137 ceph-mon[103312]: osdmap e11: 2 total, 1 up, 2 in 2023-12-01T01:12:07.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:06 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-01T01:12:07.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:06 smithi137 ceph-mon[103312]: from='osd.1 [v2:172.21.15.137:6800/224194890,v1:172.21.15.137:6801/224194890]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2023-12-01T01:12:07.253 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:06 smithi137 ceph-mon[103312]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2023-12-01T01:12:07.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:06 smithi137 ceph-mon[103312]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-01T01:12:07.262 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:06 smithi203 ceph-mon[103581]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-12-01T01:12:07.262 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:06 smithi203 ceph-mon[103581]: osdmap e11: 2 total, 1 up, 2 in 2023-12-01T01:12:07.262 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:06 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-01T01:12:07.262 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:06 smithi203 ceph-mon[103581]: from='osd.1 [v2:172.21.15.137:6800/224194890,v1:172.21.15.137:6801/224194890]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2023-12-01T01:12:07.262 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:06 smithi203 ceph-mon[103581]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2023-12-01T01:12:07.262 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:06 smithi203 ceph-mon[103581]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-01T01:12:07.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:06 smithi077 ceph-mon[98867]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-12-01T01:12:07.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:06 smithi077 ceph-mon[98867]: osdmap e11: 2 total, 1 up, 2 in 2023-12-01T01:12:07.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:06 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-01T01:12:07.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:06 smithi077 ceph-mon[98867]: from='osd.1 [v2:172.21.15.137:6800/224194890,v1:172.21.15.137:6801/224194890]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2023-12-01T01:12:07.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:06 smithi077 ceph-mon[98867]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2023-12-01T01:12:07.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:06 smithi077 ceph-mon[98867]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-01T01:12:07.438 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi203:vg_nvme/lv_4 2023-12-01T01:12:08.034 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.c/config 2023-12-01T01:12:08.262 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:07 smithi203 ceph-mon[103581]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2023-12-01T01:12:08.262 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:07 smithi203 ceph-mon[103581]: osdmap e12: 2 total, 1 up, 2 in 2023-12-01T01:12:08.262 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:07 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-01T01:12:08.262 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:07 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-01T01:12:08.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:07 smithi203 ceph-mon[103581]: Detected new or changed devices on smithi137 2023-12-01T01:12:08.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:07 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:08.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:07 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:08.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:07 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:12:08.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:07 smithi203 ceph-mon[103581]: Adjusting osd_memory_target on smithi137 to 17168M 2023-12-01T01:12:08.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:07 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:08.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:07 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:12:08.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:07 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:12:08.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:07 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:08.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:07 smithi137 ceph-mon[103312]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2023-12-01T01:12:08.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:07 smithi137 ceph-mon[103312]: osdmap e12: 2 total, 1 up, 2 in 2023-12-01T01:12:08.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:07 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-01T01:12:08.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:07 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-01T01:12:08.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:07 smithi137 ceph-mon[103312]: Detected new or changed devices on smithi137 2023-12-01T01:12:08.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:07 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:08.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:07 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:08.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:07 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:12:08.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:07 smithi137 ceph-mon[103312]: Adjusting osd_memory_target on smithi137 to 17168M 2023-12-01T01:12:08.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:07 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:08.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:07 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:12:08.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:07 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:12:08.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:07 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:08.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:07 smithi077 ceph-mon[98867]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2023-12-01T01:12:08.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:07 smithi077 ceph-mon[98867]: osdmap e12: 2 total, 1 up, 2 in 2023-12-01T01:12:08.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:07 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-01T01:12:08.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:07 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-01T01:12:08.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:07 smithi077 ceph-mon[98867]: Detected new or changed devices on smithi137 2023-12-01T01:12:08.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:07 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:08.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:07 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:08.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:07 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:12:08.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:07 smithi077 ceph-mon[98867]: Adjusting osd_memory_target on smithi137 to 17168M 2023-12-01T01:12:08.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:07 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:08.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:07 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:12:08.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:07 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:12:08.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:07 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:09.117 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:08 smithi203 ceph-mon[103581]: purged_snaps scrub starts 2023-12-01T01:12:09.117 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:08 smithi203 ceph-mon[103581]: purged_snaps scrub ok 2023-12-01T01:12:09.117 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:08 smithi203 ceph-mon[103581]: osd.1 [v2:172.21.15.137:6800/224194890,v1:172.21.15.137:6801/224194890] boot 2023-12-01T01:12:09.117 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:08 smithi203 ceph-mon[103581]: osdmap e13: 2 total, 2 up, 2 in 2023-12-01T01:12:09.118 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:08 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-01T01:12:09.118 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:08 smithi203 ceph-mon[103581]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-01T01:12:09.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:08 smithi137 ceph-mon[103312]: purged_snaps scrub starts 2023-12-01T01:12:09.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:08 smithi137 ceph-mon[103312]: purged_snaps scrub ok 2023-12-01T01:12:09.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:08 smithi137 ceph-mon[103312]: osd.1 [v2:172.21.15.137:6800/224194890,v1:172.21.15.137:6801/224194890] boot 2023-12-01T01:12:09.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:08 smithi137 ceph-mon[103312]: osdmap e13: 2 total, 2 up, 2 in 2023-12-01T01:12:09.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:08 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-01T01:12:09.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:08 smithi137 ceph-mon[103312]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-01T01:12:09.336 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:09.334+0000 7fd06f924700 1 -- 172.21.15.203:0/3617354054 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd068101950 msgr2=0x7fd068103d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:12:09.336 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:09.334+0000 7fd06f924700 1 --2- 172.21.15.203:0/3617354054 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd068101950 0x7fd068103d40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd058009990 tx=0x7fd058009ca0 comp rx=0 tx=0).stop 2023-12-01T01:12:09.336 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:09.334+0000 7fd06f924700 1 -- 172.21.15.203:0/3617354054 shutdown_connections 2023-12-01T01:12:09.336 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:09.334+0000 7fd06f924700 1 --2- 172.21.15.203:0/3617354054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd068106bb0 0x7fd068108fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:12:09.336 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:09.334+0000 7fd06f924700 1 --2- 172.21.15.203:0/3617354054 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd068101950 0x7fd068103d40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:12:09.336 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:09.334+0000 7fd06f924700 1 --2- 172.21.15.203:0/3617354054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd068104280 0x7fd068106670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:12:09.336 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:09.335+0000 7fd06f924700 1 -- 172.21.15.203:0/3617354054 >> 172.21.15.203:0/3617354054 conn(0x7fd0680fb2a0 msgr2=0x7fd0680fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:12:09.336 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:09.335+0000 7fd06f924700 1 -- 172.21.15.203:0/3617354054 shutdown_connections 2023-12-01T01:12:09.336 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:09.335+0000 7fd06f924700 1 -- 172.21.15.203:0/3617354054 wait complete. 2023-12-01T01:12:09.339 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:09.335+0000 7fd06f924700 1 Processor -- start 2023-12-01T01:12:09.339 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:09.335+0000 7fd06f924700 1 -- start start 2023-12-01T01:12:09.339 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:09.336+0000 7fd06f924700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd068101950 0x7fd068196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:12:09.339 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:09.336+0000 7fd06f924700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd068104280 0x7fd068196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:12:09.339 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:09.336+0000 7fd06f924700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd068106bb0 0x7fd06819e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:12:09.339 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:09.336+0000 7fd06f924700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06819e630 con 0x7fd068104280 2023-12-01T01:12:09.339 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:09.336+0000 7fd06f924700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06819e770 con 0x7fd068106bb0 2023-12-01T01:12:09.339 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:09.336+0000 7fd06f924700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06819e900 con 0x7fd068101950 2023-12-01T01:12:09.339 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:09.336+0000 7fd06d6c0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd068101950 0x7fd068196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:12:09.339 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:09.336+0000 7fd06d6c0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd068101950 0x7fd068196aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:59198/0 (socket says 172.21.15.203:59198) 2023-12-01T01:12:09.339 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:09.336+0000 7fd06d6c0700 1 -- 172.21.15.203:0/904814417 learned_addr learned my addr 172.21.15.203:0/904814417 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-12-01T01:12:09.339 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:09.336+0000 7fd06cebf700 1 --2- 172.21.15.203:0/904814417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd068104280 0x7fd068196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:12:09.339 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:09.336+0000 7fd06d6c0700 1 -- 172.21.15.203:0/904814417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd068106bb0 msgr2=0x7fd06819e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:12:09.339 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:09.336+0000 7fd06d6c0700 1 --2- 172.21.15.203:0/904814417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd068106bb0 0x7fd06819e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:12:09.340 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:09.336+0000 7fd06d6c0700 1 -- 172.21.15.203:0/904814417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd068104280 msgr2=0x7fd068196fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:12:09.340 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:09.336+0000 7fd06d6c0700 1 --2- 172.21.15.203:0/904814417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd068104280 0x7fd068196fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:12:09.340 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:09.336+0000 7fd06d6c0700 1 -- 172.21.15.203:0/904814417 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0580095f0 con 0x7fd068101950 2023-12-01T01:12:09.340 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:09.336+0000 7fd06cebf700 1 --2- 172.21.15.203:0/904814417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd068104280 0x7fd068196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:12:09.340 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:09.336+0000 7fd06d6c0700 1 --2- 172.21.15.203:0/904814417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd068101950 0x7fd068196aa0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd05800f6e0 tx=0x7fd05800f7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:12:09.340 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:09.336+0000 7fd05e7fc700 1 -- 172.21.15.203:0/904814417 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05801d070 con 0x7fd068101950 2023-12-01T01:12:09.340 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:09.336+0000 7fd05e7fc700 1 -- 172.21.15.203:0/904814417 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd05800fc20 con 0x7fd068101950 2023-12-01T01:12:09.340 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:09.336+0000 7fd06f924700 1 -- 172.21.15.203:0/904814417 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd06819eb80 con 0x7fd068101950 2023-12-01T01:12:09.340 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:09.336+0000 7fd05e7fc700 1 -- 172.21.15.203:0/904814417 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0580176e0 con 0x7fd068101950 2023-12-01T01:12:09.340 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:09.336+0000 7fd06f924700 1 -- 172.21.15.203:0/904814417 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd06819eff0 con 0x7fd068101950 2023-12-01T01:12:09.340 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:09.337+0000 7fd06f924700 1 -- 172.21.15.203:0/904814417 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0680fce20 con 0x7fd068101950 2023-12-01T01:12:09.341 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:09.338+0000 7fd05e7fc700 1 -- 172.21.15.203:0/904814417 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 15) v1 ==== 135014+0+0 (secure 0 0 0) 0x7fd0580229c0 con 0x7fd068101950 2023-12-01T01:12:09.341 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:09.339+0000 7fd05e7fc700 1 --2- 172.21.15.203:0/904814417 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd0540a0510 0x7fd0540a29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:12:09.341 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:09.339+0000 7fd05e7fc700 1 -- 172.21.15.203:0/904814417 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7fd0580cb4a0 con 0x7fd068101950 2023-12-01T01:12:09.341 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:09.339+0000 7fd06cebf700 1 --2- 172.21.15.203:0/904814417 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd0540a0510 0x7fd0540a29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:12:09.341 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:09.340+0000 7fd06cebf700 1 --2- 172.21.15.203:0/904814417 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd0540a0510 0x7fd0540a29d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd064007900 tx=0x7fd064008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:12:09.344 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:09.343+0000 7fd05e7fc700 1 -- 172.21.15.203:0/904814417 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd05805d0e0 con 0x7fd068101950 2023-12-01T01:12:09.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:08 smithi077 ceph-mon[98867]: purged_snaps scrub starts 2023-12-01T01:12:09.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:08 smithi077 ceph-mon[98867]: purged_snaps scrub ok 2023-12-01T01:12:09.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:08 smithi077 ceph-mon[98867]: osd.1 [v2:172.21.15.137:6800/224194890,v1:172.21.15.137:6801/224194890] boot 2023-12-01T01:12:09.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:08 smithi077 ceph-mon[98867]: osdmap e13: 2 total, 2 up, 2 in 2023-12-01T01:12:09.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:08 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-01T01:12:09.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:08 smithi077 ceph-mon[98867]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-01T01:12:09.543 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:09.542+0000 7fd06f924700 1 -- 172.21.15.203:0/904814417 --> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fd0680611d0 con 0x7fd0540a0510 2023-12-01T01:12:10.262 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:09 smithi203 ceph-mon[103581]: osdmap e14: 2 total, 2 up, 2 in 2023-12-01T01:12:10.262 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:09 smithi203 ceph-mon[103581]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:12:10.262 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:09 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-01T01:12:10.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:09 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-01T01:12:10.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:09 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:12:10.278 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:09 smithi137 ceph-mon[103312]: osdmap e14: 2 total, 2 up, 2 in 2023-12-01T01:12:10.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:09 smithi137 ceph-mon[103312]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:12:10.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:09 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-01T01:12:10.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:09 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-01T01:12:10.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:09 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:12:10.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:09 smithi077 ceph-mon[98867]: osdmap e14: 2 total, 2 up, 2 in 2023-12-01T01:12:10.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:09 smithi077 ceph-mon[98867]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:12:10.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:09 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-01T01:12:10.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:09 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-01T01:12:10.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:09 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:12:11.233 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:10 smithi203 ceph-mon[103581]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-01T01:12:11.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:10 smithi137 ceph-mon[103312]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-01T01:12:11.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:10 smithi077 ceph-mon[98867]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-01T01:12:12.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:12 smithi137 ceph-mon[103312]: from='client.? 172.21.15.203:0/1411653492' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a69802c-dede-4544-9a12-436f5c1c0038"}]: dispatch 2023-12-01T01:12:12.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:12 smithi137 ceph-mon[103312]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a69802c-dede-4544-9a12-436f5c1c0038"}]: dispatch 2023-12-01T01:12:12.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:12 smithi137 ceph-mon[103312]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a69802c-dede-4544-9a12-436f5c1c0038"}]': finished 2023-12-01T01:12:12.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:12 smithi137 ceph-mon[103312]: osdmap e15: 3 total, 2 up, 3 in 2023-12-01T01:12:12.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:12 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-01T01:12:12.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:12 smithi077 ceph-mon[98867]: from='client.? 172.21.15.203:0/1411653492' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a69802c-dede-4544-9a12-436f5c1c0038"}]: dispatch 2023-12-01T01:12:12.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:12 smithi077 ceph-mon[98867]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a69802c-dede-4544-9a12-436f5c1c0038"}]: dispatch 2023-12-01T01:12:12.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:12 smithi077 ceph-mon[98867]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a69802c-dede-4544-9a12-436f5c1c0038"}]': finished 2023-12-01T01:12:12.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:12 smithi077 ceph-mon[98867]: osdmap e15: 3 total, 2 up, 3 in 2023-12-01T01:12:12.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:12 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-01T01:12:13.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:12 smithi203 ceph-mon[103581]: from='client.? 172.21.15.203:0/1411653492' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a69802c-dede-4544-9a12-436f5c1c0038"}]: dispatch 2023-12-01T01:12:13.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:12 smithi203 ceph-mon[103581]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a69802c-dede-4544-9a12-436f5c1c0038"}]: dispatch 2023-12-01T01:12:13.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:12 smithi203 ceph-mon[103581]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a69802c-dede-4544-9a12-436f5c1c0038"}]': finished 2023-12-01T01:12:13.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:12 smithi203 ceph-mon[103581]: osdmap e15: 3 total, 2 up, 3 in 2023-12-01T01:12:13.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:12 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-01T01:12:13.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:13 smithi137 ceph-mon[103312]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-01T01:12:13.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:13 smithi137 ceph-mon[103312]: from='client.? 172.21.15.203:0/3882126018' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-01T01:12:13.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:13 smithi077 ceph-mon[98867]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-01T01:12:13.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:13 smithi077 ceph-mon[98867]: from='client.? 172.21.15.203:0/3882126018' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-01T01:12:14.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:13 smithi203 ceph-mon[103581]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-01T01:12:14.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:13 smithi203 ceph-mon[103581]: from='client.? 172.21.15.203:0/3882126018' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-01T01:12:15.778 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:15 smithi137 ceph-mon[103312]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-01T01:12:15.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:15 smithi077 ceph-mon[98867]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-01T01:12:15.907 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:15 smithi203 ceph-mon[103581]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-01T01:12:17.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:17 smithi203 ceph-mon[103581]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-01T01:12:17.778 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:17 smithi137 ceph-mon[103312]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-01T01:12:17.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:17 smithi077 ceph-mon[98867]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-01T01:12:19.659 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:19 smithi203 ceph-mon[103581]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-01T01:12:19.659 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:19 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-12-01T01:12:19.659 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:19 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:12:19.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:19 smithi077 ceph-mon[98867]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-01T01:12:19.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:19 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-12-01T01:12:19.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:19 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:12:20.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:19 smithi137 ceph-mon[103312]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-01T01:12:20.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:19 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-12-01T01:12:20.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:19 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:12:20.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:20 smithi077 ceph-mon[98867]: Deploying daemon osd.2 on smithi203 2023-12-01T01:12:20.996 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:20 smithi203 ceph-mon[103581]: Deploying daemon osd.2 on smithi203 2023-12-01T01:12:21.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:20 smithi137 ceph-mon[103312]: Deploying daemon osd.2 on smithi203 2023-12-01T01:12:21.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:21 smithi077 ceph-mon[98867]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-01T01:12:22.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:21 smithi203 ceph-mon[103581]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-01T01:12:22.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:21 smithi137 ceph-mon[103312]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-01T01:12:23.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:23 smithi077 ceph-mon[98867]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-01T01:12:23.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:23 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:23.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:23 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:23.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:23 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:12:24.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:23 smithi203 ceph-mon[103581]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-01T01:12:24.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:23 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:24.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:23 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:24.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:23 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:12:24.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:23 smithi137 ceph-mon[103312]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-01T01:12:24.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:23 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:24.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:23 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:24.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:23 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:12:25.857 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:25 smithi203 ceph-mon[103581]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-01T01:12:25.857 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:25 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:25.857 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:25 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:25.857 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:25 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:12:25.857 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:25 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:12:25.857 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:25 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:25.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:25 smithi077 ceph-mon[98867]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-01T01:12:25.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:25 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:25.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:25 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:25.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:25 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:12:25.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:25 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:12:25.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:25 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:26.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:25 smithi137 ceph-mon[103312]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-01T01:12:26.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:25 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:26.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:25 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:26.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:25 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:12:26.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:25 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:12:26.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:25 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:26.154 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:26.152+0000 7fd05e7fc700 1 -- 172.21.15.203:0/904814417 <== mgr.14152 v2:172.21.15.77:6800/3838941679 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd0680611d0 con 0x7fd0540a0510 2023-12-01T01:12:26.154 INFO:teuthology.orchestra.run.smithi203.stdout:Created osd(s) 2 on host 'smithi203' 2023-12-01T01:12:26.157 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:26.154+0000 7fd06f924700 1 -- 172.21.15.203:0/904814417 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd0540a0510 msgr2=0x7fd0540a29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:12:26.157 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:26.154+0000 7fd06f924700 1 --2- 172.21.15.203:0/904814417 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd0540a0510 0x7fd0540a29d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd064007900 tx=0x7fd064008040 comp rx=0 tx=0).stop 2023-12-01T01:12:26.157 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:26.154+0000 7fd06f924700 1 -- 172.21.15.203:0/904814417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd068101950 msgr2=0x7fd068196aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:12:26.157 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:26.154+0000 7fd06f924700 1 --2- 172.21.15.203:0/904814417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd068101950 0x7fd068196aa0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd05800f6e0 tx=0x7fd05800f7c0 comp rx=0 tx=0).stop 2023-12-01T01:12:26.157 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:26.154+0000 7fd06f924700 1 -- 172.21.15.203:0/904814417 shutdown_connections 2023-12-01T01:12:26.157 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:26.154+0000 7fd06f924700 1 --2- 172.21.15.203:0/904814417 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd0540a0510 0x7fd0540a29d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:12:26.157 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:26.154+0000 7fd06f924700 1 --2- 172.21.15.203:0/904814417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd068106bb0 0x7fd06819e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:12:26.157 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:26.154+0000 7fd06f924700 1 --2- 172.21.15.203:0/904814417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd068101950 0x7fd068196aa0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:12:26.157 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:26.154+0000 7fd06f924700 1 --2- 172.21.15.203:0/904814417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd068104280 0x7fd068196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:12:26.157 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:26.154+0000 7fd06f924700 1 -- 172.21.15.203:0/904814417 >> 172.21.15.203:0/904814417 conn(0x7fd0680fb2a0 msgr2=0x7fd0681025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:12:26.157 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:26.155+0000 7fd06f924700 1 -- 172.21.15.203:0/904814417 shutdown_connections 2023-12-01T01:12:26.157 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:12:26.155+0000 7fd06f924700 1 -- 172.21.15.203:0/904814417 wait complete. 2023-12-01T01:12:26.611 DEBUG:teuthology.orchestra.run.smithi203:osd.2> sudo journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@osd.2.service 2023-12-01T01:12:26.613 INFO:tasks.cephadm:Deploying osd.3 on smithi077 with /dev/vg_nvme/lv_3... 2023-12-01T01:12:26.613 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-12-01T01:12:26.645 INFO:journalctl@ceph.osd.2.smithi203.stdout:-- Logs begin at Fri 2023-12-01 00:55:29 UTC. -- 2023-12-01T01:12:27.179 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:12:27.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:27 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:27.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:27 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:12:27.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:27 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:27.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:27 smithi077 ceph-mon[98867]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-01T01:12:27.424 INFO:journalctl@ceph.osd.2.smithi203.stdout:Dec 01 01:12:27 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-2[109394]: 2023-12-01T01:12:27.208+0000 7f1a5d150640 -1 osd.2 0 log_to_monitors true 2023-12-01T01:12:27.425 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:27 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:27.425 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:27 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:12:27.425 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:27 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:27.425 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:27 smithi203 ceph-mon[103581]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-01T01:12:27.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:27 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:27.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:27 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:12:27.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:27 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:27.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:27 smithi137 ceph-mon[103312]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-01T01:12:28.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:28 smithi077 ceph-mon[98867]: from='osd.2 [v2:172.21.15.203:6800/4078459110,v1:172.21.15.203:6801/4078459110]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-01T01:12:28.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:28 smithi077 ceph-mon[98867]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-01T01:12:28.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:28 smithi203 ceph-mon[103581]: from='osd.2 [v2:172.21.15.203:6800/4078459110,v1:172.21.15.203:6801/4078459110]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-01T01:12:28.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:28 smithi203 ceph-mon[103581]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-01T01:12:28.528 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:28 smithi137 ceph-mon[103312]: from='osd.2 [v2:172.21.15.203:6800/4078459110,v1:172.21.15.203:6801/4078459110]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-01T01:12:28.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:28 smithi137 ceph-mon[103312]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-01T01:12:28.636 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-12-01T01:12:28.636 INFO:teuthology.orchestra.run.smithi077.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-12-01T01:12:28.636 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10+0 records in 2023-12-01T01:12:28.636 INFO:teuthology.orchestra.run.smithi077.stderr:10+0 records out 2023-12-01T01:12:28.636 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110143 s, 952 MB/s 2023-12-01T01:12:28.636 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping successful for: 2023-12-01T01:12:29.024 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi077:vg_nvme/lv_3 2023-12-01T01:12:29.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:29 smithi077 ceph-mon[98867]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-12-01T01:12:29.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:29 smithi077 ceph-mon[98867]: osdmap e16: 3 total, 2 up, 3 in 2023-12-01T01:12:29.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:29 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-01T01:12:29.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:29 smithi077 ceph-mon[98867]: from='osd.2 [v2:172.21.15.203:6800/4078459110,v1:172.21.15.203:6801/4078459110]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-12-01T01:12:29.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:29 smithi077 ceph-mon[98867]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-12-01T01:12:29.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:29 smithi077 ceph-mon[98867]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-01T01:12:29.467 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:29 smithi203 ceph-mon[103581]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-12-01T01:12:29.467 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:29 smithi203 ceph-mon[103581]: osdmap e16: 3 total, 2 up, 3 in 2023-12-01T01:12:29.467 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:29 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-01T01:12:29.467 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:29 smithi203 ceph-mon[103581]: from='osd.2 [v2:172.21.15.203:6800/4078459110,v1:172.21.15.203:6801/4078459110]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-12-01T01:12:29.467 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:29 smithi203 ceph-mon[103581]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-12-01T01:12:29.467 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:29 smithi203 ceph-mon[103581]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-01T01:12:29.468 INFO:journalctl@ceph.osd.2.smithi203.stdout:Dec 01 01:12:29 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-2[109394]: 2023-12-01T01:12:29.178+0000 7f1a51fc6700 -1 osd.2 0 waiting for initial osdmap 2023-12-01T01:12:29.468 INFO:journalctl@ceph.osd.2.smithi203.stdout:Dec 01 01:12:29 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-2[109394]: 2023-12-01T01:12:29.185+0000 7f1a4c5b8700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-01T01:12:29.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:29 smithi137 ceph-mon[103312]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-12-01T01:12:29.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:29 smithi137 ceph-mon[103312]: osdmap e16: 3 total, 2 up, 3 in 2023-12-01T01:12:29.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:29 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-01T01:12:29.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:29 smithi137 ceph-mon[103312]: from='osd.2 [v2:172.21.15.203:6800/4078459110,v1:172.21.15.203:6801/4078459110]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-12-01T01:12:29.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:29 smithi137 ceph-mon[103312]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-12-01T01:12:29.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:29 smithi137 ceph-mon[103312]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-01T01:12:29.576 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:12:30.366 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:30 smithi077 ceph-mon[98867]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2023-12-01T01:12:30.366 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:30 smithi077 ceph-mon[98867]: osdmap e17: 3 total, 2 up, 3 in 2023-12-01T01:12:30.366 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:30 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-01T01:12:30.366 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:30 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-01T01:12:30.366 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:30 smithi077 ceph-mon[98867]: Detected new or changed devices on smithi203 2023-12-01T01:12:30.366 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:30 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:30.366 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:30 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:30.366 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:30 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:12:30.366 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:30 smithi077 ceph-mon[98867]: Adjusting osd_memory_target on smithi203 to 17168M 2023-12-01T01:12:30.366 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:30 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:30.366 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:30 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:12:30.367 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:30 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:12:30.367 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:30 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:30.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:30 smithi203 ceph-mon[103581]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2023-12-01T01:12:30.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:30 smithi203 ceph-mon[103581]: osdmap e17: 3 total, 2 up, 3 in 2023-12-01T01:12:30.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:30 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-01T01:12:30.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:30 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-01T01:12:30.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:30 smithi203 ceph-mon[103581]: Detected new or changed devices on smithi203 2023-12-01T01:12:30.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:30 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:30.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:30 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:30.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:30 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:12:30.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:30 smithi203 ceph-mon[103581]: Adjusting osd_memory_target on smithi203 to 17168M 2023-12-01T01:12:30.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:30 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:30.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:30 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:12:30.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:30 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:12:30.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:30 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:30.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:30 smithi137 ceph-mon[103312]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2023-12-01T01:12:30.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:30 smithi137 ceph-mon[103312]: osdmap e17: 3 total, 2 up, 3 in 2023-12-01T01:12:30.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:30 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-01T01:12:30.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:30 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-01T01:12:30.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:30 smithi137 ceph-mon[103312]: Detected new or changed devices on smithi203 2023-12-01T01:12:30.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:30 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:30.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:30 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:30.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:30 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:12:30.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:30 smithi137 ceph-mon[103312]: Adjusting osd_memory_target on smithi203 to 17168M 2023-12-01T01:12:30.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:30 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:30.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:30 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:12:30.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:30 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:12:30.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:30 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:30.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:30.762+0000 7f994d027700 1 -- 172.21.15.77:0/3150618098 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99480fff30 msgr2=0x7f99481003b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:12:30.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:30.762+0000 7f994d027700 1 --2- 172.21.15.77:0/3150618098 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99480fff30 0x7f99481003b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f993c009990 tx=0x7f993c009ca0 comp rx=0 tx=0).stop 2023-12-01T01:12:30.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:30.763+0000 7f994d027700 1 -- 172.21.15.77:0/3150618098 shutdown_connections 2023-12-01T01:12:30.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:30.763+0000 7f994d027700 1 --2- 172.21.15.77:0/3150618098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99481008f0 0x7f9948108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:12:30.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:30.763+0000 7f994d027700 1 --2- 172.21.15.77:0/3150618098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99481012d0 0x7f99481016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:12:30.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:30.763+0000 7f994d027700 1 --2- 172.21.15.77:0/3150618098 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99480fff30 0x7f99481003b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:12:30.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:30.763+0000 7f994d027700 1 -- 172.21.15.77:0/3150618098 >> 172.21.15.77:0/3150618098 conn(0x7f99480faa40 msgr2=0x7f99480fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:12:30.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:30.763+0000 7f994d027700 1 -- 172.21.15.77:0/3150618098 shutdown_connections 2023-12-01T01:12:30.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:30.763+0000 7f994d027700 1 -- 172.21.15.77:0/3150618098 wait complete. 2023-12-01T01:12:30.766 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:30.764+0000 7f994d027700 1 Processor -- start 2023-12-01T01:12:30.766 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:30.764+0000 7f994d027700 1 -- start start 2023-12-01T01:12:30.767 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:30.764+0000 7f994d027700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99480fff30 0x7f9948195990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:12:30.767 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:30.764+0000 7f994d027700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99481008f0 0x7f9948195ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:12:30.768 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:30.764+0000 7f994d027700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99481012d0 0x7f994819cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:12:30.768 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:30.764+0000 7f994d027700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994819d520 con 0x7f99481008f0 2023-12-01T01:12:30.768 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:30.764+0000 7f994d027700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994819d660 con 0x7f99480fff30 2023-12-01T01:12:30.768 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:30.764+0000 7f994d027700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994819d7f0 con 0x7f99481012d0 2023-12-01T01:12:30.768 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:30.764+0000 7f994659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99481008f0 0x7f9948195ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:12:30.768 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:30.764+0000 7f994659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99481008f0 0x7f9948195ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56646/0 (socket says 172.21.15.77:56646) 2023-12-01T01:12:30.768 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:30.764+0000 7f994659c700 1 -- 172.21.15.77:0/1687872337 learned_addr learned my addr 172.21.15.77:0/1687872337 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:12:30.768 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:30.764+0000 7f994759e700 1 --2- 172.21.15.77:0/1687872337 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99481012d0 0x7f994819cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:12:30.768 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:30.764+0000 7f9946d9d700 1 --2- 172.21.15.77:0/1687872337 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99480fff30 0x7f9948195990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:12:30.768 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:30.764+0000 7f994659c700 1 -- 172.21.15.77:0/1687872337 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99481012d0 msgr2=0x7f994819cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:12:30.768 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:30.764+0000 7f994659c700 1 --2- 172.21.15.77:0/1687872337 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99481012d0 0x7f994819cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:12:30.768 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:30.764+0000 7f994659c700 1 -- 172.21.15.77:0/1687872337 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99480fff30 msgr2=0x7f9948195990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:12:30.768 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:30.764+0000 7f994659c700 1 --2- 172.21.15.77:0/1687872337 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99480fff30 0x7f9948195990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:12:30.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:30.765+0000 7f994659c700 1 -- 172.21.15.77:0/1687872337 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f993c0095f0 con 0x7f99481008f0 2023-12-01T01:12:30.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:30.765+0000 7f9946d9d700 1 --2- 172.21.15.77:0/1687872337 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99480fff30 0x7f9948195990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:12:30.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:30.765+0000 7f994659c700 1 --2- 172.21.15.77:0/1687872337 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99481008f0 0x7f9948195ed0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f993c005910 tx=0x7f993c00f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:12:30.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:30.765+0000 7f992ffff700 1 -- 172.21.15.77:0/1687872337 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993c01d070 con 0x7f99481008f0 2023-12-01T01:12:30.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:30.765+0000 7f994d027700 1 -- 172.21.15.77:0/1687872337 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99480fef60 con 0x7f99481008f0 2023-12-01T01:12:30.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:30.765+0000 7f992ffff700 1 -- 172.21.15.77:0/1687872337 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f993c022470 con 0x7f99481008f0 2023-12-01T01:12:30.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:30.765+0000 7f992ffff700 1 -- 172.21.15.77:0/1687872337 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993c017650 con 0x7f99481008f0 2023-12-01T01:12:30.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:30.765+0000 7f994d027700 1 -- 172.21.15.77:0/1687872337 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99480ff450 con 0x7f99481008f0 2023-12-01T01:12:30.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:30.766+0000 7f994d027700 1 -- 172.21.15.77:0/1687872337 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f994810b500 con 0x7f99481008f0 2023-12-01T01:12:30.770 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:30.767+0000 7f992ffff700 1 -- 172.21.15.77:0/1687872337 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 15) v1 ==== 135014+0+0 (secure 0 0 0) 0x7f993c0225e0 con 0x7f99481008f0 2023-12-01T01:12:30.770 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:30.768+0000 7f992ffff700 1 --2- 172.21.15.77:0/1687872337 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f99300a0510 0x7f99300a29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:12:30.770 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:30.768+0000 7f9946d9d700 1 --2- 172.21.15.77:0/1687872337 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f99300a0510 0x7f99300a29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:12:30.770 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:30.768+0000 7f992ffff700 1 -- 172.21.15.77:0/1687872337 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 2851+0+0 (secure 0 0 0) 0x7f993c0cba90 con 0x7f99481008f0 2023-12-01T01:12:30.770 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:30.768+0000 7f9946d9d700 1 --2- 172.21.15.77:0/1687872337 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f99300a0510 0x7f99300a29d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9938005d90 tx=0x7f9938005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:12:30.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:30.772+0000 7f992ffff700 1 -- 172.21.15.77:0/1687872337 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f993c096c50 con 0x7f99481008f0 2023-12-01T01:12:30.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:30.968+0000 7f994d027700 1 -- 172.21.15.77:0/1687872337 --> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f99480ff670 con 0x7f99300a0510 2023-12-01T01:12:31.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:31 smithi203 ceph-mon[103581]: purged_snaps scrub starts 2023-12-01T01:12:31.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:31 smithi203 ceph-mon[103581]: purged_snaps scrub ok 2023-12-01T01:12:31.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:31 smithi203 ceph-mon[103581]: osd.2 [v2:172.21.15.203:6800/4078459110,v1:172.21.15.203:6801/4078459110] boot 2023-12-01T01:12:31.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:31 smithi203 ceph-mon[103581]: osdmap e18: 3 total, 3 up, 3 in 2023-12-01T01:12:31.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:31 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-01T01:12:31.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:31 smithi203 ceph-mon[103581]: pgmap v71: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-12-01T01:12:31.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:31 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-12-01T01:12:31.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:31 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-01T01:12:31.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:31 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-01T01:12:31.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:31 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:12:31.528 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:31 smithi137 ceph-mon[103312]: purged_snaps scrub starts 2023-12-01T01:12:31.528 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:31 smithi137 ceph-mon[103312]: purged_snaps scrub ok 2023-12-01T01:12:31.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:31 smithi137 ceph-mon[103312]: osd.2 [v2:172.21.15.203:6800/4078459110,v1:172.21.15.203:6801/4078459110] boot 2023-12-01T01:12:31.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:31 smithi137 ceph-mon[103312]: osdmap e18: 3 total, 3 up, 3 in 2023-12-01T01:12:31.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:31 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-01T01:12:31.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:31 smithi137 ceph-mon[103312]: pgmap v71: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-12-01T01:12:31.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:31 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-12-01T01:12:31.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:31 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-01T01:12:31.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:31 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-01T01:12:31.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:31 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:12:31.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:31 smithi077 ceph-mon[98867]: purged_snaps scrub starts 2023-12-01T01:12:31.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:31 smithi077 ceph-mon[98867]: purged_snaps scrub ok 2023-12-01T01:12:31.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:31 smithi077 ceph-mon[98867]: osd.2 [v2:172.21.15.203:6800/4078459110,v1:172.21.15.203:6801/4078459110] boot 2023-12-01T01:12:31.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:31 smithi077 ceph-mon[98867]: osdmap e18: 3 total, 3 up, 3 in 2023-12-01T01:12:31.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:31 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-01T01:12:31.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:31 smithi077 ceph-mon[98867]: pgmap v71: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-12-01T01:12:31.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:31 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-12-01T01:12:31.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:31 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-01T01:12:31.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:31 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-01T01:12:31.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:31 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:12:32.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:32 smithi077 ceph-mon[98867]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:12:32.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:32 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-12-01T01:12:32.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:32 smithi077 ceph-mon[98867]: osdmap e19: 3 total, 3 up, 3 in 2023-12-01T01:12:32.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:32 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-12-01T01:12:32.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:32 smithi203 ceph-mon[103581]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:12:32.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:32 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-12-01T01:12:32.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:32 smithi203 ceph-mon[103581]: osdmap e19: 3 total, 3 up, 3 in 2023-12-01T01:12:32.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:32 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-12-01T01:12:32.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:32 smithi137 ceph-mon[103312]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:12:32.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:32 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-12-01T01:12:32.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:32 smithi137 ceph-mon[103312]: osdmap e19: 3 total, 3 up, 3 in 2023-12-01T01:12:32.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:32 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-12-01T01:12:32.802 INFO:journalctl@ceph.osd.0.smithi077.stdout:Dec 01 01:12:32 smithi077 sudo[123060]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-01T01:12:32.802 INFO:journalctl@ceph.osd.0.smithi077.stdout:Dec 01 01:12:32 smithi077 sudo[123060]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-01T01:12:32.802 INFO:journalctl@ceph.osd.0.smithi077.stdout:Dec 01 01:12:32 smithi077 sudo[123060]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-01T01:12:32.802 INFO:journalctl@ceph.osd.0.smithi077.stdout:Dec 01 01:12:32 smithi077 sudo[123060]: pam_unix(sudo:session): session closed for user root 2023-12-01T01:12:33.154 INFO:journalctl@ceph.osd.0.smithi077.stdout:Dec 01 01:12:32 smithi077 sudo[123093]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-01T01:12:33.154 INFO:journalctl@ceph.osd.0.smithi077.stdout:Dec 01 01:12:32 smithi077 sudo[123093]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-01T01:12:33.154 INFO:journalctl@ceph.osd.0.smithi077.stdout:Dec 01 01:12:32 smithi077 sudo[123093]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-01T01:12:33.154 INFO:journalctl@ceph.osd.0.smithi077.stdout:Dec 01 01:12:32 smithi077 sudo[123093]: pam_unix(sudo:session): session closed for user root 2023-12-01T01:12:33.330 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:33 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-12-01T01:12:33.331 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:33 smithi137 ceph-mon[103312]: osdmap e20: 3 total, 3 up, 3 in 2023-12-01T01:12:33.331 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:33 smithi137 ceph-mon[103312]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-12-01T01:12:33.331 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:33 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:33.331 INFO:journalctl@ceph.osd.1.smithi137.stdout:Dec 01 01:12:33 smithi137 sudo[113736]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-01T01:12:33.331 INFO:journalctl@ceph.osd.1.smithi137.stdout:Dec 01 01:12:33 smithi137 sudo[113736]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-01T01:12:33.331 INFO:journalctl@ceph.osd.1.smithi137.stdout:Dec 01 01:12:33 smithi137 sudo[113736]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-01T01:12:33.331 INFO:journalctl@ceph.osd.1.smithi137.stdout:Dec 01 01:12:33 smithi137 sudo[113736]: pam_unix(sudo:session): session closed for user root 2023-12-01T01:12:33.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:33 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-12-01T01:12:33.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:33 smithi203 ceph-mon[103581]: osdmap e20: 3 total, 3 up, 3 in 2023-12-01T01:12:33.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:33 smithi203 ceph-mon[103581]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-12-01T01:12:33.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:33 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:33.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:33 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-12-01T01:12:33.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:33 smithi077 ceph-mon[98867]: osdmap e20: 3 total, 3 up, 3 in 2023-12-01T01:12:33.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:33 smithi077 ceph-mon[98867]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-12-01T01:12:33.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:33 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:33.778 INFO:journalctl@ceph.osd.1.smithi137.stdout:Dec 01 01:12:33 smithi137 sudo[113739]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-01T01:12:33.779 INFO:journalctl@ceph.osd.1.smithi137.stdout:Dec 01 01:12:33 smithi137 sudo[113739]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-01T01:12:33.779 INFO:journalctl@ceph.osd.1.smithi137.stdout:Dec 01 01:12:33 smithi137 sudo[113739]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-01T01:12:33.779 INFO:journalctl@ceph.osd.1.smithi137.stdout:Dec 01 01:12:33 smithi137 sudo[113739]: pam_unix(sudo:session): session closed for user root 2023-12-01T01:12:33.838 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:33.837+0000 7f992ffff700 1 -- 172.21.15.77:0/1687872337 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f993c097020 con 0x7f99481008f0 2023-12-01T01:12:33.855 INFO:journalctl@ceph.osd.2.smithi203.stdout:Dec 01 01:12:33 smithi203 sudo[114839]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-01T01:12:33.855 INFO:journalctl@ceph.osd.2.smithi203.stdout:Dec 01 01:12:33 smithi203 sudo[114839]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-01T01:12:33.855 INFO:journalctl@ceph.osd.2.smithi203.stdout:Dec 01 01:12:33 smithi203 sudo[114839]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-01T01:12:33.855 INFO:journalctl@ceph.osd.2.smithi203.stdout:Dec 01 01:12:33 smithi203 sudo[114839]: pam_unix(sudo:session): session closed for user root 2023-12-01T01:12:34.262 INFO:journalctl@ceph.osd.2.smithi203.stdout:Dec 01 01:12:33 smithi203 sudo[114842]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-01T01:12:34.262 INFO:journalctl@ceph.osd.2.smithi203.stdout:Dec 01 01:12:33 smithi203 sudo[114842]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-01T01:12:34.262 INFO:journalctl@ceph.osd.2.smithi203.stdout:Dec 01 01:12:33 smithi203 sudo[114842]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-01T01:12:34.263 INFO:journalctl@ceph.osd.2.smithi203.stdout:Dec 01 01:12:33 smithi203 sudo[114842]: pam_unix(sudo:session): session closed for user root 2023-12-01T01:12:34.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:34 smithi077 sudo[123164]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-01T01:12:34.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:34 smithi077 sudo[123164]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-01T01:12:34.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:34 smithi077 sudo[123164]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-01T01:12:34.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:34 smithi077 sudo[123164]: pam_unix(sudo:session): session closed for user root 2023-12-01T01:12:34.642 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:34 smithi203 ceph-mon[103581]: osdmap e21: 3 total, 3 up, 3 in 2023-12-01T01:12:34.642 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:34 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/2771623700' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad325a97-580b-4c99-bf12-cbb0d92a6d0f"}]: dispatch 2023-12-01T01:12:34.643 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:34 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/2771623700' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad325a97-580b-4c99-bf12-cbb0d92a6d0f"}]': finished 2023-12-01T01:12:34.643 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:34 smithi203 ceph-mon[103581]: osdmap e22: 4 total, 3 up, 4 in 2023-12-01T01:12:34.643 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:34 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-01T01:12:34.643 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:34 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:34.643 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:34 smithi203 ceph-mon[103581]: mgrmap e16: x(active, since 2m), standbys: y, z 2023-12-01T01:12:34.643 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:34 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:34.643 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:34 smithi203 ceph-mon[103581]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-01T01:12:34.643 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:34 smithi203 sudo[114845]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-01T01:12:34.643 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:34 smithi203 sudo[114845]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-01T01:12:34.643 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:34 smithi203 sudo[114845]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-01T01:12:34.643 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:34 smithi203 sudo[114845]: pam_unix(sudo:session): session closed for user root 2023-12-01T01:12:34.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:34 smithi077 sudo[123167]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-01T01:12:34.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:34 smithi077 sudo[123167]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-01T01:12:34.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:34 smithi077 sudo[123167]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-01T01:12:34.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:34 smithi077 sudo[123167]: pam_unix(sudo:session): session closed for user root 2023-12-01T01:12:34.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:34 smithi077 ceph-mon[98867]: osdmap e21: 3 total, 3 up, 3 in 2023-12-01T01:12:34.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:34 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/2771623700' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad325a97-580b-4c99-bf12-cbb0d92a6d0f"}]: dispatch 2023-12-01T01:12:34.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:34 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/2771623700' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad325a97-580b-4c99-bf12-cbb0d92a6d0f"}]': finished 2023-12-01T01:12:34.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:34 smithi077 ceph-mon[98867]: osdmap e22: 4 total, 3 up, 4 in 2023-12-01T01:12:34.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:34 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-01T01:12:34.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:34 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:34.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:34 smithi077 ceph-mon[98867]: mgrmap e16: x(active, since 2m), standbys: y, z 2023-12-01T01:12:34.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:34 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:34.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:34 smithi077 ceph-mon[98867]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-01T01:12:34.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:34 smithi137 ceph-mon[103312]: osdmap e21: 3 total, 3 up, 3 in 2023-12-01T01:12:34.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:34 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/2771623700' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad325a97-580b-4c99-bf12-cbb0d92a6d0f"}]: dispatch 2023-12-01T01:12:34.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:34 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/2771623700' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad325a97-580b-4c99-bf12-cbb0d92a6d0f"}]': finished 2023-12-01T01:12:34.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:34 smithi137 ceph-mon[103312]: osdmap e22: 4 total, 3 up, 4 in 2023-12-01T01:12:34.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:34 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-01T01:12:34.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:34 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:34.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:34 smithi137 ceph-mon[103312]: mgrmap e16: x(active, since 2m), standbys: y, z 2023-12-01T01:12:34.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:34 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:34.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:34 smithi137 ceph-mon[103312]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-01T01:12:35.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:34 smithi203 sudo[114848]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-01T01:12:35.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:34 smithi203 sudo[114848]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-01T01:12:35.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:34 smithi203 sudo[114848]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-01T01:12:35.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:34 smithi203 sudo[114848]: pam_unix(sudo:session): session closed for user root 2023-12-01T01:12:35.373 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:35 smithi137 sudo[113742]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-01T01:12:35.373 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:35 smithi137 sudo[113742]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-01T01:12:35.374 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:35 smithi137 sudo[113742]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-01T01:12:35.374 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:35 smithi137 sudo[113742]: pam_unix(sudo:session): session closed for user root 2023-12-01T01:12:35.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:35 smithi077 ceph-mon[98867]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-01T01:12:35.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:35 smithi077 ceph-mon[98867]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-01T01:12:35.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:35 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-01T01:12:35.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:35 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/528500565' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-01T01:12:35.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:35 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-01T01:12:35.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:35 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-01T01:12:35.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:35 smithi077 ceph-mon[98867]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-01T01:12:35.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:35 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-01T01:12:35.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:35 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-01T01:12:35.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:35 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-01T01:12:35.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:35 smithi077 ceph-mon[98867]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-01T01:12:35.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:35 smithi077 ceph-mon[98867]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-01T01:12:35.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:35 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-01T01:12:35.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:35 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-01T01:12:35.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:35 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-01T01:12:35.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:35 smithi077 ceph-mon[98867]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-01T01:12:35.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:35 smithi203 ceph-mon[103581]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-01T01:12:35.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:35 smithi203 ceph-mon[103581]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-01T01:12:35.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:35 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-01T01:12:35.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:35 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/528500565' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-01T01:12:35.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:35 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-01T01:12:35.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:35 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-01T01:12:35.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:35 smithi203 ceph-mon[103581]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-01T01:12:35.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:35 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-01T01:12:35.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:35 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-01T01:12:35.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:35 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-01T01:12:35.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:35 smithi203 ceph-mon[103581]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-01T01:12:35.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:35 smithi203 ceph-mon[103581]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-01T01:12:35.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:35 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-01T01:12:35.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:35 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-01T01:12:35.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:35 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-01T01:12:35.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:35 smithi203 ceph-mon[103581]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-01T01:12:35.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:35 smithi137 sudo[113745]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-01T01:12:35.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:35 smithi137 sudo[113745]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-01T01:12:35.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:35 smithi137 sudo[113745]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-01T01:12:35.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:35 smithi137 sudo[113745]: pam_unix(sudo:session): session closed for user root 2023-12-01T01:12:35.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:35 smithi137 ceph-mon[103312]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-01T01:12:35.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:35 smithi137 ceph-mon[103312]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-01T01:12:35.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:35 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-01T01:12:35.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:35 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/528500565' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-01T01:12:35.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:35 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-01T01:12:35.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:35 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-01T01:12:35.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:35 smithi137 ceph-mon[103312]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-01T01:12:35.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:35 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-01T01:12:35.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:35 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-01T01:12:35.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:35 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-01T01:12:35.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:35 smithi137 ceph-mon[103312]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-01T01:12:35.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:35 smithi137 ceph-mon[103312]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-01T01:12:35.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:35 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-01T01:12:35.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:35 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-01T01:12:35.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:35 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-01T01:12:35.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:35 smithi137 ceph-mon[103312]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-01T01:12:37.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:37 smithi137 ceph-mon[103312]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-12-01T01:12:37.826 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:37 smithi077 ceph-mon[98867]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-12-01T01:12:38.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:37 smithi203 ceph-mon[103581]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-12-01T01:12:39.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:39 smithi077 ceph-mon[98867]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-12-01T01:12:40.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:39 smithi203 ceph-mon[103581]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-12-01T01:12:40.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:39 smithi137 ceph-mon[103312]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-12-01T01:12:40.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:40 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-12-01T01:12:40.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:40 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:12:41.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:40 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-12-01T01:12:41.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:40 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:12:41.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:40 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-12-01T01:12:41.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:40 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:12:41.790 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:41 smithi077 ceph-mon[98867]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2023-12-01T01:12:41.791 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:41 smithi077 ceph-mon[98867]: Deploying daemon osd.3 on smithi077 2023-12-01T01:12:42.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:41 smithi203 ceph-mon[103581]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2023-12-01T01:12:42.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:41 smithi203 ceph-mon[103581]: Deploying daemon osd.3 on smithi077 2023-12-01T01:12:42.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:41 smithi137 ceph-mon[103312]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2023-12-01T01:12:42.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:41 smithi137 ceph-mon[103312]: Deploying daemon osd.3 on smithi077 2023-12-01T01:12:43.869 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:43 smithi077 ceph-mon[98867]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2023-12-01T01:12:44.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:43 smithi203 ceph-mon[103581]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2023-12-01T01:12:44.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:43 smithi137 ceph-mon[103312]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2023-12-01T01:12:45.800 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:45 smithi077 ceph-mon[98867]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2023-12-01T01:12:45.800 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:45 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:45.800 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:45 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:45.800 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:45 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:12:46.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:45 smithi203 ceph-mon[103581]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2023-12-01T01:12:46.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:45 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:46.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:45 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:46.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:45 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:12:46.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:45 smithi137 ceph-mon[103312]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2023-12-01T01:12:46.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:45 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:46.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:45 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:46.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:45 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:12:47.318 INFO:teuthology.orchestra.run.smithi077.stdout:Created osd(s) 3 on host 'smithi077' 2023-12-01T01:12:47.318 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:47.316+0000 7f992ffff700 1 -- 172.21.15.77:0/1687872337 <== mgr.14152 v2:172.21.15.77:6800/3838941679 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f99480ff670 con 0x7f99300a0510 2023-12-01T01:12:47.320 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:47.318+0000 7f994d027700 1 -- 172.21.15.77:0/1687872337 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f99300a0510 msgr2=0x7f99300a29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:12:47.320 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:47.318+0000 7f994d027700 1 --2- 172.21.15.77:0/1687872337 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f99300a0510 0x7f99300a29d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9938005d90 tx=0x7f9938005d00 comp rx=0 tx=0).stop 2023-12-01T01:12:47.320 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:47.318+0000 7f994d027700 1 -- 172.21.15.77:0/1687872337 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99481008f0 msgr2=0x7f9948195ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:12:47.320 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:47.318+0000 7f994d027700 1 --2- 172.21.15.77:0/1687872337 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99481008f0 0x7f9948195ed0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f993c005910 tx=0x7f993c00f710 comp rx=0 tx=0).stop 2023-12-01T01:12:47.320 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:47.318+0000 7f994d027700 1 -- 172.21.15.77:0/1687872337 shutdown_connections 2023-12-01T01:12:47.320 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:47.318+0000 7f994d027700 1 --2- 172.21.15.77:0/1687872337 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f99300a0510 0x7f99300a29d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:12:47.320 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:47.318+0000 7f994d027700 1 --2- 172.21.15.77:0/1687872337 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99481012d0 0x7f994819cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:12:47.320 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:47.318+0000 7f994d027700 1 --2- 172.21.15.77:0/1687872337 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99480fff30 0x7f9948195990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:12:47.320 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:47.318+0000 7f994d027700 1 --2- 172.21.15.77:0/1687872337 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99481008f0 0x7f9948195ed0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:12:47.320 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:47.318+0000 7f994d027700 1 -- 172.21.15.77:0/1687872337 >> 172.21.15.77:0/1687872337 conn(0x7f99480faa40 msgr2=0x7f9948104470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:12:47.321 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:47.319+0000 7f994d027700 1 -- 172.21.15.77:0/1687872337 shutdown_connections 2023-12-01T01:12:47.321 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:12:47.319+0000 7f994d027700 1 -- 172.21.15.77:0/1687872337 wait complete. 2023-12-01T01:12:47.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:47 smithi077 ceph-mon[98867]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-12-01T01:12:47.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:47 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:47.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:47 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:47.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:47 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:47.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:47 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:47.773 DEBUG:teuthology.orchestra.run.smithi077:osd.3> sudo journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@osd.3.service 2023-12-01T01:12:47.775 INFO:tasks.cephadm:Deploying osd.4 on smithi137 with /dev/vg_nvme/lv_3... 2023-12-01T01:12:47.775 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-12-01T01:12:47.804 INFO:journalctl@ceph.osd.3.smithi077.stdout:-- Logs begin at Fri 2023-12-01 00:55:13 UTC. -- 2023-12-01T01:12:47.808 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:47 smithi137 ceph-mon[103312]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-12-01T01:12:47.809 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:47 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:47.809 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:47 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:47.809 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:47 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:47.809 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:47 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:48.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:47 smithi203 ceph-mon[103581]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-12-01T01:12:48.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:47 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:48.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:47 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:48.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:47 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:48.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:47 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:48.340 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.b/config 2023-12-01T01:12:48.833 INFO:journalctl@ceph.osd.3.smithi077.stdout:Dec 01 01:12:48 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-3[125398]: 2023-12-01T01:12:48.604+0000 7ff0e114a640 -1 osd.3 0 log_to_monitors true 2023-12-01T01:12:49.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:49 smithi137 ceph-mon[103312]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-12-01T01:12:49.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:49 smithi137 ceph-mon[103312]: from='osd.3 [v2:172.21.15.77:6810/1454440477,v1:172.21.15.77:6811/1454440477]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-12-01T01:12:49.847 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-12-01T01:12:49.847 INFO:teuthology.orchestra.run.smithi137.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-12-01T01:12:49.847 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10+0 records in 2023-12-01T01:12:49.847 INFO:teuthology.orchestra.run.smithi137.stderr:10+0 records out 2023-12-01T01:12:49.847 INFO:teuthology.orchestra.run.smithi137.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0149288 s, 702 MB/s 2023-12-01T01:12:49.847 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping successful for: 2023-12-01T01:12:49.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:49 smithi077 ceph-mon[98867]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-12-01T01:12:49.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:49 smithi077 ceph-mon[98867]: from='osd.3 [v2:172.21.15.77:6810/1454440477,v1:172.21.15.77:6811/1454440477]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-12-01T01:12:50.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:49 smithi203 ceph-mon[103581]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-12-01T01:12:50.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:49 smithi203 ceph-mon[103581]: from='osd.3 [v2:172.21.15.77:6810/1454440477,v1:172.21.15.77:6811/1454440477]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-12-01T01:12:50.283 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi137:vg_nvme/lv_3 2023-12-01T01:12:50.654 INFO:journalctl@ceph.osd.3.smithi077.stdout:Dec 01 01:12:50 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-3[125398]: 2023-12-01T01:12:50.553+0000 7ff0d77c3700 -1 osd.3 0 waiting for initial osdmap 2023-12-01T01:12:50.654 INFO:journalctl@ceph.osd.3.smithi077.stdout:Dec 01 01:12:50 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-3[125398]: 2023-12-01T01:12:50.560+0000 7ff0d15b4700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-01T01:12:50.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:50 smithi077 ceph-mon[98867]: from='osd.3 [v2:172.21.15.77:6810/1454440477,v1:172.21.15.77:6811/1454440477]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-12-01T01:12:50.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:50 smithi077 ceph-mon[98867]: osdmap e23: 4 total, 3 up, 4 in 2023-12-01T01:12:50.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:50 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-01T01:12:50.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:50 smithi077 ceph-mon[98867]: from='osd.3 [v2:172.21.15.77:6810/1454440477,v1:172.21.15.77:6811/1454440477]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2023-12-01T01:12:50.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:50 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:50.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:50 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:50.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:50 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:12:50.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:50 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:12:50.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:50 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:50.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:50 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:12:50.656 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:50 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:12:50.656 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:50 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:50.656 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:50 smithi077 ceph-mon[98867]: from='osd.3 [v2:172.21.15.77:6810/1454440477,v1:172.21.15.77:6811/1454440477]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2023-12-01T01:12:50.656 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:50 smithi077 ceph-mon[98867]: osdmap e24: 4 total, 3 up, 4 in 2023-12-01T01:12:50.656 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:50 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-01T01:12:50.656 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:50 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:12:50.656 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:50 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:12:50.656 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:50 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:12:50.656 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:50 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-01T01:12:50.877 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.b/config 2023-12-01T01:12:51.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:50 smithi203 ceph-mon[103581]: from='osd.3 [v2:172.21.15.77:6810/1454440477,v1:172.21.15.77:6811/1454440477]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-12-01T01:12:51.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:50 smithi203 ceph-mon[103581]: osdmap e23: 4 total, 3 up, 4 in 2023-12-01T01:12:51.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:50 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-01T01:12:51.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:50 smithi203 ceph-mon[103581]: from='osd.3 [v2:172.21.15.77:6810/1454440477,v1:172.21.15.77:6811/1454440477]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2023-12-01T01:12:51.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:50 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:51.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:50 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:51.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:50 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:12:51.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:50 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:12:51.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:50 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:51.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:50 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:12:51.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:50 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:12:51.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:50 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:51.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:50 smithi203 ceph-mon[103581]: from='osd.3 [v2:172.21.15.77:6810/1454440477,v1:172.21.15.77:6811/1454440477]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2023-12-01T01:12:51.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:50 smithi203 ceph-mon[103581]: osdmap e24: 4 total, 3 up, 4 in 2023-12-01T01:12:51.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:50 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-01T01:12:51.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:50 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:12:51.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:50 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:12:51.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:50 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:12:51.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:50 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-01T01:12:51.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:50 smithi137 ceph-mon[103312]: from='osd.3 [v2:172.21.15.77:6810/1454440477,v1:172.21.15.77:6811/1454440477]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-12-01T01:12:51.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:50 smithi137 ceph-mon[103312]: osdmap e23: 4 total, 3 up, 4 in 2023-12-01T01:12:51.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:50 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-01T01:12:51.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:50 smithi137 ceph-mon[103312]: from='osd.3 [v2:172.21.15.77:6810/1454440477,v1:172.21.15.77:6811/1454440477]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2023-12-01T01:12:51.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:50 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:51.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:50 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:51.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:50 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:12:51.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:50 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:12:51.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:50 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:51.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:50 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:12:51.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:50 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:12:51.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:50 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:51.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:50 smithi137 ceph-mon[103312]: from='osd.3 [v2:172.21.15.77:6810/1454440477,v1:172.21.15.77:6811/1454440477]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2023-12-01T01:12:51.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:50 smithi137 ceph-mon[103312]: osdmap e24: 4 total, 3 up, 4 in 2023-12-01T01:12:51.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:50 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-01T01:12:51.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:50 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:12:51.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:50 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:12:51.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:50 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:12:51.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:50 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-01T01:12:51.777 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:51 smithi137 ceph-mon[103312]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-01T01:12:51.777 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:51 smithi137 ceph-mon[103312]: Detected new or changed devices on smithi077 2023-12-01T01:12:51.777 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:51 smithi137 ceph-mon[103312]: Adjusting osd_memory_target on smithi077 to 8584M 2023-12-01T01:12:51.777 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:51 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:51.777 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:51 smithi137 ceph-mon[103312]: osd.3 [v2:172.21.15.77:6810/1454440477,v1:172.21.15.77:6811/1454440477] boot 2023-12-01T01:12:51.777 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:51 smithi137 ceph-mon[103312]: osdmap e25: 4 total, 4 up, 4 in 2023-12-01T01:12:51.777 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:51 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-01T01:12:51.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:51 smithi077 ceph-mon[98867]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-01T01:12:51.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:51 smithi077 ceph-mon[98867]: Detected new or changed devices on smithi077 2023-12-01T01:12:51.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:51 smithi077 ceph-mon[98867]: Adjusting osd_memory_target on smithi077 to 8584M 2023-12-01T01:12:51.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:51 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:51.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:51 smithi077 ceph-mon[98867]: osd.3 [v2:172.21.15.77:6810/1454440477,v1:172.21.15.77:6811/1454440477] boot 2023-12-01T01:12:51.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:51 smithi077 ceph-mon[98867]: osdmap e25: 4 total, 4 up, 4 in 2023-12-01T01:12:51.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:51 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-01T01:12:52.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:51 smithi203 ceph-mon[103581]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-01T01:12:52.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:51 smithi203 ceph-mon[103581]: Detected new or changed devices on smithi077 2023-12-01T01:12:52.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:51 smithi203 ceph-mon[103581]: Adjusting osd_memory_target on smithi077 to 8584M 2023-12-01T01:12:52.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:51 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:12:52.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:51 smithi203 ceph-mon[103581]: osd.3 [v2:172.21.15.77:6810/1454440477,v1:172.21.15.77:6811/1454440477] boot 2023-12-01T01:12:52.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:51 smithi203 ceph-mon[103581]: osdmap e25: 4 total, 4 up, 4 in 2023-12-01T01:12:52.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:51 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-01T01:12:52.291 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:52.289+0000 7f49efc82700 1 -- 172.21.15.137:0/3297997432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49e8100910 msgr2=0x7f49e81081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:12:52.291 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:52.289+0000 7f49efc82700 1 --2- 172.21.15.137:0/3297997432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49e8100910 0x7f49e81081b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f49e000b0a0 tx=0x7f49e000b3b0 comp rx=0 tx=0).stop 2023-12-01T01:12:52.291 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:52.289+0000 7f49efc82700 1 -- 172.21.15.137:0/3297997432 shutdown_connections 2023-12-01T01:12:52.291 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:52.289+0000 7f49efc82700 1 --2- 172.21.15.137:0/3297997432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49e8100910 0x7f49e81081b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:12:52.291 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:52.289+0000 7f49efc82700 1 --2- 172.21.15.137:0/3297997432 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49e81012f0 0x7f49e8101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:12:52.291 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:52.289+0000 7f49efc82700 1 --2- 172.21.15.137:0/3297997432 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49e80fff50 0x7f49e81003d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:12:52.291 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:52.289+0000 7f49efc82700 1 -- 172.21.15.137:0/3297997432 >> 172.21.15.137:0/3297997432 conn(0x7f49e80faa40 msgr2=0x7f49e80fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:12:52.291 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:52.289+0000 7f49efc82700 1 -- 172.21.15.137:0/3297997432 shutdown_connections 2023-12-01T01:12:52.291 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:52.289+0000 7f49efc82700 1 -- 172.21.15.137:0/3297997432 wait complete. 2023-12-01T01:12:52.293 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:52.290+0000 7f49efc82700 1 Processor -- start 2023-12-01T01:12:52.293 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:52.290+0000 7f49efc82700 1 -- start start 2023-12-01T01:12:52.293 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:52.290+0000 7f49efc82700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49e80fff50 0x7f49e8195a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:12:52.293 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:52.290+0000 7f49efc82700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49e8100910 0x7f49e8195fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:12:52.294 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:52.290+0000 7f49efc82700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49e81012f0 0x7f49e819d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:12:52.294 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:52.290+0000 7f49efc82700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e819d5f0 con 0x7f49e80fff50 2023-12-01T01:12:52.294 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:52.290+0000 7f49efc82700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e819d730 con 0x7f49e8100910 2023-12-01T01:12:52.294 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:52.290+0000 7f49efc82700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e819d8c0 con 0x7f49e81012f0 2023-12-01T01:12:52.294 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:52.290+0000 7f49ed21d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49e8100910 0x7f49e8195fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:12:52.294 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:52.290+0000 7f49ed21d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49e8100910 0x7f49e8195fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37280/0 (socket says 172.21.15.137:37280) 2023-12-01T01:12:52.294 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:52.290+0000 7f49ed21d700 1 -- 172.21.15.137:0/3588703420 learned_addr learned my addr 172.21.15.137:0/3588703420 (peer_addr_for_me v2:172.21.15.137:0/0) 2023-12-01T01:12:52.294 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:52.290+0000 7f49ee21f700 1 --2- 172.21.15.137:0/3588703420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49e81012f0 0x7f49e819d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:12:52.294 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:52.290+0000 7f49eda1e700 1 --2- 172.21.15.137:0/3588703420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49e80fff50 0x7f49e8195a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:12:52.294 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:52.291+0000 7f49ed21d700 1 -- 172.21.15.137:0/3588703420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49e81012f0 msgr2=0x7f49e819d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:12:52.294 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:52.291+0000 7f49ed21d700 1 --2- 172.21.15.137:0/3588703420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49e81012f0 0x7f49e819d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:12:52.294 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:52.291+0000 7f49ed21d700 1 -- 172.21.15.137:0/3588703420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49e80fff50 msgr2=0x7f49e8195a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:12:52.294 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:52.291+0000 7f49ed21d700 1 --2- 172.21.15.137:0/3588703420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49e80fff50 0x7f49e8195a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:12:52.295 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:52.291+0000 7f49ed21d700 1 -- 172.21.15.137:0/3588703420 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e0009d00 con 0x7f49e8100910 2023-12-01T01:12:52.295 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:52.291+0000 7f49eda1e700 1 --2- 172.21.15.137:0/3588703420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49e80fff50 0x7f49e8195a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:12:52.295 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:52.291+0000 7f49ed21d700 1 --2- 172.21.15.137:0/3588703420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49e8100910 0x7f49e8195fa0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f49e400c970 tx=0x7f49e400cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:12:52.295 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:52.291+0000 7f49deffd700 1 -- 172.21.15.137:0/3588703420 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e4004220 con 0x7f49e8100910 2023-12-01T01:12:52.295 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:52.291+0000 7f49deffd700 1 -- 172.21.15.137:0/3588703420 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49e4007d60 con 0x7f49e8100910 2023-12-01T01:12:52.295 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:52.291+0000 7f49efc82700 1 -- 172.21.15.137:0/3588703420 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e819dba0 con 0x7f49e8100910 2023-12-01T01:12:52.295 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:52.291+0000 7f49deffd700 1 -- 172.21.15.137:0/3588703420 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e4003d00 con 0x7f49e8100910 2023-12-01T01:12:52.295 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:52.291+0000 7f49efc82700 1 -- 172.21.15.137:0/3588703420 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e819e0a0 con 0x7f49e8100910 2023-12-01T01:12:52.295 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:52.292+0000 7f49efc82700 1 -- 172.21.15.137:0/3588703420 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49e81072b0 con 0x7f49e8100910 2023-12-01T01:12:52.296 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:52.293+0000 7f49deffd700 1 -- 172.21.15.137:0/3588703420 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f49e4004380 con 0x7f49e8100910 2023-12-01T01:12:52.296 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:52.293+0000 7f49deffd700 1 --2- 172.21.15.137:0/3588703420 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f49d40a0520 0x7f49d40a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:12:52.296 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:52.294+0000 7f49deffd700 1 -- 172.21.15.137:0/3588703420 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3702+0+0 (secure 0 0 0) 0x7f49e40ca6d0 con 0x7f49e8100910 2023-12-01T01:12:52.296 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:52.294+0000 7f49eda1e700 1 --2- 172.21.15.137:0/3588703420 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f49d40a0520 0x7f49d40a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:12:52.296 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:52.294+0000 7f49eda1e700 1 --2- 172.21.15.137:0/3588703420 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f49d40a0520 0x7f49d40a29e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f49e8068320 tx=0x7f49d800a680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:12:52.298 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:52.297+0000 7f49deffd700 1 -- 172.21.15.137:0/3588703420 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f49e4095470 con 0x7f49e8100910 2023-12-01T01:12:52.493 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:12:52.491+0000 7f49efc82700 1 -- 172.21.15.137:0/3588703420 --> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f49e80611d0 con 0x7f49d40a0520 2023-12-01T01:12:52.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:52 smithi077 ceph-mon[98867]: purged_snaps scrub starts 2023-12-01T01:12:52.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:52 smithi077 ceph-mon[98867]: purged_snaps scrub ok 2023-12-01T01:12:52.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:52 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-01T01:12:52.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:52 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-01T01:12:52.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:52 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:12:52.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:52 smithi077 ceph-mon[98867]: osdmap e26: 4 total, 4 up, 4 in 2023-12-01T01:12:53.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:52 smithi203 ceph-mon[103581]: purged_snaps scrub starts 2023-12-01T01:12:53.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:52 smithi203 ceph-mon[103581]: purged_snaps scrub ok 2023-12-01T01:12:53.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:52 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-01T01:12:53.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:52 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-01T01:12:53.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:52 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:12:53.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:52 smithi203 ceph-mon[103581]: osdmap e26: 4 total, 4 up, 4 in 2023-12-01T01:12:53.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:52 smithi137 ceph-mon[103312]: purged_snaps scrub starts 2023-12-01T01:12:53.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:52 smithi137 ceph-mon[103312]: purged_snaps scrub ok 2023-12-01T01:12:53.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:52 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-01T01:12:53.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:52 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-01T01:12:53.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:52 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:12:53.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:52 smithi137 ceph-mon[103312]: osdmap e26: 4 total, 4 up, 4 in 2023-12-01T01:12:53.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:53 smithi137 ceph-mon[103312]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-01T01:12:53.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:53 smithi137 ceph-mon[103312]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:12:53.645 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:53 smithi137 ceph-mon[103312]: osdmap e27: 4 total, 4 up, 4 in 2023-12-01T01:12:53.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:53 smithi077 ceph-mon[98867]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-01T01:12:53.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:53 smithi077 ceph-mon[98867]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:12:53.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:53 smithi077 ceph-mon[98867]: osdmap e27: 4 total, 4 up, 4 in 2023-12-01T01:12:54.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:53 smithi203 ceph-mon[103581]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-01T01:12:54.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:53 smithi203 ceph-mon[103581]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:12:54.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:53 smithi203 ceph-mon[103581]: osdmap e27: 4 total, 4 up, 4 in 2023-12-01T01:12:55.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:55 smithi077 ceph-mon[98867]: pgmap v92: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-01T01:12:55.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:55 smithi077 ceph-mon[98867]: osdmap e28: 4 total, 4 up, 4 in 2023-12-01T01:12:55.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:55 smithi077 ceph-mon[98867]: from='client.? 172.21.15.137:0/315205083' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e3ac1ba-98bb-4b8b-8e26-50064c66bf2e"}]: dispatch 2023-12-01T01:12:55.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:55 smithi077 ceph-mon[98867]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e3ac1ba-98bb-4b8b-8e26-50064c66bf2e"}]: dispatch 2023-12-01T01:12:55.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:55 smithi077 ceph-mon[98867]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e3ac1ba-98bb-4b8b-8e26-50064c66bf2e"}]': finished 2023-12-01T01:12:55.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:55 smithi077 ceph-mon[98867]: osdmap e29: 5 total, 4 up, 5 in 2023-12-01T01:12:55.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:55 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-01T01:12:56.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:55 smithi203 ceph-mon[103581]: pgmap v92: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-01T01:12:56.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:55 smithi203 ceph-mon[103581]: osdmap e28: 4 total, 4 up, 4 in 2023-12-01T01:12:56.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:55 smithi203 ceph-mon[103581]: from='client.? 172.21.15.137:0/315205083' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e3ac1ba-98bb-4b8b-8e26-50064c66bf2e"}]: dispatch 2023-12-01T01:12:56.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:55 smithi203 ceph-mon[103581]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e3ac1ba-98bb-4b8b-8e26-50064c66bf2e"}]: dispatch 2023-12-01T01:12:56.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:55 smithi203 ceph-mon[103581]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e3ac1ba-98bb-4b8b-8e26-50064c66bf2e"}]': finished 2023-12-01T01:12:56.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:55 smithi203 ceph-mon[103581]: osdmap e29: 5 total, 4 up, 5 in 2023-12-01T01:12:56.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:55 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-01T01:12:56.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:55 smithi137 ceph-mon[103312]: pgmap v92: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-01T01:12:56.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:55 smithi137 ceph-mon[103312]: osdmap e28: 4 total, 4 up, 4 in 2023-12-01T01:12:56.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:55 smithi137 ceph-mon[103312]: from='client.? 172.21.15.137:0/315205083' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e3ac1ba-98bb-4b8b-8e26-50064c66bf2e"}]: dispatch 2023-12-01T01:12:56.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:55 smithi137 ceph-mon[103312]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e3ac1ba-98bb-4b8b-8e26-50064c66bf2e"}]: dispatch 2023-12-01T01:12:56.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:55 smithi137 ceph-mon[103312]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e3ac1ba-98bb-4b8b-8e26-50064c66bf2e"}]': finished 2023-12-01T01:12:56.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:55 smithi137 ceph-mon[103312]: osdmap e29: 5 total, 4 up, 5 in 2023-12-01T01:12:56.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:55 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-01T01:12:56.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:56 smithi077 ceph-mon[98867]: from='client.? 172.21.15.137:0/1170194189' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-01T01:12:57.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:56 smithi203 ceph-mon[103581]: from='client.? 172.21.15.137:0/1170194189' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-01T01:12:57.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:56 smithi137 ceph-mon[103312]: from='client.? 172.21.15.137:0/1170194189' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-01T01:12:57.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:57 smithi077 ceph-mon[98867]: pgmap v95: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-01T01:12:58.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:57 smithi203 ceph-mon[103581]: pgmap v95: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-01T01:12:58.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:57 smithi137 ceph-mon[103312]: pgmap v95: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-01T01:12:59.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:12:59 smithi077 ceph-mon[98867]: pgmap v96: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2023-12-01T01:13:00.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:12:59 smithi203 ceph-mon[103581]: pgmap v96: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2023-12-01T01:13:00.017 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:12:59 smithi137 ceph-mon[103312]: pgmap v96: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2023-12-01T01:13:01.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:01 smithi077 ceph-mon[98867]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2023-12-01T01:13:02.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:01 smithi203 ceph-mon[103581]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2023-12-01T01:13:02.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:01 smithi137 ceph-mon[103312]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2023-12-01T01:13:02.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:02 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-12-01T01:13:02.889 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:02 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:13:02.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:02 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-12-01T01:13:02.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:02 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:13:03.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:02 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-12-01T01:13:03.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:02 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:13:03.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:03 smithi077 ceph-mon[98867]: Deploying daemon osd.4 on smithi137 2023-12-01T01:13:03.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:03 smithi077 ceph-mon[98867]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-01T01:13:03.928 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:03 smithi137 ceph-mon[103312]: Deploying daemon osd.4 on smithi137 2023-12-01T01:13:03.928 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:03 smithi137 ceph-mon[103312]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-01T01:13:04.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:03 smithi203 ceph-mon[103581]: Deploying daemon osd.4 on smithi137 2023-12-01T01:13:04.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:03 smithi203 ceph-mon[103581]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-01T01:13:04.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:04 smithi077 ceph-mon[98867]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-01T01:13:05.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:04 smithi203 ceph-mon[103581]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-01T01:13:05.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:04 smithi137 ceph-mon[103312]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-01T01:13:07.467 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:07 smithi137 ceph-mon[103312]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-01T01:13:07.467 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:07 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:07.467 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:07 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:07.467 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:07 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:13:07.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:07 smithi077 ceph-mon[98867]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-01T01:13:07.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:07 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:07.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:07 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:07.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:07 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:13:07.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:07 smithi203 ceph-mon[103581]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-01T01:13:07.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:07 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:07.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:07 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:07.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:07 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:13:09.360 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 4 on host 'smithi137' 2023-12-01T01:13:09.361 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:09.358+0000 7f49deffd700 1 -- 172.21.15.137:0/3588703420 <== mgr.14152 v2:172.21.15.77:6800/3838941679 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f49e80611d0 con 0x7f49d40a0520 2023-12-01T01:13:09.363 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:09.361+0000 7f49efc82700 1 -- 172.21.15.137:0/3588703420 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f49d40a0520 msgr2=0x7f49d40a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:13:09.363 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:09.361+0000 7f49efc82700 1 --2- 172.21.15.137:0/3588703420 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f49d40a0520 0x7f49d40a29e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f49e8068320 tx=0x7f49d800a680 comp rx=0 tx=0).stop 2023-12-01T01:13:09.364 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:09.361+0000 7f49efc82700 1 -- 172.21.15.137:0/3588703420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49e8100910 msgr2=0x7f49e8195fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:13:09.364 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:09.361+0000 7f49efc82700 1 --2- 172.21.15.137:0/3588703420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49e8100910 0x7f49e8195fa0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f49e400c970 tx=0x7f49e400cc80 comp rx=0 tx=0).stop 2023-12-01T01:13:09.364 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:09.361+0000 7f49efc82700 1 -- 172.21.15.137:0/3588703420 shutdown_connections 2023-12-01T01:13:09.364 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:09.361+0000 7f49efc82700 1 --2- 172.21.15.137:0/3588703420 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f49d40a0520 0x7f49d40a29e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:13:09.364 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:09.361+0000 7f49efc82700 1 --2- 172.21.15.137:0/3588703420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49e81012f0 0x7f49e819d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:13:09.364 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:09.361+0000 7f49efc82700 1 --2- 172.21.15.137:0/3588703420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49e80fff50 0x7f49e8195a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:13:09.364 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:09.361+0000 7f49efc82700 1 --2- 172.21.15.137:0/3588703420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49e8100910 0x7f49e8195fa0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:13:09.364 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:09.361+0000 7f49efc82700 1 -- 172.21.15.137:0/3588703420 >> 172.21.15.137:0/3588703420 conn(0x7f49e80faa40 msgr2=0x7f49e81043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:13:09.364 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:09.361+0000 7f49efc82700 1 -- 172.21.15.137:0/3588703420 shutdown_connections 2023-12-01T01:13:09.364 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:09.361+0000 7f49efc82700 1 -- 172.21.15.137:0/3588703420 wait complete. 2023-12-01T01:13:09.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:09 smithi137 ceph-mon[103312]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-01T01:13:09.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:09 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:09.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:09 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:09.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:09 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:13:09.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:09 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:13:09.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:09 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:09.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:09 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:09.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:09 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:09.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:09 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:13:09.759 DEBUG:teuthology.orchestra.run.smithi137:osd.4> sudo journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@osd.4.service 2023-12-01T01:13:09.761 INFO:tasks.cephadm:Deploying osd.5 on smithi203 with /dev/vg_nvme/lv_3... 2023-12-01T01:13:09.761 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-12-01T01:13:09.790 INFO:journalctl@ceph.osd.4.smithi137.stdout:-- Logs begin at Fri 2023-12-01 00:55:04 UTC. -- 2023-12-01T01:13:09.794 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:09 smithi203 ceph-mon[103581]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-01T01:13:09.794 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:09 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:09.794 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:09 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:09.794 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:09 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:13:09.794 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:09 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:13:09.794 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:09 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:09.794 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:09 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:09.794 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:09 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:09.794 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:09 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:13:09.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:09 smithi077 ceph-mon[98867]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-01T01:13:09.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:09 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:09.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:09 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:09.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:09 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:13:09.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:09 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:13:09.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:09 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:09.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:09 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:09.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:09 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:09.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:09 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:13:10.324 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.c/config 2023-12-01T01:13:10.523 INFO:journalctl@ceph.osd.4.smithi137.stdout:Dec 01 01:13:10 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-4[116898]: 2023-12-01T01:13:10.218+0000 7f694c254640 -1 osd.4 0 log_to_monitors true 2023-12-01T01:13:10.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:10 smithi137 ceph-mon[103312]: from='osd.4 [v2:172.21.15.137:6808/1103241451,v1:172.21.15.137:6809/1103241451]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-01T01:13:10.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:10 smithi137 ceph-mon[103312]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-01T01:13:10.859 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:10 smithi203 ceph-mon[103581]: from='osd.4 [v2:172.21.15.137:6808/1103241451,v1:172.21.15.137:6809/1103241451]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-01T01:13:10.859 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:10 smithi203 ceph-mon[103581]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-01T01:13:10.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:10 smithi077 ceph-mon[98867]: from='osd.4 [v2:172.21.15.137:6808/1103241451,v1:172.21.15.137:6809/1103241451]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-01T01:13:10.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:10 smithi077 ceph-mon[98867]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-01T01:13:11.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:11 smithi203 ceph-mon[103581]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-01T01:13:11.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:11 smithi203 ceph-mon[103581]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-12-01T01:13:11.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:11 smithi203 ceph-mon[103581]: osdmap e30: 5 total, 4 up, 5 in 2023-12-01T01:13:11.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:11 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-01T01:13:11.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:11 smithi203 ceph-mon[103581]: from='osd.4 [v2:172.21.15.137:6808/1103241451,v1:172.21.15.137:6809/1103241451]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2023-12-01T01:13:11.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:11 smithi203 ceph-mon[103581]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2023-12-01T01:13:11.889 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-12-01T01:13:11.889 INFO:teuthology.orchestra.run.smithi203.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-12-01T01:13:11.889 INFO:teuthology.orchestra.run.smithi203.stderr: stderr: 10+0 records in 2023-12-01T01:13:11.889 INFO:teuthology.orchestra.run.smithi203.stderr:10+0 records out 2023-12-01T01:13:11.889 INFO:teuthology.orchestra.run.smithi203.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0110143 s, 952 MB/s 2023-12-01T01:13:11.889 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping successful for: 2023-12-01T01:13:11.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:11 smithi077 ceph-mon[98867]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-01T01:13:11.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:11 smithi077 ceph-mon[98867]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-12-01T01:13:11.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:11 smithi077 ceph-mon[98867]: osdmap e30: 5 total, 4 up, 5 in 2023-12-01T01:13:11.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:11 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-01T01:13:11.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:11 smithi077 ceph-mon[98867]: from='osd.4 [v2:172.21.15.137:6808/1103241451,v1:172.21.15.137:6809/1103241451]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2023-12-01T01:13:11.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:11 smithi077 ceph-mon[98867]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2023-12-01T01:13:12.011 INFO:journalctl@ceph.osd.4.smithi137.stdout:Dec 01 01:13:11 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-4[116898]: 2023-12-01T01:13:11.535+0000 7f69410ca700 -1 osd.4 0 waiting for initial osdmap 2023-12-01T01:13:12.011 INFO:journalctl@ceph.osd.4.smithi137.stdout:Dec 01 01:13:11 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-4[116898]: 2023-12-01T01:13:11.542+0000 7f693cebf700 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-01T01:13:12.011 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:11 smithi137 ceph-mon[103312]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-01T01:13:12.011 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:11 smithi137 ceph-mon[103312]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-12-01T01:13:12.012 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:11 smithi137 ceph-mon[103312]: osdmap e30: 5 total, 4 up, 5 in 2023-12-01T01:13:12.012 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:11 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-01T01:13:12.012 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:11 smithi137 ceph-mon[103312]: from='osd.4 [v2:172.21.15.137:6808/1103241451,v1:172.21.15.137:6809/1103241451]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2023-12-01T01:13:12.012 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:11 smithi137 ceph-mon[103312]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2023-12-01T01:13:12.279 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi203:vg_nvme/lv_3 2023-12-01T01:13:12.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:12 smithi137 ceph-mon[103312]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2023-12-01T01:13:12.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:12 smithi137 ceph-mon[103312]: osdmap e31: 5 total, 4 up, 5 in 2023-12-01T01:13:12.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:12 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-01T01:13:12.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:12 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-01T01:13:12.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:12 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:12.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:12 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:12.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:12 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:13:12.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:12 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:13:12.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:12 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:12.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:12 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:13:12.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:12 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:13:12.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:12 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:12.876 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.c/config 2023-12-01T01:13:12.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:12 smithi077 ceph-mon[98867]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2023-12-01T01:13:12.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:12 smithi077 ceph-mon[98867]: osdmap e31: 5 total, 4 up, 5 in 2023-12-01T01:13:12.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:12 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-01T01:13:12.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:12 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-01T01:13:12.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:12 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:12.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:12 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:12.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:12 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:13:12.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:12 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:13:12.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:12 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:12.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:12 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:13:12.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:12 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:13:12.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:12 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:13.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:12 smithi203 ceph-mon[103581]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2023-12-01T01:13:13.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:12 smithi203 ceph-mon[103581]: osdmap e31: 5 total, 4 up, 5 in 2023-12-01T01:13:13.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:12 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-01T01:13:13.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:12 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-01T01:13:13.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:12 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:13.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:12 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:13.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:12 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:13:13.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:12 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:13:13.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:12 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:13.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:12 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:13:13.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:12 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:13:13.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:12 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:13.729 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:13 smithi203 ceph-mon[103581]: purged_snaps scrub starts 2023-12-01T01:13:13.729 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:13 smithi203 ceph-mon[103581]: purged_snaps scrub ok 2023-12-01T01:13:13.730 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:13 smithi203 ceph-mon[103581]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-01T01:13:13.730 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:13 smithi203 ceph-mon[103581]: Detected new or changed devices on smithi137 2023-12-01T01:13:13.730 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:13 smithi203 ceph-mon[103581]: Adjusting osd_memory_target on smithi137 to 8584M 2023-12-01T01:13:13.730 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:13 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-01T01:13:13.730 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:13 smithi203 ceph-mon[103581]: osd.4 [v2:172.21.15.137:6808/1103241451,v1:172.21.15.137:6809/1103241451] boot 2023-12-01T01:13:13.730 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:13 smithi203 ceph-mon[103581]: osdmap e32: 5 total, 5 up, 5 in 2023-12-01T01:13:13.730 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:13 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-01T01:13:13.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:13 smithi077 ceph-mon[98867]: purged_snaps scrub starts 2023-12-01T01:13:13.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:13 smithi077 ceph-mon[98867]: purged_snaps scrub ok 2023-12-01T01:13:13.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:13 smithi077 ceph-mon[98867]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-01T01:13:13.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:13 smithi077 ceph-mon[98867]: Detected new or changed devices on smithi137 2023-12-01T01:13:13.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:13 smithi077 ceph-mon[98867]: Adjusting osd_memory_target on smithi137 to 8584M 2023-12-01T01:13:13.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:13 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-01T01:13:13.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:13 smithi077 ceph-mon[98867]: osd.4 [v2:172.21.15.137:6808/1103241451,v1:172.21.15.137:6809/1103241451] boot 2023-12-01T01:13:13.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:13 smithi077 ceph-mon[98867]: osdmap e32: 5 total, 5 up, 5 in 2023-12-01T01:13:13.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:13 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-01T01:13:14.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:13 smithi137 ceph-mon[103312]: purged_snaps scrub starts 2023-12-01T01:13:14.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:13 smithi137 ceph-mon[103312]: purged_snaps scrub ok 2023-12-01T01:13:14.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:13 smithi137 ceph-mon[103312]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-01T01:13:14.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:13 smithi137 ceph-mon[103312]: Detected new or changed devices on smithi137 2023-12-01T01:13:14.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:13 smithi137 ceph-mon[103312]: Adjusting osd_memory_target on smithi137 to 8584M 2023-12-01T01:13:14.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:13 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-01T01:13:14.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:13 smithi137 ceph-mon[103312]: osd.4 [v2:172.21.15.137:6808/1103241451,v1:172.21.15.137:6809/1103241451] boot 2023-12-01T01:13:14.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:13 smithi137 ceph-mon[103312]: osdmap e32: 5 total, 5 up, 5 in 2023-12-01T01:13:14.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:13 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-01T01:13:14.200 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:14.196+0000 7f8820b18700 1 -- 172.21.15.203:0/2427299617 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f881c100e50 msgr2=0x7f881c101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:13:14.200 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:14.196+0000 7f8820b18700 1 --2- 172.21.15.203:0/2427299617 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f881c100e50 0x7f881c101270 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f880c009990 tx=0x7f880c009ca0 comp rx=0 tx=0).stop 2023-12-01T01:13:14.200 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:14.196+0000 7f8820b18700 1 -- 172.21.15.203:0/2427299617 shutdown_connections 2023-12-01T01:13:14.200 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:14.196+0000 7f8820b18700 1 --2- 172.21.15.203:0/2427299617 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f881c102970 0x7f881c109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:13:14.200 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:14.196+0000 7f8820b18700 1 --2- 172.21.15.203:0/2427299617 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f881c100e50 0x7f881c101270 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:13:14.200 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:14.196+0000 7f8820b18700 1 --2- 172.21.15.203:0/2427299617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f881c101fb0 0x7f881c102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:13:14.200 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:14.196+0000 7f8820b18700 1 -- 172.21.15.203:0/2427299617 >> 172.21.15.203:0/2427299617 conn(0x7f881c0fc410 msgr2=0x7f881c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:13:14.200 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:14.196+0000 7f8820b18700 1 -- 172.21.15.203:0/2427299617 shutdown_connections 2023-12-01T01:13:14.200 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:14.196+0000 7f8820b18700 1 -- 172.21.15.203:0/2427299617 wait complete. 2023-12-01T01:13:14.200 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:14.197+0000 7f8820b18700 1 Processor -- start 2023-12-01T01:13:14.200 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:14.197+0000 7f8820b18700 1 -- start start 2023-12-01T01:13:14.200 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:14.197+0000 7f8820b18700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f881c100e50 0x7f881c074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:13:14.201 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:14.197+0000 7f8820b18700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f881c101fb0 0x7f881c073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:13:14.201 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:14.197+0000 7f8820b18700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f881c102970 0x7f881c0736c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:13:14.201 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:14.197+0000 7f8820b18700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881c073e40 con 0x7f881c100e50 2023-12-01T01:13:14.201 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:14.197+0000 7f8820b18700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881c073fb0 con 0x7f881c102970 2023-12-01T01:13:14.201 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:14.197+0000 7f8820b18700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881c1a0b00 con 0x7f881c101fb0 2023-12-01T01:13:14.201 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:14.197+0000 7f8819d9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f881c101fb0 0x7f881c073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:13:14.201 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:14.197+0000 7f8819d9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f881c101fb0 0x7f881c073180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:59278/0 (socket says 172.21.15.203:59278) 2023-12-01T01:13:14.201 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:14.197+0000 7f8819d9b700 1 -- 172.21.15.203:0/262687018 learned_addr learned my addr 172.21.15.203:0/262687018 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-12-01T01:13:14.201 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:14.197+0000 7f881a59c700 1 --2- 172.21.15.203:0/262687018 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f881c100e50 0x7f881c074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:13:14.201 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:14.198+0000 7f881ad9d700 1 --2- 172.21.15.203:0/262687018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f881c102970 0x7f881c0736c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:13:14.201 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:14.198+0000 7f8819d9b700 1 -- 172.21.15.203:0/262687018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f881c102970 msgr2=0x7f881c0736c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:13:14.201 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:14.198+0000 7f8819d9b700 1 --2- 172.21.15.203:0/262687018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f881c102970 0x7f881c0736c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:13:14.201 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:14.198+0000 7f8819d9b700 1 -- 172.21.15.203:0/262687018 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f881c100e50 msgr2=0x7f881c074b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:13:14.201 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:14.198+0000 7f8819d9b700 1 --2- 172.21.15.203:0/262687018 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f881c100e50 0x7f881c074b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:13:14.202 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:14.198+0000 7f8819d9b700 1 -- 172.21.15.203:0/262687018 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f880c0095f0 con 0x7f881c101fb0 2023-12-01T01:13:14.202 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:14.198+0000 7f881a59c700 1 --2- 172.21.15.203:0/262687018 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f881c100e50 0x7f881c074b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:13:14.202 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:14.198+0000 7f8819d9b700 1 --2- 172.21.15.203:0/262687018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f881c101fb0 0x7f881c073180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f881000e9c0 tx=0x7f881000ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:13:14.202 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:14.198+0000 7f880b7fe700 1 -- 172.21.15.203:0/262687018 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88100097b0 con 0x7f881c101fb0 2023-12-01T01:13:14.202 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:14.198+0000 7f8820b18700 1 -- 172.21.15.203:0/262687018 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f881c1a0de0 con 0x7f881c101fb0 2023-12-01T01:13:14.202 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:14.198+0000 7f880b7fe700 1 -- 172.21.15.203:0/262687018 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8810004cf0 con 0x7f881c101fb0 2023-12-01T01:13:14.202 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:14.198+0000 7f880b7fe700 1 -- 172.21.15.203:0/262687018 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8810005700 con 0x7f881c101fb0 2023-12-01T01:13:14.202 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:14.198+0000 7f8820b18700 1 -- 172.21.15.203:0/262687018 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f881c1a1200 con 0x7f881c101fb0 2023-12-01T01:13:14.202 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:14.199+0000 7f8820b18700 1 -- 172.21.15.203:0/262687018 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f881c1083e0 con 0x7f881c101fb0 2023-12-01T01:13:14.203 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:14.200+0000 7f880b7fe700 1 -- 172.21.15.203:0/262687018 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f881000cbd0 con 0x7f881c101fb0 2023-12-01T01:13:14.203 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:14.200+0000 7f880b7fe700 1 --2- 172.21.15.203:0/262687018 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f88040a0520 0x7f88040a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:13:14.203 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:14.201+0000 7f880b7fe700 1 -- 172.21.15.203:0/262687018 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f8810014070 con 0x7f881c101fb0 2023-12-01T01:13:14.203 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:14.201+0000 7f881a59c700 1 --2- 172.21.15.203:0/262687018 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f88040a0520 0x7f88040a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:13:14.204 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:14.201+0000 7f881a59c700 1 --2- 172.21.15.203:0/262687018 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f88040a0520 0x7f88040a29e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f880c00b2e0 tx=0x7f880c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:13:14.205 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:14.205+0000 7f880b7fe700 1 -- 172.21.15.203:0/262687018 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8810099360 con 0x7f881c101fb0 2023-12-01T01:13:14.401 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:14.400+0000 7f8820b18700 1 -- 172.21.15.203:0/262687018 --> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f881c0611d0 con 0x7f88040a0520 2023-12-01T01:13:14.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:14 smithi077 ceph-mon[98867]: osdmap e33: 5 total, 5 up, 5 in 2023-12-01T01:13:14.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:14 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-01T01:13:14.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:14 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-01T01:13:14.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:14 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:13:15.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:14 smithi203 ceph-mon[103581]: osdmap e33: 5 total, 5 up, 5 in 2023-12-01T01:13:15.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:14 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-01T01:13:15.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:14 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-01T01:13:15.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:14 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:13:15.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:14 smithi137 ceph-mon[103312]: osdmap e33: 5 total, 5 up, 5 in 2023-12-01T01:13:15.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:14 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-01T01:13:15.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:14 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-01T01:13:15.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:14 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:13:15.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:15 smithi077 ceph-mon[98867]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-01T01:13:15.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:15 smithi077 ceph-mon[98867]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:13:16.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:15 smithi203 ceph-mon[103581]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-01T01:13:16.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:15 smithi203 ceph-mon[103581]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:13:16.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:15 smithi137 ceph-mon[103312]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-01T01:13:16.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:15 smithi137 ceph-mon[103312]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:13:17.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:17 smithi077 ceph-mon[98867]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-01T01:13:17.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:17 smithi077 ceph-mon[98867]: from='client.? 172.21.15.203:0/2954488037' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb32c016-abeb-44ab-abf0-19dd09235736"}]: dispatch 2023-12-01T01:13:17.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:17 smithi077 ceph-mon[98867]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb32c016-abeb-44ab-abf0-19dd09235736"}]: dispatch 2023-12-01T01:13:17.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:17 smithi077 ceph-mon[98867]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb32c016-abeb-44ab-abf0-19dd09235736"}]': finished 2023-12-01T01:13:17.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:17 smithi077 ceph-mon[98867]: osdmap e34: 6 total, 5 up, 6 in 2023-12-01T01:13:17.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:17 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-01T01:13:17.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:17 smithi077 ceph-mon[98867]: from='client.? 172.21.15.203:0/2574525700' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-01T01:13:18.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:17 smithi203 ceph-mon[103581]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-01T01:13:18.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:17 smithi203 ceph-mon[103581]: from='client.? 172.21.15.203:0/2954488037' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb32c016-abeb-44ab-abf0-19dd09235736"}]: dispatch 2023-12-01T01:13:18.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:17 smithi203 ceph-mon[103581]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb32c016-abeb-44ab-abf0-19dd09235736"}]: dispatch 2023-12-01T01:13:18.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:17 smithi203 ceph-mon[103581]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb32c016-abeb-44ab-abf0-19dd09235736"}]': finished 2023-12-01T01:13:18.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:17 smithi203 ceph-mon[103581]: osdmap e34: 6 total, 5 up, 6 in 2023-12-01T01:13:18.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:17 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-01T01:13:18.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:17 smithi203 ceph-mon[103581]: from='client.? 172.21.15.203:0/2574525700' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-01T01:13:18.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:17 smithi137 ceph-mon[103312]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-01T01:13:18.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:17 smithi137 ceph-mon[103312]: from='client.? 172.21.15.203:0/2954488037' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb32c016-abeb-44ab-abf0-19dd09235736"}]: dispatch 2023-12-01T01:13:18.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:17 smithi137 ceph-mon[103312]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb32c016-abeb-44ab-abf0-19dd09235736"}]: dispatch 2023-12-01T01:13:18.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:17 smithi137 ceph-mon[103312]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb32c016-abeb-44ab-abf0-19dd09235736"}]': finished 2023-12-01T01:13:18.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:17 smithi137 ceph-mon[103312]: osdmap e34: 6 total, 5 up, 6 in 2023-12-01T01:13:18.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:17 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-01T01:13:18.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:17 smithi137 ceph-mon[103312]: from='client.? 172.21.15.203:0/2574525700' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-01T01:13:19.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:19 smithi077 ceph-mon[98867]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-01T01:13:20.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:19 smithi203 ceph-mon[103581]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-01T01:13:20.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:19 smithi137 ceph-mon[103312]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-01T01:13:21.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:21 smithi077 ceph-mon[98867]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-01T01:13:22.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:21 smithi203 ceph-mon[103581]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-01T01:13:22.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:21 smithi137 ceph-mon[103312]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-01T01:13:23.738 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:23 smithi203 ceph-mon[103581]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-01T01:13:23.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:23 smithi077 ceph-mon[98867]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-01T01:13:24.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:23 smithi137 ceph-mon[103312]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-01T01:13:24.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:24 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-12-01T01:13:24.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:24 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:13:24.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:24 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-12-01T01:13:24.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:24 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:13:25.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:24 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-12-01T01:13:25.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:24 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:13:25.734 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:25 smithi203 ceph-mon[103581]: Deploying daemon osd.5 on smithi203 2023-12-01T01:13:25.734 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:25 smithi203 ceph-mon[103581]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-01T01:13:25.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:25 smithi077 ceph-mon[98867]: Deploying daemon osd.5 on smithi203 2023-12-01T01:13:25.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:25 smithi077 ceph-mon[98867]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-01T01:13:26.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:25 smithi137 ceph-mon[103312]: Deploying daemon osd.5 on smithi203 2023-12-01T01:13:26.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:25 smithi137 ceph-mon[103312]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-01T01:13:27.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:27 smithi077 ceph-mon[98867]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-01T01:13:28.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:27 smithi203 ceph-mon[103581]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-01T01:13:28.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:27 smithi137 ceph-mon[103312]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-01T01:13:29.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:29 smithi203 ceph-mon[103581]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-01T01:13:29.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:29 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:29.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:29 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:29.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:29 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:13:29.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:29 smithi077 ceph-mon[98867]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-01T01:13:29.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:29 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:29.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:29 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:29.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:29 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:13:29.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:29 smithi137 ceph-mon[103312]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-01T01:13:29.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:29 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:29.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:29 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:29.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:29 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:13:31.463 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:31.461+0000 7f880b7fe700 1 -- 172.21.15.203:0/262687018 <== mgr.14152 v2:172.21.15.77:6800/3838941679 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f881c0611d0 con 0x7f88040a0520 2023-12-01T01:13:31.463 INFO:teuthology.orchestra.run.smithi203.stdout:Created osd(s) 5 on host 'smithi203' 2023-12-01T01:13:31.465 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:31.464+0000 7f8820b18700 1 -- 172.21.15.203:0/262687018 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f88040a0520 msgr2=0x7f88040a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:13:31.465 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:31.464+0000 7f8820b18700 1 --2- 172.21.15.203:0/262687018 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f88040a0520 0x7f88040a29e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f880c00b2e0 tx=0x7f880c005fd0 comp rx=0 tx=0).stop 2023-12-01T01:13:31.465 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:31.464+0000 7f8820b18700 1 -- 172.21.15.203:0/262687018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f881c101fb0 msgr2=0x7f881c073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:13:31.465 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:31.464+0000 7f8820b18700 1 --2- 172.21.15.203:0/262687018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f881c101fb0 0x7f881c073180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f881000e9c0 tx=0x7f881000ed80 comp rx=0 tx=0).stop 2023-12-01T01:13:31.465 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:31.464+0000 7f8820b18700 1 -- 172.21.15.203:0/262687018 shutdown_connections 2023-12-01T01:13:31.465 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:31.464+0000 7f8820b18700 1 --2- 172.21.15.203:0/262687018 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f88040a0520 0x7f88040a29e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:13:31.465 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:31.464+0000 7f8820b18700 1 --2- 172.21.15.203:0/262687018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f881c102970 0x7f881c0736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:13:31.465 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:31.464+0000 7f8820b18700 1 --2- 172.21.15.203:0/262687018 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f881c100e50 0x7f881c074b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:13:31.465 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:31.464+0000 7f8820b18700 1 --2- 172.21.15.203:0/262687018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f881c101fb0 0x7f881c073180 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:13:31.466 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:31.464+0000 7f8820b18700 1 -- 172.21.15.203:0/262687018 >> 172.21.15.203:0/262687018 conn(0x7f881c0fc410 msgr2=0x7f881c105520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:13:31.466 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:31.464+0000 7f8820b18700 1 -- 172.21.15.203:0/262687018 shutdown_connections 2023-12-01T01:13:31.466 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:13:31.464+0000 7f8820b18700 1 -- 172.21.15.203:0/262687018 wait complete. 2023-12-01T01:13:31.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:31 smithi077 ceph-mon[98867]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-01T01:13:31.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:31 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:31.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:31 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:31.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:31 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:13:31.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:31 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:13:31.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:31 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:31.656 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:31 smithi203 ceph-mon[103581]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-01T01:13:31.656 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:31 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:31.656 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:31 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:31.656 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:31 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:13:31.656 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:31 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:13:31.656 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:31 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:31.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:31 smithi137 ceph-mon[103312]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-01T01:13:31.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:31 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:31.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:31 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:31.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:31 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:13:31.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:31 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:13:31.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:31 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:31.964 DEBUG:teuthology.orchestra.run.smithi203:osd.5> sudo journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@osd.5.service 2023-12-01T01:13:31.965 INFO:tasks.cephadm:Deploying osd.6 on smithi077 with /dev/vg_nvme/lv_2... 2023-12-01T01:13:31.966 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-12-01T01:13:32.000 INFO:journalctl@ceph.osd.5.smithi203.stdout:-- Logs begin at Fri 2023-12-01 00:55:29 UTC. -- 2023-12-01T01:13:32.533 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:13:32.639 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:32 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:32.640 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:32 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:32.640 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:32 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:13:32.640 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:32 smithi203 ceph-mon[103581]: from='osd.5 [v2:172.21.15.203:6808/2648569482,v1:172.21.15.203:6809/2648569482]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-01T01:13:32.640 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:32 smithi203 ceph-mon[103581]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-01T01:13:32.640 INFO:journalctl@ceph.osd.5.smithi203.stdout:Dec 01 01:13:32 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-5[118084]: 2023-12-01T01:13:32.336+0000 7f501864b640 -1 osd.5 0 log_to_monitors true 2023-12-01T01:13:32.717 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:32 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:32.717 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:32 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:32.717 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:32 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:13:32.717 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:32 smithi077 ceph-mon[98867]: from='osd.5 [v2:172.21.15.203:6808/2648569482,v1:172.21.15.203:6809/2648569482]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-01T01:13:32.718 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:32 smithi077 ceph-mon[98867]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-01T01:13:32.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:32 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:32.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:32 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:32.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:32 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:13:32.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:32 smithi137 ceph-mon[103312]: from='osd.5 [v2:172.21.15.203:6808/2648569482,v1:172.21.15.203:6809/2648569482]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-01T01:13:32.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:32 smithi137 ceph-mon[103312]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-01T01:13:33.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:33 smithi077 ceph-mon[98867]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-01T01:13:33.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:33 smithi077 ceph-mon[98867]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-12-01T01:13:33.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:33 smithi077 ceph-mon[98867]: osdmap e35: 6 total, 5 up, 6 in 2023-12-01T01:13:33.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:33 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-01T01:13:33.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:33 smithi077 ceph-mon[98867]: from='osd.5 [v2:172.21.15.203:6808/2648569482,v1:172.21.15.203:6809/2648569482]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-12-01T01:13:33.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:33 smithi077 ceph-mon[98867]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-12-01T01:13:34.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:33 smithi203 ceph-mon[103581]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-01T01:13:34.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:33 smithi203 ceph-mon[103581]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-12-01T01:13:34.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:33 smithi203 ceph-mon[103581]: osdmap e35: 6 total, 5 up, 6 in 2023-12-01T01:13:34.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:33 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-01T01:13:34.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:33 smithi203 ceph-mon[103581]: from='osd.5 [v2:172.21.15.203:6808/2648569482,v1:172.21.15.203:6809/2648569482]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-12-01T01:13:34.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:33 smithi203 ceph-mon[103581]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-12-01T01:13:34.013 INFO:journalctl@ceph.osd.5.smithi203.stdout:Dec 01 01:13:33 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-5[118084]: 2023-12-01T01:13:33.572+0000 7f500d4c1700 -1 osd.5 0 waiting for initial osdmap 2023-12-01T01:13:34.013 INFO:journalctl@ceph.osd.5.smithi203.stdout:Dec 01 01:13:33 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-5[118084]: 2023-12-01T01:13:33.580+0000 7f50072b2700 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-01T01:13:34.018 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-12-01T01:13:34.018 INFO:teuthology.orchestra.run.smithi077.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-12-01T01:13:34.018 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10+0 records in 2023-12-01T01:13:34.018 INFO:teuthology.orchestra.run.smithi077.stderr:10+0 records out 2023-12-01T01:13:34.018 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126447 s, 829 MB/s 2023-12-01T01:13:34.018 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping successful for: 2023-12-01T01:13:34.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:33 smithi137 ceph-mon[103312]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-01T01:13:34.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:33 smithi137 ceph-mon[103312]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-12-01T01:13:34.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:33 smithi137 ceph-mon[103312]: osdmap e35: 6 total, 5 up, 6 in 2023-12-01T01:13:34.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:33 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-01T01:13:34.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:33 smithi137 ceph-mon[103312]: from='osd.5 [v2:172.21.15.203:6808/2648569482,v1:172.21.15.203:6809/2648569482]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-12-01T01:13:34.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:33 smithi137 ceph-mon[103312]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-12-01T01:13:34.489 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi077:vg_nvme/lv_2 2023-12-01T01:13:34.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:34 smithi203 ceph-mon[103581]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2023-12-01T01:13:34.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:34 smithi203 ceph-mon[103581]: osdmap e36: 6 total, 5 up, 6 in 2023-12-01T01:13:34.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:34 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-01T01:13:34.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:34 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-01T01:13:34.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:34 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:34.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:34 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:34.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:34 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:13:34.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:34 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:13:34.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:34 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:34.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:34 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:13:34.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:34 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:13:34.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:34 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:34.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:34 smithi203 ceph-mon[103581]: osd.5 [v2:172.21.15.203:6808/2648569482,v1:172.21.15.203:6809/2648569482] boot 2023-12-01T01:13:34.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:34 smithi203 ceph-mon[103581]: osdmap e37: 6 total, 6 up, 6 in 2023-12-01T01:13:34.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:34 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-01T01:13:34.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:34 smithi077 ceph-mon[98867]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2023-12-01T01:13:34.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:34 smithi077 ceph-mon[98867]: osdmap e36: 6 total, 5 up, 6 in 2023-12-01T01:13:34.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:34 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-01T01:13:34.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:34 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-01T01:13:34.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:34 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:34.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:34 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:34.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:34 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:13:34.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:34 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:13:34.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:34 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:34.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:34 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:13:34.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:34 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:13:34.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:34 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:34.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:34 smithi077 ceph-mon[98867]: osd.5 [v2:172.21.15.203:6808/2648569482,v1:172.21.15.203:6809/2648569482] boot 2023-12-01T01:13:34.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:34 smithi077 ceph-mon[98867]: osdmap e37: 6 total, 6 up, 6 in 2023-12-01T01:13:34.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:34 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-01T01:13:35.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:34 smithi137 ceph-mon[103312]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2023-12-01T01:13:35.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:34 smithi137 ceph-mon[103312]: osdmap e36: 6 total, 5 up, 6 in 2023-12-01T01:13:35.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:34 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-01T01:13:35.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:34 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-01T01:13:35.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:34 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:35.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:34 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:35.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:34 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:13:35.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:34 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:13:35.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:34 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:35.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:34 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:13:35.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:34 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:13:35.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:34 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:35.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:34 smithi137 ceph-mon[103312]: osd.5 [v2:172.21.15.203:6808/2648569482,v1:172.21.15.203:6809/2648569482] boot 2023-12-01T01:13:35.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:34 smithi137 ceph-mon[103312]: osdmap e37: 6 total, 6 up, 6 in 2023-12-01T01:13:35.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:34 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-01T01:13:35.049 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:13:35.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:35 smithi077 ceph-mon[98867]: purged_snaps scrub starts 2023-12-01T01:13:35.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:35 smithi077 ceph-mon[98867]: purged_snaps scrub ok 2023-12-01T01:13:35.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:35 smithi077 ceph-mon[98867]: pgmap v121: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-01T01:13:35.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:35 smithi077 ceph-mon[98867]: Detected new or changed devices on smithi203 2023-12-01T01:13:35.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:35 smithi077 ceph-mon[98867]: Adjusting osd_memory_target on smithi203 to 8584M 2023-12-01T01:13:35.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:35 smithi077 ceph-mon[98867]: osdmap e38: 6 total, 6 up, 6 in 2023-12-01T01:13:36.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:35 smithi203 ceph-mon[103581]: purged_snaps scrub starts 2023-12-01T01:13:36.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:35 smithi203 ceph-mon[103581]: purged_snaps scrub ok 2023-12-01T01:13:36.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:35 smithi203 ceph-mon[103581]: pgmap v121: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-01T01:13:36.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:35 smithi203 ceph-mon[103581]: Detected new or changed devices on smithi203 2023-12-01T01:13:36.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:35 smithi203 ceph-mon[103581]: Adjusting osd_memory_target on smithi203 to 8584M 2023-12-01T01:13:36.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:35 smithi203 ceph-mon[103581]: osdmap e38: 6 total, 6 up, 6 in 2023-12-01T01:13:36.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:35 smithi137 ceph-mon[103312]: purged_snaps scrub starts 2023-12-01T01:13:36.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:35 smithi137 ceph-mon[103312]: purged_snaps scrub ok 2023-12-01T01:13:36.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:35 smithi137 ceph-mon[103312]: pgmap v121: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-01T01:13:36.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:35 smithi137 ceph-mon[103312]: Detected new or changed devices on smithi203 2023-12-01T01:13:36.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:35 smithi137 ceph-mon[103312]: Adjusting osd_memory_target on smithi203 to 8584M 2023-12-01T01:13:36.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:35 smithi137 ceph-mon[103312]: osdmap e38: 6 total, 6 up, 6 in 2023-12-01T01:13:36.276 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.271+0000 7f6d1b5ad700 1 -- 172.21.15.77:0/1868057031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d14101f90 msgr2=0x7f6d14102410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:13:36.276 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.271+0000 7f6d1b5ad700 1 --2- 172.21.15.77:0/1868057031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d14101f90 0x7f6d14102410 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6d04009990 tx=0x7f6d04009ca0 comp rx=0 tx=0).stop 2023-12-01T01:13:36.276 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.272+0000 7f6d1b5ad700 1 -- 172.21.15.77:0/1868057031 shutdown_connections 2023-12-01T01:13:36.276 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.272+0000 7f6d1b5ad700 1 --2- 172.21.15.77:0/1868057031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d14102950 0x7f6d14109270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:13:36.276 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.272+0000 7f6d1b5ad700 1 --2- 172.21.15.77:0/1868057031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d14100e30 0x7f6d14101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:13:36.276 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.272+0000 7f6d1b5ad700 1 --2- 172.21.15.77:0/1868057031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d14101f90 0x7f6d14102410 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:13:36.276 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.272+0000 7f6d1b5ad700 1 -- 172.21.15.77:0/1868057031 >> 172.21.15.77:0/1868057031 conn(0x7f6d140fc3b0 msgr2=0x7f6d140fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:13:36.277 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.272+0000 7f6d1b5ad700 1 -- 172.21.15.77:0/1868057031 shutdown_connections 2023-12-01T01:13:36.277 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.272+0000 7f6d1b5ad700 1 -- 172.21.15.77:0/1868057031 wait complete. 2023-12-01T01:13:36.277 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.272+0000 7f6d1b5ad700 1 Processor -- start 2023-12-01T01:13:36.277 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.272+0000 7f6d1b5ad700 1 -- start start 2023-12-01T01:13:36.277 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.272+0000 7f6d1b5ad700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d14100e30 0x7f6d14196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:13:36.277 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.272+0000 7f6d1b5ad700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d14101f90 0x7f6d14197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:13:36.277 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.272+0000 7f6d1b5ad700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d14102950 0x7f6d1419e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:13:36.277 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.272+0000 7f6d1b5ad700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1419e6d0 con 0x7f6d14100e30 2023-12-01T01:13:36.277 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.272+0000 7f6d1b5ad700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1419e810 con 0x7f6d14102950 2023-12-01T01:13:36.277 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.272+0000 7f6d1b5ad700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1419e9a0 con 0x7f6d14101f90 2023-12-01T01:13:36.277 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.273+0000 7f6d19349700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d14100e30 0x7f6d14196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:13:36.277 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.273+0000 7f6d19349700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d14100e30 0x7f6d14196b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56774/0 (socket says 172.21.15.77:56774) 2023-12-01T01:13:36.277 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.273+0000 7f6d19349700 1 -- 172.21.15.77:0/3121093349 learned_addr learned my addr 172.21.15.77:0/3121093349 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:13:36.277 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.273+0000 7f6d18b48700 1 --2- 172.21.15.77:0/3121093349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d14101f90 0x7f6d14197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:13:36.277 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.273+0000 7f6d19b4a700 1 --2- 172.21.15.77:0/3121093349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d14102950 0x7f6d1419e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:13:36.278 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.273+0000 7f6d19349700 1 -- 172.21.15.77:0/3121093349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d14101f90 msgr2=0x7f6d14197080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:13:36.278 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.273+0000 7f6d19349700 1 --2- 172.21.15.77:0/3121093349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d14101f90 0x7f6d14197080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:13:36.278 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.273+0000 7f6d19349700 1 -- 172.21.15.77:0/3121093349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d14102950 msgr2=0x7f6d1419e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:13:36.278 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.273+0000 7f6d19349700 1 --2- 172.21.15.77:0/3121093349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d14102950 0x7f6d1419e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:13:36.278 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.273+0000 7f6d19349700 1 -- 172.21.15.77:0/3121093349 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d040095f0 con 0x7f6d14100e30 2023-12-01T01:13:36.278 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.273+0000 7f6d18b48700 1 --2- 172.21.15.77:0/3121093349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d14101f90 0x7f6d14197080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:13:36.278 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.273+0000 7f6d19b4a700 1 --2- 172.21.15.77:0/3121093349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d14102950 0x7f6d1419e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:13:36.278 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.273+0000 7f6d19349700 1 --2- 172.21.15.77:0/3121093349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d14100e30 0x7f6d14196b40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6d1000b560 tx=0x7f6d1000b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:13:36.278 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.273+0000 7f6d0a7fc700 1 -- 172.21.15.77:0/3121093349 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d10011830 con 0x7f6d14100e30 2023-12-01T01:13:36.278 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.273+0000 7f6d0a7fc700 1 -- 172.21.15.77:0/3121093349 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d10011e50 con 0x7f6d14100e30 2023-12-01T01:13:36.278 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.273+0000 7f6d1b5ad700 1 -- 172.21.15.77:0/3121093349 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d1419ec80 con 0x7f6d14100e30 2023-12-01T01:13:36.278 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.273+0000 7f6d0a7fc700 1 -- 172.21.15.77:0/3121093349 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1000f2e0 con 0x7f6d14100e30 2023-12-01T01:13:36.278 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.273+0000 7f6d1b5ad700 1 -- 172.21.15.77:0/3121093349 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d1419f180 con 0x7f6d14100e30 2023-12-01T01:13:36.279 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.274+0000 7f6d1b5ad700 1 -- 172.21.15.77:0/3121093349 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d14108420 con 0x7f6d14100e30 2023-12-01T01:13:36.279 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.275+0000 7f6d0a7fc700 1 -- 172.21.15.77:0/3121093349 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f6d10011990 con 0x7f6d14100e30 2023-12-01T01:13:36.279 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.276+0000 7f6d0a7fc700 1 --2- 172.21.15.77:0/3121093349 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f6d000a05a0 0x7f6d000a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:13:36.279 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.276+0000 7f6d18b48700 1 --2- 172.21.15.77:0/3121093349 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f6d000a05a0 0x7f6d000a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:13:36.280 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.276+0000 7f6d0a7fc700 1 -- 172.21.15.77:0/3121093349 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4562+0+0 (secure 0 0 0) 0x7f6d100caa80 con 0x7f6d14100e30 2023-12-01T01:13:36.280 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.276+0000 7f6d18b48700 1 --2- 172.21.15.77:0/3121093349 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f6d000a05a0 0x7f6d000a2a60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6d04006010 tx=0x7f6d0400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:13:36.281 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.280+0000 7f6d0a7fc700 1 -- 172.21.15.77:0/3121093349 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6d100954c0 con 0x7f6d14100e30 2023-12-01T01:13:36.477 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:36.476+0000 7f6d1b5ad700 1 -- 172.21.15.77:0/3121093349 --> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d140611d0 con 0x7f6d000a05a0 2023-12-01T01:13:36.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:36 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-01T01:13:36.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:36 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-01T01:13:36.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:36 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:13:36.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:36 smithi077 ceph-mon[98867]: osdmap e39: 6 total, 6 up, 6 in 2023-12-01T01:13:37.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:36 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-01T01:13:37.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:36 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-01T01:13:37.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:36 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:13:37.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:36 smithi203 ceph-mon[103581]: osdmap e39: 6 total, 6 up, 6 in 2023-12-01T01:13:37.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:36 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-01T01:13:37.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:36 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-01T01:13:37.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:36 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:13:37.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:36 smithi137 ceph-mon[103312]: osdmap e39: 6 total, 6 up, 6 in 2023-12-01T01:13:37.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:37 smithi077 ceph-mon[98867]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-01T01:13:37.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:37 smithi077 ceph-mon[98867]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:13:38.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:37 smithi203 ceph-mon[103581]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-01T01:13:38.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:37 smithi203 ceph-mon[103581]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:13:38.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:37 smithi137 ceph-mon[103312]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-01T01:13:38.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:37 smithi137 ceph-mon[103312]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:13:38.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:38 smithi077 ceph-mon[98867]: osdmap e40: 6 total, 6 up, 6 in 2023-12-01T01:13:39.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:38 smithi203 ceph-mon[103581]: osdmap e40: 6 total, 6 up, 6 in 2023-12-01T01:13:39.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:38 smithi137 ceph-mon[103312]: osdmap e40: 6 total, 6 up, 6 in 2023-12-01T01:13:39.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:39 smithi077 ceph-mon[98867]: pgmap v127: 1 pgs: 1 peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-01T01:13:39.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:39 smithi077 ceph-mon[98867]: osdmap e41: 6 total, 6 up, 6 in 2023-12-01T01:13:39.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:39 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/3504271875' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e3fa97e-2ca6-418c-9e8d-a6fe3c56adb3"}]: dispatch 2023-12-01T01:13:39.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:39 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/3504271875' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e3fa97e-2ca6-418c-9e8d-a6fe3c56adb3"}]': finished 2023-12-01T01:13:39.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:39 smithi077 ceph-mon[98867]: osdmap e42: 7 total, 6 up, 7 in 2023-12-01T01:13:39.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:39 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-01T01:13:39.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:39 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/3384829503' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-01T01:13:40.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:39 smithi203 ceph-mon[103581]: pgmap v127: 1 pgs: 1 peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-01T01:13:40.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:39 smithi203 ceph-mon[103581]: osdmap e41: 6 total, 6 up, 6 in 2023-12-01T01:13:40.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:39 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/3504271875' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e3fa97e-2ca6-418c-9e8d-a6fe3c56adb3"}]: dispatch 2023-12-01T01:13:40.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:39 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/3504271875' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e3fa97e-2ca6-418c-9e8d-a6fe3c56adb3"}]': finished 2023-12-01T01:13:40.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:39 smithi203 ceph-mon[103581]: osdmap e42: 7 total, 6 up, 7 in 2023-12-01T01:13:40.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:39 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-01T01:13:40.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:39 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/3384829503' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-01T01:13:40.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:39 smithi137 ceph-mon[103312]: pgmap v127: 1 pgs: 1 peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-12-01T01:13:40.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:39 smithi137 ceph-mon[103312]: osdmap e41: 6 total, 6 up, 6 in 2023-12-01T01:13:40.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:39 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/3504271875' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e3fa97e-2ca6-418c-9e8d-a6fe3c56adb3"}]: dispatch 2023-12-01T01:13:40.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:39 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/3504271875' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e3fa97e-2ca6-418c-9e8d-a6fe3c56adb3"}]': finished 2023-12-01T01:13:40.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:39 smithi137 ceph-mon[103312]: osdmap e42: 7 total, 6 up, 7 in 2023-12-01T01:13:40.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:39 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-01T01:13:40.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:39 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/3384829503' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-01T01:13:40.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:40 smithi077 ceph-mon[98867]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-01T01:13:41.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:40 smithi203 ceph-mon[103581]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-01T01:13:41.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:40 smithi137 ceph-mon[103312]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-01T01:13:43.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:43 smithi077 ceph-mon[98867]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 102 KiB/s, 0 objects/s recovering 2023-12-01T01:13:44.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:43 smithi203 ceph-mon[103581]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 102 KiB/s, 0 objects/s recovering 2023-12-01T01:13:44.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:43 smithi137 ceph-mon[103312]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 102 KiB/s, 0 objects/s recovering 2023-12-01T01:13:45.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:45 smithi077 ceph-mon[98867]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 87 KiB/s, 0 objects/s recovering 2023-12-01T01:13:45.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:45 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-12-01T01:13:45.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:45 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:13:46.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:45 smithi203 ceph-mon[103581]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 87 KiB/s, 0 objects/s recovering 2023-12-01T01:13:46.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:45 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-12-01T01:13:46.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:45 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:13:46.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:45 smithi137 ceph-mon[103312]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 87 KiB/s, 0 objects/s recovering 2023-12-01T01:13:46.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:45 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-12-01T01:13:46.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:45 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:13:46.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:46 smithi077 ceph-mon[98867]: Deploying daemon osd.6 on smithi077 2023-12-01T01:13:47.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:46 smithi203 ceph-mon[103581]: Deploying daemon osd.6 on smithi077 2023-12-01T01:13:47.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:46 smithi137 ceph-mon[103312]: Deploying daemon osd.6 on smithi077 2023-12-01T01:13:47.821 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:47 smithi077 ceph-mon[98867]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-12-01T01:13:48.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:47 smithi203 ceph-mon[103581]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-12-01T01:13:48.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:47 smithi137 ceph-mon[103312]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-12-01T01:13:49.828 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:49 smithi077 ceph-mon[98867]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2023-12-01T01:13:50.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:49 smithi203 ceph-mon[103581]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2023-12-01T01:13:50.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:49 smithi137 ceph-mon[103312]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2023-12-01T01:13:51.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:51 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:51.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:51 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:51.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:51 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:13:51.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:51 smithi077 ceph-mon[98867]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-12-01T01:13:51.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:51 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:51.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:51 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:51.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:51 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:13:51.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:51 smithi203 ceph-mon[103581]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-12-01T01:13:51.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:51 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:51.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:51 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:51.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:51 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:13:51.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:51 smithi137 ceph-mon[103312]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-12-01T01:13:52.880 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:52.879+0000 7f6d0a7fc700 1 -- 172.21.15.77:0/3121093349 <== mgr.14152 v2:172.21.15.77:6800/3838941679 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6d140611d0 con 0x7f6d000a05a0 2023-12-01T01:13:52.880 INFO:teuthology.orchestra.run.smithi077.stdout:Created osd(s) 6 on host 'smithi077' 2023-12-01T01:13:52.882 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:52.881+0000 7f6d1b5ad700 1 -- 172.21.15.77:0/3121093349 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f6d000a05a0 msgr2=0x7f6d000a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:13:52.882 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:52.881+0000 7f6d1b5ad700 1 --2- 172.21.15.77:0/3121093349 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f6d000a05a0 0x7f6d000a2a60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6d04006010 tx=0x7f6d0400b540 comp rx=0 tx=0).stop 2023-12-01T01:13:52.882 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:52.881+0000 7f6d1b5ad700 1 -- 172.21.15.77:0/3121093349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d14100e30 msgr2=0x7f6d14196b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:13:52.883 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:52.881+0000 7f6d1b5ad700 1 --2- 172.21.15.77:0/3121093349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d14100e30 0x7f6d14196b40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6d1000b560 tx=0x7f6d1000b920 comp rx=0 tx=0).stop 2023-12-01T01:13:52.883 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:52.881+0000 7f6d1b5ad700 1 -- 172.21.15.77:0/3121093349 shutdown_connections 2023-12-01T01:13:52.883 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:52.881+0000 7f6d1b5ad700 1 --2- 172.21.15.77:0/3121093349 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f6d000a05a0 0x7f6d000a2a60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:13:52.883 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:52.881+0000 7f6d1b5ad700 1 --2- 172.21.15.77:0/3121093349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d14102950 0x7f6d1419e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:13:52.883 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:52.881+0000 7f6d1b5ad700 1 --2- 172.21.15.77:0/3121093349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d14100e30 0x7f6d14196b40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:13:52.883 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:52.881+0000 7f6d1b5ad700 1 --2- 172.21.15.77:0/3121093349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d14101f90 0x7f6d14197080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:13:52.883 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:52.881+0000 7f6d1b5ad700 1 -- 172.21.15.77:0/3121093349 >> 172.21.15.77:0/3121093349 conn(0x7f6d140fc3b0 msgr2=0x7f6d14105560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:13:52.883 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:52.881+0000 7f6d1b5ad700 1 -- 172.21.15.77:0/3121093349 shutdown_connections 2023-12-01T01:13:52.883 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:13:52.881+0000 7f6d1b5ad700 1 -- 172.21.15.77:0/3121093349 wait complete. 2023-12-01T01:13:53.255 DEBUG:teuthology.orchestra.run.smithi077:osd.6> sudo journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@osd.6.service 2023-12-01T01:13:53.257 INFO:tasks.cephadm:Deploying osd.7 on smithi137 with /dev/vg_nvme/lv_2... 2023-12-01T01:13:53.257 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-12-01T01:13:53.309 INFO:journalctl@ceph.osd.6.smithi077.stdout:-- Logs begin at Fri 2023-12-01 00:55:13 UTC. -- 2023-12-01T01:13:53.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:53 smithi077 ceph-mon[98867]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-01T01:13:53.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:53 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:53.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:53 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:53.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:53 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:53.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:53 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:53.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:53 smithi203 ceph-mon[103581]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-01T01:13:53.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:53 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:53.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:53 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:53.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:53 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:53.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:53 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:53.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:53 smithi137 ceph-mon[103312]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-01T01:13:53.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:53 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:53.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:53 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:53.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:53 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:53.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:53 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:53.836 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.b/config 2023-12-01T01:13:54.154 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:13:53 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:13:53.995+0000 7efeeddc8640 -1 osd.6 0 log_to_monitors true 2023-12-01T01:13:54.540 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:54 smithi077 ceph-mon[98867]: from='osd.6 [v2:172.21.15.77:6818/2463414651,v1:172.21.15.77:6819/2463414651]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-12-01T01:13:54.594 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:54 smithi137 ceph-mon[103312]: from='osd.6 [v2:172.21.15.77:6818/2463414651,v1:172.21.15.77:6819/2463414651]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-12-01T01:13:54.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:54 smithi203 ceph-mon[103581]: from='osd.6 [v2:172.21.15.77:6818/2463414651,v1:172.21.15.77:6819/2463414651]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-12-01T01:13:55.335 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-12-01T01:13:55.335 INFO:teuthology.orchestra.run.smithi137.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-12-01T01:13:55.335 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10+0 records in 2023-12-01T01:13:55.335 INFO:teuthology.orchestra.run.smithi137.stderr:10+0 records out 2023-12-01T01:13:55.336 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0127364 s, 823 MB/s 2023-12-01T01:13:55.336 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping successful for: 2023-12-01T01:13:55.654 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:13:55 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:13:55.361+0000 7efee2c3e700 -1 osd.6 0 waiting for initial osdmap 2023-12-01T01:13:55.654 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:13:55 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:13:55.373+0000 7efedba2d700 -1 osd.6 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-01T01:13:55.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:55 smithi077 ceph-mon[98867]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-01T01:13:55.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:55 smithi077 ceph-mon[98867]: from='osd.6 [v2:172.21.15.77:6818/2463414651,v1:172.21.15.77:6819/2463414651]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-12-01T01:13:55.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:55 smithi077 ceph-mon[98867]: osdmap e43: 7 total, 6 up, 7 in 2023-12-01T01:13:55.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:55 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-01T01:13:55.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:55 smithi077 ceph-mon[98867]: from='osd.6 [v2:172.21.15.77:6818/2463414651,v1:172.21.15.77:6819/2463414651]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2023-12-01T01:13:55.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:55 smithi203 ceph-mon[103581]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-01T01:13:55.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:55 smithi203 ceph-mon[103581]: from='osd.6 [v2:172.21.15.77:6818/2463414651,v1:172.21.15.77:6819/2463414651]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-12-01T01:13:55.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:55 smithi203 ceph-mon[103581]: osdmap e43: 7 total, 6 up, 7 in 2023-12-01T01:13:55.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:55 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-01T01:13:55.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:55 smithi203 ceph-mon[103581]: from='osd.6 [v2:172.21.15.77:6818/2463414651,v1:172.21.15.77:6819/2463414651]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2023-12-01T01:13:55.770 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:55 smithi137 ceph-mon[103312]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-01T01:13:55.770 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:55 smithi137 ceph-mon[103312]: from='osd.6 [v2:172.21.15.77:6818/2463414651,v1:172.21.15.77:6819/2463414651]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-12-01T01:13:55.770 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:55 smithi137 ceph-mon[103312]: osdmap e43: 7 total, 6 up, 7 in 2023-12-01T01:13:55.770 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:55 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-01T01:13:55.770 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:55 smithi137 ceph-mon[103312]: from='osd.6 [v2:172.21.15.77:6818/2463414651,v1:172.21.15.77:6819/2463414651]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2023-12-01T01:13:55.781 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi137:vg_nvme/lv_2 2023-12-01T01:13:56.335 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.b/config 2023-12-01T01:13:56.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:56 smithi077 ceph-mon[98867]: from='osd.6 [v2:172.21.15.77:6818/2463414651,v1:172.21.15.77:6819/2463414651]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2023-12-01T01:13:56.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:56 smithi077 ceph-mon[98867]: osdmap e44: 7 total, 6 up, 7 in 2023-12-01T01:13:56.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:56 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-01T01:13:56.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:56 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-01T01:13:56.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:56 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:56.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:56 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:56.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:56 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:13:56.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:56 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:13:56.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:56 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:13:56.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:56 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:56.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:56 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:13:56.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:56 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:13:56.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:56 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:56.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:56 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:13:56.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:56 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:13:56.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:56 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:13:56.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:56 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:56.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:56 smithi137 ceph-mon[103312]: from='osd.6 [v2:172.21.15.77:6818/2463414651,v1:172.21.15.77:6819/2463414651]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2023-12-01T01:13:56.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:56 smithi137 ceph-mon[103312]: osdmap e44: 7 total, 6 up, 7 in 2023-12-01T01:13:56.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:56 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-01T01:13:56.735 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:56 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-01T01:13:56.735 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:56 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:56.735 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:56 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:56.735 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:56 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:13:56.735 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:56 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:13:56.735 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:56 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:13:56.735 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:56 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:56.735 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:56 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:13:56.735 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:56 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:13:56.735 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:56 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:56.735 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:56 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:13:56.735 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:56 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:13:56.735 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:56 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:13:56.735 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:56 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:56.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:56 smithi203 ceph-mon[103581]: from='osd.6 [v2:172.21.15.77:6818/2463414651,v1:172.21.15.77:6819/2463414651]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2023-12-01T01:13:56.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:56 smithi203 ceph-mon[103581]: osdmap e44: 7 total, 6 up, 7 in 2023-12-01T01:13:56.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:56 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-01T01:13:56.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:56 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-01T01:13:56.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:56 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:56.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:56 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:56.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:56 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:13:56.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:56 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:13:56.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:56 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:13:56.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:56 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:56.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:56 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:13:56.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:56 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:13:56.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:56 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:56.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:56 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:13:56.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:56 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:13:56.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:56 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:13:56.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:56 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:13:57.556 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:57.555+0000 7fc1761ee700 1 -- 172.21.15.137:0/998143605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1700ffd20 msgr2=0x7fc170100140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:13:57.556 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:57.555+0000 7fc1761ee700 1 --2- 172.21.15.137:0/998143605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1700ffd20 0x7fc170100140 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc1640099c0 tx=0x7fc164009cd0 comp rx=0 tx=0).stop 2023-12-01T01:13:57.557 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:57.555+0000 7fc1761ee700 1 -- 172.21.15.137:0/998143605 shutdown_connections 2023-12-01T01:13:57.557 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:57.555+0000 7fc1761ee700 1 --2- 172.21.15.137:0/998143605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1701018c0 0x7fc170108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:13:57.557 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:57.555+0000 7fc1761ee700 1 --2- 172.21.15.137:0/998143605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1700ffd20 0x7fc170100140 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:13:57.557 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:57.555+0000 7fc1761ee700 1 --2- 172.21.15.137:0/998143605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc170100f20 0x7fc170101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:13:57.557 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:57.555+0000 7fc1761ee700 1 -- 172.21.15.137:0/998143605 >> 172.21.15.137:0/998143605 conn(0x7fc1700fb2a0 msgr2=0x7fc1700fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:13:57.557 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:57.555+0000 7fc1761ee700 1 -- 172.21.15.137:0/998143605 shutdown_connections 2023-12-01T01:13:57.557 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:57.555+0000 7fc1761ee700 1 -- 172.21.15.137:0/998143605 wait complete. 2023-12-01T01:13:57.557 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:57.555+0000 7fc1761ee700 1 Processor -- start 2023-12-01T01:13:57.559 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:57.555+0000 7fc1761ee700 1 -- start start 2023-12-01T01:13:57.559 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:57.556+0000 7fc1761ee700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1700ffd20 0x7fc1700693d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:13:57.559 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:57.556+0000 7fc1761ee700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc170100f20 0x7fc17019b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:13:57.559 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:57.556+0000 7fc1761ee700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1701018c0 0x7fc17019def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:13:57.559 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:57.556+0000 7fc1761ee700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17019e690 con 0x7fc1700ffd20 2023-12-01T01:13:57.559 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:57.556+0000 7fc1761ee700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17019e800 con 0x7fc1701018c0 2023-12-01T01:13:57.560 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:57.556+0000 7fc1761ee700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17019e9c0 con 0x7fc170100f20 2023-12-01T01:13:57.560 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:57.556+0000 7fc16ffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1701018c0 0x7fc17019def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:13:57.560 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:57.556+0000 7fc16ffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1701018c0 0x7fc17019def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37392/0 (socket says 172.21.15.137:37392) 2023-12-01T01:13:57.560 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:57.556+0000 7fc16ffff700 1 -- 172.21.15.137:0/3167345911 learned_addr learned my addr 172.21.15.137:0/3167345911 (peer_addr_for_me v2:172.21.15.137:0/0) 2023-12-01T01:13:57.560 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:57.556+0000 7fc16f7fe700 1 --2- 172.21.15.137:0/3167345911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1700ffd20 0x7fc1700693d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:13:57.560 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:57.556+0000 7fc16effd700 1 --2- 172.21.15.137:0/3167345911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc170100f20 0x7fc17019b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:13:57.560 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:57.556+0000 7fc16ffff700 1 -- 172.21.15.137:0/3167345911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc170100f20 msgr2=0x7fc17019b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:13:57.560 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:57.556+0000 7fc16ffff700 1 --2- 172.21.15.137:0/3167345911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc170100f20 0x7fc17019b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:13:57.560 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:57.556+0000 7fc16ffff700 1 -- 172.21.15.137:0/3167345911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1700ffd20 msgr2=0x7fc1700693d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:13:57.560 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:57.556+0000 7fc16ffff700 1 --2- 172.21.15.137:0/3167345911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1700ffd20 0x7fc1700693d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:13:57.560 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:57.556+0000 7fc16ffff700 1 -- 172.21.15.137:0/3167345911 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1640095f0 con 0x7fc1701018c0 2023-12-01T01:13:57.560 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:57.556+0000 7fc16ffff700 1 --2- 172.21.15.137:0/3167345911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1701018c0 0x7fc17019def0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc16800b970 tx=0x7fc16800bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:13:57.560 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:57.556+0000 7fc16cff9700 1 -- 172.21.15.137:0/3167345911 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16800c930 con 0x7fc1701018c0 2023-12-01T01:13:57.561 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:57.556+0000 7fc16cff9700 1 -- 172.21.15.137:0/3167345911 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc168014440 con 0x7fc1701018c0 2023-12-01T01:13:57.561 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:57.556+0000 7fc16cff9700 1 -- 172.21.15.137:0/3167345911 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1680075d0 con 0x7fc1701018c0 2023-12-01T01:13:57.561 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:57.556+0000 7fc1761ee700 1 -- 172.21.15.137:0/3167345911 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc17019eca0 con 0x7fc1701018c0 2023-12-01T01:13:57.561 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:57.557+0000 7fc1761ee700 1 -- 172.21.15.137:0/3167345911 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc17019f150 con 0x7fc1701018c0 2023-12-01T01:13:57.561 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:57.558+0000 7fc1761ee700 1 -- 172.21.15.137:0/3167345911 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc170107370 con 0x7fc1701018c0 2023-12-01T01:13:57.562 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:57.558+0000 7fc16cff9700 1 -- 172.21.15.137:0/3167345911 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fc1680145b0 con 0x7fc1701018c0 2023-12-01T01:13:57.562 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:57.559+0000 7fc16cff9700 1 --2- 172.21.15.137:0/3167345911 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc1580a0520 0x7fc1580a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:13:57.562 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:57.559+0000 7fc16cff9700 1 -- 172.21.15.137:0/3167345911 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 4965+0+0 (secure 0 0 0) 0x7fc168006e40 con 0x7fc1701018c0 2023-12-01T01:13:57.562 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:57.559+0000 7fc16f7fe700 1 --2- 172.21.15.137:0/3167345911 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc1580a0520 0x7fc1580a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:13:57.562 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:57.560+0000 7fc16f7fe700 1 --2- 172.21.15.137:0/3167345911 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc1580a0520 0x7fc1580a29e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc164009f50 tx=0x7fc16400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:13:57.565 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:57.563+0000 7fc16cff9700 1 -- 172.21.15.137:0/3167345911 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc168094bc0 con 0x7fc1701018c0 2023-12-01T01:13:57.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:57 smithi077 ceph-mon[98867]: purged_snaps scrub starts 2023-12-01T01:13:57.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:57 smithi077 ceph-mon[98867]: purged_snaps scrub ok 2023-12-01T01:13:57.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:57 smithi077 ceph-mon[98867]: Detected new or changed devices on smithi077 2023-12-01T01:13:57.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:57 smithi077 ceph-mon[98867]: Adjusting osd_memory_target on smithi077 to 5722M 2023-12-01T01:13:57.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:57 smithi077 ceph-mon[98867]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-01T01:13:57.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:57 smithi077 ceph-mon[98867]: osd.6 [v2:172.21.15.77:6818/2463414651,v1:172.21.15.77:6819/2463414651] boot 2023-12-01T01:13:57.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:57 smithi077 ceph-mon[98867]: osdmap e45: 7 total, 7 up, 7 in 2023-12-01T01:13:57.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:57 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-01T01:13:57.762 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:13:57.760+0000 7fc1761ee700 1 -- 172.21.15.137:0/3167345911 --> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1700611d0 con 0x7fc1580a0520 2023-12-01T01:13:57.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:57 smithi203 ceph-mon[103581]: purged_snaps scrub starts 2023-12-01T01:13:57.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:57 smithi203 ceph-mon[103581]: purged_snaps scrub ok 2023-12-01T01:13:57.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:57 smithi203 ceph-mon[103581]: Detected new or changed devices on smithi077 2023-12-01T01:13:57.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:57 smithi203 ceph-mon[103581]: Adjusting osd_memory_target on smithi077 to 5722M 2023-12-01T01:13:57.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:57 smithi203 ceph-mon[103581]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-01T01:13:57.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:57 smithi203 ceph-mon[103581]: osd.6 [v2:172.21.15.77:6818/2463414651,v1:172.21.15.77:6819/2463414651] boot 2023-12-01T01:13:57.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:57 smithi203 ceph-mon[103581]: osdmap e45: 7 total, 7 up, 7 in 2023-12-01T01:13:57.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:57 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-01T01:13:57.778 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:57 smithi137 ceph-mon[103312]: purged_snaps scrub starts 2023-12-01T01:13:57.778 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:57 smithi137 ceph-mon[103312]: purged_snaps scrub ok 2023-12-01T01:13:57.778 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:57 smithi137 ceph-mon[103312]: Detected new or changed devices on smithi077 2023-12-01T01:13:57.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:57 smithi137 ceph-mon[103312]: Adjusting osd_memory_target on smithi077 to 5722M 2023-12-01T01:13:57.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:57 smithi137 ceph-mon[103312]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-01T01:13:57.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:57 smithi137 ceph-mon[103312]: osd.6 [v2:172.21.15.77:6818/2463414651,v1:172.21.15.77:6819/2463414651] boot 2023-12-01T01:13:57.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:57 smithi137 ceph-mon[103312]: osdmap e45: 7 total, 7 up, 7 in 2023-12-01T01:13:57.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:57 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-01T01:13:58.628 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:58 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-01T01:13:58.629 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:58 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-01T01:13:58.629 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:58 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:13:58.629 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:58 smithi137 ceph-mon[103312]: osdmap e46: 7 total, 7 up, 7 in 2023-12-01T01:13:58.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:58 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-01T01:13:58.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:58 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-01T01:13:58.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:58 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:13:58.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:58 smithi077 ceph-mon[98867]: osdmap e46: 7 total, 7 up, 7 in 2023-12-01T01:13:58.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:58 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-01T01:13:58.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:58 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-01T01:13:58.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:58 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:13:58.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:58 smithi203 ceph-mon[103581]: osdmap e46: 7 total, 7 up, 7 in 2023-12-01T01:13:59.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:59 smithi077 ceph-mon[98867]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:13:59.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:13:59 smithi077 ceph-mon[98867]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-01T01:13:59.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:59 smithi203 ceph-mon[103581]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:13:59.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:13:59 smithi203 ceph-mon[103581]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-01T01:13:59.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:59 smithi137 ceph-mon[103312]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:13:59.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:13:59 smithi137 ceph-mon[103312]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-01T01:14:00.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:00 smithi077 ceph-mon[98867]: from='client.? 172.21.15.137:0/3204434287' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d7398c01-0f98-4138-8a49-c3ff0aae7e88"}]: dispatch 2023-12-01T01:14:00.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:00 smithi077 ceph-mon[98867]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d7398c01-0f98-4138-8a49-c3ff0aae7e88"}]: dispatch 2023-12-01T01:14:00.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:00 smithi077 ceph-mon[98867]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d7398c01-0f98-4138-8a49-c3ff0aae7e88"}]': finished 2023-12-01T01:14:00.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:00 smithi077 ceph-mon[98867]: osdmap e47: 8 total, 7 up, 8 in 2023-12-01T01:14:00.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:00 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-01T01:14:00.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:00 smithi203 ceph-mon[103581]: from='client.? 172.21.15.137:0/3204434287' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d7398c01-0f98-4138-8a49-c3ff0aae7e88"}]: dispatch 2023-12-01T01:14:00.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:00 smithi203 ceph-mon[103581]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d7398c01-0f98-4138-8a49-c3ff0aae7e88"}]: dispatch 2023-12-01T01:14:00.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:00 smithi203 ceph-mon[103581]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d7398c01-0f98-4138-8a49-c3ff0aae7e88"}]': finished 2023-12-01T01:14:00.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:00 smithi203 ceph-mon[103581]: osdmap e47: 8 total, 7 up, 8 in 2023-12-01T01:14:00.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:00 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-01T01:14:00.778 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:00 smithi137 ceph-mon[103312]: from='client.? 172.21.15.137:0/3204434287' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d7398c01-0f98-4138-8a49-c3ff0aae7e88"}]: dispatch 2023-12-01T01:14:00.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:00 smithi137 ceph-mon[103312]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d7398c01-0f98-4138-8a49-c3ff0aae7e88"}]: dispatch 2023-12-01T01:14:00.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:00 smithi137 ceph-mon[103312]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d7398c01-0f98-4138-8a49-c3ff0aae7e88"}]': finished 2023-12-01T01:14:00.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:00 smithi137 ceph-mon[103312]: osdmap e47: 8 total, 7 up, 8 in 2023-12-01T01:14:00.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:00 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-01T01:14:01.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:01 smithi077 ceph-mon[98867]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-01T01:14:01.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:01 smithi077 ceph-mon[98867]: from='client.? 172.21.15.137:0/2015191596' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-01T01:14:01.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:01 smithi203 ceph-mon[103581]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-01T01:14:01.765 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:01 smithi203 ceph-mon[103581]: from='client.? 172.21.15.137:0/2015191596' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-01T01:14:01.778 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:01 smithi137 ceph-mon[103312]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-01T01:14:01.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:01 smithi137 ceph-mon[103312]: from='client.? 172.21.15.137:0/2015191596' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-01T01:14:03.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:03 smithi077 ceph-mon[98867]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-01T01:14:04.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:03 smithi203 ceph-mon[103581]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-01T01:14:04.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:03 smithi137 ceph-mon[103312]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-01T01:14:05.766 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:05 smithi137 ceph-mon[103312]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-01T01:14:05.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:05 smithi077 ceph-mon[98867]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-01T01:14:06.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:05 smithi203 ceph-mon[103581]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-01T01:14:07.811 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:07 smithi137 ceph-mon[103312]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-01T01:14:07.811 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:07 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-12-01T01:14:07.811 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:07 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:14:07.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:07 smithi077 ceph-mon[98867]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-01T01:14:07.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:07 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-12-01T01:14:07.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:07 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:14:08.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:07 smithi203 ceph-mon[103581]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-01T01:14:08.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:07 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-12-01T01:14:08.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:07 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:14:08.723 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:08 smithi137 ceph-mon[103312]: Deploying daemon osd.7 on smithi137 2023-12-01T01:14:08.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:08 smithi077 ceph-mon[98867]: Deploying daemon osd.7 on smithi137 2023-12-01T01:14:09.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:08 smithi203 ceph-mon[103581]: Deploying daemon osd.7 on smithi137 2023-12-01T01:14:09.729 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:09 smithi137 ceph-mon[103312]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-01T01:14:09.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:09 smithi077 ceph-mon[98867]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-01T01:14:10.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:09 smithi203 ceph-mon[103581]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-01T01:14:11.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:11 smithi137 ceph-mon[103312]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-01T01:14:11.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:11 smithi077 ceph-mon[98867]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-01T01:14:12.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:11 smithi203 ceph-mon[103581]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-01T01:14:12.940 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:12 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:12.940 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:12 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:12.940 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:12 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:14:12.941 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:12 smithi137 ceph-mon[103312]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-01T01:14:13.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:12 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:13.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:12 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:13.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:12 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:14:13.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:12 smithi077 ceph-mon[98867]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-01T01:14:13.262 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:12 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:13.262 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:12 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:13.262 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:12 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:14:13.262 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:12 smithi203 ceph-mon[103581]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-01T01:14:14.625 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 7 on host 'smithi137' 2023-12-01T01:14:14.625 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:14:14.623+0000 7fc16cff9700 1 -- 172.21.15.137:0/3167345911 <== mgr.14152 v2:172.21.15.77:6800/3838941679 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc1700611d0 con 0x7fc1580a0520 2023-12-01T01:14:14.626 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:14:14.625+0000 7fc1761ee700 1 -- 172.21.15.137:0/3167345911 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc1580a0520 msgr2=0x7fc1580a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:14:14.626 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:14:14.625+0000 7fc1761ee700 1 --2- 172.21.15.137:0/3167345911 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc1580a0520 0x7fc1580a29e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc164009f50 tx=0x7fc16400b540 comp rx=0 tx=0).stop 2023-12-01T01:14:14.626 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:14:14.625+0000 7fc1761ee700 1 -- 172.21.15.137:0/3167345911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1701018c0 msgr2=0x7fc17019def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:14:14.626 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:14:14.625+0000 7fc1761ee700 1 --2- 172.21.15.137:0/3167345911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1701018c0 0x7fc17019def0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc16800b970 tx=0x7fc16800bc80 comp rx=0 tx=0).stop 2023-12-01T01:14:14.626 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:14:14.625+0000 7fc1761ee700 1 -- 172.21.15.137:0/3167345911 shutdown_connections 2023-12-01T01:14:14.626 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:14:14.625+0000 7fc1761ee700 1 --2- 172.21.15.137:0/3167345911 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc1580a0520 0x7fc1580a29e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:14:14.626 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:14:14.625+0000 7fc1761ee700 1 --2- 172.21.15.137:0/3167345911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1701018c0 0x7fc17019def0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:14:14.626 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:14:14.625+0000 7fc1761ee700 1 --2- 172.21.15.137:0/3167345911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1700ffd20 0x7fc1700693d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:14:14.627 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:14:14.625+0000 7fc1761ee700 1 --2- 172.21.15.137:0/3167345911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc170100f20 0x7fc17019b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:14:14.627 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:14:14.625+0000 7fc1761ee700 1 -- 172.21.15.137:0/3167345911 >> 172.21.15.137:0/3167345911 conn(0x7fc1700fb2a0 msgr2=0x7fc1701044b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:14:14.627 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:14:14.625+0000 7fc1761ee700 1 -- 172.21.15.137:0/3167345911 shutdown_connections 2023-12-01T01:14:14.627 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:14:14.625+0000 7fc1761ee700 1 -- 172.21.15.137:0/3167345911 wait complete. 2023-12-01T01:14:15.040 DEBUG:teuthology.orchestra.run.smithi137:osd.7> sudo journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@osd.7.service 2023-12-01T01:14:15.042 INFO:tasks.cephadm:Deploying osd.8 on smithi203 with /dev/vg_nvme/lv_2... 2023-12-01T01:14:15.042 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-12-01T01:14:15.074 INFO:journalctl@ceph.osd.7.smithi137.stdout:-- Logs begin at Fri 2023-12-01 00:55:04 UTC. -- 2023-12-01T01:14:15.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:15 smithi203 ceph-mon[103581]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-01T01:14:15.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:15 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:15.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:15 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:15.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:15 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:15.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:15 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:15.528 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:15 smithi137 ceph-mon[103312]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-01T01:14:15.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:15 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:15.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:15 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:15.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:15 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:15.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:15 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:15.618 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.c/config 2023-12-01T01:14:15.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:15 smithi077 ceph-mon[98867]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-01T01:14:15.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:15 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:15.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:15 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:15.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:15 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:15.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:15 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:15.973 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:14:15 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:14:15.750+0000 7f49fddc0640 -1 osd.7 0 log_to_monitors true 2023-12-01T01:14:16.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:16 smithi203 ceph-mon[103581]: from='osd.7 [v2:172.21.15.137:6816/3108330766,v1:172.21.15.137:6817/3108330766]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-01T01:14:16.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:16 smithi203 ceph-mon[103581]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-01T01:14:16.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:16 smithi137 ceph-mon[103312]: from='osd.7 [v2:172.21.15.137:6816/3108330766,v1:172.21.15.137:6817/3108330766]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-01T01:14:16.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:16 smithi137 ceph-mon[103312]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-01T01:14:16.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:16 smithi077 ceph-mon[98867]: from='osd.7 [v2:172.21.15.137:6816/3108330766,v1:172.21.15.137:6817/3108330766]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-01T01:14:16.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:16 smithi077 ceph-mon[98867]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-01T01:14:17.282 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-12-01T01:14:17.282 INFO:teuthology.orchestra.run.smithi203.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-12-01T01:14:17.282 INFO:teuthology.orchestra.run.smithi203.stderr: stderr: 10+0 records in 2023-12-01T01:14:17.283 INFO:teuthology.orchestra.run.smithi203.stderr:10+0 records out 2023-12-01T01:14:17.283 INFO:teuthology.orchestra.run.smithi203.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110586 s, 948 MB/s 2023-12-01T01:14:17.283 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping successful for: 2023-12-01T01:14:17.597 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:14:17 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:14:17.306+0000 7f49f2c36700 -1 osd.7 0 waiting for initial osdmap 2023-12-01T01:14:17.598 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:14:17 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:14:17.318+0000 7f49ed228700 -1 osd.7 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-01T01:14:17.598 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:17 smithi137 ceph-mon[103312]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-01T01:14:17.598 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:17 smithi137 ceph-mon[103312]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-12-01T01:14:17.598 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:17 smithi137 ceph-mon[103312]: osdmap e48: 8 total, 7 up, 8 in 2023-12-01T01:14:17.598 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:17 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-01T01:14:17.598 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:17 smithi137 ceph-mon[103312]: from='osd.7 [v2:172.21.15.137:6816/3108330766,v1:172.21.15.137:6817/3108330766]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2023-12-01T01:14:17.598 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:17 smithi137 ceph-mon[103312]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2023-12-01T01:14:17.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:17 smithi077 ceph-mon[98867]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-01T01:14:17.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:17 smithi077 ceph-mon[98867]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-12-01T01:14:17.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:17 smithi077 ceph-mon[98867]: osdmap e48: 8 total, 7 up, 8 in 2023-12-01T01:14:17.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:17 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-01T01:14:17.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:17 smithi077 ceph-mon[98867]: from='osd.7 [v2:172.21.15.137:6816/3108330766,v1:172.21.15.137:6817/3108330766]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2023-12-01T01:14:17.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:17 smithi077 ceph-mon[98867]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2023-12-01T01:14:17.670 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:17 smithi203 ceph-mon[103581]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-01T01:14:17.670 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:17 smithi203 ceph-mon[103581]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-12-01T01:14:17.670 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:17 smithi203 ceph-mon[103581]: osdmap e48: 8 total, 7 up, 8 in 2023-12-01T01:14:17.670 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:17 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-01T01:14:17.670 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:17 smithi203 ceph-mon[103581]: from='osd.7 [v2:172.21.15.137:6816/3108330766,v1:172.21.15.137:6817/3108330766]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2023-12-01T01:14:17.670 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:17 smithi203 ceph-mon[103581]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2023-12-01T01:14:17.671 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi203:vg_nvme/lv_2 2023-12-01T01:14:18.226 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.c/config 2023-12-01T01:14:18.642 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:18 smithi203 ceph-mon[103581]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2023-12-01T01:14:18.643 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:18 smithi203 ceph-mon[103581]: osdmap e49: 8 total, 7 up, 8 in 2023-12-01T01:14:18.643 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:18 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-01T01:14:18.643 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:18 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-01T01:14:18.643 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:18 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:18.643 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:18 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:18.643 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:18 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:14:18.643 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:18 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:14:18.643 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:18 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:14:18.643 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:18 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:18.643 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:18 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:14:18.643 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:18 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:14:18.643 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:18 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:18.644 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:18 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:14:18.644 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:18 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:14:18.644 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:18 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:14:18.644 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:18 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:18.644 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:18 smithi203 ceph-mon[103581]: osd.7 [v2:172.21.15.137:6816/3108330766,v1:172.21.15.137:6817/3108330766] boot 2023-12-01T01:14:18.644 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:18 smithi203 ceph-mon[103581]: osdmap e50: 8 total, 8 up, 8 in 2023-12-01T01:14:18.644 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:18 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-01T01:14:18.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:18 smithi077 ceph-mon[98867]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2023-12-01T01:14:18.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:18 smithi077 ceph-mon[98867]: osdmap e49: 8 total, 7 up, 8 in 2023-12-01T01:14:18.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:18 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-01T01:14:18.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:18 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-01T01:14:18.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:18 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:18.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:18 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:18.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:18 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:14:18.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:18 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:14:18.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:18 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:14:18.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:18 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:18.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:18 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:14:18.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:18 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:14:18.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:18 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:18.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:18 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:14:18.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:18 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:14:18.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:18 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:14:18.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:18 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:18.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:18 smithi077 ceph-mon[98867]: osd.7 [v2:172.21.15.137:6816/3108330766,v1:172.21.15.137:6817/3108330766] boot 2023-12-01T01:14:18.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:18 smithi077 ceph-mon[98867]: osdmap e50: 8 total, 8 up, 8 in 2023-12-01T01:14:18.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:18 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-01T01:14:18.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:18 smithi137 ceph-mon[103312]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2023-12-01T01:14:18.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:18 smithi137 ceph-mon[103312]: osdmap e49: 8 total, 7 up, 8 in 2023-12-01T01:14:18.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:18 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-01T01:14:18.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:18 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-01T01:14:18.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:18 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:18.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:18 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:18.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:18 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:14:18.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:18 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:14:18.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:18 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:14:18.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:18 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:18.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:18 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:14:18.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:18 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:14:18.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:18 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:18.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:18 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:14:18.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:18 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:14:18.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:18 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:14:18.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:18 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:18.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:18 smithi137 ceph-mon[103312]: osd.7 [v2:172.21.15.137:6816/3108330766,v1:172.21.15.137:6817/3108330766] boot 2023-12-01T01:14:18.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:18 smithi137 ceph-mon[103312]: osdmap e50: 8 total, 8 up, 8 in 2023-12-01T01:14:18.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:18 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-01T01:14:19.531 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:19.529+0000 7f447eb18700 1 -- 172.21.15.203:0/4127132091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44780ff100 msgr2=0x7f44780ff520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:14:19.531 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:19.529+0000 7f447eb18700 1 --2- 172.21.15.203:0/4127132091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44780ff100 0x7f44780ff520 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f446c009990 tx=0x7f446c009ca0 comp rx=0 tx=0).stop 2023-12-01T01:14:19.531 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:19.529+0000 7f447eb18700 1 -- 172.21.15.203:0/4127132091 shutdown_connections 2023-12-01T01:14:19.531 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:19.529+0000 7f447eb18700 1 --2- 172.21.15.203:0/4127132091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4478102b20 0x7f44781093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:14:19.531 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:19.529+0000 7f447eb18700 1 --2- 172.21.15.203:0/4127132091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44780ff100 0x7f44780ff520 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:14:19.532 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:19.529+0000 7f447eb18700 1 --2- 172.21.15.203:0/4127132091 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4478102180 0x7f44781025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:14:19.532 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:19.529+0000 7f447eb18700 1 -- 172.21.15.203:0/4127132091 >> 172.21.15.203:0/4127132091 conn(0x7f44780faa40 msgr2=0x7f44780fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:14:19.532 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:19.529+0000 7f447eb18700 1 -- 172.21.15.203:0/4127132091 shutdown_connections 2023-12-01T01:14:19.532 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:19.529+0000 7f447eb18700 1 -- 172.21.15.203:0/4127132091 wait complete. 2023-12-01T01:14:19.534 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:19.530+0000 7f447eb18700 1 Processor -- start 2023-12-01T01:14:19.534 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:19.530+0000 7f447eb18700 1 -- start start 2023-12-01T01:14:19.534 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:19.530+0000 7f447eb18700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44780ff100 0x7f4478196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:14:19.534 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:19.530+0000 7f447eb18700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4478102180 0x7f44781970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:14:19.534 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:19.530+0000 7f447eb18700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4478102b20 0x7f447819e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:14:19.534 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:19.530+0000 7f447eb18700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447819e710 con 0x7f44780ff100 2023-12-01T01:14:19.534 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:19.530+0000 7f447eb18700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447819e850 con 0x7f4478102b20 2023-12-01T01:14:19.534 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:19.530+0000 7f447eb18700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447819e9e0 con 0x7f4478102180 2023-12-01T01:14:19.534 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:19.530+0000 7f4477fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4478102180 0x7f44781970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:14:19.534 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:19.530+0000 7f4477fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4478102180 0x7f44781970c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:59388/0 (socket says 172.21.15.203:59388) 2023-12-01T01:14:19.534 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:19.530+0000 7f4477fff700 1 -- 172.21.15.203:0/633596766 learned_addr learned my addr 172.21.15.203:0/633596766 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-12-01T01:14:19.535 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:19.530+0000 7f447c8b4700 1 --2- 172.21.15.203:0/633596766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44780ff100 0x7f4478196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:14:19.535 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:19.530+0000 7f447d0b5700 1 --2- 172.21.15.203:0/633596766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4478102b20 0x7f447819e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:14:19.535 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:19.530+0000 7f4477fff700 1 -- 172.21.15.203:0/633596766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4478102b20 msgr2=0x7f447819e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:14:19.535 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:19.530+0000 7f4477fff700 1 --2- 172.21.15.203:0/633596766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4478102b20 0x7f447819e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:14:19.535 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:19.530+0000 7f4477fff700 1 -- 172.21.15.203:0/633596766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44780ff100 msgr2=0x7f4478196b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:14:19.535 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:19.530+0000 7f4477fff700 1 --2- 172.21.15.203:0/633596766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44780ff100 0x7f4478196b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:14:19.535 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:19.530+0000 7f4477fff700 1 -- 172.21.15.203:0/633596766 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f446c0095f0 con 0x7f4478102180 2023-12-01T01:14:19.535 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:19.530+0000 7f447d0b5700 1 --2- 172.21.15.203:0/633596766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4478102b20 0x7f447819e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:14:19.535 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:19.530+0000 7f4477fff700 1 --2- 172.21.15.203:0/633596766 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4478102180 0x7f44781970c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f446400b560 tx=0x7f446400b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:14:19.535 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:19.531+0000 7f4475ffb700 1 -- 172.21.15.203:0/633596766 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4464011830 con 0x7f4478102180 2023-12-01T01:14:19.535 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:19.531+0000 7f4475ffb700 1 -- 172.21.15.203:0/633596766 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4464011e50 con 0x7f4478102180 2023-12-01T01:14:19.535 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:19.531+0000 7f447eb18700 1 -- 172.21.15.203:0/633596766 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f447819ecc0 con 0x7f4478102180 2023-12-01T01:14:19.535 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:19.531+0000 7f4475ffb700 1 -- 172.21.15.203:0/633596766 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446400f2e0 con 0x7f4478102180 2023-12-01T01:14:19.535 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:19.531+0000 7f447eb18700 1 -- 172.21.15.203:0/633596766 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f447819f1c0 con 0x7f4478102180 2023-12-01T01:14:19.535 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:19.532+0000 7f447eb18700 1 -- 172.21.15.203:0/633596766 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f447810c690 con 0x7f4478102180 2023-12-01T01:14:19.536 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:19.533+0000 7f4475ffb700 1 -- 172.21.15.203:0/633596766 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f4464010580 con 0x7f4478102180 2023-12-01T01:14:19.536 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:19.533+0000 7f4475ffb700 1 --2- 172.21.15.203:0/633596766 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f44680a05a0 0x7f44680a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:14:19.536 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:19.533+0000 7f4475ffb700 1 -- 172.21.15.203:0/633596766 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5426+0+0 (secure 0 0 0) 0x7f44640cae00 con 0x7f4478102180 2023-12-01T01:14:19.536 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:19.534+0000 7f447c8b4700 1 --2- 172.21.15.203:0/633596766 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f44680a05a0 0x7f44680a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:14:19.537 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:19.534+0000 7f447c8b4700 1 --2- 172.21.15.203:0/633596766 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f44680a05a0 0x7f44680a2a60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f446c00b5c0 tx=0x7f446c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:14:19.539 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:19.537+0000 7f4475ffb700 1 -- 172.21.15.203:0/633596766 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f44640954e0 con 0x7f4478102180 2023-12-01T01:14:19.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:19 smithi077 ceph-mon[98867]: purged_snaps scrub starts 2023-12-01T01:14:19.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:19 smithi077 ceph-mon[98867]: purged_snaps scrub ok 2023-12-01T01:14:19.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:19 smithi077 ceph-mon[98867]: Detected new or changed devices on smithi137 2023-12-01T01:14:19.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:19 smithi077 ceph-mon[98867]: Adjusting osd_memory_target on smithi137 to 5722M 2023-12-01T01:14:19.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:19 smithi077 ceph-mon[98867]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-01T01:14:19.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:19 smithi077 ceph-mon[98867]: osdmap e51: 8 total, 8 up, 8 in 2023-12-01T01:14:19.738 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:19.735+0000 7f447eb18700 1 -- 172.21.15.203:0/633596766 --> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f44780611d0 con 0x7f44680a05a0 2023-12-01T01:14:21.038 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:20 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-01T01:14:21.038 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:20 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-01T01:14:21.038 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:20 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:14:21.038 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:19 smithi137 ceph-mon[103312]: purged_snaps scrub starts 2023-12-01T01:14:21.038 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:19 smithi137 ceph-mon[103312]: purged_snaps scrub ok 2023-12-01T01:14:21.039 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:19 smithi137 ceph-mon[103312]: Detected new or changed devices on smithi137 2023-12-01T01:14:21.039 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:19 smithi137 ceph-mon[103312]: Adjusting osd_memory_target on smithi137 to 5722M 2023-12-01T01:14:21.039 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:19 smithi137 ceph-mon[103312]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-01T01:14:21.039 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:19 smithi137 ceph-mon[103312]: osdmap e51: 8 total, 8 up, 8 in 2023-12-01T01:14:21.039 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:20 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-01T01:14:21.039 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:20 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-01T01:14:21.039 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:20 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:14:21.039 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:19 smithi203 ceph-mon[103581]: purged_snaps scrub starts 2023-12-01T01:14:21.039 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:19 smithi203 ceph-mon[103581]: purged_snaps scrub ok 2023-12-01T01:14:21.039 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:19 smithi203 ceph-mon[103581]: Detected new or changed devices on smithi137 2023-12-01T01:14:21.039 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:19 smithi203 ceph-mon[103581]: Adjusting osd_memory_target on smithi137 to 5722M 2023-12-01T01:14:21.039 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:19 smithi203 ceph-mon[103581]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-01T01:14:21.039 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:19 smithi203 ceph-mon[103581]: osdmap e51: 8 total, 8 up, 8 in 2023-12-01T01:14:21.040 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:20 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-01T01:14:21.040 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:20 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-01T01:14:21.040 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:20 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:14:21.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:21 smithi077 ceph-mon[98867]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:14:21.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:21 smithi077 ceph-mon[98867]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-01T01:14:21.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:21 smithi077 ceph-mon[98867]: osdmap e52: 8 total, 8 up, 8 in 2023-12-01T01:14:21.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:21 smithi203 ceph-mon[103581]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:14:21.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:21 smithi203 ceph-mon[103581]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-01T01:14:21.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:21 smithi203 ceph-mon[103581]: osdmap e52: 8 total, 8 up, 8 in 2023-12-01T01:14:21.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:21 smithi137 ceph-mon[103312]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:14:21.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:21 smithi137 ceph-mon[103312]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-01T01:14:21.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:21 smithi137 ceph-mon[103312]: osdmap e52: 8 total, 8 up, 8 in 2023-12-01T01:14:22.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:22 smithi077 ceph-mon[98867]: osdmap e53: 8 total, 8 up, 8 in 2023-12-01T01:14:22.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:22 smithi077 ceph-mon[98867]: from='client.? 172.21.15.203:0/184001395' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c09050d-3e43-410c-8cd7-bcb891af56b2"}]: dispatch 2023-12-01T01:14:22.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:22 smithi077 ceph-mon[98867]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c09050d-3e43-410c-8cd7-bcb891af56b2"}]: dispatch 2023-12-01T01:14:22.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:22 smithi077 ceph-mon[98867]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c09050d-3e43-410c-8cd7-bcb891af56b2"}]': finished 2023-12-01T01:14:22.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:22 smithi077 ceph-mon[98867]: osdmap e54: 9 total, 8 up, 9 in 2023-12-01T01:14:22.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:22 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-01T01:14:22.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:22 smithi203 ceph-mon[103581]: osdmap e53: 8 total, 8 up, 8 in 2023-12-01T01:14:22.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:22 smithi203 ceph-mon[103581]: from='client.? 172.21.15.203:0/184001395' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c09050d-3e43-410c-8cd7-bcb891af56b2"}]: dispatch 2023-12-01T01:14:22.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:22 smithi203 ceph-mon[103581]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c09050d-3e43-410c-8cd7-bcb891af56b2"}]: dispatch 2023-12-01T01:14:22.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:22 smithi203 ceph-mon[103581]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c09050d-3e43-410c-8cd7-bcb891af56b2"}]': finished 2023-12-01T01:14:22.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:22 smithi203 ceph-mon[103581]: osdmap e54: 9 total, 8 up, 9 in 2023-12-01T01:14:22.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:22 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-01T01:14:22.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:22 smithi137 ceph-mon[103312]: osdmap e53: 8 total, 8 up, 8 in 2023-12-01T01:14:22.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:22 smithi137 ceph-mon[103312]: from='client.? 172.21.15.203:0/184001395' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c09050d-3e43-410c-8cd7-bcb891af56b2"}]: dispatch 2023-12-01T01:14:22.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:22 smithi137 ceph-mon[103312]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c09050d-3e43-410c-8cd7-bcb891af56b2"}]: dispatch 2023-12-01T01:14:22.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:22 smithi137 ceph-mon[103312]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c09050d-3e43-410c-8cd7-bcb891af56b2"}]': finished 2023-12-01T01:14:22.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:22 smithi137 ceph-mon[103312]: osdmap e54: 9 total, 8 up, 9 in 2023-12-01T01:14:22.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:22 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-01T01:14:23.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:23 smithi077 ceph-mon[98867]: pgmap v163: 1 pgs: 1 remapped+peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-01T01:14:23.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:23 smithi077 ceph-mon[98867]: from='client.? 172.21.15.203:0/694649594' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-01T01:14:23.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:23 smithi203 ceph-mon[103581]: pgmap v163: 1 pgs: 1 remapped+peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-01T01:14:23.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:23 smithi203 ceph-mon[103581]: from='client.? 172.21.15.203:0/694649594' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-01T01:14:23.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:23 smithi137 ceph-mon[103312]: pgmap v163: 1 pgs: 1 remapped+peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-01T01:14:23.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:23 smithi137 ceph-mon[103312]: from='client.? 172.21.15.203:0/694649594' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-01T01:14:25.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:25 smithi077 ceph-mon[98867]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-01T01:14:25.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:25 smithi203 ceph-mon[103581]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-01T01:14:25.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:25 smithi137 ceph-mon[103312]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-01T01:14:27.893 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:27 smithi203 ceph-mon[103581]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-01T01:14:27.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:27 smithi077 ceph-mon[98867]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-01T01:14:28.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:27 smithi137 ceph-mon[103312]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-01T01:14:29.842 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:29 smithi203 ceph-mon[103581]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-01T01:14:29.843 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:29 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-12-01T01:14:29.843 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:29 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:14:29.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:29 smithi077 ceph-mon[98867]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-01T01:14:29.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:29 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-12-01T01:14:29.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:29 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:14:30.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:29 smithi137 ceph-mon[103312]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-01T01:14:30.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:29 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-12-01T01:14:30.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:29 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:14:30.645 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:30 smithi203 ceph-mon[103581]: Deploying daemon osd.8 on smithi203 2023-12-01T01:14:30.903 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:30 smithi077 ceph-mon[98867]: Deploying daemon osd.8 on smithi203 2023-12-01T01:14:31.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:30 smithi137 ceph-mon[103312]: Deploying daemon osd.8 on smithi203 2023-12-01T01:14:31.752 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:31 smithi203 ceph-mon[103581]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-01T01:14:31.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:31 smithi077 ceph-mon[98867]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-01T01:14:32.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:31 smithi137 ceph-mon[103312]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-01T01:14:33.816 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:33 smithi203 ceph-mon[103581]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-01T01:14:33.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:33 smithi077 ceph-mon[98867]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-01T01:14:34.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:33 smithi137 ceph-mon[103312]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-01T01:14:34.836 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:34 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:14:34.836 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:34 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:34.836 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:34 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:34.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:34 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:14:34.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:34 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:34.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:34 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:35.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:34 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:14:35.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:34 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:35.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:34 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:35.845 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:35 smithi203 ceph-mon[103581]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-01T01:14:35.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:35 smithi077 ceph-mon[98867]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-01T01:14:36.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:35 smithi137 ceph-mon[103312]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-01T01:14:36.904 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:36.902+0000 7f4475ffb700 1 -- 172.21.15.203:0/633596766 <== mgr.14152 v2:172.21.15.77:6800/3838941679 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f44780611d0 con 0x7f44680a05a0 2023-12-01T01:14:36.904 INFO:teuthology.orchestra.run.smithi203.stdout:Created osd(s) 8 on host 'smithi203' 2023-12-01T01:14:36.905 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:36.904+0000 7f447eb18700 1 -- 172.21.15.203:0/633596766 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f44680a05a0 msgr2=0x7f44680a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:14:36.906 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:36.904+0000 7f447eb18700 1 --2- 172.21.15.203:0/633596766 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f44680a05a0 0x7f44680a2a60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f446c00b5c0 tx=0x7f446c005fd0 comp rx=0 tx=0).stop 2023-12-01T01:14:36.906 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:36.905+0000 7f447eb18700 1 -- 172.21.15.203:0/633596766 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4478102180 msgr2=0x7f44781970c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:14:36.906 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:36.905+0000 7f447eb18700 1 --2- 172.21.15.203:0/633596766 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4478102180 0x7f44781970c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f446400b560 tx=0x7f446400b870 comp rx=0 tx=0).stop 2023-12-01T01:14:36.906 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:36.905+0000 7f447eb18700 1 -- 172.21.15.203:0/633596766 shutdown_connections 2023-12-01T01:14:36.906 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:36.905+0000 7f447eb18700 1 --2- 172.21.15.203:0/633596766 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f44680a05a0 0x7f44680a2a60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:14:36.906 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:36.905+0000 7f447eb18700 1 --2- 172.21.15.203:0/633596766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4478102b20 0x7f447819e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:14:36.906 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:36.905+0000 7f447eb18700 1 --2- 172.21.15.203:0/633596766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44780ff100 0x7f4478196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:14:36.906 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:36.905+0000 7f447eb18700 1 --2- 172.21.15.203:0/633596766 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4478102180 0x7f44781970c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:14:36.906 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:36.905+0000 7f447eb18700 1 -- 172.21.15.203:0/633596766 >> 172.21.15.203:0/633596766 conn(0x7f44780faa40 msgr2=0x7f44780fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:14:36.906 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:36.905+0000 7f447eb18700 1 -- 172.21.15.203:0/633596766 shutdown_connections 2023-12-01T01:14:36.906 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:14:36.905+0000 7f447eb18700 1 -- 172.21.15.203:0/633596766 wait complete. 2023-12-01T01:14:37.087 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:37 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:37.088 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:37 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:37.088 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:37 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:14:37.088 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:37 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:14:37.088 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:37 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:37.088 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:37 smithi203 ceph-mon[103581]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-01T01:14:37.088 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:37 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:37.088 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:37 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:37.088 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:37 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:14:37.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:37 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:37.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:37 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:37.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:37 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:14:37.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:37 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:14:37.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:37 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:37.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:37 smithi077 ceph-mon[98867]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-01T01:14:37.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:37 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:37.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:37 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:37.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:37 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:14:37.417 DEBUG:teuthology.orchestra.run.smithi203:osd.8> sudo journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@osd.8.service 2023-12-01T01:14:37.419 INFO:tasks.cephadm:Deploying osd.9 on smithi077 with /dev/vg_nvme/lv_1... 2023-12-01T01:14:37.419 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-12-01T01:14:37.448 INFO:journalctl@ceph.osd.8.smithi203.stdout:-- Logs begin at Fri 2023-12-01 00:55:29 UTC. -- 2023-12-01T01:14:37.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:37 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:37.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:37 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:37.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:37 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:14:37.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:37 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:14:37.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:37 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:37.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:37 smithi137 ceph-mon[103312]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-01T01:14:37.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:37 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:37.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:37 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:37.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:37 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:14:37.985 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:14:38.049 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:14:37 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:14:37.764+0000 7f49ccd16640 -1 osd.8 0 log_to_monitors true 2023-12-01T01:14:38.336 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:38 smithi077 ceph-mon[98867]: from='osd.8 [v2:172.21.15.203:6816/1161158776,v1:172.21.15.203:6817/1161158776]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-01T01:14:38.336 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:38 smithi077 ceph-mon[98867]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-01T01:14:38.487 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:38 smithi203 ceph-mon[103581]: from='osd.8 [v2:172.21.15.203:6816/1161158776,v1:172.21.15.203:6817/1161158776]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-01T01:14:38.487 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:38 smithi203 ceph-mon[103581]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-01T01:14:38.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:38 smithi137 ceph-mon[103312]: from='osd.8 [v2:172.21.15.203:6816/1161158776,v1:172.21.15.203:6817/1161158776]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-01T01:14:38.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:38 smithi137 ceph-mon[103312]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-01T01:14:39.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:39 smithi077 ceph-mon[98867]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-12-01T01:14:39.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:39 smithi077 ceph-mon[98867]: osdmap e55: 9 total, 8 up, 9 in 2023-12-01T01:14:39.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:39 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-01T01:14:39.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:39 smithi077 ceph-mon[98867]: from='osd.8 [v2:172.21.15.203:6816/1161158776,v1:172.21.15.203:6817/1161158776]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-12-01T01:14:39.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:39 smithi077 ceph-mon[98867]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-12-01T01:14:39.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:39 smithi077 ceph-mon[98867]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-01T01:14:39.362 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-12-01T01:14:39.362 INFO:teuthology.orchestra.run.smithi077.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-12-01T01:14:39.362 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10+0 records in 2023-12-01T01:14:39.362 INFO:teuthology.orchestra.run.smithi077.stderr:10+0 records out 2023-12-01T01:14:39.362 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112448 s, 932 MB/s 2023-12-01T01:14:39.362 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping successful for: 2023-12-01T01:14:39.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:14:39 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:14:39.062+0000 7f49c1b8c700 -1 osd.8 0 waiting for initial osdmap 2023-12-01T01:14:39.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:14:39 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:14:39.075+0000 7f49bc17e700 -1 osd.8 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-01T01:14:39.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:39 smithi203 ceph-mon[103581]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-12-01T01:14:39.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:39 smithi203 ceph-mon[103581]: osdmap e55: 9 total, 8 up, 9 in 2023-12-01T01:14:39.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:39 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-01T01:14:39.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:39 smithi203 ceph-mon[103581]: from='osd.8 [v2:172.21.15.203:6816/1161158776,v1:172.21.15.203:6817/1161158776]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-12-01T01:14:39.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:39 smithi203 ceph-mon[103581]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-12-01T01:14:39.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:39 smithi203 ceph-mon[103581]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-01T01:14:39.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:39 smithi137 ceph-mon[103312]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-12-01T01:14:39.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:39 smithi137 ceph-mon[103312]: osdmap e55: 9 total, 8 up, 9 in 2023-12-01T01:14:39.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:39 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-01T01:14:39.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:39 smithi137 ceph-mon[103312]: from='osd.8 [v2:172.21.15.203:6816/1161158776,v1:172.21.15.203:6817/1161158776]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-12-01T01:14:39.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:39 smithi137 ceph-mon[103312]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-12-01T01:14:39.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:39 smithi137 ceph-mon[103312]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-01T01:14:39.741 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi077:vg_nvme/lv_1 2023-12-01T01:14:40.296 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:14:40.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:40 smithi077 ceph-mon[98867]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2023-12-01T01:14:40.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:40 smithi077 ceph-mon[98867]: osdmap e56: 9 total, 8 up, 9 in 2023-12-01T01:14:40.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:40 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-01T01:14:40.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:40 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-01T01:14:40.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:40 smithi203 ceph-mon[103581]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2023-12-01T01:14:40.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:40 smithi203 ceph-mon[103581]: osdmap e56: 9 total, 8 up, 9 in 2023-12-01T01:14:40.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:40 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-01T01:14:40.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:40 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-01T01:14:40.528 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:40 smithi137 ceph-mon[103312]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2023-12-01T01:14:40.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:40 smithi137 ceph-mon[103312]: osdmap e56: 9 total, 8 up, 9 in 2023-12-01T01:14:40.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:40 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-01T01:14:40.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:40 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-01T01:14:41.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:41 smithi077 ceph-mon[98867]: purged_snaps scrub starts 2023-12-01T01:14:41.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:41 smithi077 ceph-mon[98867]: purged_snaps scrub ok 2023-12-01T01:14:41.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:41 smithi077 ceph-mon[98867]: osd.8 [v2:172.21.15.203:6816/1161158776,v1:172.21.15.203:6817/1161158776] boot 2023-12-01T01:14:41.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:41 smithi077 ceph-mon[98867]: osdmap e57: 9 total, 9 up, 9 in 2023-12-01T01:14:41.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:41 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-01T01:14:41.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:41 smithi077 ceph-mon[98867]: Detected new or changed devices on smithi203 2023-12-01T01:14:41.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:41 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:41.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:41 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:41.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:41 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:14:41.406 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:41 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:14:41.406 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:41 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:14:41.406 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:41 smithi077 ceph-mon[98867]: Adjusting osd_memory_target on smithi203 to 5722M 2023-12-01T01:14:41.406 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:41 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:41.406 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:41 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:14:41.406 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:41 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:14:41.406 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:41 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:41.406 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:41 smithi077 ceph-mon[98867]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-01T01:14:41.460 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:41.459+0000 7fe006051700 1 -- 172.21.15.77:0/2068622813 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0000ff140 msgr2=0x7fe0000ff560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:14:41.461 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:41.459+0000 7fe006051700 1 --2- 172.21.15.77:0/2068622813 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0000ff140 0x7fe0000ff560 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdff00098f0 tx=0x7fdff0009c00 comp rx=0 tx=0).stop 2023-12-01T01:14:41.461 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:41.459+0000 7fe006051700 1 -- 172.21.15.77:0/2068622813 shutdown_connections 2023-12-01T01:14:41.461 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:41.459+0000 7fe006051700 1 --2- 172.21.15.77:0/2068622813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe000102ac0 0x7fe000109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:14:41.461 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:41.459+0000 7fe006051700 1 --2- 172.21.15.77:0/2068622813 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0000ff140 0x7fe0000ff560 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:14:41.461 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:41.459+0000 7fe006051700 1 --2- 172.21.15.77:0/2068622813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe000102120 0x7fe000102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:14:41.461 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:41.459+0000 7fe006051700 1 -- 172.21.15.77:0/2068622813 >> 172.21.15.77:0/2068622813 conn(0x7fe0000faaa0 msgr2=0x7fe0000fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:14:41.461 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:41.459+0000 7fe006051700 1 -- 172.21.15.77:0/2068622813 shutdown_connections 2023-12-01T01:14:41.461 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:41.459+0000 7fe006051700 1 -- 172.21.15.77:0/2068622813 wait complete. 2023-12-01T01:14:41.461 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:41.460+0000 7fe006051700 1 Processor -- start 2023-12-01T01:14:41.466 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:41.460+0000 7fe006051700 1 -- start start 2023-12-01T01:14:41.466 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:41.460+0000 7fe006051700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0000ff140 0x7fe000196b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:14:41.466 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:41.460+0000 7fe006051700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe000102120 0x7fe000197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:14:41.466 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:41.460+0000 7fe006051700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe000102ac0 0x7fe00019e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:14:41.466 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:41.460+0000 7fe006051700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00019e6a0 con 0x7fe000102ac0 2023-12-01T01:14:41.466 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:41.460+0000 7fe006051700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00019e7e0 con 0x7fe000102120 2023-12-01T01:14:41.467 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:41.460+0000 7fe006051700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00019e970 con 0x7fe0000ff140 2023-12-01T01:14:41.467 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:41.460+0000 7fdffffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe000102ac0 0x7fe00019e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:14:41.467 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:41.460+0000 7fdffffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe000102ac0 0x7fe00019e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56916/0 (socket says 172.21.15.77:56916) 2023-12-01T01:14:41.467 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:41.460+0000 7fdffffff700 1 -- 172.21.15.77:0/3718364978 learned_addr learned my addr 172.21.15.77:0/3718364978 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:14:41.467 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:41.460+0000 7fdfff7fe700 1 --2- 172.21.15.77:0/3718364978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0000ff140 0x7fe000196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:14:41.467 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:41.460+0000 7fdffeffd700 1 --2- 172.21.15.77:0/3718364978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe000102120 0x7fe000197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:14:41.467 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:41.460+0000 7fdffffff700 1 -- 172.21.15.77:0/3718364978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0000ff140 msgr2=0x7fe000196b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:14:41.467 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:41.460+0000 7fdffffff700 1 --2- 172.21.15.77:0/3718364978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0000ff140 0x7fe000196b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:14:41.467 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:41.460+0000 7fdffffff700 1 -- 172.21.15.77:0/3718364978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe000102120 msgr2=0x7fe000197050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:14:41.467 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:41.460+0000 7fdffffff700 1 --2- 172.21.15.77:0/3718364978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe000102120 0x7fe000197050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:14:41.467 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:41.460+0000 7fdffffff700 1 -- 172.21.15.77:0/3718364978 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff0009520 con 0x7fe000102ac0 2023-12-01T01:14:41.467 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:41.461+0000 7fdffffff700 1 --2- 172.21.15.77:0/3718364978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe000102ac0 0x7fe00019e0d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdff800c140 tx=0x7fdff800c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:14:41.467 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:41.461+0000 7fdffcff9700 1 -- 172.21.15.77:0/3718364978 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff8017070 con 0x7fe000102ac0 2023-12-01T01:14:41.468 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:41.461+0000 7fdffcff9700 1 -- 172.21.15.77:0/3718364978 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdff800e050 con 0x7fe000102ac0 2023-12-01T01:14:41.468 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:41.461+0000 7fe006051700 1 -- 172.21.15.77:0/3718364978 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe00019ebf0 con 0x7fe000102ac0 2023-12-01T01:14:41.468 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:41.461+0000 7fdffcff9700 1 -- 172.21.15.77:0/3718364978 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff8007980 con 0x7fe000102ac0 2023-12-01T01:14:41.468 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:41.461+0000 7fe006051700 1 -- 172.21.15.77:0/3718364978 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe00019f0f0 con 0x7fe000102ac0 2023-12-01T01:14:41.468 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:41.462+0000 7fe006051700 1 -- 172.21.15.77:0/3718364978 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe000108540 con 0x7fe000102ac0 2023-12-01T01:14:41.468 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:41.463+0000 7fdffcff9700 1 -- 172.21.15.77:0/3718364978 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fdff8004ad0 con 0x7fe000102ac0 2023-12-01T01:14:41.468 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:41.463+0000 7fdffcff9700 1 --2- 172.21.15.77:0/3718364978 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fdfe80a05a0 0x7fdfe80a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:14:41.468 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:41.464+0000 7fdfff7fe700 1 --2- 172.21.15.77:0/3718364978 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fdfe80a05a0 0x7fdfe80a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:14:41.468 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:41.464+0000 7fdffcff9700 1 -- 172.21.15.77:0/3718364978 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5858+0+0 (secure 0 0 0) 0x7fdff80caab0 con 0x7fe000102ac0 2023-12-01T01:14:41.468 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:41.464+0000 7fdfff7fe700 1 --2- 172.21.15.77:0/3718364978 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fdfe80a05a0 0x7fdfe80a2a60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdff00098c0 tx=0x7fdff0019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:14:41.469 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:41.468+0000 7fdffcff9700 1 -- 172.21.15.77:0/3718364978 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fdff8094fe0 con 0x7fe000102ac0 2023-12-01T01:14:41.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:41 smithi203 ceph-mon[103581]: purged_snaps scrub starts 2023-12-01T01:14:41.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:41 smithi203 ceph-mon[103581]: purged_snaps scrub ok 2023-12-01T01:14:41.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:41 smithi203 ceph-mon[103581]: osd.8 [v2:172.21.15.203:6816/1161158776,v1:172.21.15.203:6817/1161158776] boot 2023-12-01T01:14:41.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:41 smithi203 ceph-mon[103581]: osdmap e57: 9 total, 9 up, 9 in 2023-12-01T01:14:41.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:41 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-01T01:14:41.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:41 smithi203 ceph-mon[103581]: Detected new or changed devices on smithi203 2023-12-01T01:14:41.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:41 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:41.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:41 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:41.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:41 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:14:41.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:41 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:14:41.514 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:41 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:14:41.514 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:41 smithi203 ceph-mon[103581]: Adjusting osd_memory_target on smithi203 to 5722M 2023-12-01T01:14:41.514 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:41 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:41.514 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:41 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:14:41.514 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:41 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:14:41.514 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:41 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:41.515 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:41 smithi203 ceph-mon[103581]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-01T01:14:41.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:41 smithi137 ceph-mon[103312]: purged_snaps scrub starts 2023-12-01T01:14:41.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:41 smithi137 ceph-mon[103312]: purged_snaps scrub ok 2023-12-01T01:14:41.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:41 smithi137 ceph-mon[103312]: osd.8 [v2:172.21.15.203:6816/1161158776,v1:172.21.15.203:6817/1161158776] boot 2023-12-01T01:14:41.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:41 smithi137 ceph-mon[103312]: osdmap e57: 9 total, 9 up, 9 in 2023-12-01T01:14:41.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:41 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-01T01:14:41.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:41 smithi137 ceph-mon[103312]: Detected new or changed devices on smithi203 2023-12-01T01:14:41.530 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:41 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:41.530 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:41 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:41.530 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:41 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:14:41.530 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:41 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:14:41.530 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:41 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:14:41.530 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:41 smithi137 ceph-mon[103312]: Adjusting osd_memory_target on smithi203 to 5722M 2023-12-01T01:14:41.530 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:41 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:41.530 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:41 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:14:41.530 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:41 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:14:41.530 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:41 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:41.530 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:41 smithi137 ceph-mon[103312]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-01T01:14:41.667 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:41.666+0000 7fe006051700 1 -- 172.21.15.77:0/3718364978 --> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fe0000611d0 con 0x7fdfe80a05a0 2023-12-01T01:14:42.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:42 smithi077 ceph-mon[98867]: osdmap e58: 9 total, 9 up, 9 in 2023-12-01T01:14:42.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:42 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-01T01:14:42.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:42 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-01T01:14:42.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:42 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:14:42.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:42 smithi203 ceph-mon[103581]: osdmap e58: 9 total, 9 up, 9 in 2023-12-01T01:14:42.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:42 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-01T01:14:42.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:42 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-01T01:14:42.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:42 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:14:42.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:42 smithi137 ceph-mon[103312]: osdmap e58: 9 total, 9 up, 9 in 2023-12-01T01:14:42.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:42 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-01T01:14:42.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:42 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-01T01:14:42.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:42 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:14:43.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:43 smithi077 ceph-mon[98867]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:14:43.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:43 smithi077 ceph-mon[98867]: osdmap e59: 9 total, 9 up, 9 in 2023-12-01T01:14:43.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:43 smithi077 ceph-mon[98867]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-01T01:14:43.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:43 smithi203 ceph-mon[103581]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:14:43.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:43 smithi203 ceph-mon[103581]: osdmap e59: 9 total, 9 up, 9 in 2023-12-01T01:14:43.514 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:43 smithi203 ceph-mon[103581]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-01T01:14:43.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:43 smithi137 ceph-mon[103312]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:14:43.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:43 smithi137 ceph-mon[103312]: osdmap e59: 9 total, 9 up, 9 in 2023-12-01T01:14:43.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:43 smithi137 ceph-mon[103312]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-12-01T01:14:44.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:44 smithi077 ceph-mon[98867]: osdmap e60: 9 total, 9 up, 9 in 2023-12-01T01:14:44.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:44 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/1387352684' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "527ecb49-95ec-48bd-9872-3b07fae5c5ba"}]: dispatch 2023-12-01T01:14:44.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:44 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/1387352684' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "527ecb49-95ec-48bd-9872-3b07fae5c5ba"}]': finished 2023-12-01T01:14:44.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:44 smithi077 ceph-mon[98867]: osdmap e61: 10 total, 9 up, 10 in 2023-12-01T01:14:44.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:44 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-01T01:14:44.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:44 smithi203 ceph-mon[103581]: osdmap e60: 9 total, 9 up, 9 in 2023-12-01T01:14:44.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:44 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/1387352684' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "527ecb49-95ec-48bd-9872-3b07fae5c5ba"}]: dispatch 2023-12-01T01:14:44.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:44 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/1387352684' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "527ecb49-95ec-48bd-9872-3b07fae5c5ba"}]': finished 2023-12-01T01:14:44.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:44 smithi203 ceph-mon[103581]: osdmap e61: 10 total, 9 up, 10 in 2023-12-01T01:14:44.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:44 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-01T01:14:44.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:44 smithi137 ceph-mon[103312]: osdmap e60: 9 total, 9 up, 9 in 2023-12-01T01:14:44.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:44 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/1387352684' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "527ecb49-95ec-48bd-9872-3b07fae5c5ba"}]: dispatch 2023-12-01T01:14:44.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:44 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/1387352684' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "527ecb49-95ec-48bd-9872-3b07fae5c5ba"}]': finished 2023-12-01T01:14:44.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:44 smithi137 ceph-mon[103312]: osdmap e61: 10 total, 9 up, 10 in 2023-12-01T01:14:44.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:44 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-01T01:14:45.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:45 smithi077 ceph-mon[98867]: pgmap v181: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-01T01:14:45.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:45 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/146495784' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-01T01:14:45.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:45 smithi203 ceph-mon[103581]: pgmap v181: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-01T01:14:45.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:45 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/146495784' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-01T01:14:45.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:45 smithi137 ceph-mon[103312]: pgmap v181: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-01T01:14:45.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:45 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/146495784' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-01T01:14:47.853 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:47 smithi077 ceph-mon[98867]: pgmap v182: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2023-12-01T01:14:48.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:47 smithi203 ceph-mon[103581]: pgmap v182: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2023-12-01T01:14:48.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:47 smithi137 ceph-mon[103312]: pgmap v182: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2023-12-01T01:14:49.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:49 smithi077 ceph-mon[98867]: pgmap v183: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2023-12-01T01:14:50.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:49 smithi203 ceph-mon[103581]: pgmap v183: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2023-12-01T01:14:50.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:49 smithi137 ceph-mon[103312]: pgmap v183: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2023-12-01T01:14:50.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:50 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-12-01T01:14:50.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:50 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:14:51.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:50 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-12-01T01:14:51.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:50 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:14:51.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:50 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-12-01T01:14:51.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:50 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:14:51.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:51 smithi077 ceph-mon[98867]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-12-01T01:14:51.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:51 smithi077 ceph-mon[98867]: Deploying daemon osd.9 on smithi077 2023-12-01T01:14:52.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:51 smithi203 ceph-mon[103581]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-12-01T01:14:52.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:51 smithi203 ceph-mon[103581]: Deploying daemon osd.9 on smithi077 2023-12-01T01:14:52.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:51 smithi137 ceph-mon[103312]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-12-01T01:14:52.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:51 smithi137 ceph-mon[103312]: Deploying daemon osd.9 on smithi077 2023-12-01T01:14:53.867 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:53 smithi077 ceph-mon[98867]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2023-12-01T01:14:54.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:53 smithi203 ceph-mon[103581]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2023-12-01T01:14:54.032 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:53 smithi137 ceph-mon[103312]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2023-12-01T01:14:55.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:55 smithi077 ceph-mon[98867]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-12-01T01:14:55.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:55 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:55.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:55 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:14:55.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:55 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:56.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:55 smithi203 ceph-mon[103581]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-12-01T01:14:56.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:55 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:56.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:55 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:14:56.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:55 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:56.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:55 smithi137 ceph-mon[103312]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-12-01T01:14:56.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:55 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:56.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:55 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:14:56.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:55 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:57.165 INFO:teuthology.orchestra.run.smithi077.stdout:Created osd(s) 9 on host 'smithi077' 2023-12-01T01:14:57.165 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:57.163+0000 7fdffcff9700 1 -- 172.21.15.77:0/3718364978 <== mgr.14152 v2:172.21.15.77:6800/3838941679 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe0000611d0 con 0x7fdfe80a05a0 2023-12-01T01:14:57.168 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:57.167+0000 7fe006051700 1 -- 172.21.15.77:0/3718364978 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fdfe80a05a0 msgr2=0x7fdfe80a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:14:57.168 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:57.167+0000 7fe006051700 1 --2- 172.21.15.77:0/3718364978 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fdfe80a05a0 0x7fdfe80a2a60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdff00098c0 tx=0x7fdff0019040 comp rx=0 tx=0).stop 2023-12-01T01:14:57.168 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:57.167+0000 7fe006051700 1 -- 172.21.15.77:0/3718364978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe000102ac0 msgr2=0x7fe00019e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:14:57.168 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:57.167+0000 7fe006051700 1 --2- 172.21.15.77:0/3718364978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe000102ac0 0x7fe00019e0d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdff800c140 tx=0x7fdff800c450 comp rx=0 tx=0).stop 2023-12-01T01:14:57.168 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:57.167+0000 7fe006051700 1 -- 172.21.15.77:0/3718364978 shutdown_connections 2023-12-01T01:14:57.169 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:57.167+0000 7fe006051700 1 --2- 172.21.15.77:0/3718364978 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fdfe80a05a0 0x7fdfe80a2a60 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:14:57.169 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:57.167+0000 7fe006051700 1 --2- 172.21.15.77:0/3718364978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe000102ac0 0x7fe00019e0d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:14:57.169 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:57.167+0000 7fe006051700 1 --2- 172.21.15.77:0/3718364978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0000ff140 0x7fe000196b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:14:57.169 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:57.167+0000 7fe006051700 1 --2- 172.21.15.77:0/3718364978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe000102120 0x7fe000197050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:14:57.169 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:57.167+0000 7fe006051700 1 -- 172.21.15.77:0/3718364978 >> 172.21.15.77:0/3718364978 conn(0x7fe0000faaa0 msgr2=0x7fe000105680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:14:57.169 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:57.167+0000 7fe006051700 1 -- 172.21.15.77:0/3718364978 shutdown_connections 2023-12-01T01:14:57.169 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:14:57.167+0000 7fe006051700 1 -- 172.21.15.77:0/3718364978 wait complete. 2023-12-01T01:14:57.591 DEBUG:teuthology.orchestra.run.smithi077:osd.9> sudo journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@osd.9.service 2023-12-01T01:14:57.593 INFO:tasks.cephadm:Deploying osd.10 on smithi137 with /dev/vg_nvme/lv_1... 2023-12-01T01:14:57.593 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-12-01T01:14:57.623 INFO:journalctl@ceph.osd.9.smithi077.stdout:-- Logs begin at Fri 2023-12-01 00:55:13 UTC. -- 2023-12-01T01:14:57.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:57 smithi077 ceph-mon[98867]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-12-01T01:14:57.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:57 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:57.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:57 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:57.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:57 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:57.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:57 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:58.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:57 smithi203 ceph-mon[103581]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-12-01T01:14:58.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:57 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:58.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:57 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:58.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:57 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:58.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:57 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:58.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:57 smithi137 ceph-mon[103312]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-12-01T01:14:58.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:57 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:58.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:57 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:58.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:57 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:58.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:57 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:14:58.159 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.b/config 2023-12-01T01:14:58.727 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:58 smithi137 ceph-mon[103312]: from='osd.9 [v2:172.21.15.77:6826/2409880685,v1:172.21.15.77:6827/2409880685]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-12-01T01:14:58.791 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:14:58 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:14:58.496+0000 7fe6b7ef1640 -1 osd.9 0 log_to_monitors true 2023-12-01T01:14:58.792 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:58 smithi077 ceph-mon[98867]: from='osd.9 [v2:172.21.15.77:6826/2409880685,v1:172.21.15.77:6827/2409880685]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-12-01T01:14:59.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:58 smithi203 ceph-mon[103581]: from='osd.9 [v2:172.21.15.77:6826/2409880685,v1:172.21.15.77:6827/2409880685]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-12-01T01:14:59.787 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-12-01T01:14:59.788 INFO:teuthology.orchestra.run.smithi137.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-12-01T01:14:59.788 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10+0 records in 2023-12-01T01:14:59.788 INFO:teuthology.orchestra.run.smithi137.stderr:10+0 records out 2023-12-01T01:14:59.788 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0121464 s, 863 MB/s 2023-12-01T01:14:59.788 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping successful for: 2023-12-01T01:14:59.904 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:14:59 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:14:59.577+0000 7fe6acd67700 -1 osd.9 0 waiting for initial osdmap 2023-12-01T01:14:59.904 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:14:59 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:14:59.591+0000 7fe6a835b700 -1 osd.9 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-01T01:14:59.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:59 smithi077 ceph-mon[98867]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-12-01T01:14:59.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:59 smithi077 ceph-mon[98867]: from='osd.9 [v2:172.21.15.77:6826/2409880685,v1:172.21.15.77:6827/2409880685]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-12-01T01:14:59.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:59 smithi077 ceph-mon[98867]: osdmap e62: 10 total, 9 up, 10 in 2023-12-01T01:14:59.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:59 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-01T01:14:59.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:14:59 smithi077 ceph-mon[98867]: from='osd.9 [v2:172.21.15.77:6826/2409880685,v1:172.21.15.77:6827/2409880685]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2023-12-01T01:14:59.941 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:59 smithi137 ceph-mon[103312]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-12-01T01:14:59.941 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:59 smithi137 ceph-mon[103312]: from='osd.9 [v2:172.21.15.77:6826/2409880685,v1:172.21.15.77:6827/2409880685]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-12-01T01:14:59.941 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:59 smithi137 ceph-mon[103312]: osdmap e62: 10 total, 9 up, 10 in 2023-12-01T01:14:59.941 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:59 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-01T01:14:59.941 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:14:59 smithi137 ceph-mon[103312]: from='osd.9 [v2:172.21.15.77:6826/2409880685,v1:172.21.15.77:6827/2409880685]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2023-12-01T01:15:00.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:59 smithi203 ceph-mon[103581]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-12-01T01:15:00.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:59 smithi203 ceph-mon[103581]: from='osd.9 [v2:172.21.15.77:6826/2409880685,v1:172.21.15.77:6827/2409880685]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-12-01T01:15:00.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:59 smithi203 ceph-mon[103581]: osdmap e62: 10 total, 9 up, 10 in 2023-12-01T01:15:00.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:59 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-01T01:15:00.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:14:59 smithi203 ceph-mon[103581]: from='osd.9 [v2:172.21.15.77:6826/2409880685,v1:172.21.15.77:6827/2409880685]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2023-12-01T01:15:00.205 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi137:vg_nvme/lv_1 2023-12-01T01:15:00.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:00 smithi077 ceph-mon[98867]: from='osd.9 [v2:172.21.15.77:6826/2409880685,v1:172.21.15.77:6827/2409880685]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2023-12-01T01:15:00.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:00 smithi077 ceph-mon[98867]: osdmap e63: 10 total, 9 up, 10 in 2023-12-01T01:15:00.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:00 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-01T01:15:00.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:00 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-01T01:15:00.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:00 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:00.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:00 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:00.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:00 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:15:00.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:00 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:15:00.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:00 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:15:00.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:00 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:15:00.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:00 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:00.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:00 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:15:00.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:00 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:15:00.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:00 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:00.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:00 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-01T01:15:00.754 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.b/config 2023-12-01T01:15:00.947 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:00 smithi137 ceph-mon[103312]: from='osd.9 [v2:172.21.15.77:6826/2409880685,v1:172.21.15.77:6827/2409880685]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2023-12-01T01:15:00.947 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:00 smithi137 ceph-mon[103312]: osdmap e63: 10 total, 9 up, 10 in 2023-12-01T01:15:00.947 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:00 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-01T01:15:00.947 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:00 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-01T01:15:00.947 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:00 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:00.947 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:00 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:00.947 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:00 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:15:00.947 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:00 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:15:00.947 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:00 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:15:00.947 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:00 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:15:00.947 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:00 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:00.947 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:00 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:15:00.947 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:00 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:15:00.948 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:00 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:00.948 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:00 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-01T01:15:01.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:00 smithi203 ceph-mon[103581]: from='osd.9 [v2:172.21.15.77:6826/2409880685,v1:172.21.15.77:6827/2409880685]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2023-12-01T01:15:01.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:00 smithi203 ceph-mon[103581]: osdmap e63: 10 total, 9 up, 10 in 2023-12-01T01:15:01.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:00 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-01T01:15:01.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:00 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-01T01:15:01.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:00 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:01.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:00 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:01.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:00 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:15:01.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:00 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:15:01.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:00 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:15:01.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:00 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:15:01.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:00 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:01.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:00 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:15:01.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:00 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:15:01.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:00 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:01.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:00 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-01T01:15:01.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:01 smithi077 ceph-mon[98867]: purged_snaps scrub starts 2023-12-01T01:15:01.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:01 smithi077 ceph-mon[98867]: purged_snaps scrub ok 2023-12-01T01:15:01.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:01 smithi077 ceph-mon[98867]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-12-01T01:15:01.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:01 smithi077 ceph-mon[98867]: Detected new or changed devices on smithi077 2023-12-01T01:15:01.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:01 smithi077 ceph-mon[98867]: Adjusting osd_memory_target on smithi077 to 4292M 2023-12-01T01:15:01.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:01 smithi077 ceph-mon[98867]: osd.9 [v2:172.21.15.77:6826/2409880685,v1:172.21.15.77:6827/2409880685] boot 2023-12-01T01:15:01.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:01 smithi077 ceph-mon[98867]: osdmap e64: 10 total, 10 up, 10 in 2023-12-01T01:15:01.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:01 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-01T01:15:01.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:01 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:15:01.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:01 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:15:01.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:01 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:15:01.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:01 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:01.961 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:01.956+0000 7f6185549700 1 -- 172.21.15.137:0/910757911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61801009b0 msgr2=0x7f6180108270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:01.962 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:01.956+0000 7f6185549700 1 --2- 172.21.15.137:0/910757911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61801009b0 0x7f6180108270 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f617800b0a0 tx=0x7f617800b3b0 comp rx=0 tx=0).stop 2023-12-01T01:15:01.962 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:01.957+0000 7f6185549700 1 -- 172.21.15.137:0/910757911 shutdown_connections 2023-12-01T01:15:01.962 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:01.957+0000 7f6185549700 1 --2- 172.21.15.137:0/910757911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61801009b0 0x7f6180108270 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:01.962 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:01.957+0000 7f6185549700 1 --2- 172.21.15.137:0/910757911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6180101360 0x7f6180101780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:01.962 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:01.957+0000 7f6185549700 1 --2- 172.21.15.137:0/910757911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6180100010 0x7f6180100470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:01.963 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:01.957+0000 7f6185549700 1 -- 172.21.15.137:0/910757911 >> 172.21.15.137:0/910757911 conn(0x7f61800faaa0 msgr2=0x7f61800fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:15:01.963 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:01.957+0000 7f6185549700 1 -- 172.21.15.137:0/910757911 shutdown_connections 2023-12-01T01:15:01.963 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:01.957+0000 7f6185549700 1 -- 172.21.15.137:0/910757911 wait complete. 2023-12-01T01:15:01.963 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:01.957+0000 7f6185549700 1 Processor -- start 2023-12-01T01:15:01.963 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:01.958+0000 7f6185549700 1 -- start start 2023-12-01T01:15:01.963 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:01.958+0000 7f6185549700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6180100010 0x7f6180195a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:01.964 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:01.958+0000 7f6185549700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61801009b0 0x7f6180195fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:01.964 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:01.958+0000 7f6185549700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6180101360 0x7f618019d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:01.964 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:01.958+0000 7f6185549700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618019d5f0 con 0x7f6180101360 2023-12-01T01:15:01.964 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:01.958+0000 7f6185549700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618019d730 con 0x7f6180100010 2023-12-01T01:15:01.964 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:01.958+0000 7f6185549700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618019d8c0 con 0x7f61801009b0 2023-12-01T01:15:01.964 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:01.958+0000 7f617effd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6180100010 0x7f6180195a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:01.965 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:01.958+0000 7f617effd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6180100010 0x7f6180195a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37530/0 (socket says 172.21.15.137:37530) 2023-12-01T01:15:01.965 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:01.958+0000 7f617effd700 1 -- 172.21.15.137:0/2730782625 learned_addr learned my addr 172.21.15.137:0/2730782625 (peer_addr_for_me v2:172.21.15.137:0/0) 2023-12-01T01:15:01.965 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:01.958+0000 7f617f7fe700 1 --2- 172.21.15.137:0/2730782625 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6180101360 0x7f618019d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:01.965 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:01.958+0000 7f617e7fc700 1 --2- 172.21.15.137:0/2730782625 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61801009b0 0x7f6180195fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:01.965 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:01.958+0000 7f617effd700 1 -- 172.21.15.137:0/2730782625 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61801009b0 msgr2=0x7f6180195fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:01.966 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:01.958+0000 7f617effd700 1 --2- 172.21.15.137:0/2730782625 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61801009b0 0x7f6180195fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:01.966 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:01.958+0000 7f617effd700 1 -- 172.21.15.137:0/2730782625 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6180101360 msgr2=0x7f618019d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:01.966 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:01.958+0000 7f617effd700 1 --2- 172.21.15.137:0/2730782625 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6180101360 0x7f618019d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:01.966 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:01.958+0000 7f617effd700 1 -- 172.21.15.137:0/2730782625 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6178009d00 con 0x7f6180100010 2023-12-01T01:15:01.966 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:01.958+0000 7f617effd700 1 --2- 172.21.15.137:0/2730782625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6180100010 0x7f6180195a60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f617000ea80 tx=0x7f617000ed90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:15:01.966 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:01.959+0000 7f6167fff700 1 -- 172.21.15.137:0/2730782625 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6170013070 con 0x7f6180100010 2023-12-01T01:15:01.966 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:01.959+0000 7f6185549700 1 -- 172.21.15.137:0/2730782625 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61800fef60 con 0x7f6180100010 2023-12-01T01:15:01.967 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:01.959+0000 7f6167fff700 1 -- 172.21.15.137:0/2730782625 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f617000be80 con 0x7f6180100010 2023-12-01T01:15:01.967 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:01.959+0000 7f6167fff700 1 -- 172.21.15.137:0/2730782625 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6170003ae0 con 0x7f6180100010 2023-12-01T01:15:01.967 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:01.959+0000 7f6185549700 1 -- 172.21.15.137:0/2730782625 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61800ff4b0 con 0x7f6180100010 2023-12-01T01:15:01.967 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:01.960+0000 7f6185549700 1 -- 172.21.15.137:0/2730782625 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f618010b580 con 0x7f6180100010 2023-12-01T01:15:01.968 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:01.961+0000 7f6167fff700 1 -- 172.21.15.137:0/2730782625 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f617000b9c0 con 0x7f6180100010 2023-12-01T01:15:01.969 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:01.961+0000 7f6167fff700 1 --2- 172.21.15.137:0/2730782625 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f61680a0520 0x7f61680a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:01.969 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:01.961+0000 7f6167fff700 1 -- 172.21.15.137:0/2730782625 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f61700cc9b0 con 0x7f6180100010 2023-12-01T01:15:01.969 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:01.962+0000 7f617e7fc700 1 --2- 172.21.15.137:0/2730782625 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f61680a0520 0x7f61680a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:01.969 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:01.962+0000 7f617e7fc700 1 --2- 172.21.15.137:0/2730782625 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f61680a0520 0x7f61680a29e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6174005ea0 tx=0x7f6174005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:15:01.969 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:01.965+0000 7f6167fff700 1 -- 172.21.15.137:0/2730782625 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6170096d50 con 0x7f6180100010 2023-12-01T01:15:02.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:01 smithi203 ceph-mon[103581]: purged_snaps scrub starts 2023-12-01T01:15:02.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:01 smithi203 ceph-mon[103581]: purged_snaps scrub ok 2023-12-01T01:15:02.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:01 smithi203 ceph-mon[103581]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-12-01T01:15:02.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:01 smithi203 ceph-mon[103581]: Detected new or changed devices on smithi077 2023-12-01T01:15:02.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:01 smithi203 ceph-mon[103581]: Adjusting osd_memory_target on smithi077 to 4292M 2023-12-01T01:15:02.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:01 smithi203 ceph-mon[103581]: osd.9 [v2:172.21.15.77:6826/2409880685,v1:172.21.15.77:6827/2409880685] boot 2023-12-01T01:15:02.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:01 smithi203 ceph-mon[103581]: osdmap e64: 10 total, 10 up, 10 in 2023-12-01T01:15:02.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:01 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-01T01:15:02.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:01 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:15:02.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:01 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:15:02.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:01 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:15:02.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:01 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:02.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:01 smithi137 ceph-mon[103312]: purged_snaps scrub starts 2023-12-01T01:15:02.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:01 smithi137 ceph-mon[103312]: purged_snaps scrub ok 2023-12-01T01:15:02.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:01 smithi137 ceph-mon[103312]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-12-01T01:15:02.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:01 smithi137 ceph-mon[103312]: Detected new or changed devices on smithi077 2023-12-01T01:15:02.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:01 smithi137 ceph-mon[103312]: Adjusting osd_memory_target on smithi077 to 4292M 2023-12-01T01:15:02.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:01 smithi137 ceph-mon[103312]: osd.9 [v2:172.21.15.77:6826/2409880685,v1:172.21.15.77:6827/2409880685] boot 2023-12-01T01:15:02.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:01 smithi137 ceph-mon[103312]: osdmap e64: 10 total, 10 up, 10 in 2023-12-01T01:15:02.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:01 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-01T01:15:02.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:01 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:15:02.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:01 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:15:02.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:01 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:15:02.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:01 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:02.164 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:02.162+0000 7f6185549700 1 -- 172.21.15.137:0/2730782625 --> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f61800611d0 con 0x7f61680a0520 2023-12-01T01:15:03.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:02 smithi203 ceph-mon[103581]: osdmap e65: 10 total, 10 up, 10 in 2023-12-01T01:15:03.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:02 smithi203 ceph-mon[103581]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:15:03.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:02 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-01T01:15:03.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:02 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-01T01:15:03.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:02 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:15:03.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:02 smithi203 ceph-mon[103581]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-01T01:15:03.016 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:02 smithi137 ceph-mon[103312]: osdmap e65: 10 total, 10 up, 10 in 2023-12-01T01:15:03.016 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:02 smithi137 ceph-mon[103312]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:15:03.016 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:02 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-01T01:15:03.016 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:02 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-01T01:15:03.016 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:02 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:15:03.016 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:02 smithi137 ceph-mon[103312]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-01T01:15:03.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:02 smithi077 ceph-mon[98867]: osdmap e65: 10 total, 10 up, 10 in 2023-12-01T01:15:03.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:02 smithi077 ceph-mon[98867]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:15:03.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:02 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-01T01:15:03.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:02 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-01T01:15:03.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:02 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:15:03.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:02 smithi077 ceph-mon[98867]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-01T01:15:05.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:05 smithi077 ceph-mon[98867]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-01T01:15:05.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:05 smithi077 ceph-mon[98867]: from='client.? 172.21.15.137:0/2292079110' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1680bb3e-a3c8-427b-af68-e2214f32bed3"}]: dispatch 2023-12-01T01:15:05.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:05 smithi077 ceph-mon[98867]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1680bb3e-a3c8-427b-af68-e2214f32bed3"}]: dispatch 2023-12-01T01:15:05.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:05 smithi077 ceph-mon[98867]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1680bb3e-a3c8-427b-af68-e2214f32bed3"}]': finished 2023-12-01T01:15:05.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:05 smithi077 ceph-mon[98867]: osdmap e66: 11 total, 10 up, 11 in 2023-12-01T01:15:05.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:05 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-01T01:15:05.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:05 smithi077 ceph-mon[98867]: from='client.? 172.21.15.137:0/1825568782' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-01T01:15:05.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:05 smithi203 ceph-mon[103581]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-01T01:15:05.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:05 smithi203 ceph-mon[103581]: from='client.? 172.21.15.137:0/2292079110' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1680bb3e-a3c8-427b-af68-e2214f32bed3"}]: dispatch 2023-12-01T01:15:05.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:05 smithi203 ceph-mon[103581]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1680bb3e-a3c8-427b-af68-e2214f32bed3"}]: dispatch 2023-12-01T01:15:05.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:05 smithi203 ceph-mon[103581]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1680bb3e-a3c8-427b-af68-e2214f32bed3"}]': finished 2023-12-01T01:15:05.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:05 smithi203 ceph-mon[103581]: osdmap e66: 11 total, 10 up, 11 in 2023-12-01T01:15:05.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:05 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-01T01:15:05.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:05 smithi203 ceph-mon[103581]: from='client.? 172.21.15.137:0/1825568782' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-01T01:15:05.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:05 smithi137 ceph-mon[103312]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-01T01:15:05.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:05 smithi137 ceph-mon[103312]: from='client.? 172.21.15.137:0/2292079110' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1680bb3e-a3c8-427b-af68-e2214f32bed3"}]: dispatch 2023-12-01T01:15:05.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:05 smithi137 ceph-mon[103312]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1680bb3e-a3c8-427b-af68-e2214f32bed3"}]: dispatch 2023-12-01T01:15:05.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:05 smithi137 ceph-mon[103312]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1680bb3e-a3c8-427b-af68-e2214f32bed3"}]': finished 2023-12-01T01:15:05.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:05 smithi137 ceph-mon[103312]: osdmap e66: 11 total, 10 up, 11 in 2023-12-01T01:15:05.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:05 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-01T01:15:05.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:05 smithi137 ceph-mon[103312]: from='client.? 172.21.15.137:0/1825568782' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-01T01:15:07.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:07 smithi077 ceph-mon[98867]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-01T01:15:08.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:07 smithi203 ceph-mon[103581]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-01T01:15:08.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:07 smithi137 ceph-mon[103312]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-01T01:15:09.883 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:09 smithi137 ceph-mon[103312]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-01T01:15:09.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:09 smithi077 ceph-mon[98867]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-01T01:15:10.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:09 smithi203 ceph-mon[103581]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-01T01:15:11.778 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:11 smithi137 ceph-mon[103312]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-01T01:15:11.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:11 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-12-01T01:15:11.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:11 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:15:11.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:11 smithi077 ceph-mon[98867]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-01T01:15:11.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:11 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-12-01T01:15:11.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:11 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:15:12.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:11 smithi203 ceph-mon[103581]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-01T01:15:12.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:11 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-12-01T01:15:12.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:11 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:15:12.608 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:12 smithi137 ceph-mon[103312]: Deploying daemon osd.10 on smithi137 2023-12-01T01:15:12.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:12 smithi077 ceph-mon[98867]: Deploying daemon osd.10 on smithi137 2023-12-01T01:15:13.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:12 smithi203 ceph-mon[103581]: Deploying daemon osd.10 on smithi137 2023-12-01T01:15:13.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:13 smithi077 ceph-mon[98867]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-01T01:15:13.959 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:13 smithi137 ceph-mon[103312]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-01T01:15:14.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:13 smithi203 ceph-mon[103581]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-01T01:15:15.854 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:15 smithi137 ceph-mon[103312]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-01T01:15:15.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:15 smithi077 ceph-mon[98867]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-01T01:15:16.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:15 smithi203 ceph-mon[103581]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-01T01:15:17.262 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:16 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:17.262 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:16 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:17.262 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:16 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:15:17.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:16 smithi203 ceph-mon[103581]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-01T01:15:17.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:16 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:17.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:16 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:17.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:16 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:15:17.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:16 smithi137 ceph-mon[103312]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-01T01:15:17.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:16 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:17.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:16 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:17.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:16 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:15:17.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:16 smithi077 ceph-mon[98867]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-01T01:15:18.777 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:18.775+0000 7f6167fff700 1 -- 172.21.15.137:0/2730782625 <== mgr.14152 v2:172.21.15.77:6800/3838941679 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f61800611d0 con 0x7f61680a0520 2023-12-01T01:15:18.778 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 10 on host 'smithi137' 2023-12-01T01:15:18.780 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:18.778+0000 7f6185549700 1 -- 172.21.15.137:0/2730782625 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f61680a0520 msgr2=0x7f61680a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:18.780 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:18.778+0000 7f6185549700 1 --2- 172.21.15.137:0/2730782625 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f61680a0520 0x7f61680a29e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6174005ea0 tx=0x7f6174005e30 comp rx=0 tx=0).stop 2023-12-01T01:15:18.780 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:18.778+0000 7f6185549700 1 -- 172.21.15.137:0/2730782625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6180100010 msgr2=0x7f6180195a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:18.780 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:18.778+0000 7f6185549700 1 --2- 172.21.15.137:0/2730782625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6180100010 0x7f6180195a60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f617000ea80 tx=0x7f617000ed90 comp rx=0 tx=0).stop 2023-12-01T01:15:18.780 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:18.778+0000 7f6185549700 1 -- 172.21.15.137:0/2730782625 shutdown_connections 2023-12-01T01:15:18.781 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:18.778+0000 7f6185549700 1 --2- 172.21.15.137:0/2730782625 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f61680a0520 0x7f61680a29e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:18.781 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:18.778+0000 7f6185549700 1 --2- 172.21.15.137:0/2730782625 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6180101360 0x7f618019d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:18.781 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:18.778+0000 7f6185549700 1 --2- 172.21.15.137:0/2730782625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6180100010 0x7f6180195a60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:18.781 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:18.778+0000 7f6185549700 1 --2- 172.21.15.137:0/2730782625 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61801009b0 0x7f6180195fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:18.781 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:18.778+0000 7f6185549700 1 -- 172.21.15.137:0/2730782625 >> 172.21.15.137:0/2730782625 conn(0x7f61800faaa0 msgr2=0x7f61800fcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:15:18.781 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:18.778+0000 7f6185549700 1 -- 172.21.15.137:0/2730782625 shutdown_connections 2023-12-01T01:15:18.781 INFO:teuthology.orchestra.run.smithi137.stderr:2023-12-01T01:15:18.778+0000 7f6185549700 1 -- 172.21.15.137:0/2730782625 wait complete. 2023-12-01T01:15:19.251 DEBUG:teuthology.orchestra.run.smithi137:osd.10> sudo journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@osd.10.service 2023-12-01T01:15:19.253 INFO:tasks.cephadm:Deploying osd.11 on smithi203 with /dev/vg_nvme/lv_1... 2023-12-01T01:15:19.254 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-12-01T01:15:19.283 INFO:journalctl@ceph.osd.10.smithi137.stdout:-- Logs begin at Fri 2023-12-01 00:55:04 UTC. -- 2023-12-01T01:15:19.528 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:19 smithi137 ceph-mon[103312]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-01T01:15:19.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:19 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:19.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:19 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:19.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:19 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:19.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:19 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:19.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:19 smithi203 ceph-mon[103581]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-01T01:15:19.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:19 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:19.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:19 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:19.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:19 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:19.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:19 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:19.817 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.c/config 2023-12-01T01:15:19.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:19 smithi077 ceph-mon[98867]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-01T01:15:19.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:19 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:19.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:19 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:19.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:19 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:19.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:19 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:20.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:15:19 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:15:19.926+0000 7f09d8df7640 -1 osd.10 0 log_to_monitors true 2023-12-01T01:15:20.604 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:20 smithi137 ceph-mon[103312]: from='osd.10 [v2:172.21.15.137:6824/374920146,v1:172.21.15.137:6825/374920146]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-01T01:15:20.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:20 smithi137 ceph-mon[103312]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-01T01:15:20.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:20 smithi203 ceph-mon[103581]: from='osd.10 [v2:172.21.15.137:6824/374920146,v1:172.21.15.137:6825/374920146]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-01T01:15:20.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:20 smithi203 ceph-mon[103581]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-01T01:15:20.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:20 smithi077 ceph-mon[98867]: from='osd.10 [v2:172.21.15.137:6824/374920146,v1:172.21.15.137:6825/374920146]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-01T01:15:20.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:20 smithi077 ceph-mon[98867]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-01T01:15:21.330 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-12-01T01:15:21.330 INFO:teuthology.orchestra.run.smithi203.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-12-01T01:15:21.330 INFO:teuthology.orchestra.run.smithi203.stderr: stderr: 10+0 records in 2023-12-01T01:15:21.330 INFO:teuthology.orchestra.run.smithi203.stderr:10+0 records out 2023-12-01T01:15:21.330 INFO:teuthology.orchestra.run.smithi203.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0146271 s, 717 MB/s 2023-12-01T01:15:21.330 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping successful for: 2023-12-01T01:15:21.738 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:21 smithi203 ceph-mon[103581]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-01T01:15:21.738 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:21 smithi203 ceph-mon[103581]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-12-01T01:15:21.738 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:21 smithi203 ceph-mon[103581]: osdmap e67: 11 total, 10 up, 11 in 2023-12-01T01:15:21.738 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:21 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-01T01:15:21.738 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:21 smithi203 ceph-mon[103581]: from='osd.10 [v2:172.21.15.137:6824/374920146,v1:172.21.15.137:6825/374920146]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2023-12-01T01:15:21.738 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:21 smithi203 ceph-mon[103581]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2023-12-01T01:15:21.760 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi203:vg_nvme/lv_1 2023-12-01T01:15:21.779 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:15:21 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:15:21.470+0000 7f09cdc6d700 -1 osd.10 0 waiting for initial osdmap 2023-12-01T01:15:21.779 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:15:21 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:15:21.488+0000 7f09ca263700 -1 osd.10 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-01T01:15:21.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:21 smithi137 ceph-mon[103312]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-01T01:15:21.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:21 smithi137 ceph-mon[103312]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-12-01T01:15:21.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:21 smithi137 ceph-mon[103312]: osdmap e67: 11 total, 10 up, 11 in 2023-12-01T01:15:21.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:21 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-01T01:15:21.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:21 smithi137 ceph-mon[103312]: from='osd.10 [v2:172.21.15.137:6824/374920146,v1:172.21.15.137:6825/374920146]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2023-12-01T01:15:21.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:21 smithi137 ceph-mon[103312]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2023-12-01T01:15:21.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:21 smithi077 ceph-mon[98867]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-01T01:15:21.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:21 smithi077 ceph-mon[98867]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-12-01T01:15:21.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:21 smithi077 ceph-mon[98867]: osdmap e67: 11 total, 10 up, 11 in 2023-12-01T01:15:21.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:21 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-01T01:15:21.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:21 smithi077 ceph-mon[98867]: from='osd.10 [v2:172.21.15.137:6824/374920146,v1:172.21.15.137:6825/374920146]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2023-12-01T01:15:21.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:21 smithi077 ceph-mon[98867]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2023-12-01T01:15:22.311 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.c/config 2023-12-01T01:15:22.761 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:22 smithi203 ceph-mon[103581]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2023-12-01T01:15:22.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:22 smithi203 ceph-mon[103581]: osdmap e68: 11 total, 10 up, 11 in 2023-12-01T01:15:22.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:22 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-01T01:15:22.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:22 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-01T01:15:22.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:22 smithi203 ceph-mon[103581]: osd.10 [v2:172.21.15.137:6824/374920146,v1:172.21.15.137:6825/374920146] boot 2023-12-01T01:15:22.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:22 smithi203 ceph-mon[103581]: osdmap e69: 11 total, 11 up, 11 in 2023-12-01T01:15:22.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:22 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-01T01:15:22.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:22 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:22.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:22 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:22.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:22 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:15:22.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:22 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:15:22.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:22 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:15:22.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:22 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:15:22.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:22 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:22.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:22 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:15:22.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:22 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:15:22.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:22 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:22.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:22 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:15:22.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:22 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:15:22.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:22 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:15:22.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:22 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:22.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:22 smithi137 ceph-mon[103312]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2023-12-01T01:15:22.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:22 smithi137 ceph-mon[103312]: osdmap e68: 11 total, 10 up, 11 in 2023-12-01T01:15:22.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:22 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-01T01:15:22.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:22 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-01T01:15:22.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:22 smithi137 ceph-mon[103312]: osd.10 [v2:172.21.15.137:6824/374920146,v1:172.21.15.137:6825/374920146] boot 2023-12-01T01:15:22.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:22 smithi137 ceph-mon[103312]: osdmap e69: 11 total, 11 up, 11 in 2023-12-01T01:15:22.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:22 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-01T01:15:22.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:22 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:22.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:22 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:22.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:22 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:15:22.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:22 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:15:22.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:22 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:15:22.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:22 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:15:22.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:22 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:22.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:22 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:15:22.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:22 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:15:22.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:22 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:22.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:22 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:15:22.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:22 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:15:22.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:22 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:15:22.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:22 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:22.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:22 smithi077 ceph-mon[98867]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2023-12-01T01:15:22.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:22 smithi077 ceph-mon[98867]: osdmap e68: 11 total, 10 up, 11 in 2023-12-01T01:15:22.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:22 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-01T01:15:22.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:22 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-01T01:15:22.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:22 smithi077 ceph-mon[98867]: osd.10 [v2:172.21.15.137:6824/374920146,v1:172.21.15.137:6825/374920146] boot 2023-12-01T01:15:22.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:22 smithi077 ceph-mon[98867]: osdmap e69: 11 total, 11 up, 11 in 2023-12-01T01:15:22.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:22 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-01T01:15:22.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:22 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:22.907 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:22 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:22.907 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:22 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:15:22.907 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:22 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:15:22.907 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:22 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:15:22.908 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:22 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:15:22.908 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:22 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:22.908 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:22 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:15:22.909 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:22 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:15:22.909 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:22 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:22.909 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:22 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:15:22.909 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:22 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:15:22.910 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:22 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:15:22.911 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:22 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:23.646 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:23.644+0000 7f1497c31700 1 -- 172.21.15.203:0/567381367 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1490101fb0 msgr2=0x7f1490102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:23.646 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:23.644+0000 7f1497c31700 1 --2- 172.21.15.203:0/567381367 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1490101fb0 0x7f1490102430 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f148c009990 tx=0x7f148c009ca0 comp rx=0 tx=0).stop 2023-12-01T01:15:23.646 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:23.644+0000 7f1497c31700 1 -- 172.21.15.203:0/567381367 shutdown_connections 2023-12-01T01:15:23.646 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:23.644+0000 7f1497c31700 1 --2- 172.21.15.203:0/567381367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1490102970 0x7f1490109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:23.646 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:23.644+0000 7f1497c31700 1 --2- 172.21.15.203:0/567381367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1490100e50 0x7f1490101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:23.646 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:23.644+0000 7f1497c31700 1 --2- 172.21.15.203:0/567381367 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1490101fb0 0x7f1490102430 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:23.646 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:23.645+0000 7f1497c31700 1 -- 172.21.15.203:0/567381367 >> 172.21.15.203:0/567381367 conn(0x7f14900fc410 msgr2=0x7f14900fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:15:23.646 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:23.645+0000 7f1497c31700 1 -- 172.21.15.203:0/567381367 shutdown_connections 2023-12-01T01:15:23.646 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:23.645+0000 7f1497c31700 1 -- 172.21.15.203:0/567381367 wait complete. 2023-12-01T01:15:23.648 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:23.645+0000 7f1497c31700 1 Processor -- start 2023-12-01T01:15:23.648 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:23.645+0000 7f1497c31700 1 -- start start 2023-12-01T01:15:23.648 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:23.645+0000 7f1497c31700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1490100e50 0x7f1490196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:23.648 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:23.645+0000 7f1497c31700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1490101fb0 0x7f1490196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:23.648 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:23.645+0000 7f1497c31700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1490102970 0x7f149019df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:23.649 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:23.645+0000 7f1497c31700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149019e520 con 0x7f1490102970 2023-12-01T01:15:23.649 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:23.645+0000 7f1497c31700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149019e660 con 0x7f1490101fb0 2023-12-01T01:15:23.649 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:23.645+0000 7f1497c31700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149019e7f0 con 0x7f1490100e50 2023-12-01T01:15:23.649 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:23.645+0000 7f14959cd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1490100e50 0x7f1490196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:23.649 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:23.646+0000 7f14959cd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1490100e50 0x7f1490196990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:59534/0 (socket says 172.21.15.203:59534) 2023-12-01T01:15:23.649 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:23.646+0000 7f14959cd700 1 -- 172.21.15.203:0/2235571036 learned_addr learned my addr 172.21.15.203:0/2235571036 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-12-01T01:15:23.649 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:23.646+0000 7f14951cc700 1 --2- 172.21.15.203:0/2235571036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1490101fb0 0x7f1490196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:23.649 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:23.646+0000 7f14961ce700 1 --2- 172.21.15.203:0/2235571036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1490102970 0x7f149019df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:23.649 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:23.646+0000 7f14959cd700 1 -- 172.21.15.203:0/2235571036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1490101fb0 msgr2=0x7f1490196ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:23.649 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:23.646+0000 7f14959cd700 1 --2- 172.21.15.203:0/2235571036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1490101fb0 0x7f1490196ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:23.649 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:23.646+0000 7f14959cd700 1 -- 172.21.15.203:0/2235571036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1490102970 msgr2=0x7f149019df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:23.650 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:23.646+0000 7f14959cd700 1 --2- 172.21.15.203:0/2235571036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1490102970 0x7f149019df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:23.650 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:23.646+0000 7f14959cd700 1 -- 172.21.15.203:0/2235571036 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f148c0095f0 con 0x7f1490100e50 2023-12-01T01:15:23.650 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:23.646+0000 7f14959cd700 1 --2- 172.21.15.203:0/2235571036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1490100e50 0x7f1490196990 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f148000c9e0 tx=0x7f148000ccf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:15:23.650 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:23.646+0000 7f1486ffd700 1 -- 172.21.15.203:0/2235571036 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1480007ac0 con 0x7f1490100e50 2023-12-01T01:15:23.650 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:23.646+0000 7f1486ffd700 1 -- 172.21.15.203:0/2235571036 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1480007c20 con 0x7f1490100e50 2023-12-01T01:15:23.650 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:23.646+0000 7f1497c31700 1 -- 172.21.15.203:0/2235571036 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f149019ead0 con 0x7f1490100e50 2023-12-01T01:15:23.650 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:23.646+0000 7f1486ffd700 1 -- 172.21.15.203:0/2235571036 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1480018660 con 0x7f1490100e50 2023-12-01T01:15:23.650 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:23.646+0000 7f1497c31700 1 -- 172.21.15.203:0/2235571036 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f149019efd0 con 0x7f1490100e50 2023-12-01T01:15:23.650 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:23.647+0000 7f1497c31700 1 -- 172.21.15.203:0/2235571036 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1490108350 con 0x7f1490100e50 2023-12-01T01:15:23.651 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:23.648+0000 7f1486ffd700 1 -- 172.21.15.203:0/2235571036 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f148000f450 con 0x7f1490100e50 2023-12-01T01:15:23.651 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:23.649+0000 7f1486ffd700 1 --2- 172.21.15.203:0/2235571036 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f147c0a0520 0x7f147c0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:23.651 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:23.649+0000 7f1486ffd700 1 -- 172.21.15.203:0/2235571036 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f14800cb2f0 con 0x7f1490100e50 2023-12-01T01:15:23.651 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:23.649+0000 7f14951cc700 1 --2- 172.21.15.203:0/2235571036 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f147c0a0520 0x7f147c0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:23.651 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:23.650+0000 7f14951cc700 1 --2- 172.21.15.203:0/2235571036 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f147c0a0520 0x7f147c0a29e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f148c00b5c0 tx=0x7f148c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:15:23.654 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:23.653+0000 7f1486ffd700 1 -- 172.21.15.203:0/2235571036 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f14800954c0 con 0x7f1490100e50 2023-12-01T01:15:23.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:23 smithi203 ceph-mon[103581]: purged_snaps scrub starts 2023-12-01T01:15:23.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:23 smithi203 ceph-mon[103581]: purged_snaps scrub ok 2023-12-01T01:15:23.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:23 smithi203 ceph-mon[103581]: Detected new or changed devices on smithi137 2023-12-01T01:15:23.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:23 smithi203 ceph-mon[103581]: Adjusting osd_memory_target on smithi137 to 4292M 2023-12-01T01:15:23.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:23 smithi203 ceph-mon[103581]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-01T01:15:23.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:23 smithi203 ceph-mon[103581]: osdmap e70: 11 total, 11 up, 11 in 2023-12-01T01:15:23.851 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:23.850+0000 7f1497c31700 1 -- 172.21.15.203:0/2235571036 --> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f14900611d0 con 0x7f147c0a0520 2023-12-01T01:15:23.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:23 smithi077 ceph-mon[98867]: purged_snaps scrub starts 2023-12-01T01:15:23.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:23 smithi077 ceph-mon[98867]: purged_snaps scrub ok 2023-12-01T01:15:23.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:23 smithi077 ceph-mon[98867]: Detected new or changed devices on smithi137 2023-12-01T01:15:23.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:23 smithi077 ceph-mon[98867]: Adjusting osd_memory_target on smithi137 to 4292M 2023-12-01T01:15:23.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:23 smithi077 ceph-mon[98867]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-01T01:15:23.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:23 smithi077 ceph-mon[98867]: osdmap e70: 11 total, 11 up, 11 in 2023-12-01T01:15:24.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:23 smithi137 ceph-mon[103312]: purged_snaps scrub starts 2023-12-01T01:15:24.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:23 smithi137 ceph-mon[103312]: purged_snaps scrub ok 2023-12-01T01:15:24.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:23 smithi137 ceph-mon[103312]: Detected new or changed devices on smithi137 2023-12-01T01:15:24.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:23 smithi137 ceph-mon[103312]: Adjusting osd_memory_target on smithi137 to 4292M 2023-12-01T01:15:24.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:23 smithi137 ceph-mon[103312]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-12-01T01:15:24.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:23 smithi137 ceph-mon[103312]: osdmap e70: 11 total, 11 up, 11 in 2023-12-01T01:15:24.874 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:24 smithi203 ceph-mon[103581]: osdmap e71: 11 total, 11 up, 11 in 2023-12-01T01:15:24.874 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:24 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-01T01:15:24.875 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:24 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-01T01:15:24.875 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:24 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:15:24.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:24 smithi077 ceph-mon[98867]: osdmap e71: 11 total, 11 up, 11 in 2023-12-01T01:15:24.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:24 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-01T01:15:24.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:24 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-01T01:15:24.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:24 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:15:25.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:24 smithi137 ceph-mon[103312]: osdmap e71: 11 total, 11 up, 11 in 2023-12-01T01:15:25.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:24 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-01T01:15:25.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:24 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-01T01:15:25.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:24 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:15:25.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:25 smithi077 ceph-mon[98867]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:15:25.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:25 smithi077 ceph-mon[98867]: pgmap v211: 1 pgs: 1 remapped+peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-01T01:15:25.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:25 smithi077 ceph-mon[98867]: osdmap e72: 11 total, 11 up, 11 in 2023-12-01T01:15:26.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:25 smithi203 ceph-mon[103581]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:15:26.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:25 smithi203 ceph-mon[103581]: pgmap v211: 1 pgs: 1 remapped+peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-01T01:15:26.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:25 smithi203 ceph-mon[103581]: osdmap e72: 11 total, 11 up, 11 in 2023-12-01T01:15:26.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:25 smithi137 ceph-mon[103312]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:15:26.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:25 smithi137 ceph-mon[103312]: pgmap v211: 1 pgs: 1 remapped+peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-01T01:15:26.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:25 smithi137 ceph-mon[103312]: osdmap e72: 11 total, 11 up, 11 in 2023-12-01T01:15:26.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:26 smithi077 ceph-mon[98867]: osdmap e73: 11 total, 11 up, 11 in 2023-12-01T01:15:26.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:26 smithi077 ceph-mon[98867]: from='client.? 172.21.15.203:0/3371210793' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a533466-023d-4803-ae36-9a9f1715e49f"}]: dispatch 2023-12-01T01:15:26.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:26 smithi077 ceph-mon[98867]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a533466-023d-4803-ae36-9a9f1715e49f"}]: dispatch 2023-12-01T01:15:26.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:26 smithi077 ceph-mon[98867]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8a533466-023d-4803-ae36-9a9f1715e49f"}]': finished 2023-12-01T01:15:26.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:26 smithi077 ceph-mon[98867]: osdmap e74: 12 total, 11 up, 12 in 2023-12-01T01:15:26.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:26 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-01T01:15:27.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:26 smithi203 ceph-mon[103581]: osdmap e73: 11 total, 11 up, 11 in 2023-12-01T01:15:27.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:26 smithi203 ceph-mon[103581]: from='client.? 172.21.15.203:0/3371210793' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a533466-023d-4803-ae36-9a9f1715e49f"}]: dispatch 2023-12-01T01:15:27.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:26 smithi203 ceph-mon[103581]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a533466-023d-4803-ae36-9a9f1715e49f"}]: dispatch 2023-12-01T01:15:27.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:26 smithi203 ceph-mon[103581]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8a533466-023d-4803-ae36-9a9f1715e49f"}]': finished 2023-12-01T01:15:27.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:26 smithi203 ceph-mon[103581]: osdmap e74: 12 total, 11 up, 12 in 2023-12-01T01:15:27.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:26 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-01T01:15:27.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:26 smithi137 ceph-mon[103312]: osdmap e73: 11 total, 11 up, 11 in 2023-12-01T01:15:27.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:26 smithi137 ceph-mon[103312]: from='client.? 172.21.15.203:0/3371210793' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a533466-023d-4803-ae36-9a9f1715e49f"}]: dispatch 2023-12-01T01:15:27.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:26 smithi137 ceph-mon[103312]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a533466-023d-4803-ae36-9a9f1715e49f"}]: dispatch 2023-12-01T01:15:27.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:26 smithi137 ceph-mon[103312]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8a533466-023d-4803-ae36-9a9f1715e49f"}]': finished 2023-12-01T01:15:27.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:26 smithi137 ceph-mon[103312]: osdmap e74: 12 total, 11 up, 12 in 2023-12-01T01:15:27.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:26 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-01T01:15:27.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:27 smithi077 ceph-mon[98867]: pgmap v214: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-12-01T01:15:27.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:27 smithi077 ceph-mon[98867]: from='client.? 172.21.15.203:0/4066518561' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-01T01:15:28.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:27 smithi203 ceph-mon[103581]: pgmap v214: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-12-01T01:15:28.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:27 smithi203 ceph-mon[103581]: from='client.? 172.21.15.203:0/4066518561' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-01T01:15:28.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:27 smithi137 ceph-mon[103312]: pgmap v214: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-12-01T01:15:28.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:27 smithi137 ceph-mon[103312]: from='client.? 172.21.15.203:0/4066518561' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-01T01:15:29.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:29 smithi077 ceph-mon[98867]: pgmap v216: 1 pgs: 1 remapped+peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-01T01:15:30.015 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:29 smithi203 ceph-mon[103581]: pgmap v216: 1 pgs: 1 remapped+peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-01T01:15:30.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:29 smithi137 ceph-mon[103312]: pgmap v216: 1 pgs: 1 remapped+peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-01T01:15:31.878 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:31 smithi203 ceph-mon[103581]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-01T01:15:31.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:31 smithi077 ceph-mon[98867]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-01T01:15:32.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:31 smithi137 ceph-mon[103312]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-01T01:15:33.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:33 smithi203 ceph-mon[103581]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2023-12-01T01:15:33.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:33 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-12-01T01:15:33.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:33 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:15:33.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:33 smithi077 ceph-mon[98867]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2023-12-01T01:15:33.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:33 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-12-01T01:15:33.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:33 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:15:34.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:33 smithi137 ceph-mon[103312]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2023-12-01T01:15:34.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:33 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-12-01T01:15:34.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:33 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:15:34.900 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:34 smithi203 ceph-mon[103581]: Deploying daemon osd.11 on smithi203 2023-12-01T01:15:34.903 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:34 smithi077 ceph-mon[98867]: Deploying daemon osd.11 on smithi203 2023-12-01T01:15:35.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:34 smithi137 ceph-mon[103312]: Deploying daemon osd.11 on smithi203 2023-12-01T01:15:35.889 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:35 smithi203 ceph-mon[103581]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2023-12-01T01:15:35.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:35 smithi077 ceph-mon[98867]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2023-12-01T01:15:36.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:35 smithi137 ceph-mon[103312]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2023-12-01T01:15:37.832 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:37 smithi203 ceph-mon[103581]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-12-01T01:15:37.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:37 smithi077 ceph-mon[98867]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-12-01T01:15:38.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:37 smithi137 ceph-mon[103312]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-12-01T01:15:39.262 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:39 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:39.262 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:39 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:15:39.262 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:39 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:39.262 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:39 smithi203 ceph-mon[103581]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-12-01T01:15:39.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:39 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:39.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:39 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:15:39.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:39 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:39.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:39 smithi077 ceph-mon[98867]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-12-01T01:15:39.528 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:39 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:39.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:39 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:15:39.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:39 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:39.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:39 smithi137 ceph-mon[103312]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-12-01T01:15:40.961 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:40.959+0000 7f1486ffd700 1 -- 172.21.15.203:0/2235571036 <== mgr.14152 v2:172.21.15.77:6800/3838941679 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f14900611d0 con 0x7f147c0a0520 2023-12-01T01:15:40.961 INFO:teuthology.orchestra.run.smithi203.stdout:Created osd(s) 11 on host 'smithi203' 2023-12-01T01:15:40.963 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:40.961+0000 7f1497c31700 1 -- 172.21.15.203:0/2235571036 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f147c0a0520 msgr2=0x7f147c0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:40.963 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:40.961+0000 7f1497c31700 1 --2- 172.21.15.203:0/2235571036 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f147c0a0520 0x7f147c0a29e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f148c00b5c0 tx=0x7f148c005960 comp rx=0 tx=0).stop 2023-12-01T01:15:40.963 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:40.961+0000 7f1497c31700 1 -- 172.21.15.203:0/2235571036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1490100e50 msgr2=0x7f1490196990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:40.963 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:40.961+0000 7f1497c31700 1 --2- 172.21.15.203:0/2235571036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1490100e50 0x7f1490196990 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f148000c9e0 tx=0x7f148000ccf0 comp rx=0 tx=0).stop 2023-12-01T01:15:40.963 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:40.961+0000 7f1497c31700 1 -- 172.21.15.203:0/2235571036 shutdown_connections 2023-12-01T01:15:40.964 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:40.961+0000 7f1497c31700 1 --2- 172.21.15.203:0/2235571036 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f147c0a0520 0x7f147c0a29e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:40.964 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:40.961+0000 7f1497c31700 1 --2- 172.21.15.203:0/2235571036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1490102970 0x7f149019df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:40.964 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:40.961+0000 7f1497c31700 1 --2- 172.21.15.203:0/2235571036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1490100e50 0x7f1490196990 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:40.964 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:40.961+0000 7f1497c31700 1 --2- 172.21.15.203:0/2235571036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1490101fb0 0x7f1490196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:40.964 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:40.961+0000 7f1497c31700 1 -- 172.21.15.203:0/2235571036 >> 172.21.15.203:0/2235571036 conn(0x7f14900fc410 msgr2=0x7f1490105490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:15:40.964 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:40.961+0000 7f1497c31700 1 -- 172.21.15.203:0/2235571036 shutdown_connections 2023-12-01T01:15:40.964 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:40.961+0000 7f1497c31700 1 -- 172.21.15.203:0/2235571036 wait complete. 2023-12-01T01:15:41.420 DEBUG:teuthology.orchestra.run.smithi203:osd.11> sudo journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@osd.11.service 2023-12-01T01:15:41.422 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-12-01T01:15:41.422 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph osd stat -f json 2023-12-01T01:15:41.455 INFO:journalctl@ceph.osd.11.smithi203.stdout:-- Logs begin at Fri 2023-12-01 00:55:29 UTC. -- 2023-12-01T01:15:41.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:41 smithi077 ceph-mon[98867]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-12-01T01:15:41.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:41 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:41.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:41 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:41.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:41 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:15:41.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:41 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:15:41.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:41 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:41.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:41 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:41.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:41 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:41.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:41 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:15:41.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:41 smithi203 ceph-mon[103581]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-12-01T01:15:41.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:41 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:41.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:41 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:41.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:41 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:15:41.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:41 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:15:41.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:41 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:41.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:41 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:41.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:41 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:41.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:41 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:15:41.778 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:41 smithi137 ceph-mon[103312]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-12-01T01:15:41.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:41 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:41.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:41 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:41.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:41 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:15:41.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:41 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:15:41.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:41 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:41.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:41 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:41.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:41 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:41.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:41 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:15:41.992 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:15:42.262 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:15:42 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:15:42.108+0000 7f9f36c10640 -1 osd.11 0 log_to_monitors true 2023-12-01T01:15:42.837 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:42 smithi203 ceph-mon[103581]: from='osd.11 [v2:172.21.15.203:6824/2161588962,v1:172.21.15.203:6825/2161588962]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-01T01:15:42.837 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:42 smithi203 ceph-mon[103581]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-01T01:15:42.862 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:42 smithi077 ceph-mon[98867]: from='osd.11 [v2:172.21.15.203:6824/2161588962,v1:172.21.15.203:6825/2161588962]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-01T01:15:42.862 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:42 smithi077 ceph-mon[98867]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-01T01:15:43.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:42 smithi137 ceph-mon[103312]: from='osd.11 [v2:172.21.15.203:6824/2161588962,v1:172.21.15.203:6825/2161588962]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-01T01:15:43.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:42 smithi137 ceph-mon[103312]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-01T01:15:43.055 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.052+0000 7faabf4c4700 1 -- 172.21.15.77:0/1600908636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faab8101080 msgr2=0x7faab8107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:43.056 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.052+0000 7faabf4c4700 1 --2- 172.21.15.77:0/1600908636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faab8101080 0x7faab8107930 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faab000b0d0 tx=0x7faab000b3e0 comp rx=0 tx=0).stop 2023-12-01T01:15:43.056 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.053+0000 7faabf4c4700 1 -- 172.21.15.77:0/1600908636 shutdown_connections 2023-12-01T01:15:43.056 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.053+0000 7faabf4c4700 1 --2- 172.21.15.77:0/1600908636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faab8101080 0x7faab8107930 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:43.056 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.053+0000 7faabf4c4700 1 --2- 172.21.15.77:0/1600908636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faab80ff4e0 0x7faab80ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:43.056 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.053+0000 7faabf4c4700 1 --2- 172.21.15.77:0/1600908636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faab81006e0 0x7faab8100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:43.056 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.053+0000 7faabf4c4700 1 -- 172.21.15.77:0/1600908636 >> 172.21.15.77:0/1600908636 conn(0x7faab80faa80 msgr2=0x7faab80fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:15:43.056 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.053+0000 7faabf4c4700 1 -- 172.21.15.77:0/1600908636 shutdown_connections 2023-12-01T01:15:43.056 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.053+0000 7faabf4c4700 1 -- 172.21.15.77:0/1600908636 wait complete. 2023-12-01T01:15:43.056 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.053+0000 7faabf4c4700 1 Processor -- start 2023-12-01T01:15:43.056 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.053+0000 7faabf4c4700 1 -- start start 2023-12-01T01:15:43.056 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.053+0000 7faabf4c4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faab80ff4e0 0x7faab8196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:43.056 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.053+0000 7faabf4c4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faab81006e0 0x7faab8197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:43.056 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.053+0000 7faabf4c4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faab8101080 0x7faab819e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:43.056 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.053+0000 7faabf4c4700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab819e850 con 0x7faab8101080 2023-12-01T01:15:43.056 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.053+0000 7faabf4c4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab819e990 con 0x7faab81006e0 2023-12-01T01:15:43.057 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.053+0000 7faabf4c4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab819eb50 con 0x7faab80ff4e0 2023-12-01T01:15:43.057 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.053+0000 7faabda61700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faab8101080 0x7faab819e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:43.057 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.054+0000 7faabda61700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faab8101080 0x7faab819e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57072/0 (socket says 172.21.15.77:57072) 2023-12-01T01:15:43.057 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.054+0000 7faabda61700 1 -- 172.21.15.77:0/2477203882 learned_addr learned my addr 172.21.15.77:0/2477203882 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:15:43.057 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.054+0000 7faabd260700 1 --2- 172.21.15.77:0/2477203882 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faab80ff4e0 0x7faab8196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:43.057 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.054+0000 7faabca5f700 1 --2- 172.21.15.77:0/2477203882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faab81006e0 0x7faab8197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:43.057 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.054+0000 7faabda61700 1 -- 172.21.15.77:0/2477203882 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faab80ff4e0 msgr2=0x7faab8196d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:43.057 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.054+0000 7faabda61700 1 --2- 172.21.15.77:0/2477203882 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faab80ff4e0 0x7faab8196d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:43.057 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.054+0000 7faabda61700 1 -- 172.21.15.77:0/2477203882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faab81006e0 msgr2=0x7faab8197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:43.057 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.054+0000 7faabda61700 1 --2- 172.21.15.77:0/2477203882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faab81006e0 0x7faab8197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:43.057 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.054+0000 7faabda61700 1 -- 172.21.15.77:0/2477203882 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab0009d00 con 0x7faab8101080 2023-12-01T01:15:43.057 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.054+0000 7faabca5f700 1 --2- 172.21.15.77:0/2477203882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faab81006e0 0x7faab8197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:15:43.057 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.054+0000 7faabda61700 1 --2- 172.21.15.77:0/2477203882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faab8101080 0x7faab819e280 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7faab0007a00 tx=0x7faab00094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:15:43.057 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.054+0000 7faaae7fc700 1 -- 172.21.15.77:0/2477203882 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab000e040 con 0x7faab8101080 2023-12-01T01:15:43.058 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.054+0000 7faabf4c4700 1 -- 172.21.15.77:0/2477203882 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faab8101c50 con 0x7faab8101080 2023-12-01T01:15:43.058 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.054+0000 7faaae7fc700 1 -- 172.21.15.77:0/2477203882 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faab0003e10 con 0x7faab8101080 2023-12-01T01:15:43.058 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.054+0000 7faaae7fc700 1 -- 172.21.15.77:0/2477203882 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab0012a90 con 0x7faab8101080 2023-12-01T01:15:43.058 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.054+0000 7faabf4c4700 1 -- 172.21.15.77:0/2477203882 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faab8102140 con 0x7faab8101080 2023-12-01T01:15:43.059 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.055+0000 7faabf4c4700 1 -- 172.21.15.77:0/2477203882 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faab810a510 con 0x7faab8101080 2023-12-01T01:15:43.059 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.056+0000 7faaae7fc700 1 -- 172.21.15.77:0/2477203882 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7faab00056c0 con 0x7faab8101080 2023-12-01T01:15:43.059 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.057+0000 7faaae7fc700 1 --2- 172.21.15.77:0/2477203882 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7faaa40a0620 0x7faaa40a2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:43.059 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.057+0000 7faabd260700 1 --2- 172.21.15.77:0/2477203882 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7faaa40a0620 0x7faaa40a2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:43.059 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.057+0000 7faaae7fc700 1 -- 172.21.15.77:0/2477203882 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6831+0+0 (secure 0 0 0) 0x7faab00cd260 con 0x7faab8101080 2023-12-01T01:15:43.059 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.057+0000 7faabd260700 1 --2- 172.21.15.77:0/2477203882 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7faaa40a0620 0x7faaa40a2ae0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7faab81981e0 tx=0x7faab4009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:15:43.061 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.060+0000 7faaae7fc700 1 -- 172.21.15.77:0/2477203882 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7faab00973c0 con 0x7faab8101080 2023-12-01T01:15:43.250 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.249+0000 7faabf4c4700 1 -- 172.21.15.77:0/2477203882 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7faab810a6d0 con 0x7faab8101080 2023-12-01T01:15:43.251 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.250+0000 7faaae7fc700 1 -- 172.21.15.77:0/2477203882 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) v1 ==== 74+0+133 (secure 0 0 0) 0x7faab009a9e0 con 0x7faab8101080 2023-12-01T01:15:43.251 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:15:43.253 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.252+0000 7faabf4c4700 1 -- 172.21.15.77:0/2477203882 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7faaa40a0620 msgr2=0x7faaa40a2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:43.253 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.252+0000 7faabf4c4700 1 --2- 172.21.15.77:0/2477203882 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7faaa40a0620 0x7faaa40a2ae0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7faab81981e0 tx=0x7faab4009190 comp rx=0 tx=0).stop 2023-12-01T01:15:43.253 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.252+0000 7faabf4c4700 1 -- 172.21.15.77:0/2477203882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faab8101080 msgr2=0x7faab819e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:43.253 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.252+0000 7faabf4c4700 1 --2- 172.21.15.77:0/2477203882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faab8101080 0x7faab819e280 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7faab0007a00 tx=0x7faab00094c0 comp rx=0 tx=0).stop 2023-12-01T01:15:43.253 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.252+0000 7faabf4c4700 1 -- 172.21.15.77:0/2477203882 shutdown_connections 2023-12-01T01:15:43.253 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.252+0000 7faabf4c4700 1 --2- 172.21.15.77:0/2477203882 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7faaa40a0620 0x7faaa40a2ae0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:43.253 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.252+0000 7faabf4c4700 1 --2- 172.21.15.77:0/2477203882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faab8101080 0x7faab819e280 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:43.253 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.252+0000 7faabf4c4700 1 --2- 172.21.15.77:0/2477203882 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faab80ff4e0 0x7faab8196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:43.253 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.252+0000 7faabf4c4700 1 --2- 172.21.15.77:0/2477203882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faab81006e0 0x7faab8197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:43.254 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.252+0000 7faabf4c4700 1 -- 172.21.15.77:0/2477203882 >> 172.21.15.77:0/2477203882 conn(0x7faab80faa80 msgr2=0x7faab8105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:15:43.254 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.252+0000 7faabf4c4700 1 -- 172.21.15.77:0/2477203882 shutdown_connections 2023-12-01T01:15:43.254 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:43.252+0000 7faabf4c4700 1 -- 172.21.15.77:0/2477203882 wait complete. 2023-12-01T01:15:43.632 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":75,"num_osds":12,"num_up_osds":11,"osd_up_since":1701393321,"num_in_osds":12,"osd_in_since":1701393326,"num_remapped_pgs":0} 2023-12-01T01:15:43.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:43 smithi077 ceph-mon[98867]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-01T01:15:43.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:43 smithi077 ceph-mon[98867]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-12-01T01:15:43.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:43 smithi077 ceph-mon[98867]: osdmap e75: 12 total, 11 up, 12 in 2023-12-01T01:15:43.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:43 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-01T01:15:43.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:43 smithi077 ceph-mon[98867]: from='osd.11 [v2:172.21.15.203:6824/2161588962,v1:172.21.15.203:6825/2161588962]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-12-01T01:15:43.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:43 smithi077 ceph-mon[98867]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-12-01T01:15:43.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:43 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/2477203882' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-01T01:15:44.012 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:15:43 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:15:43.588+0000 7f9f2ba86700 -1 osd.11 0 waiting for initial osdmap 2023-12-01T01:15:44.012 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:15:43 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:15:43.608+0000 7f9f2807c700 -1 osd.11 76 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-01T01:15:44.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:43 smithi203 ceph-mon[103581]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-01T01:15:44.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:43 smithi203 ceph-mon[103581]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-12-01T01:15:44.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:43 smithi203 ceph-mon[103581]: osdmap e75: 12 total, 11 up, 12 in 2023-12-01T01:15:44.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:43 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-01T01:15:44.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:43 smithi203 ceph-mon[103581]: from='osd.11 [v2:172.21.15.203:6824/2161588962,v1:172.21.15.203:6825/2161588962]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-12-01T01:15:44.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:43 smithi203 ceph-mon[103581]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-12-01T01:15:44.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:43 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/2477203882' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-01T01:15:44.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:43 smithi137 ceph-mon[103312]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-01T01:15:44.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:43 smithi137 ceph-mon[103312]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-12-01T01:15:44.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:43 smithi137 ceph-mon[103312]: osdmap e75: 12 total, 11 up, 12 in 2023-12-01T01:15:44.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:43 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-01T01:15:44.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:43 smithi137 ceph-mon[103312]: from='osd.11 [v2:172.21.15.203:6824/2161588962,v1:172.21.15.203:6825/2161588962]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-12-01T01:15:44.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:43 smithi137 ceph-mon[103312]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2023-12-01T01:15:44.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:43 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/2477203882' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-01T01:15:44.634 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph osd stat -f json 2023-12-01T01:15:44.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:44 smithi077 ceph-mon[98867]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2023-12-01T01:15:44.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:44 smithi077 ceph-mon[98867]: osdmap e76: 12 total, 11 up, 12 in 2023-12-01T01:15:44.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:44 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-01T01:15:44.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:44 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-01T01:15:44.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:44 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:44.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:44 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:44.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:44 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:15:44.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:44 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:15:44.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:44 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:15:44.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:44 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:15:44.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:44 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:44.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:44 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:15:44.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:44 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:15:44.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:44 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:45.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:44 smithi203 ceph-mon[103581]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2023-12-01T01:15:45.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:44 smithi203 ceph-mon[103581]: osdmap e76: 12 total, 11 up, 12 in 2023-12-01T01:15:45.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:44 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-01T01:15:45.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:44 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-01T01:15:45.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:44 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:45.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:44 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:45.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:44 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:15:45.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:44 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:15:45.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:44 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:15:45.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:44 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:15:45.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:44 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:45.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:44 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:15:45.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:44 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:15:45.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:44 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:45.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:44 smithi137 ceph-mon[103312]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2023-12-01T01:15:45.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:44 smithi137 ceph-mon[103312]: osdmap e76: 12 total, 11 up, 12 in 2023-12-01T01:15:45.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:44 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-01T01:15:45.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:44 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-01T01:15:45.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:44 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:45.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:44 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:45.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:44 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:15:45.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:44 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:15:45.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:44 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:15:45.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:44 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-01T01:15:45.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:44 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:45.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:44 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:15:45.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:44 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:15:45.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:44 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:45.192 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:15:45.644 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:45 smithi077 ceph-mon[98867]: purged_snaps scrub starts 2023-12-01T01:15:45.644 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:45 smithi077 ceph-mon[98867]: purged_snaps scrub ok 2023-12-01T01:15:45.644 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:45 smithi077 ceph-mon[98867]: Detected new or changed devices on smithi203 2023-12-01T01:15:45.644 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:45 smithi077 ceph-mon[98867]: Adjusting osd_memory_target on smithi203 to 4292M 2023-12-01T01:15:45.644 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:45 smithi077 ceph-mon[98867]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-01T01:15:45.644 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:45 smithi077 ceph-mon[98867]: osd.11 [v2:172.21.15.203:6824/2161588962,v1:172.21.15.203:6825/2161588962] boot 2023-12-01T01:15:45.644 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:45 smithi077 ceph-mon[98867]: osdmap e77: 12 total, 12 up, 12 in 2023-12-01T01:15:45.644 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:45 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-01T01:15:46.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:45 smithi203 ceph-mon[103581]: purged_snaps scrub starts 2023-12-01T01:15:46.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:45 smithi203 ceph-mon[103581]: purged_snaps scrub ok 2023-12-01T01:15:46.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:45 smithi203 ceph-mon[103581]: Detected new or changed devices on smithi203 2023-12-01T01:15:46.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:45 smithi203 ceph-mon[103581]: Adjusting osd_memory_target on smithi203 to 4292M 2023-12-01T01:15:46.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:45 smithi203 ceph-mon[103581]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-01T01:15:46.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:45 smithi203 ceph-mon[103581]: osd.11 [v2:172.21.15.203:6824/2161588962,v1:172.21.15.203:6825/2161588962] boot 2023-12-01T01:15:46.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:45 smithi203 ceph-mon[103581]: osdmap e77: 12 total, 12 up, 12 in 2023-12-01T01:15:46.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:45 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-01T01:15:46.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:45 smithi137 ceph-mon[103312]: purged_snaps scrub starts 2023-12-01T01:15:46.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:45 smithi137 ceph-mon[103312]: purged_snaps scrub ok 2023-12-01T01:15:46.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:45 smithi137 ceph-mon[103312]: Detected new or changed devices on smithi203 2023-12-01T01:15:46.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:45 smithi137 ceph-mon[103312]: Adjusting osd_memory_target on smithi203 to 4292M 2023-12-01T01:15:46.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:45 smithi137 ceph-mon[103312]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-12-01T01:15:46.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:45 smithi137 ceph-mon[103312]: osd.11 [v2:172.21.15.203:6824/2161588962,v1:172.21.15.203:6825/2161588962] boot 2023-12-01T01:15:46.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:45 smithi137 ceph-mon[103312]: osdmap e77: 12 total, 12 up, 12 in 2023-12-01T01:15:46.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:45 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-01T01:15:46.238 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.236+0000 7f23a92e1700 1 -- 172.21.15.77:0/1007687839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23a41018f0 msgr2=0x7f23a41081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:46.238 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.236+0000 7f23a92e1700 1 --2- 172.21.15.77:0/1007687839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23a41018f0 0x7f23a41081a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f239c00b0d0 tx=0x7f239c00b3e0 comp rx=0 tx=0).stop 2023-12-01T01:15:46.238 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.236+0000 7f23a92e1700 1 -- 172.21.15.77:0/1007687839 shutdown_connections 2023-12-01T01:15:46.238 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.236+0000 7f23a92e1700 1 --2- 172.21.15.77:0/1007687839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23a41018f0 0x7f23a41081a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:46.238 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.236+0000 7f23a92e1700 1 --2- 172.21.15.77:0/1007687839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23a40ffd50 0x7f23a4100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:46.238 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.236+0000 7f23a92e1700 1 --2- 172.21.15.77:0/1007687839 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23a4100f50 0x7f23a41013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:46.238 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.236+0000 7f23a92e1700 1 -- 172.21.15.77:0/1007687839 >> 172.21.15.77:0/1007687839 conn(0x7f23a40fb310 msgr2=0x7f23a40fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:15:46.238 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.236+0000 7f23a92e1700 1 -- 172.21.15.77:0/1007687839 shutdown_connections 2023-12-01T01:15:46.238 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.236+0000 7f23a92e1700 1 -- 172.21.15.77:0/1007687839 wait complete. 2023-12-01T01:15:46.238 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.237+0000 7f23a92e1700 1 Processor -- start 2023-12-01T01:15:46.240 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.237+0000 7f23a92e1700 1 -- start start 2023-12-01T01:15:46.240 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.237+0000 7f23a92e1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23a40ffd50 0x7f23a4196c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:46.241 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.237+0000 7f23a92e1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23a4100f50 0x7f23a41971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:46.241 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.237+0000 7f23a92e1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23a41018f0 0x7f23a419e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:46.241 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.237+0000 7f23a92e1700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a419e7c0 con 0x7f23a41018f0 2023-12-01T01:15:46.241 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.237+0000 7f23a92e1700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a419e900 con 0x7f23a40ffd50 2023-12-01T01:15:46.241 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.237+0000 7f23a92e1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a419eac0 con 0x7f23a4100f50 2023-12-01T01:15:46.241 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.237+0000 7f23a37fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23a41018f0 0x7f23a419e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:46.241 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.237+0000 7f23a37fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23a41018f0 0x7f23a419e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57088/0 (socket says 172.21.15.77:57088) 2023-12-01T01:15:46.241 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.237+0000 7f23a37fe700 1 -- 172.21.15.77:0/2083863415 learned_addr learned my addr 172.21.15.77:0/2083863415 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:15:46.241 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.237+0000 7f23a2ffd700 1 --2- 172.21.15.77:0/2083863415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23a40ffd50 0x7f23a4196c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:46.241 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.237+0000 7f23a27fc700 1 --2- 172.21.15.77:0/2083863415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23a4100f50 0x7f23a41971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:46.241 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.238+0000 7f23a37fe700 1 -- 172.21.15.77:0/2083863415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23a4100f50 msgr2=0x7f23a41971b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:46.241 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.238+0000 7f23a37fe700 1 --2- 172.21.15.77:0/2083863415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23a4100f50 0x7f23a41971b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:46.241 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.238+0000 7f23a37fe700 1 -- 172.21.15.77:0/2083863415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23a40ffd50 msgr2=0x7f23a4196c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:46.242 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.238+0000 7f23a37fe700 1 --2- 172.21.15.77:0/2083863415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23a40ffd50 0x7f23a4196c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:46.242 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.238+0000 7f23a37fe700 1 -- 172.21.15.77:0/2083863415 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f239c009d00 con 0x7f23a41018f0 2023-12-01T01:15:46.242 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.238+0000 7f23a2ffd700 1 --2- 172.21.15.77:0/2083863415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23a40ffd50 0x7f23a4196c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:15:46.242 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.238+0000 7f23a37fe700 1 --2- 172.21.15.77:0/2083863415 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23a41018f0 0x7f23a419e1f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f239c007a00 tx=0x7f239c009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:15:46.242 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.238+0000 7f238bfff700 1 -- 172.21.15.77:0/2083863415 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239c00e040 con 0x7f23a41018f0 2023-12-01T01:15:46.242 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.238+0000 7f238bfff700 1 -- 172.21.15.77:0/2083863415 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f239c003e10 con 0x7f23a41018f0 2023-12-01T01:15:46.242 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.238+0000 7f23a92e1700 1 -- 172.21.15.77:0/2083863415 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23a419ed40 con 0x7f23a41018f0 2023-12-01T01:15:46.242 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.238+0000 7f238bfff700 1 -- 172.21.15.77:0/2083863415 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239c005610 con 0x7f23a41018f0 2023-12-01T01:15:46.242 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.238+0000 7f23a92e1700 1 -- 172.21.15.77:0/2083863415 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23a419f150 con 0x7f23a41018f0 2023-12-01T01:15:46.242 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.239+0000 7f23a92e1700 1 -- 172.21.15.77:0/2083863415 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23a410b640 con 0x7f23a41018f0 2023-12-01T01:15:46.243 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.241+0000 7f238bfff700 1 -- 172.21.15.77:0/2083863415 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f239c005770 con 0x7f23a41018f0 2023-12-01T01:15:46.243 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.241+0000 7f238bfff700 1 --2- 172.21.15.77:0/2083863415 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f238c0a05a0 0x7f238c0a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:46.243 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.241+0000 7f238bfff700 1 -- 172.21.15.77:0/2083863415 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f239c0cd570 con 0x7f23a41018f0 2023-12-01T01:15:46.243 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.241+0000 7f23a2ffd700 1 --2- 172.21.15.77:0/2083863415 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f238c0a05a0 0x7f238c0a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:46.243 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.242+0000 7f23a2ffd700 1 --2- 172.21.15.77:0/2083863415 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f238c0a05a0 0x7f238c0a2a60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2398005950 tx=0x7f23980058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:15:46.246 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.245+0000 7f238bfff700 1 -- 172.21.15.77:0/2083863415 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f239c0975a0 con 0x7f23a41018f0 2023-12-01T01:15:46.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.436+0000 7f23a92e1700 1 -- 172.21.15.77:0/2083863415 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f23a419f760 con 0x7f23a41018f0 2023-12-01T01:15:46.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.436+0000 7f238bfff700 1 -- 172.21.15.77:0/2083863415 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v78) v1 ==== 74+0+133 (secure 0 0 0) 0x7f239c09abc0 con 0x7f23a41018f0 2023-12-01T01:15:46.437 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:15:46.439 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.438+0000 7f23a92e1700 1 -- 172.21.15.77:0/2083863415 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f238c0a05a0 msgr2=0x7f238c0a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:46.439 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.438+0000 7f23a92e1700 1 --2- 172.21.15.77:0/2083863415 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f238c0a05a0 0x7f238c0a2a60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2398005950 tx=0x7f23980058e0 comp rx=0 tx=0).stop 2023-12-01T01:15:46.439 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.438+0000 7f23a92e1700 1 -- 172.21.15.77:0/2083863415 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23a41018f0 msgr2=0x7f23a419e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:46.439 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.438+0000 7f23a92e1700 1 --2- 172.21.15.77:0/2083863415 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23a41018f0 0x7f23a419e1f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f239c007a00 tx=0x7f239c009570 comp rx=0 tx=0).stop 2023-12-01T01:15:46.439 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.438+0000 7f23a92e1700 1 -- 172.21.15.77:0/2083863415 shutdown_connections 2023-12-01T01:15:46.440 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.438+0000 7f23a92e1700 1 --2- 172.21.15.77:0/2083863415 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f238c0a05a0 0x7f238c0a2a60 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:46.440 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.438+0000 7f23a92e1700 1 --2- 172.21.15.77:0/2083863415 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23a41018f0 0x7f23a419e1f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:46.440 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.438+0000 7f23a92e1700 1 --2- 172.21.15.77:0/2083863415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23a40ffd50 0x7f23a4196c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:46.440 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.438+0000 7f23a92e1700 1 --2- 172.21.15.77:0/2083863415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23a4100f50 0x7f23a41971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:46.440 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.438+0000 7f23a92e1700 1 -- 172.21.15.77:0/2083863415 >> 172.21.15.77:0/2083863415 conn(0x7f23a40fb310 msgr2=0x7f23a40fbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:15:46.440 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.438+0000 7f23a92e1700 1 -- 172.21.15.77:0/2083863415 shutdown_connections 2023-12-01T01:15:46.440 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:46.438+0000 7f23a92e1700 1 -- 172.21.15.77:0/2083863415 wait complete. 2023-12-01T01:15:46.833 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":78,"num_osds":12,"num_up_osds":12,"osd_up_since":1701393344,"num_in_osds":12,"osd_in_since":1701393326,"num_remapped_pgs":0} 2023-12-01T01:15:46.834 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph osd dump --format=json 2023-12-01T01:15:47.385 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:15:47.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:47 smithi077 ceph-mon[98867]: osdmap e78: 12 total, 12 up, 12 in 2023-12-01T01:15:47.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:47 smithi077 ceph-mon[98867]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:15:47.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:47 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/2083863415' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-01T01:15:47.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:47 smithi203 ceph-mon[103581]: osdmap e78: 12 total, 12 up, 12 in 2023-12-01T01:15:47.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:47 smithi203 ceph-mon[103581]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:15:47.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:47 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/2083863415' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-01T01:15:47.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:47 smithi137 ceph-mon[103312]: osdmap e78: 12 total, 12 up, 12 in 2023-12-01T01:15:47.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:47 smithi137 ceph-mon[103312]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:15:47.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:47 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/2083863415' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-01T01:15:48.447 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.444+0000 7f013d678700 1 -- 172.21.15.77:0/3326849152 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01381022b0 msgr2=0x7f013810cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:48.447 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.444+0000 7f013d678700 1 --2- 172.21.15.77:0/3326849152 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01381022b0 0x7f013810cc90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f013000b0d0 tx=0x7f013000b3e0 comp rx=0 tx=0).stop 2023-12-01T01:15:48.447 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.444+0000 7f013d678700 1 -- 172.21.15.77:0/3326849152 shutdown_connections 2023-12-01T01:15:48.448 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.444+0000 7f013d678700 1 --2- 172.21.15.77:0/3326849152 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01381022b0 0x7f013810cc90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:48.448 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.444+0000 7f013d678700 1 --2- 172.21.15.77:0/3326849152 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0138100010 0x7f01381003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:48.448 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.444+0000 7f013d678700 1 --2- 172.21.15.77:0/3326849152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01381018f0 0x7f0138101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:48.448 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.444+0000 7f013d678700 1 -- 172.21.15.77:0/3326849152 >> 172.21.15.77:0/3326849152 conn(0x7f0138076d80 msgr2=0x7f0138077190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:15:48.448 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.444+0000 7f013d678700 1 -- 172.21.15.77:0/3326849152 shutdown_connections 2023-12-01T01:15:48.448 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.444+0000 7f013d678700 1 -- 172.21.15.77:0/3326849152 wait complete. 2023-12-01T01:15:48.448 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.445+0000 7f013d678700 1 Processor -- start 2023-12-01T01:15:48.448 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.445+0000 7f013d678700 1 -- start start 2023-12-01T01:15:48.448 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.445+0000 7f013d678700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0138100010 0x7f0138102ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:48.448 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.445+0000 7f013d678700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01381018f0 0x7f0138103010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:48.448 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.445+0000 7f013d678700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01381022b0 0x7f0138103590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:48.448 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.445+0000 7f013d678700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01381a55b0 con 0x7f0138100010 2023-12-01T01:15:48.448 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.445+0000 7f013d678700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01381a5720 con 0x7f01381018f0 2023-12-01T01:15:48.448 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.445+0000 7f013d678700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01381a58e0 con 0x7f01381022b0 2023-12-01T01:15:48.449 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.445+0000 7f0136ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0138100010 0x7f0138102ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:48.449 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.445+0000 7f0136ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0138100010 0x7f0138102ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57104/0 (socket says 172.21.15.77:57104) 2023-12-01T01:15:48.449 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.445+0000 7f0136ffd700 1 -- 172.21.15.77:0/3941018729 learned_addr learned my addr 172.21.15.77:0/3941018729 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:15:48.449 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.445+0000 7f01377fe700 1 --2- 172.21.15.77:0/3941018729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01381022b0 0x7f0138103590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:48.449 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.445+0000 7f01367fc700 1 --2- 172.21.15.77:0/3941018729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01381018f0 0x7f0138103010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:48.449 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.446+0000 7f0136ffd700 1 -- 172.21.15.77:0/3941018729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01381022b0 msgr2=0x7f0138103590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:48.449 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.446+0000 7f0136ffd700 1 --2- 172.21.15.77:0/3941018729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01381022b0 0x7f0138103590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:48.449 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.446+0000 7f0136ffd700 1 -- 172.21.15.77:0/3941018729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01381018f0 msgr2=0x7f0138103010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:48.449 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.446+0000 7f0136ffd700 1 --2- 172.21.15.77:0/3941018729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01381018f0 0x7f0138103010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:48.449 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.446+0000 7f0136ffd700 1 -- 172.21.15.77:0/3941018729 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0130009d00 con 0x7f0138100010 2023-12-01T01:15:48.449 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.446+0000 7f01367fc700 1 --2- 172.21.15.77:0/3941018729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01381018f0 0x7f0138103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:15:48.449 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.446+0000 7f0136ffd700 1 --2- 172.21.15.77:0/3941018729 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0138100010 0x7f0138102ad0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f012000d980 tx=0x7f012000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:15:48.449 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.446+0000 7f012ffff700 1 -- 172.21.15.77:0/3941018729 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0120009b40 con 0x7f0138100010 2023-12-01T01:15:48.449 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.446+0000 7f012ffff700 1 -- 172.21.15.77:0/3941018729 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f012000de60 con 0x7f0138100010 2023-12-01T01:15:48.450 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.446+0000 7f013d678700 1 -- 172.21.15.77:0/3941018729 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01381a5bc0 con 0x7f0138100010 2023-12-01T01:15:48.450 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.446+0000 7f012ffff700 1 -- 172.21.15.77:0/3941018729 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0120005580 con 0x7f0138100010 2023-12-01T01:15:48.450 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.446+0000 7f013d678700 1 -- 172.21.15.77:0/3941018729 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01381a6070 con 0x7f0138100010 2023-12-01T01:15:48.451 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.447+0000 7f013d678700 1 -- 172.21.15.77:0/3941018729 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0138109380 con 0x7f0138100010 2023-12-01T01:15:48.451 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.448+0000 7f012ffff700 1 -- 172.21.15.77:0/3941018729 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f0120004750 con 0x7f0138100010 2023-12-01T01:15:48.451 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.449+0000 7f012ffff700 1 --2- 172.21.15.77:0/3941018729 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f01240a0520 0x7f01240a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:48.451 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.449+0000 7f01367fc700 1 --2- 172.21.15.77:0/3941018729 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f01240a0520 0x7f01240a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:48.451 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.449+0000 7f012ffff700 1 -- 172.21.15.77:0/3941018729 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f01200cb860 con 0x7f0138100010 2023-12-01T01:15:48.451 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.449+0000 7f01367fc700 1 --2- 172.21.15.77:0/3941018729 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f01240a0520 0x7f01240a29e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0128005fd0 tx=0x7f0128005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:15:48.453 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.452+0000 7f012ffff700 1 -- 172.21.15.77:0/3941018729 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f01200958a0 con 0x7f0138100010 2023-12-01T01:15:48.644 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.642+0000 7f013d678700 1 -- 172.21.15.77:0/3941018729 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f013804ea90 con 0x7f0138100010 2023-12-01T01:15:48.645 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.643+0000 7f012ffff700 1 -- 172.21.15.77:0/3941018729 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+18434 (secure 0 0 0) 0x7f0120017090 con 0x7f0138100010 2023-12-01T01:15:48.645 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:15:48.645 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":78,"fsid":"2a06e29a-8fe6-11ee-95a2-87774f69a715","created":"2023-12-01T01:09:30.210398+0000","modified":"2023-12-01T01:15:46.072360+0000","last_up_change":"2023-12-01T01:15:44.586092+0000","last_in_change":"2023-12-01T01:15:26.305439+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:12:30.514229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fc651e62-a103-4ab8-a76d-d3708ea6aeec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6803","nonce":2244799094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6805","nonce":2244799094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6809","nonce":2244799094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6807","nonce":2244799094}]},"public_addr":"172.21.15.77:6803/2244799094","cluster_addr":"172.21.15.77:6805/2244799094","heartbeat_back_addr":"172.21.15.77:6809/2244799094","heartbeat_front_addr":"172.21.15.77:6807/2244799094","state":["exists","up"]},{"osd":1,"uuid":"0adf8c54-49ec-49a0-8bf1-af1f20e60729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6801","nonce":224194890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6803","nonce":224194890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6807","nonce":224194890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6805","nonce":224194890}]},"public_addr":"172.21.15.137:6801/224194890","cluster_addr":"172.21.15.137:6803/224194890","heartbeat_back_addr":"172.21.15.137:6807/224194890","heartbeat_front_addr":"172.21.15.137:6805/224194890","state":["exists","up"]},{"osd":2,"uuid":"5a69802c-dede-4544-9a12-436f5c1c0038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6801","nonce":4078459110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6803","nonce":4078459110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6807","nonce":4078459110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6805","nonce":4078459110}]},"public_addr":"172.21.15.203:6801/4078459110","cluster_addr":"172.21.15.203:6803/4078459110","heartbeat_back_addr":"172.21.15.203:6807/4078459110","heartbeat_front_addr":"172.21.15.203:6805/4078459110","state":["exists","up"]},{"osd":3,"uuid":"ad325a97-580b-4c99-bf12-cbb0d92a6d0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6811","nonce":1454440477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6813","nonce":1454440477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6817","nonce":1454440477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6815","nonce":1454440477}]},"public_addr":"172.21.15.77:6811/1454440477","cluster_addr":"172.21.15.77:6813/1454440477","heartbeat_back_addr":"172.21.15.77:6817/1454440477","heartbeat_front_addr":"172.21.15.77:6815/1454440477","state":["exists","up"]},{"osd":4,"uuid":"1e3ac1ba-98bb-4b8b-8e26-50064c66bf2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6809","nonce":1103241451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6811","nonce":1103241451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6815","nonce":1103241451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6813","nonce":1103241451}]},"public_addr":"172.21.15.137:6809/1103241451","cluster_addr":"172.21.15.137:6811/1103241451","heartbeat_back_addr":"172.21.15.137:6815/1103241451","heartbeat_front_addr":"172.21.15.137:6813/1103241451","state":["exists","up"]},{"osd":5,"uuid":"eb32c016-abeb-44ab-abf0-19dd09235736","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6809","nonce":2648569482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6811","nonce":2648569482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6815","nonce":2648569482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6813","nonce":2648569482}]},"public_addr":"172.21.15.203:6809/2648569482","cluster_addr":"172.21.15.203:6811/2648569482","heartbeat_back_addr":"172.21.15.203:6815/2648569482","heartbeat_front_addr":"172.21.15.203:6813/2648569482","state":["exists","up"]},{"osd":6,"uuid":"0e3fa97e-2ca6-418c-9e8d-a6fe3c56adb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6819","nonce":2463414651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6821","nonce":2463414651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6825","nonce":2463414651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6823","nonce":2463414651}]},"public_addr":"172.21.15.77:6819/2463414651","cluster_addr":"172.21.15.77:6821/2463414651","heartbeat_back_addr":"172.21.15.77:6825/2463414651","heartbeat_front_addr":"172.21.15.77:6823/2463414651","state":["exists","up"]},{"osd":7,"uuid":"d7398c01-0f98-4138-8a49-c3ff0aae7e88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6817","nonce":3108330766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6819","nonce":3108330766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6823","nonce":3108330766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6821","nonce":3108330766}]},"public_addr":"172.21.15.137:6817/3108330766","cluster_addr":"172.21.15.137:6819/3108330766","heartbeat_back_addr":"172.21.15.137:6823/3108330766","heartbeat_front_addr":"172.21.15.137:6821/3108330766","state":["exists","up"]},{"osd":8,"uuid":"0c09050d-3e43-410c-8cd7-bcb891af56b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6817","nonce":1161158776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6819","nonce":1161158776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6823","nonce":1161158776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6821","nonce":1161158776}]},"public_addr":"172.21.15.203:6817/1161158776","cluster_addr":"172.21.15.203:6819/1161158776","heartbeat_back_addr":"172.21.15.203:6823/1161158776","heartbeat_front_addr":"172.21.15.203:6821/1161158776","state":["exists","up"]},{"osd":9,"uuid":"527ecb49-95ec-48bd-9872-3b07fae5c5ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6827","nonce":2409880685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6829","nonce":2409880685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6833","nonce":2409880685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6831","nonce":2409880685}]},"public_addr":"172.21.15.77:6827/2409880685","cluster_addr":"172.21.15.77:6829/2409880685","heartbeat_back_addr":"172.21.15.77:6833/2409880685","heartbeat_front_addr":"172.21.15.77:6831/2409880685","state":["exists","up"]},{"osd":10,"uuid":"1680bb3e-a3c8-427b-af68-e2214f32bed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6825","nonce":374920146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6827","nonce":374920146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6831","nonce":374920146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6829","nonce":374920146}]},"public_addr":"172.21.15.137:6825/374920146","cluster_addr":"172.21.15.137:6827/374920146","heartbeat_back_addr":"172.21.15.137:6831/374920146","heartbeat_front_addr":"172.21.15.137:6829/374920146","state":["exists","up"]},{"osd":11,"uuid":"8a533466-023d-4803-ae36-9a9f1715e49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6825","nonce":2161588962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6827","nonce":2161588962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6831","nonce":2161588962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6829","nonce":2161588962}]},"public_addr":"172.21.15.203:6825/2161588962","cluster_addr":"172.21.15.203:6827/2161588962","heartbeat_back_addr":"172.21.15.203:6831/2161588962","heartbeat_front_addr":"172.21.15.203:6829/2161588962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:11:45.420509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:06.210115+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:28.191307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:49.594116+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:11.261689+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:33.376541+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:54.968336+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:16.775310+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:38.733792+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:59.456699+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:15:20.924059+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:15:43.094773+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/2260035254":"2023-12-02T01:10:24.191835+0000","172.21.15.77:6800/1456621747":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/219838828":"2023-12-02T01:10:24.191835+0000","172.21.15.77:6801/1456621747":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/2937149266":"2023-12-02T01:10:02.232948+0000","172.21.15.77:0/807520018":"2023-12-02T01:10:02.232948+0000","172.21.15.77:6801/1661788026":"2023-12-02T01:10:02.232948+0000","172.21.15.77:6800/1661788026":"2023-12-02T01:10:02.232948+0000","172.21.15.77:0/3551282473":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/1202222558":"2023-12-02T01:10:02.232948+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:15:48.647 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.646+0000 7f013d678700 1 -- 172.21.15.77:0/3941018729 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f01240a0520 msgr2=0x7f01240a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:48.647 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.646+0000 7f013d678700 1 --2- 172.21.15.77:0/3941018729 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f01240a0520 0x7f01240a29e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0128005fd0 tx=0x7f0128005d40 comp rx=0 tx=0).stop 2023-12-01T01:15:48.647 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.646+0000 7f013d678700 1 -- 172.21.15.77:0/3941018729 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0138100010 msgr2=0x7f0138102ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:48.647 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.646+0000 7f013d678700 1 --2- 172.21.15.77:0/3941018729 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0138100010 0x7f0138102ad0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f012000d980 tx=0x7f012000dd40 comp rx=0 tx=0).stop 2023-12-01T01:15:48.647 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.646+0000 7f013d678700 1 -- 172.21.15.77:0/3941018729 shutdown_connections 2023-12-01T01:15:48.648 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.646+0000 7f013d678700 1 --2- 172.21.15.77:0/3941018729 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f01240a0520 0x7f01240a29e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:48.648 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.646+0000 7f013d678700 1 --2- 172.21.15.77:0/3941018729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01381022b0 0x7f0138103590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:48.648 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.646+0000 7f013d678700 1 --2- 172.21.15.77:0/3941018729 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0138100010 0x7f0138102ad0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:48.648 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.646+0000 7f013d678700 1 --2- 172.21.15.77:0/3941018729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01381018f0 0x7f0138103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:48.648 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.646+0000 7f013d678700 1 -- 172.21.15.77:0/3941018729 >> 172.21.15.77:0/3941018729 conn(0x7f0138076d80 msgr2=0x7f0138069220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:15:48.648 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.646+0000 7f013d678700 1 -- 172.21.15.77:0/3941018729 shutdown_connections 2023-12-01T01:15:48.648 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:48.646+0000 7f013d678700 1 -- 172.21.15.77:0/3941018729 wait complete. 2023-12-01T01:15:49.025 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:12:30.514229+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:15:49.026 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph osd pool get .mgr pg_num 2023-12-01T01:15:49.577 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:15:49.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:49 smithi077 ceph-mon[98867]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:15:49.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:49 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/3941018729' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:15:49.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:49 smithi203 ceph-mon[103581]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:15:49.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:49 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/3941018729' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:15:49.778 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:49 smithi137 ceph-mon[103312]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:15:49.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:49 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/3941018729' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:15:50.710 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.707+0000 7f952d34d700 1 -- 172.21.15.77:0/2555049091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f95280ff6b0 msgr2=0x7f95280ffb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:50.710 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.707+0000 7f952d34d700 1 --2- 172.21.15.77:0/2555049091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f95280ff6b0 0x7f95280ffb30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9518009990 tx=0x7f9518009ca0 comp rx=0 tx=0).stop 2023-12-01T01:15:50.711 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.707+0000 7f952d34d700 1 -- 172.21.15.77:0/2555049091 shutdown_connections 2023-12-01T01:15:50.711 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.707+0000 7f952d34d700 1 --2- 172.21.15.77:0/2555049091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9528100070 0x7f952810dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:50.711 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.707+0000 7f952d34d700 1 --2- 172.21.15.77:0/2555049091 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9528105e40 0x7f9528106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:50.711 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.707+0000 7f952d34d700 1 --2- 172.21.15.77:0/2555049091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f95280ff6b0 0x7f95280ffb30 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:50.711 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.707+0000 7f952d34d700 1 -- 172.21.15.77:0/2555049091 >> 172.21.15.77:0/2555049091 conn(0x7f9528074fb0 msgr2=0x7f95280753c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:15:50.711 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.707+0000 7f952d34d700 1 -- 172.21.15.77:0/2555049091 shutdown_connections 2023-12-01T01:15:50.711 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.707+0000 7f952d34d700 1 -- 172.21.15.77:0/2555049091 wait complete. 2023-12-01T01:15:50.711 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.708+0000 7f952d34d700 1 Processor -- start 2023-12-01T01:15:50.711 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.708+0000 7f952d34d700 1 -- start start 2023-12-01T01:15:50.711 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.708+0000 7f952d34d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95280ff6b0 0x7f9528196ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:50.711 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.708+0000 7f952d34d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9528100070 0x7f9528197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:50.711 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.708+0000 7f952d34d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9528105e40 0x7f952819b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:50.711 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.708+0000 7f952d34d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952819beb0 con 0x7f9528100070 2023-12-01T01:15:50.712 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.708+0000 7f952d34d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952819c020 con 0x7f95280ff6b0 2023-12-01T01:15:50.712 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.708+0000 7f952d34d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952819c1e0 con 0x7f9528105e40 2023-12-01T01:15:50.712 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.708+0000 7f95267fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9528100070 0x7f9528197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:50.712 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.708+0000 7f95267fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9528100070 0x7f9528197430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57118/0 (socket says 172.21.15.77:57118) 2023-12-01T01:15:50.712 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.708+0000 7f95267fc700 1 -- 172.21.15.77:0/778174647 learned_addr learned my addr 172.21.15.77:0/778174647 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:15:50.712 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.708+0000 7f95277fe700 1 --2- 172.21.15.77:0/778174647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9528105e40 0x7f952819b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:50.712 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.709+0000 7f9526ffd700 1 --2- 172.21.15.77:0/778174647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95280ff6b0 0x7f9528196ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:50.712 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.709+0000 7f95267fc700 1 -- 172.21.15.77:0/778174647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9528105e40 msgr2=0x7f952819b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:50.712 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.709+0000 7f95267fc700 1 --2- 172.21.15.77:0/778174647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9528105e40 0x7f952819b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:50.712 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.709+0000 7f95267fc700 1 -- 172.21.15.77:0/778174647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95280ff6b0 msgr2=0x7f9528196ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:50.712 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.709+0000 7f95267fc700 1 --2- 172.21.15.77:0/778174647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95280ff6b0 0x7f9528196ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:50.712 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.709+0000 7f95267fc700 1 -- 172.21.15.77:0/778174647 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95180095f0 con 0x7f9528100070 2023-12-01T01:15:50.712 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.709+0000 7f9526ffd700 1 --2- 172.21.15.77:0/778174647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95280ff6b0 0x7f9528196ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:15:50.713 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.709+0000 7f95267fc700 1 --2- 172.21.15.77:0/778174647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9528100070 0x7f9528197430 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9518000c00 tx=0x7f951800f830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:15:50.713 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.709+0000 7f950ffff700 1 -- 172.21.15.77:0/778174647 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951801d070 con 0x7f9528100070 2023-12-01T01:15:50.713 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.709+0000 7f952d34d700 1 -- 172.21.15.77:0/778174647 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f952819c460 con 0x7f9528100070 2023-12-01T01:15:50.713 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.709+0000 7f950ffff700 1 -- 172.21.15.77:0/778174647 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f951800fce0 con 0x7f9528100070 2023-12-01T01:15:50.713 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.709+0000 7f950ffff700 1 -- 172.21.15.77:0/778174647 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95180176e0 con 0x7f9528100070 2023-12-01T01:15:50.714 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.709+0000 7f952d34d700 1 -- 172.21.15.77:0/778174647 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95281a3d20 con 0x7f9528100070 2023-12-01T01:15:50.714 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.710+0000 7f952d34d700 1 -- 172.21.15.77:0/778174647 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f952804ea90 con 0x7f9528100070 2023-12-01T01:15:50.714 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.711+0000 7f950ffff700 1 -- 172.21.15.77:0/778174647 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f9518022470 con 0x7f9528100070 2023-12-01T01:15:50.714 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.711+0000 7f950ffff700 1 --2- 172.21.15.77:0/778174647 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f95100a0520 0x7f95100a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:50.714 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.711+0000 7f9526ffd700 1 --2- 172.21.15.77:0/778174647 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f95100a0520 0x7f95100a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:50.714 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.712+0000 7f950ffff700 1 -- 172.21.15.77:0/778174647 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f95180cc8d0 con 0x7f9528100070 2023-12-01T01:15:50.714 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.712+0000 7f9526ffd700 1 --2- 172.21.15.77:0/778174647 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f95100a0520 0x7f95100a29e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f951c005fd0 tx=0x7f951c005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:15:50.717 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.716+0000 7f950ffff700 1 -- 172.21.15.77:0/778174647 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f951805d0e0 con 0x7f9528100070 2023-12-01T01:15:50.904 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.903+0000 7f952d34d700 1 -- 172.21.15.77:0/778174647 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f95281a3eb0 con 0x7f9528100070 2023-12-01T01:15:50.905 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.903+0000 7f950ffff700 1 -- 172.21.15.77:0/778174647 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7f95180969a0 con 0x7f9528100070 2023-12-01T01:15:50.905 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 1 2023-12-01T01:15:50.907 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.905+0000 7f952d34d700 1 -- 172.21.15.77:0/778174647 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f95100a0520 msgr2=0x7f95100a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:50.907 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.905+0000 7f952d34d700 1 --2- 172.21.15.77:0/778174647 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f95100a0520 0x7f95100a29e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f951c005fd0 tx=0x7f951c005d40 comp rx=0 tx=0).stop 2023-12-01T01:15:50.907 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.905+0000 7f952d34d700 1 -- 172.21.15.77:0/778174647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9528100070 msgr2=0x7f9528197430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:50.907 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.905+0000 7f952d34d700 1 --2- 172.21.15.77:0/778174647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9528100070 0x7f9528197430 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9518000c00 tx=0x7f951800f830 comp rx=0 tx=0).stop 2023-12-01T01:15:50.907 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.905+0000 7f952d34d700 1 -- 172.21.15.77:0/778174647 shutdown_connections 2023-12-01T01:15:50.907 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.905+0000 7f952d34d700 1 --2- 172.21.15.77:0/778174647 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f95100a0520 0x7f95100a29e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:50.907 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.905+0000 7f952d34d700 1 --2- 172.21.15.77:0/778174647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9528105e40 0x7f952819b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:50.907 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.905+0000 7f952d34d700 1 --2- 172.21.15.77:0/778174647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95280ff6b0 0x7f9528196ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:50.907 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.905+0000 7f952d34d700 1 --2- 172.21.15.77:0/778174647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9528100070 0x7f9528197430 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:50.907 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.905+0000 7f952d34d700 1 -- 172.21.15.77:0/778174647 >> 172.21.15.77:0/778174647 conn(0x7f9528074fb0 msgr2=0x7f95280fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:15:50.907 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.905+0000 7f952d34d700 1 -- 172.21.15.77:0/778174647 shutdown_connections 2023-12-01T01:15:50.907 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:50.906+0000 7f952d34d700 1 -- 172.21.15.77:0/778174647 wait complete. 2023-12-01T01:15:51.316 INFO:tasks.cephadm:Adding mds.a on smithi077 2023-12-01T01:15:51.317 INFO:tasks.cephadm:Adding mds.d on smithi077 2023-12-01T01:15:51.317 INFO:tasks.cephadm:Adding mds.g on smithi077 2023-12-01T01:15:51.317 INFO:tasks.cephadm:Adding mds.j on smithi077 2023-12-01T01:15:51.317 INFO:tasks.cephadm:Adding mds.b on smithi137 2023-12-01T01:15:51.317 INFO:tasks.cephadm:Adding mds.e on smithi137 2023-12-01T01:15:51.317 INFO:tasks.cephadm:Adding mds.h on smithi137 2023-12-01T01:15:51.318 INFO:tasks.cephadm:Adding mds.k on smithi137 2023-12-01T01:15:51.318 INFO:tasks.cephadm:Adding mds.c on smithi203 2023-12-01T01:15:51.318 INFO:tasks.cephadm:Adding mds.f on smithi203 2023-12-01T01:15:51.318 INFO:tasks.cephadm:Adding mds.i on smithi203 2023-12-01T01:15:51.318 INFO:tasks.cephadm:Adding mds.l on smithi203 2023-12-01T01:15:51.319 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph orch apply mds all '12;smithi077=a;smithi077=d;smithi077=g;smithi077=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l' 2023-12-01T01:15:51.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:51 smithi077 ceph-mon[98867]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:15:51.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:51 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/778174647' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-01T01:15:51.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:51 smithi203 ceph-mon[103581]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:15:51.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:51 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/778174647' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-01T01:15:51.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:51 smithi137 ceph-mon[103312]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:15:51.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:51 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/778174647' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-01T01:15:51.885 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.c/config 2023-12-01T01:15:53.464 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.461+0000 7effa39bd700 1 -- 172.21.15.203:0/2613281149 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff9c101f80 msgr2=0x7eff9c102400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:53.464 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.461+0000 7effa39bd700 1 --2- 172.21.15.203:0/2613281149 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff9c101f80 0x7eff9c102400 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7eff98009990 tx=0x7eff98009ca0 comp rx=0 tx=0).stop 2023-12-01T01:15:53.464 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.461+0000 7effa39bd700 1 -- 172.21.15.203:0/2613281149 shutdown_connections 2023-12-01T01:15:53.464 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.461+0000 7effa39bd700 1 --2- 172.21.15.203:0/2613281149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff9c102940 0x7eff9c109260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:53.464 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.461+0000 7effa39bd700 1 --2- 172.21.15.203:0/2613281149 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff9c100e20 0x7eff9c101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:53.464 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.461+0000 7effa39bd700 1 --2- 172.21.15.203:0/2613281149 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff9c101f80 0x7eff9c102400 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:53.464 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.461+0000 7effa39bd700 1 -- 172.21.15.203:0/2613281149 >> 172.21.15.203:0/2613281149 conn(0x7eff9c0fc380 msgr2=0x7eff9c0fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:15:53.464 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.461+0000 7effa39bd700 1 -- 172.21.15.203:0/2613281149 shutdown_connections 2023-12-01T01:15:53.465 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.461+0000 7effa39bd700 1 -- 172.21.15.203:0/2613281149 wait complete. 2023-12-01T01:15:53.465 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.462+0000 7effa39bd700 1 Processor -- start 2023-12-01T01:15:53.465 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.462+0000 7effa39bd700 1 -- start start 2023-12-01T01:15:53.465 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.462+0000 7effa39bd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff9c100e20 0x7eff9c074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:53.465 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.462+0000 7effa39bd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff9c101f80 0x7eff9c073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:53.465 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.462+0000 7effa39bd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff9c102940 0x7eff9c0736c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:53.465 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.462+0000 7effa39bd700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c073e40 con 0x7eff9c100e20 2023-12-01T01:15:53.465 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.462+0000 7effa39bd700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c073fb0 con 0x7eff9c101f80 2023-12-01T01:15:53.465 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.462+0000 7effa39bd700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c1a0b40 con 0x7eff9c102940 2023-12-01T01:15:53.465 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.462+0000 7effa1f5a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff9c102940 0x7eff9c0736c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:53.465 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.462+0000 7effa1f5a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff9c102940 0x7eff9c0736c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:59662/0 (socket says 172.21.15.203:59662) 2023-12-01T01:15:53.465 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.462+0000 7effa1f5a700 1 -- 172.21.15.203:0/616573447 learned_addr learned my addr 172.21.15.203:0/616573447 (peer_addr_for_me v2:172.21.15.203:0/0) 2023-12-01T01:15:53.465 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.462+0000 7effa1759700 1 --2- 172.21.15.203:0/616573447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff9c100e20 0x7eff9c074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:53.465 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.462+0000 7effa0f58700 1 --2- 172.21.15.203:0/616573447 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff9c101f80 0x7eff9c073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:53.465 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.462+0000 7effa1f5a700 1 -- 172.21.15.203:0/616573447 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff9c101f80 msgr2=0x7eff9c073180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:53.466 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.462+0000 7effa1f5a700 1 --2- 172.21.15.203:0/616573447 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff9c101f80 0x7eff9c073180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:53.466 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.462+0000 7effa1f5a700 1 -- 172.21.15.203:0/616573447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff9c100e20 msgr2=0x7eff9c074b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:53.466 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.462+0000 7effa1f5a700 1 --2- 172.21.15.203:0/616573447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff9c100e20 0x7eff9c074b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:53.466 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.462+0000 7effa1f5a700 1 -- 172.21.15.203:0/616573447 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff980095f0 con 0x7eff9c102940 2023-12-01T01:15:53.466 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.462+0000 7effa1f5a700 1 --2- 172.21.15.203:0/616573447 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff9c102940 0x7eff9c0736c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7eff94007ee0 tx=0x7eff9400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:15:53.466 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.463+0000 7eff927fc700 1 -- 172.21.15.203:0/616573447 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff94017070 con 0x7eff9c102940 2023-12-01T01:15:53.466 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.463+0000 7eff927fc700 1 -- 172.21.15.203:0/616573447 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff9400f040 con 0x7eff9c102940 2023-12-01T01:15:53.466 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.463+0000 7effa39bd700 1 -- 172.21.15.203:0/616573447 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff9c1a0e20 con 0x7eff9c102940 2023-12-01T01:15:53.466 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.463+0000 7eff927fc700 1 -- 172.21.15.203:0/616573447 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff94005740 con 0x7eff9c102940 2023-12-01T01:15:53.466 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.463+0000 7effa39bd700 1 -- 172.21.15.203:0/616573447 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff9c1a1290 con 0x7eff9c102940 2023-12-01T01:15:53.467 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.464+0000 7effa39bd700 1 -- 172.21.15.203:0/616573447 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff9c108410 con 0x7eff9c102940 2023-12-01T01:15:53.467 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.465+0000 7eff927fc700 1 -- 172.21.15.203:0/616573447 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7eff9400dc60 con 0x7eff9c102940 2023-12-01T01:15:53.467 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.465+0000 7eff927fc700 1 --2- 172.21.15.203:0/616573447 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7eff880a05a0 0x7eff880a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:53.467 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.466+0000 7eff927fc700 1 -- 172.21.15.203:0/616573447 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7eff940cb260 con 0x7eff9c102940 2023-12-01T01:15:53.467 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.466+0000 7effa1759700 1 --2- 172.21.15.203:0/616573447 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7eff880a05a0 0x7eff880a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:53.468 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.466+0000 7effa1759700 1 --2- 172.21.15.203:0/616573447 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7eff880a05a0 0x7eff880a2a60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7eff8c009a10 tx=0x7eff8c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:15:53.470 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.469+0000 7eff927fc700 1 -- 172.21.15.203:0/616573447 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7eff940952a0 con 0x7eff9c102940 2023-12-01T01:15:53.668 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.667+0000 7effa39bd700 1 -- 172.21.15.203:0/616573447 --> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi077=a;smithi077=d;smithi077=g;smithi077=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l", "target": ["mon-mgr", ""]}) v1 -- 0x7eff9c061490 con 0x7eff880a05a0 2023-12-01T01:15:53.676 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.675+0000 7eff927fc700 1 -- 172.21.15.203:0/616573447 <== mgr.14152 v2:172.21.15.77:6800/3838941679 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7eff9c061490 con 0x7eff880a05a0 2023-12-01T01:15:53.676 INFO:teuthology.orchestra.run.smithi203.stdout:Scheduled mds.all update... 2023-12-01T01:15:53.678 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.677+0000 7effa39bd700 1 -- 172.21.15.203:0/616573447 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7eff880a05a0 msgr2=0x7eff880a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:53.678 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.677+0000 7effa39bd700 1 --2- 172.21.15.203:0/616573447 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7eff880a05a0 0x7eff880a2a60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7eff8c009a10 tx=0x7eff8c009190 comp rx=0 tx=0).stop 2023-12-01T01:15:53.678 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.677+0000 7effa39bd700 1 -- 172.21.15.203:0/616573447 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff9c102940 msgr2=0x7eff9c0736c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:53.678 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.677+0000 7effa39bd700 1 --2- 172.21.15.203:0/616573447 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff9c102940 0x7eff9c0736c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7eff94007ee0 tx=0x7eff9400d3b0 comp rx=0 tx=0).stop 2023-12-01T01:15:53.678 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.677+0000 7effa39bd700 1 -- 172.21.15.203:0/616573447 shutdown_connections 2023-12-01T01:15:53.679 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.677+0000 7effa39bd700 1 --2- 172.21.15.203:0/616573447 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7eff880a05a0 0x7eff880a2a60 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:53.679 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.677+0000 7effa39bd700 1 --2- 172.21.15.203:0/616573447 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff9c102940 0x7eff9c0736c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:53.679 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.677+0000 7effa39bd700 1 --2- 172.21.15.203:0/616573447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff9c100e20 0x7eff9c074b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:53.679 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.677+0000 7effa39bd700 1 --2- 172.21.15.203:0/616573447 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff9c101f80 0x7eff9c073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:53.679 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.677+0000 7effa39bd700 1 -- 172.21.15.203:0/616573447 >> 172.21.15.203:0/616573447 conn(0x7eff9c0fc380 msgr2=0x7eff9c105550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:15:53.679 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.677+0000 7effa39bd700 1 -- 172.21.15.203:0/616573447 shutdown_connections 2023-12-01T01:15:53.679 INFO:teuthology.orchestra.run.smithi203.stderr:2023-12-01T01:15:53.677+0000 7effa39bd700 1 -- 172.21.15.203:0/616573447 wait complete. 2023-12-01T01:15:53.841 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:53 smithi203 ceph-mon[103581]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:15:53.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:53 smithi077 ceph-mon[98867]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:15:54.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:53 smithi137 ceph-mon[103312]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:15:54.147 DEBUG:teuthology.orchestra.run.smithi077:mds.a> sudo journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mds.a.service 2023-12-01T01:15:54.149 DEBUG:teuthology.orchestra.run.smithi077:mds.d> sudo journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mds.d.service 2023-12-01T01:15:54.187 INFO:journalctl@ceph.mds.a.smithi077.stdout:-- Logs begin at Fri 2023-12-01 00:55:13 UTC. -- 2023-12-01T01:15:54.188 DEBUG:teuthology.orchestra.run.smithi077:mds.g> sudo journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mds.g.service 2023-12-01T01:15:54.217 INFO:journalctl@ceph.mds.d.smithi077.stdout:-- Logs begin at Fri 2023-12-01 00:55:13 UTC. -- 2023-12-01T01:15:54.218 DEBUG:teuthology.orchestra.run.smithi077:mds.j> sudo journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mds.j.service 2023-12-01T01:15:54.245 INFO:journalctl@ceph.mds.g.smithi077.stdout:-- Logs begin at Fri 2023-12-01 00:55:13 UTC. -- 2023-12-01T01:15:54.246 DEBUG:teuthology.orchestra.run.smithi137:mds.b> sudo journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mds.b.service 2023-12-01T01:15:54.249 DEBUG:teuthology.orchestra.run.smithi137:mds.e> sudo journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mds.e.service 2023-12-01T01:15:54.273 INFO:journalctl@ceph.mds.j.smithi077.stdout:-- Logs begin at Fri 2023-12-01 00:55:13 UTC. -- 2023-12-01T01:15:54.286 INFO:journalctl@ceph.mds.b.smithi137.stdout:-- Logs begin at Fri 2023-12-01 00:55:04 UTC. -- 2023-12-01T01:15:54.287 DEBUG:teuthology.orchestra.run.smithi137:mds.h> sudo journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mds.h.service 2023-12-01T01:15:54.316 INFO:journalctl@ceph.mds.e.smithi137.stdout:-- Logs begin at Fri 2023-12-01 00:55:04 UTC. -- 2023-12-01T01:15:54.317 DEBUG:teuthology.orchestra.run.smithi137:mds.k> sudo journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mds.k.service 2023-12-01T01:15:54.346 INFO:journalctl@ceph.mds.h.smithi137.stdout:-- Logs begin at Fri 2023-12-01 00:55:04 UTC. -- 2023-12-01T01:15:54.347 DEBUG:teuthology.orchestra.run.smithi203:mds.c> sudo journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mds.c.service 2023-12-01T01:15:54.349 DEBUG:teuthology.orchestra.run.smithi203:mds.f> sudo journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mds.f.service 2023-12-01T01:15:54.374 INFO:journalctl@ceph.mds.k.smithi137.stdout:-- Logs begin at Fri 2023-12-01 00:55:04 UTC. -- 2023-12-01T01:15:54.378 INFO:journalctl@ceph.mds.c.smithi203.stdout:-- Logs begin at Fri 2023-12-01 00:55:29 UTC. -- 2023-12-01T01:15:54.379 DEBUG:teuthology.orchestra.run.smithi203:mds.i> sudo journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mds.i.service 2023-12-01T01:15:54.405 INFO:journalctl@ceph.mds.f.smithi203.stdout:-- Logs begin at Fri 2023-12-01 00:55:29 UTC. -- 2023-12-01T01:15:54.406 DEBUG:teuthology.orchestra.run.smithi203:mds.l> sudo journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mds.l.service 2023-12-01T01:15:54.432 INFO:journalctl@ceph.mds.i.smithi203.stdout:-- Logs begin at Fri 2023-12-01 00:55:29 UTC. -- 2023-12-01T01:15:54.433 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-12-01T01:15:54.434 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:15:54.460 INFO:journalctl@ceph.mds.l.smithi203.stdout:-- Logs begin at Fri 2023-12-01 00:55:29 UTC. -- 2023-12-01T01:15:54.566 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.563+0000 7f5c62ef2700 1 -- 172.21.15.77:0/1289882626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c5c101f80 msgr2=0x7f5c5c1023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:54.567 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.563+0000 7f5c62ef2700 1 --2- 172.21.15.77:0/1289882626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c5c101f80 0x7f5c5c1023e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5c480099c0 tx=0x7f5c48009cd0 comp rx=0 tx=0).stop 2023-12-01T01:15:54.567 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.563+0000 7f5c62ef2700 1 -- 172.21.15.77:0/1289882626 shutdown_connections 2023-12-01T01:15:54.567 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.563+0000 7f5c62ef2700 1 --2- 172.21.15.77:0/1289882626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c5c102920 0x7f5c5c1091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:54.567 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.563+0000 7f5c62ef2700 1 --2- 172.21.15.77:0/1289882626 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c5c100d90 0x7f5c5c1011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:54.567 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.563+0000 7f5c62ef2700 1 --2- 172.21.15.77:0/1289882626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c5c101f80 0x7f5c5c1023e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:54.567 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.563+0000 7f5c62ef2700 1 -- 172.21.15.77:0/1289882626 >> 172.21.15.77:0/1289882626 conn(0x7f5c5c0fc2e0 msgr2=0x7f5c5c0fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:15:54.567 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.563+0000 7f5c62ef2700 1 -- 172.21.15.77:0/1289882626 shutdown_connections 2023-12-01T01:15:54.567 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.563+0000 7f5c62ef2700 1 -- 172.21.15.77:0/1289882626 wait complete. 2023-12-01T01:15:54.567 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.563+0000 7f5c62ef2700 1 Processor -- start 2023-12-01T01:15:54.567 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.563+0000 7f5c62ef2700 1 -- start start 2023-12-01T01:15:54.567 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.563+0000 7f5c62ef2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c5c100d90 0x7f5c5c196a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:54.567 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.563+0000 7f5c62ef2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c5c101f80 0x7f5c5c196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:54.568 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.563+0000 7f5c62ef2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c5c102920 0x7f5c5c19e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:54.568 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.563+0000 7f5c62ef2700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5c19e600 con 0x7f5c5c100d90 2023-12-01T01:15:54.568 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.563+0000 7f5c62ef2700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5c19e740 con 0x7f5c5c102920 2023-12-01T01:15:54.568 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.563+0000 7f5c62ef2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5c19e8d0 con 0x7f5c5c101f80 2023-12-01T01:15:54.568 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.563+0000 7f5c60c8e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c5c100d90 0x7f5c5c196a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:54.568 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.563+0000 7f5c60c8e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c5c100d90 0x7f5c5c196a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57128/0 (socket says 172.21.15.77:57128) 2023-12-01T01:15:54.568 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.564+0000 7f5c60c8e700 1 -- 172.21.15.77:0/933036411 learned_addr learned my addr 172.21.15.77:0/933036411 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:15:54.568 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.564+0000 7f5c5bfff700 1 --2- 172.21.15.77:0/933036411 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c5c101f80 0x7f5c5c196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:54.568 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.564+0000 7f5c6148f700 1 --2- 172.21.15.77:0/933036411 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c5c102920 0x7f5c5c19e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:54.568 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.564+0000 7f5c60c8e700 1 -- 172.21.15.77:0/933036411 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c5c101f80 msgr2=0x7f5c5c196fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:54.568 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.564+0000 7f5c60c8e700 1 --2- 172.21.15.77:0/933036411 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c5c101f80 0x7f5c5c196fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:54.568 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.564+0000 7f5c60c8e700 1 -- 172.21.15.77:0/933036411 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c5c102920 msgr2=0x7f5c5c19e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:54.569 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.564+0000 7f5c60c8e700 1 --2- 172.21.15.77:0/933036411 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c5c102920 0x7f5c5c19e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:54.569 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.564+0000 7f5c60c8e700 1 -- 172.21.15.77:0/933036411 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c480095f0 con 0x7f5c5c100d90 2023-12-01T01:15:54.569 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.564+0000 7f5c60c8e700 1 --2- 172.21.15.77:0/933036411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c5c100d90 0x7f5c5c196a80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5c5000e970 tx=0x7f5c5000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:15:54.569 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.564+0000 7f5c59ffb700 1 -- 172.21.15.77:0/933036411 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c50009790 con 0x7f5c5c100d90 2023-12-01T01:15:54.569 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.564+0000 7f5c59ffb700 1 -- 172.21.15.77:0/933036411 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c5000c7e0 con 0x7f5c5c100d90 2023-12-01T01:15:54.569 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.564+0000 7f5c62ef2700 1 -- 172.21.15.77:0/933036411 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c5c19ebb0 con 0x7f5c5c100d90 2023-12-01T01:15:54.569 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.564+0000 7f5c59ffb700 1 -- 172.21.15.77:0/933036411 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c50003c70 con 0x7f5c5c100d90 2023-12-01T01:15:54.569 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.564+0000 7f5c62ef2700 1 -- 172.21.15.77:0/933036411 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c5c19f0c0 con 0x7f5c5c100d90 2023-12-01T01:15:54.569 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.565+0000 7f5c62ef2700 1 -- 172.21.15.77:0/933036411 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c5c108330 con 0x7f5c5c100d90 2023-12-01T01:15:54.570 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.566+0000 7f5c59ffb700 1 -- 172.21.15.77:0/933036411 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f5c5000cc50 con 0x7f5c5c100d90 2023-12-01T01:15:54.570 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.567+0000 7f5c59ffb700 1 --2- 172.21.15.77:0/933036411 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f5c4c0a0570 0x7f5c4c0a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:54.570 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.567+0000 7f5c59ffb700 1 -- 172.21.15.77:0/933036411 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5c50014070 con 0x7f5c5c100d90 2023-12-01T01:15:54.570 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.567+0000 7f5c5bfff700 1 --2- 172.21.15.77:0/933036411 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f5c4c0a0570 0x7f5c4c0a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:54.570 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.567+0000 7f5c5bfff700 1 --2- 172.21.15.77:0/933036411 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f5c4c0a0570 0x7f5c4c0a2a30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5c4800b580 tx=0x7f5c48005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:15:54.572 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.571+0000 7f5c59ffb700 1 -- 172.21.15.77:0/933036411 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5c500956e0 con 0x7f5c5c100d90 2023-12-01T01:15:54.767 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.766+0000 7f5c62ef2700 1 -- 172.21.15.77:0/933036411 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c5c02ce70 con 0x7f5c5c100d90 2023-12-01T01:15:54.768 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.767+0000 7f5c59ffb700 1 -- 172.21.15.77:0/933036411 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+18434 (secure 0 0 0) 0x7f5c50003dd0 con 0x7f5c5c100d90 2023-12-01T01:15:54.769 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:15:54.769 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":78,"fsid":"2a06e29a-8fe6-11ee-95a2-87774f69a715","created":"2023-12-01T01:09:30.210398+0000","modified":"2023-12-01T01:15:46.072360+0000","last_up_change":"2023-12-01T01:15:44.586092+0000","last_in_change":"2023-12-01T01:15:26.305439+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:12:30.514229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fc651e62-a103-4ab8-a76d-d3708ea6aeec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6803","nonce":2244799094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6805","nonce":2244799094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6809","nonce":2244799094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6807","nonce":2244799094}]},"public_addr":"172.21.15.77:6803/2244799094","cluster_addr":"172.21.15.77:6805/2244799094","heartbeat_back_addr":"172.21.15.77:6809/2244799094","heartbeat_front_addr":"172.21.15.77:6807/2244799094","state":["exists","up"]},{"osd":1,"uuid":"0adf8c54-49ec-49a0-8bf1-af1f20e60729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6801","nonce":224194890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6803","nonce":224194890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6807","nonce":224194890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6805","nonce":224194890}]},"public_addr":"172.21.15.137:6801/224194890","cluster_addr":"172.21.15.137:6803/224194890","heartbeat_back_addr":"172.21.15.137:6807/224194890","heartbeat_front_addr":"172.21.15.137:6805/224194890","state":["exists","up"]},{"osd":2,"uuid":"5a69802c-dede-4544-9a12-436f5c1c0038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6801","nonce":4078459110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6803","nonce":4078459110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6807","nonce":4078459110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6805","nonce":4078459110}]},"public_addr":"172.21.15.203:6801/4078459110","cluster_addr":"172.21.15.203:6803/4078459110","heartbeat_back_addr":"172.21.15.203:6807/4078459110","heartbeat_front_addr":"172.21.15.203:6805/4078459110","state":["exists","up"]},{"osd":3,"uuid":"ad325a97-580b-4c99-bf12-cbb0d92a6d0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6811","nonce":1454440477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6813","nonce":1454440477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6817","nonce":1454440477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6815","nonce":1454440477}]},"public_addr":"172.21.15.77:6811/1454440477","cluster_addr":"172.21.15.77:6813/1454440477","heartbeat_back_addr":"172.21.15.77:6817/1454440477","heartbeat_front_addr":"172.21.15.77:6815/1454440477","state":["exists","up"]},{"osd":4,"uuid":"1e3ac1ba-98bb-4b8b-8e26-50064c66bf2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6809","nonce":1103241451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6811","nonce":1103241451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6815","nonce":1103241451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6813","nonce":1103241451}]},"public_addr":"172.21.15.137:6809/1103241451","cluster_addr":"172.21.15.137:6811/1103241451","heartbeat_back_addr":"172.21.15.137:6815/1103241451","heartbeat_front_addr":"172.21.15.137:6813/1103241451","state":["exists","up"]},{"osd":5,"uuid":"eb32c016-abeb-44ab-abf0-19dd09235736","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6809","nonce":2648569482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6811","nonce":2648569482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6815","nonce":2648569482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6813","nonce":2648569482}]},"public_addr":"172.21.15.203:6809/2648569482","cluster_addr":"172.21.15.203:6811/2648569482","heartbeat_back_addr":"172.21.15.203:6815/2648569482","heartbeat_front_addr":"172.21.15.203:6813/2648569482","state":["exists","up"]},{"osd":6,"uuid":"0e3fa97e-2ca6-418c-9e8d-a6fe3c56adb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6819","nonce":2463414651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6821","nonce":2463414651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6825","nonce":2463414651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6823","nonce":2463414651}]},"public_addr":"172.21.15.77:6819/2463414651","cluster_addr":"172.21.15.77:6821/2463414651","heartbeat_back_addr":"172.21.15.77:6825/2463414651","heartbeat_front_addr":"172.21.15.77:6823/2463414651","state":["exists","up"]},{"osd":7,"uuid":"d7398c01-0f98-4138-8a49-c3ff0aae7e88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6817","nonce":3108330766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6819","nonce":3108330766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6823","nonce":3108330766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6821","nonce":3108330766}]},"public_addr":"172.21.15.137:6817/3108330766","cluster_addr":"172.21.15.137:6819/3108330766","heartbeat_back_addr":"172.21.15.137:6823/3108330766","heartbeat_front_addr":"172.21.15.137:6821/3108330766","state":["exists","up"]},{"osd":8,"uuid":"0c09050d-3e43-410c-8cd7-bcb891af56b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6817","nonce":1161158776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6819","nonce":1161158776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6823","nonce":1161158776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6821","nonce":1161158776}]},"public_addr":"172.21.15.203:6817/1161158776","cluster_addr":"172.21.15.203:6819/1161158776","heartbeat_back_addr":"172.21.15.203:6823/1161158776","heartbeat_front_addr":"172.21.15.203:6821/1161158776","state":["exists","up"]},{"osd":9,"uuid":"527ecb49-95ec-48bd-9872-3b07fae5c5ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6827","nonce":2409880685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6829","nonce":2409880685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6833","nonce":2409880685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6831","nonce":2409880685}]},"public_addr":"172.21.15.77:6827/2409880685","cluster_addr":"172.21.15.77:6829/2409880685","heartbeat_back_addr":"172.21.15.77:6833/2409880685","heartbeat_front_addr":"172.21.15.77:6831/2409880685","state":["exists","up"]},{"osd":10,"uuid":"1680bb3e-a3c8-427b-af68-e2214f32bed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6825","nonce":374920146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6827","nonce":374920146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6831","nonce":374920146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6829","nonce":374920146}]},"public_addr":"172.21.15.137:6825/374920146","cluster_addr":"172.21.15.137:6827/374920146","heartbeat_back_addr":"172.21.15.137:6831/374920146","heartbeat_front_addr":"172.21.15.137:6829/374920146","state":["exists","up"]},{"osd":11,"uuid":"8a533466-023d-4803-ae36-9a9f1715e49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6825","nonce":2161588962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6827","nonce":2161588962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6831","nonce":2161588962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6829","nonce":2161588962}]},"public_addr":"172.21.15.203:6825/2161588962","cluster_addr":"172.21.15.203:6827/2161588962","heartbeat_back_addr":"172.21.15.203:6831/2161588962","heartbeat_front_addr":"172.21.15.203:6829/2161588962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:11:45.420509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:06.210115+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:28.191307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:49.594116+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:11.261689+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:33.376541+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:54.968336+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:16.775310+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:38.733792+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:59.456699+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:15:20.924059+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:15:43.094773+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/2260035254":"2023-12-02T01:10:24.191835+0000","172.21.15.77:6800/1456621747":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/219838828":"2023-12-02T01:10:24.191835+0000","172.21.15.77:6801/1456621747":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/2937149266":"2023-12-02T01:10:02.232948+0000","172.21.15.77:0/807520018":"2023-12-02T01:10:02.232948+0000","172.21.15.77:6801/1661788026":"2023-12-02T01:10:02.232948+0000","172.21.15.77:6800/1661788026":"2023-12-02T01:10:02.232948+0000","172.21.15.77:0/3551282473":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/1202222558":"2023-12-02T01:10:02.232948+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:15:54.771 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.769+0000 7f5c62ef2700 1 -- 172.21.15.77:0/933036411 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f5c4c0a0570 msgr2=0x7f5c4c0a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:54.771 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.769+0000 7f5c62ef2700 1 --2- 172.21.15.77:0/933036411 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f5c4c0a0570 0x7f5c4c0a2a30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5c4800b580 tx=0x7f5c48005fb0 comp rx=0 tx=0).stop 2023-12-01T01:15:54.771 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.769+0000 7f5c62ef2700 1 -- 172.21.15.77:0/933036411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c5c100d90 msgr2=0x7f5c5c196a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:54.771 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.769+0000 7f5c62ef2700 1 --2- 172.21.15.77:0/933036411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c5c100d90 0x7f5c5c196a80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5c5000e970 tx=0x7f5c5000ec80 comp rx=0 tx=0).stop 2023-12-01T01:15:54.771 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.769+0000 7f5c62ef2700 1 -- 172.21.15.77:0/933036411 shutdown_connections 2023-12-01T01:15:54.771 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.769+0000 7f5c62ef2700 1 --2- 172.21.15.77:0/933036411 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f5c4c0a0570 0x7f5c4c0a2a30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:54.771 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.769+0000 7f5c62ef2700 1 --2- 172.21.15.77:0/933036411 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c5c102920 0x7f5c5c19e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:54.771 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.769+0000 7f5c62ef2700 1 --2- 172.21.15.77:0/933036411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c5c100d90 0x7f5c5c196a80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:54.771 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.769+0000 7f5c62ef2700 1 --2- 172.21.15.77:0/933036411 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c5c101f80 0x7f5c5c196fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:54.772 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.769+0000 7f5c62ef2700 1 -- 172.21.15.77:0/933036411 >> 172.21.15.77:0/933036411 conn(0x7f5c5c0fc2e0 msgr2=0x7f5c5c105470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:15:54.772 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.769+0000 7f5c62ef2700 1 -- 172.21.15.77:0/933036411 shutdown_connections 2023-12-01T01:15:54.772 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.769+0000 7f5c62ef2700 1 -- 172.21.15.77:0/933036411 wait complete. 2023-12-01T01:15:54.782 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:54 smithi077 ceph-mon[98867]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi077=a;smithi077=d;smithi077=g;smithi077=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:15:54.782 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:54 smithi077 ceph-mon[98867]: Saving service mds.all spec with placement smithi077=a;smithi077=d;smithi077=g;smithi077=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l;count:12 2023-12-01T01:15:54.782 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:54 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:54.782 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:54 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:15:54.782 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:54 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:15:54.783 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:54 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:15:54.783 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:54 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:54.783 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:54 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-01T01:15:54.783 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:54 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-01T01:15:54.783 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:54 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:15:54.783 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:54 smithi077 ceph-mon[98867]: Deploying daemon mds.l on smithi203 2023-12-01T01:15:54.783 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:54 smithi077 ceph-mon[98867]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:15:54.784 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:12:30.514229+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:15:54.785 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:15:54.912 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.907+0000 7fd35dc17700 1 -- 172.21.15.77:0/1391878712 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd358101830 msgr2=0x7fd358103c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:54.913 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.907+0000 7fd35dc17700 1 --2- 172.21.15.77:0/1391878712 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd358101830 0x7fd358103c20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd34c009990 tx=0x7fd34c009ca0 comp rx=0 tx=0).stop 2023-12-01T01:15:54.913 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.908+0000 7fd35dc17700 1 -- 172.21.15.77:0/1391878712 shutdown_connections 2023-12-01T01:15:54.913 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.908+0000 7fd35dc17700 1 --2- 172.21.15.77:0/1391878712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd358106a90 0x7fd358108f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:54.913 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.908+0000 7fd35dc17700 1 --2- 172.21.15.77:0/1391878712 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd358101830 0x7fd358103c20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:54.913 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.908+0000 7fd35dc17700 1 --2- 172.21.15.77:0/1391878712 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd358104160 0x7fd358106550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:54.913 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.908+0000 7fd35dc17700 1 -- 172.21.15.77:0/1391878712 >> 172.21.15.77:0/1391878712 conn(0x7fd3580fb220 msgr2=0x7fd3580fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:15:54.913 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.908+0000 7fd35dc17700 1 -- 172.21.15.77:0/1391878712 shutdown_connections 2023-12-01T01:15:54.913 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.908+0000 7fd35dc17700 1 -- 172.21.15.77:0/1391878712 wait complete. 2023-12-01T01:15:54.913 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.908+0000 7fd35dc17700 1 Processor -- start 2023-12-01T01:15:54.913 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.908+0000 7fd35dc17700 1 -- start start 2023-12-01T01:15:54.913 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.908+0000 7fd35dc17700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd358101830 0x7fd358196ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:54.913 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.908+0000 7fd35dc17700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd358104160 0x7fd358196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:54.913 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.908+0000 7fd35dc17700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd358106a90 0x7fd35819e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:54.914 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.909+0000 7fd35dc17700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35819e630 con 0x7fd358104160 2023-12-01T01:15:54.914 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.909+0000 7fd35dc17700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35819e770 con 0x7fd358101830 2023-12-01T01:15:54.914 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.909+0000 7fd35dc17700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35819e900 con 0x7fd358106a90 2023-12-01T01:15:54.914 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.909+0000 7fd356ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd358104160 0x7fd358196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:54.914 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.909+0000 7fd356ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd358104160 0x7fd358196ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57144/0 (socket says 172.21.15.77:57144) 2023-12-01T01:15:54.914 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.909+0000 7fd356ffd700 1 -- 172.21.15.77:0/2846509593 learned_addr learned my addr 172.21.15.77:0/2846509593 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:15:54.914 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.909+0000 7fd3577fe700 1 --2- 172.21.15.77:0/2846509593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd358101830 0x7fd358196ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:54.914 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.909+0000 7fd356ffd700 1 -- 172.21.15.77:0/2846509593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd358106a90 msgr2=0x7fd35819e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:54.914 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.909+0000 7fd357fff700 1 --2- 172.21.15.77:0/2846509593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd358106a90 0x7fd35819e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:54.914 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.909+0000 7fd356ffd700 1 --2- 172.21.15.77:0/2846509593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd358106a90 0x7fd35819e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:54.914 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.909+0000 7fd356ffd700 1 -- 172.21.15.77:0/2846509593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd358101830 msgr2=0x7fd358196ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:54.914 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.909+0000 7fd356ffd700 1 --2- 172.21.15.77:0/2846509593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd358101830 0x7fd358196ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:54.914 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.909+0000 7fd356ffd700 1 -- 172.21.15.77:0/2846509593 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd34c0095f0 con 0x7fd358104160 2023-12-01T01:15:54.914 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.909+0000 7fd3577fe700 1 --2- 172.21.15.77:0/2846509593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd358101830 0x7fd358196ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:15:54.915 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.909+0000 7fd356ffd700 1 --2- 172.21.15.77:0/2846509593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd358104160 0x7fd358196ff0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd34800d980 tx=0x7fd34800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:15:54.915 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.909+0000 7fd354ff9700 1 -- 172.21.15.77:0/2846509593 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd348009b40 con 0x7fd358104160 2023-12-01T01:15:54.915 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.909+0000 7fd354ff9700 1 -- 172.21.15.77:0/2846509593 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd34800de60 con 0x7fd358104160 2023-12-01T01:15:54.915 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.909+0000 7fd35dc17700 1 -- 172.21.15.77:0/2846509593 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd35819ebe0 con 0x7fd358104160 2023-12-01T01:15:54.915 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.909+0000 7fd354ff9700 1 -- 172.21.15.77:0/2846509593 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd348005580 con 0x7fd358104160 2023-12-01T01:15:54.915 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.909+0000 7fd35dc17700 1 -- 172.21.15.77:0/2846509593 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd35819f0a0 con 0x7fd358104160 2023-12-01T01:15:54.915 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.910+0000 7fd35dc17700 1 -- 172.21.15.77:0/2846509593 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd35810c5b0 con 0x7fd358104160 2023-12-01T01:15:54.916 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.911+0000 7fd354ff9700 1 -- 172.21.15.77:0/2846509593 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fd348004750 con 0x7fd358104160 2023-12-01T01:15:54.916 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.912+0000 7fd354ff9700 1 --2- 172.21.15.77:0/2846509593 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd3400a05a0 0x7fd3400a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:54.916 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.912+0000 7fd3577fe700 1 --2- 172.21.15.77:0/2846509593 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd3400a05a0 0x7fd3400a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:54.916 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.912+0000 7fd354ff9700 1 -- 172.21.15.77:0/2846509593 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd3480cb590 con 0x7fd358104160 2023-12-01T01:15:54.916 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.912+0000 7fd3577fe700 1 --2- 172.21.15.77:0/2846509593 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd3400a05a0 0x7fd3400a2a60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd34c00b5c0 tx=0x7fd34c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:15:54.917 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:54.916+0000 7fd354ff9700 1 -- 172.21.15.77:0/2846509593 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd348095530 con 0x7fd358104160 2023-12-01T01:15:55.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:54 smithi203 ceph-mon[103581]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi077=a;smithi077=d;smithi077=g;smithi077=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:15:55.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:54 smithi203 ceph-mon[103581]: Saving service mds.all spec with placement smithi077=a;smithi077=d;smithi077=g;smithi077=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l;count:12 2023-12-01T01:15:55.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:54 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:55.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:54 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:15:55.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:54 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:15:55.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:54 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:15:55.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:54 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:55.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:54 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-01T01:15:55.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:54 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-01T01:15:55.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:54 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:15:55.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:54 smithi203 ceph-mon[103581]: Deploying daemon mds.l on smithi203 2023-12-01T01:15:55.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:54 smithi203 ceph-mon[103581]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:15:55.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:54 smithi137 ceph-mon[103312]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi077=a;smithi077=d;smithi077=g;smithi077=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:15:55.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:54 smithi137 ceph-mon[103312]: Saving service mds.all spec with placement smithi077=a;smithi077=d;smithi077=g;smithi077=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l;count:12 2023-12-01T01:15:55.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:54 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:55.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:54 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:15:55.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:54 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:15:55.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:54 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:15:55.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:54 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:55.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:54 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-01T01:15:55.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:54 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-01T01:15:55.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:54 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:15:55.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:54 smithi137 ceph-mon[103312]: Deploying daemon mds.l on smithi203 2023-12-01T01:15:55.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:54 smithi137 ceph-mon[103312]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:15:55.106 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.105+0000 7fd35dc17700 1 -- 172.21.15.77:0/2846509593 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd35819f9e0 con 0x7fd358104160 2023-12-01T01:15:55.106 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.105+0000 7fd354ff9700 1 -- 172.21.15.77:0/2846509593 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd348098b50 con 0x7fd358104160 2023-12-01T01:15:55.106 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 1 2023-12-01T01:15:55.108 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.107+0000 7fd35dc17700 1 -- 172.21.15.77:0/2846509593 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd3400a05a0 msgr2=0x7fd3400a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:55.109 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.107+0000 7fd35dc17700 1 --2- 172.21.15.77:0/2846509593 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd3400a05a0 0x7fd3400a2a60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd34c00b5c0 tx=0x7fd34c005fb0 comp rx=0 tx=0).stop 2023-12-01T01:15:55.109 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.107+0000 7fd35dc17700 1 -- 172.21.15.77:0/2846509593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd358104160 msgr2=0x7fd358196ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:55.109 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.107+0000 7fd35dc17700 1 --2- 172.21.15.77:0/2846509593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd358104160 0x7fd358196ff0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd34800d980 tx=0x7fd34800dd40 comp rx=0 tx=0).stop 2023-12-01T01:15:55.109 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.107+0000 7fd35dc17700 1 -- 172.21.15.77:0/2846509593 shutdown_connections 2023-12-01T01:15:55.109 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.107+0000 7fd35dc17700 1 --2- 172.21.15.77:0/2846509593 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd3400a05a0 0x7fd3400a2a60 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:55.109 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.107+0000 7fd35dc17700 1 --2- 172.21.15.77:0/2846509593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd358106a90 0x7fd35819e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:55.109 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.107+0000 7fd35dc17700 1 --2- 172.21.15.77:0/2846509593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd358101830 0x7fd358196ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:55.109 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.107+0000 7fd35dc17700 1 --2- 172.21.15.77:0/2846509593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd358104160 0x7fd358196ff0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:55.109 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.107+0000 7fd35dc17700 1 -- 172.21.15.77:0/2846509593 >> 172.21.15.77:0/2846509593 conn(0x7fd3580fb220 msgr2=0x7fd358102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:15:55.109 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.107+0000 7fd35dc17700 1 -- 172.21.15.77:0/2846509593 shutdown_connections 2023-12-01T01:15:55.109 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.107+0000 7fd35dc17700 1 -- 172.21.15.77:0/2846509593 wait complete. 2023-12-01T01:15:55.121 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:15:55.244 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.241+0000 7f0687bc1700 1 -- 172.21.15.77:0/3845287174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06801008e0 msgr2=0x7f06801081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:55.244 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.241+0000 7f0687bc1700 1 --2- 172.21.15.77:0/3845287174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06801008e0 0x7f06801081a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f067800b0a0 tx=0x7f067800b3b0 comp rx=0 tx=0).stop 2023-12-01T01:15:55.244 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.241+0000 7f0687bc1700 1 -- 172.21.15.77:0/3845287174 shutdown_connections 2023-12-01T01:15:55.244 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.241+0000 7f0687bc1700 1 --2- 172.21.15.77:0/3845287174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06801008e0 0x7f06801081a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:55.245 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.241+0000 7f0687bc1700 1 --2- 172.21.15.77:0/3845287174 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0680101210 0x7f0680101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:55.245 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.241+0000 7f0687bc1700 1 --2- 172.21.15.77:0/3845287174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06800fff40 0x7f06801003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:55.245 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.241+0000 7f0687bc1700 1 -- 172.21.15.77:0/3845287174 >> 172.21.15.77:0/3845287174 conn(0x7f06800fa950 msgr2=0x7f06800fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:15:55.245 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.241+0000 7f0687bc1700 1 -- 172.21.15.77:0/3845287174 shutdown_connections 2023-12-01T01:15:55.245 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.241+0000 7f0687bc1700 1 -- 172.21.15.77:0/3845287174 wait complete. 2023-12-01T01:15:55.245 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.241+0000 7f0687bc1700 1 Processor -- start 2023-12-01T01:15:55.245 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.241+0000 7f0687bc1700 1 -- start start 2023-12-01T01:15:55.245 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.242+0000 7f0687bc1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06800fff40 0x7f0680103cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:55.245 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.242+0000 7f0687bc1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06801008e0 0x7f0680102330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:55.245 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.242+0000 7f0687bc1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0680101210 0x7f0680102870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:55.245 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.242+0000 7f0687bc1700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0680103010 con 0x7f06800fff40 2023-12-01T01:15:55.245 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.242+0000 7f0687bc1700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0680103180 con 0x7f0680101210 2023-12-01T01:15:55.245 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.242+0000 7f0687bc1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06801a0c00 con 0x7f06801008e0 2023-12-01T01:15:55.245 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.242+0000 7f068595d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06800fff40 0x7f0680103cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:55.246 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.242+0000 7f068595d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06800fff40 0x7f0680103cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57156/0 (socket says 172.21.15.77:57156) 2023-12-01T01:15:55.246 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.242+0000 7f068595d700 1 -- 172.21.15.77:0/3669231287 learned_addr learned my addr 172.21.15.77:0/3669231287 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:15:55.246 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.242+0000 7f068615e700 1 --2- 172.21.15.77:0/3669231287 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0680101210 0x7f0680102870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:55.246 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.242+0000 7f068515c700 1 --2- 172.21.15.77:0/3669231287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06801008e0 0x7f0680102330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:55.246 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.242+0000 7f068595d700 1 -- 172.21.15.77:0/3669231287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06801008e0 msgr2=0x7f0680102330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:55.246 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.242+0000 7f068595d700 1 --2- 172.21.15.77:0/3669231287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06801008e0 0x7f0680102330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:55.246 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.242+0000 7f068595d700 1 -- 172.21.15.77:0/3669231287 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0680101210 msgr2=0x7f0680102870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:55.246 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.242+0000 7f068595d700 1 --2- 172.21.15.77:0/3669231287 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0680101210 0x7f0680102870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:55.246 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.242+0000 7f068595d700 1 -- 172.21.15.77:0/3669231287 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0678009d00 con 0x7f06800fff40 2023-12-01T01:15:55.246 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.242+0000 7f068595d700 1 --2- 172.21.15.77:0/3669231287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06800fff40 0x7f0680103cf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f067000b500 tx=0x7f067000b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:15:55.246 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.242+0000 7f0676ffd700 1 -- 172.21.15.77:0/3669231287 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0670011830 con 0x7f06800fff40 2023-12-01T01:15:55.246 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.242+0000 7f0676ffd700 1 -- 172.21.15.77:0/3669231287 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0670011e50 con 0x7f06800fff40 2023-12-01T01:15:55.246 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.242+0000 7f0687bc1700 1 -- 172.21.15.77:0/3669231287 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06801a0ee0 con 0x7f06800fff40 2023-12-01T01:15:55.246 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.242+0000 7f0676ffd700 1 -- 172.21.15.77:0/3669231287 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0670010dc0 con 0x7f06800fff40 2023-12-01T01:15:55.247 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.243+0000 7f0687bc1700 1 -- 172.21.15.77:0/3669231287 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06801a1380 con 0x7f06800fff40 2023-12-01T01:15:55.247 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.244+0000 7f0687bc1700 1 -- 172.21.15.77:0/3669231287 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f068010b580 con 0x7f06800fff40 2023-12-01T01:15:55.248 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.245+0000 7f0676ffd700 1 -- 172.21.15.77:0/3669231287 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f0670010380 con 0x7f06800fff40 2023-12-01T01:15:55.248 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.245+0000 7f0676ffd700 1 --2- 172.21.15.77:0/3669231287 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f066c0a0570 0x7f066c0a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:55.248 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.245+0000 7f068515c700 1 --2- 172.21.15.77:0/3669231287 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f066c0a0570 0x7f066c0a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:55.248 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.246+0000 7f0676ffd700 1 -- 172.21.15.77:0/3669231287 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f06700caf20 con 0x7f06800fff40 2023-12-01T01:15:55.248 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.246+0000 7f068515c700 1 --2- 172.21.15.77:0/3669231287 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f066c0a0570 0x7f066c0a2a30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f067c009c60 tx=0x7f067c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:15:55.251 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.250+0000 7f0676ffd700 1 -- 172.21.15.77:0/3669231287 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0670094f60 con 0x7f06800fff40 2023-12-01T01:15:55.393 INFO:journalctl@ceph.mds.l.smithi203.stdout:Dec 01 01:15:55 smithi203 systemd[1]: Starting Ceph mds.l for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:15:55.486 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.484+0000 7f0687bc1700 1 -- 172.21.15.77:0/3669231287 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f068010b740 con 0x7f06800fff40 2023-12-01T01:15:55.486 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.485+0000 7f0676ffd700 1 -- 172.21.15.77:0/3669231287 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7f0670098580 con 0x7f06800fff40 2023-12-01T01:15:55.486 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:15:55.486 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-12-01T01:15:55.489 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.487+0000 7f0687bc1700 1 -- 172.21.15.77:0/3669231287 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f066c0a0570 msgr2=0x7f066c0a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:55.489 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.487+0000 7f0687bc1700 1 --2- 172.21.15.77:0/3669231287 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f066c0a0570 0x7f066c0a2a30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f067c009c60 tx=0x7f067c009260 comp rx=0 tx=0).stop 2023-12-01T01:15:55.489 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.487+0000 7f0687bc1700 1 -- 172.21.15.77:0/3669231287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06800fff40 msgr2=0x7f0680103cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:55.489 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.487+0000 7f0687bc1700 1 --2- 172.21.15.77:0/3669231287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06800fff40 0x7f0680103cf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f067000b500 tx=0x7f067000b810 comp rx=0 tx=0).stop 2023-12-01T01:15:55.489 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.487+0000 7f0687bc1700 1 -- 172.21.15.77:0/3669231287 shutdown_connections 2023-12-01T01:15:55.489 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.487+0000 7f0687bc1700 1 --2- 172.21.15.77:0/3669231287 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f066c0a0570 0x7f066c0a2a30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:55.489 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.487+0000 7f0687bc1700 1 --2- 172.21.15.77:0/3669231287 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0680101210 0x7f0680102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:55.489 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.487+0000 7f0687bc1700 1 --2- 172.21.15.77:0/3669231287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06800fff40 0x7f0680103cf0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:55.489 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.487+0000 7f0687bc1700 1 --2- 172.21.15.77:0/3669231287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06801008e0 0x7f0680102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:55.489 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.487+0000 7f0687bc1700 1 -- 172.21.15.77:0/3669231287 >> 172.21.15.77:0/3669231287 conn(0x7f06800fa950 msgr2=0x7f06800fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:15:55.489 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.487+0000 7f0687bc1700 1 -- 172.21.15.77:0/3669231287 shutdown_connections 2023-12-01T01:15:55.489 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:55.487+0000 7f0687bc1700 1 -- 172.21.15.77:0/3669231287 wait complete. 2023-12-01T01:15:55.490 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 1 2023-12-01T01:15:55.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:55 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/933036411' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:15:55.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:55 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/2846509593' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-01T01:15:55.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:55 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/3669231287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:15:55.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:55 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/933036411' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:15:55.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:55 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/2846509593' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-01T01:15:55.989 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:55 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/3669231287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:15:55.989 INFO:journalctl@ceph.mds.l.smithi203.stdout:Dec 01 01:15:55 smithi203 podman[140354]: 2023-12-01T01:15:55.989 INFO:journalctl@ceph.mds.l.smithi203.stdout:Dec 01 01:15:55 smithi203 ceph-mds[140394]: -- 172.21.15.203:0/1574518924 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ff4ed531e0 con 0x55ff4ed29000 2023-12-01T01:15:56.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:55 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/933036411' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:15:56.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:55 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/2846509593' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-01T01:15:56.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:55 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/3669231287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:15:56.262 INFO:journalctl@ceph.mds.l.smithi203.stdout:Dec 01 01:15:55 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-l[140370]: starting mds.l at 2023-12-01T01:15:56.262 INFO:journalctl@ceph.mds.l.smithi203.stdout:Dec 01 01:15:55 smithi203 bash[140354]: c554faf3c27af77ef5064f7aae7ffede188118bed4d04b5a040e5c1e2684c104 2023-12-01T01:15:56.262 INFO:journalctl@ceph.mds.l.smithi203.stdout:Dec 01 01:15:56 smithi203 systemd[1]: Started Ceph mds.l for 2a06e29a-8fe6-11ee-95a2-87774f69a715. 2023-12-01T01:15:57.089 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:56 smithi137 ceph-mon[103312]: mds.? [v2:172.21.15.203:6832/1201599774,v1:172.21.15.203:6833/1201599774] up:boot 2023-12-01T01:15:57.090 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:56 smithi137 ceph-mon[103312]: fsmap 1 up:standby 2023-12-01T01:15:57.090 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:56 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-01T01:15:57.090 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:56 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:57.090 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:56 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:57.090 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:56 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:57.090 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:56 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-01T01:15:57.090 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:56 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-01T01:15:57.090 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:56 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:15:57.090 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:56 smithi137 ceph-mon[103312]: Deploying daemon mds.k on smithi137 2023-12-01T01:15:57.090 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:56 smithi137 ceph-mon[103312]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:15:57.262 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:56 smithi203 ceph-mon[103581]: mds.? [v2:172.21.15.203:6832/1201599774,v1:172.21.15.203:6833/1201599774] up:boot 2023-12-01T01:15:57.262 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:56 smithi203 ceph-mon[103581]: fsmap 1 up:standby 2023-12-01T01:15:57.262 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:56 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-01T01:15:57.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:56 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:57.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:56 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:57.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:56 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:57.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:56 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-01T01:15:57.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:56 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-01T01:15:57.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:56 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:15:57.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:56 smithi203 ceph-mon[103581]: Deploying daemon mds.k on smithi137 2023-12-01T01:15:57.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:56 smithi203 ceph-mon[103581]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:15:57.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:56 smithi077 ceph-mon[98867]: mds.? [v2:172.21.15.203:6832/1201599774,v1:172.21.15.203:6833/1201599774] up:boot 2023-12-01T01:15:57.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:56 smithi077 ceph-mon[98867]: fsmap 1 up:standby 2023-12-01T01:15:57.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:56 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-01T01:15:57.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:56 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:57.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:56 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:57.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:56 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:57.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:56 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-01T01:15:57.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:56 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-01T01:15:57.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:56 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:15:57.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:56 smithi077 ceph-mon[98867]: Deploying daemon mds.k on smithi137 2023-12-01T01:15:57.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:56 smithi077 ceph-mon[98867]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:15:57.501 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:15:57.637 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.634+0000 7fd695a96700 1 -- 172.21.15.77:0/3566744746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd690102930 msgr2=0x7fd6901091e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:57.637 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.634+0000 7fd695a96700 1 --2- 172.21.15.77:0/3566744746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd690102930 0x7fd6901091e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd68800b0d0 tx=0x7fd68800b3e0 comp rx=0 tx=0).stop 2023-12-01T01:15:57.637 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.634+0000 7fd695a96700 1 -- 172.21.15.77:0/3566744746 shutdown_connections 2023-12-01T01:15:57.637 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.634+0000 7fd695a96700 1 --2- 172.21.15.77:0/3566744746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd690102930 0x7fd6901091e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:57.638 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.634+0000 7fd695a96700 1 --2- 172.21.15.77:0/3566744746 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd690100da0 0x7fd6901011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:57.638 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.634+0000 7fd695a96700 1 --2- 172.21.15.77:0/3566744746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd690101f90 0x7fd6901023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:57.638 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.634+0000 7fd695a96700 1 -- 172.21.15.77:0/3566744746 >> 172.21.15.77:0/3566744746 conn(0x7fd6900fc2d0 msgr2=0x7fd6900fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:15:57.638 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.634+0000 7fd695a96700 1 -- 172.21.15.77:0/3566744746 shutdown_connections 2023-12-01T01:15:57.638 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.634+0000 7fd695a96700 1 -- 172.21.15.77:0/3566744746 wait complete. 2023-12-01T01:15:57.638 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.634+0000 7fd695a96700 1 Processor -- start 2023-12-01T01:15:57.638 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.635+0000 7fd695a96700 1 -- start start 2023-12-01T01:15:57.638 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.635+0000 7fd695a96700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd690100da0 0x7fd690196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:57.638 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.635+0000 7fd695a96700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd690101f90 0x7fd690196fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:57.638 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.635+0000 7fd695a96700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd690102930 0x7fd69019e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:57.638 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.635+0000 7fd695a96700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69019e5e0 con 0x7fd690100da0 2023-12-01T01:15:57.638 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.635+0000 7fd695a96700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69019e720 con 0x7fd690102930 2023-12-01T01:15:57.638 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.635+0000 7fd695a96700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69019e8b0 con 0x7fd690101f90 2023-12-01T01:15:57.639 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.635+0000 7fd68f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd690100da0 0x7fd690196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:57.639 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.635+0000 7fd68f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd690100da0 0x7fd690196a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57170/0 (socket says 172.21.15.77:57170) 2023-12-01T01:15:57.639 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.635+0000 7fd68f7fe700 1 -- 172.21.15.77:0/3649722969 learned_addr learned my addr 172.21.15.77:0/3649722969 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:15:57.639 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.635+0000 7fd68ffff700 1 --2- 172.21.15.77:0/3649722969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd690102930 0x7fd69019e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:57.639 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.635+0000 7fd68effd700 1 --2- 172.21.15.77:0/3649722969 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd690101f90 0x7fd690196fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:57.639 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.635+0000 7fd68f7fe700 1 -- 172.21.15.77:0/3649722969 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd690101f90 msgr2=0x7fd690196fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:57.639 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.635+0000 7fd68f7fe700 1 --2- 172.21.15.77:0/3649722969 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd690101f90 0x7fd690196fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:57.639 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.635+0000 7fd68f7fe700 1 -- 172.21.15.77:0/3649722969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd690102930 msgr2=0x7fd69019e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:57.639 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.635+0000 7fd68f7fe700 1 --2- 172.21.15.77:0/3649722969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd690102930 0x7fd69019e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:57.639 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.635+0000 7fd68f7fe700 1 -- 172.21.15.77:0/3649722969 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd688009d00 con 0x7fd690100da0 2023-12-01T01:15:57.639 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.635+0000 7fd68f7fe700 1 --2- 172.21.15.77:0/3649722969 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd690100da0 0x7fd690196a60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd68400c990 tx=0x7fd68400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:15:57.639 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.635+0000 7fd68cff9700 1 -- 172.21.15.77:0/3649722969 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd684007b00 con 0x7fd690100da0 2023-12-01T01:15:57.639 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.636+0000 7fd68cff9700 1 -- 172.21.15.77:0/3649722969 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd684007c60 con 0x7fd690100da0 2023-12-01T01:15:57.640 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.636+0000 7fd695a96700 1 -- 172.21.15.77:0/3649722969 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd69019eb90 con 0x7fd690100da0 2023-12-01T01:15:57.640 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.636+0000 7fd68cff9700 1 -- 172.21.15.77:0/3649722969 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd684018660 con 0x7fd690100da0 2023-12-01T01:15:57.640 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.636+0000 7fd695a96700 1 -- 172.21.15.77:0/3649722969 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd69019f0a0 con 0x7fd690100da0 2023-12-01T01:15:57.640 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.636+0000 7fd695a96700 1 -- 172.21.15.77:0/3649722969 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd69010c5c0 con 0x7fd690100da0 2023-12-01T01:15:57.641 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.637+0000 7fd68cff9700 1 -- 172.21.15.77:0/3649722969 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fd68400f450 con 0x7fd690100da0 2023-12-01T01:15:57.641 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.638+0000 7fd68cff9700 1 --2- 172.21.15.77:0/3649722969 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd6780a0570 0x7fd6780a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:15:57.641 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.638+0000 7fd68effd700 1 --2- 172.21.15.77:0/3649722969 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd6780a0570 0x7fd6780a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:15:57.641 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.638+0000 7fd68cff9700 1 -- 172.21.15.77:0/3649722969 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd6840cb4f0 con 0x7fd690100da0 2023-12-01T01:15:57.641 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.638+0000 7fd68effd700 1 --2- 172.21.15.77:0/3649722969 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd6780a0570 0x7fd6780a2a30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd680009b30 tx=0x7fd680009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:15:57.643 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.642+0000 7fd68cff9700 1 -- 172.21.15.77:0/3649722969 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd6840955c0 con 0x7fd690100da0 2023-12-01T01:15:57.779 INFO:journalctl@ceph.mds.k.smithi137.stdout:Dec 01 01:15:57 smithi137 systemd[1]: Starting Ceph mds.k for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:15:57.878 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.877+0000 7fd695a96700 1 -- 172.21.15.77:0/3649722969 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd69010c7e0 con 0x7fd690100da0 2023-12-01T01:15:57.879 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.878+0000 7fd68cff9700 1 -- 172.21.15.77:0/3649722969 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1260 (secure 0 0 0) 0x7fd684098be0 con 0x7fd690100da0 2023-12-01T01:15:57.879 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:15:57.879 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/1201599774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1201599774},{"type":"v1","addr":"172.21.15.203:6833","nonce":1201599774}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-12-01T01:15:57.881 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.880+0000 7fd695a96700 1 -- 172.21.15.77:0/3649722969 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd6780a0570 msgr2=0x7fd6780a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:57.881 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.880+0000 7fd695a96700 1 --2- 172.21.15.77:0/3649722969 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd6780a0570 0x7fd6780a2a30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd680009b30 tx=0x7fd680009260 comp rx=0 tx=0).stop 2023-12-01T01:15:57.881 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.880+0000 7fd695a96700 1 -- 172.21.15.77:0/3649722969 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd690100da0 msgr2=0x7fd690196a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:15:57.881 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.880+0000 7fd695a96700 1 --2- 172.21.15.77:0/3649722969 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd690100da0 0x7fd690196a60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd68400c990 tx=0x7fd68400cca0 comp rx=0 tx=0).stop 2023-12-01T01:15:57.881 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.880+0000 7fd695a96700 1 -- 172.21.15.77:0/3649722969 shutdown_connections 2023-12-01T01:15:57.882 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.880+0000 7fd695a96700 1 --2- 172.21.15.77:0/3649722969 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd6780a0570 0x7fd6780a2a30 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:57.882 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.880+0000 7fd695a96700 1 --2- 172.21.15.77:0/3649722969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd690102930 0x7fd69019e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:57.882 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.880+0000 7fd695a96700 1 --2- 172.21.15.77:0/3649722969 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd690100da0 0x7fd690196a60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:57.882 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.880+0000 7fd695a96700 1 --2- 172.21.15.77:0/3649722969 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd690101f90 0x7fd690196fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:15:57.882 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.880+0000 7fd695a96700 1 -- 172.21.15.77:0/3649722969 >> 172.21.15.77:0/3649722969 conn(0x7fd6900fc2d0 msgr2=0x7fd6900fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:15:57.882 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.880+0000 7fd695a96700 1 -- 172.21.15.77:0/3649722969 shutdown_connections 2023-12-01T01:15:57.882 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:15:57.880+0000 7fd695a96700 1 -- 172.21.15.77:0/3649722969 wait complete. 2023-12-01T01:15:57.882 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 2 2023-12-01T01:15:58.238 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:58 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/3649722969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:15:58.238 INFO:journalctl@ceph.mds.k.smithi137.stdout:Dec 01 01:15:57 smithi137 podman[140328]: 2023-12-01T01:15:58.238 INFO:journalctl@ceph.mds.k.smithi137.stdout:Dec 01 01:15:58 smithi137 ceph-mds[140369]: -- 172.21.15.137:0/3145954006 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55da6e7ad1e0 con 0x55da6e783000 2023-12-01T01:15:58.403 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:58 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/3649722969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:15:58.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:58 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/3649722969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:15:58.528 INFO:journalctl@ceph.mds.k.smithi137.stdout:Dec 01 01:15:58 smithi137 bash[140328]: b5339df18575275671fb8ea7ec2583ef8cc7becab133e8981c122c55ffe44bfe 2023-12-01T01:15:58.528 INFO:journalctl@ceph.mds.k.smithi137.stdout:Dec 01 01:15:58 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-k[140344]: starting mds.k at 2023-12-01T01:15:58.529 INFO:journalctl@ceph.mds.k.smithi137.stdout:Dec 01 01:15:58 smithi137 systemd[1]: Started Ceph mds.k for 2a06e29a-8fe6-11ee-95a2-87774f69a715. 2023-12-01T01:15:59.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:59 smithi077 ceph-mon[98867]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:15:59.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:59 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:59.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:59 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:59.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:59 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:59.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:59 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-01T01:15:59.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:59 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-01T01:15:59.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:59 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:15:59.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:59 smithi077 ceph-mon[98867]: Deploying daemon mds.j on smithi077 2023-12-01T01:15:59.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:59 smithi077 ceph-mon[98867]: mds.? [v2:172.21.15.137:6832/2249638445,v1:172.21.15.137:6833/2249638445] up:boot 2023-12-01T01:15:59.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:59 smithi077 ceph-mon[98867]: fsmap 2 up:standby 2023-12-01T01:15:59.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:15:59 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-01T01:15:59.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:59 smithi203 ceph-mon[103581]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:15:59.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:59 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:59.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:59 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:59.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:59 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:59.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:59 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-01T01:15:59.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:59 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-01T01:15:59.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:59 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:15:59.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:59 smithi203 ceph-mon[103581]: Deploying daemon mds.j on smithi077 2023-12-01T01:15:59.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:59 smithi203 ceph-mon[103581]: mds.? [v2:172.21.15.137:6832/2249638445,v1:172.21.15.137:6833/2249638445] up:boot 2023-12-01T01:15:59.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:59 smithi203 ceph-mon[103581]: fsmap 2 up:standby 2023-12-01T01:15:59.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:15:59 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-01T01:15:59.778 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:59 smithi137 ceph-mon[103312]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:15:59.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:59 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:59.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:59 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:59.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:59 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:15:59.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:59 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-01T01:15:59.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:59 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-01T01:15:59.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:59 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:15:59.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:59 smithi137 ceph-mon[103312]: Deploying daemon mds.j on smithi077 2023-12-01T01:15:59.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:59 smithi137 ceph-mon[103312]: mds.? [v2:172.21.15.137:6832/2249638445,v1:172.21.15.137:6833/2249638445] up:boot 2023-12-01T01:15:59.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:59 smithi137 ceph-mon[103312]: fsmap 2 up:standby 2023-12-01T01:15:59.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:15:59 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-01T01:15:59.893 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:15:59.903 INFO:journalctl@ceph.mds.j.smithi077.stdout:Dec 01 01:15:59 smithi077 systemd[1]: Starting Ceph mds.j for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:16:00.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.024+0000 7fa38b35d700 1 -- 172.21.15.77:0/3950420625 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa384101800 msgr2=0x7fa3841080b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:00.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.024+0000 7fa38b35d700 1 --2- 172.21.15.77:0/3950420625 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa384101800 0x7fa3841080b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa37c00b0d0 tx=0x7fa37c00b3e0 comp rx=0 tx=0).stop 2023-12-01T01:16:00.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.024+0000 7fa38b35d700 1 -- 172.21.15.77:0/3950420625 shutdown_connections 2023-12-01T01:16:00.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.024+0000 7fa38b35d700 1 --2- 172.21.15.77:0/3950420625 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa384101800 0x7fa3841080b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:00.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.024+0000 7fa38b35d700 1 --2- 172.21.15.77:0/3950420625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3840ffc70 0x7fa384100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:00.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.024+0000 7fa38b35d700 1 --2- 172.21.15.77:0/3950420625 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa384100e60 0x7fa3841012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:00.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.024+0000 7fa38b35d700 1 -- 172.21.15.77:0/3950420625 >> 172.21.15.77:0/3950420625 conn(0x7fa3840fb1c0 msgr2=0x7fa3840fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:00.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.024+0000 7fa38b35d700 1 -- 172.21.15.77:0/3950420625 shutdown_connections 2023-12-01T01:16:00.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.024+0000 7fa38b35d700 1 -- 172.21.15.77:0/3950420625 wait complete. 2023-12-01T01:16:00.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.025+0000 7fa38b35d700 1 Processor -- start 2023-12-01T01:16:00.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.025+0000 7fa38b35d700 1 -- start start 2023-12-01T01:16:00.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.025+0000 7fa38b35d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3840ffc70 0x7fa384196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:00.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.025+0000 7fa38b35d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa384100e60 0x7fa384196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:00.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.025+0000 7fa38b35d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa384101800 0x7fa38419e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:00.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.025+0000 7fa38b35d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38419e610 con 0x7fa384100e60 2023-12-01T01:16:00.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.025+0000 7fa38b35d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38419e750 con 0x7fa3840ffc70 2023-12-01T01:16:00.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.025+0000 7fa38b35d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38419e8e0 con 0x7fa384101800 2023-12-01T01:16:00.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.025+0000 7fa3888f8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa384100e60 0x7fa384196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:00.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.025+0000 7fa3888f8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa384100e60 0x7fa384196fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57186/0 (socket says 172.21.15.77:57186) 2023-12-01T01:16:00.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.025+0000 7fa3888f8700 1 -- 172.21.15.77:0/1916014055 learned_addr learned my addr 172.21.15.77:0/1916014055 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:00.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.025+0000 7fa3898fa700 1 --2- 172.21.15.77:0/1916014055 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa384101800 0x7fa38419e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:00.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.025+0000 7fa3890f9700 1 --2- 172.21.15.77:0/1916014055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3840ffc70 0x7fa384196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:00.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.025+0000 7fa3888f8700 1 -- 172.21.15.77:0/1916014055 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa384101800 msgr2=0x7fa38419e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:00.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.025+0000 7fa3888f8700 1 --2- 172.21.15.77:0/1916014055 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa384101800 0x7fa38419e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:00.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.025+0000 7fa3888f8700 1 -- 172.21.15.77:0/1916014055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3840ffc70 msgr2=0x7fa384196a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:00.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.025+0000 7fa3888f8700 1 --2- 172.21.15.77:0/1916014055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3840ffc70 0x7fa384196a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:00.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.025+0000 7fa3888f8700 1 -- 172.21.15.77:0/1916014055 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa37c009d00 con 0x7fa384100e60 2023-12-01T01:16:00.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.025+0000 7fa3898fa700 1 --2- 172.21.15.77:0/1916014055 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa384101800 0x7fa38419e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:16:00.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.026+0000 7fa3888f8700 1 --2- 172.21.15.77:0/1916014055 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa384100e60 0x7fa384196fd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa37400c920 tx=0x7fa37400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:00.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.026+0000 7fa37a7fc700 1 -- 172.21.15.77:0/1916014055 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa374007b00 con 0x7fa384100e60 2023-12-01T01:16:00.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.026+0000 7fa37a7fc700 1 -- 172.21.15.77:0/1916014055 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa374004cf0 con 0x7fa384100e60 2023-12-01T01:16:00.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.026+0000 7fa38b35d700 1 -- 172.21.15.77:0/1916014055 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa38419ebc0 con 0x7fa384100e60 2023-12-01T01:16:00.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.026+0000 7fa37a7fc700 1 -- 172.21.15.77:0/1916014055 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa374005610 con 0x7fa384100e60 2023-12-01T01:16:00.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.026+0000 7fa38b35d700 1 -- 172.21.15.77:0/1916014055 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa38419f0d0 con 0x7fa384100e60 2023-12-01T01:16:00.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.027+0000 7fa38b35d700 1 -- 172.21.15.77:0/1916014055 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa38410b4e0 con 0x7fa384100e60 2023-12-01T01:16:00.031 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.028+0000 7fa37a7fc700 1 -- 172.21.15.77:0/1916014055 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fa374004750 con 0x7fa384100e60 2023-12-01T01:16:00.031 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.029+0000 7fa37a7fc700 1 --2- 172.21.15.77:0/1916014055 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa3700a0570 0x7fa3700a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:00.031 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.029+0000 7fa3890f9700 1 --2- 172.21.15.77:0/1916014055 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa3700a0570 0x7fa3700a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:00.031 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.029+0000 7fa37a7fc700 1 -- 172.21.15.77:0/1916014055 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa3740cb4d0 con 0x7fa384100e60 2023-12-01T01:16:00.031 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.029+0000 7fa3890f9700 1 --2- 172.21.15.77:0/1916014055 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa3700a0570 0x7fa3700a2a30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa380005fd0 tx=0x7fa380005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:00.033 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.032+0000 7fa37a7fc700 1 -- 172.21.15.77:0/1916014055 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa374095510 con 0x7fa384100e60 2023-12-01T01:16:00.215 INFO:journalctl@ceph.mds.j.smithi077.stdout:Dec 01 01:15:59 smithi077 podman[148906]: 2023-12-01T01:16:00.283 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.282+0000 7fa38b35d700 1 -- 172.21.15.77:0/1916014055 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa38419f680 con 0x7fa384100e60 2023-12-01T01:16:00.284 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.282+0000 7fa37a7fc700 1 -- 172.21.15.77:0/1916014055 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2020 (secure 0 0 0) 0x7fa374098b30 con 0x7fa384100e60 2023-12-01T01:16:00.284 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:16:00.284 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/2249638445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2249638445},{"type":"v1","addr":"172.21.15.137:6833","nonce":2249638445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/1201599774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1201599774},{"type":"v1","addr":"172.21.15.203:6833","nonce":1201599774}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-12-01T01:16:00.287 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.285+0000 7fa38b35d700 1 -- 172.21.15.77:0/1916014055 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa3700a0570 msgr2=0x7fa3700a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:00.287 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.285+0000 7fa38b35d700 1 --2- 172.21.15.77:0/1916014055 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa3700a0570 0x7fa3700a2a30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa380005fd0 tx=0x7fa380005e50 comp rx=0 tx=0).stop 2023-12-01T01:16:00.287 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.285+0000 7fa38b35d700 1 -- 172.21.15.77:0/1916014055 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa384100e60 msgr2=0x7fa384196fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:00.287 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.285+0000 7fa38b35d700 1 --2- 172.21.15.77:0/1916014055 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa384100e60 0x7fa384196fd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa37400c920 tx=0x7fa37400cce0 comp rx=0 tx=0).stop 2023-12-01T01:16:00.287 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.285+0000 7fa38b35d700 1 -- 172.21.15.77:0/1916014055 shutdown_connections 2023-12-01T01:16:00.287 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.285+0000 7fa38b35d700 1 --2- 172.21.15.77:0/1916014055 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa3700a0570 0x7fa3700a2a30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:00.287 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.285+0000 7fa38b35d700 1 --2- 172.21.15.77:0/1916014055 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa384101800 0x7fa38419e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:00.287 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.285+0000 7fa38b35d700 1 --2- 172.21.15.77:0/1916014055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa3840ffc70 0x7fa384196a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:00.287 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.285+0000 7fa38b35d700 1 --2- 172.21.15.77:0/1916014055 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa384100e60 0x7fa384196fd0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:00.287 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.285+0000 7fa38b35d700 1 -- 172.21.15.77:0/1916014055 >> 172.21.15.77:0/1916014055 conn(0x7fa3840fb1c0 msgr2=0x7fa384067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:00.288 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.285+0000 7fa38b35d700 1 -- 172.21.15.77:0/1916014055 shutdown_connections 2023-12-01T01:16:00.288 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:00.285+0000 7fa38b35d700 1 -- 172.21.15.77:0/1916014055 wait complete. 2023-12-01T01:16:00.288 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 3 2023-12-01T01:16:00.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:00 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/1916014055' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:16:00.654 INFO:journalctl@ceph.mds.j.smithi077.stdout:Dec 01 01:16:00 smithi077 ceph-mds[148989]: -- 172.21.15.77:0/444733489 <== mon.0 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b09f7471e0 con 0x55b09f71c800 2023-12-01T01:16:00.654 INFO:journalctl@ceph.mds.j.smithi077.stdout:Dec 01 01:16:00 smithi077 bash[148906]: a78b678a6a672c7da59b7b232a2f0b00953fa322397d229a92a33a4f8aed09b6 2023-12-01T01:16:00.654 INFO:journalctl@ceph.mds.j.smithi077.stdout:Dec 01 01:16:00 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-j[148946]: starting mds.j at 2023-12-01T01:16:00.654 INFO:journalctl@ceph.mds.j.smithi077.stdout:Dec 01 01:16:00 smithi077 systemd[1]: Started Ceph mds.j for 2a06e29a-8fe6-11ee-95a2-87774f69a715. 2023-12-01T01:16:00.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:00 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/1916014055' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:16:00.778 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:00 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/1916014055' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:16:01.469 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:01 smithi077 ceph-mon[98867]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:16:01.469 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:01 smithi077 ceph-mon[98867]: mds.? [v2:172.21.15.77:6834/1435145620,v1:172.21.15.77:6835/1435145620] up:boot 2023-12-01T01:16:01.469 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:01 smithi077 ceph-mon[98867]: fsmap 3 up:standby 2023-12-01T01:16:01.469 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:01 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-12-01T01:16:01.470 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:01 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:01.470 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:01 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:01.470 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:01 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:01.470 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:01 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-01T01:16:01.470 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:01 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-01T01:16:01.470 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:01 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:16:01.470 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:01 smithi077 ceph-mon[98867]: Deploying daemon mds.d on smithi077 2023-12-01T01:16:01.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:01 smithi203 ceph-mon[103581]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:16:01.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:01 smithi203 ceph-mon[103581]: mds.? [v2:172.21.15.77:6834/1435145620,v1:172.21.15.77:6835/1435145620] up:boot 2023-12-01T01:16:01.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:01 smithi203 ceph-mon[103581]: fsmap 3 up:standby 2023-12-01T01:16:01.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:01 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-12-01T01:16:01.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:01 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:01.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:01 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:01.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:01 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:01.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:01 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-01T01:16:01.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:01 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-01T01:16:01.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:01 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:16:01.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:01 smithi203 ceph-mon[103581]: Deploying daemon mds.d on smithi077 2023-12-01T01:16:01.778 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:01 smithi137 ceph-mon[103312]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:16:01.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:01 smithi137 ceph-mon[103312]: mds.? [v2:172.21.15.77:6834/1435145620,v1:172.21.15.77:6835/1435145620] up:boot 2023-12-01T01:16:01.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:01 smithi137 ceph-mon[103312]: fsmap 3 up:standby 2023-12-01T01:16:01.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:01 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-12-01T01:16:01.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:01 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:01.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:01 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:01.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:01 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:01.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:01 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-01T01:16:01.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:01 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-01T01:16:01.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:01 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:16:01.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:01 smithi137 ceph-mon[103312]: Deploying daemon mds.d on smithi077 2023-12-01T01:16:01.904 INFO:journalctl@ceph.mds.d.smithi077.stdout:Dec 01 01:16:01 smithi077 systemd[1]: Starting Ceph mds.d for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:16:02.285 INFO:journalctl@ceph.mds.d.smithi077.stdout:Dec 01 01:16:02 smithi077 podman[149293]: 2023-12-01T01:16:02.285 INFO:journalctl@ceph.mds.d.smithi077.stdout:Dec 01 01:16:02 smithi077 ceph-mds[149333]: -- 172.21.15.77:0/972478161 <== mon.0 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560dad94d1e0 con 0x560dad923000 2023-12-01T01:16:02.286 INFO:journalctl@ceph.mds.d.smithi077.stdout:Dec 01 01:16:02 smithi077 bash[149293]: f40a9289b76ccf81fd04d959b1946d037eca2661a97bb3ce7a2718e7811b84e2 2023-12-01T01:16:02.286 INFO:journalctl@ceph.mds.d.smithi077.stdout:Dec 01 01:16:02 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-d[149309]: starting mds.d at 2023-12-01T01:16:02.301 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:16:02.432 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.430+0000 7ff6dceab700 1 -- 172.21.15.77:0/1019018717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6d8104120 msgr2=0x7ff6d8106510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:02.432 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.430+0000 7ff6dceab700 1 --2- 172.21.15.77:0/1019018717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6d8104120 0x7ff6d8106510 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc009990 tx=0x7ff6cc009ca0 comp rx=0 tx=0).stop 2023-12-01T01:16:02.432 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.430+0000 7ff6dceab700 1 -- 172.21.15.77:0/1019018717 shutdown_connections 2023-12-01T01:16:02.432 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.430+0000 7ff6dceab700 1 --2- 172.21.15.77:0/1019018717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6d8106a50 0x7ff6d8108ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:02.432 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.430+0000 7ff6dceab700 1 --2- 172.21.15.77:0/1019018717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6d81017f0 0x7ff6d8103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:02.432 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.430+0000 7ff6dceab700 1 --2- 172.21.15.77:0/1019018717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6d8104120 0x7ff6d8106510 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:02.432 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.430+0000 7ff6dceab700 1 -- 172.21.15.77:0/1019018717 >> 172.21.15.77:0/1019018717 conn(0x7ff6d80fb1c0 msgr2=0x7ff6d80fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:02.432 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.430+0000 7ff6dceab700 1 -- 172.21.15.77:0/1019018717 shutdown_connections 2023-12-01T01:16:02.432 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.430+0000 7ff6dceab700 1 -- 172.21.15.77:0/1019018717 wait complete. 2023-12-01T01:16:02.434 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.431+0000 7ff6dceab700 1 Processor -- start 2023-12-01T01:16:02.434 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.431+0000 7ff6dceab700 1 -- start start 2023-12-01T01:16:02.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.431+0000 7ff6dceab700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6d81017f0 0x7ff6d8101160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:02.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.431+0000 7ff6dceab700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6d8104120 0x7ff6d80ff7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:02.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.431+0000 7ff6dceab700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6d8106a50 0x7ff6d80ffce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:02.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.431+0000 7ff6dceab700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d8100480 con 0x7ff6d8106a50 2023-12-01T01:16:02.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.431+0000 7ff6dceab700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d81005f0 con 0x7ff6d81017f0 2023-12-01T01:16:02.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.431+0000 7ff6dceab700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d81a0a50 con 0x7ff6d8104120 2023-12-01T01:16:02.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.431+0000 7ff6d6d9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6d8106a50 0x7ff6d80ffce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:02.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.431+0000 7ff6d6d9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6d8106a50 0x7ff6d80ffce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57226/0 (socket says 172.21.15.77:57226) 2023-12-01T01:16:02.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.431+0000 7ff6d6d9d700 1 -- 172.21.15.77:0/3061240434 learned_addr learned my addr 172.21.15.77:0/3061240434 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:02.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.431+0000 7ff6d659c700 1 --2- 172.21.15.77:0/3061240434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6d81017f0 0x7ff6d8101160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:02.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.431+0000 7ff6d5d9b700 1 --2- 172.21.15.77:0/3061240434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6d8104120 0x7ff6d80ff7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:02.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.431+0000 7ff6d6d9d700 1 -- 172.21.15.77:0/3061240434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6d8104120 msgr2=0x7ff6d80ff7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:02.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.432+0000 7ff6d6d9d700 1 --2- 172.21.15.77:0/3061240434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6d8104120 0x7ff6d80ff7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:02.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.432+0000 7ff6d6d9d700 1 -- 172.21.15.77:0/3061240434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6d81017f0 msgr2=0x7ff6d8101160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:02.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.432+0000 7ff6d6d9d700 1 --2- 172.21.15.77:0/3061240434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6d81017f0 0x7ff6d8101160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:02.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.432+0000 7ff6d6d9d700 1 -- 172.21.15.77:0/3061240434 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6cc0095f0 con 0x7ff6d8106a50 2023-12-01T01:16:02.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.432+0000 7ff6d6d9d700 1 --2- 172.21.15.77:0/3061240434 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6d8106a50 0x7ff6d80ffce0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0007ee0 tx=0x7ff6d000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:02.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.432+0000 7ff6c77fe700 1 -- 172.21.15.77:0/3061240434 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d000dd20 con 0x7ff6d8106a50 2023-12-01T01:16:02.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.432+0000 7ff6dceab700 1 -- 172.21.15.77:0/3061240434 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6d81a0d30 con 0x7ff6d8106a50 2023-12-01T01:16:02.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.432+0000 7ff6c77fe700 1 -- 172.21.15.77:0/3061240434 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6d000f040 con 0x7ff6d8106a50 2023-12-01T01:16:02.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.432+0000 7ff6c77fe700 1 -- 172.21.15.77:0/3061240434 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d00048a0 con 0x7ff6d8106a50 2023-12-01T01:16:02.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.432+0000 7ff6dceab700 1 -- 172.21.15.77:0/3061240434 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d81a1190 con 0x7ff6d8106a50 2023-12-01T01:16:02.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.433+0000 7ff6dceab700 1 -- 172.21.15.77:0/3061240434 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6d80fcd40 con 0x7ff6d8106a50 2023-12-01T01:16:02.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.434+0000 7ff6c77fe700 1 -- 172.21.15.77:0/3061240434 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7ff6d0003a00 con 0x7ff6d8106a50 2023-12-01T01:16:02.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.435+0000 7ff6c77fe700 1 --2- 172.21.15.77:0/3061240434 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff6c00a0570 0x7ff6c00a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:02.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.435+0000 7ff6d659c700 1 --2- 172.21.15.77:0/3061240434 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff6c00a0570 0x7ff6c00a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:02.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.435+0000 7ff6c77fe700 1 -- 172.21.15.77:0/3061240434 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff6d00cb4b0 con 0x7ff6d8106a50 2023-12-01T01:16:02.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.435+0000 7ff6d659c700 1 --2- 172.21.15.77:0/3061240434 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff6c00a0570 0x7ff6c00a2a30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8007c60 tx=0x7ff6c80073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:02.439 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.438+0000 7ff6c77fe700 1 -- 172.21.15.77:0/3061240434 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff6d00954d0 con 0x7ff6d8106a50 2023-12-01T01:16:02.654 INFO:journalctl@ceph.mds.d.smithi077.stdout:Dec 01 01:16:02 smithi077 systemd[1]: Started Ceph mds.d for 2a06e29a-8fe6-11ee-95a2-87774f69a715. 2023-12-01T01:16:02.679 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.678+0000 7ff6dceab700 1 -- 172.21.15.77:0/3061240434 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6d802cc70 con 0x7ff6d8106a50 2023-12-01T01:16:02.680 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.679+0000 7ff6c77fe700 1 -- 172.21.15.77:0/3061240434 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3531 (secure 0 0 0) 0x7ff6d0023de0 con 0x7ff6d8106a50 2023-12-01T01:16:02.680 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:16:02.680 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1435145620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1435145620},{"type":"v1","addr":"172.21.15.77:6835","nonce":1435145620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/841838908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":841838908},{"type":"v1","addr":"172.21.15.77:6837","nonce":841838908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/2249638445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2249638445},{"type":"v1","addr":"172.21.15.137:6833","nonce":2249638445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/1201599774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1201599774},{"type":"v1","addr":"172.21.15.203:6833","nonce":1201599774}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-12-01T01:16:02.682 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.681+0000 7ff6dceab700 1 -- 172.21.15.77:0/3061240434 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff6c00a0570 msgr2=0x7ff6c00a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:02.682 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.681+0000 7ff6dceab700 1 --2- 172.21.15.77:0/3061240434 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff6c00a0570 0x7ff6c00a2a30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8007c60 tx=0x7ff6c80073d0 comp rx=0 tx=0).stop 2023-12-01T01:16:02.683 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.681+0000 7ff6dceab700 1 -- 172.21.15.77:0/3061240434 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6d8106a50 msgr2=0x7ff6d80ffce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:02.683 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.681+0000 7ff6dceab700 1 --2- 172.21.15.77:0/3061240434 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6d8106a50 0x7ff6d80ffce0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0007ee0 tx=0x7ff6d000d3b0 comp rx=0 tx=0).stop 2023-12-01T01:16:02.683 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.681+0000 7ff6dceab700 1 -- 172.21.15.77:0/3061240434 shutdown_connections 2023-12-01T01:16:02.683 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.681+0000 7ff6dceab700 1 --2- 172.21.15.77:0/3061240434 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff6c00a0570 0x7ff6c00a2a30 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:02.683 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.681+0000 7ff6dceab700 1 --2- 172.21.15.77:0/3061240434 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6d8106a50 0x7ff6d80ffce0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:02.683 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.681+0000 7ff6dceab700 1 --2- 172.21.15.77:0/3061240434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6d81017f0 0x7ff6d8101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:02.683 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.681+0000 7ff6dceab700 1 --2- 172.21.15.77:0/3061240434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff6d8104120 0x7ff6d80ff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:02.683 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.681+0000 7ff6dceab700 1 -- 172.21.15.77:0/3061240434 >> 172.21.15.77:0/3061240434 conn(0x7ff6d80fb1c0 msgr2=0x7ff6d8102460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:02.683 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.681+0000 7ff6dceab700 1 -- 172.21.15.77:0/3061240434 shutdown_connections 2023-12-01T01:16:02.683 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:02.681+0000 7ff6dceab700 1 -- 172.21.15.77:0/3061240434 wait complete. 2023-12-01T01:16:02.683 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 5 2023-12-01T01:16:03.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:03 smithi077 ceph-mon[98867]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:16:03.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:03 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:03.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:03 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:03.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:03 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:03.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:03 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-01T01:16:03.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:03 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-01T01:16:03.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:03 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:16:03.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:03 smithi077 ceph-mon[98867]: Deploying daemon mds.e on smithi137 2023-12-01T01:16:03.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:03 smithi077 ceph-mon[98867]: mds.? [v2:172.21.15.77:6836/841838908,v1:172.21.15.77:6837/841838908] up:boot 2023-12-01T01:16:03.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:03 smithi077 ceph-mon[98867]: fsmap 4 up:standby 2023-12-01T01:16:03.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:03 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-12-01T01:16:03.655 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:03 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/3061240434' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:16:03.659 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:03 smithi137 ceph-mon[103312]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:16:03.659 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:03 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:03.659 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:03 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:03.659 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:03 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:03.659 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:03 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-01T01:16:03.659 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:03 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-01T01:16:03.659 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:03 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:16:03.659 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:03 smithi137 ceph-mon[103312]: Deploying daemon mds.e on smithi137 2023-12-01T01:16:03.659 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:03 smithi137 ceph-mon[103312]: mds.? [v2:172.21.15.77:6836/841838908,v1:172.21.15.77:6837/841838908] up:boot 2023-12-01T01:16:03.659 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:03 smithi137 ceph-mon[103312]: fsmap 4 up:standby 2023-12-01T01:16:03.660 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:03 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-12-01T01:16:03.660 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:03 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/3061240434' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:16:03.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:03 smithi203 ceph-mon[103581]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:16:03.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:03 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:03.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:03 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:03.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:03 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:03.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:03 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-01T01:16:03.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:03 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-01T01:16:03.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:03 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:16:03.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:03 smithi203 ceph-mon[103581]: Deploying daemon mds.e on smithi137 2023-12-01T01:16:03.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:03 smithi203 ceph-mon[103581]: mds.? [v2:172.21.15.77:6836/841838908,v1:172.21.15.77:6837/841838908] up:boot 2023-12-01T01:16:03.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:03 smithi203 ceph-mon[103581]: fsmap 4 up:standby 2023-12-01T01:16:03.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:03 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-12-01T01:16:03.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:03 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/3061240434' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:16:04.029 INFO:journalctl@ceph.mds.e.smithi137.stdout:Dec 01 01:16:03 smithi137 systemd[1]: Starting Ceph mds.e for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:16:04.420 INFO:journalctl@ceph.mds.e.smithi137.stdout:Dec 01 01:16:04 smithi137 podman[140666]: 2023-12-01T01:16:04.420 INFO:journalctl@ceph.mds.e.smithi137.stdout:Dec 01 01:16:04 smithi137 ceph-mds[140706]: -- 172.21.15.137:0/1947137254 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5629e11171e0 con 0x5629e10ecc00 2023-12-01T01:16:04.420 INFO:journalctl@ceph.mds.e.smithi137.stdout:Dec 01 01:16:04 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-e[140682]: starting mds.e at 2023-12-01T01:16:04.695 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:16:04.778 INFO:journalctl@ceph.mds.e.smithi137.stdout:Dec 01 01:16:04 smithi137 bash[140666]: afce27568d7f387784cca97e3c120fa88203afd56553ea604926df12cc8645a5 2023-12-01T01:16:04.779 INFO:journalctl@ceph.mds.e.smithi137.stdout:Dec 01 01:16:04 smithi137 systemd[1]: Started Ceph mds.e for 2a06e29a-8fe6-11ee-95a2-87774f69a715. 2023-12-01T01:16:04.828 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:04.824+0000 7f6384463700 1 -- 172.21.15.77:0/1737956871 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f637c0ffc80 msgr2=0x7f637c1000a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:04.828 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:04.824+0000 7f6384463700 1 --2- 172.21.15.77:0/1737956871 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f637c0ffc80 0x7f637c1000a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f636c0099c0 tx=0x7f636c009cd0 comp rx=0 tx=0).stop 2023-12-01T01:16:04.828 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:04.824+0000 7f6384463700 1 -- 172.21.15.77:0/1737956871 shutdown_connections 2023-12-01T01:16:04.828 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:04.824+0000 7f6384463700 1 --2- 172.21.15.77:0/1737956871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f637c101810 0x7f637c1080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:04.828 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:04.824+0000 7f6384463700 1 --2- 172.21.15.77:0/1737956871 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f637c0ffc80 0x7f637c1000a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:04.828 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:04.824+0000 7f6384463700 1 --2- 172.21.15.77:0/1737956871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f637c100e70 0x7f637c1012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:04.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:04.824+0000 7f6384463700 1 -- 172.21.15.77:0/1737956871 >> 172.21.15.77:0/1737956871 conn(0x7f637c0fb1b0 msgr2=0x7f637c0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:04.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:04.824+0000 7f6384463700 1 -- 172.21.15.77:0/1737956871 shutdown_connections 2023-12-01T01:16:04.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:04.824+0000 7f6384463700 1 -- 172.21.15.77:0/1737956871 wait complete. 2023-12-01T01:16:04.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:04.825+0000 7f6384463700 1 Processor -- start 2023-12-01T01:16:04.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:04.825+0000 7f6384463700 1 -- start start 2023-12-01T01:16:04.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:04.825+0000 7f6384463700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f637c0ffc80 0x7f637c198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:04.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:04.825+0000 7f6384463700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f637c100e70 0x7f637c1991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:04.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:04.825+0000 7f6384463700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f637c101810 0x7f637c1a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:04.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:04.825+0000 7f6384463700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c1a07b0 con 0x7f637c101810 2023-12-01T01:16:04.830 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:04.825+0000 7f6384463700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c1a0920 con 0x7f637c100e70 2023-12-01T01:16:04.830 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:04.825+0000 7f6384463700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c1a0ae0 con 0x7f637c0ffc80 2023-12-01T01:16:04.830 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:04.825+0000 7f6382a00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f637c101810 0x7f637c1a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:04.830 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:04.825+0000 7f6382a00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f637c101810 0x7f637c1a01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57242/0 (socket says 172.21.15.77:57242) 2023-12-01T01:16:04.830 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:04.825+0000 7f6382a00700 1 -- 172.21.15.77:0/298386343 learned_addr learned my addr 172.21.15.77:0/298386343 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:04.830 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:04.825+0000 7f63821ff700 1 --2- 172.21.15.77:0/298386343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f637c0ffc80 0x7f637c198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:04.830 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:04.825+0000 7f63819fe700 1 --2- 172.21.15.77:0/298386343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f637c100e70 0x7f637c1991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:04.830 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:04.826+0000 7f6382a00700 1 -- 172.21.15.77:0/298386343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f637c0ffc80 msgr2=0x7f637c198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:04.830 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:04.826+0000 7f6382a00700 1 --2- 172.21.15.77:0/298386343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f637c0ffc80 0x7f637c198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:04.830 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:04.826+0000 7f6382a00700 1 -- 172.21.15.77:0/298386343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f637c100e70 msgr2=0x7f637c1991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:04.830 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:04.826+0000 7f6382a00700 1 --2- 172.21.15.77:0/298386343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f637c100e70 0x7f637c1991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:04.830 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:04.826+0000 7f6382a00700 1 -- 172.21.15.77:0/298386343 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f636c0095f0 con 0x7f637c101810 2023-12-01T01:16:04.830 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:04.826+0000 7f6382a00700 1 --2- 172.21.15.77:0/298386343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f637c101810 0x7f637c1a01e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6374007ee0 tx=0x7f637400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:04.830 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:04.826+0000 7f63737fe700 1 -- 172.21.15.77:0/298386343 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637400dd20 con 0x7f637c101810 2023-12-01T01:16:04.831 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:04.826+0000 7f63737fe700 1 -- 172.21.15.77:0/298386343 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f637400f040 con 0x7f637c101810 2023-12-01T01:16:04.831 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:04.826+0000 7f6384463700 1 -- 172.21.15.77:0/298386343 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f637c1a0dc0 con 0x7f637c101810 2023-12-01T01:16:04.831 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:04.826+0000 7f63737fe700 1 -- 172.21.15.77:0/298386343 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63740048a0 con 0x7f637c101810 2023-12-01T01:16:04.831 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:04.826+0000 7f6384463700 1 -- 172.21.15.77:0/298386343 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f637c1a11c0 con 0x7f637c101810 2023-12-01T01:16:04.831 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:04.827+0000 7f6384463700 1 -- 172.21.15.77:0/298386343 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f637c10b4e0 con 0x7f637c101810 2023-12-01T01:16:04.831 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:04.828+0000 7f63737fe700 1 -- 172.21.15.77:0/298386343 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f6374003a00 con 0x7f637c101810 2023-12-01T01:16:04.832 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:04.828+0000 7f63737fe700 1 --2- 172.21.15.77:0/298386343 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f63680a05f0 0x7f63680a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:04.832 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:04.828+0000 7f63821ff700 1 --2- 172.21.15.77:0/298386343 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f63680a05f0 0x7f63680a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:04.832 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:04.828+0000 7f63737fe700 1 -- 172.21.15.77:0/298386343 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f63740cb620 con 0x7f637c101810 2023-12-01T01:16:04.832 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:04.829+0000 7f63821ff700 1 --2- 172.21.15.77:0/298386343 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f63680a05f0 0x7f63680a2ab0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f637c19a140 tx=0x7f636c009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:04.833 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:04.832+0000 7f63737fe700 1 -- 172.21.15.77:0/298386343 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f63740955c0 con 0x7f637c101810 2023-12-01T01:16:05.067 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:05.066+0000 7f6384463700 1 -- 172.21.15.77:0/298386343 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f637c1a17d0 con 0x7f637c101810 2023-12-01T01:16:05.067 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:05.066+0000 7f63737fe700 1 -- 172.21.15.77:0/298386343 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3531 (secure 0 0 0) 0x7f6374023de0 con 0x7f637c101810 2023-12-01T01:16:05.068 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:16:05.068 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1435145620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1435145620},{"type":"v1","addr":"172.21.15.77:6835","nonce":1435145620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/841838908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":841838908},{"type":"v1","addr":"172.21.15.77:6837","nonce":841838908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/2249638445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2249638445},{"type":"v1","addr":"172.21.15.137:6833","nonce":2249638445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/1201599774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1201599774},{"type":"v1","addr":"172.21.15.203:6833","nonce":1201599774}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-12-01T01:16:05.070 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:05.068+0000 7f6384463700 1 -- 172.21.15.77:0/298386343 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f63680a05f0 msgr2=0x7f63680a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:05.070 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:05.068+0000 7f6384463700 1 --2- 172.21.15.77:0/298386343 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f63680a05f0 0x7f63680a2ab0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f637c19a140 tx=0x7f636c009f10 comp rx=0 tx=0).stop 2023-12-01T01:16:05.070 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:05.068+0000 7f6384463700 1 -- 172.21.15.77:0/298386343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f637c101810 msgr2=0x7f637c1a01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:05.070 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:05.068+0000 7f6384463700 1 --2- 172.21.15.77:0/298386343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f637c101810 0x7f637c1a01e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6374007ee0 tx=0x7f637400d3b0 comp rx=0 tx=0).stop 2023-12-01T01:16:05.070 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:05.068+0000 7f6384463700 1 -- 172.21.15.77:0/298386343 shutdown_connections 2023-12-01T01:16:05.070 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:05.068+0000 7f6384463700 1 --2- 172.21.15.77:0/298386343 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f63680a05f0 0x7f63680a2ab0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:05.070 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:05.068+0000 7f6384463700 1 --2- 172.21.15.77:0/298386343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f637c101810 0x7f637c1a01e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:05.070 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:05.068+0000 7f6384463700 1 --2- 172.21.15.77:0/298386343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f637c0ffc80 0x7f637c198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:05.071 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:05.068+0000 7f6384463700 1 --2- 172.21.15.77:0/298386343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f637c100e70 0x7f637c1991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:05.071 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:05.068+0000 7f6384463700 1 -- 172.21.15.77:0/298386343 >> 172.21.15.77:0/298386343 conn(0x7f637c0fb1b0 msgr2=0x7f637c0fba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:05.071 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:05.069+0000 7f6384463700 1 -- 172.21.15.77:0/298386343 shutdown_connections 2023-12-01T01:16:05.071 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:05.069+0000 7f6384463700 1 -- 172.21.15.77:0/298386343 wait complete. 2023-12-01T01:16:05.071 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 5 2023-12-01T01:16:05.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:05 smithi137 ceph-mon[103312]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:16:05.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:05 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:05.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:05 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:05.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:05 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:05.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:05 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-01T01:16:05.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:05 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-01T01:16:05.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:05 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:16:05.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:05 smithi137 ceph-mon[103312]: Deploying daemon mds.f on smithi203 2023-12-01T01:16:05.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:05 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/298386343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:16:05.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:05 smithi137 ceph-mon[103312]: mds.? [v2:172.21.15.137:6834/3935354878,v1:172.21.15.137:6835/3935354878] up:boot 2023-12-01T01:16:05.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:05 smithi137 ceph-mon[103312]: fsmap 5 up:standby 2023-12-01T01:16:05.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:05 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-01T01:16:05.798 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:05 smithi203 ceph-mon[103581]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:16:05.798 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:05 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:05.798 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:05 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:05.798 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:05 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:05.798 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:05 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-01T01:16:05.798 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:05 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-01T01:16:05.798 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:05 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:16:05.799 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:05 smithi203 ceph-mon[103581]: Deploying daemon mds.f on smithi203 2023-12-01T01:16:05.799 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:05 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/298386343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:16:05.799 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:05 smithi203 ceph-mon[103581]: mds.? [v2:172.21.15.137:6834/3935354878,v1:172.21.15.137:6835/3935354878] up:boot 2023-12-01T01:16:05.799 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:05 smithi203 ceph-mon[103581]: fsmap 5 up:standby 2023-12-01T01:16:05.799 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:05 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-01T01:16:05.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:05 smithi077 ceph-mon[98867]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:16:05.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:05 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:05.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:05 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:05.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:05 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:05.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:05 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-01T01:16:05.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:05 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-01T01:16:05.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:05 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:16:05.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:05 smithi077 ceph-mon[98867]: Deploying daemon mds.f on smithi203 2023-12-01T01:16:05.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:05 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/298386343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:16:05.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:05 smithi077 ceph-mon[98867]: mds.? [v2:172.21.15.137:6834/3935354878,v1:172.21.15.137:6835/3935354878] up:boot 2023-12-01T01:16:05.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:05 smithi077 ceph-mon[98867]: fsmap 5 up:standby 2023-12-01T01:16:05.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:05 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-01T01:16:06.437 INFO:journalctl@ceph.mds.f.smithi203.stdout:Dec 01 01:16:06 smithi203 systemd[1]: Starting Ceph mds.f for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:16:06.695 INFO:journalctl@ceph.mds.f.smithi203.stdout:Dec 01 01:16:06 smithi203 podman[140695]: 2023-12-01T01:16:07.012 INFO:journalctl@ceph.mds.f.smithi203.stdout:Dec 01 01:16:06 smithi203 ceph-mds[140736]: -- 172.21.15.203:0/481487579 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555b42eb71e0 con 0x555b42e8d000 2023-12-01T01:16:07.012 INFO:journalctl@ceph.mds.f.smithi203.stdout:Dec 01 01:16:06 smithi203 bash[140695]: a068deb4a29da0c14b8ae3cff028719dd8c67be23d16229549a99a0b97a4f20c 2023-12-01T01:16:07.012 INFO:journalctl@ceph.mds.f.smithi203.stdout:Dec 01 01:16:06 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-f[140711]: starting mds.f at 2023-12-01T01:16:07.012 INFO:journalctl@ceph.mds.f.smithi203.stdout:Dec 01 01:16:06 smithi203 systemd[1]: Started Ceph mds.f for 2a06e29a-8fe6-11ee-95a2-87774f69a715. 2023-12-01T01:16:07.083 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:16:07.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.216+0000 7f1b003d3700 1 -- 172.21.15.77:0/311731809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1af8101870 msgr2=0x7f1af8108100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:07.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.216+0000 7f1b003d3700 1 --2- 172.21.15.77:0/311731809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1af8101870 0x7f1af8108100 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1af000b0a0 tx=0x7f1af000b3b0 comp rx=0 tx=0).stop 2023-12-01T01:16:07.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.216+0000 7f1b003d3700 1 -- 172.21.15.77:0/311731809 shutdown_connections 2023-12-01T01:16:07.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.216+0000 7f1b003d3700 1 --2- 172.21.15.77:0/311731809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1af8101870 0x7f1af8108100 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:07.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.216+0000 7f1b003d3700 1 --2- 172.21.15.77:0/311731809 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1af81001a0 0x7f1af81005c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:07.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.216+0000 7f1b003d3700 1 --2- 172.21.15.77:0/311731809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1af8100eb0 0x7f1af8101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:07.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.216+0000 7f1b003d3700 1 -- 172.21.15.77:0/311731809 >> 172.21.15.77:0/311731809 conn(0x7f1af80fa950 msgr2=0x7f1af80fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:07.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.216+0000 7f1b003d3700 1 -- 172.21.15.77:0/311731809 shutdown_connections 2023-12-01T01:16:07.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.216+0000 7f1b003d3700 1 -- 172.21.15.77:0/311731809 wait complete. 2023-12-01T01:16:07.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.216+0000 7f1b003d3700 1 Processor -- start 2023-12-01T01:16:07.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.216+0000 7f1b003d3700 1 -- start start 2023-12-01T01:16:07.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.217+0000 7f1b003d3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1af81001a0 0x7f1af80ff860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:07.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.217+0000 7f1b003d3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1af8100eb0 0x7f1af819b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:07.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.217+0000 7f1b003d3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1af8101870 0x7f1af819de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:07.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.217+0000 7f1b003d3700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af819e630 con 0x7f1af81001a0 2023-12-01T01:16:07.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.217+0000 7f1b003d3700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af819e7a0 con 0x7f1af8100eb0 2023-12-01T01:16:07.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.217+0000 7f1b003d3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af819e960 con 0x7f1af8101870 2023-12-01T01:16:07.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.217+0000 7f1afe16f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1af81001a0 0x7f1af80ff860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:07.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.217+0000 7f1afe16f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1af81001a0 0x7f1af80ff860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57252/0 (socket says 172.21.15.77:57252) 2023-12-01T01:16:07.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.217+0000 7f1afe16f700 1 -- 172.21.15.77:0/2528528124 learned_addr learned my addr 172.21.15.77:0/2528528124 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:07.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.217+0000 7f1afe970700 1 --2- 172.21.15.77:0/2528528124 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1af8101870 0x7f1af819de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:07.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.217+0000 7f1afd96e700 1 --2- 172.21.15.77:0/2528528124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1af8100eb0 0x7f1af819b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:07.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.217+0000 7f1afe16f700 1 -- 172.21.15.77:0/2528528124 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1af8101870 msgr2=0x7f1af819de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:07.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.217+0000 7f1afe16f700 1 --2- 172.21.15.77:0/2528528124 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1af8101870 0x7f1af819de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:07.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.217+0000 7f1afe16f700 1 -- 172.21.15.77:0/2528528124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1af8100eb0 msgr2=0x7f1af819b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:07.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.217+0000 7f1afe16f700 1 --2- 172.21.15.77:0/2528528124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1af8100eb0 0x7f1af819b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:07.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.217+0000 7f1afe16f700 1 -- 172.21.15.77:0/2528528124 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af0009d00 con 0x7f1af81001a0 2023-12-01T01:16:07.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.217+0000 7f1afe16f700 1 --2- 172.21.15.77:0/2528528124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1af81001a0 0x7f1af80ff860 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1af400e970 tx=0x7f1af400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:07.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.217+0000 7f1aef7fe700 1 -- 172.21.15.77:0/2528528124 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af4009790 con 0x7f1af81001a0 2023-12-01T01:16:07.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.217+0000 7f1b003d3700 1 -- 172.21.15.77:0/2528528124 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1af819ec40 con 0x7f1af81001a0 2023-12-01T01:16:07.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.217+0000 7f1aef7fe700 1 -- 172.21.15.77:0/2528528124 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1af400eec0 con 0x7f1af81001a0 2023-12-01T01:16:07.222 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.217+0000 7f1aef7fe700 1 -- 172.21.15.77:0/2528528124 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af4003cf0 con 0x7f1af81001a0 2023-12-01T01:16:07.222 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.217+0000 7f1b003d3700 1 -- 172.21.15.77:0/2528528124 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1af819f150 con 0x7f1af81001a0 2023-12-01T01:16:07.222 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.218+0000 7f1b003d3700 1 -- 172.21.15.77:0/2528528124 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1af8106ba0 con 0x7f1af81001a0 2023-12-01T01:16:07.222 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.219+0000 7f1aef7fe700 1 -- 172.21.15.77:0/2528528124 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f1af400ccd0 con 0x7f1af81001a0 2023-12-01T01:16:07.223 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.220+0000 7f1aef7fe700 1 --2- 172.21.15.77:0/2528528124 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f1ae40a0570 0x7f1ae40a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:07.223 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.220+0000 7f1afd96e700 1 --2- 172.21.15.77:0/2528528124 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f1ae40a0570 0x7f1ae40a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:07.223 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.220+0000 7f1aef7fe700 1 -- 172.21.15.77:0/2528528124 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1af4014070 con 0x7f1af81001a0 2023-12-01T01:16:07.223 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.220+0000 7f1afd96e700 1 --2- 172.21.15.77:0/2528528124 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f1ae40a0570 0x7f1ae40a2a30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8005950 tx=0x7f1ae80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:07.225 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.224+0000 7f1aef7fe700 1 -- 172.21.15.77:0/2528528124 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1af4095770 con 0x7f1af81001a0 2023-12-01T01:16:07.461 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.460+0000 7f1b003d3700 1 -- 172.21.15.77:0/2528528124 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1af802cc70 con 0x7f1af81001a0 2023-12-01T01:16:07.462 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.460+0000 7f1aef7fe700 1 -- 172.21.15.77:0/2528528124 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4291 (secure 0 0 0) 0x7f1af4098d90 con 0x7f1af81001a0 2023-12-01T01:16:07.462 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:16:07.462 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1435145620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1435145620},{"type":"v1","addr":"172.21.15.77:6835","nonce":1435145620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/841838908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":841838908},{"type":"v1","addr":"172.21.15.77:6837","nonce":841838908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/2249638445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2249638445},{"type":"v1","addr":"172.21.15.137:6833","nonce":2249638445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/1201599774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1201599774},{"type":"v1","addr":"172.21.15.203:6833","nonce":1201599774}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3935354878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3935354878},{"type":"v1","addr":"172.21.15.137:6835","nonce":3935354878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2023-12-01T01:16:07.464 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.462+0000 7f1b003d3700 1 -- 172.21.15.77:0/2528528124 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f1ae40a0570 msgr2=0x7f1ae40a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:07.464 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.462+0000 7f1b003d3700 1 --2- 172.21.15.77:0/2528528124 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f1ae40a0570 0x7f1ae40a2a30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8005950 tx=0x7f1ae80058e0 comp rx=0 tx=0).stop 2023-12-01T01:16:07.464 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.462+0000 7f1b003d3700 1 -- 172.21.15.77:0/2528528124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1af81001a0 msgr2=0x7f1af80ff860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:07.464 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.462+0000 7f1b003d3700 1 --2- 172.21.15.77:0/2528528124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1af81001a0 0x7f1af80ff860 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1af400e970 tx=0x7f1af400ec80 comp rx=0 tx=0).stop 2023-12-01T01:16:07.464 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.462+0000 7f1b003d3700 1 -- 172.21.15.77:0/2528528124 shutdown_connections 2023-12-01T01:16:07.464 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.463+0000 7f1b003d3700 1 --2- 172.21.15.77:0/2528528124 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f1ae40a0570 0x7f1ae40a2a30 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:07.464 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.463+0000 7f1b003d3700 1 --2- 172.21.15.77:0/2528528124 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1af8101870 0x7f1af819de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:07.465 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.463+0000 7f1b003d3700 1 --2- 172.21.15.77:0/2528528124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1af81001a0 0x7f1af80ff860 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:07.465 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.463+0000 7f1b003d3700 1 --2- 172.21.15.77:0/2528528124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1af8100eb0 0x7f1af819b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:07.465 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.463+0000 7f1b003d3700 1 -- 172.21.15.77:0/2528528124 >> 172.21.15.77:0/2528528124 conn(0x7f1af80fa950 msgr2=0x7f1af80fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:07.465 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.463+0000 7f1b003d3700 1 -- 172.21.15.77:0/2528528124 shutdown_connections 2023-12-01T01:16:07.465 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:07.463+0000 7f1b003d3700 1 -- 172.21.15.77:0/2528528124 wait complete. 2023-12-01T01:16:07.465 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 6 2023-12-01T01:16:07.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:07 smithi203 ceph-mon[103581]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:16:07.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:07 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:07.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:07 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:07.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:07 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:07.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:07 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-01T01:16:07.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:07 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-01T01:16:07.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:07 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:16:07.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:07 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/2528528124' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:16:07.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:07 smithi077 ceph-mon[98867]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:16:07.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:07 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:07.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:07 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:07.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:07 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:07.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:07 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-01T01:16:07.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:07 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-01T01:16:07.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:07 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:16:07.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:07 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/2528528124' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:16:08.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:07 smithi137 ceph-mon[103312]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:16:08.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:07 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:08.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:07 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:08.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:07 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:08.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:07 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-01T01:16:08.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:07 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-01T01:16:08.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:07 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:16:08.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:07 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/2528528124' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:16:08.512 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:16:08 smithi203 systemd[1]: Starting Ceph mds.c for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:16:08.887 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:08 smithi203 ceph-mon[103581]: Deploying daemon mds.c on smithi203 2023-12-01T01:16:08.887 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:08 smithi203 ceph-mon[103581]: mds.? [v2:172.21.15.203:6834/3237538176,v1:172.21.15.203:6835/3237538176] up:boot 2023-12-01T01:16:08.888 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:08 smithi203 ceph-mon[103581]: fsmap 6 up:standby 2023-12-01T01:16:08.888 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:08 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-01T01:16:08.888 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:16:08 smithi203 podman[141033]: 2023-12-01T01:16:08.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:08 smithi077 ceph-mon[98867]: Deploying daemon mds.c on smithi203 2023-12-01T01:16:08.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:08 smithi077 ceph-mon[98867]: mds.? [v2:172.21.15.203:6834/3237538176,v1:172.21.15.203:6835/3237538176] up:boot 2023-12-01T01:16:08.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:08 smithi077 ceph-mon[98867]: fsmap 6 up:standby 2023-12-01T01:16:08.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:08 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-01T01:16:09.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:08 smithi137 ceph-mon[103312]: Deploying daemon mds.c on smithi203 2023-12-01T01:16:09.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:08 smithi137 ceph-mon[103312]: mds.? [v2:172.21.15.203:6834/3237538176,v1:172.21.15.203:6835/3237538176] up:boot 2023-12-01T01:16:09.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:08 smithi137 ceph-mon[103312]: fsmap 6 up:standby 2023-12-01T01:16:09.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:08 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-01T01:16:09.262 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:16:08 smithi203 ceph-mds[141073]: -- 172.21.15.203:0/233808344 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ea5defb1e0 con 0x55ea5ded1000 2023-12-01T01:16:09.262 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:16:08 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: starting mds.c at 2023-12-01T01:16:09.262 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:16:08 smithi203 bash[141033]: 1415fa67b514261298859ee3cddd1973e2f9180e4f562a1024c0e676aa10bbf4 2023-12-01T01:16:09.263 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:16:08 smithi203 systemd[1]: Started Ceph mds.c for 2a06e29a-8fe6-11ee-95a2-87774f69a715. 2023-12-01T01:16:09.476 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:16:09.614 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.610+0000 7fbe83e4f700 1 -- 172.21.15.77:0/2983471914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe7c101800 msgr2=0x7fbe7c103bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:09.614 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.610+0000 7fbe83e4f700 1 --2- 172.21.15.77:0/2983471914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe7c101800 0x7fbe7c103bf0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbe78009990 tx=0x7fbe78009ca0 comp rx=0 tx=0).stop 2023-12-01T01:16:09.614 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.610+0000 7fbe83e4f700 1 -- 172.21.15.77:0/2983471914 shutdown_connections 2023-12-01T01:16:09.615 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.610+0000 7fbe83e4f700 1 --2- 172.21.15.77:0/2983471914 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe7c106a60 0x7fbe7c108ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:09.615 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.610+0000 7fbe83e4f700 1 --2- 172.21.15.77:0/2983471914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe7c101800 0x7fbe7c103bf0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:09.615 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.610+0000 7fbe83e4f700 1 --2- 172.21.15.77:0/2983471914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe7c104130 0x7fbe7c106520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:09.615 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.610+0000 7fbe83e4f700 1 -- 172.21.15.77:0/2983471914 >> 172.21.15.77:0/2983471914 conn(0x7fbe7c0fb1f0 msgr2=0x7fbe7c0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:09.615 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.611+0000 7fbe83e4f700 1 -- 172.21.15.77:0/2983471914 shutdown_connections 2023-12-01T01:16:09.615 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.611+0000 7fbe83e4f700 1 -- 172.21.15.77:0/2983471914 wait complete. 2023-12-01T01:16:09.615 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.611+0000 7fbe83e4f700 1 Processor -- start 2023-12-01T01:16:09.615 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.611+0000 7fbe83e4f700 1 -- start start 2023-12-01T01:16:09.615 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.611+0000 7fbe83e4f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe7c101800 0x7fbe7c101170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:09.615 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.611+0000 7fbe83e4f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe7c104130 0x7fbe7c0ff7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:09.615 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.611+0000 7fbe83e4f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe7c106a60 0x7fbe7c0ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:09.615 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.611+0000 7fbe83e4f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7c100490 con 0x7fbe7c106a60 2023-12-01T01:16:09.616 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.611+0000 7fbe83e4f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7c100600 con 0x7fbe7c101800 2023-12-01T01:16:09.616 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.611+0000 7fbe83e4f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7c1a0a30 con 0x7fbe7c104130 2023-12-01T01:16:09.616 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.611+0000 7fbe823ec700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe7c106a60 0x7fbe7c0ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:09.616 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.611+0000 7fbe823ec700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe7c106a60 0x7fbe7c0ffcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57270/0 (socket says 172.21.15.77:57270) 2023-12-01T01:16:09.616 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.611+0000 7fbe823ec700 1 -- 172.21.15.77:0/972901233 learned_addr learned my addr 172.21.15.77:0/972901233 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:09.616 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.611+0000 7fbe81beb700 1 --2- 172.21.15.77:0/972901233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe7c101800 0x7fbe7c101170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:09.616 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.611+0000 7fbe813ea700 1 --2- 172.21.15.77:0/972901233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe7c104130 0x7fbe7c0ff7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:09.616 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.612+0000 7fbe823ec700 1 -- 172.21.15.77:0/972901233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe7c104130 msgr2=0x7fbe7c0ff7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:09.616 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.612+0000 7fbe823ec700 1 --2- 172.21.15.77:0/972901233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe7c104130 0x7fbe7c0ff7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:09.616 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.612+0000 7fbe823ec700 1 -- 172.21.15.77:0/972901233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe7c101800 msgr2=0x7fbe7c101170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:09.617 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.612+0000 7fbe823ec700 1 --2- 172.21.15.77:0/972901233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe7c101800 0x7fbe7c101170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:09.617 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.612+0000 7fbe823ec700 1 -- 172.21.15.77:0/972901233 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe780095f0 con 0x7fbe7c106a60 2023-12-01T01:16:09.617 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.612+0000 7fbe823ec700 1 --2- 172.21.15.77:0/972901233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe7c106a60 0x7fbe7c0ffcf0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbe7400ef30 tx=0x7fbe7400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:09.617 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.612+0000 7fbe6effd700 1 -- 172.21.15.77:0/972901233 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe74018070 con 0x7fbe7c106a60 2023-12-01T01:16:09.617 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.612+0000 7fbe6effd700 1 -- 172.21.15.77:0/972901233 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe7400f040 con 0x7fbe7c106a60 2023-12-01T01:16:09.617 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.612+0000 7fbe83e4f700 1 -- 172.21.15.77:0/972901233 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe7c1a0cb0 con 0x7fbe7c106a60 2023-12-01T01:16:09.617 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.612+0000 7fbe6effd700 1 -- 172.21.15.77:0/972901233 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7400cd00 con 0x7fbe7c106a60 2023-12-01T01:16:09.617 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.612+0000 7fbe83e4f700 1 -- 172.21.15.77:0/972901233 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe7c1a11c0 con 0x7fbe7c106a60 2023-12-01T01:16:09.617 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.613+0000 7fbe83e4f700 1 -- 172.21.15.77:0/972901233 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe7c0fcd70 con 0x7fbe7c106a60 2023-12-01T01:16:09.619 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.614+0000 7fbe6effd700 1 -- 172.21.15.77:0/972901233 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fbe74003a00 con 0x7fbe7c106a60 2023-12-01T01:16:09.619 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.615+0000 7fbe6effd700 1 --2- 172.21.15.77:0/972901233 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fbe680a05f0 0x7fbe680a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:09.619 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.615+0000 7fbe81beb700 1 --2- 172.21.15.77:0/972901233 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fbe680a05f0 0x7fbe680a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:09.619 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.615+0000 7fbe6effd700 1 -- 172.21.15.77:0/972901233 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbe740cb980 con 0x7fbe7c106a60 2023-12-01T01:16:09.619 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.615+0000 7fbe81beb700 1 --2- 172.21.15.77:0/972901233 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fbe680a05f0 0x7fbe680a2ab0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbe7800b2e0 tx=0x7fbe78005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:09.620 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.618+0000 7fbe6effd700 1 -- 172.21.15.77:0/972901233 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fbe74095a50 con 0x7fbe7c106a60 2023-12-01T01:16:09.854 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.853+0000 7fbe83e4f700 1 -- 172.21.15.77:0/972901233 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe7c02cc70 con 0x7fbe7c106a60 2023-12-01T01:16:09.855 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.854+0000 7fbe6effd700 1 -- 172.21.15.77:0/972901233 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5811 (secure 0 0 0) 0x7fbe74099070 con 0x7fbe7c106a60 2023-12-01T01:16:09.855 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:16:09.855 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1435145620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1435145620},{"type":"v1","addr":"172.21.15.77:6835","nonce":1435145620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/841838908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":841838908},{"type":"v1","addr":"172.21.15.77:6837","nonce":841838908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/2249638445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2249638445},{"type":"v1","addr":"172.21.15.137:6833","nonce":2249638445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/1201599774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1201599774},{"type":"v1","addr":"172.21.15.203:6833","nonce":1201599774}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3935354878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3935354878},{"type":"v1","addr":"172.21.15.137:6835","nonce":3935354878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3237538176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3237538176},{"type":"v1","addr":"172.21.15.203:6835","nonce":3237538176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2887080977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2887080977},{"type":"v1","addr":"172.21.15.203:6837","nonce":2887080977}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2023-12-01T01:16:09.857 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.856+0000 7fbe83e4f700 1 -- 172.21.15.77:0/972901233 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fbe680a05f0 msgr2=0x7fbe680a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:09.857 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.856+0000 7fbe83e4f700 1 --2- 172.21.15.77:0/972901233 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fbe680a05f0 0x7fbe680a2ab0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbe7800b2e0 tx=0x7fbe78005fb0 comp rx=0 tx=0).stop 2023-12-01T01:16:09.857 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.856+0000 7fbe83e4f700 1 -- 172.21.15.77:0/972901233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe7c106a60 msgr2=0x7fbe7c0ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:09.857 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.856+0000 7fbe83e4f700 1 --2- 172.21.15.77:0/972901233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe7c106a60 0x7fbe7c0ffcf0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbe7400ef30 tx=0x7fbe7400c3a0 comp rx=0 tx=0).stop 2023-12-01T01:16:09.857 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.856+0000 7fbe83e4f700 1 -- 172.21.15.77:0/972901233 shutdown_connections 2023-12-01T01:16:09.857 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.856+0000 7fbe83e4f700 1 --2- 172.21.15.77:0/972901233 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fbe680a05f0 0x7fbe680a2ab0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:09.858 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.856+0000 7fbe83e4f700 1 --2- 172.21.15.77:0/972901233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe7c106a60 0x7fbe7c0ffcf0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:09.858 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.856+0000 7fbe83e4f700 1 --2- 172.21.15.77:0/972901233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe7c101800 0x7fbe7c101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:09.858 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.856+0000 7fbe83e4f700 1 --2- 172.21.15.77:0/972901233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe7c104130 0x7fbe7c0ff7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:09.858 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.856+0000 7fbe83e4f700 1 -- 172.21.15.77:0/972901233 >> 172.21.15.77:0/972901233 conn(0x7fbe7c0fb1f0 msgr2=0x7fbe7c102470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:09.858 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.856+0000 7fbe83e4f700 1 -- 172.21.15.77:0/972901233 shutdown_connections 2023-12-01T01:16:09.858 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:09.856+0000 7fbe83e4f700 1 -- 172.21.15.77:0/972901233 wait complete. 2023-12-01T01:16:09.858 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 8 2023-12-01T01:16:09.868 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:09 smithi077 ceph-mon[98867]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:16:09.868 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:09 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:09.868 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:09 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:09.868 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:09 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:09.868 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:09 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-01T01:16:09.868 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:09 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-01T01:16:09.868 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:09 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:16:09.908 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:09 smithi137 ceph-mon[103312]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:16:09.909 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:09 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:09.909 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:09 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:09.909 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:09 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:09.909 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:09 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-01T01:16:09.909 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:09 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-01T01:16:09.909 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:09 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:16:10.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:09 smithi203 ceph-mon[103581]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:16:10.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:09 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:10.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:09 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:10.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:09 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:10.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:09 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-01T01:16:10.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:09 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-01T01:16:10.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:09 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:16:10.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:10 smithi077 ceph-mon[98867]: Deploying daemon mds.h on smithi137 2023-12-01T01:16:10.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:10 smithi077 ceph-mon[98867]: mds.? [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] up:boot 2023-12-01T01:16:10.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:10 smithi077 ceph-mon[98867]: fsmap 7 up:standby 2023-12-01T01:16:10.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:10 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-01T01:16:10.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:10 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/972901233' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:16:10.934 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:10 smithi137 ceph-mon[103312]: Deploying daemon mds.h on smithi137 2023-12-01T01:16:10.934 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:10 smithi137 ceph-mon[103312]: mds.? [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] up:boot 2023-12-01T01:16:10.934 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:10 smithi137 ceph-mon[103312]: fsmap 7 up:standby 2023-12-01T01:16:10.934 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:10 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-01T01:16:10.934 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:10 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/972901233' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:16:10.934 INFO:journalctl@ceph.mds.h.smithi137.stdout:Dec 01 01:16:10 smithi137 systemd[1]: Starting Ceph mds.h for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:16:11.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:10 smithi203 ceph-mon[103581]: Deploying daemon mds.h on smithi137 2023-12-01T01:16:11.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:10 smithi203 ceph-mon[103581]: mds.? [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] up:boot 2023-12-01T01:16:11.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:10 smithi203 ceph-mon[103581]: fsmap 7 up:standby 2023-12-01T01:16:11.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:10 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-01T01:16:11.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:10 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/972901233' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:16:11.199 INFO:journalctl@ceph.mds.h.smithi137.stdout:Dec 01 01:16:10 smithi137 podman[141008]: 2023-12-01T01:16:11.529 INFO:journalctl@ceph.mds.h.smithi137.stdout:Dec 01 01:16:11 smithi137 ceph-mds[141056]: -- 172.21.15.137:0/3117795305 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b4fe4951e0 con 0x55b4fe46a800 2023-12-01T01:16:11.529 INFO:journalctl@ceph.mds.h.smithi137.stdout:Dec 01 01:16:11 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-h[141032]: starting mds.h at 2023-12-01T01:16:11.529 INFO:journalctl@ceph.mds.h.smithi137.stdout:Dec 01 01:16:11 smithi137 bash[141008]: e3dafb8bd166b03971aa119c5b3557040d6aa88b407e5badcbe1c33e876b301d 2023-12-01T01:16:11.529 INFO:journalctl@ceph.mds.h.smithi137.stdout:Dec 01 01:16:11 smithi137 systemd[1]: Started Ceph mds.h for 2a06e29a-8fe6-11ee-95a2-87774f69a715. 2023-12-01T01:16:11.870 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:16:11.899 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:11 smithi077 ceph-mon[98867]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:16:11.899 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:11 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:11.899 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:11 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:11.899 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:11 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:11.899 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:11 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-01T01:16:11.899 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:11 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-01T01:16:11.900 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:11 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:16:11.944 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:11 smithi203 ceph-mon[103581]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:16:11.944 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:11 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:11.944 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:11 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:11.944 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:11 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:11.945 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:11 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-01T01:16:11.945 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:11 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-01T01:16:11.945 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:11 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:16:12.007 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.004+0000 7fd7cb153700 1 -- 172.21.15.77:0/3186744392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7c4100e60 msgr2=0x7fd7c41012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:12.007 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.004+0000 7fd7cb153700 1 --2- 172.21.15.77:0/3186744392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7c4100e60 0x7fd7c41012c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd7b00099c0 tx=0x7fd7b0009cd0 comp rx=0 tx=0).stop 2023-12-01T01:16:12.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.004+0000 7fd7cb153700 1 -- 172.21.15.77:0/3186744392 shutdown_connections 2023-12-01T01:16:12.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.004+0000 7fd7cb153700 1 --2- 172.21.15.77:0/3186744392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7c4101800 0x7fd7c41080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:12.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.004+0000 7fd7cb153700 1 --2- 172.21.15.77:0/3186744392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7c40ffc70 0x7fd7c4100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:12.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.004+0000 7fd7cb153700 1 --2- 172.21.15.77:0/3186744392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7c4100e60 0x7fd7c41012c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:12.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.004+0000 7fd7cb153700 1 -- 172.21.15.77:0/3186744392 >> 172.21.15.77:0/3186744392 conn(0x7fd7c40fb1c0 msgr2=0x7fd7c40fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:12.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.004+0000 7fd7cb153700 1 -- 172.21.15.77:0/3186744392 shutdown_connections 2023-12-01T01:16:12.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.004+0000 7fd7cb153700 1 -- 172.21.15.77:0/3186744392 wait complete. 2023-12-01T01:16:12.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.004+0000 7fd7cb153700 1 Processor -- start 2023-12-01T01:16:12.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.004+0000 7fd7cb153700 1 -- start start 2023-12-01T01:16:12.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.004+0000 7fd7cb153700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7c40ffc70 0x7fd7c4195910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:12.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.004+0000 7fd7cb153700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7c4100e60 0x7fd7c4195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:12.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.004+0000 7fd7cb153700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7c4101800 0x7fd7c419cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:12.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.004+0000 7fd7cb153700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c419d490 con 0x7fd7c4100e60 2023-12-01T01:16:12.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.004+0000 7fd7cb153700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c419d5d0 con 0x7fd7c4101800 2023-12-01T01:16:12.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.004+0000 7fd7cb153700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c419d760 con 0x7fd7c40ffc70 2023-12-01T01:16:12.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.005+0000 7fd7c3fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7c4100e60 0x7fd7c4195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:12.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.005+0000 7fd7c8eef700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7c40ffc70 0x7fd7c4195910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:12.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.005+0000 7fd7c3fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7c4100e60 0x7fd7c4195e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57282/0 (socket says 172.21.15.77:57282) 2023-12-01T01:16:12.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.005+0000 7fd7c3fff700 1 -- 172.21.15.77:0/1723194475 learned_addr learned my addr 172.21.15.77:0/1723194475 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:12.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.005+0000 7fd7c96f0700 1 --2- 172.21.15.77:0/1723194475 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7c4101800 0x7fd7c419cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:12.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.005+0000 7fd7c3fff700 1 -- 172.21.15.77:0/1723194475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7c40ffc70 msgr2=0x7fd7c4195910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:12.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.005+0000 7fd7c3fff700 1 --2- 172.21.15.77:0/1723194475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7c40ffc70 0x7fd7c4195910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:12.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.005+0000 7fd7c3fff700 1 -- 172.21.15.77:0/1723194475 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7c4101800 msgr2=0x7fd7c419cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:12.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.005+0000 7fd7c3fff700 1 --2- 172.21.15.77:0/1723194475 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7c4101800 0x7fd7c419cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:12.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.005+0000 7fd7c3fff700 1 -- 172.21.15.77:0/1723194475 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b00095f0 con 0x7fd7c4100e60 2023-12-01T01:16:12.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.005+0000 7fd7c96f0700 1 --2- 172.21.15.77:0/1723194475 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7c4101800 0x7fd7c419cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:16:12.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.005+0000 7fd7c3fff700 1 --2- 172.21.15.77:0/1723194475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7c4100e60 0x7fd7c4195e50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0009990 tx=0x7fd7b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:12.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.005+0000 7fd7c1ffb700 1 -- 172.21.15.77:0/1723194475 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b001d070 con 0x7fd7c4100e60 2023-12-01T01:16:12.010 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.005+0000 7fd7c1ffb700 1 -- 172.21.15.77:0/1723194475 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7b0004090 con 0x7fd7c4100e60 2023-12-01T01:16:12.010 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.005+0000 7fd7c1ffb700 1 -- 172.21.15.77:0/1723194475 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0004a00 con 0x7fd7c4100e60 2023-12-01T01:16:12.010 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.005+0000 7fd7cb153700 1 -- 172.21.15.77:0/1723194475 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7c419d9e0 con 0x7fd7c4100e60 2023-12-01T01:16:12.010 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.005+0000 7fd7cb153700 1 -- 172.21.15.77:0/1723194475 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7c419de40 con 0x7fd7c4100e60 2023-12-01T01:16:12.010 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.006+0000 7fd7cb153700 1 -- 172.21.15.77:0/1723194475 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7c410b480 con 0x7fd7c4100e60 2023-12-01T01:16:12.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.007+0000 7fd7c1ffb700 1 -- 172.21.15.77:0/1723194475 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fd7b0003bd0 con 0x7fd7c4100e60 2023-12-01T01:16:12.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.008+0000 7fd7c1ffb700 1 --2- 172.21.15.77:0/1723194475 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd7b40a0570 0x7fd7b40a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:12.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.008+0000 7fd7c1ffb700 1 -- 172.21.15.77:0/1723194475 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd7b00cca00 con 0x7fd7c4100e60 2023-12-01T01:16:12.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.008+0000 7fd7c8eef700 1 --2- 172.21.15.77:0/1723194475 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd7b40a0570 0x7fd7b40a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:12.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.008+0000 7fd7c8eef700 1 --2- 172.21.15.77:0/1723194475 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd7b40a0570 0x7fd7b40a2a30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4100cc0 tx=0x7fd7b800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:12.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.012+0000 7fd7c1ffb700 1 -- 172.21.15.77:0/1723194475 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd7b0096a30 con 0x7fd7c4100e60 2023-12-01T01:16:12.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:11 smithi137 ceph-mon[103312]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:16:12.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:11 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:12.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:11 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:12.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:11 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:12.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:11 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-01T01:16:12.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:11 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-01T01:16:12.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:11 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:16:12.250 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.249+0000 7fd7cb153700 1 -- 172.21.15.77:0/1723194475 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7c4106ba0 con 0x7fd7c4100e60 2023-12-01T01:16:12.251 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.250+0000 7fd7c1ffb700 1 -- 172.21.15.77:0/1723194475 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6571 (secure 0 0 0) 0x7fd7b009a050 con 0x7fd7c4100e60 2023-12-01T01:16:12.251 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:16:12.251 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1435145620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1435145620},{"type":"v1","addr":"172.21.15.77:6835","nonce":1435145620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/841838908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":841838908},{"type":"v1","addr":"172.21.15.77:6837","nonce":841838908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/2249638445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2249638445},{"type":"v1","addr":"172.21.15.137:6833","nonce":2249638445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/1201599774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1201599774},{"type":"v1","addr":"172.21.15.203:6833","nonce":1201599774}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3935354878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3935354878},{"type":"v1","addr":"172.21.15.137:6835","nonce":3935354878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3237538176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3237538176},{"type":"v1","addr":"172.21.15.203:6835","nonce":3237538176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24473,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/3518881867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3518881867},{"type":"v1","addr":"172.21.15.137:6837","nonce":3518881867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2887080977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2887080977},{"type":"v1","addr":"172.21.15.203:6837","nonce":2887080977}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2023-12-01T01:16:12.253 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.252+0000 7fd7cb153700 1 -- 172.21.15.77:0/1723194475 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd7b40a0570 msgr2=0x7fd7b40a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:12.253 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.252+0000 7fd7cb153700 1 --2- 172.21.15.77:0/1723194475 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd7b40a0570 0x7fd7b40a2a30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4100cc0 tx=0x7fd7b800b410 comp rx=0 tx=0).stop 2023-12-01T01:16:12.254 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.252+0000 7fd7cb153700 1 -- 172.21.15.77:0/1723194475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7c4100e60 msgr2=0x7fd7c4195e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:12.254 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.252+0000 7fd7cb153700 1 --2- 172.21.15.77:0/1723194475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7c4100e60 0x7fd7c4195e50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0009990 tx=0x7fd7b0004730 comp rx=0 tx=0).stop 2023-12-01T01:16:12.254 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.252+0000 7fd7cb153700 1 -- 172.21.15.77:0/1723194475 shutdown_connections 2023-12-01T01:16:12.254 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.252+0000 7fd7cb153700 1 --2- 172.21.15.77:0/1723194475 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd7b40a0570 0x7fd7b40a2a30 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:12.254 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.252+0000 7fd7cb153700 1 --2- 172.21.15.77:0/1723194475 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd7c4101800 0x7fd7c419cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:12.254 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.252+0000 7fd7cb153700 1 --2- 172.21.15.77:0/1723194475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7c40ffc70 0x7fd7c4195910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:12.254 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.252+0000 7fd7cb153700 1 --2- 172.21.15.77:0/1723194475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd7c4100e60 0x7fd7c4195e50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:12.254 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.252+0000 7fd7cb153700 1 -- 172.21.15.77:0/1723194475 >> 172.21.15.77:0/1723194475 conn(0x7fd7c40fb1c0 msgr2=0x7fd7c40fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:12.254 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.252+0000 7fd7cb153700 1 -- 172.21.15.77:0/1723194475 shutdown_connections 2023-12-01T01:16:12.254 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:12.252+0000 7fd7cb153700 1 -- 172.21.15.77:0/1723194475 wait complete. 2023-12-01T01:16:12.255 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 9 2023-12-01T01:16:12.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:12 smithi077 ceph-mon[98867]: Deploying daemon mds.i on smithi203 2023-12-01T01:16:12.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:12 smithi077 ceph-mon[98867]: mds.? [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] up:boot 2023-12-01T01:16:12.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:12 smithi077 ceph-mon[98867]: fsmap 8 up:standby 2023-12-01T01:16:12.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:12 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-01T01:16:12.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:12 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/1723194475' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:16:13.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:12 smithi203 ceph-mon[103581]: Deploying daemon mds.i on smithi203 2023-12-01T01:16:13.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:12 smithi203 ceph-mon[103581]: mds.? [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] up:boot 2023-12-01T01:16:13.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:12 smithi203 ceph-mon[103581]: fsmap 8 up:standby 2023-12-01T01:16:13.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:12 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-01T01:16:13.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:12 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/1723194475' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:16:13.014 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:16:12 smithi203 systemd[1]: Starting Ceph mds.i for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:16:13.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:12 smithi137 ceph-mon[103312]: Deploying daemon mds.i on smithi203 2023-12-01T01:16:13.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:12 smithi137 ceph-mon[103312]: mds.? [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] up:boot 2023-12-01T01:16:13.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:12 smithi137 ceph-mon[103312]: fsmap 8 up:standby 2023-12-01T01:16:13.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:12 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-01T01:16:13.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:12 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/1723194475' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:16:13.349 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:16:13 smithi203 podman[141370]: 2023-12-01T01:16:13.618 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:16:13 smithi203 ceph-mds[141410]: -- 172.21.15.203:0/1495195516 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563634daf1e0 con 0x563634d84800 2023-12-01T01:16:13.618 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:16:13 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: starting mds.i at 2023-12-01T01:16:13.618 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:16:13 smithi203 bash[141370]: 6d5196850fb8cfc91e59bb00927b32cc6a934d540a5454ceca70c8ea87c5384f 2023-12-01T01:16:13.618 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:16:13 smithi203 systemd[1]: Started Ceph mds.i for 2a06e29a-8fe6-11ee-95a2-87774f69a715. 2023-12-01T01:16:13.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:13 smithi077 ceph-mon[98867]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:16:13.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:13 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:13.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:13 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:13.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:13 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:13.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:13 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-01T01:16:13.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:13 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-01T01:16:13.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:13 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:16:14.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:13 smithi203 ceph-mon[103581]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:16:14.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:13 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:14.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:13 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:14.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:13 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:14.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:13 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-01T01:16:14.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:13 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-01T01:16:14.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:13 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:16:14.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:13 smithi137 ceph-mon[103312]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:16:14.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:13 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:14.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:13 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:14.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:13 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:14.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:13 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-01T01:16:14.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:13 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-01T01:16:14.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:13 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:16:14.266 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:16:14.400 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.397+0000 7f7087df2700 1 -- 172.21.15.77:0/1682636736 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7080101830 msgr2=0x7f70801080e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:14.400 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.397+0000 7f7087df2700 1 --2- 172.21.15.77:0/1682636736 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7080101830 0x7f70801080e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f70780099c0 tx=0x7f7078009cd0 comp rx=0 tx=0).stop 2023-12-01T01:16:14.401 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.397+0000 7f7087df2700 1 -- 172.21.15.77:0/1682636736 shutdown_connections 2023-12-01T01:16:14.401 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.397+0000 7f7087df2700 1 --2- 172.21.15.77:0/1682636736 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7080101830 0x7f70801080e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:14.401 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.397+0000 7f7087df2700 1 --2- 172.21.15.77:0/1682636736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70800ffca0 0x7f70801000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:14.401 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.397+0000 7f7087df2700 1 --2- 172.21.15.77:0/1682636736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7080100e90 0x7f70801012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:14.401 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.397+0000 7f7087df2700 1 -- 172.21.15.77:0/1682636736 >> 172.21.15.77:0/1682636736 conn(0x7f70800fb210 msgr2=0x7f70800fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:14.401 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.397+0000 7f7087df2700 1 -- 172.21.15.77:0/1682636736 shutdown_connections 2023-12-01T01:16:14.401 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.397+0000 7f7087df2700 1 -- 172.21.15.77:0/1682636736 wait complete. 2023-12-01T01:16:14.401 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.398+0000 7f7087df2700 1 Processor -- start 2023-12-01T01:16:14.401 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.398+0000 7f7087df2700 1 -- start start 2023-12-01T01:16:14.401 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.398+0000 7f7087df2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70800ffca0 0x7f7080069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:14.401 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.398+0000 7f7087df2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7080100e90 0x7f708019b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:14.401 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.398+0000 7f7087df2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7080101830 0x7f708019dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:14.401 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.398+0000 7f7087df2700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708019e530 con 0x7f7080100e90 2023-12-01T01:16:14.401 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.398+0000 7f7087df2700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708019e6a0 con 0x7f70800ffca0 2023-12-01T01:16:14.402 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.398+0000 7f7087df2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708019e860 con 0x7f7080101830 2023-12-01T01:16:14.402 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.398+0000 7f708538d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7080100e90 0x7f708019b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:14.402 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.398+0000 7f708538d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7080100e90 0x7f708019b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57294/0 (socket says 172.21.15.77:57294) 2023-12-01T01:16:14.402 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.398+0000 7f708538d700 1 -- 172.21.15.77:0/2036261576 learned_addr learned my addr 172.21.15.77:0/2036261576 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:14.402 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.398+0000 7f708638f700 1 --2- 172.21.15.77:0/2036261576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7080101830 0x7f708019dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:14.402 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.398+0000 7f7085b8e700 1 --2- 172.21.15.77:0/2036261576 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70800ffca0 0x7f7080069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:14.402 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.398+0000 7f708538d700 1 -- 172.21.15.77:0/2036261576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7080101830 msgr2=0x7f708019dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:14.402 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.398+0000 7f708538d700 1 --2- 172.21.15.77:0/2036261576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7080101830 0x7f708019dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:14.402 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.398+0000 7f708538d700 1 -- 172.21.15.77:0/2036261576 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70800ffca0 msgr2=0x7f7080069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:14.402 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.398+0000 7f708538d700 1 --2- 172.21.15.77:0/2036261576 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70800ffca0 0x7f7080069ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:14.402 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.398+0000 7f708538d700 1 -- 172.21.15.77:0/2036261576 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70780095f0 con 0x7f7080100e90 2023-12-01T01:16:14.402 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.398+0000 7f7085b8e700 1 --2- 172.21.15.77:0/2036261576 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70800ffca0 0x7f7080069ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:16:14.402 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.399+0000 7f708538d700 1 --2- 172.21.15.77:0/2036261576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7080100e90 0x7f708019b710 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f707400d950 tx=0x7f707400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:14.403 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.399+0000 7f707effd700 1 -- 172.21.15.77:0/2036261576 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7074009940 con 0x7f7080100e90 2023-12-01T01:16:14.403 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.399+0000 7f7087df2700 1 -- 172.21.15.77:0/2036261576 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f708019eb40 con 0x7f7080100e90 2023-12-01T01:16:14.403 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.399+0000 7f707effd700 1 -- 172.21.15.77:0/2036261576 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7074010460 con 0x7f7080100e90 2023-12-01T01:16:14.403 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.399+0000 7f707effd700 1 -- 172.21.15.77:0/2036261576 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707400f620 con 0x7f7080100e90 2023-12-01T01:16:14.403 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.399+0000 7f7087df2700 1 -- 172.21.15.77:0/2036261576 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f708019f050 con 0x7f7080100e90 2023-12-01T01:16:14.404 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.400+0000 7f7087df2700 1 -- 172.21.15.77:0/2036261576 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7080107140 con 0x7f7080100e90 2023-12-01T01:16:14.404 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.401+0000 7f707effd700 1 -- 172.21.15.77:0/2036261576 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f70740108d0 con 0x7f7080100e90 2023-12-01T01:16:14.404 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.401+0000 7f707effd700 1 --2- 172.21.15.77:0/2036261576 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f70680a0570 0x7f70680a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:14.404 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.401+0000 7f7085b8e700 1 --2- 172.21.15.77:0/2036261576 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f70680a0570 0x7f70680a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:14.404 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.402+0000 7f707effd700 1 -- 172.21.15.77:0/2036261576 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f70740cb310 con 0x7f7080100e90 2023-12-01T01:16:14.404 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.402+0000 7f7085b8e700 1 --2- 172.21.15.77:0/2036261576 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f70680a0570 0x7f70680a2a30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7080100cf0 tx=0x7f70700073c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:14.406 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.405+0000 7f707effd700 1 -- 172.21.15.77:0/2036261576 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7074095350 con 0x7f7080100e90 2023-12-01T01:16:14.647 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.646+0000 7f7087df2700 1 -- 172.21.15.77:0/2036261576 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f708002cc70 con 0x7f7080100e90 2023-12-01T01:16:14.648 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.647+0000 7f707effd700 1 -- 172.21.15.77:0/2036261576 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7333 (secure 0 0 0) 0x7f7074098970 con 0x7f7080100e90 2023-12-01T01:16:14.648 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:16:14.648 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1435145620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1435145620},{"type":"v1","addr":"172.21.15.77:6835","nonce":1435145620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/841838908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":841838908},{"type":"v1","addr":"172.21.15.77:6837","nonce":841838908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/2249638445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2249638445},{"type":"v1","addr":"172.21.15.137:6833","nonce":2249638445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/1201599774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1201599774},{"type":"v1","addr":"172.21.15.203:6833","nonce":1201599774}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3935354878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3935354878},{"type":"v1","addr":"172.21.15.137:6835","nonce":3935354878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3237538176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3237538176},{"type":"v1","addr":"172.21.15.203:6835","nonce":3237538176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24473,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/3518881867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3518881867},{"type":"v1","addr":"172.21.15.137:6837","nonce":3518881867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2887080977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2887080977},{"type":"v1","addr":"172.21.15.203:6837","nonce":2887080977}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2888221613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2888221613},{"type":"v1","addr":"172.21.15.203:6839","nonce":2888221613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-12-01T01:16:14.650 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.649+0000 7f7087df2700 1 -- 172.21.15.77:0/2036261576 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f70680a0570 msgr2=0x7f70680a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:14.650 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.649+0000 7f7087df2700 1 --2- 172.21.15.77:0/2036261576 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f70680a0570 0x7f70680a2a30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7080100cf0 tx=0x7f70700073c0 comp rx=0 tx=0).stop 2023-12-01T01:16:14.650 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.649+0000 7f7087df2700 1 -- 172.21.15.77:0/2036261576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7080100e90 msgr2=0x7f708019b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:14.651 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.649+0000 7f7087df2700 1 --2- 172.21.15.77:0/2036261576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7080100e90 0x7f708019b710 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f707400d950 tx=0x7f707400dc60 comp rx=0 tx=0).stop 2023-12-01T01:16:14.651 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.649+0000 7f7087df2700 1 -- 172.21.15.77:0/2036261576 shutdown_connections 2023-12-01T01:16:14.651 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.649+0000 7f7087df2700 1 --2- 172.21.15.77:0/2036261576 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f70680a0570 0x7f70680a2a30 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:14.651 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.649+0000 7f7087df2700 1 --2- 172.21.15.77:0/2036261576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7080101830 0x7f708019dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:14.651 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.649+0000 7f7087df2700 1 --2- 172.21.15.77:0/2036261576 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70800ffca0 0x7f7080069ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:14.651 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.649+0000 7f7087df2700 1 --2- 172.21.15.77:0/2036261576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7080100e90 0x7f708019b710 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:14.651 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.649+0000 7f7087df2700 1 -- 172.21.15.77:0/2036261576 >> 172.21.15.77:0/2036261576 conn(0x7f70800fb210 msgr2=0x7f7080104280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:14.651 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.649+0000 7f7087df2700 1 -- 172.21.15.77:0/2036261576 shutdown_connections 2023-12-01T01:16:14.651 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:14.649+0000 7f7087df2700 1 -- 172.21.15.77:0/2036261576 wait complete. 2023-12-01T01:16:14.651 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 10 2023-12-01T01:16:14.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:14 smithi077 ceph-mon[98867]: Deploying daemon mds.g on smithi077 2023-12-01T01:16:14.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:14 smithi077 ceph-mon[98867]: mds.? [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] up:boot 2023-12-01T01:16:14.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:14 smithi077 ceph-mon[98867]: fsmap 9 up:standby 2023-12-01T01:16:14.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:14 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-01T01:16:15.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:14 smithi203 ceph-mon[103581]: Deploying daemon mds.g on smithi077 2023-12-01T01:16:15.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:14 smithi203 ceph-mon[103581]: mds.? [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] up:boot 2023-12-01T01:16:15.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:14 smithi203 ceph-mon[103581]: fsmap 9 up:standby 2023-12-01T01:16:15.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:14 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-01T01:16:15.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:14 smithi137 ceph-mon[103312]: Deploying daemon mds.g on smithi077 2023-12-01T01:16:15.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:14 smithi137 ceph-mon[103312]: mds.? [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] up:boot 2023-12-01T01:16:15.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:14 smithi137 ceph-mon[103312]: fsmap 9 up:standby 2023-12-01T01:16:15.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:14 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-01T01:16:15.222 INFO:journalctl@ceph.mds.g.smithi077.stdout:Dec 01 01:16:14 smithi077 systemd[1]: Starting Ceph mds.g for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:16:15.514 INFO:journalctl@ceph.mds.g.smithi077.stdout:Dec 01 01:16:15 smithi077 podman[149902]: 2023-12-01T01:16:15.515 INFO:journalctl@ceph.mds.g.smithi077.stdout:Dec 01 01:16:15 smithi077 ceph-mds[149942]: -- 172.21.15.77:0/2067257714 <== mon.0 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f60c9131e0 con 0x55f60c8e8800 2023-12-01T01:16:15.515 INFO:journalctl@ceph.mds.g.smithi077.stdout:Dec 01 01:16:15 smithi077 bash[149902]: 6472af40a78327546839950728bc9cc514da22ffecc38c1be31e9901ec48e9d7 2023-12-01T01:16:15.515 INFO:journalctl@ceph.mds.g.smithi077.stdout:Dec 01 01:16:15 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-g[149918]: starting mds.g at 2023-12-01T01:16:15.904 INFO:journalctl@ceph.mds.g.smithi077.stdout:Dec 01 01:16:15 smithi077 systemd[1]: Started Ceph mds.g for 2a06e29a-8fe6-11ee-95a2-87774f69a715. 2023-12-01T01:16:15.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:15 smithi077 ceph-mon[98867]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:16:15.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:15 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/2036261576' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:16:15.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:15 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:15.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:15 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:15.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:15 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:15.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:15 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-01T01:16:15.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:15 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-01T01:16:15.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:15 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:16:16.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:15 smithi203 ceph-mon[103581]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:16:16.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:15 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/2036261576' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:16:16.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:15 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:16.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:15 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:16.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:15 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:16.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:15 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-01T01:16:16.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:15 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-01T01:16:16.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:15 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:16:16.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:15 smithi137 ceph-mon[103312]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:16:16.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:15 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/2036261576' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:16:16.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:15 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:16.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:15 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:16.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:15 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:16.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:15 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-01T01:16:16.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:15 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-01T01:16:16.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:15 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:16:16.663 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:16:16.793 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:16.790+0000 7f0620e4c700 1 -- 172.21.15.77:0/4169062522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f061c0ffc70 msgr2=0x7f061c100090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:16.794 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:16.790+0000 7f0620e4c700 1 --2- 172.21.15.77:0/4169062522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f061c0ffc70 0x7f061c100090 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f060c0099c0 tx=0x7f060c009cd0 comp rx=0 tx=0).stop 2023-12-01T01:16:16.794 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:16.790+0000 7f0620e4c700 1 -- 172.21.15.77:0/4169062522 shutdown_connections 2023-12-01T01:16:16.794 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:16.790+0000 7f0620e4c700 1 --2- 172.21.15.77:0/4169062522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f061c101800 0x7f061c1080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:16.794 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:16.790+0000 7f0620e4c700 1 --2- 172.21.15.77:0/4169062522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f061c0ffc70 0x7f061c100090 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:16.794 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:16.790+0000 7f0620e4c700 1 --2- 172.21.15.77:0/4169062522 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f061c100e60 0x7f061c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:16.794 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:16.790+0000 7f0620e4c700 1 -- 172.21.15.77:0/4169062522 >> 172.21.15.77:0/4169062522 conn(0x7f061c0fb1c0 msgr2=0x7f061c0fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:16.794 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:16.790+0000 7f0620e4c700 1 -- 172.21.15.77:0/4169062522 shutdown_connections 2023-12-01T01:16:16.794 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:16.790+0000 7f0620e4c700 1 -- 172.21.15.77:0/4169062522 wait complete. 2023-12-01T01:16:16.794 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:16.791+0000 7f0620e4c700 1 Processor -- start 2023-12-01T01:16:16.794 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:16.791+0000 7f0620e4c700 1 -- start start 2023-12-01T01:16:16.794 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:16.791+0000 7f0620e4c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f061c0ffc70 0x7f061c068a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:16.795 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:16.791+0000 7f0620e4c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f061c100e60 0x7f061c19dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:16.795 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:16.791+0000 7f0620e4c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f061c101800 0x7f061c1a0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:16.795 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:16.791+0000 7f0620e4c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c1a07c0 con 0x7f061c100e60 2023-12-01T01:16:16.795 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:16.791+0000 7f0620e4c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c1a0930 con 0x7f061c101800 2023-12-01T01:16:16.795 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:16.791+0000 7f0620e4c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c1a0af0 con 0x7f061c0ffc70 2023-12-01T01:16:16.795 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:16.791+0000 7f0619d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f061c100e60 0x7f061c19dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:16.795 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:16.791+0000 7f0619d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f061c100e60 0x7f061c19dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57320/0 (socket says 172.21.15.77:57320) 2023-12-01T01:16:16.795 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:16.791+0000 7f0619d9b700 1 -- 172.21.15.77:0/209084694 learned_addr learned my addr 172.21.15.77:0/209084694 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:16.795 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:16.791+0000 7f061a59c700 1 --2- 172.21.15.77:0/209084694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f061c0ffc70 0x7f061c068a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:16.795 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:16.792+0000 7f061ad9d700 1 --2- 172.21.15.77:0/209084694 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f061c101800 0x7f061c1a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:16.795 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:16.792+0000 7f0619d9b700 1 -- 172.21.15.77:0/209084694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f061c0ffc70 msgr2=0x7f061c068a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:16.795 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:16.792+0000 7f0619d9b700 1 --2- 172.21.15.77:0/209084694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f061c0ffc70 0x7f061c068a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:16.795 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:16.792+0000 7f0619d9b700 1 -- 172.21.15.77:0/209084694 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f061c101800 msgr2=0x7f061c1a0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:16.796 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:16.792+0000 7f0619d9b700 1 --2- 172.21.15.77:0/209084694 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f061c101800 0x7f061c1a0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:16.796 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:16.792+0000 7f0619d9b700 1 -- 172.21.15.77:0/209084694 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f060c0095f0 con 0x7f061c100e60 2023-12-01T01:16:16.796 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:16.792+0000 7f061ad9d700 1 --2- 172.21.15.77:0/209084694 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f061c101800 0x7f061c1a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:16:16.796 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:16.792+0000 7f0619d9b700 1 --2- 172.21.15.77:0/209084694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f061c100e60 0x7f061c19dbc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f060400d950 tx=0x7f060400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:16.796 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:16.792+0000 7f06137fe700 1 -- 172.21.15.77:0/209084694 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0604009940 con 0x7f061c100e60 2023-12-01T01:16:16.796 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:16.792+0000 7f06137fe700 1 -- 172.21.15.77:0/209084694 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0604010460 con 0x7f061c100e60 2023-12-01T01:16:16.796 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:16.792+0000 7f0620e4c700 1 -- 172.21.15.77:0/209084694 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f061c1a0dd0 con 0x7f061c100e60 2023-12-01T01:16:16.796 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:16.792+0000 7f06137fe700 1 -- 172.21.15.77:0/209084694 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060400f620 con 0x7f061c100e60 2023-12-01T01:16:16.796 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:16.792+0000 7f0620e4c700 1 -- 172.21.15.77:0/209084694 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f061c1a1260 con 0x7f061c100e60 2023-12-01T01:16:16.797 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:16.793+0000 7f0620e4c700 1 -- 172.21.15.77:0/209084694 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f061c10b4d0 con 0x7f061c100e60 2023-12-01T01:16:16.797 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:16.794+0000 7f06137fe700 1 -- 172.21.15.77:0/209084694 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f06040108d0 con 0x7f061c100e60 2023-12-01T01:16:16.797 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:16.794+0000 7f06137fe700 1 --2- 172.21.15.77:0/209084694 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f06080a05f0 0x7f06080a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:16.797 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:16.794+0000 7f061a59c700 1 --2- 172.21.15.77:0/209084694 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f06080a05f0 0x7f06080a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:16.797 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:16.794+0000 7f06137fe700 1 -- 172.21.15.77:0/209084694 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f06040cb670 con 0x7f061c100e60 2023-12-01T01:16:16.798 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:16.795+0000 7f061a59c700 1 --2- 172.21.15.77:0/209084694 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f06080a05f0 0x7f06080a2ab0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f060c00b580 tx=0x7f060c009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:16.800 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:16.799+0000 7f06137fe700 1 -- 172.21.15.77:0/209084694 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f06040956b0 con 0x7f061c100e60 2023-12-01T01:16:16.824 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:16 smithi077 ceph-mon[98867]: Deploying daemon mds.a on smithi077 2023-12-01T01:16:16.824 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:16 smithi077 ceph-mon[98867]: mds.? [v2:172.21.15.77:6838/3726913446,v1:172.21.15.77:6839/3726913446] up:boot 2023-12-01T01:16:16.825 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:16 smithi077 ceph-mon[98867]: fsmap 10 up:standby 2023-12-01T01:16:16.825 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:16 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-01T01:16:17.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:16 smithi203 ceph-mon[103581]: Deploying daemon mds.a on smithi077 2023-12-01T01:16:17.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:16 smithi203 ceph-mon[103581]: mds.? [v2:172.21.15.77:6838/3726913446,v1:172.21.15.77:6839/3726913446] up:boot 2023-12-01T01:16:17.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:16 smithi203 ceph-mon[103581]: fsmap 10 up:standby 2023-12-01T01:16:17.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:16 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-01T01:16:17.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:16 smithi137 ceph-mon[103312]: Deploying daemon mds.a on smithi077 2023-12-01T01:16:17.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:16 smithi137 ceph-mon[103312]: mds.? [v2:172.21.15.77:6838/3726913446,v1:172.21.15.77:6839/3726913446] up:boot 2023-12-01T01:16:17.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:16 smithi137 ceph-mon[103312]: fsmap 10 up:standby 2023-12-01T01:16:17.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:16 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-01T01:16:17.057 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:17.056+0000 7f0620e4c700 1 -- 172.21.15.77:0/209084694 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f061c106b50 con 0x7f061c100e60 2023-12-01T01:16:17.058 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:17.057+0000 7f06137fe700 1 -- 172.21.15.77:0/209084694 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8091 (secure 0 0 0) 0x7f0604098cd0 con 0x7f061c100e60 2023-12-01T01:16:17.058 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:16:17.059 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1435145620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1435145620},{"type":"v1","addr":"172.21.15.77:6835","nonce":1435145620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/841838908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":841838908},{"type":"v1","addr":"172.21.15.77:6837","nonce":841838908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6839/3726913446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3726913446},{"type":"v1","addr":"172.21.15.77:6839","nonce":3726913446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/2249638445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2249638445},{"type":"v1","addr":"172.21.15.137:6833","nonce":2249638445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/1201599774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1201599774},{"type":"v1","addr":"172.21.15.203:6833","nonce":1201599774}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3935354878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3935354878},{"type":"v1","addr":"172.21.15.137:6835","nonce":3935354878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3237538176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3237538176},{"type":"v1","addr":"172.21.15.203:6835","nonce":3237538176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24473,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/3518881867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3518881867},{"type":"v1","addr":"172.21.15.137:6837","nonce":3518881867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2887080977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2887080977},{"type":"v1","addr":"172.21.15.203:6837","nonce":2887080977}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2888221613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2888221613},{"type":"v1","addr":"172.21.15.203:6839","nonce":2888221613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-12-01T01:16:17.062 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:17.060+0000 7f0620e4c700 1 -- 172.21.15.77:0/209084694 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f06080a05f0 msgr2=0x7f06080a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:17.062 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:17.060+0000 7f0620e4c700 1 --2- 172.21.15.77:0/209084694 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f06080a05f0 0x7f06080a2ab0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f060c00b580 tx=0x7f060c009f10 comp rx=0 tx=0).stop 2023-12-01T01:16:17.062 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:17.060+0000 7f0620e4c700 1 -- 172.21.15.77:0/209084694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f061c100e60 msgr2=0x7f061c19dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:17.062 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:17.060+0000 7f0620e4c700 1 --2- 172.21.15.77:0/209084694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f061c100e60 0x7f061c19dbc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f060400d950 tx=0x7f060400dc60 comp rx=0 tx=0).stop 2023-12-01T01:16:17.062 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:17.060+0000 7f0620e4c700 1 -- 172.21.15.77:0/209084694 shutdown_connections 2023-12-01T01:16:17.062 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:17.060+0000 7f0620e4c700 1 --2- 172.21.15.77:0/209084694 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f06080a05f0 0x7f06080a2ab0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:17.062 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:17.060+0000 7f0620e4c700 1 --2- 172.21.15.77:0/209084694 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f061c101800 0x7f061c1a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:17.062 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:17.060+0000 7f0620e4c700 1 --2- 172.21.15.77:0/209084694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f061c0ffc70 0x7f061c068a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:17.062 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:17.060+0000 7f0620e4c700 1 --2- 172.21.15.77:0/209084694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f061c100e60 0x7f061c19dbc0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:17.062 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:17.060+0000 7f0620e4c700 1 -- 172.21.15.77:0/209084694 >> 172.21.15.77:0/209084694 conn(0x7f061c0fb1c0 msgr2=0x7f061c0fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:17.063 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:17.060+0000 7f0620e4c700 1 -- 172.21.15.77:0/209084694 shutdown_connections 2023-12-01T01:16:17.063 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:17.061+0000 7f0620e4c700 1 -- 172.21.15.77:0/209084694 wait complete. 2023-12-01T01:16:17.063 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 11 2023-12-01T01:16:17.074 INFO:journalctl@ceph.mds.a.smithi077.stdout:Dec 01 01:16:17 smithi077 systemd[1]: Starting Ceph mds.a for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:16:17.604 INFO:journalctl@ceph.mds.a.smithi077.stdout:Dec 01 01:16:17 smithi077 podman[150288]: 2023-12-01T01:16:17.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:17 smithi077 ceph-mon[98867]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:16:17.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:17 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/209084694' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:16:17.904 INFO:journalctl@ceph.mds.a.smithi077.stdout:Dec 01 01:16:17 smithi077 ceph-mds[150328]: -- 172.21.15.77:0/2472090945 <== mon.0 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b7b81cb1e0 con 0x55b7b81a0c00 2023-12-01T01:16:17.904 INFO:journalctl@ceph.mds.a.smithi077.stdout:Dec 01 01:16:17 smithi077 bash[150288]: 93e77b2a14a9704aaff4d92c0694e292d63b8b99ff6c77766e1453ea7b9a60a3 2023-12-01T01:16:17.904 INFO:journalctl@ceph.mds.a.smithi077.stdout:Dec 01 01:16:17 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-a[150304]: starting mds.a at 2023-12-01T01:16:17.904 INFO:journalctl@ceph.mds.a.smithi077.stdout:Dec 01 01:16:17 smithi077 systemd[1]: Started Ceph mds.a for 2a06e29a-8fe6-11ee-95a2-87774f69a715. 2023-12-01T01:16:18.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:17 smithi203 ceph-mon[103581]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:16:18.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:17 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/209084694' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:16:18.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:17 smithi137 ceph-mon[103312]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:16:18.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:17 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/209084694' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:16:18.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:18 smithi077 ceph-mon[98867]: mds.? [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] up:boot 2023-12-01T01:16:18.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:18 smithi077 ceph-mon[98867]: fsmap 11 up:standby 2023-12-01T01:16:18.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:18 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-01T01:16:18.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:18 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:18.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:18 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:18.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:18 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:18.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:18 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-01T01:16:18.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:18 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-01T01:16:18.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:18 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:16:18.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:18 smithi077 ceph-mon[98867]: Deploying daemon mds.b on smithi137 2023-12-01T01:16:18.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:18 smithi077 ceph-mon[98867]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:16:18.962 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:18 smithi137 ceph-mon[103312]: mds.? [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] up:boot 2023-12-01T01:16:18.962 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:18 smithi137 ceph-mon[103312]: fsmap 11 up:standby 2023-12-01T01:16:18.962 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:18 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-01T01:16:18.962 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:18 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:18.962 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:18 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:18.962 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:18 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:18.962 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:18 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-01T01:16:18.962 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:18 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-01T01:16:18.962 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:18 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:16:18.963 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:18 smithi137 ceph-mon[103312]: Deploying daemon mds.b on smithi137 2023-12-01T01:16:18.963 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:18 smithi137 ceph-mon[103312]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:16:19.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:18 smithi203 ceph-mon[103581]: mds.? [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] up:boot 2023-12-01T01:16:19.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:18 smithi203 ceph-mon[103581]: fsmap 11 up:standby 2023-12-01T01:16:19.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:18 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-01T01:16:19.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:18 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:19.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:18 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:19.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:18 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:19.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:18 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-01T01:16:19.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:18 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-01T01:16:19.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:18 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:16:19.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:18 smithi203 ceph-mon[103581]: Deploying daemon mds.b on smithi137 2023-12-01T01:16:19.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:18 smithi203 ceph-mon[103581]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:16:19.077 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:16:19.218 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.215+0000 7fd5f3907700 1 -- 172.21.15.77:0/3218029812 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5ec0ffcb0 msgr2=0x7fd5ec1000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:19.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.215+0000 7fd5f3907700 1 --2- 172.21.15.77:0/3218029812 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5ec0ffcb0 0x7fd5ec1000d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd5e80099c0 tx=0x7fd5e8009cd0 comp rx=0 tx=0).stop 2023-12-01T01:16:19.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.215+0000 7fd5f3907700 1 -- 172.21.15.77:0/3218029812 shutdown_connections 2023-12-01T01:16:19.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.215+0000 7fd5f3907700 1 --2- 172.21.15.77:0/3218029812 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5ec101840 0x7fd5ec1080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:19.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.215+0000 7fd5f3907700 1 --2- 172.21.15.77:0/3218029812 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5ec0ffcb0 0x7fd5ec1000d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:19.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.215+0000 7fd5f3907700 1 --2- 172.21.15.77:0/3218029812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5ec100ea0 0x7fd5ec101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:19.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.215+0000 7fd5f3907700 1 -- 172.21.15.77:0/3218029812 >> 172.21.15.77:0/3218029812 conn(0x7fd5ec0fb220 msgr2=0x7fd5ec0fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:19.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.215+0000 7fd5f3907700 1 -- 172.21.15.77:0/3218029812 shutdown_connections 2023-12-01T01:16:19.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.215+0000 7fd5f3907700 1 -- 172.21.15.77:0/3218029812 wait complete. 2023-12-01T01:16:19.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.215+0000 7fd5f3907700 1 Processor -- start 2023-12-01T01:16:19.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.215+0000 7fd5f3907700 1 -- start start 2023-12-01T01:16:19.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.216+0000 7fd5f3907700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5ec0ffcb0 0x7fd5ec196ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:19.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.216+0000 7fd5f3907700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5ec100ea0 0x7fd5ec197010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:19.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.216+0000 7fd5f3907700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5ec101840 0x7fd5ec19e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:19.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.216+0000 7fd5f3907700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec19e650 con 0x7fd5ec101840 2023-12-01T01:16:19.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.216+0000 7fd5f3907700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec19e790 con 0x7fd5ec0ffcb0 2023-12-01T01:16:19.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.216+0000 7fd5f3907700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec19e920 con 0x7fd5ec100ea0 2023-12-01T01:16:19.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.216+0000 7fd5f1ea4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5ec101840 0x7fd5ec19e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:19.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.216+0000 7fd5f1ea4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5ec101840 0x7fd5ec19e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57346/0 (socket says 172.21.15.77:57346) 2023-12-01T01:16:19.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.216+0000 7fd5f1ea4700 1 -- 172.21.15.77:0/1851599781 learned_addr learned my addr 172.21.15.77:0/1851599781 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:19.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.216+0000 7fd5f0ea2700 1 --2- 172.21.15.77:0/1851599781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5ec100ea0 0x7fd5ec197010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:19.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.216+0000 7fd5f16a3700 1 --2- 172.21.15.77:0/1851599781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5ec0ffcb0 0x7fd5ec196ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:19.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.216+0000 7fd5f1ea4700 1 -- 172.21.15.77:0/1851599781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5ec100ea0 msgr2=0x7fd5ec197010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:19.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.216+0000 7fd5f1ea4700 1 --2- 172.21.15.77:0/1851599781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5ec100ea0 0x7fd5ec197010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:19.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.216+0000 7fd5f1ea4700 1 -- 172.21.15.77:0/1851599781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5ec0ffcb0 msgr2=0x7fd5ec196ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:19.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.216+0000 7fd5f1ea4700 1 --2- 172.21.15.77:0/1851599781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5ec0ffcb0 0x7fd5ec196ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:19.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.216+0000 7fd5f1ea4700 1 -- 172.21.15.77:0/1851599781 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e80095f0 con 0x7fd5ec101840 2023-12-01T01:16:19.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.216+0000 7fd5f16a3700 1 --2- 172.21.15.77:0/1851599781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5ec0ffcb0 0x7fd5ec196ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:16:19.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.216+0000 7fd5f1ea4700 1 --2- 172.21.15.77:0/1851599781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5ec101840 0x7fd5ec19e080 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd5e400b970 tx=0x7fd5e400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:19.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.216+0000 7fd5de7fc700 1 -- 172.21.15.77:0/1851599781 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e400c930 con 0x7fd5ec101840 2023-12-01T01:16:19.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.216+0000 7fd5de7fc700 1 -- 172.21.15.77:0/1851599781 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5e4014440 con 0x7fd5ec101840 2023-12-01T01:16:19.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.216+0000 7fd5f3907700 1 -- 172.21.15.77:0/1851599781 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5ec19ec00 con 0x7fd5ec101840 2023-12-01T01:16:19.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.216+0000 7fd5de7fc700 1 -- 172.21.15.77:0/1851599781 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e40075d0 con 0x7fd5ec101840 2023-12-01T01:16:19.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.216+0000 7fd5f3907700 1 -- 172.21.15.77:0/1851599781 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5ec19f0c0 con 0x7fd5ec101840 2023-12-01T01:16:19.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.217+0000 7fd5f3907700 1 -- 172.21.15.77:0/1851599781 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5ec10b590 con 0x7fd5ec101840 2023-12-01T01:16:19.222 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.218+0000 7fd5de7fc700 1 -- 172.21.15.77:0/1851599781 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fd5e40145b0 con 0x7fd5ec101840 2023-12-01T01:16:19.222 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.219+0000 7fd5de7fc700 1 --2- 172.21.15.77:0/1851599781 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd5d80a0570 0x7fd5d80a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:19.222 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.219+0000 7fd5f16a3700 1 --2- 172.21.15.77:0/1851599781 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd5d80a0570 0x7fd5d80a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:19.222 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.219+0000 7fd5de7fc700 1 -- 172.21.15.77:0/1851599781 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd5e4006e40 con 0x7fd5ec101840 2023-12-01T01:16:19.222 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.219+0000 7fd5f16a3700 1 --2- 172.21.15.77:0/1851599781 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd5d80a0570 0x7fd5d80a2a30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8009990 tx=0x7fd5e8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:19.224 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.223+0000 7fd5de7fc700 1 -- 172.21.15.77:0/1851599781 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd5e4094bc0 con 0x7fd5ec101840 2023-12-01T01:16:19.460 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.459+0000 7fd5f3907700 1 -- 172.21.15.77:0/1851599781 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5ec10b780 con 0x7fd5ec101840 2023-12-01T01:16:19.461 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.460+0000 7fd5de7fc700 1 -- 172.21.15.77:0/1851599781 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8849 (secure 0 0 0) 0x7fd5e40981e0 con 0x7fd5ec101840 2023-12-01T01:16:19.461 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:16:19.461 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1435145620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1435145620},{"type":"v1","addr":"172.21.15.77:6835","nonce":1435145620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/841838908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":841838908},{"type":"v1","addr":"172.21.15.77:6837","nonce":841838908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6839/3726913446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3726913446},{"type":"v1","addr":"172.21.15.77:6839","nonce":3726913446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6841/3023184965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3023184965},{"type":"v1","addr":"172.21.15.77:6841","nonce":3023184965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/2249638445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2249638445},{"type":"v1","addr":"172.21.15.137:6833","nonce":2249638445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/1201599774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1201599774},{"type":"v1","addr":"172.21.15.203:6833","nonce":1201599774}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3935354878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3935354878},{"type":"v1","addr":"172.21.15.137:6835","nonce":3935354878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3237538176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3237538176},{"type":"v1","addr":"172.21.15.203:6835","nonce":3237538176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24473,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/3518881867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3518881867},{"type":"v1","addr":"172.21.15.137:6837","nonce":3518881867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2887080977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2887080977},{"type":"v1","addr":"172.21.15.203:6837","nonce":2887080977}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2888221613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2888221613},{"type":"v1","addr":"172.21.15.203:6839","nonce":2888221613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-12-01T01:16:19.463 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.462+0000 7fd5f3907700 1 -- 172.21.15.77:0/1851599781 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd5d80a0570 msgr2=0x7fd5d80a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:19.463 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.462+0000 7fd5f3907700 1 --2- 172.21.15.77:0/1851599781 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd5d80a0570 0x7fd5d80a2a30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8009990 tx=0x7fd5e8005fb0 comp rx=0 tx=0).stop 2023-12-01T01:16:19.463 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.462+0000 7fd5f3907700 1 -- 172.21.15.77:0/1851599781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5ec101840 msgr2=0x7fd5ec19e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:19.463 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.462+0000 7fd5f3907700 1 --2- 172.21.15.77:0/1851599781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5ec101840 0x7fd5ec19e080 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd5e400b970 tx=0x7fd5e400bc80 comp rx=0 tx=0).stop 2023-12-01T01:16:19.463 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.462+0000 7fd5f3907700 1 -- 172.21.15.77:0/1851599781 shutdown_connections 2023-12-01T01:16:19.463 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.462+0000 7fd5f3907700 1 --2- 172.21.15.77:0/1851599781 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd5d80a0570 0x7fd5d80a2a30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:19.463 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.462+0000 7fd5f3907700 1 --2- 172.21.15.77:0/1851599781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5ec101840 0x7fd5ec19e080 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:19.463 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.462+0000 7fd5f3907700 1 --2- 172.21.15.77:0/1851599781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5ec0ffcb0 0x7fd5ec196ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:19.463 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.462+0000 7fd5f3907700 1 --2- 172.21.15.77:0/1851599781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5ec100ea0 0x7fd5ec197010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:19.464 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.462+0000 7fd5f3907700 1 -- 172.21.15.77:0/1851599781 >> 172.21.15.77:0/1851599781 conn(0x7fd5ec0fb220 msgr2=0x7fd5ec068ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:19.464 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.462+0000 7fd5f3907700 1 -- 172.21.15.77:0/1851599781 shutdown_connections 2023-12-01T01:16:19.464 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:19.462+0000 7fd5f3907700 1 -- 172.21.15.77:0/1851599781 wait complete. 2023-12-01T01:16:19.464 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 12 2023-12-01T01:16:19.605 INFO:journalctl@ceph.mds.b.smithi137.stdout:Dec 01 01:16:19 smithi137 systemd[1]: Starting Ceph mds.b for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:16:19.868 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:19 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/1851599781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:16:19.868 INFO:journalctl@ceph.mds.b.smithi137.stdout:Dec 01 01:16:19 smithi137 podman[141359]: 2023-12-01T01:16:19.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:19 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/1851599781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:16:20.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:19 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/1851599781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:16:20.278 INFO:journalctl@ceph.mds.b.smithi137.stdout:Dec 01 01:16:19 smithi137 ceph-mds[141399]: -- 172.21.15.137:0/1706076867 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5630f1df51e0 con 0x5630f1dca800 2023-12-01T01:16:20.279 INFO:journalctl@ceph.mds.b.smithi137.stdout:Dec 01 01:16:19 smithi137 bash[141359]: 58d18dc2de8bda39250f21801ed1193e45d415cdbaef15c6a0b9153285b263bc 2023-12-01T01:16:20.279 INFO:journalctl@ceph.mds.b.smithi137.stdout:Dec 01 01:16:19 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-b[141375]: starting mds.b at 2023-12-01T01:16:20.279 INFO:journalctl@ceph.mds.b.smithi137.stdout:Dec 01 01:16:19 smithi137 systemd[1]: Started Ceph mds.b for 2a06e29a-8fe6-11ee-95a2-87774f69a715. 2023-12-01T01:16:21.262 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:20 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:21.262 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:20 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:21.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:20 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:21.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:20 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:21.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:20 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:21.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:20 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:16:21.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:20 smithi203 ceph-mon[103581]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:16:21.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:20 smithi203 ceph-mon[103581]: mds.? [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] up:boot 2023-12-01T01:16:21.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:20 smithi203 ceph-mon[103581]: fsmap 12 up:standby 2023-12-01T01:16:21.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:20 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-01T01:16:21.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:20 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:21.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:20 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:21.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:20 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:21.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:20 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:21.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:20 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:21.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:20 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:16:21.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:20 smithi137 ceph-mon[103312]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:16:21.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:20 smithi137 ceph-mon[103312]: mds.? [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] up:boot 2023-12-01T01:16:21.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:20 smithi137 ceph-mon[103312]: fsmap 12 up:standby 2023-12-01T01:16:21.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:20 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-01T01:16:21.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:20 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:21.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:20 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:21.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:20 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:21.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:20 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:21.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:20 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:21.405 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:20 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:16:21.406 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:20 smithi077 ceph-mon[98867]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:16:21.406 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:20 smithi077 ceph-mon[98867]: mds.? [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] up:boot 2023-12-01T01:16:21.406 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:20 smithi077 ceph-mon[98867]: fsmap 12 up:standby 2023-12-01T01:16:21.406 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:20 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-01T01:16:21.477 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:16:21.616 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.612+0000 7f7a88ba6700 1 -- 172.21.15.77:0/2968066851 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a840ffc70 msgr2=0x7f7a84100090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:21.616 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.612+0000 7f7a88ba6700 1 --2- 172.21.15.77:0/2968066851 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a840ffc70 0x7f7a84100090 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c0099c0 tx=0x7f7a7c009cd0 comp rx=0 tx=0).stop 2023-12-01T01:16:21.616 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.612+0000 7f7a88ba6700 1 -- 172.21.15.77:0/2968066851 shutdown_connections 2023-12-01T01:16:21.616 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.612+0000 7f7a88ba6700 1 --2- 172.21.15.77:0/2968066851 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a84101800 0x7f7a841080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:21.616 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.612+0000 7f7a88ba6700 1 --2- 172.21.15.77:0/2968066851 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a840ffc70 0x7f7a84100090 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:21.616 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.612+0000 7f7a88ba6700 1 --2- 172.21.15.77:0/2968066851 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a84100e60 0x7f7a841012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:21.616 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.612+0000 7f7a88ba6700 1 -- 172.21.15.77:0/2968066851 >> 172.21.15.77:0/2968066851 conn(0x7f7a840fb1c0 msgr2=0x7f7a840fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:21.616 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.612+0000 7f7a88ba6700 1 -- 172.21.15.77:0/2968066851 shutdown_connections 2023-12-01T01:16:21.616 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.612+0000 7f7a88ba6700 1 -- 172.21.15.77:0/2968066851 wait complete. 2023-12-01T01:16:21.616 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.613+0000 7f7a88ba6700 1 Processor -- start 2023-12-01T01:16:21.616 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.613+0000 7f7a88ba6700 1 -- start start 2023-12-01T01:16:21.616 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.613+0000 7f7a88ba6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a840ffc70 0x7f7a84195890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:21.617 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.613+0000 7f7a88ba6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a84100e60 0x7f7a84195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:21.617 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.613+0000 7f7a88ba6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a84101800 0x7f7a8419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:21.617 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.613+0000 7f7a88ba6700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8419d410 con 0x7f7a84100e60 2023-12-01T01:16:21.617 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.613+0000 7f7a88ba6700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8419d550 con 0x7f7a840ffc70 2023-12-01T01:16:21.617 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.613+0000 7f7a88ba6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8419d6e0 con 0x7f7a84101800 2023-12-01T01:16:21.617 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.613+0000 7f7a81d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a84100e60 0x7f7a84195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:21.617 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.613+0000 7f7a81d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a84100e60 0x7f7a84195dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57356/0 (socket says 172.21.15.77:57356) 2023-12-01T01:16:21.617 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.613+0000 7f7a81d9b700 1 -- 172.21.15.77:0/2103324324 learned_addr learned my addr 172.21.15.77:0/2103324324 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:21.617 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.613+0000 7f7a8259c700 1 --2- 172.21.15.77:0/2103324324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a840ffc70 0x7f7a84195890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:21.617 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.613+0000 7f7a82d9d700 1 --2- 172.21.15.77:0/2103324324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a84101800 0x7f7a8419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:21.617 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.614+0000 7f7a81d9b700 1 -- 172.21.15.77:0/2103324324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a84101800 msgr2=0x7f7a8419ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:21.617 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.614+0000 7f7a81d9b700 1 --2- 172.21.15.77:0/2103324324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a84101800 0x7f7a8419ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:21.617 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.614+0000 7f7a81d9b700 1 -- 172.21.15.77:0/2103324324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a840ffc70 msgr2=0x7f7a84195890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:21.617 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.614+0000 7f7a81d9b700 1 --2- 172.21.15.77:0/2103324324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a840ffc70 0x7f7a84195890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:21.617 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.614+0000 7f7a81d9b700 1 -- 172.21.15.77:0/2103324324 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a7c0095f0 con 0x7f7a84100e60 2023-12-01T01:16:21.618 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.614+0000 7f7a81d9b700 1 --2- 172.21.15.77:0/2103324324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a84100e60 0x7f7a84195dd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c00d950 tx=0x7f7a6c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:21.618 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.614+0000 7f7a7b7fe700 1 -- 172.21.15.77:0/2103324324 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c00f890 con 0x7f7a84100e60 2023-12-01T01:16:21.618 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.614+0000 7f7a7b7fe700 1 -- 172.21.15.77:0/2103324324 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a6c010460 con 0x7f7a84100e60 2023-12-01T01:16:21.618 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.614+0000 7f7a88ba6700 1 -- 172.21.15.77:0/2103324324 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a8419d9c0 con 0x7f7a84100e60 2023-12-01T01:16:21.618 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.614+0000 7f7a7b7fe700 1 -- 172.21.15.77:0/2103324324 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c00e610 con 0x7f7a84100e60 2023-12-01T01:16:21.618 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.614+0000 7f7a88ba6700 1 -- 172.21.15.77:0/2103324324 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a8419df20 con 0x7f7a84100e60 2023-12-01T01:16:21.619 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.615+0000 7f7a88ba6700 1 -- 172.21.15.77:0/2103324324 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a84107160 con 0x7f7a84100e60 2023-12-01T01:16:21.619 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.616+0000 7f7a7b7fe700 1 -- 172.21.15.77:0/2103324324 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f7a6c0108d0 con 0x7f7a84100e60 2023-12-01T01:16:21.619 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.617+0000 7f7a7b7fe700 1 --2- 172.21.15.77:0/2103324324 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7a700a05f0 0x7f7a700a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:21.619 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.617+0000 7f7a8259c700 1 --2- 172.21.15.77:0/2103324324 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7a700a05f0 0x7f7a700a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:21.619 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.617+0000 7f7a7b7fe700 1 -- 172.21.15.77:0/2103324324 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7a6c0cb250 con 0x7f7a84100e60 2023-12-01T01:16:21.619 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.617+0000 7f7a8259c700 1 --2- 172.21.15.77:0/2103324324 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7a700a05f0 0x7f7a700a2ab0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c009f50 tx=0x7f7a7c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:21.623 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.621+0000 7f7a7b7fe700 1 -- 172.21.15.77:0/2103324324 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7a6c095270 con 0x7f7a84100e60 2023-12-01T01:16:21.866 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.865+0000 7f7a88ba6700 1 -- 172.21.15.77:0/2103324324 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a8402cc70 con 0x7f7a84100e60 2023-12-01T01:16:21.867 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.866+0000 7f7a7b7fe700 1 -- 172.21.15.77:0/2103324324 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9610 (secure 0 0 0) 0x7f7a6c098890 con 0x7f7a84100e60 2023-12-01T01:16:21.867 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:16:21.867 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1435145620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1435145620},{"type":"v1","addr":"172.21.15.77:6835","nonce":1435145620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/841838908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":841838908},{"type":"v1","addr":"172.21.15.77:6837","nonce":841838908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6839/3726913446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3726913446},{"type":"v1","addr":"172.21.15.77:6839","nonce":3726913446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6841/3023184965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3023184965},{"type":"v1","addr":"172.21.15.77:6841","nonce":3023184965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/2249638445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2249638445},{"type":"v1","addr":"172.21.15.137:6833","nonce":2249638445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/1201599774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1201599774},{"type":"v1","addr":"172.21.15.203:6833","nonce":1201599774}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3935354878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3935354878},{"type":"v1","addr":"172.21.15.137:6835","nonce":3935354878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/3237538176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3237538176},{"type":"v1","addr":"172.21.15.203:6835","nonce":3237538176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24473,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/3518881867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3518881867},{"type":"v1","addr":"172.21.15.137:6837","nonce":3518881867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2887080977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2887080977},{"type":"v1","addr":"172.21.15.203:6837","nonce":2887080977}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24496,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2888221613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2888221613},{"type":"v1","addr":"172.21.15.203:6839","nonce":2888221613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24500,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/1099966660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1099966660},{"type":"v1","addr":"172.21.15.137:6839","nonce":1099966660}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2023-12-01T01:16:21.869 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.868+0000 7f7a88ba6700 1 -- 172.21.15.77:0/2103324324 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7a700a05f0 msgr2=0x7f7a700a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:21.869 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.868+0000 7f7a88ba6700 1 --2- 172.21.15.77:0/2103324324 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7a700a05f0 0x7f7a700a2ab0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c009f50 tx=0x7f7a7c00b540 comp rx=0 tx=0).stop 2023-12-01T01:16:21.870 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.868+0000 7f7a88ba6700 1 -- 172.21.15.77:0/2103324324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a84100e60 msgr2=0x7f7a84195dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:21.870 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.868+0000 7f7a88ba6700 1 --2- 172.21.15.77:0/2103324324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a84100e60 0x7f7a84195dd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c00d950 tx=0x7f7a6c00dc60 comp rx=0 tx=0).stop 2023-12-01T01:16:21.870 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.868+0000 7f7a88ba6700 1 -- 172.21.15.77:0/2103324324 shutdown_connections 2023-12-01T01:16:21.870 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.868+0000 7f7a88ba6700 1 --2- 172.21.15.77:0/2103324324 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7a700a05f0 0x7f7a700a2ab0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:21.870 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.868+0000 7f7a88ba6700 1 --2- 172.21.15.77:0/2103324324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a84101800 0x7f7a8419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:21.870 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.868+0000 7f7a88ba6700 1 --2- 172.21.15.77:0/2103324324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a840ffc70 0x7f7a84195890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:21.870 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.868+0000 7f7a88ba6700 1 --2- 172.21.15.77:0/2103324324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a84100e60 0x7f7a84195dd0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:21.870 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.868+0000 7f7a88ba6700 1 -- 172.21.15.77:0/2103324324 >> 172.21.15.77:0/2103324324 conn(0x7f7a840fb1c0 msgr2=0x7f7a841042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:21.870 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.868+0000 7f7a88ba6700 1 -- 172.21.15.77:0/2103324324 shutdown_connections 2023-12-01T01:16:21.870 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:21.868+0000 7f7a88ba6700 1 -- 172.21.15.77:0/2103324324 wait complete. 2023-12-01T01:16:21.870 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 13 2023-12-01T01:16:21.882 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:16:22.010 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.007+0000 7f4d9f454700 1 -- 172.21.15.77:0/2707827178 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4d98074d90 msgr2=0x7f4d980731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:22.010 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.007+0000 7f4d9f454700 1 --2- 172.21.15.77:0/2707827178 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4d98074d90 0x7f4d980731e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4d94009990 tx=0x7f4d94009ca0 comp rx=0 tx=0).stop 2023-12-01T01:16:22.010 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.007+0000 7f4d9f454700 1 -- 172.21.15.77:0/2707827178 shutdown_connections 2023-12-01T01:16:22.010 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.007+0000 7f4d9f454700 1 --2- 172.21.15.77:0/2707827178 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d98108c50 0x7f4d9810b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:22.010 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.007+0000 7f4d9f454700 1 --2- 172.21.15.77:0/2707827178 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4d98074d90 0x7f4d980731e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:22.010 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.007+0000 7f4d9f454700 1 --2- 172.21.15.77:0/2707827178 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d980737b0 0x7f4d98073c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:22.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.007+0000 7f4d9f454700 1 -- 172.21.15.77:0/2707827178 >> 172.21.15.77:0/2707827178 conn(0x7f4d980fc2d0 msgr2=0x7f4d980fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:22.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.007+0000 7f4d9f454700 1 -- 172.21.15.77:0/2707827178 shutdown_connections 2023-12-01T01:16:22.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.007+0000 7f4d9f454700 1 -- 172.21.15.77:0/2707827178 wait complete. 2023-12-01T01:16:22.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.007+0000 7f4d9f454700 1 Processor -- start 2023-12-01T01:16:22.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.008+0000 7f4d9f454700 1 -- start start 2023-12-01T01:16:22.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.008+0000 7f4d9f454700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4d980737b0 0x7f4d98198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:22.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.008+0000 7f4d9f454700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d98074d90 0x7f4d981991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:22.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.008+0000 7f4d9f454700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d98108c50 0x7f4d981a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:22.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.008+0000 7f4d9f454700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d981a0800 con 0x7f4d980737b0 2023-12-01T01:16:22.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.008+0000 7f4d9f454700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d981a0970 con 0x7f4d98108c50 2023-12-01T01:16:22.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.008+0000 7f4d9f454700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d981a0b30 con 0x7f4d98074d90 2023-12-01T01:16:22.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.008+0000 7f4d9d1f0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4d980737b0 0x7f4d98198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:22.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.008+0000 7f4d9d1f0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4d980737b0 0x7f4d98198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57368/0 (socket says 172.21.15.77:57368) 2023-12-01T01:16:22.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.008+0000 7f4d9d1f0700 1 -- 172.21.15.77:0/1470978693 learned_addr learned my addr 172.21.15.77:0/1470978693 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:22.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.008+0000 7f4d9c9ef700 1 --2- 172.21.15.77:0/1470978693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d98074d90 0x7f4d981991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:22.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.008+0000 7f4d9d9f1700 1 --2- 172.21.15.77:0/1470978693 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d98108c50 0x7f4d981a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:22.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.008+0000 7f4d9d1f0700 1 -- 172.21.15.77:0/1470978693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d98074d90 msgr2=0x7f4d981991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:22.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.008+0000 7f4d9d1f0700 1 --2- 172.21.15.77:0/1470978693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d98074d90 0x7f4d981991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:22.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.008+0000 7f4d9d1f0700 1 -- 172.21.15.77:0/1470978693 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d98108c50 msgr2=0x7f4d981a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:22.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.008+0000 7f4d9d1f0700 1 --2- 172.21.15.77:0/1470978693 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d98108c50 0x7f4d981a0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:22.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.008+0000 7f4d9d1f0700 1 -- 172.21.15.77:0/1470978693 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d940095f0 con 0x7f4d980737b0 2023-12-01T01:16:22.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.008+0000 7f4d9d9f1700 1 --2- 172.21.15.77:0/1470978693 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d98108c50 0x7f4d981a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:16:22.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.008+0000 7f4d9c9ef700 1 --2- 172.21.15.77:0/1470978693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d98074d90 0x7f4d981991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:16:22.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.008+0000 7f4d9d1f0700 1 --2- 172.21.15.77:0/1470978693 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4d980737b0 0x7f4d98198c90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4d94004730 tx=0x7f4d94005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:22.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.008+0000 7f4d8e7fc700 1 -- 172.21.15.77:0/1470978693 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9401d070 con 0x7f4d980737b0 2023-12-01T01:16:22.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.008+0000 7f4d8e7fc700 1 -- 172.21.15.77:0/1470978693 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d94004cd0 con 0x7f4d980737b0 2023-12-01T01:16:22.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.008+0000 7f4d9f454700 1 -- 172.21.15.77:0/1470978693 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d981a0db0 con 0x7f4d980737b0 2023-12-01T01:16:22.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.008+0000 7f4d8e7fc700 1 -- 172.21.15.77:0/1470978693 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d94005610 con 0x7f4d980737b0 2023-12-01T01:16:22.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.008+0000 7f4d9f454700 1 -- 172.21.15.77:0/1470978693 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d981a1170 con 0x7f4d980737b0 2023-12-01T01:16:22.013 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.009+0000 7f4d9f454700 1 -- 172.21.15.77:0/1470978693 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d98107350 con 0x7f4d980737b0 2023-12-01T01:16:22.013 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.010+0000 7f4d8e7fc700 1 -- 172.21.15.77:0/1470978693 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f4d9400fc70 con 0x7f4d980737b0 2023-12-01T01:16:22.013 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.011+0000 7f4d8e7fc700 1 --2- 172.21.15.77:0/1470978693 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f4d840a0570 0x7f4d840a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:22.013 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.011+0000 7f4d9c9ef700 1 --2- 172.21.15.77:0/1470978693 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f4d840a0570 0x7f4d840a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:22.014 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.011+0000 7f4d8e7fc700 1 -- 172.21.15.77:0/1470978693 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4d940cce60 con 0x7f4d980737b0 2023-12-01T01:16:22.014 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.012+0000 7f4d9c9ef700 1 --2- 172.21.15.77:0/1470978693 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f4d840a0570 0x7f4d840a2a30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4d9819a190 tx=0x7f4d88009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:22.016 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.015+0000 7f4d8e7fc700 1 -- 172.21.15.77:0/1470978693 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4d94096e90 con 0x7f4d980737b0 2023-12-01T01:16:22.217 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.216+0000 7f4d9f454700 1 -- 172.21.15.77:0/1470978693 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4d9802ce40 con 0x7f4d980737b0 2023-12-01T01:16:22.218 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.217+0000 7f4d8e7fc700 1 -- 172.21.15.77:0/1470978693 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+18434 (secure 0 0 0) 0x7f4d9409a4b0 con 0x7f4d980737b0 2023-12-01T01:16:22.219 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:16:22.219 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":78,"fsid":"2a06e29a-8fe6-11ee-95a2-87774f69a715","created":"2023-12-01T01:09:30.210398+0000","modified":"2023-12-01T01:15:46.072360+0000","last_up_change":"2023-12-01T01:15:44.586092+0000","last_in_change":"2023-12-01T01:15:26.305439+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:12:30.514229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fc651e62-a103-4ab8-a76d-d3708ea6aeec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6803","nonce":2244799094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6805","nonce":2244799094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6809","nonce":2244799094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6807","nonce":2244799094}]},"public_addr":"172.21.15.77:6803/2244799094","cluster_addr":"172.21.15.77:6805/2244799094","heartbeat_back_addr":"172.21.15.77:6809/2244799094","heartbeat_front_addr":"172.21.15.77:6807/2244799094","state":["exists","up"]},{"osd":1,"uuid":"0adf8c54-49ec-49a0-8bf1-af1f20e60729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6801","nonce":224194890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6803","nonce":224194890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6807","nonce":224194890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6805","nonce":224194890}]},"public_addr":"172.21.15.137:6801/224194890","cluster_addr":"172.21.15.137:6803/224194890","heartbeat_back_addr":"172.21.15.137:6807/224194890","heartbeat_front_addr":"172.21.15.137:6805/224194890","state":["exists","up"]},{"osd":2,"uuid":"5a69802c-dede-4544-9a12-436f5c1c0038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6801","nonce":4078459110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6803","nonce":4078459110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6807","nonce":4078459110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6805","nonce":4078459110}]},"public_addr":"172.21.15.203:6801/4078459110","cluster_addr":"172.21.15.203:6803/4078459110","heartbeat_back_addr":"172.21.15.203:6807/4078459110","heartbeat_front_addr":"172.21.15.203:6805/4078459110","state":["exists","up"]},{"osd":3,"uuid":"ad325a97-580b-4c99-bf12-cbb0d92a6d0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6811","nonce":1454440477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6813","nonce":1454440477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6817","nonce":1454440477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6815","nonce":1454440477}]},"public_addr":"172.21.15.77:6811/1454440477","cluster_addr":"172.21.15.77:6813/1454440477","heartbeat_back_addr":"172.21.15.77:6817/1454440477","heartbeat_front_addr":"172.21.15.77:6815/1454440477","state":["exists","up"]},{"osd":4,"uuid":"1e3ac1ba-98bb-4b8b-8e26-50064c66bf2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6809","nonce":1103241451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6811","nonce":1103241451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6815","nonce":1103241451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6813","nonce":1103241451}]},"public_addr":"172.21.15.137:6809/1103241451","cluster_addr":"172.21.15.137:6811/1103241451","heartbeat_back_addr":"172.21.15.137:6815/1103241451","heartbeat_front_addr":"172.21.15.137:6813/1103241451","state":["exists","up"]},{"osd":5,"uuid":"eb32c016-abeb-44ab-abf0-19dd09235736","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6809","nonce":2648569482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6811","nonce":2648569482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6815","nonce":2648569482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6813","nonce":2648569482}]},"public_addr":"172.21.15.203:6809/2648569482","cluster_addr":"172.21.15.203:6811/2648569482","heartbeat_back_addr":"172.21.15.203:6815/2648569482","heartbeat_front_addr":"172.21.15.203:6813/2648569482","state":["exists","up"]},{"osd":6,"uuid":"0e3fa97e-2ca6-418c-9e8d-a6fe3c56adb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6819","nonce":2463414651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6821","nonce":2463414651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6825","nonce":2463414651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6823","nonce":2463414651}]},"public_addr":"172.21.15.77:6819/2463414651","cluster_addr":"172.21.15.77:6821/2463414651","heartbeat_back_addr":"172.21.15.77:6825/2463414651","heartbeat_front_addr":"172.21.15.77:6823/2463414651","state":["exists","up"]},{"osd":7,"uuid":"d7398c01-0f98-4138-8a49-c3ff0aae7e88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6817","nonce":3108330766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6819","nonce":3108330766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6823","nonce":3108330766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6821","nonce":3108330766}]},"public_addr":"172.21.15.137:6817/3108330766","cluster_addr":"172.21.15.137:6819/3108330766","heartbeat_back_addr":"172.21.15.137:6823/3108330766","heartbeat_front_addr":"172.21.15.137:6821/3108330766","state":["exists","up"]},{"osd":8,"uuid":"0c09050d-3e43-410c-8cd7-bcb891af56b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6817","nonce":1161158776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6819","nonce":1161158776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6823","nonce":1161158776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6821","nonce":1161158776}]},"public_addr":"172.21.15.203:6817/1161158776","cluster_addr":"172.21.15.203:6819/1161158776","heartbeat_back_addr":"172.21.15.203:6823/1161158776","heartbeat_front_addr":"172.21.15.203:6821/1161158776","state":["exists","up"]},{"osd":9,"uuid":"527ecb49-95ec-48bd-9872-3b07fae5c5ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6827","nonce":2409880685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6829","nonce":2409880685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6833","nonce":2409880685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6831","nonce":2409880685}]},"public_addr":"172.21.15.77:6827/2409880685","cluster_addr":"172.21.15.77:6829/2409880685","heartbeat_back_addr":"172.21.15.77:6833/2409880685","heartbeat_front_addr":"172.21.15.77:6831/2409880685","state":["exists","up"]},{"osd":10,"uuid":"1680bb3e-a3c8-427b-af68-e2214f32bed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6825","nonce":374920146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6827","nonce":374920146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6831","nonce":374920146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6829","nonce":374920146}]},"public_addr":"172.21.15.137:6825/374920146","cluster_addr":"172.21.15.137:6827/374920146","heartbeat_back_addr":"172.21.15.137:6831/374920146","heartbeat_front_addr":"172.21.15.137:6829/374920146","state":["exists","up"]},{"osd":11,"uuid":"8a533466-023d-4803-ae36-9a9f1715e49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6825","nonce":2161588962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6827","nonce":2161588962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6831","nonce":2161588962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6829","nonce":2161588962}]},"public_addr":"172.21.15.203:6825/2161588962","cluster_addr":"172.21.15.203:6827/2161588962","heartbeat_back_addr":"172.21.15.203:6831/2161588962","heartbeat_front_addr":"172.21.15.203:6829/2161588962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:11:45.420509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:06.210115+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:28.191307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:49.594116+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:11.261689+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:33.376541+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:54.968336+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:16.775310+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:38.733792+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:59.456699+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:15:20.924059+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:15:43.094773+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/2260035254":"2023-12-02T01:10:24.191835+0000","172.21.15.77:6800/1456621747":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/219838828":"2023-12-02T01:10:24.191835+0000","172.21.15.77:6801/1456621747":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/2937149266":"2023-12-02T01:10:02.232948+0000","172.21.15.77:0/807520018":"2023-12-02T01:10:02.232948+0000","172.21.15.77:6801/1661788026":"2023-12-02T01:10:02.232948+0000","172.21.15.77:6800/1661788026":"2023-12-02T01:10:02.232948+0000","172.21.15.77:0/3551282473":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/1202222558":"2023-12-02T01:10:02.232948+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:16:22.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.219+0000 7f4d9f454700 1 -- 172.21.15.77:0/1470978693 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f4d840a0570 msgr2=0x7f4d840a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:22.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.219+0000 7f4d9f454700 1 --2- 172.21.15.77:0/1470978693 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f4d840a0570 0x7f4d840a2a30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4d9819a190 tx=0x7f4d88009260 comp rx=0 tx=0).stop 2023-12-01T01:16:22.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.219+0000 7f4d9f454700 1 -- 172.21.15.77:0/1470978693 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4d980737b0 msgr2=0x7f4d98198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:22.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.219+0000 7f4d9f454700 1 --2- 172.21.15.77:0/1470978693 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4d980737b0 0x7f4d98198c90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4d94004730 tx=0x7f4d94005ef0 comp rx=0 tx=0).stop 2023-12-01T01:16:22.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.220+0000 7f4d9f454700 1 -- 172.21.15.77:0/1470978693 shutdown_connections 2023-12-01T01:16:22.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.220+0000 7f4d9f454700 1 --2- 172.21.15.77:0/1470978693 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f4d840a0570 0x7f4d840a2a30 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:22.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.220+0000 7f4d9f454700 1 --2- 172.21.15.77:0/1470978693 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d98108c50 0x7f4d981a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:22.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.220+0000 7f4d9f454700 1 --2- 172.21.15.77:0/1470978693 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4d980737b0 0x7f4d98198c90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:22.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.220+0000 7f4d9f454700 1 --2- 172.21.15.77:0/1470978693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d98074d90 0x7f4d981991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:22.222 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.220+0000 7f4d9f454700 1 -- 172.21.15.77:0/1470978693 >> 172.21.15.77:0/1470978693 conn(0x7f4d980fc2d0 msgr2=0x7f4d980fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:22.222 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.220+0000 7f4d9f454700 1 -- 172.21.15.77:0/1470978693 shutdown_connections 2023-12-01T01:16:22.222 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.220+0000 7f4d9f454700 1 -- 172.21.15.77:0/1470978693 wait complete. 2023-12-01T01:16:22.233 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:12:30.514229+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:16:22.233 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:16:22.364 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.360+0000 7fef221ce700 1 -- 172.21.15.77:0/3412554951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef1c100da0 msgr2=0x7fef1c1011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:22.364 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.360+0000 7fef221ce700 1 --2- 172.21.15.77:0/3412554951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef1c100da0 0x7fef1c1011c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fef100099c0 tx=0x7fef10009cd0 comp rx=0 tx=0).stop 2023-12-01T01:16:22.364 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.360+0000 7fef221ce700 1 -- 172.21.15.77:0/3412554951 shutdown_connections 2023-12-01T01:16:22.364 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.360+0000 7fef221ce700 1 --2- 172.21.15.77:0/3412554951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef1c102930 0x7fef1c1091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:22.364 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.360+0000 7fef221ce700 1 --2- 172.21.15.77:0/3412554951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef1c100da0 0x7fef1c1011c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:22.364 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.360+0000 7fef221ce700 1 --2- 172.21.15.77:0/3412554951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef1c101f90 0x7fef1c1023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:22.364 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.360+0000 7fef221ce700 1 -- 172.21.15.77:0/3412554951 >> 172.21.15.77:0/3412554951 conn(0x7fef1c0fc310 msgr2=0x7fef1c0fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:22.364 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.360+0000 7fef221ce700 1 -- 172.21.15.77:0/3412554951 shutdown_connections 2023-12-01T01:16:22.364 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.360+0000 7fef221ce700 1 -- 172.21.15.77:0/3412554951 wait complete. 2023-12-01T01:16:22.364 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.360+0000 7fef221ce700 1 Processor -- start 2023-12-01T01:16:22.364 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.361+0000 7fef221ce700 1 -- start start 2023-12-01T01:16:22.364 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.361+0000 7fef221ce700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef1c100da0 0x7fef1c074b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:22.365 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.361+0000 7fef221ce700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef1c101f90 0x7fef1c073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:22.365 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.361+0000 7fef221ce700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef1c102930 0x7fef1c0736c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:22.365 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.361+0000 7fef221ce700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c073e60 con 0x7fef1c102930 2023-12-01T01:16:22.365 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.361+0000 7fef221ce700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c073fd0 con 0x7fef1c100da0 2023-12-01T01:16:22.365 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.361+0000 7fef221ce700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c1a0b50 con 0x7fef1c101f90 2023-12-01T01:16:22.365 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.361+0000 7fef1bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef1c102930 0x7fef1c0736c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:22.365 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.361+0000 7fef1bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef1c102930 0x7fef1c0736c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57386/0 (socket says 172.21.15.77:57386) 2023-12-01T01:16:22.365 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.361+0000 7fef1bfff700 1 -- 172.21.15.77:0/4181048196 learned_addr learned my addr 172.21.15.77:0/4181048196 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:22.365 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.361+0000 7fef1affd700 1 --2- 172.21.15.77:0/4181048196 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef1c101f90 0x7fef1c073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:22.365 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.361+0000 7fef1b7fe700 1 --2- 172.21.15.77:0/4181048196 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef1c100da0 0x7fef1c074b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:22.365 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.361+0000 7fef1bfff700 1 -- 172.21.15.77:0/4181048196 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef1c101f90 msgr2=0x7fef1c073180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:22.365 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.361+0000 7fef1bfff700 1 --2- 172.21.15.77:0/4181048196 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef1c101f90 0x7fef1c073180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:22.365 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.361+0000 7fef1bfff700 1 -- 172.21.15.77:0/4181048196 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef1c100da0 msgr2=0x7fef1c074b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:22.365 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.361+0000 7fef1bfff700 1 --2- 172.21.15.77:0/4181048196 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef1c100da0 0x7fef1c074b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:22.366 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.361+0000 7fef1bfff700 1 -- 172.21.15.77:0/4181048196 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef100095f0 con 0x7fef1c102930 2023-12-01T01:16:22.366 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.361+0000 7fef1b7fe700 1 --2- 172.21.15.77:0/4181048196 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef1c100da0 0x7fef1c074b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:16:22.366 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.361+0000 7fef1bfff700 1 --2- 172.21.15.77:0/4181048196 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef1c102930 0x7fef1c0736c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fef14007ee0 tx=0x7fef1400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:22.366 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.361+0000 7fef18ff9700 1 -- 172.21.15.77:0/4181048196 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef14017070 con 0x7fef1c102930 2023-12-01T01:16:22.366 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.362+0000 7fef18ff9700 1 -- 172.21.15.77:0/4181048196 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef1400f040 con 0x7fef1c102930 2023-12-01T01:16:22.366 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.362+0000 7fef221ce700 1 -- 172.21.15.77:0/4181048196 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef1c1a0e30 con 0x7fef1c102930 2023-12-01T01:16:22.366 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.362+0000 7fef18ff9700 1 -- 172.21.15.77:0/4181048196 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef14005740 con 0x7fef1c102930 2023-12-01T01:16:22.366 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.362+0000 7fef221ce700 1 -- 172.21.15.77:0/4181048196 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef1c1a12b0 con 0x7fef1c102930 2023-12-01T01:16:22.366 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.362+0000 7fef221ce700 1 -- 172.21.15.77:0/4181048196 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef1c1083b0 con 0x7fef1c102930 2023-12-01T01:16:22.367 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.363+0000 7fef18ff9700 1 -- 172.21.15.77:0/4181048196 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fef1400dc60 con 0x7fef1c102930 2023-12-01T01:16:22.367 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.364+0000 7fef18ff9700 1 --2- 172.21.15.77:0/4181048196 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fef040a05a0 0x7fef040a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:22.367 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.364+0000 7fef1b7fe700 1 --2- 172.21.15.77:0/4181048196 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fef040a05a0 0x7fef040a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:22.367 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.364+0000 7fef18ff9700 1 -- 172.21.15.77:0/4181048196 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fef140cacf0 con 0x7fef1c102930 2023-12-01T01:16:22.367 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.364+0000 7fef1b7fe700 1 --2- 172.21.15.77:0/4181048196 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fef040a05a0 0x7fef040a2a60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fef10009990 tx=0x7fef1001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:22.369 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.368+0000 7fef18ff9700 1 -- 172.21.15.77:0/4181048196 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fef14094c90 con 0x7fef1c102930 2023-12-01T01:16:22.571 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.570+0000 7fef221ce700 1 -- 172.21.15.77:0/4181048196 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fef1c061a40 con 0x7fef1c102930 2023-12-01T01:16:22.572 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.571+0000 7fef18ff9700 1 -- 172.21.15.77:0/4181048196 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7fef140982b0 con 0x7fef1c102930 2023-12-01T01:16:22.572 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 1 2023-12-01T01:16:22.575 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.573+0000 7fef221ce700 1 -- 172.21.15.77:0/4181048196 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fef040a05a0 msgr2=0x7fef040a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:22.575 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.573+0000 7fef221ce700 1 --2- 172.21.15.77:0/4181048196 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fef040a05a0 0x7fef040a2a60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fef10009990 tx=0x7fef1001a040 comp rx=0 tx=0).stop 2023-12-01T01:16:22.575 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.573+0000 7fef221ce700 1 -- 172.21.15.77:0/4181048196 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef1c102930 msgr2=0x7fef1c0736c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:22.575 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.573+0000 7fef221ce700 1 --2- 172.21.15.77:0/4181048196 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef1c102930 0x7fef1c0736c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fef14007ee0 tx=0x7fef1400d3b0 comp rx=0 tx=0).stop 2023-12-01T01:16:22.575 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.573+0000 7fef221ce700 1 -- 172.21.15.77:0/4181048196 shutdown_connections 2023-12-01T01:16:22.575 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.573+0000 7fef221ce700 1 --2- 172.21.15.77:0/4181048196 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fef040a05a0 0x7fef040a2a60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:22.575 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.573+0000 7fef221ce700 1 --2- 172.21.15.77:0/4181048196 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef1c102930 0x7fef1c0736c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:22.575 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.573+0000 7fef221ce700 1 --2- 172.21.15.77:0/4181048196 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef1c100da0 0x7fef1c074b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:22.575 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.573+0000 7fef221ce700 1 --2- 172.21.15.77:0/4181048196 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef1c101f90 0x7fef1c073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:22.575 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.573+0000 7fef221ce700 1 -- 172.21.15.77:0/4181048196 >> 172.21.15.77:0/4181048196 conn(0x7fef1c0fc310 msgr2=0x7fef1c1054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:22.575 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.574+0000 7fef221ce700 1 -- 172.21.15.77:0/4181048196 shutdown_connections 2023-12-01T01:16:22.575 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.574+0000 7fef221ce700 1 -- 172.21.15.77:0/4181048196 wait complete. 2023-12-01T01:16:22.589 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-01T01:16:22.715 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.713+0000 7f4aa2f9e700 1 -- 172.21.15.77:0/3031732335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a9c100cb0 msgr2=0x7f4a9c101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:22.715 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.713+0000 7f4aa2f9e700 1 --2- 172.21.15.77:0/3031732335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a9c100cb0 0x7f4a9c101110 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c0099c0 tx=0x7f4a8c009cd0 comp rx=0 tx=0).stop 2023-12-01T01:16:22.715 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.713+0000 7f4aa2f9e700 1 -- 172.21.15.77:0/3031732335 shutdown_connections 2023-12-01T01:16:22.715 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.713+0000 7f4aa2f9e700 1 --2- 172.21.15.77:0/3031732335 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a9c101650 0x7f4a9c107f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:22.715 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.713+0000 7f4aa2f9e700 1 --2- 172.21.15.77:0/3031732335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a9c0ffac0 0x7f4a9c0ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:22.715 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.713+0000 7f4aa2f9e700 1 --2- 172.21.15.77:0/3031732335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a9c100cb0 0x7f4a9c101110 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:22.715 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.713+0000 7f4aa2f9e700 1 -- 172.21.15.77:0/3031732335 >> 172.21.15.77:0/3031732335 conn(0x7f4a9c0fb050 msgr2=0x7f4a9c0fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:22.716 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.713+0000 7f4aa2f9e700 1 -- 172.21.15.77:0/3031732335 shutdown_connections 2023-12-01T01:16:22.716 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.713+0000 7f4aa2f9e700 1 -- 172.21.15.77:0/3031732335 wait complete. 2023-12-01T01:16:22.718 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.714+0000 7f4aa2f9e700 1 Processor -- start 2023-12-01T01:16:22.718 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.714+0000 7f4aa2f9e700 1 -- start start 2023-12-01T01:16:22.718 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.714+0000 7f4aa2f9e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a9c0ffac0 0x7f4a9c195650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:22.718 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.714+0000 7f4aa2f9e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a9c100cb0 0x7f4a9c195b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:22.718 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.714+0000 7f4aa2f9e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a9c101650 0x7f4a9c19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:22.718 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.714+0000 7f4aa2f9e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9c19d1d0 con 0x7f4a9c101650 2023-12-01T01:16:22.718 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.714+0000 7f4aa2f9e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9c19d310 con 0x7f4a9c100cb0 2023-12-01T01:16:22.718 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.714+0000 7f4aa2f9e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9c19d4a0 con 0x7f4a9c0ffac0 2023-12-01T01:16:22.718 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.715+0000 7f4aa279d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a9c101650 0x7f4a9c19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:22.718 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.715+0000 7f4aa279d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a9c101650 0x7f4a9c19cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57398/0 (socket says 172.21.15.77:57398) 2023-12-01T01:16:22.718 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.715+0000 7f4aa179b700 1 --2- 172.21.15.77:0/1041780548 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a9c100cb0 0x7f4a9c195b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:22.718 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.715+0000 7f4aa279d700 1 -- 172.21.15.77:0/1041780548 learned_addr learned my addr 172.21.15.77:0/1041780548 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:22.719 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.715+0000 7f4aa1f9c700 1 --2- 172.21.15.77:0/1041780548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a9c0ffac0 0x7f4a9c195650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:22.719 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.715+0000 7f4aa279d700 1 -- 172.21.15.77:0/1041780548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a9c0ffac0 msgr2=0x7f4a9c195650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:22.719 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.715+0000 7f4aa279d700 1 --2- 172.21.15.77:0/1041780548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a9c0ffac0 0x7f4a9c195650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:22.719 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.715+0000 7f4aa279d700 1 -- 172.21.15.77:0/1041780548 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a9c100cb0 msgr2=0x7f4a9c195b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:22.719 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.715+0000 7f4aa279d700 1 --2- 172.21.15.77:0/1041780548 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a9c100cb0 0x7f4a9c195b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:22.719 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.715+0000 7f4aa279d700 1 -- 172.21.15.77:0/1041780548 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a8c0095f0 con 0x7f4a9c101650 2023-12-01T01:16:22.719 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.715+0000 7f4aa179b700 1 --2- 172.21.15.77:0/1041780548 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a9c100cb0 0x7f4a9c195b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:16:22.719 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.715+0000 7f4aa279d700 1 --2- 172.21.15.77:0/1041780548 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a9c101650 0x7f4a9c19cc00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4a94007ee0 tx=0x7f4a9400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:22.719 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.715+0000 7f4a92ffd700 1 -- 172.21.15.77:0/1041780548 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a94017070 con 0x7f4a9c101650 2023-12-01T01:16:22.719 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.715+0000 7f4aa2f9e700 1 -- 172.21.15.77:0/1041780548 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a9c19d780 con 0x7f4a9c101650 2023-12-01T01:16:22.719 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.715+0000 7f4a92ffd700 1 -- 172.21.15.77:0/1041780548 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a9400f040 con 0x7f4a9c101650 2023-12-01T01:16:22.719 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.715+0000 7f4a92ffd700 1 -- 172.21.15.77:0/1041780548 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a94005550 con 0x7f4a9c101650 2023-12-01T01:16:22.719 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.715+0000 7f4aa2f9e700 1 -- 172.21.15.77:0/1041780548 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a9c19dc90 con 0x7f4a9c101650 2023-12-01T01:16:22.719 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.716+0000 7f4aa2f9e700 1 -- 172.21.15.77:0/1041780548 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a9c1068d0 con 0x7f4a9c101650 2023-12-01T01:16:22.720 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.718+0000 7f4a92ffd700 1 -- 172.21.15.77:0/1041780548 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f4a9400db20 con 0x7f4a9c101650 2023-12-01T01:16:22.720 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.718+0000 7f4a92ffd700 1 --2- 172.21.15.77:0/1041780548 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f4a880a0610 0x7f4a880a2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:22.720 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.719+0000 7f4a92ffd700 1 -- 172.21.15.77:0/1041780548 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4a940cacf0 con 0x7f4a9c101650 2023-12-01T01:16:22.720 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.719+0000 7f4aa1f9c700 1 --2- 172.21.15.77:0/1041780548 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f4a880a0610 0x7f4a880a2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:22.720 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.719+0000 7f4aa1f9c700 1 --2- 172.21.15.77:0/1041780548 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f4a880a0610 0x7f4a880a2ad0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c100b10 tx=0x7f4a98009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:22.724 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.723+0000 7f4a92ffd700 1 -- 172.21.15.77:0/1041780548 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4a94094c90 con 0x7f4a9c101650 2023-12-01T01:16:22.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:22 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/2103324324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:16:22.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:22 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/1470978693' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:16:22.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:22 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/4181048196' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-01T01:16:22.979 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.978+0000 7f4aa2f9e700 1 -- 172.21.15.77:0/1041780548 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4a9c02cc70 con 0x7f4a9c101650 2023-12-01T01:16:22.979 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.978+0000 7f4a92ffd700 1 -- 172.21.15.77:0/1041780548 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v78) v1 ==== 84+0+65 (secure 0 0 0) 0x7f4a940982b0 con 0x7f4a9c101650 2023-12-01T01:16:22.979 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:16:22.980 INFO:teuthology.orchestra.run.smithi077.stdout:[ 2023-12-01T01:16:22.980 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:16:22.980 INFO:teuthology.orchestra.run.smithi077.stdout: "poolnum": 1, 2023-12-01T01:16:22.980 INFO:teuthology.orchestra.run.smithi077.stdout: "poolname": ".mgr" 2023-12-01T01:16:22.980 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:16:22.980 INFO:teuthology.orchestra.run.smithi077.stdout:] 2023-12-01T01:16:22.981 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.980+0000 7f4aa2f9e700 1 -- 172.21.15.77:0/1041780548 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f4a880a0610 msgr2=0x7f4a880a2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:22.982 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.980+0000 7f4aa2f9e700 1 --2- 172.21.15.77:0/1041780548 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f4a880a0610 0x7f4a880a2ad0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c100b10 tx=0x7f4a98009260 comp rx=0 tx=0).stop 2023-12-01T01:16:22.982 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.980+0000 7f4aa2f9e700 1 -- 172.21.15.77:0/1041780548 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a9c101650 msgr2=0x7f4a9c19cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:22.982 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.980+0000 7f4aa2f9e700 1 --2- 172.21.15.77:0/1041780548 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a9c101650 0x7f4a9c19cc00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4a94007ee0 tx=0x7f4a9400d3b0 comp rx=0 tx=0).stop 2023-12-01T01:16:22.982 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.980+0000 7f4aa2f9e700 1 -- 172.21.15.77:0/1041780548 shutdown_connections 2023-12-01T01:16:22.982 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.980+0000 7f4aa2f9e700 1 --2- 172.21.15.77:0/1041780548 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f4a880a0610 0x7f4a880a2ad0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:22.982 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.980+0000 7f4aa2f9e700 1 --2- 172.21.15.77:0/1041780548 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a9c101650 0x7f4a9c19cc00 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:22.982 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.980+0000 7f4aa2f9e700 1 --2- 172.21.15.77:0/1041780548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a9c0ffac0 0x7f4a9c195650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:22.982 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.980+0000 7f4aa2f9e700 1 --2- 172.21.15.77:0/1041780548 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a9c100cb0 0x7f4a9c195b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:22.982 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.980+0000 7f4aa2f9e700 1 -- 172.21.15.77:0/1041780548 >> 172.21.15.77:0/1041780548 conn(0x7f4a9c0fb050 msgr2=0x7f4a9c0fd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:22.982 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.980+0000 7f4aa2f9e700 1 -- 172.21.15.77:0/1041780548 shutdown_connections 2023-12-01T01:16:22.982 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:22.980+0000 7f4aa2f9e700 1 -- 172.21.15.77:0/1041780548 wait complete. 2023-12-01T01:16:22.994 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-01T01:16:22.994 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-01T01:16:23.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:22 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/2103324324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:16:23.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:22 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/1470978693' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:16:23.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:22 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/4181048196' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-01T01:16:23.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:22 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/2103324324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:16:23.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:22 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/1470978693' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:16:23.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:22 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/4181048196' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-01T01:16:23.125 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.122+0000 7f5c826e8700 1 -- 172.21.15.77:0/4123350044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c7c101870 msgr2=0x7f5c7c103c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:23.126 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.122+0000 7f5c826e8700 1 --2- 172.21.15.77:0/4123350044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c7c101870 0x7f5c7c103c60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c009990 tx=0x7f5c6c009ca0 comp rx=0 tx=0).stop 2023-12-01T01:16:23.126 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.122+0000 7f5c826e8700 1 -- 172.21.15.77:0/4123350044 shutdown_connections 2023-12-01T01:16:23.126 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.122+0000 7f5c826e8700 1 --2- 172.21.15.77:0/4123350044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c7c106ad0 0x7f5c7c108ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:23.126 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.122+0000 7f5c826e8700 1 --2- 172.21.15.77:0/4123350044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c7c101870 0x7f5c7c103c60 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:23.126 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.122+0000 7f5c826e8700 1 --2- 172.21.15.77:0/4123350044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c7c1041a0 0x7f5c7c106590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:23.126 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.122+0000 7f5c826e8700 1 -- 172.21.15.77:0/4123350044 >> 172.21.15.77:0/4123350044 conn(0x7f5c7c0fb220 msgr2=0x7f5c7c0fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:23.126 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.122+0000 7f5c826e8700 1 -- 172.21.15.77:0/4123350044 shutdown_connections 2023-12-01T01:16:23.126 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.122+0000 7f5c826e8700 1 -- 172.21.15.77:0/4123350044 wait complete. 2023-12-01T01:16:23.126 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.123+0000 7f5c826e8700 1 Processor -- start 2023-12-01T01:16:23.126 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.123+0000 7f5c826e8700 1 -- start start 2023-12-01T01:16:23.126 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.123+0000 7f5c826e8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c7c101870 0x7f5c7c1011e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:23.126 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.123+0000 7f5c826e8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c7c1041a0 0x7f5c7c0ff820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:23.126 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.123+0000 7f5c826e8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c7c106ad0 0x7f5c7c0ffd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:23.126 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.123+0000 7f5c826e8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c100500 con 0x7f5c7c1041a0 2023-12-01T01:16:23.127 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.123+0000 7f5c826e8700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c100670 con 0x7f5c7c106ad0 2023-12-01T01:16:23.127 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.123+0000 7f5c826e8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c1a0b00 con 0x7f5c7c101870 2023-12-01T01:16:23.127 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.123+0000 7f5c7b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c7c1041a0 0x7f5c7c0ff820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:23.127 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.123+0000 7f5c7b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c7c1041a0 0x7f5c7c0ff820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57410/0 (socket says 172.21.15.77:57410) 2023-12-01T01:16:23.127 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.123+0000 7f5c7b7fe700 1 -- 172.21.15.77:0/542687315 learned_addr learned my addr 172.21.15.77:0/542687315 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:23.127 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.123+0000 7f5c80c85700 1 --2- 172.21.15.77:0/542687315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c7c106ad0 0x7f5c7c0ffd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:23.127 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.123+0000 7f5c7bfff700 1 --2- 172.21.15.77:0/542687315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c7c101870 0x7f5c7c1011e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:23.127 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.123+0000 7f5c7b7fe700 1 -- 172.21.15.77:0/542687315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c7c101870 msgr2=0x7f5c7c1011e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:23.127 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.123+0000 7f5c7b7fe700 1 --2- 172.21.15.77:0/542687315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c7c101870 0x7f5c7c1011e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:23.127 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.123+0000 7f5c7b7fe700 1 -- 172.21.15.77:0/542687315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c7c106ad0 msgr2=0x7f5c7c0ffd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:23.127 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.123+0000 7f5c7b7fe700 1 --2- 172.21.15.77:0/542687315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c7c106ad0 0x7f5c7c0ffd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:23.127 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.123+0000 7f5c7b7fe700 1 -- 172.21.15.77:0/542687315 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c6c0095f0 con 0x7f5c7c1041a0 2023-12-01T01:16:23.127 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.123+0000 7f5c7b7fe700 1 --2- 172.21.15.77:0/542687315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c7c1041a0 0x7f5c7c0ff820 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000d950 tx=0x7f5c7000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:23.127 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.124+0000 7f5c797fa700 1 -- 172.21.15.77:0/542687315 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c70009940 con 0x7f5c7c1041a0 2023-12-01T01:16:23.127 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.124+0000 7f5c797fa700 1 -- 172.21.15.77:0/542687315 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c70010460 con 0x7f5c7c1041a0 2023-12-01T01:16:23.128 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.124+0000 7f5c826e8700 1 -- 172.21.15.77:0/542687315 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c7c1a0de0 con 0x7f5c7c1041a0 2023-12-01T01:16:23.128 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.124+0000 7f5c797fa700 1 -- 172.21.15.77:0/542687315 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7000f620 con 0x7f5c7c1041a0 2023-12-01T01:16:23.128 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.124+0000 7f5c826e8700 1 -- 172.21.15.77:0/542687315 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c7c1a1260 con 0x7f5c7c1041a0 2023-12-01T01:16:23.128 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.125+0000 7f5c826e8700 1 -- 172.21.15.77:0/542687315 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c7c0fcda0 con 0x7f5c7c1041a0 2023-12-01T01:16:23.129 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.126+0000 7f5c797fa700 1 -- 172.21.15.77:0/542687315 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f5c700108d0 con 0x7f5c7c1041a0 2023-12-01T01:16:23.129 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.126+0000 7f5c797fa700 1 --2- 172.21.15.77:0/542687315 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f5c640a0520 0x7f5c640a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:23.129 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.126+0000 7f5c7bfff700 1 --2- 172.21.15.77:0/542687315 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f5c640a0520 0x7f5c640a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:23.129 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.126+0000 7f5c797fa700 1 -- 172.21.15.77:0/542687315 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5c700cb360 con 0x7f5c7c1041a0 2023-12-01T01:16:23.129 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.127+0000 7f5c7bfff700 1 --2- 172.21.15.77:0/542687315 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f5c640a0520 0x7f5c640a29e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00b2e0 tx=0x7f5c6c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:23.132 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.131+0000 7f5c797fa700 1 -- 172.21.15.77:0/542687315 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5c700953a0 con 0x7f5c7c1041a0 2023-12-01T01:16:23.330 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.328+0000 7f5c826e8700 1 -- 172.21.15.77:0/542687315 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f5c7c02cc70 con 0x7f5c7c1041a0 2023-12-01T01:16:23.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:23 smithi077 ceph-mon[98867]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:16:23.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:23 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:23.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:23 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:23.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:23 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:23.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:23 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:23.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:23 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:23.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:23 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:23.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:23 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/1041780548' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-12-01T01:16:23.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:23 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/542687315' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-12-01T01:16:23.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:23 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:16:23.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:23 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:16:23.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:23 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:23.919 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.917+0000 7f5c797fa700 1 -- 172.21.15.77:0/542687315 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v79) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5c700989c0 con 0x7f5c7c1041a0 2023-12-01T01:16:23.923 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.922+0000 7f5c826e8700 1 -- 172.21.15.77:0/542687315 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f5c640a0520 msgr2=0x7f5c640a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:23.923 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.922+0000 7f5c826e8700 1 --2- 172.21.15.77:0/542687315 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f5c640a0520 0x7f5c640a29e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00b2e0 tx=0x7f5c6c005fb0 comp rx=0 tx=0).stop 2023-12-01T01:16:23.923 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.922+0000 7f5c826e8700 1 -- 172.21.15.77:0/542687315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c7c1041a0 msgr2=0x7f5c7c0ff820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:23.923 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.922+0000 7f5c826e8700 1 --2- 172.21.15.77:0/542687315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c7c1041a0 0x7f5c7c0ff820 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000d950 tx=0x7f5c7000dc60 comp rx=0 tx=0).stop 2023-12-01T01:16:23.923 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.922+0000 7f5c826e8700 1 -- 172.21.15.77:0/542687315 shutdown_connections 2023-12-01T01:16:23.923 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.922+0000 7f5c826e8700 1 --2- 172.21.15.77:0/542687315 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f5c640a0520 0x7f5c640a29e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:23.923 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.922+0000 7f5c826e8700 1 --2- 172.21.15.77:0/542687315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c7c106ad0 0x7f5c7c0ffd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:23.924 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.922+0000 7f5c826e8700 1 --2- 172.21.15.77:0/542687315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c7c101870 0x7f5c7c1011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:23.924 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.922+0000 7f5c826e8700 1 --2- 172.21.15.77:0/542687315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c7c1041a0 0x7f5c7c0ff820 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:23.924 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.922+0000 7f5c826e8700 1 -- 172.21.15.77:0/542687315 >> 172.21.15.77:0/542687315 conn(0x7f5c7c0fb220 msgr2=0x7f5c7c1024e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:23.924 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.922+0000 7f5c826e8700 1 -- 172.21.15.77:0/542687315 shutdown_connections 2023-12-01T01:16:23.924 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:23.922+0000 7f5c826e8700 1 -- 172.21.15.77:0/542687315 wait complete. 2023-12-01T01:16:23.924 INFO:teuthology.orchestra.run.smithi077.stderr:pool 'cephfs_metadata' created 2023-12-01T01:16:23.936 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-01T01:16:24.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:23 smithi203 ceph-mon[103581]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:16:24.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:23 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:24.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:23 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:24.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:23 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:24.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:23 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:24.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:23 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:24.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:23 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:24.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:23 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/1041780548' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-12-01T01:16:24.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:23 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/542687315' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-12-01T01:16:24.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:23 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:16:24.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:23 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:16:24.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:23 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:24.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:23 smithi137 ceph-mon[103312]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:16:24.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:23 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:24.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:23 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:24.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:23 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:24.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:23 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:24.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:23 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:24.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:23 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:24.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:23 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/1041780548' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-12-01T01:16:24.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:23 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/542687315' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-12-01T01:16:24.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:23 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:16:24.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:23 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:16:24.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:23 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:24.072 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.068+0000 7f7bca9dd700 1 -- 172.21.15.77:0/2185198783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7bc4102930 msgr2=0x7f7bc41091e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:24.072 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.068+0000 7f7bca9dd700 1 --2- 172.21.15.77:0/2185198783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7bc4102930 0x7f7bc41091e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc00b0d0 tx=0x7f7bbc00b3e0 comp rx=0 tx=0).stop 2023-12-01T01:16:24.072 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.068+0000 7f7bca9dd700 1 -- 172.21.15.77:0/2185198783 shutdown_connections 2023-12-01T01:16:24.072 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.068+0000 7f7bca9dd700 1 --2- 172.21.15.77:0/2185198783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7bc4102930 0x7f7bc41091e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:24.072 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.068+0000 7f7bca9dd700 1 --2- 172.21.15.77:0/2185198783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bc4100da0 0x7f7bc41011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:24.072 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.068+0000 7f7bca9dd700 1 --2- 172.21.15.77:0/2185198783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7bc4101f90 0x7f7bc41023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:24.072 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.068+0000 7f7bca9dd700 1 -- 172.21.15.77:0/2185198783 >> 172.21.15.77:0/2185198783 conn(0x7f7bc40fc310 msgr2=0x7f7bc40fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:24.072 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.068+0000 7f7bca9dd700 1 -- 172.21.15.77:0/2185198783 shutdown_connections 2023-12-01T01:16:24.072 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.068+0000 7f7bca9dd700 1 -- 172.21.15.77:0/2185198783 wait complete. 2023-12-01T01:16:24.072 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.068+0000 7f7bca9dd700 1 Processor -- start 2023-12-01T01:16:24.073 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.069+0000 7f7bca9dd700 1 -- start start 2023-12-01T01:16:24.073 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.069+0000 7f7bca9dd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7bc4100da0 0x7f7bc4196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:24.073 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.069+0000 7f7bca9dd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7bc4101f90 0x7f7bc4197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:24.073 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.069+0000 7f7bca9dd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bc4102930 0x7f7bc419e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:24.073 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.069+0000 7f7bca9dd700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc419e6b0 con 0x7f7bc4100da0 2023-12-01T01:16:24.073 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.069+0000 7f7bca9dd700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc419e7f0 con 0x7f7bc4102930 2023-12-01T01:16:24.073 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.069+0000 7f7bca9dd700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc419e980 con 0x7f7bc4101f90 2023-12-01T01:16:24.073 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.069+0000 7f7bc3fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7bc4100da0 0x7f7bc4196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:24.073 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.069+0000 7f7bc3fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7bc4100da0 0x7f7bc4196b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57422/0 (socket says 172.21.15.77:57422) 2023-12-01T01:16:24.073 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.069+0000 7f7bc3fff700 1 -- 172.21.15.77:0/4247075960 learned_addr learned my addr 172.21.15.77:0/4247075960 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:24.073 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.069+0000 7f7bc37fe700 1 --2- 172.21.15.77:0/4247075960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7bc4101f90 0x7f7bc4197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:24.073 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.069+0000 7f7bc8f7a700 1 --2- 172.21.15.77:0/4247075960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bc4102930 0x7f7bc419e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:24.073 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.069+0000 7f7bc3fff700 1 -- 172.21.15.77:0/4247075960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7bc4101f90 msgr2=0x7f7bc4197070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:24.073 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.069+0000 7f7bc3fff700 1 --2- 172.21.15.77:0/4247075960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7bc4101f90 0x7f7bc4197070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:24.074 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.069+0000 7f7bc3fff700 1 -- 172.21.15.77:0/4247075960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bc4102930 msgr2=0x7f7bc419e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:24.074 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.069+0000 7f7bc3fff700 1 --2- 172.21.15.77:0/4247075960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bc4102930 0x7f7bc419e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:24.074 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.069+0000 7f7bc3fff700 1 -- 172.21.15.77:0/4247075960 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bbc009d00 con 0x7f7bc4100da0 2023-12-01T01:16:24.074 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.069+0000 7f7bc37fe700 1 --2- 172.21.15.77:0/4247075960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7bc4101f90 0x7f7bc4197070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:16:24.074 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.069+0000 7f7bc3fff700 1 --2- 172.21.15.77:0/4247075960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7bc4100da0 0x7f7bc4196b30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7bb800e970 tx=0x7f7bb800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:24.074 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.069+0000 7f7bc17fa700 1 -- 172.21.15.77:0/4247075960 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb8009790 con 0x7f7bc4100da0 2023-12-01T01:16:24.074 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.070+0000 7f7bc17fa700 1 -- 172.21.15.77:0/4247075960 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bb80044e0 con 0x7f7bc4100da0 2023-12-01T01:16:24.074 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.070+0000 7f7bc17fa700 1 -- 172.21.15.77:0/4247075960 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb8003e30 con 0x7f7bc4100da0 2023-12-01T01:16:24.074 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.070+0000 7f7bca9dd700 1 -- 172.21.15.77:0/4247075960 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bc419ec60 con 0x7f7bc4100da0 2023-12-01T01:16:24.074 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.070+0000 7f7bca9dd700 1 -- 172.21.15.77:0/4247075960 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bc419f170 con 0x7f7bc4100da0 2023-12-01T01:16:24.074 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.070+0000 7f7bca9dd700 1 -- 172.21.15.77:0/4247075960 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bc41082c0 con 0x7f7bc4100da0 2023-12-01T01:16:24.075 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.072+0000 7f7bc17fa700 1 -- 172.21.15.77:0/4247075960 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f7bb8003f90 con 0x7f7bc4100da0 2023-12-01T01:16:24.075 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.072+0000 7f7bc17fa700 1 --2- 172.21.15.77:0/4247075960 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7bac0a05a0 0x7f7bac0a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:24.075 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.072+0000 7f7bc37fe700 1 --2- 172.21.15.77:0/4247075960 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7bac0a05a0 0x7f7bac0a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:24.075 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.072+0000 7f7bc17fa700 1 -- 172.21.15.77:0/4247075960 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f7bb8014070 con 0x7f7bc4100da0 2023-12-01T01:16:24.075 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.073+0000 7f7bc37fe700 1 --2- 172.21.15.77:0/4247075960 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7bac0a05a0 0x7f7bac0a2a60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4009cb0 tx=0x7f7bb4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:24.077 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.076+0000 7f7bc17fa700 1 -- 172.21.15.77:0/4247075960 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7bb80958b0 con 0x7f7bc4100da0 2023-12-01T01:16:24.277 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.276+0000 7f7bca9dd700 1 -- 172.21.15.77:0/4247075960 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f7bc4061af0 con 0x7f7bc4100da0 2023-12-01T01:16:24.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:24 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:16:24.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:24 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/542687315' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-12-01T01:16:24.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:24 smithi077 ceph-mon[98867]: osdmap e79: 12 total, 12 up, 12 in 2023-12-01T01:16:24.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:24 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:24.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:24 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/4247075960' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-12-01T01:16:24.931 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.930+0000 7f7bc17fa700 1 -- 172.21.15.77:0/4247075960 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v80) v1 ==== 170+0+0 (secure 0 0 0) 0x7f7bb8098ed0 con 0x7f7bc4100da0 2023-12-01T01:16:24.937 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.936+0000 7f7bca9dd700 1 -- 172.21.15.77:0/4247075960 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7bac0a05a0 msgr2=0x7f7bac0a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:24.937 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.936+0000 7f7bca9dd700 1 --2- 172.21.15.77:0/4247075960 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7bac0a05a0 0x7f7bac0a2a60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4009cb0 tx=0x7f7bb4009260 comp rx=0 tx=0).stop 2023-12-01T01:16:24.937 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.936+0000 7f7bca9dd700 1 -- 172.21.15.77:0/4247075960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7bc4100da0 msgr2=0x7f7bc4196b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:24.938 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.936+0000 7f7bca9dd700 1 --2- 172.21.15.77:0/4247075960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7bc4100da0 0x7f7bc4196b30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7bb800e970 tx=0x7f7bb800ed30 comp rx=0 tx=0).stop 2023-12-01T01:16:24.938 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.937+0000 7f7bca9dd700 1 -- 172.21.15.77:0/4247075960 shutdown_connections 2023-12-01T01:16:24.938 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.937+0000 7f7bca9dd700 1 --2- 172.21.15.77:0/4247075960 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7bac0a05a0 0x7f7bac0a2a60 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:24.938 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.937+0000 7f7bca9dd700 1 --2- 172.21.15.77:0/4247075960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bc4102930 0x7f7bc419e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:24.939 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.937+0000 7f7bca9dd700 1 --2- 172.21.15.77:0/4247075960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7bc4100da0 0x7f7bc4196b30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:24.939 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.937+0000 7f7bca9dd700 1 --2- 172.21.15.77:0/4247075960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7bc4101f90 0x7f7bc4197070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:24.939 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.937+0000 7f7bca9dd700 1 -- 172.21.15.77:0/4247075960 >> 172.21.15.77:0/4247075960 conn(0x7f7bc40fc310 msgr2=0x7f7bc4105400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:24.939 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.937+0000 7f7bca9dd700 1 -- 172.21.15.77:0/4247075960 shutdown_connections 2023-12-01T01:16:24.939 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:24.937+0000 7f7bca9dd700 1 -- 172.21.15.77:0/4247075960 wait complete. 2023-12-01T01:16:24.944 INFO:teuthology.orchestra.run.smithi077.stderr:pool 'cephfs_data' created 2023-12-01T01:16:24.958 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-01T01:16:25.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:24 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:16:25.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:24 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/542687315' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-12-01T01:16:25.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:24 smithi203 ceph-mon[103581]: osdmap e79: 12 total, 12 up, 12 in 2023-12-01T01:16:25.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:24 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:25.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:24 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/4247075960' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-12-01T01:16:25.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:24 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:16:25.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:24 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/542687315' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-12-01T01:16:25.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:24 smithi137 ceph-mon[103312]: osdmap e79: 12 total, 12 up, 12 in 2023-12-01T01:16:25.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:24 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:25.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:24 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/4247075960' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-12-01T01:16:25.084 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.082+0000 7fe2c76ee700 1 -- 172.21.15.77:0/340387499 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2c00ffc80 msgr2=0x7fe2c01000a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:25.084 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.082+0000 7fe2c76ee700 1 --2- 172.21.15.77:0/340387499 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2c00ffc80 0x7fe2c01000a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe2b000b0d0 tx=0x7fe2b000b3e0 comp rx=0 tx=0).stop 2023-12-01T01:16:25.084 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.082+0000 7fe2c76ee700 1 -- 172.21.15.77:0/340387499 shutdown_connections 2023-12-01T01:16:25.084 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.082+0000 7fe2c76ee700 1 --2- 172.21.15.77:0/340387499 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe2c0101810 0x7fe2c01080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:25.084 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.082+0000 7fe2c76ee700 1 --2- 172.21.15.77:0/340387499 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2c00ffc80 0x7fe2c01000a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:25.084 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.082+0000 7fe2c76ee700 1 --2- 172.21.15.77:0/340387499 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2c0100e70 0x7fe2c01012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:25.084 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.082+0000 7fe2c76ee700 1 -- 172.21.15.77:0/340387499 >> 172.21.15.77:0/340387499 conn(0x7fe2c00fb1f0 msgr2=0x7fe2c00fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:25.084 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.083+0000 7fe2c76ee700 1 -- 172.21.15.77:0/340387499 shutdown_connections 2023-12-01T01:16:25.084 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.083+0000 7fe2c76ee700 1 -- 172.21.15.77:0/340387499 wait complete. 2023-12-01T01:16:25.087 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.083+0000 7fe2c76ee700 1 Processor -- start 2023-12-01T01:16:25.087 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.083+0000 7fe2c76ee700 1 -- start start 2023-12-01T01:16:25.087 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.083+0000 7fe2c76ee700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2c00ffc80 0x7fe2c0068a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:25.087 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.083+0000 7fe2c76ee700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2c0100e70 0x7fe2c019b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:25.087 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.083+0000 7fe2c76ee700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe2c0101810 0x7fe2c019df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:25.087 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.083+0000 7fe2c76ee700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c019e710 con 0x7fe2c0100e70 2023-12-01T01:16:25.087 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.083+0000 7fe2c76ee700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c019e880 con 0x7fe2c00ffc80 2023-12-01T01:16:25.087 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.083+0000 7fe2c76ee700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c019ea40 con 0x7fe2c0101810 2023-12-01T01:16:25.087 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.083+0000 7fe2c4c89700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2c0100e70 0x7fe2c019b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:25.087 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.083+0000 7fe2c4c89700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2c0100e70 0x7fe2c019b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57438/0 (socket says 172.21.15.77:57438) 2023-12-01T01:16:25.088 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.083+0000 7fe2c4c89700 1 -- 172.21.15.77:0/2212310524 learned_addr learned my addr 172.21.15.77:0/2212310524 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:25.088 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.083+0000 7fe2c5c8b700 1 --2- 172.21.15.77:0/2212310524 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe2c0101810 0x7fe2c019df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:25.088 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.084+0000 7fe2c548a700 1 --2- 172.21.15.77:0/2212310524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2c00ffc80 0x7fe2c0068a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:25.088 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.084+0000 7fe2c4c89700 1 -- 172.21.15.77:0/2212310524 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe2c0101810 msgr2=0x7fe2c019df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:25.088 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.084+0000 7fe2c4c89700 1 --2- 172.21.15.77:0/2212310524 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe2c0101810 0x7fe2c019df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:25.088 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.084+0000 7fe2c4c89700 1 -- 172.21.15.77:0/2212310524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2c00ffc80 msgr2=0x7fe2c0068a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:25.088 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.084+0000 7fe2c4c89700 1 --2- 172.21.15.77:0/2212310524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2c00ffc80 0x7fe2c0068a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:25.088 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.084+0000 7fe2c4c89700 1 -- 172.21.15.77:0/2212310524 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b0009d00 con 0x7fe2c0100e70 2023-12-01T01:16:25.088 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.084+0000 7fe2c5c8b700 1 --2- 172.21.15.77:0/2212310524 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe2c0101810 0x7fe2c019df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:16:25.088 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.084+0000 7fe2c4c89700 1 --2- 172.21.15.77:0/2212310524 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2c0100e70 0x7fe2c019b930 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe2b400e9a0 tx=0x7fe2b400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:25.088 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.084+0000 7fe2be7fc700 1 -- 172.21.15.77:0/2212310524 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4009790 con 0x7fe2c0100e70 2023-12-01T01:16:25.088 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.084+0000 7fe2be7fc700 1 -- 172.21.15.77:0/2212310524 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2b4010430 con 0x7fe2c0100e70 2023-12-01T01:16:25.088 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.084+0000 7fe2c76ee700 1 -- 172.21.15.77:0/2212310524 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2c019ed20 con 0x7fe2c0100e70 2023-12-01T01:16:25.088 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.084+0000 7fe2be7fc700 1 -- 172.21.15.77:0/2212310524 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4018680 con 0x7fe2c0100e70 2023-12-01T01:16:25.089 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.084+0000 7fe2c76ee700 1 -- 172.21.15.77:0/2212310524 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2c019f220 con 0x7fe2c0100e70 2023-12-01T01:16:25.089 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.085+0000 7fe2c76ee700 1 -- 172.21.15.77:0/2212310524 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2c010b490 con 0x7fe2c0100e70 2023-12-01T01:16:25.089 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.087+0000 7fe2be7fc700 1 -- 172.21.15.77:0/2212310524 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fe2b40105a0 con 0x7fe2c0100e70 2023-12-01T01:16:25.089 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.088+0000 7fe2be7fc700 1 --2- 172.21.15.77:0/2212310524 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fe2a80a05a0 0x7fe2a80a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:25.089 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.088+0000 7fe2be7fc700 1 -- 172.21.15.77:0/2212310524 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7879+0+0 (secure 0 0 0) 0x7fe2b4014070 con 0x7fe2c0100e70 2023-12-01T01:16:25.090 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.088+0000 7fe2c548a700 1 --2- 172.21.15.77:0/2212310524 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fe2a80a05a0 0x7fe2a80a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:25.090 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.088+0000 7fe2c548a700 1 --2- 172.21.15.77:0/2212310524 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fe2a80a05a0 0x7fe2a80a2a60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009fd0 tx=0x7fe2b000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:25.092 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.091+0000 7fe2be7fc700 1 -- 172.21.15.77:0/2212310524 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe2b40958b0 con 0x7fe2c0100e70 2023-12-01T01:16:25.344 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.343+0000 7fe2c76ee700 1 -- 172.21.15.77:0/2212310524 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe2c0106bd0 con 0x7fe2c0100e70 2023-12-01T01:16:25.350 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.349+0000 7fe2be7fc700 1 -- 172.21.15.77:0/2212310524 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7fe2b4026020 con 0x7fe2c0100e70 2023-12-01T01:16:25.360 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.358+0000 7fe2c76ee700 1 -- 172.21.15.77:0/2212310524 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fe2a80a05a0 msgr2=0x7fe2a80a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:25.360 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.358+0000 7fe2c76ee700 1 --2- 172.21.15.77:0/2212310524 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fe2a80a05a0 0x7fe2a80a2a60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009fd0 tx=0x7fe2b000b040 comp rx=0 tx=0).stop 2023-12-01T01:16:25.360 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.358+0000 7fe2c76ee700 1 -- 172.21.15.77:0/2212310524 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2c0100e70 msgr2=0x7fe2c019b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:25.360 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.358+0000 7fe2c76ee700 1 --2- 172.21.15.77:0/2212310524 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2c0100e70 0x7fe2c019b930 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe2b400e9a0 tx=0x7fe2b400ecb0 comp rx=0 tx=0).stop 2023-12-01T01:16:25.360 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.359+0000 7fe2c76ee700 1 -- 172.21.15.77:0/2212310524 shutdown_connections 2023-12-01T01:16:25.360 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.359+0000 7fe2c76ee700 1 --2- 172.21.15.77:0/2212310524 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fe2a80a05a0 0x7fe2a80a2a60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:25.360 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.359+0000 7fe2c76ee700 1 --2- 172.21.15.77:0/2212310524 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe2c0101810 0x7fe2c019df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:25.360 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.359+0000 7fe2c76ee700 1 --2- 172.21.15.77:0/2212310524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2c00ffc80 0x7fe2c0068a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:25.360 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.359+0000 7fe2c76ee700 1 --2- 172.21.15.77:0/2212310524 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2c0100e70 0x7fe2c019b930 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:25.361 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.359+0000 7fe2c76ee700 1 -- 172.21.15.77:0/2212310524 >> 172.21.15.77:0/2212310524 conn(0x7fe2c00fb1f0 msgr2=0x7fe2c00fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:25.365 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.363+0000 7fe2c76ee700 1 -- 172.21.15.77:0/2212310524 shutdown_connections 2023-12-01T01:16:25.365 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.364+0000 7fe2c76ee700 1 -- 172.21.15.77:0/2212310524 wait complete. 2023-12-01T01:16:25.369 INFO:teuthology.orchestra.run.smithi077.stderr:new fs with metadata pool 2 and data pool 3 2023-12-01T01:16:25.384 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:16:25.512 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.509+0000 7f2432f0b700 1 -- 172.21.15.77:0/811379705 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f242c073770 msgr2=0x7f242c073bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:25.513 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.509+0000 7f2432f0b700 1 --2- 172.21.15.77:0/811379705 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f242c073770 0x7f242c073bf0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2418009990 tx=0x7f2418009ca0 comp rx=0 tx=0).stop 2023-12-01T01:16:25.513 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.509+0000 7f2432f0b700 1 -- 172.21.15.77:0/811379705 shutdown_connections 2023-12-01T01:16:25.513 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.509+0000 7f2432f0b700 1 --2- 172.21.15.77:0/811379705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f242c108cc0 0x7f242c10b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:25.513 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.509+0000 7f2432f0b700 1 --2- 172.21.15.77:0/811379705 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f242c074d50 0x7f242c0731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:25.513 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.509+0000 7f2432f0b700 1 --2- 172.21.15.77:0/811379705 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f242c073770 0x7f242c073bf0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:25.513 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.509+0000 7f2432f0b700 1 -- 172.21.15.77:0/811379705 >> 172.21.15.77:0/811379705 conn(0x7f242c0fc340 msgr2=0x7f242c0fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:25.513 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.509+0000 7f2432f0b700 1 -- 172.21.15.77:0/811379705 shutdown_connections 2023-12-01T01:16:25.513 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.509+0000 7f2432f0b700 1 -- 172.21.15.77:0/811379705 wait complete. 2023-12-01T01:16:25.513 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.510+0000 7f2432f0b700 1 Processor -- start 2023-12-01T01:16:25.513 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.510+0000 7f2432f0b700 1 -- start start 2023-12-01T01:16:25.513 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.510+0000 7f2432f0b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f242c073770 0x7f242c198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:25.513 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.510+0000 7f2432f0b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f242c074d50 0x7f242c199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:25.513 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.510+0000 7f2432f0b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f242c108cc0 0x7f242c1a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:25.513 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.510+0000 7f2432f0b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242c1a0770 con 0x7f242c073770 2023-12-01T01:16:25.514 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.510+0000 7f2432f0b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242c1a08b0 con 0x7f242c074d50 2023-12-01T01:16:25.514 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.510+0000 7f2432f0b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242c1a0a40 con 0x7f242c108cc0 2023-12-01T01:16:25.514 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.510+0000 7f2430ca7700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f242c073770 0x7f242c198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:25.514 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.510+0000 7f2430ca7700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f242c073770 0x7f242c198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57456/0 (socket says 172.21.15.77:57456) 2023-12-01T01:16:25.514 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.510+0000 7f2430ca7700 1 -- 172.21.15.77:0/2426224369 learned_addr learned my addr 172.21.15.77:0/2426224369 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:25.514 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.510+0000 7f242bfff700 1 --2- 172.21.15.77:0/2426224369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f242c074d50 0x7f242c199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:25.514 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.510+0000 7f24314a8700 1 --2- 172.21.15.77:0/2426224369 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f242c108cc0 0x7f242c1a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:25.514 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.510+0000 7f2430ca7700 1 -- 172.21.15.77:0/2426224369 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f242c108cc0 msgr2=0x7f242c1a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:25.514 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.510+0000 7f2430ca7700 1 --2- 172.21.15.77:0/2426224369 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f242c108cc0 0x7f242c1a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:25.514 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.510+0000 7f2430ca7700 1 -- 172.21.15.77:0/2426224369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f242c074d50 msgr2=0x7f242c199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:25.514 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.510+0000 7f2430ca7700 1 --2- 172.21.15.77:0/2426224369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f242c074d50 0x7f242c199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:25.514 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.510+0000 7f2430ca7700 1 -- 172.21.15.77:0/2426224369 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24180095f0 con 0x7f242c073770 2023-12-01T01:16:25.515 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.510+0000 7f2430ca7700 1 --2- 172.21.15.77:0/2426224369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f242c073770 0x7f242c198bf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f242000d950 tx=0x7f242000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:25.515 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.511+0000 7f2429ffb700 1 -- 172.21.15.77:0/2426224369 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242000f890 con 0x7f242c073770 2023-12-01T01:16:25.515 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.511+0000 7f2429ffb700 1 -- 172.21.15.77:0/2426224369 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2420010460 con 0x7f242c073770 2023-12-01T01:16:25.515 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.511+0000 7f2429ffb700 1 -- 172.21.15.77:0/2426224369 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242000e610 con 0x7f242c073770 2023-12-01T01:16:25.515 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.511+0000 7f2432f0b700 1 -- 172.21.15.77:0/2426224369 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f242c1a0d20 con 0x7f242c073770 2023-12-01T01:16:25.515 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.511+0000 7f2432f0b700 1 -- 172.21.15.77:0/2426224369 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f242c1a1180 con 0x7f242c073770 2023-12-01T01:16:25.516 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.512+0000 7f2432f0b700 1 -- 172.21.15.77:0/2426224369 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f242c107ad0 con 0x7f242c073770 2023-12-01T01:16:25.516 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.513+0000 7f2429ffb700 1 -- 172.21.15.77:0/2426224369 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f24200108d0 con 0x7f242c073770 2023-12-01T01:16:25.516 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.513+0000 7f2429ffb700 1 --2- 172.21.15.77:0/2426224369 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f241c0a0570 0x7f241c0a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:25.516 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.514+0000 7f242bfff700 1 --2- 172.21.15.77:0/2426224369 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f241c0a0570 0x7f241c0a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:25.516 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.514+0000 7f2429ffb700 1 -- 172.21.15.77:0/2426224369 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f24200cb5e0 con 0x7f242c073770 2023-12-01T01:16:25.516 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.514+0000 7f242bfff700 1 --2- 172.21.15.77:0/2426224369 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f241c0a0570 0x7f241c0a2a30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f241800b2e0 tx=0x7f2418005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:25.519 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.518+0000 7f2429ffb700 1 -- 172.21.15.77:0/2426224369 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f24200952d0 con 0x7f242c073770 2023-12-01T01:16:25.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:25 smithi077 ceph-mon[98867]: pgmap v249: 33 pgs: 1 creating+peering, 1 active+clean, 31 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:16:25.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:25 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/4247075960' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-12-01T01:16:25.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:25 smithi077 ceph-mon[98867]: osdmap e80: 12 total, 12 up, 12 in 2023-12-01T01:16:25.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:25 smithi077 ceph-mon[98867]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-12-01T01:16:25.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:25 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/2212310524' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-12-01T01:16:25.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:25 smithi077 ceph-mon[98867]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-12-01T01:16:25.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:25 smithi077 ceph-mon[98867]: osdmap e81: 12 total, 12 up, 12 in 2023-12-01T01:16:25.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:25 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/2212310524' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-12-01T01:16:25.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:25 smithi077 ceph-mon[98867]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-12-01T01:16:25.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:25 smithi077 ceph-mon[98867]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-12-01T01:16:25.733 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.732+0000 7f2432f0b700 1 -- 172.21.15.77:0/2426224369 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f242c02cf20 con 0x7f242c073770 2023-12-01T01:16:25.735 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.734+0000 7f2429ffb700 1 -- 172.21.15.77:0/2426224369 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22144 (secure 0 0 0) 0x7f24200b8020 con 0x7f242c073770 2023-12-01T01:16:25.735 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:16:25.735 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":81,"fsid":"2a06e29a-8fe6-11ee-95a2-87774f69a715","created":"2023-12-01T01:09:30.210398+0000","modified":"2023-12-01T01:16:25.344878+0000","last_up_change":"2023-12-01T01:15:44.586092+0000","last_in_change":"2023-12-01T01:15:26.305439+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:12:30.514229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:16:23.568860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-01T01:16:24.531137+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fc651e62-a103-4ab8-a76d-d3708ea6aeec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6803","nonce":2244799094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6805","nonce":2244799094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6809","nonce":2244799094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6807","nonce":2244799094}]},"public_addr":"172.21.15.77:6803/2244799094","cluster_addr":"172.21.15.77:6805/2244799094","heartbeat_back_addr":"172.21.15.77:6809/2244799094","heartbeat_front_addr":"172.21.15.77:6807/2244799094","state":["exists","up"]},{"osd":1,"uuid":"0adf8c54-49ec-49a0-8bf1-af1f20e60729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6801","nonce":224194890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6803","nonce":224194890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6807","nonce":224194890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6805","nonce":224194890}]},"public_addr":"172.21.15.137:6801/224194890","cluster_addr":"172.21.15.137:6803/224194890","heartbeat_back_addr":"172.21.15.137:6807/224194890","heartbeat_front_addr":"172.21.15.137:6805/224194890","state":["exists","up"]},{"osd":2,"uuid":"5a69802c-dede-4544-9a12-436f5c1c0038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6801","nonce":4078459110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6803","nonce":4078459110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6807","nonce":4078459110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6805","nonce":4078459110}]},"public_addr":"172.21.15.203:6801/4078459110","cluster_addr":"172.21.15.203:6803/4078459110","heartbeat_back_addr":"172.21.15.203:6807/4078459110","heartbeat_front_addr":"172.21.15.203:6805/4078459110","state":["exists","up"]},{"osd":3,"uuid":"ad325a97-580b-4c99-bf12-cbb0d92a6d0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6811","nonce":1454440477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6813","nonce":1454440477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6817","nonce":1454440477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6815","nonce":1454440477}]},"public_addr":"172.21.15.77:6811/1454440477","cluster_addr":"172.21.15.77:6813/1454440477","heartbeat_back_addr":"172.21.15.77:6817/1454440477","heartbeat_front_addr":"172.21.15.77:6815/1454440477","state":["exists","up"]},{"osd":4,"uuid":"1e3ac1ba-98bb-4b8b-8e26-50064c66bf2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6809","nonce":1103241451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6811","nonce":1103241451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6815","nonce":1103241451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6813","nonce":1103241451}]},"public_addr":"172.21.15.137:6809/1103241451","cluster_addr":"172.21.15.137:6811/1103241451","heartbeat_back_addr":"172.21.15.137:6815/1103241451","heartbeat_front_addr":"172.21.15.137:6813/1103241451","state":["exists","up"]},{"osd":5,"uuid":"eb32c016-abeb-44ab-abf0-19dd09235736","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6809","nonce":2648569482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6811","nonce":2648569482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6815","nonce":2648569482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6813","nonce":2648569482}]},"public_addr":"172.21.15.203:6809/2648569482","cluster_addr":"172.21.15.203:6811/2648569482","heartbeat_back_addr":"172.21.15.203:6815/2648569482","heartbeat_front_addr":"172.21.15.203:6813/2648569482","state":["exists","up"]},{"osd":6,"uuid":"0e3fa97e-2ca6-418c-9e8d-a6fe3c56adb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6819","nonce":2463414651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6821","nonce":2463414651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6825","nonce":2463414651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6823","nonce":2463414651}]},"public_addr":"172.21.15.77:6819/2463414651","cluster_addr":"172.21.15.77:6821/2463414651","heartbeat_back_addr":"172.21.15.77:6825/2463414651","heartbeat_front_addr":"172.21.15.77:6823/2463414651","state":["exists","up"]},{"osd":7,"uuid":"d7398c01-0f98-4138-8a49-c3ff0aae7e88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6817","nonce":3108330766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6819","nonce":3108330766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6823","nonce":3108330766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6821","nonce":3108330766}]},"public_addr":"172.21.15.137:6817/3108330766","cluster_addr":"172.21.15.137:6819/3108330766","heartbeat_back_addr":"172.21.15.137:6823/3108330766","heartbeat_front_addr":"172.21.15.137:6821/3108330766","state":["exists","up"]},{"osd":8,"uuid":"0c09050d-3e43-410c-8cd7-bcb891af56b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6817","nonce":1161158776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6819","nonce":1161158776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6823","nonce":1161158776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6821","nonce":1161158776}]},"public_addr":"172.21.15.203:6817/1161158776","cluster_addr":"172.21.15.203:6819/1161158776","heartbeat_back_addr":"172.21.15.203:6823/1161158776","heartbeat_front_addr":"172.21.15.203:6821/1161158776","state":["exists","up"]},{"osd":9,"uuid":"527ecb49-95ec-48bd-9872-3b07fae5c5ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6827","nonce":2409880685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6829","nonce":2409880685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6833","nonce":2409880685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6831","nonce":2409880685}]},"public_addr":"172.21.15.77:6827/2409880685","cluster_addr":"172.21.15.77:6829/2409880685","heartbeat_back_addr":"172.21.15.77:6833/2409880685","heartbeat_front_addr":"172.21.15.77:6831/2409880685","state":["exists","up"]},{"osd":10,"uuid":"1680bb3e-a3c8-427b-af68-e2214f32bed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6825","nonce":374920146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6827","nonce":374920146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6831","nonce":374920146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6829","nonce":374920146}]},"public_addr":"172.21.15.137:6825/374920146","cluster_addr":"172.21.15.137:6827/374920146","heartbeat_back_addr":"172.21.15.137:6831/374920146","heartbeat_front_addr":"172.21.15.137:6829/374920146","state":["exists","up"]},{"osd":11,"uuid":"8a533466-023d-4803-ae36-9a9f1715e49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6825","nonce":2161588962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6827","nonce":2161588962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6831","nonce":2161588962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6829","nonce":2161588962}]},"public_addr":"172.21.15.203:6825/2161588962","cluster_addr":"172.21.15.203:6827/2161588962","heartbeat_back_addr":"172.21.15.203:6831/2161588962","heartbeat_front_addr":"172.21.15.203:6829/2161588962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:11:45.420509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:06.210115+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:28.191307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:49.594116+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:11.261689+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:33.376541+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:54.968336+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:16.775310+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:38.733792+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:59.456699+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:15:20.924059+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:15:43.094773+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/2260035254":"2023-12-02T01:10:24.191835+0000","172.21.15.77:6800/1456621747":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/219838828":"2023-12-02T01:10:24.191835+0000","172.21.15.77:6801/1456621747":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/2937149266":"2023-12-02T01:10:02.232948+0000","172.21.15.77:0/807520018":"2023-12-02T01:10:02.232948+0000","172.21.15.77:6801/1661788026":"2023-12-02T01:10:02.232948+0000","172.21.15.77:6800/1661788026":"2023-12-02T01:10:02.232948+0000","172.21.15.77:0/3551282473":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/1202222558":"2023-12-02T01:10:02.232948+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:16:25.737 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.736+0000 7f2432f0b700 1 -- 172.21.15.77:0/2426224369 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f241c0a0570 msgr2=0x7f241c0a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:25.737 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.736+0000 7f2432f0b700 1 --2- 172.21.15.77:0/2426224369 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f241c0a0570 0x7f241c0a2a30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f241800b2e0 tx=0x7f2418005fb0 comp rx=0 tx=0).stop 2023-12-01T01:16:25.737 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.736+0000 7f2432f0b700 1 -- 172.21.15.77:0/2426224369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f242c073770 msgr2=0x7f242c198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:25.737 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.736+0000 7f2432f0b700 1 --2- 172.21.15.77:0/2426224369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f242c073770 0x7f242c198bf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f242000d950 tx=0x7f242000dc60 comp rx=0 tx=0).stop 2023-12-01T01:16:25.738 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.736+0000 7f2432f0b700 1 -- 172.21.15.77:0/2426224369 shutdown_connections 2023-12-01T01:16:25.738 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.736+0000 7f2432f0b700 1 --2- 172.21.15.77:0/2426224369 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f241c0a0570 0x7f241c0a2a30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:25.738 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.736+0000 7f2432f0b700 1 --2- 172.21.15.77:0/2426224369 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f242c108cc0 0x7f242c1a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:25.738 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.736+0000 7f2432f0b700 1 --2- 172.21.15.77:0/2426224369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f242c073770 0x7f242c198bf0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:25.738 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.736+0000 7f2432f0b700 1 --2- 172.21.15.77:0/2426224369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f242c074d50 0x7f242c199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:25.738 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.736+0000 7f2432f0b700 1 -- 172.21.15.77:0/2426224369 >> 172.21.15.77:0/2426224369 conn(0x7f242c0fc340 msgr2=0x7f242c104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:25.738 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.736+0000 7f2432f0b700 1 -- 172.21.15.77:0/2426224369 shutdown_connections 2023-12-01T01:16:25.738 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.736+0000 7f2432f0b700 1 -- 172.21.15.77:0/2426224369 wait complete. 2023-12-01T01:16:25.749 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:16:25.874 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.872+0000 7faa78f16700 1 -- 172.21.15.77:0/2802344028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa74100e60 msgr2=0x7faa741012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:25.874 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.872+0000 7faa78f16700 1 --2- 172.21.15.77:0/2802344028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa74100e60 0x7faa741012c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7faa640099c0 tx=0x7faa64009cd0 comp rx=0 tx=0).stop 2023-12-01T01:16:25.874 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.872+0000 7faa78f16700 1 -- 172.21.15.77:0/2802344028 shutdown_connections 2023-12-01T01:16:25.874 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.872+0000 7faa78f16700 1 --2- 172.21.15.77:0/2802344028 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa74101800 0x7faa741080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:25.874 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.872+0000 7faa78f16700 1 --2- 172.21.15.77:0/2802344028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa740ffc70 0x7faa74100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:25.874 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.872+0000 7faa78f16700 1 --2- 172.21.15.77:0/2802344028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa74100e60 0x7faa741012c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:25.874 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.872+0000 7faa78f16700 1 -- 172.21.15.77:0/2802344028 >> 172.21.15.77:0/2802344028 conn(0x7faa740fb1c0 msgr2=0x7faa740fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:25.874 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.872+0000 7faa78f16700 1 -- 172.21.15.77:0/2802344028 shutdown_connections 2023-12-01T01:16:25.875 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.872+0000 7faa78f16700 1 -- 172.21.15.77:0/2802344028 wait complete. 2023-12-01T01:16:25.876 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.873+0000 7faa78f16700 1 Processor -- start 2023-12-01T01:16:25.876 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.875+0000 7faa78f16700 1 -- start start 2023-12-01T01:16:25.877 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.875+0000 7faa78f16700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa740ffc70 0x7faa741958a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:25.877 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.875+0000 7faa78f16700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa74100e60 0x7faa74195de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:25.877 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.875+0000 7faa78f16700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa74101800 0x7faa7419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:25.878 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.875+0000 7faa78f16700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7419d420 con 0x7faa74101800 2023-12-01T01:16:25.878 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.875+0000 7faa78f16700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7419d560 con 0x7faa74100e60 2023-12-01T01:16:25.879 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.875+0000 7faa78f16700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7419d6f0 con 0x7faa740ffc70 2023-12-01T01:16:25.880 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.875+0000 7faa72d9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa74101800 0x7faa7419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:25.880 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.875+0000 7faa7259c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa740ffc70 0x7faa741958a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:25.880 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.875+0000 7faa71d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa74100e60 0x7faa74195de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:25.880 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.875+0000 7faa72d9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa74101800 0x7faa7419ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57474/0 (socket says 172.21.15.77:57474) 2023-12-01T01:16:25.880 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.875+0000 7faa7259c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa740ffc70 0x7faa741958a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:43658/0 (socket says 172.21.15.77:43658) 2023-12-01T01:16:25.880 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.875+0000 7faa71d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa74100e60 0x7faa74195de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:36392/0 (socket says 172.21.15.77:36392) 2023-12-01T01:16:25.880 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.875+0000 7faa72d9d700 1 -- 172.21.15.77:0/1812878601 learned_addr learned my addr 172.21.15.77:0/1812878601 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:25.880 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.876+0000 7faa72d9d700 1 -- 172.21.15.77:0/1812878601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa740ffc70 msgr2=0x7faa741958a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:25.880 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.876+0000 7faa72d9d700 1 --2- 172.21.15.77:0/1812878601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa740ffc70 0x7faa741958a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:25.880 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.876+0000 7faa72d9d700 1 -- 172.21.15.77:0/1812878601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa74100e60 msgr2=0x7faa74195de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:25.880 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.876+0000 7faa72d9d700 1 --2- 172.21.15.77:0/1812878601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa74100e60 0x7faa74195de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:25.880 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.876+0000 7faa72d9d700 1 -- 172.21.15.77:0/1812878601 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa640095f0 con 0x7faa74101800 2023-12-01T01:16:25.880 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.876+0000 7faa72d9d700 1 --2- 172.21.15.77:0/1812878601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa74101800 0x7faa7419ce50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7faa6c007f10 tx=0x7faa6c00f5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:25.881 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.876+0000 7faa637fe700 1 -- 172.21.15.77:0/1812878601 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6c00fd30 con 0x7faa74101800 2023-12-01T01:16:25.881 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.876+0000 7faa637fe700 1 -- 172.21.15.77:0/1812878601 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa6c00fe90 con 0x7faa74101800 2023-12-01T01:16:25.881 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.876+0000 7faa78f16700 1 -- 172.21.15.77:0/1812878601 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa74067f50 con 0x7faa74101800 2023-12-01T01:16:25.881 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.876+0000 7faa637fe700 1 -- 172.21.15.77:0/1812878601 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6c005020 con 0x7faa74101800 2023-12-01T01:16:25.881 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.876+0000 7faa78f16700 1 -- 172.21.15.77:0/1812878601 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa74068410 con 0x7faa74101800 2023-12-01T01:16:25.882 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.877+0000 7faa78f16700 1 -- 172.21.15.77:0/1812878601 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa74067b80 con 0x7faa74101800 2023-12-01T01:16:25.882 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.878+0000 7faa637fe700 1 -- 172.21.15.77:0/1812878601 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7faa6c012460 con 0x7faa74101800 2023-12-01T01:16:25.882 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.879+0000 7faa637fe700 1 --2- 172.21.15.77:0/1812878601 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7faa5c0a0570 0x7faa5c0a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:25.882 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.879+0000 7faa7259c700 1 --2- 172.21.15.77:0/1812878601 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7faa5c0a0570 0x7faa5c0a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:25.882 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.879+0000 7faa637fe700 1 -- 172.21.15.77:0/1812878601 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7faa6c0cde90 con 0x7faa74101800 2023-12-01T01:16:25.882 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.879+0000 7faa7259c700 1 --2- 172.21.15.77:0/1812878601 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7faa5c0a0570 0x7faa5c0a2a30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7faa74100cc0 tx=0x7faa6800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:25.883 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:25.882+0000 7faa637fe700 1 -- 172.21.15.77:0/1812878601 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7faa6c097b80 con 0x7faa74101800 2023-12-01T01:16:26.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:25 smithi203 ceph-mon[103581]: pgmap v249: 33 pgs: 1 creating+peering, 1 active+clean, 31 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:16:26.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:25 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/4247075960' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-12-01T01:16:26.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:25 smithi203 ceph-mon[103581]: osdmap e80: 12 total, 12 up, 12 in 2023-12-01T01:16:26.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:25 smithi203 ceph-mon[103581]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-12-01T01:16:26.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:25 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/2212310524' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-12-01T01:16:26.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:25 smithi203 ceph-mon[103581]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-12-01T01:16:26.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:25 smithi203 ceph-mon[103581]: osdmap e81: 12 total, 12 up, 12 in 2023-12-01T01:16:26.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:25 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/2212310524' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-12-01T01:16:26.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:25 smithi203 ceph-mon[103581]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-12-01T01:16:26.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:25 smithi203 ceph-mon[103581]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-12-01T01:16:26.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:25 smithi137 ceph-mon[103312]: pgmap v249: 33 pgs: 1 creating+peering, 1 active+clean, 31 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:16:26.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:25 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/4247075960' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-12-01T01:16:26.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:25 smithi137 ceph-mon[103312]: osdmap e80: 12 total, 12 up, 12 in 2023-12-01T01:16:26.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:25 smithi137 ceph-mon[103312]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-12-01T01:16:26.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:25 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/2212310524' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-12-01T01:16:26.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:25 smithi137 ceph-mon[103312]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-12-01T01:16:26.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:25 smithi137 ceph-mon[103312]: osdmap e81: 12 total, 12 up, 12 in 2023-12-01T01:16:26.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:25 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/2212310524' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-12-01T01:16:26.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:25 smithi137 ceph-mon[103312]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-12-01T01:16:26.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:25 smithi137 ceph-mon[103312]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-12-01T01:16:26.082 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.081+0000 7faa78f16700 1 -- 172.21.15.77:0/1812878601 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa74106e30 con 0x7faa74101800 2023-12-01T01:16:26.083 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.082+0000 7faa637fe700 1 -- 172.21.15.77:0/1812878601 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+22144 (secure 0 0 0) 0x7faa6c020090 con 0x7faa74101800 2023-12-01T01:16:26.084 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:16:26.084 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":81,"fsid":"2a06e29a-8fe6-11ee-95a2-87774f69a715","created":"2023-12-01T01:09:30.210398+0000","modified":"2023-12-01T01:16:25.344878+0000","last_up_change":"2023-12-01T01:15:44.586092+0000","last_in_change":"2023-12-01T01:15:26.305439+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:12:30.514229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:16:23.568860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-01T01:16:24.531137+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fc651e62-a103-4ab8-a76d-d3708ea6aeec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6803","nonce":2244799094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6805","nonce":2244799094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6809","nonce":2244799094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6807","nonce":2244799094}]},"public_addr":"172.21.15.77:6803/2244799094","cluster_addr":"172.21.15.77:6805/2244799094","heartbeat_back_addr":"172.21.15.77:6809/2244799094","heartbeat_front_addr":"172.21.15.77:6807/2244799094","state":["exists","up"]},{"osd":1,"uuid":"0adf8c54-49ec-49a0-8bf1-af1f20e60729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6801","nonce":224194890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6803","nonce":224194890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6807","nonce":224194890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6805","nonce":224194890}]},"public_addr":"172.21.15.137:6801/224194890","cluster_addr":"172.21.15.137:6803/224194890","heartbeat_back_addr":"172.21.15.137:6807/224194890","heartbeat_front_addr":"172.21.15.137:6805/224194890","state":["exists","up"]},{"osd":2,"uuid":"5a69802c-dede-4544-9a12-436f5c1c0038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6801","nonce":4078459110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6803","nonce":4078459110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6807","nonce":4078459110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6805","nonce":4078459110}]},"public_addr":"172.21.15.203:6801/4078459110","cluster_addr":"172.21.15.203:6803/4078459110","heartbeat_back_addr":"172.21.15.203:6807/4078459110","heartbeat_front_addr":"172.21.15.203:6805/4078459110","state":["exists","up"]},{"osd":3,"uuid":"ad325a97-580b-4c99-bf12-cbb0d92a6d0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6811","nonce":1454440477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6813","nonce":1454440477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6817","nonce":1454440477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6815","nonce":1454440477}]},"public_addr":"172.21.15.77:6811/1454440477","cluster_addr":"172.21.15.77:6813/1454440477","heartbeat_back_addr":"172.21.15.77:6817/1454440477","heartbeat_front_addr":"172.21.15.77:6815/1454440477","state":["exists","up"]},{"osd":4,"uuid":"1e3ac1ba-98bb-4b8b-8e26-50064c66bf2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6809","nonce":1103241451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6811","nonce":1103241451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6815","nonce":1103241451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6813","nonce":1103241451}]},"public_addr":"172.21.15.137:6809/1103241451","cluster_addr":"172.21.15.137:6811/1103241451","heartbeat_back_addr":"172.21.15.137:6815/1103241451","heartbeat_front_addr":"172.21.15.137:6813/1103241451","state":["exists","up"]},{"osd":5,"uuid":"eb32c016-abeb-44ab-abf0-19dd09235736","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6809","nonce":2648569482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6811","nonce":2648569482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6815","nonce":2648569482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6813","nonce":2648569482}]},"public_addr":"172.21.15.203:6809/2648569482","cluster_addr":"172.21.15.203:6811/2648569482","heartbeat_back_addr":"172.21.15.203:6815/2648569482","heartbeat_front_addr":"172.21.15.203:6813/2648569482","state":["exists","up"]},{"osd":6,"uuid":"0e3fa97e-2ca6-418c-9e8d-a6fe3c56adb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6819","nonce":2463414651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6821","nonce":2463414651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6825","nonce":2463414651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6823","nonce":2463414651}]},"public_addr":"172.21.15.77:6819/2463414651","cluster_addr":"172.21.15.77:6821/2463414651","heartbeat_back_addr":"172.21.15.77:6825/2463414651","heartbeat_front_addr":"172.21.15.77:6823/2463414651","state":["exists","up"]},{"osd":7,"uuid":"d7398c01-0f98-4138-8a49-c3ff0aae7e88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6817","nonce":3108330766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6819","nonce":3108330766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6823","nonce":3108330766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6821","nonce":3108330766}]},"public_addr":"172.21.15.137:6817/3108330766","cluster_addr":"172.21.15.137:6819/3108330766","heartbeat_back_addr":"172.21.15.137:6823/3108330766","heartbeat_front_addr":"172.21.15.137:6821/3108330766","state":["exists","up"]},{"osd":8,"uuid":"0c09050d-3e43-410c-8cd7-bcb891af56b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6817","nonce":1161158776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6819","nonce":1161158776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6823","nonce":1161158776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6821","nonce":1161158776}]},"public_addr":"172.21.15.203:6817/1161158776","cluster_addr":"172.21.15.203:6819/1161158776","heartbeat_back_addr":"172.21.15.203:6823/1161158776","heartbeat_front_addr":"172.21.15.203:6821/1161158776","state":["exists","up"]},{"osd":9,"uuid":"527ecb49-95ec-48bd-9872-3b07fae5c5ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6827","nonce":2409880685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6829","nonce":2409880685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6833","nonce":2409880685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6831","nonce":2409880685}]},"public_addr":"172.21.15.77:6827/2409880685","cluster_addr":"172.21.15.77:6829/2409880685","heartbeat_back_addr":"172.21.15.77:6833/2409880685","heartbeat_front_addr":"172.21.15.77:6831/2409880685","state":["exists","up"]},{"osd":10,"uuid":"1680bb3e-a3c8-427b-af68-e2214f32bed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6825","nonce":374920146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6827","nonce":374920146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6831","nonce":374920146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6829","nonce":374920146}]},"public_addr":"172.21.15.137:6825/374920146","cluster_addr":"172.21.15.137:6827/374920146","heartbeat_back_addr":"172.21.15.137:6831/374920146","heartbeat_front_addr":"172.21.15.137:6829/374920146","state":["exists","up"]},{"osd":11,"uuid":"8a533466-023d-4803-ae36-9a9f1715e49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6825","nonce":2161588962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6827","nonce":2161588962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6831","nonce":2161588962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6829","nonce":2161588962}]},"public_addr":"172.21.15.203:6825/2161588962","cluster_addr":"172.21.15.203:6827/2161588962","heartbeat_back_addr":"172.21.15.203:6831/2161588962","heartbeat_front_addr":"172.21.15.203:6829/2161588962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:11:45.420509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:06.210115+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:28.191307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:49.594116+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:11.261689+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:33.376541+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:54.968336+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:16.775310+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:38.733792+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:59.456699+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:15:20.924059+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:15:43.094773+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/2260035254":"2023-12-02T01:10:24.191835+0000","172.21.15.77:6800/1456621747":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/219838828":"2023-12-02T01:10:24.191835+0000","172.21.15.77:6801/1456621747":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/2937149266":"2023-12-02T01:10:02.232948+0000","172.21.15.77:0/807520018":"2023-12-02T01:10:02.232948+0000","172.21.15.77:6801/1661788026":"2023-12-02T01:10:02.232948+0000","172.21.15.77:6800/1661788026":"2023-12-02T01:10:02.232948+0000","172.21.15.77:0/3551282473":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/1202222558":"2023-12-02T01:10:02.232948+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:16:26.085 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.084+0000 7faa78f16700 1 -- 172.21.15.77:0/1812878601 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7faa5c0a0570 msgr2=0x7faa5c0a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:26.086 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.084+0000 7faa78f16700 1 --2- 172.21.15.77:0/1812878601 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7faa5c0a0570 0x7faa5c0a2a30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7faa74100cc0 tx=0x7faa6800a380 comp rx=0 tx=0).stop 2023-12-01T01:16:26.086 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.084+0000 7faa78f16700 1 -- 172.21.15.77:0/1812878601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa74101800 msgr2=0x7faa7419ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:26.086 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.084+0000 7faa78f16700 1 --2- 172.21.15.77:0/1812878601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa74101800 0x7faa7419ce50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7faa6c007f10 tx=0x7faa6c00f5c0 comp rx=0 tx=0).stop 2023-12-01T01:16:26.086 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.084+0000 7faa78f16700 1 -- 172.21.15.77:0/1812878601 shutdown_connections 2023-12-01T01:16:26.086 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.084+0000 7faa78f16700 1 --2- 172.21.15.77:0/1812878601 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7faa5c0a0570 0x7faa5c0a2a30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:26.086 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.084+0000 7faa78f16700 1 --2- 172.21.15.77:0/1812878601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa74101800 0x7faa7419ce50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:26.086 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.084+0000 7faa78f16700 1 --2- 172.21.15.77:0/1812878601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa740ffc70 0x7faa741958a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:26.086 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.084+0000 7faa78f16700 1 --2- 172.21.15.77:0/1812878601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa74100e60 0x7faa74195de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:26.086 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.084+0000 7faa78f16700 1 -- 172.21.15.77:0/1812878601 >> 172.21.15.77:0/1812878601 conn(0x7faa740fb1c0 msgr2=0x7faa74104390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:26.086 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.084+0000 7faa78f16700 1 -- 172.21.15.77:0/1812878601 shutdown_connections 2023-12-01T01:16:26.086 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.084+0000 7faa78f16700 1 -- 172.21.15.77:0/1812878601 wait complete. 2023-12-01T01:16:26.098 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-01T01:16:26.236 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.233+0000 7f0641ad4700 1 -- 172.21.15.77:0/4065370227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f063c100e70 msgr2=0x7f063c1012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:26.237 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.233+0000 7f0641ad4700 1 --2- 172.21.15.77:0/4065370227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f063c100e70 0x7f063c1012d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f06300099c0 tx=0x7f0630009cd0 comp rx=0 tx=0).stop 2023-12-01T01:16:26.237 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.233+0000 7f0641ad4700 1 -- 172.21.15.77:0/4065370227 shutdown_connections 2023-12-01T01:16:26.237 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.233+0000 7f0641ad4700 1 --2- 172.21.15.77:0/4065370227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f063c101810 0x7f063c1080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:26.237 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.233+0000 7f0641ad4700 1 --2- 172.21.15.77:0/4065370227 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f063c0ffc80 0x7f063c1000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:26.237 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.233+0000 7f0641ad4700 1 --2- 172.21.15.77:0/4065370227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f063c100e70 0x7f063c1012d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:26.237 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.233+0000 7f0641ad4700 1 -- 172.21.15.77:0/4065370227 >> 172.21.15.77:0/4065370227 conn(0x7f063c0fb1f0 msgr2=0x7f063c0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:26.237 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.233+0000 7f0641ad4700 1 -- 172.21.15.77:0/4065370227 shutdown_connections 2023-12-01T01:16:26.237 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.233+0000 7f0641ad4700 1 -- 172.21.15.77:0/4065370227 wait complete. 2023-12-01T01:16:26.237 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.234+0000 7f0641ad4700 1 Processor -- start 2023-12-01T01:16:26.237 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.234+0000 7f0641ad4700 1 -- start start 2023-12-01T01:16:26.237 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.234+0000 7f0641ad4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f063c0ffc80 0x7f063c103c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:26.237 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.234+0000 7f0641ad4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f063c100e70 0x7f063c102250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:26.238 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.234+0000 7f0641ad4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f063c101810 0x7f063c102790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:26.238 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.234+0000 7f0641ad4700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063c102f30 con 0x7f063c0ffc80 2023-12-01T01:16:26.238 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.234+0000 7f0641ad4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063c1030a0 con 0x7f063c101810 2023-12-01T01:16:26.238 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.234+0000 7f0641ad4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063c1a0ac0 con 0x7f063c100e70 2023-12-01T01:16:26.238 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.234+0000 7f063b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f063c0ffc80 0x7f063c103c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:26.238 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.234+0000 7f063b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f063c0ffc80 0x7f063c103c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57484/0 (socket says 172.21.15.77:57484) 2023-12-01T01:16:26.238 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.234+0000 7f063b7fe700 1 -- 172.21.15.77:0/412336373 learned_addr learned my addr 172.21.15.77:0/412336373 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:26.238 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.234+0000 7f063affd700 1 --2- 172.21.15.77:0/412336373 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f063c100e70 0x7f063c102250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:26.238 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.234+0000 7f063bfff700 1 --2- 172.21.15.77:0/412336373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f063c101810 0x7f063c102790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:26.238 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.235+0000 7f063b7fe700 1 -- 172.21.15.77:0/412336373 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f063c100e70 msgr2=0x7f063c102250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:26.238 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.235+0000 7f063b7fe700 1 --2- 172.21.15.77:0/412336373 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f063c100e70 0x7f063c102250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:26.238 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.235+0000 7f063b7fe700 1 -- 172.21.15.77:0/412336373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f063c101810 msgr2=0x7f063c102790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:26.238 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.235+0000 7f063b7fe700 1 --2- 172.21.15.77:0/412336373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f063c101810 0x7f063c102790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:26.238 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.235+0000 7f063b7fe700 1 -- 172.21.15.77:0/412336373 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06300095f0 con 0x7f063c0ffc80 2023-12-01T01:16:26.239 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.235+0000 7f063b7fe700 1 --2- 172.21.15.77:0/412336373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f063c0ffc80 0x7f063c103c10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f062c00e970 tx=0x7f062c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:26.239 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.235+0000 7f0638ff9700 1 -- 172.21.15.77:0/412336373 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c009790 con 0x7f063c0ffc80 2023-12-01T01:16:26.239 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.235+0000 7f0638ff9700 1 -- 172.21.15.77:0/412336373 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f062c00eec0 con 0x7f063c0ffc80 2023-12-01T01:16:26.239 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.235+0000 7f0641ad4700 1 -- 172.21.15.77:0/412336373 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f063c1a0da0 con 0x7f063c0ffc80 2023-12-01T01:16:26.239 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.235+0000 7f0638ff9700 1 -- 172.21.15.77:0/412336373 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c003cf0 con 0x7f063c0ffc80 2023-12-01T01:16:26.239 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.235+0000 7f0641ad4700 1 -- 172.21.15.77:0/412336373 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f063c1a11f0 con 0x7f063c0ffc80 2023-12-01T01:16:26.240 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.236+0000 7f0641ad4700 1 -- 172.21.15.77:0/412336373 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f063c10b490 con 0x7f063c0ffc80 2023-12-01T01:16:26.240 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.237+0000 7f0638ff9700 1 -- 172.21.15.77:0/412336373 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f062c00ccd0 con 0x7f063c0ffc80 2023-12-01T01:16:26.240 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.238+0000 7f0638ff9700 1 --2- 172.21.15.77:0/412336373 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f06240a0520 0x7f06240a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:26.240 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.238+0000 7f063affd700 1 --2- 172.21.15.77:0/412336373 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f06240a0520 0x7f06240a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:26.240 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.238+0000 7f0638ff9700 1 -- 172.21.15.77:0/412336373 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f062c014070 con 0x7f063c0ffc80 2023-12-01T01:16:26.240 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.238+0000 7f063affd700 1 --2- 172.21.15.77:0/412336373 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f06240a0520 0x7f06240a29e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f063000b580 tx=0x7f0630005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:26.242 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.242+0000 7f0638ff9700 1 -- 172.21.15.77:0/412336373 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f062c095930 con 0x7f063c0ffc80 2023-12-01T01:16:26.536 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:26.535+0000 7f0641ad4700 1 -- 172.21.15.77:0/412336373 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f063c1a1860 con 0x7f063c0ffc80 2023-12-01T01:16:26.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:26 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/2426224369' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:16:26.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:26 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/1812878601' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:16:26.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:26 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:26.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:26 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:26.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:26 smithi077 ceph-mon[98867]: osdmap e82: 12 total, 12 up, 12 in 2023-12-01T01:16:26.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:26 smithi077 ceph-mon[98867]: mds.? [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] up:active 2023-12-01T01:16:26.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:26 smithi077 ceph-mon[98867]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-01T01:16:26.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:26 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:26.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:26 smithi077 ceph-mon[98867]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-01T01:16:26.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:26 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:26.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:26 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:26.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:26 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:26.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:26 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:16:26.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:26 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:16:26.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:26 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:26.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:26 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/412336373' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-12-01T01:16:27.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:26 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/2426224369' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:16:27.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:26 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/1812878601' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:16:27.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:26 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:27.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:26 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:27.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:26 smithi203 ceph-mon[103581]: osdmap e82: 12 total, 12 up, 12 in 2023-12-01T01:16:27.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:26 smithi203 ceph-mon[103581]: mds.? [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] up:active 2023-12-01T01:16:27.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:26 smithi203 ceph-mon[103581]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-01T01:16:27.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:26 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:27.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:26 smithi203 ceph-mon[103581]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-01T01:16:27.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:26 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:27.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:26 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:27.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:26 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:27.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:26 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:16:27.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:26 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:16:27.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:26 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:27.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:26 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/412336373' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-12-01T01:16:27.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:26 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/2426224369' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:16:27.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:26 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/1812878601' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:16:27.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:26 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:27.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:26 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:27.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:26 smithi137 ceph-mon[103312]: osdmap e82: 12 total, 12 up, 12 in 2023-12-01T01:16:27.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:26 smithi137 ceph-mon[103312]: mds.? [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] up:active 2023-12-01T01:16:27.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:26 smithi137 ceph-mon[103312]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-01T01:16:27.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:26 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:27.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:26 smithi137 ceph-mon[103312]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-01T01:16:27.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:26 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:27.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:26 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:27.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:26 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:27.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:26 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:16:27.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:26 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:16:27.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:26 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:16:27.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:26 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/412336373' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-12-01T01:16:27.389 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.388+0000 7f0638ff9700 1 -- 172.21.15.77:0/412336373 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f062c098f50 con 0x7f063c0ffc80 2023-12-01T01:16:27.392 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.391+0000 7f0641ad4700 1 -- 172.21.15.77:0/412336373 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f06240a0520 msgr2=0x7f06240a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:27.392 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.391+0000 7f0641ad4700 1 --2- 172.21.15.77:0/412336373 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f06240a0520 0x7f06240a29e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f063000b580 tx=0x7f0630005fb0 comp rx=0 tx=0).stop 2023-12-01T01:16:27.392 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.391+0000 7f0641ad4700 1 -- 172.21.15.77:0/412336373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f063c0ffc80 msgr2=0x7f063c103c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:27.392 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.391+0000 7f0641ad4700 1 --2- 172.21.15.77:0/412336373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f063c0ffc80 0x7f063c103c10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f062c00e970 tx=0x7f062c00ec80 comp rx=0 tx=0).stop 2023-12-01T01:16:27.392 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.391+0000 7f0641ad4700 1 -- 172.21.15.77:0/412336373 shutdown_connections 2023-12-01T01:16:27.392 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.391+0000 7f0641ad4700 1 --2- 172.21.15.77:0/412336373 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f06240a0520 0x7f06240a29e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:27.393 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.391+0000 7f0641ad4700 1 --2- 172.21.15.77:0/412336373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f063c101810 0x7f063c102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:27.393 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.391+0000 7f0641ad4700 1 --2- 172.21.15.77:0/412336373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f063c0ffc80 0x7f063c103c10 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:27.393 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.391+0000 7f0641ad4700 1 --2- 172.21.15.77:0/412336373 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f063c100e70 0x7f063c102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:27.393 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.391+0000 7f0641ad4700 1 -- 172.21.15.77:0/412336373 >> 172.21.15.77:0/412336373 conn(0x7f063c0fb1f0 msgr2=0x7f063c068fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:27.393 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.391+0000 7f0641ad4700 1 -- 172.21.15.77:0/412336373 shutdown_connections 2023-12-01T01:16:27.393 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.391+0000 7f0641ad4700 1 -- 172.21.15.77:0/412336373 wait complete. 2023-12-01T01:16:27.407 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2023-12-01T01:16:27.407 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2023-12-01T01:16:27.540 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.536+0000 7f07eeb0d700 1 -- 172.21.15.77:0/2570317894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07e8100e70 msgr2=0x7f07e81012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:27.540 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.536+0000 7f07eeb0d700 1 --2- 172.21.15.77:0/2570317894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07e8100e70 0x7f07e81012d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f07d40099c0 tx=0x7f07d4009cd0 comp rx=0 tx=0).stop 2023-12-01T01:16:27.540 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.536+0000 7f07eeb0d700 1 -- 172.21.15.77:0/2570317894 shutdown_connections 2023-12-01T01:16:27.540 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.536+0000 7f07eeb0d700 1 --2- 172.21.15.77:0/2570317894 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07e8101810 0x7f07e81080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:27.540 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.536+0000 7f07eeb0d700 1 --2- 172.21.15.77:0/2570317894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07e80ffc80 0x7f07e81000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:27.541 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.536+0000 7f07eeb0d700 1 --2- 172.21.15.77:0/2570317894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07e8100e70 0x7f07e81012d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:27.541 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.536+0000 7f07eeb0d700 1 -- 172.21.15.77:0/2570317894 >> 172.21.15.77:0/2570317894 conn(0x7f07e80fb1f0 msgr2=0x7f07e80fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:27.541 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.536+0000 7f07eeb0d700 1 -- 172.21.15.77:0/2570317894 shutdown_connections 2023-12-01T01:16:27.541 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.536+0000 7f07eeb0d700 1 -- 172.21.15.77:0/2570317894 wait complete. 2023-12-01T01:16:27.541 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.536+0000 7f07eeb0d700 1 Processor -- start 2023-12-01T01:16:27.541 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.536+0000 7f07eeb0d700 1 -- start start 2023-12-01T01:16:27.541 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.537+0000 7f07eeb0d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07e80ffc80 0x7f07e8068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:27.541 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.537+0000 7f07eeb0d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07e8100e70 0x7f07e819b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:27.541 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.537+0000 7f07eeb0d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07e8101810 0x7f07e819de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:27.541 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.537+0000 7f07eeb0d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e819e610 con 0x7f07e80ffc80 2023-12-01T01:16:27.541 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.537+0000 7f07eeb0d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e819e780 con 0x7f07e8101810 2023-12-01T01:16:27.541 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.537+0000 7f07eeb0d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e819e940 con 0x7f07e8100e70 2023-12-01T01:16:27.541 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.537+0000 7f07ec8a9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07e80ffc80 0x7f07e8068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:27.541 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.537+0000 7f07ec8a9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07e80ffc80 0x7f07e8068a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57498/0 (socket says 172.21.15.77:57498) 2023-12-01T01:16:27.542 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.537+0000 7f07ec8a9700 1 -- 172.21.15.77:0/2422083924 learned_addr learned my addr 172.21.15.77:0/2422083924 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:27.542 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.537+0000 7f07ed0aa700 1 --2- 172.21.15.77:0/2422083924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07e8101810 0x7f07e819de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:27.542 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.537+0000 7f07e7fff700 1 --2- 172.21.15.77:0/2422083924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07e8100e70 0x7f07e819b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:27.542 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.537+0000 7f07ec8a9700 1 -- 172.21.15.77:0/2422083924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07e8100e70 msgr2=0x7f07e819b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:27.542 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.537+0000 7f07ec8a9700 1 --2- 172.21.15.77:0/2422083924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07e8100e70 0x7f07e819b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:27.542 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.537+0000 7f07ec8a9700 1 -- 172.21.15.77:0/2422083924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07e8101810 msgr2=0x7f07e819de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:27.542 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.537+0000 7f07ec8a9700 1 --2- 172.21.15.77:0/2422083924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07e8101810 0x7f07e819de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:27.542 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.537+0000 7f07ec8a9700 1 -- 172.21.15.77:0/2422083924 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d40095f0 con 0x7f07e80ffc80 2023-12-01T01:16:27.542 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.537+0000 7f07ec8a9700 1 --2- 172.21.15.77:0/2422083924 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07e80ffc80 0x7f07e8068a80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f07dc00d980 tx=0x7f07dc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:27.542 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.537+0000 7f07e5ffb700 1 -- 172.21.15.77:0/2422083924 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07dc004020 con 0x7f07e80ffc80 2023-12-01T01:16:27.542 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.537+0000 7f07eeb0d700 1 -- 172.21.15.77:0/2422083924 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07e819ec20 con 0x7f07e80ffc80 2023-12-01T01:16:27.542 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.537+0000 7f07e5ffb700 1 -- 172.21.15.77:0/2422083924 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07dc009730 con 0x7f07e80ffc80 2023-12-01T01:16:27.542 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.537+0000 7f07e5ffb700 1 -- 172.21.15.77:0/2422083924 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07dc003a30 con 0x7f07e80ffc80 2023-12-01T01:16:27.542 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.537+0000 7f07eeb0d700 1 -- 172.21.15.77:0/2422083924 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e819f150 con 0x7f07e80ffc80 2023-12-01T01:16:27.542 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.538+0000 7f07eeb0d700 1 -- 172.21.15.77:0/2422083924 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07e8106c00 con 0x7f07e80ffc80 2023-12-01T01:16:27.543 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.539+0000 7f07e5ffb700 1 -- 172.21.15.77:0/2422083924 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f07dc0098a0 con 0x7f07e80ffc80 2023-12-01T01:16:27.543 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.540+0000 7f07e5ffb700 1 --2- 172.21.15.77:0/2422083924 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f07d80a0520 0x7f07d80a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:27.543 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.540+0000 7f07e7fff700 1 --2- 172.21.15.77:0/2422083924 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f07d80a0520 0x7f07d80a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:27.543 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.540+0000 7f07e5ffb700 1 -- 172.21.15.77:0/2422083924 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f07dc0cb530 con 0x7f07e80ffc80 2023-12-01T01:16:27.544 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.540+0000 7f07e7fff700 1 --2- 172.21.15.77:0/2422083924 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f07d80a0520 0x7f07d80a29e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f07d4000c00 tx=0x7f07d4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:27.545 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.544+0000 7f07e5ffb700 1 -- 172.21.15.77:0/2422083924 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f07dc095220 con 0x7f07e80ffc80 2023-12-01T01:16:27.783 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:27.782+0000 7f07eeb0d700 1 -- 172.21.15.77:0/2422083924 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7f07e802ce40 con 0x7f07e80ffc80 2023-12-01T01:16:27.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:27 smithi077 ceph-mon[98867]: pgmap v252: 97 pgs: 10 creating+peering, 21 active+clean, 66 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 3 op/s 2023-12-01T01:16:27.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:27 smithi077 ceph-mon[98867]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-12-01T01:16:27.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:27 smithi077 ceph-mon[98867]: Cluster is now healthy 2023-12-01T01:16:27.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:27 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/412336373' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-12-01T01:16:27.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:27 smithi077 ceph-mon[98867]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-01T01:16:28.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:27 smithi203 ceph-mon[103581]: pgmap v252: 97 pgs: 10 creating+peering, 21 active+clean, 66 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 3 op/s 2023-12-01T01:16:28.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:27 smithi203 ceph-mon[103581]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-12-01T01:16:28.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:27 smithi203 ceph-mon[103581]: Cluster is now healthy 2023-12-01T01:16:28.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:27 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/412336373' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-12-01T01:16:28.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:27 smithi203 ceph-mon[103581]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-01T01:16:28.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:27 smithi137 ceph-mon[103312]: pgmap v252: 97 pgs: 10 creating+peering, 21 active+clean, 66 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 3 op/s 2023-12-01T01:16:28.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:27 smithi137 ceph-mon[103312]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-12-01T01:16:28.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:27 smithi137 ceph-mon[103312]: Cluster is now healthy 2023-12-01T01:16:28.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:27 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/412336373' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-12-01T01:16:28.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:27 smithi137 ceph-mon[103312]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-01T01:16:28.618 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.617+0000 7f07e5ffb700 1 -- 172.21.15.77:0/2422083924 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7f07dc098840 con 0x7f07e80ffc80 2023-12-01T01:16:28.621 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.620+0000 7f07eeb0d700 1 -- 172.21.15.77:0/2422083924 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f07d80a0520 msgr2=0x7f07d80a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:28.622 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.620+0000 7f07eeb0d700 1 --2- 172.21.15.77:0/2422083924 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f07d80a0520 0x7f07d80a29e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f07d4000c00 tx=0x7f07d4005fb0 comp rx=0 tx=0).stop 2023-12-01T01:16:28.622 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.620+0000 7f07eeb0d700 1 -- 172.21.15.77:0/2422083924 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07e80ffc80 msgr2=0x7f07e8068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:28.622 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.620+0000 7f07eeb0d700 1 --2- 172.21.15.77:0/2422083924 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07e80ffc80 0x7f07e8068a80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f07dc00d980 tx=0x7f07dc00dc90 comp rx=0 tx=0).stop 2023-12-01T01:16:28.622 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.620+0000 7f07eeb0d700 1 -- 172.21.15.77:0/2422083924 shutdown_connections 2023-12-01T01:16:28.622 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.620+0000 7f07eeb0d700 1 --2- 172.21.15.77:0/2422083924 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f07d80a0520 0x7f07d80a29e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:28.622 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.620+0000 7f07eeb0d700 1 --2- 172.21.15.77:0/2422083924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07e8101810 0x7f07e819de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:28.622 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.620+0000 7f07eeb0d700 1 --2- 172.21.15.77:0/2422083924 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07e80ffc80 0x7f07e8068a80 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:28.622 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.620+0000 7f07eeb0d700 1 --2- 172.21.15.77:0/2422083924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07e8100e70 0x7f07e819b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:28.622 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.620+0000 7f07eeb0d700 1 -- 172.21.15.77:0/2422083924 >> 172.21.15.77:0/2422083924 conn(0x7f07e80fb1f0 msgr2=0x7f07e80fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:28.622 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.620+0000 7f07eeb0d700 1 -- 172.21.15.77:0/2422083924 shutdown_connections 2023-12-01T01:16:28.622 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.620+0000 7f07eeb0d700 1 -- 172.21.15.77:0/2422083924 wait complete. 2023-12-01T01:16:28.638 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-12-01T01:16:28.766 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.765+0000 7f5efcb8b700 1 -- 172.21.15.77:0/2318298162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ef8100da0 msgr2=0x7f5ef81011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:28.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.765+0000 7f5efcb8b700 1 --2- 172.21.15.77:0/2318298162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ef8100da0 0x7f5ef81011c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5ee80099c0 tx=0x7f5ee8009cd0 comp rx=0 tx=0).stop 2023-12-01T01:16:28.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.765+0000 7f5efcb8b700 1 -- 172.21.15.77:0/2318298162 shutdown_connections 2023-12-01T01:16:28.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.765+0000 7f5efcb8b700 1 --2- 172.21.15.77:0/2318298162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ef8102930 0x7f5ef81091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:28.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.765+0000 7f5efcb8b700 1 --2- 172.21.15.77:0/2318298162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ef8100da0 0x7f5ef81011c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:28.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.765+0000 7f5efcb8b700 1 --2- 172.21.15.77:0/2318298162 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ef8101f90 0x7f5ef81023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:28.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.765+0000 7f5efcb8b700 1 -- 172.21.15.77:0/2318298162 >> 172.21.15.77:0/2318298162 conn(0x7f5ef80fc310 msgr2=0x7f5ef80fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:28.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.765+0000 7f5efcb8b700 1 -- 172.21.15.77:0/2318298162 shutdown_connections 2023-12-01T01:16:28.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.765+0000 7f5efcb8b700 1 -- 172.21.15.77:0/2318298162 wait complete. 2023-12-01T01:16:28.772 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.766+0000 7f5efcb8b700 1 Processor -- start 2023-12-01T01:16:28.772 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.766+0000 7f5efcb8b700 1 -- start start 2023-12-01T01:16:28.772 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.766+0000 7f5efcb8b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ef8100da0 0x7f5ef8074b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:28.772 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.766+0000 7f5efcb8b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ef8101f90 0x7f5ef8073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:28.772 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.766+0000 7f5efcb8b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ef8102930 0x7f5ef80736c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:28.772 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.766+0000 7f5efcb8b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef8073e60 con 0x7f5ef8102930 2023-12-01T01:16:28.772 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.766+0000 7f5efcb8b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef8073fd0 con 0x7f5ef8101f90 2023-12-01T01:16:28.772 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.766+0000 7f5efcb8b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef81a0b20 con 0x7f5ef8100da0 2023-12-01T01:16:28.772 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.766+0000 7f5ef6d9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ef8102930 0x7f5ef80736c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:28.772 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.766+0000 7f5ef6d9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ef8102930 0x7f5ef80736c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57516/0 (socket says 172.21.15.77:57516) 2023-12-01T01:16:28.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.766+0000 7f5ef6d9d700 1 -- 172.21.15.77:0/2380553063 learned_addr learned my addr 172.21.15.77:0/2380553063 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:28.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.766+0000 7f5ef5d9b700 1 --2- 172.21.15.77:0/2380553063 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ef8101f90 0x7f5ef8073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:28.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.766+0000 7f5ef659c700 1 --2- 172.21.15.77:0/2380553063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ef8100da0 0x7f5ef8074b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:28.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.766+0000 7f5ef6d9d700 1 -- 172.21.15.77:0/2380553063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ef8100da0 msgr2=0x7f5ef8074b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:28.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.766+0000 7f5ef6d9d700 1 --2- 172.21.15.77:0/2380553063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ef8100da0 0x7f5ef8074b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:28.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.766+0000 7f5ef6d9d700 1 -- 172.21.15.77:0/2380553063 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ef8101f90 msgr2=0x7f5ef8073180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:28.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.766+0000 7f5ef6d9d700 1 --2- 172.21.15.77:0/2380553063 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ef8101f90 0x7f5ef8073180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:28.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.766+0000 7f5ef6d9d700 1 -- 172.21.15.77:0/2380553063 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee80095f0 con 0x7f5ef8102930 2023-12-01T01:16:28.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.766+0000 7f5ef659c700 1 --2- 172.21.15.77:0/2380553063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ef8100da0 0x7f5ef8074b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:16:28.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.766+0000 7f5ef6d9d700 1 --2- 172.21.15.77:0/2380553063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ef8102930 0x7f5ef80736c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0007ee0 tx=0x7f5ef000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:28.774 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.766+0000 7f5ee77fe700 1 -- 172.21.15.77:0/2380553063 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0017070 con 0x7f5ef8102930 2023-12-01T01:16:28.774 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.767+0000 7f5ee77fe700 1 -- 172.21.15.77:0/2380553063 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ef000f040 con 0x7f5ef8102930 2023-12-01T01:16:28.774 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.767+0000 7f5efcb8b700 1 -- 172.21.15.77:0/2380553063 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ef81a0e00 con 0x7f5ef8102930 2023-12-01T01:16:28.774 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.767+0000 7f5ee77fe700 1 -- 172.21.15.77:0/2380553063 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0005740 con 0x7f5ef8102930 2023-12-01T01:16:28.774 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.767+0000 7f5efcb8b700 1 -- 172.21.15.77:0/2380553063 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef81a1230 con 0x7f5ef8102930 2023-12-01T01:16:28.774 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.767+0000 7f5efcb8b700 1 -- 172.21.15.77:0/2380553063 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ef8107d50 con 0x7f5ef8102930 2023-12-01T01:16:28.774 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.768+0000 7f5ee77fe700 1 -- 172.21.15.77:0/2380553063 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f5ef000dc60 con 0x7f5ef8102930 2023-12-01T01:16:28.774 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.769+0000 7f5ee77fe700 1 --2- 172.21.15.77:0/2380553063 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f5ee00a05a0 0x7f5ee00a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:28.774 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.769+0000 7f5ef659c700 1 --2- 172.21.15.77:0/2380553063 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f5ee00a05a0 0x7f5ee00a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:28.774 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.769+0000 7f5ee77fe700 1 -- 172.21.15.77:0/2380553063 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5ef00cb6a0 con 0x7f5ef8102930 2023-12-01T01:16:28.774 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.769+0000 7f5ef659c700 1 --2- 172.21.15.77:0/2380553063 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f5ee00a05a0 0x7f5ee00a2a60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800b580 tx=0x7f5ee8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:28.775 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:28.772+0000 7f5ee77fe700 1 -- 172.21.15.77:0/2380553063 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5ef00952f0 con 0x7f5ef8102930 2023-12-01T01:16:28.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:28 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/2422083924' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-12-01T01:16:29.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:28 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/2422083924' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-12-01T01:16:29.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.011+0000 7f5efcb8b700 1 -- 172.21.15.77:0/2380553063 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f5ef802d090 con 0x7f5ef8102930 2023-12-01T01:16:29.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:28 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/2422083924' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-12-01T01:16:29.626 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.624+0000 7f5ee77fe700 1 -- 172.21.15.77:0/2380553063 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7f5ef0098910 con 0x7f5ef8102930 2023-12-01T01:16:29.630 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.627+0000 7f5efcb8b700 1 -- 172.21.15.77:0/2380553063 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f5ee00a05a0 msgr2=0x7f5ee00a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:29.630 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.627+0000 7f5efcb8b700 1 --2- 172.21.15.77:0/2380553063 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f5ee00a05a0 0x7f5ee00a2a60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800b580 tx=0x7f5ee8005fb0 comp rx=0 tx=0).stop 2023-12-01T01:16:29.630 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.627+0000 7f5efcb8b700 1 -- 172.21.15.77:0/2380553063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ef8102930 msgr2=0x7f5ef80736c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:29.630 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.627+0000 7f5efcb8b700 1 --2- 172.21.15.77:0/2380553063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ef8102930 0x7f5ef80736c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0007ee0 tx=0x7f5ef000d3b0 comp rx=0 tx=0).stop 2023-12-01T01:16:29.630 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.628+0000 7f5efcb8b700 1 -- 172.21.15.77:0/2380553063 shutdown_connections 2023-12-01T01:16:29.630 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.628+0000 7f5efcb8b700 1 --2- 172.21.15.77:0/2380553063 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f5ee00a05a0 0x7f5ee00a2a60 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:29.631 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.628+0000 7f5efcb8b700 1 --2- 172.21.15.77:0/2380553063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ef8102930 0x7f5ef80736c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:29.631 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.628+0000 7f5efcb8b700 1 --2- 172.21.15.77:0/2380553063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ef8100da0 0x7f5ef8074b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:29.631 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.628+0000 7f5efcb8b700 1 --2- 172.21.15.77:0/2380553063 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ef8101f90 0x7f5ef8073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:29.631 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.628+0000 7f5efcb8b700 1 -- 172.21.15.77:0/2380553063 >> 172.21.15.77:0/2380553063 conn(0x7f5ef80fc310 msgr2=0x7f5ef80fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:29.631 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.628+0000 7f5efcb8b700 1 -- 172.21.15.77:0/2380553063 shutdown_connections 2023-12-01T01:16:29.631 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.628+0000 7f5efcb8b700 1 -- 172.21.15.77:0/2380553063 wait complete. 2023-12-01T01:16:29.647 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-12-01T01:16:29.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.774+0000 7f0ea9b0a700 1 -- 172.21.15.77:0/3058501606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ea4106af0 msgr2=0x7f0ea4108f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:29.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.774+0000 7f0ea9b0a700 1 --2- 172.21.15.77:0/3058501606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ea4106af0 0x7f0ea4108f00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c00b0a0 tx=0x7f0e9c00b3b0 comp rx=0 tx=0).stop 2023-12-01T01:16:29.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.774+0000 7f0ea9b0a700 1 -- 172.21.15.77:0/3058501606 shutdown_connections 2023-12-01T01:16:29.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.774+0000 7f0ea9b0a700 1 --2- 172.21.15.77:0/3058501606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ea4106af0 0x7f0ea4108f00 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:29.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.774+0000 7f0ea9b0a700 1 --2- 172.21.15.77:0/3058501606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ea4101890 0x7f0ea4103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:29.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.774+0000 7f0ea9b0a700 1 --2- 172.21.15.77:0/3058501606 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ea41041c0 0x7f0ea41065b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:29.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.774+0000 7f0ea9b0a700 1 -- 172.21.15.77:0/3058501606 >> 172.21.15.77:0/3058501606 conn(0x7f0ea40fb220 msgr2=0x7f0ea40fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:29.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.775+0000 7f0ea9b0a700 1 -- 172.21.15.77:0/3058501606 shutdown_connections 2023-12-01T01:16:29.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.775+0000 7f0ea9b0a700 1 -- 172.21.15.77:0/3058501606 wait complete. 2023-12-01T01:16:29.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.775+0000 7f0ea9b0a700 1 Processor -- start 2023-12-01T01:16:29.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.775+0000 7f0ea9b0a700 1 -- start start 2023-12-01T01:16:29.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.775+0000 7f0ea9b0a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ea4101890 0x7f0ea4101200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:29.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.775+0000 7f0ea9b0a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ea41041c0 0x7f0ea40ff840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:29.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.775+0000 7f0ea9b0a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ea4106af0 0x7f0ea40ffd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:29.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.775+0000 7f0ea9b0a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea4100520 con 0x7f0ea41041c0 2023-12-01T01:16:29.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.775+0000 7f0ea9b0a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea4100690 con 0x7f0ea4106af0 2023-12-01T01:16:29.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.775+0000 7f0ea9b0a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea41a0a90 con 0x7f0ea4101890 2023-12-01T01:16:29.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.776+0000 7f0ea2ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ea41041c0 0x7f0ea40ff840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:29.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.776+0000 7f0ea2ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ea41041c0 0x7f0ea40ff840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57528/0 (socket says 172.21.15.77:57528) 2023-12-01T01:16:29.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.776+0000 7f0ea2ffd700 1 -- 172.21.15.77:0/314433087 learned_addr learned my addr 172.21.15.77:0/314433087 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:29.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.776+0000 7f0ea3fff700 1 --2- 172.21.15.77:0/314433087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ea4106af0 0x7f0ea40ffd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:29.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.776+0000 7f0ea37fe700 1 --2- 172.21.15.77:0/314433087 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ea4101890 0x7f0ea4101200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:29.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.776+0000 7f0ea2ffd700 1 -- 172.21.15.77:0/314433087 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ea4101890 msgr2=0x7f0ea4101200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:29.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.776+0000 7f0ea2ffd700 1 --2- 172.21.15.77:0/314433087 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ea4101890 0x7f0ea4101200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:29.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.776+0000 7f0ea2ffd700 1 -- 172.21.15.77:0/314433087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ea4106af0 msgr2=0x7f0ea40ffd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:29.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.776+0000 7f0ea2ffd700 1 --2- 172.21.15.77:0/314433087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ea4106af0 0x7f0ea40ffd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:29.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.776+0000 7f0ea2ffd700 1 -- 172.21.15.77:0/314433087 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e9c009d00 con 0x7f0ea41041c0 2023-12-01T01:16:29.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.776+0000 7f0ea2ffd700 1 --2- 172.21.15.77:0/314433087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ea41041c0 0x7f0ea40ff840 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0e9400c9b0 tx=0x7f0e9400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:29.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.776+0000 7f0ea0ff9700 1 -- 172.21.15.77:0/314433087 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e94007b00 con 0x7f0ea41041c0 2023-12-01T01:16:29.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.776+0000 7f0ea0ff9700 1 -- 172.21.15.77:0/314433087 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e94007c60 con 0x7f0ea41041c0 2023-12-01T01:16:29.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.776+0000 7f0ea9b0a700 1 -- 172.21.15.77:0/314433087 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ea41a0d70 con 0x7f0ea41041c0 2023-12-01T01:16:29.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.776+0000 7f0ea0ff9700 1 -- 172.21.15.77:0/314433087 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e94018660 con 0x7f0ea41041c0 2023-12-01T01:16:29.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.776+0000 7f0ea9b0a700 1 -- 172.21.15.77:0/314433087 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ea41a1220 con 0x7f0ea41041c0 2023-12-01T01:16:29.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.777+0000 7f0ea9b0a700 1 -- 172.21.15.77:0/314433087 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ea40fcda0 con 0x7f0ea41041c0 2023-12-01T01:16:29.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.778+0000 7f0ea0ff9700 1 -- 172.21.15.77:0/314433087 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f0e9400f450 con 0x7f0ea41041c0 2023-12-01T01:16:29.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.779+0000 7f0ea0ff9700 1 --2- 172.21.15.77:0/314433087 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f0e8c0a05a0 0x7f0e8c0a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:29.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.779+0000 7f0ea37fe700 1 --2- 172.21.15.77:0/314433087 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f0e8c0a05a0 0x7f0e8c0a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:29.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.779+0000 7f0ea0ff9700 1 -- 172.21.15.77:0/314433087 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0e940cbb00 con 0x7f0ea41041c0 2023-12-01T01:16:29.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.779+0000 7f0ea37fe700 1 --2- 172.21.15.77:0/314433087 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f0e8c0a05a0 0x7f0e8c0a2a60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0e98005ea0 tx=0x7f0e98005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:29.783 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:29.782+0000 7f0ea0ff9700 1 -- 172.21.15.77:0/314433087 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0e940957e0 con 0x7f0ea41041c0 2023-12-01T01:16:29.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:29 smithi077 ceph-mon[98867]: pgmap v254: 97 pgs: 10 creating+peering, 60 active+clean, 27 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 7 op/s 2023-12-01T01:16:29.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:29 smithi077 ceph-mon[98867]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-12-01T01:16:29.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:29 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/2422083924' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-12-01T01:16:29.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:29 smithi077 ceph-mon[98867]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-12-01T01:16:29.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:29 smithi077 ceph-mon[98867]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-01T01:16:29.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:29 smithi077 ceph-mon[98867]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-12-01T01:16:29.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:29 smithi077 ceph-mon[98867]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-12-01T01:16:29.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:29 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/2380553063' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-12-01T01:16:30.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:29 smithi203 ceph-mon[103581]: pgmap v254: 97 pgs: 10 creating+peering, 60 active+clean, 27 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 7 op/s 2023-12-01T01:16:30.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:29 smithi203 ceph-mon[103581]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-12-01T01:16:30.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:29 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/2422083924' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-12-01T01:16:30.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:29 smithi203 ceph-mon[103581]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-12-01T01:16:30.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:29 smithi203 ceph-mon[103581]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-01T01:16:30.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:29 smithi203 ceph-mon[103581]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-12-01T01:16:30.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:29 smithi203 ceph-mon[103581]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-12-01T01:16:30.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:29 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/2380553063' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-12-01T01:16:30.021 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.020+0000 7f0ea9b0a700 1 -- 172.21.15.77:0/314433087 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f0ea402cec0 con 0x7f0ea41041c0 2023-12-01T01:16:30.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:29 smithi137 ceph-mon[103312]: pgmap v254: 97 pgs: 10 creating+peering, 60 active+clean, 27 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 7 op/s 2023-12-01T01:16:30.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:29 smithi137 ceph-mon[103312]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-12-01T01:16:30.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:29 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/2422083924' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-12-01T01:16:30.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:29 smithi137 ceph-mon[103312]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-12-01T01:16:30.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:29 smithi137 ceph-mon[103312]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-01T01:16:30.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:29 smithi137 ceph-mon[103312]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-12-01T01:16:30.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:29 smithi137 ceph-mon[103312]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-12-01T01:16:30.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:29 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/2380553063' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-12-01T01:16:30.639 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.638+0000 7f0ea0ff9700 1 -- 172.21.15.77:0/314433087 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7f0e94098e00 con 0x7f0ea41041c0 2023-12-01T01:16:30.642 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.640+0000 7f0ea9b0a700 1 -- 172.21.15.77:0/314433087 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f0e8c0a05a0 msgr2=0x7f0e8c0a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:30.642 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.640+0000 7f0ea9b0a700 1 --2- 172.21.15.77:0/314433087 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f0e8c0a05a0 0x7f0e8c0a2a60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0e98005ea0 tx=0x7f0e98005e30 comp rx=0 tx=0).stop 2023-12-01T01:16:30.642 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.640+0000 7f0ea9b0a700 1 -- 172.21.15.77:0/314433087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ea41041c0 msgr2=0x7f0ea40ff840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:30.642 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.640+0000 7f0ea9b0a700 1 --2- 172.21.15.77:0/314433087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ea41041c0 0x7f0ea40ff840 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0e9400c9b0 tx=0x7f0e9400ccc0 comp rx=0 tx=0).stop 2023-12-01T01:16:30.642 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.640+0000 7f0ea9b0a700 1 -- 172.21.15.77:0/314433087 shutdown_connections 2023-12-01T01:16:30.642 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.640+0000 7f0ea9b0a700 1 --2- 172.21.15.77:0/314433087 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f0e8c0a05a0 0x7f0e8c0a2a60 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:30.642 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.640+0000 7f0ea9b0a700 1 --2- 172.21.15.77:0/314433087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ea4106af0 0x7f0ea40ffd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:30.642 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.640+0000 7f0ea9b0a700 1 --2- 172.21.15.77:0/314433087 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ea4101890 0x7f0ea4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:30.642 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.640+0000 7f0ea9b0a700 1 --2- 172.21.15.77:0/314433087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ea41041c0 0x7f0ea40ff840 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:30.642 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.640+0000 7f0ea9b0a700 1 -- 172.21.15.77:0/314433087 >> 172.21.15.77:0/314433087 conn(0x7f0ea40fb220 msgr2=0x7f0ea4102500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:30.643 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.640+0000 7f0ea9b0a700 1 -- 172.21.15.77:0/314433087 shutdown_connections 2023-12-01T01:16:30.643 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.641+0000 7f0ea9b0a700 1 -- 172.21.15.77:0/314433087 wait complete. 2023-12-01T01:16:30.657 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2023-12-01T01:16:30.657 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-12-01T01:16:30.784 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.781+0000 7f364ef01700 1 -- 172.21.15.77:0/2714001777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3648100890 msgr2=0x7f3648108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:30.784 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.781+0000 7f364ef01700 1 --2- 172.21.15.77:0/2714001777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3648100890 0x7f3648108130 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f364000b0a0 tx=0x7f364000b3b0 comp rx=0 tx=0).stop 2023-12-01T01:16:30.784 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.781+0000 7f364ef01700 1 -- 172.21.15.77:0/2714001777 shutdown_connections 2023-12-01T01:16:30.784 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.781+0000 7f364ef01700 1 --2- 172.21.15.77:0/2714001777 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3648100890 0x7f3648108130 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:30.784 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.781+0000 7f364ef01700 1 --2- 172.21.15.77:0/2714001777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3648101280 0x7f36481016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:30.784 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.781+0000 7f364ef01700 1 --2- 172.21.15.77:0/2714001777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36480ffed0 0x7f3648100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:30.785 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.781+0000 7f364ef01700 1 -- 172.21.15.77:0/2714001777 >> 172.21.15.77:0/2714001777 conn(0x7f36480fa960 msgr2=0x7f36480fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:30.785 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.781+0000 7f364ef01700 1 -- 172.21.15.77:0/2714001777 shutdown_connections 2023-12-01T01:16:30.785 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.781+0000 7f364ef01700 1 -- 172.21.15.77:0/2714001777 wait complete. 2023-12-01T01:16:30.785 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.781+0000 7f364ef01700 1 Processor -- start 2023-12-01T01:16:30.785 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.781+0000 7f364ef01700 1 -- start start 2023-12-01T01:16:30.785 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.782+0000 7f364ef01700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36480ffed0 0x7f36480ff950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:30.785 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.782+0000 7f364ef01700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3648100890 0x7f364819b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:30.785 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.782+0000 7f364ef01700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3648101280 0x7f364819dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:30.785 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.782+0000 7f364ef01700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364819e6b0 con 0x7f3648101280 2023-12-01T01:16:30.785 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.782+0000 7f364ef01700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364819e820 con 0x7f3648100890 2023-12-01T01:16:30.785 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.782+0000 7f364ef01700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364819e9e0 con 0x7f36480ffed0 2023-12-01T01:16:30.785 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.782+0000 7f364d49e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3648101280 0x7f364819dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:30.785 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.782+0000 7f364d49e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3648101280 0x7f364819dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57542/0 (socket says 172.21.15.77:57542) 2023-12-01T01:16:30.785 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.782+0000 7f364d49e700 1 -- 172.21.15.77:0/1844131106 learned_addr learned my addr 172.21.15.77:0/1844131106 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:30.786 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.782+0000 7f3647fff700 1 --2- 172.21.15.77:0/1844131106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3648100890 0x7f364819b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:30.786 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.782+0000 7f364cc9d700 1 --2- 172.21.15.77:0/1844131106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36480ffed0 0x7f36480ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:30.786 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.782+0000 7f364d49e700 1 -- 172.21.15.77:0/1844131106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36480ffed0 msgr2=0x7f36480ff950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:30.786 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.782+0000 7f364d49e700 1 --2- 172.21.15.77:0/1844131106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36480ffed0 0x7f36480ff950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:30.786 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.782+0000 7f364d49e700 1 -- 172.21.15.77:0/1844131106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3648100890 msgr2=0x7f364819b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:30.786 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.782+0000 7f364d49e700 1 --2- 172.21.15.77:0/1844131106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3648100890 0x7f364819b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:30.786 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.782+0000 7f364d49e700 1 -- 172.21.15.77:0/1844131106 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3640009d00 con 0x7f3648101280 2023-12-01T01:16:30.786 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.782+0000 7f364cc9d700 1 --2- 172.21.15.77:0/1844131106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36480ffed0 0x7f36480ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:16:30.786 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.782+0000 7f3647fff700 1 --2- 172.21.15.77:0/1844131106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3648100890 0x7f364819b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:16:30.786 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.782+0000 7f364d49e700 1 --2- 172.21.15.77:0/1844131106 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3648101280 0x7f364819dee0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f364000b070 tx=0x7f3640004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:30.786 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.782+0000 7f3645ffb700 1 -- 172.21.15.77:0/1844131106 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364000e040 con 0x7f3648101280 2023-12-01T01:16:30.786 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.782+0000 7f3645ffb700 1 -- 172.21.15.77:0/1844131106 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f364001bd20 con 0x7f3648101280 2023-12-01T01:16:30.786 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.782+0000 7f3645ffb700 1 -- 172.21.15.77:0/1844131106 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3640005910 con 0x7f3648101280 2023-12-01T01:16:30.786 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.782+0000 7f364ef01700 1 -- 172.21.15.77:0/1844131106 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f364819ec60 con 0x7f3648101280 2023-12-01T01:16:30.787 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.783+0000 7f364ef01700 1 -- 172.21.15.77:0/1844131106 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f364819f090 con 0x7f3648101280 2023-12-01T01:16:30.787 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.783+0000 7f364ef01700 1 -- 172.21.15.77:0/1844131106 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f364810b470 con 0x7f3648101280 2023-12-01T01:16:30.787 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.785+0000 7f3645ffb700 1 -- 172.21.15.77:0/1844131106 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f3640005040 con 0x7f3648101280 2023-12-01T01:16:30.787 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.785+0000 7f3645ffb700 1 --2- 172.21.15.77:0/1844131106 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f36380a05a0 0x7f36380a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:30.788 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.785+0000 7f3645ffb700 1 -- 172.21.15.77:0/1844131106 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f36400cdaf0 con 0x7f3648101280 2023-12-01T01:16:30.788 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.785+0000 7f364cc9d700 1 --2- 172.21.15.77:0/1844131106 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f36380a05a0 0x7f36380a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:30.788 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.786+0000 7f364cc9d700 1 --2- 172.21.15.77:0/1844131106 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f36380a05a0 0x7f36380a2a60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f363c005fd0 tx=0x7f363c005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:30.790 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:30.789+0000 7f3645ffb700 1 -- 172.21.15.77:0/1844131106 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3640097990 con 0x7f3648101280 2023-12-01T01:16:30.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:30 smithi077 ceph-mon[98867]: mds.? [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] up:active 2023-12-01T01:16:30.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:30 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/2380553063' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-12-01T01:16:30.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:30 smithi077 ceph-mon[98867]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-12-01T01:16:30.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:30 smithi077 ceph-mon[98867]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-12-01T01:16:30.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:30 smithi077 ceph-mon[98867]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2023-12-01T01:16:30.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:30 smithi077 ceph-mon[98867]: daemon mds.c is now active in filesystem cephfs as rank 2 2023-12-01T01:16:30.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:30 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/314433087' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-12-01T01:16:31.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:30 smithi203 ceph-mon[103581]: mds.? [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] up:active 2023-12-01T01:16:31.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:30 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/2380553063' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-12-01T01:16:31.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:30 smithi203 ceph-mon[103581]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-12-01T01:16:31.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:30 smithi203 ceph-mon[103581]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-12-01T01:16:31.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:30 smithi203 ceph-mon[103581]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2023-12-01T01:16:31.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:30 smithi203 ceph-mon[103581]: daemon mds.c is now active in filesystem cephfs as rank 2 2023-12-01T01:16:31.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:30 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/314433087' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-12-01T01:16:31.013 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.012+0000 7f364ef01700 1 -- 172.21.15.77:0/1844131106 --> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f36480611d0 con 0x7f36380a05a0 2023-12-01T01:16:31.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:30 smithi137 ceph-mon[103312]: mds.? [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] up:active 2023-12-01T01:16:31.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:30 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/2380553063' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-12-01T01:16:31.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:30 smithi137 ceph-mon[103312]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-12-01T01:16:31.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:30 smithi137 ceph-mon[103312]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-12-01T01:16:31.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:30 smithi137 ceph-mon[103312]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2023-12-01T01:16:31.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:30 smithi137 ceph-mon[103312]: daemon mds.c is now active in filesystem cephfs as rank 2 2023-12-01T01:16:31.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:30 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/314433087' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-12-01T01:16:31.029 INFO:journalctl@ceph.mds.b.smithi137.stdout:Dec 01 01:16:30 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-b[141375]: 2023-12-01T01:16:30.730+0000 7f1cc1df2700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-01T01:16:31.029 INFO:journalctl@ceph.mds.b.smithi137.stdout:Dec 01 01:16:30 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-b[141375]: 2023-12-01T01:16:30.730+0000 7f1cc1df2700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-12-01T01:16:31.029 INFO:journalctl@ceph.mds.b.smithi137.stdout:Dec 01 01:16:30 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-b[141375]: 2023-12-01T01:16:30.730+0000 7f1cc1df2700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-12-01T01:16:31.404 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:16:31 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:16:31.017+0000 7f73ed995700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:16:31.404 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:16:31 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:16:31.017+0000 7f73ed995700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:16:31.404 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:16:31 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:16:31.017+0000 7f73ed995700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:16:31.404 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:16:31 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:16:31.017+0000 7f73ed995700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:16:31.404 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:16:31 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:16:31.017+0000 7f73ed995700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-01T01:16:31.629 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.627+0000 7f3645ffb700 1 -- 172.21.15.77:0/1844131106 <== mgr.14152 v2:172.21.15.77:6800/3838941679 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f36480611d0 con 0x7f36380a05a0 2023-12-01T01:16:31.631 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.630+0000 7f364ef01700 1 -- 172.21.15.77:0/1844131106 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f36380a05a0 msgr2=0x7f36380a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:31.631 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.630+0000 7f364ef01700 1 --2- 172.21.15.77:0/1844131106 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f36380a05a0 0x7f36380a2a60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f363c005fd0 tx=0x7f363c005dc0 comp rx=0 tx=0).stop 2023-12-01T01:16:31.631 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.630+0000 7f364ef01700 1 -- 172.21.15.77:0/1844131106 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3648101280 msgr2=0x7f364819dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:31.632 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.630+0000 7f364ef01700 1 --2- 172.21.15.77:0/1844131106 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3648101280 0x7f364819dee0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f364000b070 tx=0x7f3640004db0 comp rx=0 tx=0).stop 2023-12-01T01:16:31.632 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.630+0000 7f364ef01700 1 -- 172.21.15.77:0/1844131106 shutdown_connections 2023-12-01T01:16:31.632 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.630+0000 7f364ef01700 1 --2- 172.21.15.77:0/1844131106 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f36380a05a0 0x7f36380a2a60 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:31.632 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.630+0000 7f364ef01700 1 --2- 172.21.15.77:0/1844131106 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3648101280 0x7f364819dee0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:31.632 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.630+0000 7f364ef01700 1 --2- 172.21.15.77:0/1844131106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36480ffed0 0x7f36480ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:31.632 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.630+0000 7f364ef01700 1 --2- 172.21.15.77:0/1844131106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3648100890 0x7f364819b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:31.632 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.630+0000 7f364ef01700 1 -- 172.21.15.77:0/1844131106 >> 172.21.15.77:0/1844131106 conn(0x7f36480fa960 msgr2=0x7f36480fcc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:31.632 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.630+0000 7f364ef01700 1 -- 172.21.15.77:0/1844131106 shutdown_connections 2023-12-01T01:16:31.632 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.630+0000 7f364ef01700 1 -- 172.21.15.77:0/1844131106 wait complete. 2023-12-01T01:16:31.647 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-12-01T01:16:31.776 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.773+0000 7fe486c79700 1 -- 172.21.15.77:0/2195252131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe4801008d0 msgr2=0x7fe480108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:31.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.773+0000 7fe486c79700 1 --2- 172.21.15.77:0/2195252131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe4801008d0 0x7fe480108170 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe47800b0a0 tx=0x7fe47800b3b0 comp rx=0 tx=0).stop 2023-12-01T01:16:31.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.773+0000 7fe486c79700 1 -- 172.21.15.77:0/2195252131 shutdown_connections 2023-12-01T01:16:31.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.773+0000 7fe486c79700 1 --2- 172.21.15.77:0/2195252131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe4801008d0 0x7fe480108170 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:31.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.773+0000 7fe486c79700 1 --2- 172.21.15.77:0/2195252131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4801012c0 0x7fe4801016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:31.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.773+0000 7fe486c79700 1 --2- 172.21.15.77:0/2195252131 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4800fff10 0x7fe480100390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:31.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.773+0000 7fe486c79700 1 -- 172.21.15.77:0/2195252131 >> 172.21.15.77:0/2195252131 conn(0x7fe4800fa980 msgr2=0x7fe4800fcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:31.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.773+0000 7fe486c79700 1 -- 172.21.15.77:0/2195252131 shutdown_connections 2023-12-01T01:16:31.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.773+0000 7fe486c79700 1 -- 172.21.15.77:0/2195252131 wait complete. 2023-12-01T01:16:31.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.774+0000 7fe486c79700 1 Processor -- start 2023-12-01T01:16:31.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.774+0000 7fe486c79700 1 -- start start 2023-12-01T01:16:31.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.774+0000 7fe486c79700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4800fff10 0x7fe4800ff970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:31.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.774+0000 7fe486c79700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4801008d0 0x7fe48019b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:31.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.774+0000 7fe486c79700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe4801012c0 0x7fe48019de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:31.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.774+0000 7fe486c79700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48019e5d0 con 0x7fe4801012c0 2023-12-01T01:16:31.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.774+0000 7fe486c79700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48019e740 con 0x7fe4801008d0 2023-12-01T01:16:31.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.774+0000 7fe486c79700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48019e900 con 0x7fe4800fff10 2023-12-01T01:16:31.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.774+0000 7fe485216700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe4801012c0 0x7fe48019de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:31.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.774+0000 7fe485216700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe4801012c0 0x7fe48019de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57592/0 (socket says 172.21.15.77:57592) 2023-12-01T01:16:31.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.774+0000 7fe485216700 1 -- 172.21.15.77:0/927562074 learned_addr learned my addr 172.21.15.77:0/927562074 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:31.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.774+0000 7fe484a15700 1 --2- 172.21.15.77:0/927562074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4800fff10 0x7fe4800ff970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:31.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.774+0000 7fe47ffff700 1 --2- 172.21.15.77:0/927562074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4801008d0 0x7fe48019b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:31.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.774+0000 7fe485216700 1 -- 172.21.15.77:0/927562074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4800fff10 msgr2=0x7fe4800ff970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:31.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.774+0000 7fe485216700 1 --2- 172.21.15.77:0/927562074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4800fff10 0x7fe4800ff970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:31.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.774+0000 7fe485216700 1 -- 172.21.15.77:0/927562074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4801008d0 msgr2=0x7fe48019b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:31.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.774+0000 7fe485216700 1 --2- 172.21.15.77:0/927562074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4801008d0 0x7fe48019b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:31.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.774+0000 7fe485216700 1 -- 172.21.15.77:0/927562074 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe478009d00 con 0x7fe4801012c0 2023-12-01T01:16:31.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.774+0000 7fe484a15700 1 --2- 172.21.15.77:0/927562074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4800fff10 0x7fe4800ff970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:16:31.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.775+0000 7fe485216700 1 --2- 172.21.15.77:0/927562074 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe4801012c0 0x7fe48019de30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe47800b070 tx=0x7fe478004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:31.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.775+0000 7fe47dffb700 1 -- 172.21.15.77:0/927562074 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47800e040 con 0x7fe4801012c0 2023-12-01T01:16:31.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.775+0000 7fe486c79700 1 -- 172.21.15.77:0/927562074 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe48019eb80 con 0x7fe4801012c0 2023-12-01T01:16:31.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.775+0000 7fe47dffb700 1 -- 172.21.15.77:0/927562074 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe47801bd20 con 0x7fe4801012c0 2023-12-01T01:16:31.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.775+0000 7fe47dffb700 1 -- 172.21.15.77:0/927562074 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe478005910 con 0x7fe4801012c0 2023-12-01T01:16:31.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.775+0000 7fe486c79700 1 -- 172.21.15.77:0/927562074 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe48019f000 con 0x7fe4801012c0 2023-12-01T01:16:31.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.776+0000 7fe486c79700 1 -- 172.21.15.77:0/927562074 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe480106bc0 con 0x7fe4801012c0 2023-12-01T01:16:31.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.777+0000 7fe47dffb700 1 -- 172.21.15.77:0/927562074 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fe478005040 con 0x7fe4801012c0 2023-12-01T01:16:31.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.777+0000 7fe47dffb700 1 --2- 172.21.15.77:0/927562074 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fe4700a0520 0x7fe4700a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:31.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.777+0000 7fe484a15700 1 --2- 172.21.15.77:0/927562074 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fe4700a0520 0x7fe4700a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:31.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.777+0000 7fe47dffb700 1 -- 172.21.15.77:0/927562074 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe4780cd660 con 0x7fe4801012c0 2023-12-01T01:16:31.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.778+0000 7fe484a15700 1 --2- 172.21.15.77:0/927562074 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fe4700a0520 0x7fe4700a29e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe480068ec0 tx=0x7fe474005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:31.783 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:31.782+0000 7fe47dffb700 1 -- 172.21.15.77:0/927562074 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe4780973e0 con 0x7fe4801012c0 2023-12-01T01:16:31.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:31 smithi077 ceph-mon[98867]: pgmap v255: 97 pgs: 97 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 10 op/s 2023-12-01T01:16:31.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:31 smithi077 ceph-mon[98867]: mds.? [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] up:active 2023-12-01T01:16:31.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:31 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/314433087' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-12-01T01:16:31.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:31 smithi077 ceph-mon[98867]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-12-01T01:16:31.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:31 smithi077 ceph-mon[98867]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2023-12-01T01:16:31.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:31 smithi077 ceph-mon[98867]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2023-12-01T01:16:31.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:31 smithi077 ceph-mon[98867]: daemon mds.h is now active in filesystem cephfs as rank 3 2023-12-01T01:16:31.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:31 smithi077 ceph-mon[98867]: mds.? [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] up:active 2023-12-01T01:16:31.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:31 smithi077 ceph-mon[98867]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-12-01T01:16:31.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:31 smithi077 ceph-mon[98867]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-12-01T01:16:31.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:31 smithi077 ceph-mon[98867]: Cluster is now healthy 2023-12-01T01:16:31.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:31 smithi077 ceph-mon[98867]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 4 up:standby-replay 4 up:standby 2023-12-01T01:16:31.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:31 smithi077 ceph-mon[98867]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2023-12-01T01:16:31.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:31 smithi077 ceph-mon[98867]: daemon mds.a is now active in filesystem cephfs as rank 4 2023-12-01T01:16:32.005 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.004+0000 7fe486c79700 1 -- 172.21.15.77:0/927562074 --> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7fe4800611d0 con 0x7fe4700a0520 2023-12-01T01:16:32.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.008+0000 7fe47dffb700 1 -- 172.21.15.77:0/927562074 <== mgr.14152 v2:172.21.15.77:6800/3838941679 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7fe4800611d0 con 0x7fe4700a0520 2023-12-01T01:16:32.009 INFO:teuthology.orchestra.run.smithi077.stdout:/volumes/_nogroup/sv_0/d9e05e04-b5a0-4815-a37d-0443b2ddabbb 2023-12-01T01:16:32.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.010+0000 7fe486c79700 1 -- 172.21.15.77:0/927562074 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fe4700a0520 msgr2=0x7fe4700a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:32.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.010+0000 7fe486c79700 1 --2- 172.21.15.77:0/927562074 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fe4700a0520 0x7fe4700a29e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe480068ec0 tx=0x7fe474005f90 comp rx=0 tx=0).stop 2023-12-01T01:16:32.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.010+0000 7fe486c79700 1 -- 172.21.15.77:0/927562074 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe4801012c0 msgr2=0x7fe48019de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:32.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.010+0000 7fe486c79700 1 --2- 172.21.15.77:0/927562074 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe4801012c0 0x7fe48019de30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe47800b070 tx=0x7fe478004db0 comp rx=0 tx=0).stop 2023-12-01T01:16:32.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.010+0000 7fe486c79700 1 -- 172.21.15.77:0/927562074 shutdown_connections 2023-12-01T01:16:32.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.010+0000 7fe486c79700 1 --2- 172.21.15.77:0/927562074 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fe4700a0520 0x7fe4700a29e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:32.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.010+0000 7fe486c79700 1 --2- 172.21.15.77:0/927562074 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe4801012c0 0x7fe48019de30 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:32.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.010+0000 7fe486c79700 1 --2- 172.21.15.77:0/927562074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4800fff10 0x7fe4800ff970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:32.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.010+0000 7fe486c79700 1 --2- 172.21.15.77:0/927562074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4801008d0 0x7fe48019b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:32.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.010+0000 7fe486c79700 1 -- 172.21.15.77:0/927562074 >> 172.21.15.77:0/927562074 conn(0x7fe4800fa980 msgr2=0x7fe4800fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:32.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.010+0000 7fe486c79700 1 -- 172.21.15.77:0/927562074 shutdown_connections 2023-12-01T01:16:32.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.010+0000 7fe486c79700 1 -- 172.21.15.77:0/927562074 wait complete. 2023-12-01T01:16:32.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:31 smithi203 ceph-mon[103581]: pgmap v255: 97 pgs: 97 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 10 op/s 2023-12-01T01:16:32.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:31 smithi203 ceph-mon[103581]: mds.? [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] up:active 2023-12-01T01:16:32.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:31 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/314433087' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-12-01T01:16:32.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:31 smithi203 ceph-mon[103581]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-12-01T01:16:32.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:31 smithi203 ceph-mon[103581]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2023-12-01T01:16:32.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:31 smithi203 ceph-mon[103581]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2023-12-01T01:16:32.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:31 smithi203 ceph-mon[103581]: daemon mds.h is now active in filesystem cephfs as rank 3 2023-12-01T01:16:32.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:31 smithi203 ceph-mon[103581]: mds.? [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] up:active 2023-12-01T01:16:32.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:31 smithi203 ceph-mon[103581]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-12-01T01:16:32.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:31 smithi203 ceph-mon[103581]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-12-01T01:16:32.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:31 smithi203 ceph-mon[103581]: Cluster is now healthy 2023-12-01T01:16:32.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:31 smithi203 ceph-mon[103581]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 4 up:standby-replay 4 up:standby 2023-12-01T01:16:32.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:31 smithi203 ceph-mon[103581]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2023-12-01T01:16:32.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:31 smithi203 ceph-mon[103581]: daemon mds.a is now active in filesystem cephfs as rank 4 2023-12-01T01:16:32.024 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-12-01T01:16:32.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:31 smithi137 ceph-mon[103312]: pgmap v255: 97 pgs: 97 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 10 op/s 2023-12-01T01:16:32.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:31 smithi137 ceph-mon[103312]: mds.? [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] up:active 2023-12-01T01:16:32.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:31 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/314433087' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-12-01T01:16:32.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:31 smithi137 ceph-mon[103312]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-12-01T01:16:32.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:31 smithi137 ceph-mon[103312]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2023-12-01T01:16:32.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:31 smithi137 ceph-mon[103312]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2023-12-01T01:16:32.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:31 smithi137 ceph-mon[103312]: daemon mds.h is now active in filesystem cephfs as rank 3 2023-12-01T01:16:32.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:31 smithi137 ceph-mon[103312]: mds.? [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] up:active 2023-12-01T01:16:32.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:31 smithi137 ceph-mon[103312]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-12-01T01:16:32.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:31 smithi137 ceph-mon[103312]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-12-01T01:16:32.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:31 smithi137 ceph-mon[103312]: Cluster is now healthy 2023-12-01T01:16:32.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:31 smithi137 ceph-mon[103312]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 4 up:standby-replay 4 up:standby 2023-12-01T01:16:32.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:31 smithi137 ceph-mon[103312]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2023-12-01T01:16:32.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:31 smithi137 ceph-mon[103312]: daemon mds.a is now active in filesystem cephfs as rank 4 2023-12-01T01:16:32.150 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.147+0000 7f2c0fd63700 1 -- 172.21.15.77:0/1543375921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c080ff050 msgr2=0x7f2c080ff470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:32.150 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.147+0000 7f2c0fd63700 1 --2- 172.21.15.77:0/1543375921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c080ff050 0x7f2c080ff470 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8009990 tx=0x7f2bf8009ca0 comp rx=0 tx=0).stop 2023-12-01T01:16:32.150 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.147+0000 7f2c0fd63700 1 -- 172.21.15.77:0/1543375921 shutdown_connections 2023-12-01T01:16:32.150 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.147+0000 7f2c0fd63700 1 --2- 172.21.15.77:0/1543375921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c08102a60 0x7f2c08109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:32.151 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.147+0000 7f2c0fd63700 1 --2- 172.21.15.77:0/1543375921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c080ff050 0x7f2c080ff470 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:32.151 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.147+0000 7f2c0fd63700 1 --2- 172.21.15.77:0/1543375921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c081020c0 0x7f2c08102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:32.151 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.147+0000 7f2c0fd63700 1 -- 172.21.15.77:0/1543375921 >> 172.21.15.77:0/1543375921 conn(0x7f2c080fa960 msgr2=0x7f2c080fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:32.151 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.147+0000 7f2c0fd63700 1 -- 172.21.15.77:0/1543375921 shutdown_connections 2023-12-01T01:16:32.151 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.147+0000 7f2c0fd63700 1 -- 172.21.15.77:0/1543375921 wait complete. 2023-12-01T01:16:32.151 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.147+0000 7f2c0fd63700 1 Processor -- start 2023-12-01T01:16:32.151 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.147+0000 7f2c0fd63700 1 -- start start 2023-12-01T01:16:32.151 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.147+0000 7f2c0fd63700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c080ff050 0x7f2c08196ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:32.151 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.147+0000 7f2c0fd63700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c081020c0 0x7f2c08197010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:32.151 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.147+0000 7f2c0fd63700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c08102a60 0x7f2c0819e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:32.151 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.147+0000 7f2c0fd63700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0819e650 con 0x7f2c08102a60 2023-12-01T01:16:32.151 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.147+0000 7f2c0fd63700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0819e790 con 0x7f2c080ff050 2023-12-01T01:16:32.151 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.147+0000 7f2c0fd63700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0819e920 con 0x7f2c081020c0 2023-12-01T01:16:32.151 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.148+0000 7f2c0e300700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c08102a60 0x7f2c0819e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:32.151 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.148+0000 7f2c0e300700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c08102a60 0x7f2c0819e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57606/0 (socket says 172.21.15.77:57606) 2023-12-01T01:16:32.152 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.148+0000 7f2c0e300700 1 -- 172.21.15.77:0/1548725083 learned_addr learned my addr 172.21.15.77:0/1548725083 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:32.152 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.148+0000 7f2c0d2fe700 1 --2- 172.21.15.77:0/1548725083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c081020c0 0x7f2c08197010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:32.152 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.148+0000 7f2c0daff700 1 --2- 172.21.15.77:0/1548725083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c080ff050 0x7f2c08196ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:32.152 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.148+0000 7f2c0e300700 1 -- 172.21.15.77:0/1548725083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c081020c0 msgr2=0x7f2c08197010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:32.152 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.148+0000 7f2c0e300700 1 --2- 172.21.15.77:0/1548725083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c081020c0 0x7f2c08197010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:32.152 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.148+0000 7f2c0e300700 1 -- 172.21.15.77:0/1548725083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c080ff050 msgr2=0x7f2c08196ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:32.152 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.148+0000 7f2c0e300700 1 --2- 172.21.15.77:0/1548725083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c080ff050 0x7f2c08196ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:32.152 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.148+0000 7f2c0e300700 1 -- 172.21.15.77:0/1548725083 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf80095f0 con 0x7f2c08102a60 2023-12-01T01:16:32.152 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.148+0000 7f2c0d2fe700 1 --2- 172.21.15.77:0/1548725083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c081020c0 0x7f2c08197010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:16:32.152 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.148+0000 7f2c0e300700 1 --2- 172.21.15.77:0/1548725083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c08102a60 0x7f2c0819e080 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2c00007ee0 tx=0x7f2c0000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:32.152 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.148+0000 7f2bfeffd700 1 -- 172.21.15.77:0/1548725083 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0000ddd0 con 0x7f2c08102a60 2023-12-01T01:16:32.152 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.148+0000 7f2bfeffd700 1 -- 172.21.15.77:0/1548725083 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c0000f040 con 0x7f2c08102a60 2023-12-01T01:16:32.152 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.148+0000 7f2c0fd63700 1 -- 172.21.15.77:0/1548725083 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c0819ec00 con 0x7f2c08102a60 2023-12-01T01:16:32.152 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.148+0000 7f2bfeffd700 1 -- 172.21.15.77:0/1548725083 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c00015400 con 0x7f2c08102a60 2023-12-01T01:16:32.153 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.148+0000 7f2c0fd63700 1 -- 172.21.15.77:0/1548725083 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c0819f0c0 con 0x7f2c08102a60 2023-12-01T01:16:32.153 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.149+0000 7f2c0fd63700 1 -- 172.21.15.77:0/1548725083 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c0810c560 con 0x7f2c08102a60 2023-12-01T01:16:32.153 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.150+0000 7f2bfeffd700 1 -- 172.21.15.77:0/1548725083 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f2c00004020 con 0x7f2c08102a60 2023-12-01T01:16:32.154 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.151+0000 7f2bfeffd700 1 --2- 172.21.15.77:0/1548725083 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f2bf40a05a0 0x7f2bf40a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:32.154 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.151+0000 7f2c0daff700 1 --2- 172.21.15.77:0/1548725083 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f2bf40a05a0 0x7f2bf40a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:32.154 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.151+0000 7f2bfeffd700 1 -- 172.21.15.77:0/1548725083 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2c000cb260 con 0x7f2c08102a60 2023-12-01T01:16:32.154 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.151+0000 7f2c0daff700 1 --2- 172.21.15.77:0/1548725083 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f2bf40a05a0 0x7f2bf40a2a60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8006010 tx=0x7f2bf800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:32.156 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.155+0000 7f2bfeffd700 1 -- 172.21.15.77:0/1548725083 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2c0004f0e0 con 0x7f2c08102a60 2023-12-01T01:16:32.377 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.376+0000 7f2c0fd63700 1 -- 172.21.15.77:0/1548725083 --> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c080611d0 con 0x7f2bf40a05a0 2023-12-01T01:16:32.402 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.401+0000 7f2bfeffd700 1 -- 172.21.15.77:0/1548725083 <== mgr.14152 v2:172.21.15.77:6800/3838941679 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2c080611d0 con 0x7f2bf40a05a0 2023-12-01T01:16:32.404 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.403+0000 7f2c0fd63700 1 -- 172.21.15.77:0/1548725083 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f2bf40a05a0 msgr2=0x7f2bf40a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:32.405 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.403+0000 7f2c0fd63700 1 --2- 172.21.15.77:0/1548725083 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f2bf40a05a0 0x7f2bf40a2a60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8006010 tx=0x7f2bf800b540 comp rx=0 tx=0).stop 2023-12-01T01:16:32.405 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.403+0000 7f2c0fd63700 1 -- 172.21.15.77:0/1548725083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c08102a60 msgr2=0x7f2c0819e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:32.405 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.403+0000 7f2c0fd63700 1 --2- 172.21.15.77:0/1548725083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c08102a60 0x7f2c0819e080 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2c00007ee0 tx=0x7f2c0000d3b0 comp rx=0 tx=0).stop 2023-12-01T01:16:32.405 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.403+0000 7f2c0fd63700 1 -- 172.21.15.77:0/1548725083 shutdown_connections 2023-12-01T01:16:32.405 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.403+0000 7f2c0fd63700 1 --2- 172.21.15.77:0/1548725083 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f2bf40a05a0 0x7f2bf40a2a60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:32.405 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.403+0000 7f2c0fd63700 1 --2- 172.21.15.77:0/1548725083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c08102a60 0x7f2c0819e080 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:32.405 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.403+0000 7f2c0fd63700 1 --2- 172.21.15.77:0/1548725083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c080ff050 0x7f2c08196ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:32.405 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.403+0000 7f2c0fd63700 1 --2- 172.21.15.77:0/1548725083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c081020c0 0x7f2c08197010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:32.405 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.403+0000 7f2c0fd63700 1 -- 172.21.15.77:0/1548725083 >> 172.21.15.77:0/1548725083 conn(0x7f2c080fa960 msgr2=0x7f2c080fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:32.405 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.403+0000 7f2c0fd63700 1 -- 172.21.15.77:0/1548725083 shutdown_connections 2023-12-01T01:16:32.405 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.403+0000 7f2c0fd63700 1 -- 172.21.15.77:0/1548725083 wait complete. 2023-12-01T01:16:32.416 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-12-01T01:16:32.542 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.539+0000 7f7835339700 1 -- 172.21.15.77:0/152754730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7830101240 msgr2=0x7f7830101660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:32.542 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.539+0000 7f7835339700 1 --2- 172.21.15.77:0/152754730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7830101240 0x7f7830101660 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7824009990 tx=0x7f7824009ca0 comp rx=0 tx=0).stop 2023-12-01T01:16:32.542 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.539+0000 7f7835339700 1 -- 172.21.15.77:0/152754730 shutdown_connections 2023-12-01T01:16:32.542 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.539+0000 7f7835339700 1 --2- 172.21.15.77:0/152754730 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7830100910 0x7f78301081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:32.543 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.539+0000 7f7835339700 1 --2- 172.21.15.77:0/152754730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7830101240 0x7f7830101660 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:32.543 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.539+0000 7f7835339700 1 --2- 172.21.15.77:0/152754730 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78300fff70 0x7f78301003d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:32.543 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.539+0000 7f7835339700 1 -- 172.21.15.77:0/152754730 >> 172.21.15.77:0/152754730 conn(0x7f78300fa980 msgr2=0x7f78300fcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:32.543 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.539+0000 7f7835339700 1 -- 172.21.15.77:0/152754730 shutdown_connections 2023-12-01T01:16:32.543 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.539+0000 7f7835339700 1 -- 172.21.15.77:0/152754730 wait complete. 2023-12-01T01:16:32.543 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.539+0000 7f7835339700 1 Processor -- start 2023-12-01T01:16:32.543 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.539+0000 7f7835339700 1 -- start start 2023-12-01T01:16:32.543 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.540+0000 7f7835339700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f78300fff70 0x7f78301958e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:32.543 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.540+0000 7f7835339700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7830100910 0x7f7830195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:32.543 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.540+0000 7f7835339700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7830101240 0x7f783019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:32.543 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.540+0000 7f7835339700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783019d460 con 0x7f7830101240 2023-12-01T01:16:32.543 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.540+0000 7f7835339700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783019d5a0 con 0x7f78300fff70 2023-12-01T01:16:32.543 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.540+0000 7f7835339700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783019d730 con 0x7f7830100910 2023-12-01T01:16:32.543 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.540+0000 7f782f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7830101240 0x7f783019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:32.543 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.540+0000 7f782f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7830101240 0x7f783019ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57622/0 (socket says 172.21.15.77:57622) 2023-12-01T01:16:32.544 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.540+0000 7f782f7fe700 1 -- 172.21.15.77:0/378106232 learned_addr learned my addr 172.21.15.77:0/378106232 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:32.544 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.540+0000 7f782effd700 1 --2- 172.21.15.77:0/378106232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f78300fff70 0x7f78301958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:32.544 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.540+0000 7f782e7fc700 1 --2- 172.21.15.77:0/378106232 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7830100910 0x7f7830195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:32.544 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.540+0000 7f782f7fe700 1 -- 172.21.15.77:0/378106232 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7830100910 msgr2=0x7f7830195e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:32.544 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.540+0000 7f782f7fe700 1 --2- 172.21.15.77:0/378106232 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7830100910 0x7f7830195e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:32.544 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.540+0000 7f782f7fe700 1 -- 172.21.15.77:0/378106232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f78300fff70 msgr2=0x7f78301958e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:32.544 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.540+0000 7f782f7fe700 1 --2- 172.21.15.77:0/378106232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f78300fff70 0x7f78301958e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:32.544 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.540+0000 7f782f7fe700 1 -- 172.21.15.77:0/378106232 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78240095f0 con 0x7f7830101240 2023-12-01T01:16:32.544 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.540+0000 7f782f7fe700 1 --2- 172.21.15.77:0/378106232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7830101240 0x7f783019ce90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f782800efb0 tx=0x7f782800c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:32.544 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.540+0000 7f7817fff700 1 -- 172.21.15.77:0/378106232 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7828011670 con 0x7f7830101240 2023-12-01T01:16:32.544 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.540+0000 7f7817fff700 1 -- 172.21.15.77:0/378106232 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7828011c90 con 0x7f7830101240 2023-12-01T01:16:32.544 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.540+0000 7f7835339700 1 -- 172.21.15.77:0/378106232 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78300feec0 con 0x7f7830101240 2023-12-01T01:16:32.544 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.540+0000 7f7817fff700 1 -- 172.21.15.77:0/378106232 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782801ac70 con 0x7f7830101240 2023-12-01T01:16:32.544 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.540+0000 7f7835339700 1 -- 172.21.15.77:0/378106232 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78300ff3d0 con 0x7f7830101240 2023-12-01T01:16:32.545 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.541+0000 7f7835339700 1 -- 172.21.15.77:0/378106232 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f783010b4a0 con 0x7f7830101240 2023-12-01T01:16:32.545 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.543+0000 7f7817fff700 1 -- 172.21.15.77:0/378106232 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f78280117d0 con 0x7f7830101240 2023-12-01T01:16:32.545 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.544+0000 7f7817fff700 1 --2- 172.21.15.77:0/378106232 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f78180a05a0 0x7f78180a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:32.545 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.544+0000 7f782effd700 1 --2- 172.21.15.77:0/378106232 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f78180a05a0 0x7f78180a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:32.545 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.544+0000 7f7817fff700 1 -- 172.21.15.77:0/378106232 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f78280ce120 con 0x7f7830101240 2023-12-01T01:16:32.546 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.544+0000 7f782effd700 1 --2- 172.21.15.77:0/378106232 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f78180a05a0 0x7f78180a2a60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f782400b5c0 tx=0x7f7824005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:32.548 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.547+0000 7f7817fff700 1 -- 172.21.15.77:0/378106232 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7828097e90 con 0x7f7830101240 2023-12-01T01:16:32.653 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.652+0000 7f7817fff700 1 -- 172.21.15.77:0/378106232 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f7828098260 con 0x7f7830101240 2023-12-01T01:16:32.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.779+0000 7f7835339700 1 -- 172.21.15.77:0/378106232 --> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f78300611d0 con 0x7f78180a05a0 2023-12-01T01:16:32.784 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.783+0000 7f7817fff700 1 -- 172.21.15.77:0/378106232 <== mgr.14152 v2:172.21.15.77:6800/3838941679 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f78300611d0 con 0x7f78180a05a0 2023-12-01T01:16:32.784 INFO:teuthology.orchestra.run.smithi077.stdout:/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 2023-12-01T01:16:32.786 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.785+0000 7f7835339700 1 -- 172.21.15.77:0/378106232 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f78180a05a0 msgr2=0x7f78180a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:32.787 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.785+0000 7f7835339700 1 --2- 172.21.15.77:0/378106232 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f78180a05a0 0x7f78180a2a60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f782400b5c0 tx=0x7f7824005fd0 comp rx=0 tx=0).stop 2023-12-01T01:16:32.787 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.785+0000 7f7835339700 1 -- 172.21.15.77:0/378106232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7830101240 msgr2=0x7f783019ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:32.787 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.785+0000 7f7835339700 1 --2- 172.21.15.77:0/378106232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7830101240 0x7f783019ce90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f782800efb0 tx=0x7f782800c550 comp rx=0 tx=0).stop 2023-12-01T01:16:32.787 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.785+0000 7f7835339700 1 -- 172.21.15.77:0/378106232 shutdown_connections 2023-12-01T01:16:32.787 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.785+0000 7f7835339700 1 --2- 172.21.15.77:0/378106232 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f78180a05a0 0x7f78180a2a60 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:32.787 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.785+0000 7f7835339700 1 --2- 172.21.15.77:0/378106232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7830101240 0x7f783019ce90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:32.787 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.785+0000 7f7835339700 1 --2- 172.21.15.77:0/378106232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f78300fff70 0x7f78301958e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:32.787 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.785+0000 7f7835339700 1 --2- 172.21.15.77:0/378106232 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7830100910 0x7f7830195e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:32.787 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.785+0000 7f7835339700 1 -- 172.21.15.77:0/378106232 >> 172.21.15.77:0/378106232 conn(0x7f78300fa980 msgr2=0x7f78300fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:32.787 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.785+0000 7f7835339700 1 -- 172.21.15.77:0/378106232 shutdown_connections 2023-12-01T01:16:32.787 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.785+0000 7f7835339700 1 -- 172.21.15.77:0/378106232 wait complete. 2023-12-01T01:16:32.798 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:32 smithi077 ceph-mon[98867]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:16:32.798 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:32 smithi077 ceph-mon[98867]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:16:32.798 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:32 smithi077 ceph-mon[98867]: pgmap v256: 97 pgs: 97 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s wr, 14 op/s 2023-12-01T01:16:32.798 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:32 smithi077 ceph-mon[98867]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:16:32.798 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:32 smithi077 ceph-mon[98867]: mds.? [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] up:active 2023-12-01T01:16:32.798 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:32 smithi077 ceph-mon[98867]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-12-01T01:16:32.798 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:32 smithi077 ceph-mon[98867]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:16:32.799 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:16:32.922 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.918+0000 7fc18706a700 1 -- 172.21.15.77:0/1977247849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc180100da0 msgr2=0x7fc1801011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:32.922 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.918+0000 7fc18706a700 1 --2- 172.21.15.77:0/1977247849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc180100da0 0x7fc1801011c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc1740099c0 tx=0x7fc174009cd0 comp rx=0 tx=0).stop 2023-12-01T01:16:32.922 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.918+0000 7fc18706a700 1 -- 172.21.15.77:0/1977247849 shutdown_connections 2023-12-01T01:16:32.922 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.918+0000 7fc18706a700 1 --2- 172.21.15.77:0/1977247849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc180102930 0x7fc1801091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:32.922 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.918+0000 7fc18706a700 1 --2- 172.21.15.77:0/1977247849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc180100da0 0x7fc1801011c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:32.923 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.918+0000 7fc18706a700 1 --2- 172.21.15.77:0/1977247849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc180101f90 0x7fc1801023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:32.923 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.918+0000 7fc18706a700 1 -- 172.21.15.77:0/1977247849 >> 172.21.15.77:0/1977247849 conn(0x7fc1800fc2d0 msgr2=0x7fc1800fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:32.923 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.919+0000 7fc18706a700 1 -- 172.21.15.77:0/1977247849 shutdown_connections 2023-12-01T01:16:32.923 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.919+0000 7fc18706a700 1 -- 172.21.15.77:0/1977247849 wait complete. 2023-12-01T01:16:32.923 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.919+0000 7fc18706a700 1 Processor -- start 2023-12-01T01:16:32.923 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.919+0000 7fc18706a700 1 -- start start 2023-12-01T01:16:32.923 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.919+0000 7fc18706a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc180100da0 0x7fc180074b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:32.923 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.919+0000 7fc18706a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc180101f90 0x7fc180073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:32.923 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.919+0000 7fc18706a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc180102930 0x7fc1800736c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:32.923 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.919+0000 7fc18706a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc180073e60 con 0x7fc180100da0 2023-12-01T01:16:32.923 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.919+0000 7fc18706a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc180073fd0 con 0x7fc180101f90 2023-12-01T01:16:32.923 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.919+0000 7fc18706a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1801a0950 con 0x7fc180102930 2023-12-01T01:16:32.923 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.919+0000 7fc184e06700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc180100da0 0x7fc180074b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:32.924 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.919+0000 7fc184e06700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc180100da0 0x7fc180074b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57646/0 (socket says 172.21.15.77:57646) 2023-12-01T01:16:32.924 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.919+0000 7fc184e06700 1 -- 172.21.15.77:0/709623757 learned_addr learned my addr 172.21.15.77:0/709623757 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:32.924 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.919+0000 7fc17ffff700 1 --2- 172.21.15.77:0/709623757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc180101f90 0x7fc180073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:32.924 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.920+0000 7fc185607700 1 --2- 172.21.15.77:0/709623757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc180102930 0x7fc1800736c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:32.924 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.920+0000 7fc184e06700 1 -- 172.21.15.77:0/709623757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc180102930 msgr2=0x7fc1800736c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:32.924 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.920+0000 7fc184e06700 1 --2- 172.21.15.77:0/709623757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc180102930 0x7fc1800736c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:32.924 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.920+0000 7fc184e06700 1 -- 172.21.15.77:0/709623757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc180101f90 msgr2=0x7fc180073180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:32.924 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.920+0000 7fc184e06700 1 --2- 172.21.15.77:0/709623757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc180101f90 0x7fc180073180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:32.924 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.920+0000 7fc184e06700 1 -- 172.21.15.77:0/709623757 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1740095f0 con 0x7fc180100da0 2023-12-01T01:16:32.924 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.920+0000 7fc185607700 1 --2- 172.21.15.77:0/709623757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc180102930 0x7fc1800736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:16:32.924 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.920+0000 7fc17ffff700 1 --2- 172.21.15.77:0/709623757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc180101f90 0x7fc180073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:16:32.924 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.920+0000 7fc184e06700 1 --2- 172.21.15.77:0/709623757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc180100da0 0x7fc180074b40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc1740036f0 tx=0x7fc1740057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:32.924 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.920+0000 7fc17dffb700 1 -- 172.21.15.77:0/709623757 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17401d070 con 0x7fc180100da0 2023-12-01T01:16:32.924 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.920+0000 7fc17dffb700 1 -- 172.21.15.77:0/709623757 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1740041c0 con 0x7fc180100da0 2023-12-01T01:16:32.925 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.920+0000 7fc17dffb700 1 -- 172.21.15.77:0/709623757 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17400f460 con 0x7fc180100da0 2023-12-01T01:16:32.925 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.920+0000 7fc18706a700 1 -- 172.21.15.77:0/709623757 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1801a0bd0 con 0x7fc180100da0 2023-12-01T01:16:32.925 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.920+0000 7fc18706a700 1 -- 172.21.15.77:0/709623757 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1801a1030 con 0x7fc180100da0 2023-12-01T01:16:32.925 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.921+0000 7fc18706a700 1 -- 172.21.15.77:0/709623757 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc180107b90 con 0x7fc180100da0 2023-12-01T01:16:32.926 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.922+0000 7fc17dffb700 1 -- 172.21.15.77:0/709623757 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fc174022600 con 0x7fc180100da0 2023-12-01T01:16:32.926 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.923+0000 7fc17dffb700 1 --2- 172.21.15.77:0/709623757 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc1700a05f0 0x7fc1700a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:32.926 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.923+0000 7fc17ffff700 1 --2- 172.21.15.77:0/709623757 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc1700a05f0 0x7fc1700a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:32.926 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.923+0000 7fc17dffb700 1 -- 172.21.15.77:0/709623757 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc1740ccff0 con 0x7fc180100da0 2023-12-01T01:16:32.926 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.923+0000 7fc17ffff700 1 --2- 172.21.15.77:0/709623757 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc1700a05f0 0x7fc1700a2ab0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc16c005fd0 tx=0x7fc16c005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:32.928 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:32.928+0000 7fc17dffb700 1 -- 172.21.15.77:0/709623757 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc174096b70 con 0x7fc180100da0 2023-12-01T01:16:33.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:32 smithi203 ceph-mon[103581]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:16:33.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:32 smithi203 ceph-mon[103581]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:16:33.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:32 smithi203 ceph-mon[103581]: pgmap v256: 97 pgs: 97 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s wr, 14 op/s 2023-12-01T01:16:33.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:32 smithi203 ceph-mon[103581]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:16:33.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:32 smithi203 ceph-mon[103581]: mds.? [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] up:active 2023-12-01T01:16:33.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:32 smithi203 ceph-mon[103581]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-12-01T01:16:33.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:32 smithi203 ceph-mon[103581]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:16:33.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:32 smithi137 ceph-mon[103312]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:16:33.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:32 smithi137 ceph-mon[103312]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:16:33.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:32 smithi137 ceph-mon[103312]: pgmap v256: 97 pgs: 97 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s wr, 14 op/s 2023-12-01T01:16:33.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:32 smithi137 ceph-mon[103312]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:16:33.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:32 smithi137 ceph-mon[103312]: mds.? [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] up:active 2023-12-01T01:16:33.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:32 smithi137 ceph-mon[103312]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2023-12-01T01:16:33.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:32 smithi137 ceph-mon[103312]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:16:33.170 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.168+0000 7fc18706a700 1 -- 172.21.15.77:0/709623757 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc18002cee0 con 0x7fc180100da0 2023-12-01T01:16:33.170 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.169+0000 7fc17dffb700 1 -- 172.21.15.77:0/709623757 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10838 (secure 0 0 0) 0x7fc174027020 con 0x7fc180100da0 2023-12-01T01:16:33.170 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:16:33.171 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1435145620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1435145620},{"type":"v1","addr":"172.21.15.77:6835","nonce":1435145620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/841838908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":841838908},{"type":"v1","addr":"172.21.15.77:6837","nonce":841838908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-01T01:16:25.345276+0000","modified":"2023-12-01T01:16:32.596617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24496,"mds_2":24475,"mds_3":24473,"mds_4":14649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/3726913446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3726913446},{"type":"v1","addr":"172.21.15.77:6839","nonce":3726913446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14649":{"gid":14649,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":5,"addr":"172.21.15.77:6841/3023184965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3023184965},{"type":"v1","addr":"172.21.15.77:6841","nonce":3023184965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/2249638445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2249638445},{"type":"v1","addr":"172.21.15.137:6833","nonce":2249638445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6833/1201599774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1201599774},{"type":"v1","addr":"172.21.15.203:6833","nonce":1201599774}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3935354878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3935354878},{"type":"v1","addr":"172.21.15.137:6835","nonce":3935354878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3237538176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3237538176},{"type":"v1","addr":"172.21.15.203:6835","nonce":3237538176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3518881867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3518881867},{"type":"v1","addr":"172.21.15.137:6837","nonce":3518881867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.203:6837/2887080977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2887080977},{"type":"v1","addr":"172.21.15.203:6837","nonce":2887080977}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.203:6839/2888221613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2888221613},{"type":"v1","addr":"172.21.15.203:6839","nonce":2888221613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/1099966660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1099966660},{"type":"v1","addr":"172.21.15.137:6839","nonce":1099966660}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-01T01:16:33.172 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.171+0000 7fc18706a700 1 -- 172.21.15.77:0/709623757 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc1700a05f0 msgr2=0x7fc1700a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:33.173 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.171+0000 7fc18706a700 1 --2- 172.21.15.77:0/709623757 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc1700a05f0 0x7fc1700a2ab0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc16c005fd0 tx=0x7fc16c005e30 comp rx=0 tx=0).stop 2023-12-01T01:16:33.173 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.171+0000 7fc18706a700 1 -- 172.21.15.77:0/709623757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc180100da0 msgr2=0x7fc180074b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:33.173 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.171+0000 7fc18706a700 1 --2- 172.21.15.77:0/709623757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc180100da0 0x7fc180074b40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc1740036f0 tx=0x7fc1740057d0 comp rx=0 tx=0).stop 2023-12-01T01:16:33.173 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.171+0000 7fc18706a700 1 -- 172.21.15.77:0/709623757 shutdown_connections 2023-12-01T01:16:33.173 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.171+0000 7fc18706a700 1 --2- 172.21.15.77:0/709623757 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc1700a05f0 0x7fc1700a2ab0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:33.173 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.171+0000 7fc18706a700 1 --2- 172.21.15.77:0/709623757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc180102930 0x7fc1800736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:33.173 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.171+0000 7fc18706a700 1 --2- 172.21.15.77:0/709623757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc180100da0 0x7fc180074b40 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:33.173 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.171+0000 7fc18706a700 1 --2- 172.21.15.77:0/709623757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc180101f90 0x7fc180073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:33.173 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.171+0000 7fc18706a700 1 -- 172.21.15.77:0/709623757 >> 172.21.15.77:0/709623757 conn(0x7fc1800fc2d0 msgr2=0x7fc1800fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:33.173 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.171+0000 7fc18706a700 1 -- 172.21.15.77:0/709623757 shutdown_connections 2023-12-01T01:16:33.173 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.171+0000 7fc18706a700 1 -- 172.21.15.77:0/709623757 wait complete. 2023-12-01T01:16:33.174 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 27 2023-12-01T01:16:33.185 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:16:33.305 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.304+0000 7eff21f14700 1 -- 172.21.15.77:0/3430162847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff1c100d90 msgr2=0x7eff1c1011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:33.306 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.304+0000 7eff21f14700 1 --2- 172.21.15.77:0/3430162847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff1c100d90 0x7eff1c1011b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7eff100099c0 tx=0x7eff10009cd0 comp rx=0 tx=0).stop 2023-12-01T01:16:33.306 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.304+0000 7eff21f14700 1 -- 172.21.15.77:0/3430162847 shutdown_connections 2023-12-01T01:16:33.306 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.304+0000 7eff21f14700 1 --2- 172.21.15.77:0/3430162847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff1c102920 0x7eff1c1091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:33.306 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.304+0000 7eff21f14700 1 --2- 172.21.15.77:0/3430162847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff1c100d90 0x7eff1c1011b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:33.306 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.304+0000 7eff21f14700 1 --2- 172.21.15.77:0/3430162847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff1c101f80 0x7eff1c1023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:33.306 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.304+0000 7eff21f14700 1 -- 172.21.15.77:0/3430162847 >> 172.21.15.77:0/3430162847 conn(0x7eff1c0fc2e0 msgr2=0x7eff1c0fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:33.306 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.304+0000 7eff21f14700 1 -- 172.21.15.77:0/3430162847 shutdown_connections 2023-12-01T01:16:33.306 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.304+0000 7eff21f14700 1 -- 172.21.15.77:0/3430162847 wait complete. 2023-12-01T01:16:33.306 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.304+0000 7eff21f14700 1 Processor -- start 2023-12-01T01:16:33.306 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.305+0000 7eff21f14700 1 -- start start 2023-12-01T01:16:33.308 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.305+0000 7eff21f14700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff1c100d90 0x7eff1c074b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:33.308 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.305+0000 7eff21f14700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff1c101f80 0x7eff1c073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:33.309 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.305+0000 7eff21f14700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff1c102920 0x7eff1c0736c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:33.309 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.305+0000 7eff21f14700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1c073e60 con 0x7eff1c101f80 2023-12-01T01:16:33.309 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.305+0000 7eff21f14700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1c073fd0 con 0x7eff1c102920 2023-12-01T01:16:33.309 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.305+0000 7eff21f14700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1c1a0aa0 con 0x7eff1c100d90 2023-12-01T01:16:33.309 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.305+0000 7eff1affd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff1c101f80 0x7eff1c073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:33.309 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.305+0000 7eff1affd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff1c101f80 0x7eff1c073180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57662/0 (socket says 172.21.15.77:57662) 2023-12-01T01:16:33.309 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.305+0000 7eff1affd700 1 -- 172.21.15.77:0/3602740592 learned_addr learned my addr 172.21.15.77:0/3602740592 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:33.309 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.305+0000 7eff1b7fe700 1 --2- 172.21.15.77:0/3602740592 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff1c100d90 0x7eff1c074b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:33.309 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.305+0000 7eff1bfff700 1 --2- 172.21.15.77:0/3602740592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff1c102920 0x7eff1c0736c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:33.309 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.305+0000 7eff1affd700 1 -- 172.21.15.77:0/3602740592 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff1c100d90 msgr2=0x7eff1c074b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:33.309 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.305+0000 7eff1affd700 1 --2- 172.21.15.77:0/3602740592 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff1c100d90 0x7eff1c074b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:33.309 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.305+0000 7eff1affd700 1 -- 172.21.15.77:0/3602740592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff1c102920 msgr2=0x7eff1c0736c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:33.309 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.305+0000 7eff1affd700 1 --2- 172.21.15.77:0/3602740592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff1c102920 0x7eff1c0736c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:33.309 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.305+0000 7eff1affd700 1 -- 172.21.15.77:0/3602740592 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff100095f0 con 0x7eff1c101f80 2023-12-01T01:16:33.310 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.305+0000 7eff1bfff700 1 --2- 172.21.15.77:0/3602740592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff1c102920 0x7eff1c0736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:16:33.310 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.305+0000 7eff1affd700 1 --2- 172.21.15.77:0/3602740592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff1c101f80 0x7eff1c073180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7eff0c00e9a0 tx=0x7eff0c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:33.310 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.305+0000 7eff18ff9700 1 -- 172.21.15.77:0/3602740592 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0c009790 con 0x7eff1c101f80 2023-12-01T01:16:33.310 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.305+0000 7eff18ff9700 1 -- 172.21.15.77:0/3602740592 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff0c00cbd0 con 0x7eff1c101f80 2023-12-01T01:16:33.310 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.305+0000 7eff21f14700 1 -- 172.21.15.77:0/3602740592 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff1c1a0d80 con 0x7eff1c101f80 2023-12-01T01:16:33.310 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.306+0000 7eff18ff9700 1 -- 172.21.15.77:0/3602740592 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0c018680 con 0x7eff1c101f80 2023-12-01T01:16:33.310 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.306+0000 7eff21f14700 1 -- 172.21.15.77:0/3602740592 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff1c1a1230 con 0x7eff1c101f80 2023-12-01T01:16:33.310 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.306+0000 7eff21f14700 1 -- 172.21.15.77:0/3602740592 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff1c10c5a0 con 0x7eff1c101f80 2023-12-01T01:16:33.310 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.307+0000 7eff18ff9700 1 -- 172.21.15.77:0/3602740592 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7eff0c00cd40 con 0x7eff1c101f80 2023-12-01T01:16:33.311 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.308+0000 7eff18ff9700 1 --2- 172.21.15.77:0/3602740592 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7eff040a05f0 0x7eff040a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:33.311 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.308+0000 7eff1b7fe700 1 --2- 172.21.15.77:0/3602740592 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7eff040a05f0 0x7eff040a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:33.311 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.308+0000 7eff18ff9700 1 -- 172.21.15.77:0/3602740592 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7eff0c014070 con 0x7eff1c101f80 2023-12-01T01:16:33.311 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.308+0000 7eff1b7fe700 1 --2- 172.21.15.77:0/3602740592 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7eff040a05f0 0x7eff040a2ab0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7eff10009990 tx=0x7eff10005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:33.312 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.312+0000 7eff18ff9700 1 -- 172.21.15.77:0/3602740592 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7eff0c095b10 con 0x7eff1c101f80 2023-12-01T01:16:33.511 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.510+0000 7eff21f14700 1 -- 172.21.15.77:0/3602740592 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff1c107cd0 con 0x7eff1c101f80 2023-12-01T01:16:33.513 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.512+0000 7eff18ff9700 1 -- 172.21.15.77:0/3602740592 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22131 (secure 0 0 0) 0x7eff0c099130 con 0x7eff1c101f80 2023-12-01T01:16:33.513 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:16:33.513 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":82,"fsid":"2a06e29a-8fe6-11ee-95a2-87774f69a715","created":"2023-12-01T01:09:30.210398+0000","modified":"2023-12-01T01:16:26.346791+0000","last_up_change":"2023-12-01T01:15:44.586092+0000","last_in_change":"2023-12-01T01:15:26.305439+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:12:30.514229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:16:23.568860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-01T01:16:24.531137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fc651e62-a103-4ab8-a76d-d3708ea6aeec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6803","nonce":2244799094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6805","nonce":2244799094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6809","nonce":2244799094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6807","nonce":2244799094}]},"public_addr":"172.21.15.77:6803/2244799094","cluster_addr":"172.21.15.77:6805/2244799094","heartbeat_back_addr":"172.21.15.77:6809/2244799094","heartbeat_front_addr":"172.21.15.77:6807/2244799094","state":["exists","up"]},{"osd":1,"uuid":"0adf8c54-49ec-49a0-8bf1-af1f20e60729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6801","nonce":224194890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6803","nonce":224194890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6807","nonce":224194890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6805","nonce":224194890}]},"public_addr":"172.21.15.137:6801/224194890","cluster_addr":"172.21.15.137:6803/224194890","heartbeat_back_addr":"172.21.15.137:6807/224194890","heartbeat_front_addr":"172.21.15.137:6805/224194890","state":["exists","up"]},{"osd":2,"uuid":"5a69802c-dede-4544-9a12-436f5c1c0038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6801","nonce":4078459110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6803","nonce":4078459110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6807","nonce":4078459110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6805","nonce":4078459110}]},"public_addr":"172.21.15.203:6801/4078459110","cluster_addr":"172.21.15.203:6803/4078459110","heartbeat_back_addr":"172.21.15.203:6807/4078459110","heartbeat_front_addr":"172.21.15.203:6805/4078459110","state":["exists","up"]},{"osd":3,"uuid":"ad325a97-580b-4c99-bf12-cbb0d92a6d0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6811","nonce":1454440477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6813","nonce":1454440477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6817","nonce":1454440477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6815","nonce":1454440477}]},"public_addr":"172.21.15.77:6811/1454440477","cluster_addr":"172.21.15.77:6813/1454440477","heartbeat_back_addr":"172.21.15.77:6817/1454440477","heartbeat_front_addr":"172.21.15.77:6815/1454440477","state":["exists","up"]},{"osd":4,"uuid":"1e3ac1ba-98bb-4b8b-8e26-50064c66bf2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6809","nonce":1103241451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6811","nonce":1103241451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6815","nonce":1103241451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6813","nonce":1103241451}]},"public_addr":"172.21.15.137:6809/1103241451","cluster_addr":"172.21.15.137:6811/1103241451","heartbeat_back_addr":"172.21.15.137:6815/1103241451","heartbeat_front_addr":"172.21.15.137:6813/1103241451","state":["exists","up"]},{"osd":5,"uuid":"eb32c016-abeb-44ab-abf0-19dd09235736","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6809","nonce":2648569482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6811","nonce":2648569482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6815","nonce":2648569482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6813","nonce":2648569482}]},"public_addr":"172.21.15.203:6809/2648569482","cluster_addr":"172.21.15.203:6811/2648569482","heartbeat_back_addr":"172.21.15.203:6815/2648569482","heartbeat_front_addr":"172.21.15.203:6813/2648569482","state":["exists","up"]},{"osd":6,"uuid":"0e3fa97e-2ca6-418c-9e8d-a6fe3c56adb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6819","nonce":2463414651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6821","nonce":2463414651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6825","nonce":2463414651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6823","nonce":2463414651}]},"public_addr":"172.21.15.77:6819/2463414651","cluster_addr":"172.21.15.77:6821/2463414651","heartbeat_back_addr":"172.21.15.77:6825/2463414651","heartbeat_front_addr":"172.21.15.77:6823/2463414651","state":["exists","up"]},{"osd":7,"uuid":"d7398c01-0f98-4138-8a49-c3ff0aae7e88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6817","nonce":3108330766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6819","nonce":3108330766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6823","nonce":3108330766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6821","nonce":3108330766}]},"public_addr":"172.21.15.137:6817/3108330766","cluster_addr":"172.21.15.137:6819/3108330766","heartbeat_back_addr":"172.21.15.137:6823/3108330766","heartbeat_front_addr":"172.21.15.137:6821/3108330766","state":["exists","up"]},{"osd":8,"uuid":"0c09050d-3e43-410c-8cd7-bcb891af56b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6817","nonce":1161158776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6819","nonce":1161158776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6823","nonce":1161158776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6821","nonce":1161158776}]},"public_addr":"172.21.15.203:6817/1161158776","cluster_addr":"172.21.15.203:6819/1161158776","heartbeat_back_addr":"172.21.15.203:6823/1161158776","heartbeat_front_addr":"172.21.15.203:6821/1161158776","state":["exists","up"]},{"osd":9,"uuid":"527ecb49-95ec-48bd-9872-3b07fae5c5ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6827","nonce":2409880685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6829","nonce":2409880685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6833","nonce":2409880685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6831","nonce":2409880685}]},"public_addr":"172.21.15.77:6827/2409880685","cluster_addr":"172.21.15.77:6829/2409880685","heartbeat_back_addr":"172.21.15.77:6833/2409880685","heartbeat_front_addr":"172.21.15.77:6831/2409880685","state":["exists","up"]},{"osd":10,"uuid":"1680bb3e-a3c8-427b-af68-e2214f32bed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6825","nonce":374920146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6827","nonce":374920146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6831","nonce":374920146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6829","nonce":374920146}]},"public_addr":"172.21.15.137:6825/374920146","cluster_addr":"172.21.15.137:6827/374920146","heartbeat_back_addr":"172.21.15.137:6831/374920146","heartbeat_front_addr":"172.21.15.137:6829/374920146","state":["exists","up"]},{"osd":11,"uuid":"8a533466-023d-4803-ae36-9a9f1715e49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6825","nonce":2161588962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6827","nonce":2161588962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6831","nonce":2161588962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6829","nonce":2161588962}]},"public_addr":"172.21.15.203:6825/2161588962","cluster_addr":"172.21.15.203:6827/2161588962","heartbeat_back_addr":"172.21.15.203:6831/2161588962","heartbeat_front_addr":"172.21.15.203:6829/2161588962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:11:45.420509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:06.210115+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:28.191307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:49.594116+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:11.261689+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:33.376541+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:54.968336+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:16.775310+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:38.733792+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:59.456699+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:15:20.924059+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:15:43.094773+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/2260035254":"2023-12-02T01:10:24.191835+0000","172.21.15.77:6800/1456621747":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/219838828":"2023-12-02T01:10:24.191835+0000","172.21.15.77:6801/1456621747":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/2937149266":"2023-12-02T01:10:02.232948+0000","172.21.15.77:0/807520018":"2023-12-02T01:10:02.232948+0000","172.21.15.77:6801/1661788026":"2023-12-02T01:10:02.232948+0000","172.21.15.77:6800/1661788026":"2023-12-02T01:10:02.232948+0000","172.21.15.77:0/3551282473":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/1202222558":"2023-12-02T01:10:02.232948+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:16:33.516 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.514+0000 7eff21f14700 1 -- 172.21.15.77:0/3602740592 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7eff040a05f0 msgr2=0x7eff040a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:33.516 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.514+0000 7eff21f14700 1 --2- 172.21.15.77:0/3602740592 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7eff040a05f0 0x7eff040a2ab0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7eff10009990 tx=0x7eff10005fb0 comp rx=0 tx=0).stop 2023-12-01T01:16:33.516 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.514+0000 7eff21f14700 1 -- 172.21.15.77:0/3602740592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff1c101f80 msgr2=0x7eff1c073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:33.516 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.514+0000 7eff21f14700 1 --2- 172.21.15.77:0/3602740592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff1c101f80 0x7eff1c073180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7eff0c00e9a0 tx=0x7eff0c00ecb0 comp rx=0 tx=0).stop 2023-12-01T01:16:33.516 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.514+0000 7eff21f14700 1 -- 172.21.15.77:0/3602740592 shutdown_connections 2023-12-01T01:16:33.516 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.514+0000 7eff21f14700 1 --2- 172.21.15.77:0/3602740592 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7eff040a05f0 0x7eff040a2ab0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:33.516 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.514+0000 7eff21f14700 1 --2- 172.21.15.77:0/3602740592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff1c102920 0x7eff1c0736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:33.516 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.514+0000 7eff21f14700 1 --2- 172.21.15.77:0/3602740592 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff1c100d90 0x7eff1c074b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:33.516 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.514+0000 7eff21f14700 1 --2- 172.21.15.77:0/3602740592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff1c101f80 0x7eff1c073180 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:33.517 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.514+0000 7eff21f14700 1 -- 172.21.15.77:0/3602740592 >> 172.21.15.77:0/3602740592 conn(0x7eff1c0fc2e0 msgr2=0x7eff1c0fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:33.517 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.514+0000 7eff21f14700 1 -- 172.21.15.77:0/3602740592 shutdown_connections 2023-12-01T01:16:33.517 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.514+0000 7eff21f14700 1 -- 172.21.15.77:0/3602740592 wait complete. 2023-12-01T01:16:33.528 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-01T01:16:33.528 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T01:16:33.653 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.650+0000 7fed2ab4a700 1 -- 172.21.15.77:0/2698984492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed240ffc70 msgr2=0x7fed24100090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:33.654 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.650+0000 7fed2ab4a700 1 --2- 172.21.15.77:0/2698984492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed240ffc70 0x7fed24100090 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fed180099c0 tx=0x7fed18009cd0 comp rx=0 tx=0).stop 2023-12-01T01:16:33.654 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.650+0000 7fed2ab4a700 1 -- 172.21.15.77:0/2698984492 shutdown_connections 2023-12-01T01:16:33.654 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.650+0000 7fed2ab4a700 1 --2- 172.21.15.77:0/2698984492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed24101800 0x7fed241080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:33.654 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.650+0000 7fed2ab4a700 1 --2- 172.21.15.77:0/2698984492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed240ffc70 0x7fed24100090 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:33.654 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.650+0000 7fed2ab4a700 1 --2- 172.21.15.77:0/2698984492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed24100e60 0x7fed241012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:33.654 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.650+0000 7fed2ab4a700 1 -- 172.21.15.77:0/2698984492 >> 172.21.15.77:0/2698984492 conn(0x7fed240fb1c0 msgr2=0x7fed240fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:33.654 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.650+0000 7fed2ab4a700 1 -- 172.21.15.77:0/2698984492 shutdown_connections 2023-12-01T01:16:33.654 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.650+0000 7fed2ab4a700 1 -- 172.21.15.77:0/2698984492 wait complete. 2023-12-01T01:16:33.654 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.650+0000 7fed2ab4a700 1 Processor -- start 2023-12-01T01:16:33.654 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.650+0000 7fed2ab4a700 1 -- start start 2023-12-01T01:16:33.654 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.650+0000 7fed2ab4a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed240ffc70 0x7fed24196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:33.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.650+0000 7fed2ab4a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed24100e60 0x7fed24196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:33.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.651+0000 7fed2ab4a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed24101800 0x7fed2419dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:33.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.651+0000 7fed2ab4a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2419e5b0 con 0x7fed24100e60 2023-12-01T01:16:33.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.651+0000 7fed2ab4a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2419e6f0 con 0x7fed24101800 2023-12-01T01:16:33.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.651+0000 7fed2ab4a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2419e880 con 0x7fed240ffc70 2023-12-01T01:16:33.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.651+0000 7fed23fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed24100e60 0x7fed24196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:33.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.651+0000 7fed23fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed24100e60 0x7fed24196f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57676/0 (socket says 172.21.15.77:57676) 2023-12-01T01:16:33.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.651+0000 7fed23fff700 1 -- 172.21.15.77:0/880988240 learned_addr learned my addr 172.21.15.77:0/880988240 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:33.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.651+0000 7fed288e6700 1 --2- 172.21.15.77:0/880988240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed240ffc70 0x7fed24196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:33.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.651+0000 7fed290e7700 1 --2- 172.21.15.77:0/880988240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed24101800 0x7fed2419dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:33.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.651+0000 7fed23fff700 1 -- 172.21.15.77:0/880988240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed240ffc70 msgr2=0x7fed24196a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:33.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.651+0000 7fed23fff700 1 --2- 172.21.15.77:0/880988240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed240ffc70 0x7fed24196a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:33.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.651+0000 7fed23fff700 1 -- 172.21.15.77:0/880988240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed24101800 msgr2=0x7fed2419dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:33.656 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.651+0000 7fed23fff700 1 --2- 172.21.15.77:0/880988240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed24101800 0x7fed2419dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:33.656 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.651+0000 7fed23fff700 1 -- 172.21.15.77:0/880988240 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed180095f0 con 0x7fed24100e60 2023-12-01T01:16:33.656 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.651+0000 7fed288e6700 1 --2- 172.21.15.77:0/880988240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed240ffc70 0x7fed24196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:16:33.656 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.651+0000 7fed290e7700 1 --2- 172.21.15.77:0/880988240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed24101800 0x7fed2419dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:16:33.656 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.651+0000 7fed23fff700 1 --2- 172.21.15.77:0/880988240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed24100e60 0x7fed24196f70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fed1000c9b0 tx=0x7fed1000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:33.656 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.651+0000 7fed21ffb700 1 -- 172.21.15.77:0/880988240 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed10007b00 con 0x7fed24100e60 2023-12-01T01:16:33.656 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.651+0000 7fed21ffb700 1 -- 172.21.15.77:0/880988240 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed10007c60 con 0x7fed24100e60 2023-12-01T01:16:33.656 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.651+0000 7fed2ab4a700 1 -- 172.21.15.77:0/880988240 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed2419eb60 con 0x7fed24100e60 2023-12-01T01:16:33.656 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.651+0000 7fed21ffb700 1 -- 172.21.15.77:0/880988240 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed10018660 con 0x7fed24100e60 2023-12-01T01:16:33.656 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.651+0000 7fed2ab4a700 1 -- 172.21.15.77:0/880988240 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed2419f0c0 con 0x7fed24100e60 2023-12-01T01:16:33.656 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.652+0000 7fed2ab4a700 1 -- 172.21.15.77:0/880988240 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed2410c690 con 0x7fed24100e60 2023-12-01T01:16:33.657 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.653+0000 7fed21ffb700 1 -- 172.21.15.77:0/880988240 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fed10018880 con 0x7fed24100e60 2023-12-01T01:16:33.657 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.654+0000 7fed21ffb700 1 --2- 172.21.15.77:0/880988240 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fed140a06f0 0x7fed140a2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:33.657 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.654+0000 7fed288e6700 1 --2- 172.21.15.77:0/880988240 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fed140a06f0 0x7fed140a2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:33.657 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.654+0000 7fed21ffb700 1 -- 172.21.15.77:0/880988240 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fed100cba70 con 0x7fed24100e60 2023-12-01T01:16:33.657 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.654+0000 7fed288e6700 1 --2- 172.21.15.77:0/880988240 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fed140a06f0 0x7fed140a2bb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fed1800b580 tx=0x7fed18005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:33.659 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.658+0000 7fed21ffb700 1 -- 172.21.15.77:0/880988240 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fed10095560 con 0x7fed24100e60 2023-12-01T01:16:33.846 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.845+0000 7fed2ab4a700 1 -- 172.21.15.77:0/880988240 --> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fed240611d0 con 0x7fed140a06f0 2023-12-01T01:16:33.857 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.855+0000 7fed21ffb700 1 -- 172.21.15.77:0/880988240 <== mgr.14152 v2:172.21.15.77:6800/3838941679 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+336051 (secure 0 0 0) 0x7fed240611d0 con 0x7fed140a06f0 2023-12-01T01:16:33.861 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:16:33.864 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.859+0000 7fed2ab4a700 1 -- 172.21.15.77:0/880988240 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fed140a06f0 msgr2=0x7fed140a2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:33.864 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.859+0000 7fed2ab4a700 1 --2- 172.21.15.77:0/880988240 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fed140a06f0 0x7fed140a2bb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fed1800b580 tx=0x7fed18005fd0 comp rx=0 tx=0).stop 2023-12-01T01:16:33.864 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.859+0000 7fed2ab4a700 1 -- 172.21.15.77:0/880988240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed24100e60 msgr2=0x7fed24196f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:33.864 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.859+0000 7fed2ab4a700 1 --2- 172.21.15.77:0/880988240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed24100e60 0x7fed24196f70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fed1000c9b0 tx=0x7fed1000cd70 comp rx=0 tx=0).stop 2023-12-01T01:16:33.864 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.859+0000 7fed2ab4a700 1 -- 172.21.15.77:0/880988240 shutdown_connections 2023-12-01T01:16:33.864 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.859+0000 7fed2ab4a700 1 --2- 172.21.15.77:0/880988240 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fed140a06f0 0x7fed140a2bb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:33.864 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.859+0000 7fed2ab4a700 1 --2- 172.21.15.77:0/880988240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed24101800 0x7fed2419dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:33.864 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.859+0000 7fed2ab4a700 1 --2- 172.21.15.77:0/880988240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed240ffc70 0x7fed24196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:33.864 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.859+0000 7fed2ab4a700 1 --2- 172.21.15.77:0/880988240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed24100e60 0x7fed24196f70 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:33.864 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.859+0000 7fed2ab4a700 1 -- 172.21.15.77:0/880988240 >> 172.21.15.77:0/880988240 conn(0x7fed240fb1c0 msgr2=0x7fed240fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:33.864 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.859+0000 7fed2ab4a700 1 -- 172.21.15.77:0/880988240 shutdown_connections 2023-12-01T01:16:33.864 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:33.859+0000 7fed2ab4a700 1 -- 172.21.15.77:0/880988240 wait complete. 2023-12-01T01:16:33.864 INFO:teuthology.orchestra.run.smithi077.stderr:dumped all 2023-12-01T01:16:33.871 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:33 smithi077 ceph-mon[98867]: mgrmap e17: x(active, since 6m), standbys: y, z 2023-12-01T01:16:33.872 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:33 smithi077 ceph-mon[98867]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:16:33.872 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:33 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/709623757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:16:33.872 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:33 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/3602740592' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:16:33.873 INFO:teuthology.orchestra.run.smithi077.stdout:{"pg_ready":true,"pg_map":{"version":256,"stamp":"2023-12-01T01:16:32.267694+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594692,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":291,"num_write_kb":2966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":260,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335636,"kb_used_data":12212,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606188,"statfs":{"total":1151940427776,"available":1151596736512,"internally_reserved":0,"allocated":12505088,"data_stored":6845892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4324,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.341501"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.592825+0000","last_change":"2023-12-01T01:16:25.373817+0000","last_active":"2023-12-01T01:16:25.592825+0000","last_peered":"2023-12-01T01:16:25.592825+0000","last_clean":"2023-12-01T01:16:25.592825+0000","last_became_active":"2023-12-01T01:16:25.373154+0000","last_became_peered":"2023-12-01T01:16:25.373154+0000","last_unstale":"2023-12-01T01:16:25.592825+0000","last_undegraded":"2023-12-01T01:16:25.592825+0000","last_fullsized":"2023-12-01T01:16:25.592825+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:33:43.284106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.324984+0000","last_change":"2023-12-01T01:16:25.364855+0000","last_active":"2023-12-01T01:16:27.324984+0000","last_peered":"2023-12-01T01:16:27.324984+0000","last_clean":"2023-12-01T01:16:27.324984+0000","last_became_active":"2023-12-01T01:16:25.364536+0000","last_became_peered":"2023-12-01T01:16:25.364536+0000","last_unstale":"2023-12-01T01:16:27.324984+0000","last_undegraded":"2023-12-01T01:16:27.324984+0000","last_fullsized":"2023-12-01T01:16:27.324984+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:45:23.471557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.324702+0000","last_change":"2023-12-01T01:16:25.372443+0000","last_active":"2023-12-01T01:16:27.324702+0000","last_peered":"2023-12-01T01:16:27.324702+0000","last_clean":"2023-12-01T01:16:27.324702+0000","last_became_active":"2023-12-01T01:16:25.372133+0000","last_became_peered":"2023-12-01T01:16:25.372133+0000","last_unstale":"2023-12-01T01:16:27.324702+0000","last_undegraded":"2023-12-01T01:16:27.324702+0000","last_fullsized":"2023-12-01T01:16:27.324702+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:54:19.439408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.554816+0000","last_change":"2023-12-01T01:16:25.365857+0000","last_active":"2023-12-01T01:16:26.554816+0000","last_peered":"2023-12-01T01:16:26.554816+0000","last_clean":"2023-12-01T01:16:26.554816+0000","last_became_active":"2023-12-01T01:16:25.364992+0000","last_became_peered":"2023-12-01T01:16:25.364992+0000","last_unstale":"2023-12-01T01:16:26.554816+0000","last_undegraded":"2023-12-01T01:16:26.554816+0000","last_fullsized":"2023-12-01T01:16:26.554816+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:01:11.064587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.071460+0000","last_change":"2023-12-01T01:16:25.364081+0000","last_active":"2023-12-01T01:16:26.071460+0000","last_peered":"2023-12-01T01:16:26.071460+0000","last_clean":"2023-12-01T01:16:26.071460+0000","last_became_active":"2023-12-01T01:16:25.363187+0000","last_became_peered":"2023-12-01T01:16:25.363187+0000","last_unstale":"2023-12-01T01:16:26.071460+0000","last_undegraded":"2023-12-01T01:16:26.071460+0000","last_fullsized":"2023-12-01T01:16:26.071460+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:05:42.852128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325515+0000","last_change":"2023-12-01T01:16:25.366104+0000","last_active":"2023-12-01T01:16:27.325515+0000","last_peered":"2023-12-01T01:16:27.325515+0000","last_clean":"2023-12-01T01:16:27.325515+0000","last_became_active":"2023-12-01T01:16:25.365778+0000","last_became_peered":"2023-12-01T01:16:25.365778+0000","last_unstale":"2023-12-01T01:16:27.325515+0000","last_undegraded":"2023-12-01T01:16:27.325515+0000","last_fullsized":"2023-12-01T01:16:27.325515+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:30:09.485609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.349881+0000","last_change":"2023-12-01T01:16:25.371354+0000","last_active":"2023-12-01T01:16:26.349881+0000","last_peered":"2023-12-01T01:16:26.349881+0000","last_clean":"2023-12-01T01:16:26.349881+0000","last_became_active":"2023-12-01T01:16:25.370955+0000","last_became_peered":"2023-12-01T01:16:25.370955+0000","last_unstale":"2023-12-01T01:16:26.349881+0000","last_undegraded":"2023-12-01T01:16:26.349881+0000","last_fullsized":"2023-12-01T01:16:26.349881+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:38:53.508285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.571448+0000","last_change":"2023-12-01T01:16:25.368187+0000","last_active":"2023-12-01T01:16:26.571448+0000","last_peered":"2023-12-01T01:16:26.571448+0000","last_clean":"2023-12-01T01:16:26.571448+0000","last_became_active":"2023-12-01T01:16:25.367854+0000","last_became_peered":"2023-12-01T01:16:25.367854+0000","last_unstale":"2023-12-01T01:16:26.571448+0000","last_undegraded":"2023-12-01T01:16:26.571448+0000","last_fullsized":"2023-12-01T01:16:26.571448+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:22:35.159185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325090+0000","last_change":"2023-12-01T01:16:25.372829+0000","last_active":"2023-12-01T01:16:27.325090+0000","last_peered":"2023-12-01T01:16:27.325090+0000","last_clean":"2023-12-01T01:16:27.325090+0000","last_became_active":"2023-12-01T01:16:25.372469+0000","last_became_peered":"2023-12-01T01:16:25.372469+0000","last_unstale":"2023-12-01T01:16:27.325090+0000","last_undegraded":"2023-12-01T01:16:27.325090+0000","last_fullsized":"2023-12-01T01:16:27.325090+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:25:10.300138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325829+0000","last_change":"2023-12-01T01:16:25.379135+0000","last_active":"2023-12-01T01:16:27.325829+0000","last_peered":"2023-12-01T01:16:27.325829+0000","last_clean":"2023-12-01T01:16:27.325829+0000","last_became_active":"2023-12-01T01:16:25.378852+0000","last_became_peered":"2023-12-01T01:16:25.378852+0000","last_unstale":"2023-12-01T01:16:27.325829+0000","last_undegraded":"2023-12-01T01:16:27.325829+0000","last_fullsized":"2023-12-01T01:16:27.325829+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:42:16.803389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.350179+0000","last_change":"2023-12-01T01:16:25.367602+0000","last_active":"2023-12-01T01:16:26.350179+0000","last_peered":"2023-12-01T01:16:26.350179+0000","last_clean":"2023-12-01T01:16:26.350179+0000","last_became_active":"2023-12-01T01:16:25.366565+0000","last_became_peered":"2023-12-01T01:16:25.366565+0000","last_unstale":"2023-12-01T01:16:26.350179+0000","last_undegraded":"2023-12-01T01:16:26.350179+0000","last_fullsized":"2023-12-01T01:16:26.350179+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:28:52.576838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.554806+0000","last_change":"2023-12-01T01:16:25.375004+0000","last_active":"2023-12-01T01:16:26.554806+0000","last_peered":"2023-12-01T01:16:26.554806+0000","last_clean":"2023-12-01T01:16:26.554806+0000","last_became_active":"2023-12-01T01:16:25.374320+0000","last_became_peered":"2023-12-01T01:16:25.374320+0000","last_unstale":"2023-12-01T01:16:26.554806+0000","last_undegraded":"2023-12-01T01:16:26.554806+0000","last_fullsized":"2023-12-01T01:16:26.554806+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:40:49.482061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.352492+0000","last_change":"2023-12-01T01:16:25.367858+0000","last_active":"2023-12-01T01:16:26.352492+0000","last_peered":"2023-12-01T01:16:26.352492+0000","last_clean":"2023-12-01T01:16:26.352492+0000","last_became_active":"2023-12-01T01:16:25.367560+0000","last_became_peered":"2023-12-01T01:16:25.367560+0000","last_unstale":"2023-12-01T01:16:26.352492+0000","last_undegraded":"2023-12-01T01:16:26.352492+0000","last_fullsized":"2023-12-01T01:16:26.352492+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:33:21.048672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.324783+0000","last_change":"2023-12-01T01:16:25.375654+0000","last_active":"2023-12-01T01:16:27.324783+0000","last_peered":"2023-12-01T01:16:27.324783+0000","last_clean":"2023-12-01T01:16:27.324783+0000","last_became_active":"2023-12-01T01:16:25.375379+0000","last_became_peered":"2023-12-01T01:16:25.375379+0000","last_unstale":"2023-12-01T01:16:27.324783+0000","last_undegraded":"2023-12-01T01:16:27.324783+0000","last_fullsized":"2023-12-01T01:16:27.324783+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:33:42.335365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325759+0000","last_change":"2023-12-01T01:16:25.372546+0000","last_active":"2023-12-01T01:16:27.325759+0000","last_peered":"2023-12-01T01:16:27.325759+0000","last_clean":"2023-12-01T01:16:27.325759+0000","last_became_active":"2023-12-01T01:16:25.372327+0000","last_became_peered":"2023-12-01T01:16:25.372327+0000","last_unstale":"2023-12-01T01:16:27.325759+0000","last_undegraded":"2023-12-01T01:16:27.325759+0000","last_fullsized":"2023-12-01T01:16:27.325759+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:12:34.171834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.324599+0000","last_change":"2023-12-01T01:16:25.366449+0000","last_active":"2023-12-01T01:16:27.324599+0000","last_peered":"2023-12-01T01:16:27.324599+0000","last_clean":"2023-12-01T01:16:27.324599+0000","last_became_active":"2023-12-01T01:16:25.366147+0000","last_became_peered":"2023-12-01T01:16:25.366147+0000","last_unstale":"2023-12-01T01:16:27.324599+0000","last_undegraded":"2023-12-01T01:16:27.324599+0000","last_fullsized":"2023-12-01T01:16:27.324599+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:01:29.670329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.592834+0000","last_change":"2023-12-01T01:16:25.374016+0000","last_active":"2023-12-01T01:16:25.592834+0000","last_peered":"2023-12-01T01:16:25.592834+0000","last_clean":"2023-12-01T01:16:25.592834+0000","last_became_active":"2023-12-01T01:16:25.373543+0000","last_became_peered":"2023-12-01T01:16:25.373543+0000","last_unstale":"2023-12-01T01:16:25.592834+0000","last_undegraded":"2023-12-01T01:16:25.592834+0000","last_fullsized":"2023-12-01T01:16:25.592834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:36:22.858446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.584312+0000","last_change":"2023-12-01T01:16:25.363598+0000","last_active":"2023-12-01T01:16:26.584312+0000","last_peered":"2023-12-01T01:16:26.584312+0000","last_clean":"2023-12-01T01:16:26.584312+0000","last_became_active":"2023-12-01T01:16:25.363111+0000","last_became_peered":"2023-12-01T01:16:25.363111+0000","last_unstale":"2023-12-01T01:16:26.584312+0000","last_undegraded":"2023-12-01T01:16:26.584312+0000","last_fullsized":"2023-12-01T01:16:26.584312+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:42:08.171789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.375829+0000","last_change":"2023-12-01T01:16:25.368446+0000","last_active":"2023-12-01T01:16:25.375829+0000","last_peered":"2023-12-01T01:16:25.375829+0000","last_clean":"2023-12-01T01:16:25.375829+0000","last_became_active":"2023-12-01T01:16:25.361893+0000","last_became_peered":"2023-12-01T01:16:25.361893+0000","last_unstale":"2023-12-01T01:16:25.375829+0000","last_undegraded":"2023-12-01T01:16:25.375829+0000","last_fullsized":"2023-12-01T01:16:25.375829+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:05:45.697338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'11","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:30.651957+0000","last_change":"2023-12-01T01:16:24.934939+0000","last_active":"2023-12-01T01:16:30.651957+0000","last_peered":"2023-12-01T01:16:30.651957+0000","last_clean":"2023-12-01T01:16:30.651957+0000","last_became_active":"2023-12-01T01:16:24.934481+0000","last_became_peered":"2023-12-01T01:16:24.934481+0000","last_unstale":"2023-12-01T01:16:30.651957+0000","last_undegraded":"2023-12-01T01:16:30.651957+0000","last_fullsized":"2023-12-01T01:16:30.651957+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:39:49.804567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.571536+0000","last_change":"2023-12-01T01:16:25.364061+0000","last_active":"2023-12-01T01:16:26.571536+0000","last_peered":"2023-12-01T01:16:26.571536+0000","last_clean":"2023-12-01T01:16:26.571536+0000","last_became_active":"2023-12-01T01:16:25.363678+0000","last_became_peered":"2023-12-01T01:16:25.363678+0000","last_unstale":"2023-12-01T01:16:26.571536+0000","last_undegraded":"2023-12-01T01:16:26.571536+0000","last_fullsized":"2023-12-01T01:16:26.571536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:24:07.107604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325949+0000","last_change":"2023-12-01T01:16:24.939338+0000","last_active":"2023-12-01T01:16:27.325949+0000","last_peered":"2023-12-01T01:16:27.325949+0000","last_clean":"2023-12-01T01:16:27.325949+0000","last_became_active":"2023-12-01T01:16:24.939007+0000","last_became_peered":"2023-12-01T01:16:24.939007+0000","last_unstale":"2023-12-01T01:16:27.325949+0000","last_undegraded":"2023-12-01T01:16:27.325949+0000","last_fullsized":"2023-12-01T01:16:27.325949+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:56:50.732188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325303+0000","last_change":"2023-12-01T01:16:25.362231+0000","last_active":"2023-12-01T01:16:27.325303+0000","last_peered":"2023-12-01T01:16:27.325303+0000","last_clean":"2023-12-01T01:16:27.325303+0000","last_became_active":"2023-12-01T01:16:25.361920+0000","last_became_peered":"2023-12-01T01:16:25.361920+0000","last_unstale":"2023-12-01T01:16:27.325303+0000","last_undegraded":"2023-12-01T01:16:27.325303+0000","last_fullsized":"2023-12-01T01:16:27.325303+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:22:22.104480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'7","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:30.653095+0000","last_change":"2023-12-01T01:16:24.931801+0000","last_active":"2023-12-01T01:16:30.653095+0000","last_peered":"2023-12-01T01:16:30.653095+0000","last_clean":"2023-12-01T01:16:30.653095+0000","last_became_active":"2023-12-01T01:16:24.931451+0000","last_became_peered":"2023-12-01T01:16:24.931451+0000","last_unstale":"2023-12-01T01:16:30.653095+0000","last_undegraded":"2023-12-01T01:16:30.653095+0000","last_fullsized":"2023-12-01T01:16:30.653095+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:22:41.666502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.324789+0000","last_change":"2023-12-01T01:16:25.361789+0000","last_active":"2023-12-01T01:16:27.324789+0000","last_peered":"2023-12-01T01:16:27.324789+0000","last_clean":"2023-12-01T01:16:27.324789+0000","last_became_active":"2023-12-01T01:16:25.361480+0000","last_became_peered":"2023-12-01T01:16:25.361480+0000","last_unstale":"2023-12-01T01:16:27.324789+0000","last_undegraded":"2023-12-01T01:16:27.324789+0000","last_fullsized":"2023-12-01T01:16:27.324789+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:07:00.742167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.071493+0000","last_change":"2023-12-01T01:16:24.933362+0000","last_active":"2023-12-01T01:16:26.071493+0000","last_peered":"2023-12-01T01:16:26.071493+0000","last_clean":"2023-12-01T01:16:26.071493+0000","last_became_active":"2023-12-01T01:16:24.933049+0000","last_became_peered":"2023-12-01T01:16:24.933049+0000","last_unstale":"2023-12-01T01:16:26.071493+0000","last_undegraded":"2023-12-01T01:16:26.071493+0000","last_fullsized":"2023-12-01T01:16:26.071493+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:47:37.153344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325647+0000","last_change":"2023-12-01T01:16:25.364419+0000","last_active":"2023-12-01T01:16:27.325647+0000","last_peered":"2023-12-01T01:16:27.325647+0000","last_clean":"2023-12-01T01:16:27.325647+0000","last_became_active":"2023-12-01T01:16:25.363022+0000","last_became_peered":"2023-12-01T01:16:25.363022+0000","last_unstale":"2023-12-01T01:16:27.325647+0000","last_undegraded":"2023-12-01T01:16:27.325647+0000","last_fullsized":"2023-12-01T01:16:27.325647+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:58:45.660625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:28.632500+0000","last_change":"2023-12-01T01:16:24.935947+0000","last_active":"2023-12-01T01:16:28.632500+0000","last_peered":"2023-12-01T01:16:28.632500+0000","last_clean":"2023-12-01T01:16:28.632500+0000","last_became_active":"2023-12-01T01:16:24.935061+0000","last_became_peered":"2023-12-01T01:16:24.935061+0000","last_unstale":"2023-12-01T01:16:28.632500+0000","last_undegraded":"2023-12-01T01:16:28.632500+0000","last_fullsized":"2023-12-01T01:16:28.632500+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:00:31.551362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.071450+0000","last_change":"2023-12-01T01:16:25.363229+0000","last_active":"2023-12-01T01:16:26.071450+0000","last_peered":"2023-12-01T01:16:26.071450+0000","last_clean":"2023-12-01T01:16:26.071450+0000","last_became_active":"2023-12-01T01:16:25.362770+0000","last_became_peered":"2023-12-01T01:16:25.362770+0000","last_unstale":"2023-12-01T01:16:26.071450+0000","last_undegraded":"2023-12-01T01:16:26.071450+0000","last_fullsized":"2023-12-01T01:16:26.071450+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:45:02.782201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325488+0000","last_change":"2023-12-01T01:16:24.930509+0000","last_active":"2023-12-01T01:16:27.325488+0000","last_peered":"2023-12-01T01:16:27.325488+0000","last_clean":"2023-12-01T01:16:27.325488+0000","last_became_active":"2023-12-01T01:16:24.930188+0000","last_became_peered":"2023-12-01T01:16:24.930188+0000","last_unstale":"2023-12-01T01:16:27.325488+0000","last_undegraded":"2023-12-01T01:16:27.325488+0000","last_fullsized":"2023-12-01T01:16:27.325488+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:04:17.748363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325530+0000","last_change":"2023-12-01T01:16:25.363383+0000","last_active":"2023-12-01T01:16:27.325530+0000","last_peered":"2023-12-01T01:16:27.325530+0000","last_clean":"2023-12-01T01:16:27.325530+0000","last_became_active":"2023-12-01T01:16:25.363085+0000","last_became_peered":"2023-12-01T01:16:25.363085+0000","last_unstale":"2023-12-01T01:16:27.325530+0000","last_undegraded":"2023-12-01T01:16:27.325530+0000","last_fullsized":"2023-12-01T01:16:27.325530+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:34:36.921706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.352206+0000","last_change":"2023-12-01T01:16:24.929081+0000","last_active":"2023-12-01T01:16:26.352206+0000","last_peered":"2023-12-01T01:16:26.352206+0000","last_clean":"2023-12-01T01:16:26.352206+0000","last_became_active":"2023-12-01T01:16:24.928747+0000","last_became_peered":"2023-12-01T01:16:24.928747+0000","last_unstale":"2023-12-01T01:16:26.352206+0000","last_undegraded":"2023-12-01T01:16:26.352206+0000","last_fullsized":"2023-12-01T01:16:26.352206+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:02:24.727835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.571467+0000","last_change":"2023-12-01T01:16:25.360959+0000","last_active":"2023-12-01T01:16:26.571467+0000","last_peered":"2023-12-01T01:16:26.571467+0000","last_clean":"2023-12-01T01:16:26.571467+0000","last_became_active":"2023-12-01T01:16:25.360647+0000","last_became_peered":"2023-12-01T01:16:25.360647+0000","last_unstale":"2023-12-01T01:16:26.571467+0000","last_undegraded":"2023-12-01T01:16:26.571467+0000","last_fullsized":"2023-12-01T01:16:26.571467+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:50:24.151273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325262+0000","last_change":"2023-12-01T01:16:24.938322+0000","last_active":"2023-12-01T01:16:27.325262+0000","last_peered":"2023-12-01T01:16:27.325262+0000","last_clean":"2023-12-01T01:16:27.325262+0000","last_became_active":"2023-12-01T01:16:24.937747+0000","last_became_peered":"2023-12-01T01:16:24.937747+0000","last_unstale":"2023-12-01T01:16:27.325262+0000","last_undegraded":"2023-12-01T01:16:27.325262+0000","last_fullsized":"2023-12-01T01:16:27.325262+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:37:59.999748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.584270+0000","last_change":"2023-12-01T01:16:25.362779+0000","last_active":"2023-12-01T01:16:26.584270+0000","last_peered":"2023-12-01T01:16:26.584270+0000","last_clean":"2023-12-01T01:16:26.584270+0000","last_became_active":"2023-12-01T01:16:25.362487+0000","last_became_peered":"2023-12-01T01:16:25.362487+0000","last_unstale":"2023-12-01T01:16:26.584270+0000","last_undegraded":"2023-12-01T01:16:26.584270+0000","last_fullsized":"2023-12-01T01:16:26.584270+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:04:42.409498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325796+0000","last_change":"2023-12-01T01:16:24.933187+0000","last_active":"2023-12-01T01:16:27.325796+0000","last_peered":"2023-12-01T01:16:27.325796+0000","last_clean":"2023-12-01T01:16:27.325796+0000","last_became_active":"2023-12-01T01:16:24.932819+0000","last_became_peered":"2023-12-01T01:16:24.932819+0000","last_unstale":"2023-12-01T01:16:27.325796+0000","last_undegraded":"2023-12-01T01:16:27.325796+0000","last_fullsized":"2023-12-01T01:16:27.325796+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:54:34.739815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.324904+0000","last_change":"2023-12-01T01:16:25.361183+0000","last_active":"2023-12-01T01:16:27.324904+0000","last_peered":"2023-12-01T01:16:27.324904+0000","last_clean":"2023-12-01T01:16:27.324904+0000","last_became_active":"2023-12-01T01:16:25.360871+0000","last_became_peered":"2023-12-01T01:16:25.360871+0000","last_unstale":"2023-12-01T01:16:27.324904+0000","last_undegraded":"2023-12-01T01:16:27.324904+0000","last_fullsized":"2023-12-01T01:16:27.324904+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:06:04.748159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.350065+0000","last_change":"2023-12-01T01:16:24.939355+0000","last_active":"2023-12-01T01:16:26.350065+0000","last_peered":"2023-12-01T01:16:26.350065+0000","last_clean":"2023-12-01T01:16:26.350065+0000","last_became_active":"2023-12-01T01:16:24.938967+0000","last_became_peered":"2023-12-01T01:16:24.938967+0000","last_unstale":"2023-12-01T01:16:26.350065+0000","last_undegraded":"2023-12-01T01:16:26.350065+0000","last_fullsized":"2023-12-01T01:16:26.350065+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:20:51.172753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325217+0000","last_change":"2023-12-01T01:16:25.366752+0000","last_active":"2023-12-01T01:16:27.325217+0000","last_peered":"2023-12-01T01:16:27.325217+0000","last_clean":"2023-12-01T01:16:27.325217+0000","last_became_active":"2023-12-01T01:16:25.366466+0000","last_became_peered":"2023-12-01T01:16:25.366466+0000","last_unstale":"2023-12-01T01:16:27.325217+0000","last_undegraded":"2023-12-01T01:16:27.325217+0000","last_fullsized":"2023-12-01T01:16:27.325217+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:09:53.716127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325703+0000","last_change":"2023-12-01T01:16:24.940827+0000","last_active":"2023-12-01T01:16:27.325703+0000","last_peered":"2023-12-01T01:16:27.325703+0000","last_clean":"2023-12-01T01:16:27.325703+0000","last_became_active":"2023-12-01T01:16:24.940515+0000","last_became_peered":"2023-12-01T01:16:24.940515+0000","last_unstale":"2023-12-01T01:16:27.325703+0000","last_undegraded":"2023-12-01T01:16:27.325703+0000","last_fullsized":"2023-12-01T01:16:27.325703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:08:26.409014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325032+0000","last_change":"2023-12-01T01:16:25.369662+0000","last_active":"2023-12-01T01:16:27.325032+0000","last_peered":"2023-12-01T01:16:27.325032+0000","last_clean":"2023-12-01T01:16:27.325032+0000","last_became_active":"2023-12-01T01:16:25.369178+0000","last_became_peered":"2023-12-01T01:16:25.369178+0000","last_unstale":"2023-12-01T01:16:27.325032+0000","last_undegraded":"2023-12-01T01:16:27.325032+0000","last_fullsized":"2023-12-01T01:16:27.325032+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:40:39.919414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'6","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325641+0000","last_change":"2023-12-01T01:16:24.936733+0000","last_active":"2023-12-01T01:16:27.325641+0000","last_peered":"2023-12-01T01:16:27.325641+0000","last_clean":"2023-12-01T01:16:27.325641+0000","last_became_active":"2023-12-01T01:16:24.936459+0000","last_became_peered":"2023-12-01T01:16:24.936459+0000","last_unstale":"2023-12-01T01:16:27.325641+0000","last_undegraded":"2023-12-01T01:16:27.325641+0000","last_fullsized":"2023-12-01T01:16:27.325641+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:19:58.387650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.071505+0000","last_change":"2023-12-01T01:16:25.364292+0000","last_active":"2023-12-01T01:16:26.071505+0000","last_peered":"2023-12-01T01:16:26.071505+0000","last_clean":"2023-12-01T01:16:26.071505+0000","last_became_active":"2023-12-01T01:16:25.362615+0000","last_became_peered":"2023-12-01T01:16:25.362615+0000","last_unstale":"2023-12-01T01:16:26.071505+0000","last_undegraded":"2023-12-01T01:16:26.071505+0000","last_fullsized":"2023-12-01T01:16:26.071505+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:06:07.350607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:29.636705+0000","last_change":"2023-12-01T01:16:24.941007+0000","last_active":"2023-12-01T01:16:29.636705+0000","last_peered":"2023-12-01T01:16:29.636705+0000","last_clean":"2023-12-01T01:16:29.636705+0000","last_became_active":"2023-12-01T01:16:24.940669+0000","last_became_peered":"2023-12-01T01:16:24.940669+0000","last_unstale":"2023-12-01T01:16:29.636705+0000","last_undegraded":"2023-12-01T01:16:29.636705+0000","last_fullsized":"2023-12-01T01:16:29.636705+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:26:21.907942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.352205+0000","last_change":"2023-12-01T01:16:25.360104+0000","last_active":"2023-12-01T01:16:26.352205+0000","last_peered":"2023-12-01T01:16:26.352205+0000","last_clean":"2023-12-01T01:16:26.352205+0000","last_became_active":"2023-12-01T01:16:25.359807+0000","last_became_peered":"2023-12-01T01:16:25.359807+0000","last_unstale":"2023-12-01T01:16:26.352205+0000","last_undegraded":"2023-12-01T01:16:26.352205+0000","last_fullsized":"2023-12-01T01:16:26.352205+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:04:05.100258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:28.632442+0000","last_change":"2023-12-01T01:16:24.945074+0000","last_active":"2023-12-01T01:16:28.632442+0000","last_peered":"2023-12-01T01:16:28.632442+0000","last_clean":"2023-12-01T01:16:28.632442+0000","last_became_active":"2023-12-01T01:16:24.944737+0000","last_became_peered":"2023-12-01T01:16:24.944737+0000","last_unstale":"2023-12-01T01:16:28.632442+0000","last_undegraded":"2023-12-01T01:16:28.632442+0000","last_fullsized":"2023-12-01T01:16:28.632442+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:44:53.931454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.350184+0000","last_change":"2023-12-01T01:16:25.364995+0000","last_active":"2023-12-01T01:16:26.350184+0000","last_peered":"2023-12-01T01:16:26.350184+0000","last_clean":"2023-12-01T01:16:26.350184+0000","last_became_active":"2023-12-01T01:16:25.364684+0000","last_became_peered":"2023-12-01T01:16:25.364684+0000","last_unstale":"2023-12-01T01:16:26.350184+0000","last_undegraded":"2023-12-01T01:16:26.350184+0000","last_fullsized":"2023-12-01T01:16:26.350184+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:41:35.422238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.554690+0000","last_change":"2023-12-01T01:16:24.929103+0000","last_active":"2023-12-01T01:16:26.554690+0000","last_peered":"2023-12-01T01:16:26.554690+0000","last_clean":"2023-12-01T01:16:26.554690+0000","last_became_active":"2023-12-01T01:16:24.928769+0000","last_became_peered":"2023-12-01T01:16:24.928769+0000","last_unstale":"2023-12-01T01:16:26.554690+0000","last_undegraded":"2023-12-01T01:16:26.554690+0000","last_fullsized":"2023-12-01T01:16:26.554690+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:45:24.542610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.349997+0000","last_change":"2023-12-01T01:16:25.370816+0000","last_active":"2023-12-01T01:16:26.349997+0000","last_peered":"2023-12-01T01:16:26.349997+0000","last_clean":"2023-12-01T01:16:26.349997+0000","last_became_active":"2023-12-01T01:16:25.370465+0000","last_became_peered":"2023-12-01T01:16:25.370465+0000","last_unstale":"2023-12-01T01:16:26.349997+0000","last_undegraded":"2023-12-01T01:16:26.349997+0000","last_fullsized":"2023-12-01T01:16:26.349997+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:52:28.990416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'16","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:29.638905+0000","last_change":"2023-12-01T01:16:24.933022+0000","last_active":"2023-12-01T01:16:29.638905+0000","last_peered":"2023-12-01T01:16:29.638905+0000","last_clean":"2023-12-01T01:16:29.638905+0000","last_became_active":"2023-12-01T01:16:24.932705+0000","last_became_peered":"2023-12-01T01:16:24.932705+0000","last_unstale":"2023-12-01T01:16:29.638905+0000","last_undegraded":"2023-12-01T01:16:29.638905+0000","last_fullsized":"2023-12-01T01:16:29.638905+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:36:44.747984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325386+0000","last_change":"2023-12-01T01:16:25.370881+0000","last_active":"2023-12-01T01:16:27.325386+0000","last_peered":"2023-12-01T01:16:27.325386+0000","last_clean":"2023-12-01T01:16:27.325386+0000","last_became_active":"2023-12-01T01:16:25.368242+0000","last_became_peered":"2023-12-01T01:16:25.368242+0000","last_unstale":"2023-12-01T01:16:27.325386+0000","last_undegraded":"2023-12-01T01:16:27.325386+0000","last_fullsized":"2023-12-01T01:16:27.325386+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:35:41.128841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.071546+0000","last_change":"2023-12-01T01:16:24.933579+0000","last_active":"2023-12-01T01:16:26.071546+0000","last_peered":"2023-12-01T01:16:26.071546+0000","last_clean":"2023-12-01T01:16:26.071546+0000","last_became_active":"2023-12-01T01:16:24.933207+0000","last_became_peered":"2023-12-01T01:16:24.933207+0000","last_unstale":"2023-12-01T01:16:26.071546+0000","last_undegraded":"2023-12-01T01:16:26.071546+0000","last_fullsized":"2023-12-01T01:16:26.071546+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:28.309551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.592816+0000","last_change":"2023-12-01T01:16:25.368611+0000","last_active":"2023-12-01T01:16:25.592816+0000","last_peered":"2023-12-01T01:16:25.592816+0000","last_clean":"2023-12-01T01:16:25.592816+0000","last_became_active":"2023-12-01T01:16:25.368291+0000","last_became_peered":"2023-12-01T01:16:25.368291+0000","last_unstale":"2023-12-01T01:16:25.592816+0000","last_undegraded":"2023-12-01T01:16:25.592816+0000","last_fullsized":"2023-12-01T01:16:25.592816+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:17:21.824737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325140+0000","last_change":"2023-12-01T01:16:24.941410+0000","last_active":"2023-12-01T01:16:27.325140+0000","last_peered":"2023-12-01T01:16:27.325140+0000","last_clean":"2023-12-01T01:16:27.325140+0000","last_became_active":"2023-12-01T01:16:24.941127+0000","last_became_peered":"2023-12-01T01:16:24.941127+0000","last_unstale":"2023-12-01T01:16:27.325140+0000","last_undegraded":"2023-12-01T01:16:27.325140+0000","last_fullsized":"2023-12-01T01:16:27.325140+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:25:20.911892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325740+0000","last_change":"2023-12-01T01:16:25.368235+0000","last_active":"2023-12-01T01:16:27.325740+0000","last_peered":"2023-12-01T01:16:27.325740+0000","last_clean":"2023-12-01T01:16:27.325740+0000","last_became_active":"2023-12-01T01:16:25.367918+0000","last_became_peered":"2023-12-01T01:16:25.367918+0000","last_unstale":"2023-12-01T01:16:27.325740+0000","last_undegraded":"2023-12-01T01:16:27.325740+0000","last_fullsized":"2023-12-01T01:16:27.325740+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:12:26.665404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.350039+0000","last_change":"2023-12-01T01:16:24.932057+0000","last_active":"2023-12-01T01:16:26.350039+0000","last_peered":"2023-12-01T01:16:26.350039+0000","last_clean":"2023-12-01T01:16:26.350039+0000","last_became_active":"2023-12-01T01:16:24.931817+0000","last_became_peered":"2023-12-01T01:16:24.931817+0000","last_unstale":"2023-12-01T01:16:26.350039+0000","last_undegraded":"2023-12-01T01:16:26.350039+0000","last_fullsized":"2023-12-01T01:16:26.350039+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:19:02.039334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325595+0000","last_change":"2023-12-01T01:16:25.367431+0000","last_active":"2023-12-01T01:16:27.325595+0000","last_peered":"2023-12-01T01:16:27.325595+0000","last_clean":"2023-12-01T01:16:27.325595+0000","last_became_active":"2023-12-01T01:16:25.367184+0000","last_became_peered":"2023-12-01T01:16:25.367184+0000","last_unstale":"2023-12-01T01:16:27.325595+0000","last_undegraded":"2023-12-01T01:16:27.325595+0000","last_fullsized":"2023-12-01T01:16:27.325595+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:21:19.132725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325383+0000","last_change":"2023-12-01T01:16:25.370593+0000","last_active":"2023-12-01T01:16:27.325383+0000","last_peered":"2023-12-01T01:16:27.325383+0000","last_clean":"2023-12-01T01:16:27.325383+0000","last_became_active":"2023-12-01T01:16:25.367614+0000","last_became_peered":"2023-12-01T01:16:25.367614+0000","last_unstale":"2023-12-01T01:16:27.325383+0000","last_undegraded":"2023-12-01T01:16:27.325383+0000","last_fullsized":"2023-12-01T01:16:27.325383+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:56:34.388180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.352111+0000","last_change":"2023-12-01T01:16:24.931188+0000","last_active":"2023-12-01T01:16:26.352111+0000","last_peered":"2023-12-01T01:16:26.352111+0000","last_clean":"2023-12-01T01:16:26.352111+0000","last_became_active":"2023-12-01T01:16:24.930875+0000","last_became_peered":"2023-12-01T01:16:24.930875+0000","last_unstale":"2023-12-01T01:16:26.352111+0000","last_undegraded":"2023-12-01T01:16:26.352111+0000","last_fullsized":"2023-12-01T01:16:26.352111+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:17.729286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.571499+0000","last_change":"2023-12-01T01:16:25.361359+0000","last_active":"2023-12-01T01:16:26.571499+0000","last_peered":"2023-12-01T01:16:26.571499+0000","last_clean":"2023-12-01T01:16:26.571499+0000","last_became_active":"2023-12-01T01:16:25.361083+0000","last_became_peered":"2023-12-01T01:16:25.361083+0000","last_unstale":"2023-12-01T01:16:26.571499+0000","last_undegraded":"2023-12-01T01:16:26.571499+0000","last_fullsized":"2023-12-01T01:16:26.571499+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:42:09.417021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.554872+0000","last_change":"2023-12-01T01:16:24.927614+0000","last_active":"2023-12-01T01:16:26.554872+0000","last_peered":"2023-12-01T01:16:26.554872+0000","last_clean":"2023-12-01T01:16:26.554872+0000","last_became_active":"2023-12-01T01:16:24.926459+0000","last_became_peered":"2023-12-01T01:16:24.926459+0000","last_unstale":"2023-12-01T01:16:26.554872+0000","last_undegraded":"2023-12-01T01:16:26.554872+0000","last_fullsized":"2023-12-01T01:16:26.554872+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:41:25.435261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":323,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.358040+0000","last_change":"2023-12-01T01:15:25.599551+0000","last_active":"2023-12-01T01:16:25.358040+0000","last_peered":"2023-12-01T01:16:25.358040+0000","last_clean":"2023-12-01T01:16:25.358040+0000","last_became_active":"2023-12-01T01:15:25.598951+0000","last_became_peered":"2023-12-01T01:15:25.598951+0000","last_unstale":"2023-12-01T01:16:25.358040+0000","last_undegraded":"2023-12-01T01:16:25.358040+0000","last_fullsized":"2023-12-01T01:16:25.358040+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:12:31.173756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:12:31.173756+0000","last_clean_scrub_stamp":"2023-12-01T01:12:31.173756+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:00:11.343823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.352149+0000","last_change":"2023-12-01T01:16:25.358617+0000","last_active":"2023-12-01T01:16:26.352149+0000","last_peered":"2023-12-01T01:16:26.352149+0000","last_clean":"2023-12-01T01:16:26.352149+0000","last_became_active":"2023-12-01T01:16:25.358325+0000","last_became_peered":"2023-12-01T01:16:25.358325+0000","last_unstale":"2023-12-01T01:16:26.352149+0000","last_undegraded":"2023-12-01T01:16:26.352149+0000","last_fullsized":"2023-12-01T01:16:26.352149+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:40:01.111777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.349804+0000","last_change":"2023-12-01T01:16:24.935103+0000","last_active":"2023-12-01T01:16:26.349804+0000","last_peered":"2023-12-01T01:16:26.349804+0000","last_clean":"2023-12-01T01:16:26.349804+0000","last_became_active":"2023-12-01T01:16:24.934900+0000","last_became_peered":"2023-12-01T01:16:24.934900+0000","last_unstale":"2023-12-01T01:16:26.349804+0000","last_undegraded":"2023-12-01T01:16:26.349804+0000","last_fullsized":"2023-12-01T01:16:26.349804+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:59:37.138090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"82'3","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:30.653485+0000","last_change":"2023-12-01T01:16:24.938260+0000","last_active":"2023-12-01T01:16:30.653485+0000","last_peered":"2023-12-01T01:16:30.653485+0000","last_clean":"2023-12-01T01:16:30.653485+0000","last_became_active":"2023-12-01T01:16:24.937939+0000","last_became_peered":"2023-12-01T01:16:24.937939+0000","last_unstale":"2023-12-01T01:16:30.653485+0000","last_undegraded":"2023-12-01T01:16:30.653485+0000","last_fullsized":"2023-12-01T01:16:30.653485+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:07:05.090877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.352470+0000","last_change":"2023-12-01T01:16:25.362137+0000","last_active":"2023-12-01T01:16:26.352470+0000","last_peered":"2023-12-01T01:16:26.352470+0000","last_clean":"2023-12-01T01:16:26.352470+0000","last_became_active":"2023-12-01T01:16:25.361563+0000","last_became_peered":"2023-12-01T01:16:25.361563+0000","last_unstale":"2023-12-01T01:16:26.352470+0000","last_undegraded":"2023-12-01T01:16:26.352470+0000","last_fullsized":"2023-12-01T01:16:26.352470+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:37:56.473920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'7","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:30.654598+0000","last_change":"2023-12-01T01:16:24.935692+0000","last_active":"2023-12-01T01:16:30.654598+0000","last_peered":"2023-12-01T01:16:30.654598+0000","last_clean":"2023-12-01T01:16:30.654598+0000","last_became_active":"2023-12-01T01:16:24.934799+0000","last_became_peered":"2023-12-01T01:16:24.934799+0000","last_unstale":"2023-12-01T01:16:30.654598+0000","last_undegraded":"2023-12-01T01:16:30.654598+0000","last_fullsized":"2023-12-01T01:16:30.654598+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:22:51.586089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.571589+0000","last_change":"2023-12-01T01:16:25.368672+0000","last_active":"2023-12-01T01:16:26.571589+0000","last_peered":"2023-12-01T01:16:26.571589+0000","last_clean":"2023-12-01T01:16:26.571589+0000","last_became_active":"2023-12-01T01:16:25.368309+0000","last_became_peered":"2023-12-01T01:16:25.368309+0000","last_unstale":"2023-12-01T01:16:26.571589+0000","last_undegraded":"2023-12-01T01:16:26.571589+0000","last_fullsized":"2023-12-01T01:16:26.571589+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:30:59.029250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.554938+0000","last_change":"2023-12-01T01:16:24.931322+0000","last_active":"2023-12-01T01:16:26.554938+0000","last_peered":"2023-12-01T01:16:26.554938+0000","last_clean":"2023-12-01T01:16:26.554938+0000","last_became_active":"2023-12-01T01:16:24.930961+0000","last_became_peered":"2023-12-01T01:16:24.930961+0000","last_unstale":"2023-12-01T01:16:26.554938+0000","last_undegraded":"2023-12-01T01:16:26.554938+0000","last_fullsized":"2023-12-01T01:16:26.554938+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:47:16.317035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325104+0000","last_change":"2023-12-01T01:16:25.364065+0000","last_active":"2023-12-01T01:16:27.325104+0000","last_peered":"2023-12-01T01:16:27.325104+0000","last_clean":"2023-12-01T01:16:27.325104+0000","last_became_active":"2023-12-01T01:16:25.363694+0000","last_became_peered":"2023-12-01T01:16:25.363694+0000","last_unstale":"2023-12-01T01:16:27.325104+0000","last_undegraded":"2023-12-01T01:16:27.325104+0000","last_fullsized":"2023-12-01T01:16:27.325104+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:49:39.091587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:28.633891+0000","last_change":"2023-12-01T01:16:24.932730+0000","last_active":"2023-12-01T01:16:28.633891+0000","last_peered":"2023-12-01T01:16:28.633891+0000","last_clean":"2023-12-01T01:16:28.633891+0000","last_became_active":"2023-12-01T01:16:24.932404+0000","last_became_peered":"2023-12-01T01:16:24.932404+0000","last_unstale":"2023-12-01T01:16:28.633891+0000","last_undegraded":"2023-12-01T01:16:28.633891+0000","last_fullsized":"2023-12-01T01:16:28.633891+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:41:33.104418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.071437+0000","last_change":"2023-12-01T01:16:25.375795+0000","last_active":"2023-12-01T01:16:26.071437+0000","last_peered":"2023-12-01T01:16:26.071437+0000","last_clean":"2023-12-01T01:16:26.071437+0000","last_became_active":"2023-12-01T01:16:25.375485+0000","last_became_peered":"2023-12-01T01:16:25.375485+0000","last_unstale":"2023-12-01T01:16:26.071437+0000","last_undegraded":"2023-12-01T01:16:26.071437+0000","last_fullsized":"2023-12-01T01:16:26.071437+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:10:04.009511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'5","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:30.653691+0000","last_change":"2023-12-01T01:16:24.936770+0000","last_active":"2023-12-01T01:16:30.653691+0000","last_peered":"2023-12-01T01:16:30.653691+0000","last_clean":"2023-12-01T01:16:30.653691+0000","last_became_active":"2023-12-01T01:16:24.936058+0000","last_became_peered":"2023-12-01T01:16:24.936058+0000","last_unstale":"2023-12-01T01:16:30.653691+0000","last_undegraded":"2023-12-01T01:16:30.653691+0000","last_fullsized":"2023-12-01T01:16:30.653691+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:22:21.414131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.592744+0000","last_change":"2023-12-01T01:16:25.372183+0000","last_active":"2023-12-01T01:16:25.592744+0000","last_peered":"2023-12-01T01:16:25.592744+0000","last_clean":"2023-12-01T01:16:25.592744+0000","last_became_active":"2023-12-01T01:16:25.371865+0000","last_became_peered":"2023-12-01T01:16:25.371865+0000","last_unstale":"2023-12-01T01:16:25.592744+0000","last_undegraded":"2023-12-01T01:16:25.592744+0000","last_fullsized":"2023-12-01T01:16:25.592744+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:19:54.675425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.356576+0000","last_change":"2023-12-01T01:16:24.939397+0000","last_active":"2023-12-01T01:16:25.356576+0000","last_peered":"2023-12-01T01:16:25.356576+0000","last_clean":"2023-12-01T01:16:25.356576+0000","last_became_active":"2023-12-01T01:16:24.939081+0000","last_became_peered":"2023-12-01T01:16:24.939081+0000","last_unstale":"2023-12-01T01:16:25.356576+0000","last_undegraded":"2023-12-01T01:16:25.356576+0000","last_fullsized":"2023-12-01T01:16:25.356576+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:21:34.967255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.571515+0000","last_change":"2023-12-01T01:16:25.368484+0000","last_active":"2023-12-01T01:16:26.571515+0000","last_peered":"2023-12-01T01:16:26.571515+0000","last_clean":"2023-12-01T01:16:26.571515+0000","last_became_active":"2023-12-01T01:16:25.367638+0000","last_became_peered":"2023-12-01T01:16:25.367638+0000","last_unstale":"2023-12-01T01:16:26.571515+0000","last_undegraded":"2023-12-01T01:16:26.571515+0000","last_fullsized":"2023-12-01T01:16:26.571515+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:06:05.354443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:28.633543+0000","last_change":"2023-12-01T01:16:24.933304+0000","last_active":"2023-12-01T01:16:28.633543+0000","last_peered":"2023-12-01T01:16:28.633543+0000","last_clean":"2023-12-01T01:16:28.633543+0000","last_became_active":"2023-12-01T01:16:24.933028+0000","last_became_peered":"2023-12-01T01:16:24.933028+0000","last_unstale":"2023-12-01T01:16:28.633543+0000","last_undegraded":"2023-12-01T01:16:28.633543+0000","last_fullsized":"2023-12-01T01:16:28.633543+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:26:51.810995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.350012+0000","last_change":"2023-12-01T01:16:25.364888+0000","last_active":"2023-12-01T01:16:26.350012+0000","last_peered":"2023-12-01T01:16:26.350012+0000","last_clean":"2023-12-01T01:16:26.350012+0000","last_became_active":"2023-12-01T01:16:25.364595+0000","last_became_peered":"2023-12-01T01:16:25.364595+0000","last_unstale":"2023-12-01T01:16:26.350012+0000","last_undegraded":"2023-12-01T01:16:26.350012+0000","last_fullsized":"2023-12-01T01:16:26.350012+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:54:23.178824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:29.638923+0000","last_change":"2023-12-01T01:16:24.945771+0000","last_active":"2023-12-01T01:16:29.638923+0000","last_peered":"2023-12-01T01:16:29.638923+0000","last_clean":"2023-12-01T01:16:29.638923+0000","last_became_active":"2023-12-01T01:16:24.945441+0000","last_became_peered":"2023-12-01T01:16:24.945441+0000","last_unstale":"2023-12-01T01:16:29.638923+0000","last_undegraded":"2023-12-01T01:16:29.638923+0000","last_fullsized":"2023-12-01T01:16:29.638923+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:49:41.234472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.350239+0000","last_change":"2023-12-01T01:16:25.365246+0000","last_active":"2023-12-01T01:16:26.350239+0000","last_peered":"2023-12-01T01:16:26.350239+0000","last_clean":"2023-12-01T01:16:26.350239+0000","last_became_active":"2023-12-01T01:16:25.364364+0000","last_became_peered":"2023-12-01T01:16:25.364364+0000","last_unstale":"2023-12-01T01:16:26.350239+0000","last_undegraded":"2023-12-01T01:16:26.350239+0000","last_fullsized":"2023-12-01T01:16:26.350239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:56:44.815172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.375188+0000","last_change":"2023-12-01T01:16:24.944229+0000","last_active":"2023-12-01T01:16:25.375188+0000","last_peered":"2023-12-01T01:16:25.375188+0000","last_clean":"2023-12-01T01:16:25.375188+0000","last_became_active":"2023-12-01T01:16:24.943880+0000","last_became_peered":"2023-12-01T01:16:24.943880+0000","last_unstale":"2023-12-01T01:16:25.375188+0000","last_undegraded":"2023-12-01T01:16:25.375188+0000","last_fullsized":"2023-12-01T01:16:25.375188+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:44:26.769358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.350055+0000","last_change":"2023-12-01T01:16:25.361028+0000","last_active":"2023-12-01T01:16:26.350055+0000","last_peered":"2023-12-01T01:16:26.350055+0000","last_clean":"2023-12-01T01:16:26.350055+0000","last_became_active":"2023-12-01T01:16:25.360702+0000","last_became_peered":"2023-12-01T01:16:25.360702+0000","last_unstale":"2023-12-01T01:16:26.350055+0000","last_undegraded":"2023-12-01T01:16:26.350055+0000","last_fullsized":"2023-12-01T01:16:26.350055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:45:09.472497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.352416+0000","last_change":"2023-12-01T01:16:24.945567+0000","last_active":"2023-12-01T01:16:26.352416+0000","last_peered":"2023-12-01T01:16:26.352416+0000","last_clean":"2023-12-01T01:16:26.352416+0000","last_became_active":"2023-12-01T01:16:24.945256+0000","last_became_peered":"2023-12-01T01:16:24.945256+0000","last_unstale":"2023-12-01T01:16:26.352416+0000","last_undegraded":"2023-12-01T01:16:26.352416+0000","last_fullsized":"2023-12-01T01:16:26.352416+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:19:30.944573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.350244+0000","last_change":"2023-12-01T01:16:25.367247+0000","last_active":"2023-12-01T01:16:26.350244+0000","last_peered":"2023-12-01T01:16:26.350244+0000","last_clean":"2023-12-01T01:16:26.350244+0000","last_became_active":"2023-12-01T01:16:25.366872+0000","last_became_peered":"2023-12-01T01:16:25.366872+0000","last_unstale":"2023-12-01T01:16:26.350244+0000","last_undegraded":"2023-12-01T01:16:26.350244+0000","last_fullsized":"2023-12-01T01:16:26.350244+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:03:07.697233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.592768+0000","last_change":"2023-12-01T01:16:25.376005+0000","last_active":"2023-12-01T01:16:25.592768+0000","last_peered":"2023-12-01T01:16:25.592768+0000","last_clean":"2023-12-01T01:16:25.592768+0000","last_became_active":"2023-12-01T01:16:25.375702+0000","last_became_peered":"2023-12-01T01:16:25.375702+0000","last_unstale":"2023-12-01T01:16:25.592768+0000","last_undegraded":"2023-12-01T01:16:25.592768+0000","last_fullsized":"2023-12-01T01:16:25.592768+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:14:00.955730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.378145+0000","last_change":"2023-12-01T01:16:25.371023+0000","last_active":"2023-12-01T01:16:25.378145+0000","last_peered":"2023-12-01T01:16:25.378145+0000","last_clean":"2023-12-01T01:16:25.378145+0000","last_became_active":"2023-12-01T01:16:25.370694+0000","last_became_peered":"2023-12-01T01:16:25.370694+0000","last_unstale":"2023-12-01T01:16:25.378145+0000","last_undegraded":"2023-12-01T01:16:25.378145+0000","last_fullsized":"2023-12-01T01:16:25.378145+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:03:00.715413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.375644+0000","last_change":"2023-12-01T01:16:25.361795+0000","last_active":"2023-12-01T01:16:25.375644+0000","last_peered":"2023-12-01T01:16:25.375644+0000","last_clean":"2023-12-01T01:16:25.375644+0000","last_became_active":"2023-12-01T01:16:25.361459+0000","last_became_peered":"2023-12-01T01:16:25.361459+0000","last_unstale":"2023-12-01T01:16:25.375644+0000","last_undegraded":"2023-12-01T01:16:25.375644+0000","last_fullsized":"2023-12-01T01:16:25.375644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:18:57.531240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.324741+0000","last_change":"2023-12-01T01:16:25.363773+0000","last_active":"2023-12-01T01:16:27.324741+0000","last_peered":"2023-12-01T01:16:27.324741+0000","last_clean":"2023-12-01T01:16:27.324741+0000","last_became_active":"2023-12-01T01:16:25.363422+0000","last_became_peered":"2023-12-01T01:16:25.363422+0000","last_unstale":"2023-12-01T01:16:27.324741+0000","last_undegraded":"2023-12-01T01:16:27.324741+0000","last_fullsized":"2023-12-01T01:16:27.324741+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:48:33.395566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.352367+0000","last_change":"2023-12-01T01:16:25.359431+0000","last_active":"2023-12-01T01:16:26.352367+0000","last_peered":"2023-12-01T01:16:26.352367+0000","last_clean":"2023-12-01T01:16:26.352367+0000","last_became_active":"2023-12-01T01:16:25.359116+0000","last_became_peered":"2023-12-01T01:16:25.359116+0000","last_unstale":"2023-12-01T01:16:26.352367+0000","last_undegraded":"2023-12-01T01:16:26.352367+0000","last_fullsized":"2023-12-01T01:16:26.352367+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:32:26.545269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.592804+0000","last_change":"2023-12-01T01:16:25.368008+0000","last_active":"2023-12-01T01:16:25.592804+0000","last_peered":"2023-12-01T01:16:25.592804+0000","last_clean":"2023-12-01T01:16:25.592804+0000","last_became_active":"2023-12-01T01:16:25.367726+0000","last_became_peered":"2023-12-01T01:16:25.367726+0000","last_unstale":"2023-12-01T01:16:25.592804+0000","last_undegraded":"2023-12-01T01:16:25.592804+0000","last_fullsized":"2023-12-01T01:16:25.592804+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:38:33.837846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.378494+0000","last_change":"2023-12-01T01:16:25.372781+0000","last_active":"2023-12-01T01:16:25.378494+0000","last_peered":"2023-12-01T01:16:25.378494+0000","last_clean":"2023-12-01T01:16:25.378494+0000","last_became_active":"2023-12-01T01:16:25.371443+0000","last_became_peered":"2023-12-01T01:16:25.371443+0000","last_unstale":"2023-12-01T01:16:25.378494+0000","last_undegraded":"2023-12-01T01:16:25.378494+0000","last_fullsized":"2023-12-01T01:16:25.378494+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:49:06.422055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.571571+0000","last_change":"2023-12-01T01:16:25.363048+0000","last_active":"2023-12-01T01:16:26.571571+0000","last_peered":"2023-12-01T01:16:26.571571+0000","last_clean":"2023-12-01T01:16:26.571571+0000","last_became_active":"2023-12-01T01:16:25.362752+0000","last_became_peered":"2023-12-01T01:16:25.362752+0000","last_unstale":"2023-12-01T01:16:26.571571+0000","last_undegraded":"2023-12-01T01:16:26.571571+0000","last_fullsized":"2023-12-01T01:16:26.571571+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:54:16.492562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325256+0000","last_change":"2023-12-01T01:16:25.359998+0000","last_active":"2023-12-01T01:16:27.325256+0000","last_peered":"2023-12-01T01:16:27.325256+0000","last_clean":"2023-12-01T01:16:27.325256+0000","last_became_active":"2023-12-01T01:16:25.359744+0000","last_became_peered":"2023-12-01T01:16:25.359744+0000","last_unstale":"2023-12-01T01:16:27.325256+0000","last_undegraded":"2023-12-01T01:16:27.325256+0000","last_fullsized":"2023-12-01T01:16:27.325256+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:12:33.947134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.571477+0000","last_change":"2023-12-01T01:16:25.373615+0000","last_active":"2023-12-01T01:16:26.571477+0000","last_peered":"2023-12-01T01:16:26.571477+0000","last_clean":"2023-12-01T01:16:26.571477+0000","last_became_active":"2023-12-01T01:16:25.373247+0000","last_became_peered":"2023-12-01T01:16:25.373247+0000","last_unstale":"2023-12-01T01:16:26.571477+0000","last_undegraded":"2023-12-01T01:16:26.571477+0000","last_fullsized":"2023-12-01T01:16:26.571477+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:28:10.815906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.324878+0000","last_change":"2023-12-01T01:16:25.367404+0000","last_active":"2023-12-01T01:16:27.324878+0000","last_peered":"2023-12-01T01:16:27.324878+0000","last_clean":"2023-12-01T01:16:27.324878+0000","last_became_active":"2023-12-01T01:16:25.366404+0000","last_became_peered":"2023-12-01T01:16:25.366404+0000","last_unstale":"2023-12-01T01:16:27.324878+0000","last_undegraded":"2023-12-01T01:16:27.324878+0000","last_fullsized":"2023-12-01T01:16:27.324878+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:54:49.668036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.352405+0000","last_change":"2023-12-01T01:16:25.360315+0000","last_active":"2023-12-01T01:16:26.352405+0000","last_peered":"2023-12-01T01:16:26.352405+0000","last_clean":"2023-12-01T01:16:26.352405+0000","last_became_active":"2023-12-01T01:16:25.360031+0000","last_became_peered":"2023-12-01T01:16:25.360031+0000","last_unstale":"2023-12-01T01:16:26.352405+0000","last_undegraded":"2023-12-01T01:16:26.352405+0000","last_fullsized":"2023-12-01T01:16:26.352405+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:13:24.437697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.592792+0000","last_change":"2023-12-01T01:16:25.367820+0000","last_active":"2023-12-01T01:16:25.592792+0000","last_peered":"2023-12-01T01:16:25.592792+0000","last_clean":"2023-12-01T01:16:25.592792+0000","last_became_active":"2023-12-01T01:16:25.367525+0000","last_became_peered":"2023-12-01T01:16:25.367525+0000","last_unstale":"2023-12-01T01:16:25.592792+0000","last_undegraded":"2023-12-01T01:16:25.592792+0000","last_fullsized":"2023-12-01T01:16:25.592792+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:28:01.552342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4324,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":188416,"data_stored":10946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":95,"ondisk_log_size":95,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481804,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":864,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":884736,"data_stored":423396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]}]},{"osd":10,"up_from":69,"seq":296352743439,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28400,"kb_used_data":1448,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716752,"statfs":{"total":95995035648,"available":95965954048,"internally_reserved":0,"allocated":1482752,"data_stored":1006039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":1,"last update":"Fri Dec 1 01:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.438}]},{"osd":2,"last update":"Fri Dec 1 01:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.54000000000000004}]},{"osd":3,"last update":"Fri Dec 1 01:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":5.7240000000000002,"5min":5.7240000000000002,"15min":5.7240000000000002},"last":0.56799999999999995}]},{"osd":4,"last update":"Fri Dec 1 01:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.59899999999999998}]},{"osd":5,"last update":"Fri Dec 1 01:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.50800000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":5.7510000000000003,"5min":5.7510000000000003,"15min":5.7510000000000003},"last":0.66300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":8,"last update":"Fri Dec 1 01:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.63},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.69499999999999995}]},{"osd":9,"last update":"Fri Dec 1 01:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.71199999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]}]},{"osd":9,"up_from":64,"seq":274877906965,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":905216,"data_stored":423722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.374},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.40000000000000002}]},{"osd":1,"last update":"Fri Dec 1 01:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.64300000000000002}]},{"osd":2,"last update":"Fri Dec 1 01:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.55400000000000005}]},{"osd":3,"last update":"Fri Dec 1 01:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.42699999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.67800000000000005}]},{"osd":5,"last update":"Fri Dec 1 01:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.626},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.70999999999999996}]},{"osd":6,"last update":"Fri Dec 1 01:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.72599999999999998}]},{"osd":7,"last update":"Fri Dec 1 01:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.749}]},{"osd":8,"last update":"Fri Dec 1 01:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.76900000000000002}]},{"osd":10,"last update":"Fri Dec 1 01:16:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.80100000000000005}]}]},{"osd":8,"up_from":57,"seq":244813135896,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":901120,"data_stored":423552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:15:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.505},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.48399999999999999}]},{"osd":1,"last update":"Fri Dec 1 01:15:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.63700000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:15:42 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.71199999999999997}]},{"osd":3,"last update":"Fri Dec 1 01:15:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.66000000000000003}]},{"osd":4,"last update":"Fri Dec 1 01:15:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.73499999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:15:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.87}]},{"osd":6,"last update":"Fri Dec 1 01:15:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.83799999999999997}]},{"osd":7,"last update":"Fri Dec 1 01:15:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.82399999999999995}]},{"osd":9,"last update":"Fri Dec 1 01:16:01 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":5.9690000000000003,"5min":5.9690000000000003,"15min":5.9690000000000003},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":3.5259999999999998,"5min":3.5259999999999998,"15min":3.5259999999999998},"last":0.77800000000000002}]},{"osd":10,"last update":"Fri Dec 1 01:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":6.0469999999999997,"5min":6.0469999999999997,"15min":6.0469999999999997},"last":0.93600000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]}]},{"osd":7,"up_from":50,"seq":214748364828,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28396,"kb_used_data":1444,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716756,"statfs":{"total":95995035648,"available":95965958144,"internally_reserved":0,"allocated":1478656,"data_stored":1013764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.45000000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.40100000000000002,"15min":0.39300000000000002},"min":{"1min":0.27700000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.436},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.41099999999999998,"15min":0.40200000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.26900000000000002}]},{"osd":2,"last update":"Fri Dec 1 01:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56200000000000006,"15min":0.55800000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.64300000000000002}]},{"osd":3,"last update":"Fri Dec 1 01:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.55800000000000005,"15min":0.55000000000000004},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.77700000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61899999999999999,"15min":0.61599999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.85099999999999998,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.45000000000000001}]},{"osd":4,"last update":"Fri Dec 1 01:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.51600000000000001,"15min":0.51200000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.51600000000000001,"15min":0.51100000000000001},"min":{"1min":0.32200000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.35899999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.435,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.61499999999999999,"15min":0.60699999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.56000000000000005}]},{"osd":6,"last update":"Fri Dec 1 01:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.64800000000000002,"15min":0.64300000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.66400000000000003,"15min":0.65800000000000003},"min":{"1min":0.42099999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0600000000000001,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.42099999999999999}]},{"osd":8,"last update":"Fri Dec 1 01:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.63200000000000001}]},{"osd":9,"last update":"Fri Dec 1 01:16:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.40899999999999997}]},{"osd":10,"last update":"Fri Dec 1 01:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.52500000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]}]},{"osd":6,"up_from":45,"seq":193273528353,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":860,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":880640,"data_stored":422814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.41199999999999998,"15min":0.40699999999999997},"min":{"1min":0.29799999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.437,"5min":0.41399999999999998,"15min":0.40999999999999998},"min":{"1min":0.29899999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.45500000000000002}]},{"osd":1,"last update":"Fri Dec 1 01:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.623,"15min":0.626},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82199999999999995,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":1.5840000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.59099999999999997,"15min":0.59899999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.86899999999999999,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.68100000000000005}]},{"osd":2,"last update":"Fri Dec 1 01:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57699999999999996,"15min":0.57099999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.629,"15min":0.623},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":1.506}]},{"osd":3,"last update":"Fri Dec 1 01:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.47899999999999998,"15min":0.46800000000000003},"min":{"1min":0.35399999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":1.456},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.46700000000000003,"15min":0.46400000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.48499999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.83899999999999997,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.65600000000000003,"15min":0.65200000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":1.6830000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63800000000000001,"15min":0.64000000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.871,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.65800000000000003,"15min":0.65200000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.65200000000000002}]},{"osd":7,"last update":"Fri Dec 1 01:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.69999999999999996,"15min":0.70399999999999996},"min":{"1min":0.45600000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92000000000000004,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.93300000000000005,"5min":0.995,"15min":0.995},"last":1.6100000000000001}]},{"osd":8,"last update":"Fri Dec 1 01:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":1.635},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.76100000000000001}]},{"osd":9,"last update":"Fri Dec 1 01:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":1.653}]},{"osd":10,"last update":"Fri Dec 1 01:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":1.712}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":892928,"data_stored":424378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:16:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.39500000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.79300000000000004,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53200000000000003,"15min":0.51900000000000002},"min":{"1min":0.32800000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.69299999999999995,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.38100000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.51100000000000001,"15min":0.48499999999999999},"min":{"1min":0.52200000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.52400000000000002,"15min":0.51600000000000001},"min":{"1min":0.47099999999999997,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78400000000000003,"5min":1.115,"15min":1.115},"last":0.61099999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.57699999999999996,"15min":0.55700000000000005},"min":{"1min":0.48899999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88800000000000001,"5min":1.198,"15min":1.198},"last":0.628},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.59399999999999997,"15min":0.57199999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.47999999999999998}]},{"osd":4,"last update":"Fri Dec 1 01:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.42899999999999999,"15min":0.41499999999999998},"min":{"1min":0.30299999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.70799999999999996,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.497,"15min":0.48099999999999998},"min":{"1min":0.36599999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.91100000000000003,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.60199999999999998}]},{"osd":5,"last update":"Fri Dec 1 01:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.53400000000000003,"15min":0.51000000000000001},"min":{"1min":0.45200000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.53100000000000003,"15min":0.51100000000000001},"min":{"1min":0.53000000000000003,"5min":0.316,"15min":0.316},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.63800000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.60299999999999998,"15min":0.59499999999999997},"min":{"1min":0.43099999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.622,"15min":0.61399999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.46400000000000002}]},{"osd":7,"last update":"Fri Dec 1 01:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.55400000000000005,"15min":0.53800000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.88100000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.56399999999999995,"15min":0.54600000000000004},"min":{"1min":0.40300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.878,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.56299999999999994}]},{"osd":8,"last update":"Fri Dec 1 01:15:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.621}]},{"osd":9,"last update":"Fri Dec 1 01:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.69799999999999995}]},{"osd":10,"last update":"Fri Dec 1 01:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1,"5min":1,"15min":1},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.68799999999999994}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738427,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":864,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":884736,"data_stored":422904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 01:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.498,"15min":0.46800000000000003},"min":{"1min":0.35699999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.80600000000000005,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.53000000000000003,"15min":0.49099999999999999},"min":{"1min":0.39500000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.879,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":1.5229999999999999}]},{"osd":2,"last update":"Fri Dec 1 01:15:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.51500000000000001,"15min":0.495},"min":{"1min":0.39300000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":3.4590000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.51200000000000001,"15min":0.5},"min":{"1min":0.35899999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":3.1429999999999998}]},{"osd":3,"last update":"Fri Dec 1 01:14:57 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.36699999999999999,"15min":0.35899999999999999},"min":{"1min":0.247,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.441},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.40799999999999997,"15min":0.40200000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.41599999999999998}]},{"osd":4,"last update":"Fri Dec 1 01:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54300000000000004,"15min":0.53300000000000003},"min":{"1min":0.40300000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.81200000000000006,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.58999999999999997,"15min":0.57499999999999996},"min":{"1min":0.42299999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.92300000000000004,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":1.5649999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.41099999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.83199999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":1.2470000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.60899999999999999,"15min":0.59999999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":1.708}]},{"osd":6,"last update":"Fri Dec 1 01:15:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.54300000000000004,"15min":0.53700000000000003},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":1.679},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.55900000000000005,"15min":0.55300000000000005},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.51900000000000002}]},{"osd":7,"last update":"Fri Dec 1 01:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.623,"15min":0.621},"min":{"1min":0.43099999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67800000000000005,"15min":0.68200000000000005},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":1.6220000000000001}]},{"osd":8,"last update":"Fri Dec 1 01:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":1.431},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":1.4530000000000001}]},{"osd":9,"last update":"Fri Dec 1 01:16:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":1.595},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.54300000000000004}]},{"osd":10,"last update":"Fri Dec 1 01:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.999},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":1.6479999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8740000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411378,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":905216,"data_stored":424986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:15:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.49399999999999999,"15min":0.47899999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.76400000000000001,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.49099999999999999,"15min":0.47699999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.503}]},{"osd":1,"last update":"Fri Dec 1 01:15:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.51600000000000001,"15min":0.50600000000000001},"min":{"1min":0.39100000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.38800000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.438}]},{"osd":3,"last update":"Fri Dec 1 01:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.58599999999999997,"15min":0.52600000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":5.4710000000000001,"5min":5.4710000000000001,"15min":5.4710000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.55400000000000005,"15min":0.52400000000000002},"min":{"1min":0.40899999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.76700000000000002,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.58499999999999996}]},{"osd":4,"last update":"Fri Dec 1 01:16:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.53200000000000003,"15min":0.51800000000000002},"min":{"1min":0.378,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.623},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.53100000000000003,"15min":0.52900000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.45700000000000002}]},{"osd":5,"last update":"Fri Dec 1 01:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.41599999999999998,"15min":0.41499999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.47199999999999998,"15min":0.46400000000000002},"min":{"1min":0.29899999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.497}]},{"osd":6,"last update":"Fri Dec 1 01:16:00 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.64300000000000002,"15min":0.60399999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":5.5019999999999998,"5min":5.5019999999999998,"15min":5.5019999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.63800000000000001,"15min":0.63400000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.56799999999999995}]},{"osd":7,"last update":"Fri Dec 1 01:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.54000000000000004}]},{"osd":8,"last update":"Fri Dec 1 01:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.47899999999999998}]},{"osd":9,"last update":"Fri Dec 1 01:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.63800000000000001}]},{"osd":10,"last update":"Fri Dec 1 01:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":5.4450000000000003,"5min":5.4450000000000003,"15min":5.4450000000000003},"last":0.67400000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]}]},{"osd":3,"up_from":25,"seq":107374182446,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":905216,"data_stored":423222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:16:00 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.33400000000000002,"15min":0.316},"min":{"1min":0.24299999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.35299999999999998,"15min":0.33700000000000002},"min":{"1min":0.29199999999999998,"5min":0.224,"15min":0.224},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.35299999999999998}]},{"osd":1,"last update":"Fri Dec 1 01:16:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.52200000000000002,"15min":0.5},"min":{"1min":0.376,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.52300000000000002,"15min":0.51800000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.60599999999999998}]},{"osd":2,"last update":"Fri Dec 1 01:16:00 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53100000000000003,"15min":0.51500000000000001},"min":{"1min":0.42599999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.74099999999999999,"5min":0.88,"15min":0.88},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.54200000000000004,"15min":0.52500000000000002},"min":{"1min":0.41399999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.432}]},{"osd":4,"last update":"Fri Dec 1 01:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.57799999999999996,"15min":0.55600000000000005},"min":{"1min":0.40899999999999997,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.60399999999999998,"15min":0.58699999999999997},"min":{"1min":0.44500000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.623}]},{"osd":5,"last update":"Fri Dec 1 01:15:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56100000000000005,"15min":0.55600000000000005},"min":{"1min":0.443,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.58299999999999996,"15min":0.57599999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.52100000000000002}]},{"osd":6,"last update":"Fri Dec 1 01:16:00 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.373},{"interface":"front","average":{"1min":0.627,"5min":0.55900000000000005,"15min":0.54700000000000004},"min":{"1min":0.35499999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.50600000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.63300000000000001,"15min":0.63100000000000001},"min":{"1min":0.46300000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.82999999999999996,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.65600000000000003,"15min":0.65100000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.57299999999999995}]},{"osd":8,"last update":"Fri Dec 1 01:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.51200000000000001}]},{"osd":9,"last update":"Fri Dec 1 01:16:05 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.53900000000000003}]},{"osd":10,"last update":"Fri Dec 1 01:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.67900000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]}]},{"osd":4,"up_from":32,"seq":137438953514,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":864,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":884736,"data_stored":422836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.54300000000000004,"15min":0.50900000000000001},"min":{"1min":0.48799999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.373},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.53400000000000003,"15min":0.499},"min":{"1min":0.47199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.497}]},{"osd":1,"last update":"Fri Dec 1 01:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.40500000000000003,"15min":0.378},"min":{"1min":0.29599999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.39800000000000002,"15min":0.371},"min":{"1min":0.32100000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.40500000000000003}]},{"osd":2,"last update":"Fri Dec 1 01:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.55900000000000005,"15min":0.53000000000000003},"min":{"1min":0.48199999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.59499999999999997,"15min":0.56799999999999995},"min":{"1min":0.40400000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.751}]},{"osd":3,"last update":"Fri Dec 1 01:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.53300000000000003,"15min":0.499},"min":{"1min":0.45700000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.97899999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.52200000000000002,"15min":0.504},"min":{"1min":0.373,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.89300000000000002,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.54800000000000004}]},{"osd":5,"last update":"Fri Dec 1 01:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.56699999999999995,"15min":0.55500000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.745,"5min":0.61899999999999999,"15min":0.59799999999999998},"min":{"1min":0.52800000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.78600000000000003}]},{"osd":6,"last update":"Fri Dec 1 01:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.61699999999999999,"15min":0.60799999999999998},"min":{"1min":0.46300000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.747,"5min":0.67100000000000004,"15min":0.65900000000000003},"min":{"1min":0.47899999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.64900000000000002}]},{"osd":7,"last update":"Fri Dec 1 01:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.57699999999999996,"15min":0.55700000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.503},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.56599999999999995,"15min":0.55000000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.48099999999999998}]},{"osd":8,"last update":"Fri Dec 1 01:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.81200000000000006}]},{"osd":9,"last update":"Fri Dec 1 01:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.70199999999999996}]},{"osd":10,"last update":"Fri Dec 1 01:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.52800000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]}]},{"osd":5,"up_from":37,"seq":158913789989,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28416,"kb_used_data":1464,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716736,"statfs":{"total":95995035648,"available":95965937664,"internally_reserved":0,"allocated":1499136,"data_stored":1014279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:15:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.51600000000000001,"15min":0.50600000000000001},"min":{"1min":0.378,"5min":0.307,"15min":0.307},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.52700000000000002,"15min":0.52400000000000002},"min":{"1min":0.36699999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.45400000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:15:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.48999999999999999,"15min":0.47999999999999998},"min":{"1min":0.35999999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.50600000000000001,"15min":0.49199999999999999},"min":{"1min":0.439,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.43099999999999999}]},{"osd":2,"last update":"Fri Dec 1 01:15:39 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.38800000000000001,"15min":0.38400000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.377,"15min":0.372},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.34899999999999998}]},{"osd":3,"last update":"Fri Dec 1 01:15:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.53900000000000003,"15min":0.53100000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.53500000000000003,"15min":0.52700000000000002},"min":{"1min":0.39000000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.40600000000000003}]},{"osd":4,"last update":"Fri Dec 1 01:15:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.53700000000000003,"15min":0.52500000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.53100000000000003,"15min":0.52200000000000002},"min":{"1min":0.45200000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.63600000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:15:56 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.629,"15min":0.622},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.63700000000000001,"15min":0.623},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.58599999999999997}]},{"osd":7,"last update":"Fri Dec 1 01:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.64900000000000002,"15min":0.65100000000000002},"min":{"1min":0.49399999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.85299999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.61299999999999999,"15min":0.61099999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.53000000000000003}]},{"osd":8,"last update":"Fri Dec 1 01:15:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.501}]},{"osd":9,"last update":"Fri Dec 1 01:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.48199999999999998}]},{"osd":10,"last update":"Fri Dec 1 01:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.624},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.60999999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T01:16:33.881 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T01:16:34.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:33 smithi203 ceph-mon[103581]: mgrmap e17: x(active, since 6m), standbys: y, z 2023-12-01T01:16:34.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:33 smithi203 ceph-mon[103581]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:16:34.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:33 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/709623757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:16:34.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:33 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/3602740592' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:16:34.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:33 smithi137 ceph-mon[103312]: mgrmap e17: x(active, since 6m), standbys: y, z 2023-12-01T01:16:34.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:33 smithi137 ceph-mon[103312]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:16:34.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:33 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/709623757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:16:34.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:33 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/3602740592' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:16:34.034 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.033+0000 7f02c11e3700 1 -- 172.21.15.77:0/1256873389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02bc101f80 msgr2=0x7f02bc1023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:34.034 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.033+0000 7f02c11e3700 1 --2- 172.21.15.77:0/1256873389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02bc101f80 0x7f02bc1023e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f02b00099c0 tx=0x7f02b0009cd0 comp rx=0 tx=0).stop 2023-12-01T01:16:34.035 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.033+0000 7f02c11e3700 1 -- 172.21.15.77:0/1256873389 shutdown_connections 2023-12-01T01:16:34.035 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.033+0000 7f02c11e3700 1 --2- 172.21.15.77:0/1256873389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02bc102920 0x7f02bc1091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:34.035 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.033+0000 7f02c11e3700 1 --2- 172.21.15.77:0/1256873389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02bc100d90 0x7f02bc1011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:34.035 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.033+0000 7f02c11e3700 1 --2- 172.21.15.77:0/1256873389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02bc101f80 0x7f02bc1023e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:34.035 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.033+0000 7f02c11e3700 1 -- 172.21.15.77:0/1256873389 >> 172.21.15.77:0/1256873389 conn(0x7f02bc0fc2e0 msgr2=0x7f02bc0fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:34.035 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.033+0000 7f02c11e3700 1 -- 172.21.15.77:0/1256873389 shutdown_connections 2023-12-01T01:16:34.035 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.033+0000 7f02c11e3700 1 -- 172.21.15.77:0/1256873389 wait complete. 2023-12-01T01:16:34.035 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.034+0000 7f02c11e3700 1 Processor -- start 2023-12-01T01:16:34.037 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.034+0000 7f02c11e3700 1 -- start start 2023-12-01T01:16:34.037 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.034+0000 7f02c11e3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02bc100d90 0x7f02bc196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:34.037 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.034+0000 7f02c11e3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02bc101f80 0x7f02bc196ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:34.037 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.034+0000 7f02c11e3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02bc102920 0x7f02bc19df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:34.037 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.034+0000 7f02c11e3700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02bc19e4e0 con 0x7f02bc102920 2023-12-01T01:16:34.037 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.034+0000 7f02c11e3700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02bc19e620 con 0x7f02bc101f80 2023-12-01T01:16:34.037 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.034+0000 7f02c11e3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02bc19e7b0 con 0x7f02bc100d90 2023-12-01T01:16:34.038 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.034+0000 7f02bb59e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02bc102920 0x7f02bc19df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:34.038 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.034+0000 7f02bb59e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02bc102920 0x7f02bc19df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57692/0 (socket says 172.21.15.77:57692) 2023-12-01T01:16:34.038 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.034+0000 7f02bb59e700 1 -- 172.21.15.77:0/1545390442 learned_addr learned my addr 172.21.15.77:0/1545390442 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:34.038 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.034+0000 7f02ba59c700 1 --2- 172.21.15.77:0/1545390442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02bc101f80 0x7f02bc196ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:34.038 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.034+0000 7f02bad9d700 1 --2- 172.21.15.77:0/1545390442 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02bc100d90 0x7f02bc196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:34.038 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.035+0000 7f02bb59e700 1 -- 172.21.15.77:0/1545390442 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02bc100d90 msgr2=0x7f02bc196960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:34.038 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.035+0000 7f02bb59e700 1 --2- 172.21.15.77:0/1545390442 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02bc100d90 0x7f02bc196960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:34.038 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.035+0000 7f02bb59e700 1 -- 172.21.15.77:0/1545390442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02bc101f80 msgr2=0x7f02bc196ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:34.038 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.035+0000 7f02bb59e700 1 --2- 172.21.15.77:0/1545390442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02bc101f80 0x7f02bc196ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:34.038 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.035+0000 7f02bb59e700 1 -- 172.21.15.77:0/1545390442 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b00095f0 con 0x7f02bc102920 2023-12-01T01:16:34.038 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.035+0000 7f02bb59e700 1 --2- 172.21.15.77:0/1545390442 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02bc102920 0x7f02bc19df10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f02b4007d70 tx=0x7f02b400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:34.038 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.035+0000 7f02a3fff700 1 -- 172.21.15.77:0/1545390442 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b400ac90 con 0x7f02bc102920 2023-12-01T01:16:34.038 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.035+0000 7f02c11e3700 1 -- 172.21.15.77:0/1545390442 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02bc19ea90 con 0x7f02bc102920 2023-12-01T01:16:34.038 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.035+0000 7f02a3fff700 1 -- 172.21.15.77:0/1545390442 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02b400adf0 con 0x7f02bc102920 2023-12-01T01:16:34.038 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.035+0000 7f02a3fff700 1 -- 172.21.15.77:0/1545390442 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b40115e0 con 0x7f02bc102920 2023-12-01T01:16:34.039 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.035+0000 7f02c11e3700 1 -- 172.21.15.77:0/1545390442 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02bc19efa0 con 0x7f02bc102920 2023-12-01T01:16:34.039 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.036+0000 7f02c11e3700 1 -- 172.21.15.77:0/1545390442 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02bc108290 con 0x7f02bc102920 2023-12-01T01:16:34.039 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.038+0000 7f02a3fff700 1 -- 172.21.15.77:0/1545390442 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f02b4004ad0 con 0x7f02bc102920 2023-12-01T01:16:34.040 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.038+0000 7f02a3fff700 1 --2- 172.21.15.77:0/1545390442 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f02a40a05f0 0x7f02a40a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:34.040 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.038+0000 7f02bad9d700 1 --2- 172.21.15.77:0/1545390442 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f02a40a05f0 0x7f02a40a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:34.040 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.038+0000 7f02a3fff700 1 -- 172.21.15.77:0/1545390442 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f02b40cbe20 con 0x7f02bc102920 2023-12-01T01:16:34.040 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.039+0000 7f02bad9d700 1 --2- 172.21.15.77:0/1545390442 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f02a40a05f0 0x7f02a40a2ab0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f02bc101de0 tx=0x7f02ac00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:34.042 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.041+0000 7f02a3fff700 1 -- 172.21.15.77:0/1545390442 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f02b4095b20 con 0x7f02bc102920 2023-12-01T01:16:34.231 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.230+0000 7f02c11e3700 1 -- 172.21.15.77:0/1545390442 --> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f02bc0611d0 con 0x7f02a40a05f0 2023-12-01T01:16:34.241 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.240+0000 7f02a3fff700 1 -- 172.21.15.77:0/1545390442 <== mgr.14152 v2:172.21.15.77:6800/3838941679 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+336051 (secure 0 0 0) 0x7f02bc0611d0 con 0x7f02a40a05f0 2023-12-01T01:16:34.244 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:16:34.247 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.243+0000 7f02c11e3700 1 -- 172.21.15.77:0/1545390442 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f02a40a05f0 msgr2=0x7f02a40a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:34.247 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.243+0000 7f02c11e3700 1 --2- 172.21.15.77:0/1545390442 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f02a40a05f0 0x7f02a40a2ab0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f02bc101de0 tx=0x7f02ac00b410 comp rx=0 tx=0).stop 2023-12-01T01:16:34.247 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.243+0000 7f02c11e3700 1 -- 172.21.15.77:0/1545390442 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02bc102920 msgr2=0x7f02bc19df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:34.247 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.243+0000 7f02c11e3700 1 --2- 172.21.15.77:0/1545390442 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02bc102920 0x7f02bc19df10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f02b4007d70 tx=0x7f02b400a340 comp rx=0 tx=0).stop 2023-12-01T01:16:34.247 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.243+0000 7f02c11e3700 1 -- 172.21.15.77:0/1545390442 shutdown_connections 2023-12-01T01:16:34.247 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.243+0000 7f02c11e3700 1 --2- 172.21.15.77:0/1545390442 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f02a40a05f0 0x7f02a40a2ab0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:34.247 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.243+0000 7f02c11e3700 1 --2- 172.21.15.77:0/1545390442 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02bc102920 0x7f02bc19df10 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:34.248 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.243+0000 7f02c11e3700 1 --2- 172.21.15.77:0/1545390442 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02bc100d90 0x7f02bc196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:34.248 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.243+0000 7f02c11e3700 1 --2- 172.21.15.77:0/1545390442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02bc101f80 0x7f02bc196ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:34.248 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.243+0000 7f02c11e3700 1 -- 172.21.15.77:0/1545390442 >> 172.21.15.77:0/1545390442 conn(0x7f02bc0fc2e0 msgr2=0x7f02bc1053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:34.248 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.244+0000 7f02c11e3700 1 -- 172.21.15.77:0/1545390442 shutdown_connections 2023-12-01T01:16:34.248 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:34.244+0000 7f02c11e3700 1 -- 172.21.15.77:0/1545390442 wait complete. 2023-12-01T01:16:34.248 INFO:teuthology.orchestra.run.smithi077.stderr:dumped all 2023-12-01T01:16:34.257 INFO:teuthology.orchestra.run.smithi077.stdout:{"pg_ready":true,"pg_map":{"version":256,"stamp":"2023-12-01T01:16:32.267694+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594692,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":291,"num_write_kb":2966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":260,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335636,"kb_used_data":12212,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606188,"statfs":{"total":1151940427776,"available":1151596736512,"internally_reserved":0,"allocated":12505088,"data_stored":6845892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4324,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.341501"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.592825+0000","last_change":"2023-12-01T01:16:25.373817+0000","last_active":"2023-12-01T01:16:25.592825+0000","last_peered":"2023-12-01T01:16:25.592825+0000","last_clean":"2023-12-01T01:16:25.592825+0000","last_became_active":"2023-12-01T01:16:25.373154+0000","last_became_peered":"2023-12-01T01:16:25.373154+0000","last_unstale":"2023-12-01T01:16:25.592825+0000","last_undegraded":"2023-12-01T01:16:25.592825+0000","last_fullsized":"2023-12-01T01:16:25.592825+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:33:43.284106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.324984+0000","last_change":"2023-12-01T01:16:25.364855+0000","last_active":"2023-12-01T01:16:27.324984+0000","last_peered":"2023-12-01T01:16:27.324984+0000","last_clean":"2023-12-01T01:16:27.324984+0000","last_became_active":"2023-12-01T01:16:25.364536+0000","last_became_peered":"2023-12-01T01:16:25.364536+0000","last_unstale":"2023-12-01T01:16:27.324984+0000","last_undegraded":"2023-12-01T01:16:27.324984+0000","last_fullsized":"2023-12-01T01:16:27.324984+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:45:23.471557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.324702+0000","last_change":"2023-12-01T01:16:25.372443+0000","last_active":"2023-12-01T01:16:27.324702+0000","last_peered":"2023-12-01T01:16:27.324702+0000","last_clean":"2023-12-01T01:16:27.324702+0000","last_became_active":"2023-12-01T01:16:25.372133+0000","last_became_peered":"2023-12-01T01:16:25.372133+0000","last_unstale":"2023-12-01T01:16:27.324702+0000","last_undegraded":"2023-12-01T01:16:27.324702+0000","last_fullsized":"2023-12-01T01:16:27.324702+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:54:19.439408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.554816+0000","last_change":"2023-12-01T01:16:25.365857+0000","last_active":"2023-12-01T01:16:26.554816+0000","last_peered":"2023-12-01T01:16:26.554816+0000","last_clean":"2023-12-01T01:16:26.554816+0000","last_became_active":"2023-12-01T01:16:25.364992+0000","last_became_peered":"2023-12-01T01:16:25.364992+0000","last_unstale":"2023-12-01T01:16:26.554816+0000","last_undegraded":"2023-12-01T01:16:26.554816+0000","last_fullsized":"2023-12-01T01:16:26.554816+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:01:11.064587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.071460+0000","last_change":"2023-12-01T01:16:25.364081+0000","last_active":"2023-12-01T01:16:26.071460+0000","last_peered":"2023-12-01T01:16:26.071460+0000","last_clean":"2023-12-01T01:16:26.071460+0000","last_became_active":"2023-12-01T01:16:25.363187+0000","last_became_peered":"2023-12-01T01:16:25.363187+0000","last_unstale":"2023-12-01T01:16:26.071460+0000","last_undegraded":"2023-12-01T01:16:26.071460+0000","last_fullsized":"2023-12-01T01:16:26.071460+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:05:42.852128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325515+0000","last_change":"2023-12-01T01:16:25.366104+0000","last_active":"2023-12-01T01:16:27.325515+0000","last_peered":"2023-12-01T01:16:27.325515+0000","last_clean":"2023-12-01T01:16:27.325515+0000","last_became_active":"2023-12-01T01:16:25.365778+0000","last_became_peered":"2023-12-01T01:16:25.365778+0000","last_unstale":"2023-12-01T01:16:27.325515+0000","last_undegraded":"2023-12-01T01:16:27.325515+0000","last_fullsized":"2023-12-01T01:16:27.325515+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:30:09.485609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.349881+0000","last_change":"2023-12-01T01:16:25.371354+0000","last_active":"2023-12-01T01:16:26.349881+0000","last_peered":"2023-12-01T01:16:26.349881+0000","last_clean":"2023-12-01T01:16:26.349881+0000","last_became_active":"2023-12-01T01:16:25.370955+0000","last_became_peered":"2023-12-01T01:16:25.370955+0000","last_unstale":"2023-12-01T01:16:26.349881+0000","last_undegraded":"2023-12-01T01:16:26.349881+0000","last_fullsized":"2023-12-01T01:16:26.349881+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:38:53.508285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.571448+0000","last_change":"2023-12-01T01:16:25.368187+0000","last_active":"2023-12-01T01:16:26.571448+0000","last_peered":"2023-12-01T01:16:26.571448+0000","last_clean":"2023-12-01T01:16:26.571448+0000","last_became_active":"2023-12-01T01:16:25.367854+0000","last_became_peered":"2023-12-01T01:16:25.367854+0000","last_unstale":"2023-12-01T01:16:26.571448+0000","last_undegraded":"2023-12-01T01:16:26.571448+0000","last_fullsized":"2023-12-01T01:16:26.571448+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:22:35.159185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325090+0000","last_change":"2023-12-01T01:16:25.372829+0000","last_active":"2023-12-01T01:16:27.325090+0000","last_peered":"2023-12-01T01:16:27.325090+0000","last_clean":"2023-12-01T01:16:27.325090+0000","last_became_active":"2023-12-01T01:16:25.372469+0000","last_became_peered":"2023-12-01T01:16:25.372469+0000","last_unstale":"2023-12-01T01:16:27.325090+0000","last_undegraded":"2023-12-01T01:16:27.325090+0000","last_fullsized":"2023-12-01T01:16:27.325090+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:25:10.300138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325829+0000","last_change":"2023-12-01T01:16:25.379135+0000","last_active":"2023-12-01T01:16:27.325829+0000","last_peered":"2023-12-01T01:16:27.325829+0000","last_clean":"2023-12-01T01:16:27.325829+0000","last_became_active":"2023-12-01T01:16:25.378852+0000","last_became_peered":"2023-12-01T01:16:25.378852+0000","last_unstale":"2023-12-01T01:16:27.325829+0000","last_undegraded":"2023-12-01T01:16:27.325829+0000","last_fullsized":"2023-12-01T01:16:27.325829+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:42:16.803389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.350179+0000","last_change":"2023-12-01T01:16:25.367602+0000","last_active":"2023-12-01T01:16:26.350179+0000","last_peered":"2023-12-01T01:16:26.350179+0000","last_clean":"2023-12-01T01:16:26.350179+0000","last_became_active":"2023-12-01T01:16:25.366565+0000","last_became_peered":"2023-12-01T01:16:25.366565+0000","last_unstale":"2023-12-01T01:16:26.350179+0000","last_undegraded":"2023-12-01T01:16:26.350179+0000","last_fullsized":"2023-12-01T01:16:26.350179+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:28:52.576838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.554806+0000","last_change":"2023-12-01T01:16:25.375004+0000","last_active":"2023-12-01T01:16:26.554806+0000","last_peered":"2023-12-01T01:16:26.554806+0000","last_clean":"2023-12-01T01:16:26.554806+0000","last_became_active":"2023-12-01T01:16:25.374320+0000","last_became_peered":"2023-12-01T01:16:25.374320+0000","last_unstale":"2023-12-01T01:16:26.554806+0000","last_undegraded":"2023-12-01T01:16:26.554806+0000","last_fullsized":"2023-12-01T01:16:26.554806+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:40:49.482061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.352492+0000","last_change":"2023-12-01T01:16:25.367858+0000","last_active":"2023-12-01T01:16:26.352492+0000","last_peered":"2023-12-01T01:16:26.352492+0000","last_clean":"2023-12-01T01:16:26.352492+0000","last_became_active":"2023-12-01T01:16:25.367560+0000","last_became_peered":"2023-12-01T01:16:25.367560+0000","last_unstale":"2023-12-01T01:16:26.352492+0000","last_undegraded":"2023-12-01T01:16:26.352492+0000","last_fullsized":"2023-12-01T01:16:26.352492+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:33:21.048672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.324783+0000","last_change":"2023-12-01T01:16:25.375654+0000","last_active":"2023-12-01T01:16:27.324783+0000","last_peered":"2023-12-01T01:16:27.324783+0000","last_clean":"2023-12-01T01:16:27.324783+0000","last_became_active":"2023-12-01T01:16:25.375379+0000","last_became_peered":"2023-12-01T01:16:25.375379+0000","last_unstale":"2023-12-01T01:16:27.324783+0000","last_undegraded":"2023-12-01T01:16:27.324783+0000","last_fullsized":"2023-12-01T01:16:27.324783+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:33:42.335365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325759+0000","last_change":"2023-12-01T01:16:25.372546+0000","last_active":"2023-12-01T01:16:27.325759+0000","last_peered":"2023-12-01T01:16:27.325759+0000","last_clean":"2023-12-01T01:16:27.325759+0000","last_became_active":"2023-12-01T01:16:25.372327+0000","last_became_peered":"2023-12-01T01:16:25.372327+0000","last_unstale":"2023-12-01T01:16:27.325759+0000","last_undegraded":"2023-12-01T01:16:27.325759+0000","last_fullsized":"2023-12-01T01:16:27.325759+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:12:34.171834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.324599+0000","last_change":"2023-12-01T01:16:25.366449+0000","last_active":"2023-12-01T01:16:27.324599+0000","last_peered":"2023-12-01T01:16:27.324599+0000","last_clean":"2023-12-01T01:16:27.324599+0000","last_became_active":"2023-12-01T01:16:25.366147+0000","last_became_peered":"2023-12-01T01:16:25.366147+0000","last_unstale":"2023-12-01T01:16:27.324599+0000","last_undegraded":"2023-12-01T01:16:27.324599+0000","last_fullsized":"2023-12-01T01:16:27.324599+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:01:29.670329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.592834+0000","last_change":"2023-12-01T01:16:25.374016+0000","last_active":"2023-12-01T01:16:25.592834+0000","last_peered":"2023-12-01T01:16:25.592834+0000","last_clean":"2023-12-01T01:16:25.592834+0000","last_became_active":"2023-12-01T01:16:25.373543+0000","last_became_peered":"2023-12-01T01:16:25.373543+0000","last_unstale":"2023-12-01T01:16:25.592834+0000","last_undegraded":"2023-12-01T01:16:25.592834+0000","last_fullsized":"2023-12-01T01:16:25.592834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:36:22.858446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.584312+0000","last_change":"2023-12-01T01:16:25.363598+0000","last_active":"2023-12-01T01:16:26.584312+0000","last_peered":"2023-12-01T01:16:26.584312+0000","last_clean":"2023-12-01T01:16:26.584312+0000","last_became_active":"2023-12-01T01:16:25.363111+0000","last_became_peered":"2023-12-01T01:16:25.363111+0000","last_unstale":"2023-12-01T01:16:26.584312+0000","last_undegraded":"2023-12-01T01:16:26.584312+0000","last_fullsized":"2023-12-01T01:16:26.584312+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:42:08.171789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.375829+0000","last_change":"2023-12-01T01:16:25.368446+0000","last_active":"2023-12-01T01:16:25.375829+0000","last_peered":"2023-12-01T01:16:25.375829+0000","last_clean":"2023-12-01T01:16:25.375829+0000","last_became_active":"2023-12-01T01:16:25.361893+0000","last_became_peered":"2023-12-01T01:16:25.361893+0000","last_unstale":"2023-12-01T01:16:25.375829+0000","last_undegraded":"2023-12-01T01:16:25.375829+0000","last_fullsized":"2023-12-01T01:16:25.375829+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:05:45.697338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'11","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:30.651957+0000","last_change":"2023-12-01T01:16:24.934939+0000","last_active":"2023-12-01T01:16:30.651957+0000","last_peered":"2023-12-01T01:16:30.651957+0000","last_clean":"2023-12-01T01:16:30.651957+0000","last_became_active":"2023-12-01T01:16:24.934481+0000","last_became_peered":"2023-12-01T01:16:24.934481+0000","last_unstale":"2023-12-01T01:16:30.651957+0000","last_undegraded":"2023-12-01T01:16:30.651957+0000","last_fullsized":"2023-12-01T01:16:30.651957+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:39:49.804567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.571536+0000","last_change":"2023-12-01T01:16:25.364061+0000","last_active":"2023-12-01T01:16:26.571536+0000","last_peered":"2023-12-01T01:16:26.571536+0000","last_clean":"2023-12-01T01:16:26.571536+0000","last_became_active":"2023-12-01T01:16:25.363678+0000","last_became_peered":"2023-12-01T01:16:25.363678+0000","last_unstale":"2023-12-01T01:16:26.571536+0000","last_undegraded":"2023-12-01T01:16:26.571536+0000","last_fullsized":"2023-12-01T01:16:26.571536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:24:07.107604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325949+0000","last_change":"2023-12-01T01:16:24.939338+0000","last_active":"2023-12-01T01:16:27.325949+0000","last_peered":"2023-12-01T01:16:27.325949+0000","last_clean":"2023-12-01T01:16:27.325949+0000","last_became_active":"2023-12-01T01:16:24.939007+0000","last_became_peered":"2023-12-01T01:16:24.939007+0000","last_unstale":"2023-12-01T01:16:27.325949+0000","last_undegraded":"2023-12-01T01:16:27.325949+0000","last_fullsized":"2023-12-01T01:16:27.325949+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:56:50.732188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325303+0000","last_change":"2023-12-01T01:16:25.362231+0000","last_active":"2023-12-01T01:16:27.325303+0000","last_peered":"2023-12-01T01:16:27.325303+0000","last_clean":"2023-12-01T01:16:27.325303+0000","last_became_active":"2023-12-01T01:16:25.361920+0000","last_became_peered":"2023-12-01T01:16:25.361920+0000","last_unstale":"2023-12-01T01:16:27.325303+0000","last_undegraded":"2023-12-01T01:16:27.325303+0000","last_fullsized":"2023-12-01T01:16:27.325303+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:22:22.104480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'7","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:30.653095+0000","last_change":"2023-12-01T01:16:24.931801+0000","last_active":"2023-12-01T01:16:30.653095+0000","last_peered":"2023-12-01T01:16:30.653095+0000","last_clean":"2023-12-01T01:16:30.653095+0000","last_became_active":"2023-12-01T01:16:24.931451+0000","last_became_peered":"2023-12-01T01:16:24.931451+0000","last_unstale":"2023-12-01T01:16:30.653095+0000","last_undegraded":"2023-12-01T01:16:30.653095+0000","last_fullsized":"2023-12-01T01:16:30.653095+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:22:41.666502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.324789+0000","last_change":"2023-12-01T01:16:25.361789+0000","last_active":"2023-12-01T01:16:27.324789+0000","last_peered":"2023-12-01T01:16:27.324789+0000","last_clean":"2023-12-01T01:16:27.324789+0000","last_became_active":"2023-12-01T01:16:25.361480+0000","last_became_peered":"2023-12-01T01:16:25.361480+0000","last_unstale":"2023-12-01T01:16:27.324789+0000","last_undegraded":"2023-12-01T01:16:27.324789+0000","last_fullsized":"2023-12-01T01:16:27.324789+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:07:00.742167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.071493+0000","last_change":"2023-12-01T01:16:24.933362+0000","last_active":"2023-12-01T01:16:26.071493+0000","last_peered":"2023-12-01T01:16:26.071493+0000","last_clean":"2023-12-01T01:16:26.071493+0000","last_became_active":"2023-12-01T01:16:24.933049+0000","last_became_peered":"2023-12-01T01:16:24.933049+0000","last_unstale":"2023-12-01T01:16:26.071493+0000","last_undegraded":"2023-12-01T01:16:26.071493+0000","last_fullsized":"2023-12-01T01:16:26.071493+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:47:37.153344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325647+0000","last_change":"2023-12-01T01:16:25.364419+0000","last_active":"2023-12-01T01:16:27.325647+0000","last_peered":"2023-12-01T01:16:27.325647+0000","last_clean":"2023-12-01T01:16:27.325647+0000","last_became_active":"2023-12-01T01:16:25.363022+0000","last_became_peered":"2023-12-01T01:16:25.363022+0000","last_unstale":"2023-12-01T01:16:27.325647+0000","last_undegraded":"2023-12-01T01:16:27.325647+0000","last_fullsized":"2023-12-01T01:16:27.325647+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:58:45.660625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:28.632500+0000","last_change":"2023-12-01T01:16:24.935947+0000","last_active":"2023-12-01T01:16:28.632500+0000","last_peered":"2023-12-01T01:16:28.632500+0000","last_clean":"2023-12-01T01:16:28.632500+0000","last_became_active":"2023-12-01T01:16:24.935061+0000","last_became_peered":"2023-12-01T01:16:24.935061+0000","last_unstale":"2023-12-01T01:16:28.632500+0000","last_undegraded":"2023-12-01T01:16:28.632500+0000","last_fullsized":"2023-12-01T01:16:28.632500+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:00:31.551362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.071450+0000","last_change":"2023-12-01T01:16:25.363229+0000","last_active":"2023-12-01T01:16:26.071450+0000","last_peered":"2023-12-01T01:16:26.071450+0000","last_clean":"2023-12-01T01:16:26.071450+0000","last_became_active":"2023-12-01T01:16:25.362770+0000","last_became_peered":"2023-12-01T01:16:25.362770+0000","last_unstale":"2023-12-01T01:16:26.071450+0000","last_undegraded":"2023-12-01T01:16:26.071450+0000","last_fullsized":"2023-12-01T01:16:26.071450+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:45:02.782201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325488+0000","last_change":"2023-12-01T01:16:24.930509+0000","last_active":"2023-12-01T01:16:27.325488+0000","last_peered":"2023-12-01T01:16:27.325488+0000","last_clean":"2023-12-01T01:16:27.325488+0000","last_became_active":"2023-12-01T01:16:24.930188+0000","last_became_peered":"2023-12-01T01:16:24.930188+0000","last_unstale":"2023-12-01T01:16:27.325488+0000","last_undegraded":"2023-12-01T01:16:27.325488+0000","last_fullsized":"2023-12-01T01:16:27.325488+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:04:17.748363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325530+0000","last_change":"2023-12-01T01:16:25.363383+0000","last_active":"2023-12-01T01:16:27.325530+0000","last_peered":"2023-12-01T01:16:27.325530+0000","last_clean":"2023-12-01T01:16:27.325530+0000","last_became_active":"2023-12-01T01:16:25.363085+0000","last_became_peered":"2023-12-01T01:16:25.363085+0000","last_unstale":"2023-12-01T01:16:27.325530+0000","last_undegraded":"2023-12-01T01:16:27.325530+0000","last_fullsized":"2023-12-01T01:16:27.325530+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:34:36.921706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.352206+0000","last_change":"2023-12-01T01:16:24.929081+0000","last_active":"2023-12-01T01:16:26.352206+0000","last_peered":"2023-12-01T01:16:26.352206+0000","last_clean":"2023-12-01T01:16:26.352206+0000","last_became_active":"2023-12-01T01:16:24.928747+0000","last_became_peered":"2023-12-01T01:16:24.928747+0000","last_unstale":"2023-12-01T01:16:26.352206+0000","last_undegraded":"2023-12-01T01:16:26.352206+0000","last_fullsized":"2023-12-01T01:16:26.352206+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:02:24.727835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.571467+0000","last_change":"2023-12-01T01:16:25.360959+0000","last_active":"2023-12-01T01:16:26.571467+0000","last_peered":"2023-12-01T01:16:26.571467+0000","last_clean":"2023-12-01T01:16:26.571467+0000","last_became_active":"2023-12-01T01:16:25.360647+0000","last_became_peered":"2023-12-01T01:16:25.360647+0000","last_unstale":"2023-12-01T01:16:26.571467+0000","last_undegraded":"2023-12-01T01:16:26.571467+0000","last_fullsized":"2023-12-01T01:16:26.571467+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:50:24.151273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325262+0000","last_change":"2023-12-01T01:16:24.938322+0000","last_active":"2023-12-01T01:16:27.325262+0000","last_peered":"2023-12-01T01:16:27.325262+0000","last_clean":"2023-12-01T01:16:27.325262+0000","last_became_active":"2023-12-01T01:16:24.937747+0000","last_became_peered":"2023-12-01T01:16:24.937747+0000","last_unstale":"2023-12-01T01:16:27.325262+0000","last_undegraded":"2023-12-01T01:16:27.325262+0000","last_fullsized":"2023-12-01T01:16:27.325262+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:37:59.999748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.584270+0000","last_change":"2023-12-01T01:16:25.362779+0000","last_active":"2023-12-01T01:16:26.584270+0000","last_peered":"2023-12-01T01:16:26.584270+0000","last_clean":"2023-12-01T01:16:26.584270+0000","last_became_active":"2023-12-01T01:16:25.362487+0000","last_became_peered":"2023-12-01T01:16:25.362487+0000","last_unstale":"2023-12-01T01:16:26.584270+0000","last_undegraded":"2023-12-01T01:16:26.584270+0000","last_fullsized":"2023-12-01T01:16:26.584270+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:04:42.409498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325796+0000","last_change":"2023-12-01T01:16:24.933187+0000","last_active":"2023-12-01T01:16:27.325796+0000","last_peered":"2023-12-01T01:16:27.325796+0000","last_clean":"2023-12-01T01:16:27.325796+0000","last_became_active":"2023-12-01T01:16:24.932819+0000","last_became_peered":"2023-12-01T01:16:24.932819+0000","last_unstale":"2023-12-01T01:16:27.325796+0000","last_undegraded":"2023-12-01T01:16:27.325796+0000","last_fullsized":"2023-12-01T01:16:27.325796+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:54:34.739815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.324904+0000","last_change":"2023-12-01T01:16:25.361183+0000","last_active":"2023-12-01T01:16:27.324904+0000","last_peered":"2023-12-01T01:16:27.324904+0000","last_clean":"2023-12-01T01:16:27.324904+0000","last_became_active":"2023-12-01T01:16:25.360871+0000","last_became_peered":"2023-12-01T01:16:25.360871+0000","last_unstale":"2023-12-01T01:16:27.324904+0000","last_undegraded":"2023-12-01T01:16:27.324904+0000","last_fullsized":"2023-12-01T01:16:27.324904+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:06:04.748159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.350065+0000","last_change":"2023-12-01T01:16:24.939355+0000","last_active":"2023-12-01T01:16:26.350065+0000","last_peered":"2023-12-01T01:16:26.350065+0000","last_clean":"2023-12-01T01:16:26.350065+0000","last_became_active":"2023-12-01T01:16:24.938967+0000","last_became_peered":"2023-12-01T01:16:24.938967+0000","last_unstale":"2023-12-01T01:16:26.350065+0000","last_undegraded":"2023-12-01T01:16:26.350065+0000","last_fullsized":"2023-12-01T01:16:26.350065+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:20:51.172753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325217+0000","last_change":"2023-12-01T01:16:25.366752+0000","last_active":"2023-12-01T01:16:27.325217+0000","last_peered":"2023-12-01T01:16:27.325217+0000","last_clean":"2023-12-01T01:16:27.325217+0000","last_became_active":"2023-12-01T01:16:25.366466+0000","last_became_peered":"2023-12-01T01:16:25.366466+0000","last_unstale":"2023-12-01T01:16:27.325217+0000","last_undegraded":"2023-12-01T01:16:27.325217+0000","last_fullsized":"2023-12-01T01:16:27.325217+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:09:53.716127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325703+0000","last_change":"2023-12-01T01:16:24.940827+0000","last_active":"2023-12-01T01:16:27.325703+0000","last_peered":"2023-12-01T01:16:27.325703+0000","last_clean":"2023-12-01T01:16:27.325703+0000","last_became_active":"2023-12-01T01:16:24.940515+0000","last_became_peered":"2023-12-01T01:16:24.940515+0000","last_unstale":"2023-12-01T01:16:27.325703+0000","last_undegraded":"2023-12-01T01:16:27.325703+0000","last_fullsized":"2023-12-01T01:16:27.325703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:08:26.409014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325032+0000","last_change":"2023-12-01T01:16:25.369662+0000","last_active":"2023-12-01T01:16:27.325032+0000","last_peered":"2023-12-01T01:16:27.325032+0000","last_clean":"2023-12-01T01:16:27.325032+0000","last_became_active":"2023-12-01T01:16:25.369178+0000","last_became_peered":"2023-12-01T01:16:25.369178+0000","last_unstale":"2023-12-01T01:16:27.325032+0000","last_undegraded":"2023-12-01T01:16:27.325032+0000","last_fullsized":"2023-12-01T01:16:27.325032+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:40:39.919414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'6","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325641+0000","last_change":"2023-12-01T01:16:24.936733+0000","last_active":"2023-12-01T01:16:27.325641+0000","last_peered":"2023-12-01T01:16:27.325641+0000","last_clean":"2023-12-01T01:16:27.325641+0000","last_became_active":"2023-12-01T01:16:24.936459+0000","last_became_peered":"2023-12-01T01:16:24.936459+0000","last_unstale":"2023-12-01T01:16:27.325641+0000","last_undegraded":"2023-12-01T01:16:27.325641+0000","last_fullsized":"2023-12-01T01:16:27.325641+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:19:58.387650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.071505+0000","last_change":"2023-12-01T01:16:25.364292+0000","last_active":"2023-12-01T01:16:26.071505+0000","last_peered":"2023-12-01T01:16:26.071505+0000","last_clean":"2023-12-01T01:16:26.071505+0000","last_became_active":"2023-12-01T01:16:25.362615+0000","last_became_peered":"2023-12-01T01:16:25.362615+0000","last_unstale":"2023-12-01T01:16:26.071505+0000","last_undegraded":"2023-12-01T01:16:26.071505+0000","last_fullsized":"2023-12-01T01:16:26.071505+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:06:07.350607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:29.636705+0000","last_change":"2023-12-01T01:16:24.941007+0000","last_active":"2023-12-01T01:16:29.636705+0000","last_peered":"2023-12-01T01:16:29.636705+0000","last_clean":"2023-12-01T01:16:29.636705+0000","last_became_active":"2023-12-01T01:16:24.940669+0000","last_became_peered":"2023-12-01T01:16:24.940669+0000","last_unstale":"2023-12-01T01:16:29.636705+0000","last_undegraded":"2023-12-01T01:16:29.636705+0000","last_fullsized":"2023-12-01T01:16:29.636705+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:26:21.907942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.352205+0000","last_change":"2023-12-01T01:16:25.360104+0000","last_active":"2023-12-01T01:16:26.352205+0000","last_peered":"2023-12-01T01:16:26.352205+0000","last_clean":"2023-12-01T01:16:26.352205+0000","last_became_active":"2023-12-01T01:16:25.359807+0000","last_became_peered":"2023-12-01T01:16:25.359807+0000","last_unstale":"2023-12-01T01:16:26.352205+0000","last_undegraded":"2023-12-01T01:16:26.352205+0000","last_fullsized":"2023-12-01T01:16:26.352205+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:04:05.100258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:28.632442+0000","last_change":"2023-12-01T01:16:24.945074+0000","last_active":"2023-12-01T01:16:28.632442+0000","last_peered":"2023-12-01T01:16:28.632442+0000","last_clean":"2023-12-01T01:16:28.632442+0000","last_became_active":"2023-12-01T01:16:24.944737+0000","last_became_peered":"2023-12-01T01:16:24.944737+0000","last_unstale":"2023-12-01T01:16:28.632442+0000","last_undegraded":"2023-12-01T01:16:28.632442+0000","last_fullsized":"2023-12-01T01:16:28.632442+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:44:53.931454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.350184+0000","last_change":"2023-12-01T01:16:25.364995+0000","last_active":"2023-12-01T01:16:26.350184+0000","last_peered":"2023-12-01T01:16:26.350184+0000","last_clean":"2023-12-01T01:16:26.350184+0000","last_became_active":"2023-12-01T01:16:25.364684+0000","last_became_peered":"2023-12-01T01:16:25.364684+0000","last_unstale":"2023-12-01T01:16:26.350184+0000","last_undegraded":"2023-12-01T01:16:26.350184+0000","last_fullsized":"2023-12-01T01:16:26.350184+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:41:35.422238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.554690+0000","last_change":"2023-12-01T01:16:24.929103+0000","last_active":"2023-12-01T01:16:26.554690+0000","last_peered":"2023-12-01T01:16:26.554690+0000","last_clean":"2023-12-01T01:16:26.554690+0000","last_became_active":"2023-12-01T01:16:24.928769+0000","last_became_peered":"2023-12-01T01:16:24.928769+0000","last_unstale":"2023-12-01T01:16:26.554690+0000","last_undegraded":"2023-12-01T01:16:26.554690+0000","last_fullsized":"2023-12-01T01:16:26.554690+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:45:24.542610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.349997+0000","last_change":"2023-12-01T01:16:25.370816+0000","last_active":"2023-12-01T01:16:26.349997+0000","last_peered":"2023-12-01T01:16:26.349997+0000","last_clean":"2023-12-01T01:16:26.349997+0000","last_became_active":"2023-12-01T01:16:25.370465+0000","last_became_peered":"2023-12-01T01:16:25.370465+0000","last_unstale":"2023-12-01T01:16:26.349997+0000","last_undegraded":"2023-12-01T01:16:26.349997+0000","last_fullsized":"2023-12-01T01:16:26.349997+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:52:28.990416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'16","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:29.638905+0000","last_change":"2023-12-01T01:16:24.933022+0000","last_active":"2023-12-01T01:16:29.638905+0000","last_peered":"2023-12-01T01:16:29.638905+0000","last_clean":"2023-12-01T01:16:29.638905+0000","last_became_active":"2023-12-01T01:16:24.932705+0000","last_became_peered":"2023-12-01T01:16:24.932705+0000","last_unstale":"2023-12-01T01:16:29.638905+0000","last_undegraded":"2023-12-01T01:16:29.638905+0000","last_fullsized":"2023-12-01T01:16:29.638905+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:36:44.747984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325386+0000","last_change":"2023-12-01T01:16:25.370881+0000","last_active":"2023-12-01T01:16:27.325386+0000","last_peered":"2023-12-01T01:16:27.325386+0000","last_clean":"2023-12-01T01:16:27.325386+0000","last_became_active":"2023-12-01T01:16:25.368242+0000","last_became_peered":"2023-12-01T01:16:25.368242+0000","last_unstale":"2023-12-01T01:16:27.325386+0000","last_undegraded":"2023-12-01T01:16:27.325386+0000","last_fullsized":"2023-12-01T01:16:27.325386+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:35:41.128841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.071546+0000","last_change":"2023-12-01T01:16:24.933579+0000","last_active":"2023-12-01T01:16:26.071546+0000","last_peered":"2023-12-01T01:16:26.071546+0000","last_clean":"2023-12-01T01:16:26.071546+0000","last_became_active":"2023-12-01T01:16:24.933207+0000","last_became_peered":"2023-12-01T01:16:24.933207+0000","last_unstale":"2023-12-01T01:16:26.071546+0000","last_undegraded":"2023-12-01T01:16:26.071546+0000","last_fullsized":"2023-12-01T01:16:26.071546+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:28.309551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.592816+0000","last_change":"2023-12-01T01:16:25.368611+0000","last_active":"2023-12-01T01:16:25.592816+0000","last_peered":"2023-12-01T01:16:25.592816+0000","last_clean":"2023-12-01T01:16:25.592816+0000","last_became_active":"2023-12-01T01:16:25.368291+0000","last_became_peered":"2023-12-01T01:16:25.368291+0000","last_unstale":"2023-12-01T01:16:25.592816+0000","last_undegraded":"2023-12-01T01:16:25.592816+0000","last_fullsized":"2023-12-01T01:16:25.592816+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:17:21.824737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325140+0000","last_change":"2023-12-01T01:16:24.941410+0000","last_active":"2023-12-01T01:16:27.325140+0000","last_peered":"2023-12-01T01:16:27.325140+0000","last_clean":"2023-12-01T01:16:27.325140+0000","last_became_active":"2023-12-01T01:16:24.941127+0000","last_became_peered":"2023-12-01T01:16:24.941127+0000","last_unstale":"2023-12-01T01:16:27.325140+0000","last_undegraded":"2023-12-01T01:16:27.325140+0000","last_fullsized":"2023-12-01T01:16:27.325140+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:25:20.911892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325740+0000","last_change":"2023-12-01T01:16:25.368235+0000","last_active":"2023-12-01T01:16:27.325740+0000","last_peered":"2023-12-01T01:16:27.325740+0000","last_clean":"2023-12-01T01:16:27.325740+0000","last_became_active":"2023-12-01T01:16:25.367918+0000","last_became_peered":"2023-12-01T01:16:25.367918+0000","last_unstale":"2023-12-01T01:16:27.325740+0000","last_undegraded":"2023-12-01T01:16:27.325740+0000","last_fullsized":"2023-12-01T01:16:27.325740+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:12:26.665404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.350039+0000","last_change":"2023-12-01T01:16:24.932057+0000","last_active":"2023-12-01T01:16:26.350039+0000","last_peered":"2023-12-01T01:16:26.350039+0000","last_clean":"2023-12-01T01:16:26.350039+0000","last_became_active":"2023-12-01T01:16:24.931817+0000","last_became_peered":"2023-12-01T01:16:24.931817+0000","last_unstale":"2023-12-01T01:16:26.350039+0000","last_undegraded":"2023-12-01T01:16:26.350039+0000","last_fullsized":"2023-12-01T01:16:26.350039+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:19:02.039334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325595+0000","last_change":"2023-12-01T01:16:25.367431+0000","last_active":"2023-12-01T01:16:27.325595+0000","last_peered":"2023-12-01T01:16:27.325595+0000","last_clean":"2023-12-01T01:16:27.325595+0000","last_became_active":"2023-12-01T01:16:25.367184+0000","last_became_peered":"2023-12-01T01:16:25.367184+0000","last_unstale":"2023-12-01T01:16:27.325595+0000","last_undegraded":"2023-12-01T01:16:27.325595+0000","last_fullsized":"2023-12-01T01:16:27.325595+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:21:19.132725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325383+0000","last_change":"2023-12-01T01:16:25.370593+0000","last_active":"2023-12-01T01:16:27.325383+0000","last_peered":"2023-12-01T01:16:27.325383+0000","last_clean":"2023-12-01T01:16:27.325383+0000","last_became_active":"2023-12-01T01:16:25.367614+0000","last_became_peered":"2023-12-01T01:16:25.367614+0000","last_unstale":"2023-12-01T01:16:27.325383+0000","last_undegraded":"2023-12-01T01:16:27.325383+0000","last_fullsized":"2023-12-01T01:16:27.325383+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:56:34.388180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.352111+0000","last_change":"2023-12-01T01:16:24.931188+0000","last_active":"2023-12-01T01:16:26.352111+0000","last_peered":"2023-12-01T01:16:26.352111+0000","last_clean":"2023-12-01T01:16:26.352111+0000","last_became_active":"2023-12-01T01:16:24.930875+0000","last_became_peered":"2023-12-01T01:16:24.930875+0000","last_unstale":"2023-12-01T01:16:26.352111+0000","last_undegraded":"2023-12-01T01:16:26.352111+0000","last_fullsized":"2023-12-01T01:16:26.352111+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:17.729286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.571499+0000","last_change":"2023-12-01T01:16:25.361359+0000","last_active":"2023-12-01T01:16:26.571499+0000","last_peered":"2023-12-01T01:16:26.571499+0000","last_clean":"2023-12-01T01:16:26.571499+0000","last_became_active":"2023-12-01T01:16:25.361083+0000","last_became_peered":"2023-12-01T01:16:25.361083+0000","last_unstale":"2023-12-01T01:16:26.571499+0000","last_undegraded":"2023-12-01T01:16:26.571499+0000","last_fullsized":"2023-12-01T01:16:26.571499+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:42:09.417021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.554872+0000","last_change":"2023-12-01T01:16:24.927614+0000","last_active":"2023-12-01T01:16:26.554872+0000","last_peered":"2023-12-01T01:16:26.554872+0000","last_clean":"2023-12-01T01:16:26.554872+0000","last_became_active":"2023-12-01T01:16:24.926459+0000","last_became_peered":"2023-12-01T01:16:24.926459+0000","last_unstale":"2023-12-01T01:16:26.554872+0000","last_undegraded":"2023-12-01T01:16:26.554872+0000","last_fullsized":"2023-12-01T01:16:26.554872+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:41:25.435261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":323,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.358040+0000","last_change":"2023-12-01T01:15:25.599551+0000","last_active":"2023-12-01T01:16:25.358040+0000","last_peered":"2023-12-01T01:16:25.358040+0000","last_clean":"2023-12-01T01:16:25.358040+0000","last_became_active":"2023-12-01T01:15:25.598951+0000","last_became_peered":"2023-12-01T01:15:25.598951+0000","last_unstale":"2023-12-01T01:16:25.358040+0000","last_undegraded":"2023-12-01T01:16:25.358040+0000","last_fullsized":"2023-12-01T01:16:25.358040+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:12:31.173756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:12:31.173756+0000","last_clean_scrub_stamp":"2023-12-01T01:12:31.173756+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:00:11.343823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.352149+0000","last_change":"2023-12-01T01:16:25.358617+0000","last_active":"2023-12-01T01:16:26.352149+0000","last_peered":"2023-12-01T01:16:26.352149+0000","last_clean":"2023-12-01T01:16:26.352149+0000","last_became_active":"2023-12-01T01:16:25.358325+0000","last_became_peered":"2023-12-01T01:16:25.358325+0000","last_unstale":"2023-12-01T01:16:26.352149+0000","last_undegraded":"2023-12-01T01:16:26.352149+0000","last_fullsized":"2023-12-01T01:16:26.352149+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:40:01.111777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.349804+0000","last_change":"2023-12-01T01:16:24.935103+0000","last_active":"2023-12-01T01:16:26.349804+0000","last_peered":"2023-12-01T01:16:26.349804+0000","last_clean":"2023-12-01T01:16:26.349804+0000","last_became_active":"2023-12-01T01:16:24.934900+0000","last_became_peered":"2023-12-01T01:16:24.934900+0000","last_unstale":"2023-12-01T01:16:26.349804+0000","last_undegraded":"2023-12-01T01:16:26.349804+0000","last_fullsized":"2023-12-01T01:16:26.349804+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:59:37.138090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"82'3","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:30.653485+0000","last_change":"2023-12-01T01:16:24.938260+0000","last_active":"2023-12-01T01:16:30.653485+0000","last_peered":"2023-12-01T01:16:30.653485+0000","last_clean":"2023-12-01T01:16:30.653485+0000","last_became_active":"2023-12-01T01:16:24.937939+0000","last_became_peered":"2023-12-01T01:16:24.937939+0000","last_unstale":"2023-12-01T01:16:30.653485+0000","last_undegraded":"2023-12-01T01:16:30.653485+0000","last_fullsized":"2023-12-01T01:16:30.653485+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:07:05.090877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.352470+0000","last_change":"2023-12-01T01:16:25.362137+0000","last_active":"2023-12-01T01:16:26.352470+0000","last_peered":"2023-12-01T01:16:26.352470+0000","last_clean":"2023-12-01T01:16:26.352470+0000","last_became_active":"2023-12-01T01:16:25.361563+0000","last_became_peered":"2023-12-01T01:16:25.361563+0000","last_unstale":"2023-12-01T01:16:26.352470+0000","last_undegraded":"2023-12-01T01:16:26.352470+0000","last_fullsized":"2023-12-01T01:16:26.352470+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:37:56.473920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'7","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:30.654598+0000","last_change":"2023-12-01T01:16:24.935692+0000","last_active":"2023-12-01T01:16:30.654598+0000","last_peered":"2023-12-01T01:16:30.654598+0000","last_clean":"2023-12-01T01:16:30.654598+0000","last_became_active":"2023-12-01T01:16:24.934799+0000","last_became_peered":"2023-12-01T01:16:24.934799+0000","last_unstale":"2023-12-01T01:16:30.654598+0000","last_undegraded":"2023-12-01T01:16:30.654598+0000","last_fullsized":"2023-12-01T01:16:30.654598+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:22:51.586089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.571589+0000","last_change":"2023-12-01T01:16:25.368672+0000","last_active":"2023-12-01T01:16:26.571589+0000","last_peered":"2023-12-01T01:16:26.571589+0000","last_clean":"2023-12-01T01:16:26.571589+0000","last_became_active":"2023-12-01T01:16:25.368309+0000","last_became_peered":"2023-12-01T01:16:25.368309+0000","last_unstale":"2023-12-01T01:16:26.571589+0000","last_undegraded":"2023-12-01T01:16:26.571589+0000","last_fullsized":"2023-12-01T01:16:26.571589+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:30:59.029250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.554938+0000","last_change":"2023-12-01T01:16:24.931322+0000","last_active":"2023-12-01T01:16:26.554938+0000","last_peered":"2023-12-01T01:16:26.554938+0000","last_clean":"2023-12-01T01:16:26.554938+0000","last_became_active":"2023-12-01T01:16:24.930961+0000","last_became_peered":"2023-12-01T01:16:24.930961+0000","last_unstale":"2023-12-01T01:16:26.554938+0000","last_undegraded":"2023-12-01T01:16:26.554938+0000","last_fullsized":"2023-12-01T01:16:26.554938+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:47:16.317035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325104+0000","last_change":"2023-12-01T01:16:25.364065+0000","last_active":"2023-12-01T01:16:27.325104+0000","last_peered":"2023-12-01T01:16:27.325104+0000","last_clean":"2023-12-01T01:16:27.325104+0000","last_became_active":"2023-12-01T01:16:25.363694+0000","last_became_peered":"2023-12-01T01:16:25.363694+0000","last_unstale":"2023-12-01T01:16:27.325104+0000","last_undegraded":"2023-12-01T01:16:27.325104+0000","last_fullsized":"2023-12-01T01:16:27.325104+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:49:39.091587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:28.633891+0000","last_change":"2023-12-01T01:16:24.932730+0000","last_active":"2023-12-01T01:16:28.633891+0000","last_peered":"2023-12-01T01:16:28.633891+0000","last_clean":"2023-12-01T01:16:28.633891+0000","last_became_active":"2023-12-01T01:16:24.932404+0000","last_became_peered":"2023-12-01T01:16:24.932404+0000","last_unstale":"2023-12-01T01:16:28.633891+0000","last_undegraded":"2023-12-01T01:16:28.633891+0000","last_fullsized":"2023-12-01T01:16:28.633891+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:41:33.104418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.071437+0000","last_change":"2023-12-01T01:16:25.375795+0000","last_active":"2023-12-01T01:16:26.071437+0000","last_peered":"2023-12-01T01:16:26.071437+0000","last_clean":"2023-12-01T01:16:26.071437+0000","last_became_active":"2023-12-01T01:16:25.375485+0000","last_became_peered":"2023-12-01T01:16:25.375485+0000","last_unstale":"2023-12-01T01:16:26.071437+0000","last_undegraded":"2023-12-01T01:16:26.071437+0000","last_fullsized":"2023-12-01T01:16:26.071437+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:10:04.009511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'5","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:30.653691+0000","last_change":"2023-12-01T01:16:24.936770+0000","last_active":"2023-12-01T01:16:30.653691+0000","last_peered":"2023-12-01T01:16:30.653691+0000","last_clean":"2023-12-01T01:16:30.653691+0000","last_became_active":"2023-12-01T01:16:24.936058+0000","last_became_peered":"2023-12-01T01:16:24.936058+0000","last_unstale":"2023-12-01T01:16:30.653691+0000","last_undegraded":"2023-12-01T01:16:30.653691+0000","last_fullsized":"2023-12-01T01:16:30.653691+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:22:21.414131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.592744+0000","last_change":"2023-12-01T01:16:25.372183+0000","last_active":"2023-12-01T01:16:25.592744+0000","last_peered":"2023-12-01T01:16:25.592744+0000","last_clean":"2023-12-01T01:16:25.592744+0000","last_became_active":"2023-12-01T01:16:25.371865+0000","last_became_peered":"2023-12-01T01:16:25.371865+0000","last_unstale":"2023-12-01T01:16:25.592744+0000","last_undegraded":"2023-12-01T01:16:25.592744+0000","last_fullsized":"2023-12-01T01:16:25.592744+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:19:54.675425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.356576+0000","last_change":"2023-12-01T01:16:24.939397+0000","last_active":"2023-12-01T01:16:25.356576+0000","last_peered":"2023-12-01T01:16:25.356576+0000","last_clean":"2023-12-01T01:16:25.356576+0000","last_became_active":"2023-12-01T01:16:24.939081+0000","last_became_peered":"2023-12-01T01:16:24.939081+0000","last_unstale":"2023-12-01T01:16:25.356576+0000","last_undegraded":"2023-12-01T01:16:25.356576+0000","last_fullsized":"2023-12-01T01:16:25.356576+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:21:34.967255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.571515+0000","last_change":"2023-12-01T01:16:25.368484+0000","last_active":"2023-12-01T01:16:26.571515+0000","last_peered":"2023-12-01T01:16:26.571515+0000","last_clean":"2023-12-01T01:16:26.571515+0000","last_became_active":"2023-12-01T01:16:25.367638+0000","last_became_peered":"2023-12-01T01:16:25.367638+0000","last_unstale":"2023-12-01T01:16:26.571515+0000","last_undegraded":"2023-12-01T01:16:26.571515+0000","last_fullsized":"2023-12-01T01:16:26.571515+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:06:05.354443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:28.633543+0000","last_change":"2023-12-01T01:16:24.933304+0000","last_active":"2023-12-01T01:16:28.633543+0000","last_peered":"2023-12-01T01:16:28.633543+0000","last_clean":"2023-12-01T01:16:28.633543+0000","last_became_active":"2023-12-01T01:16:24.933028+0000","last_became_peered":"2023-12-01T01:16:24.933028+0000","last_unstale":"2023-12-01T01:16:28.633543+0000","last_undegraded":"2023-12-01T01:16:28.633543+0000","last_fullsized":"2023-12-01T01:16:28.633543+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:26:51.810995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.350012+0000","last_change":"2023-12-01T01:16:25.364888+0000","last_active":"2023-12-01T01:16:26.350012+0000","last_peered":"2023-12-01T01:16:26.350012+0000","last_clean":"2023-12-01T01:16:26.350012+0000","last_became_active":"2023-12-01T01:16:25.364595+0000","last_became_peered":"2023-12-01T01:16:25.364595+0000","last_unstale":"2023-12-01T01:16:26.350012+0000","last_undegraded":"2023-12-01T01:16:26.350012+0000","last_fullsized":"2023-12-01T01:16:26.350012+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:54:23.178824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:29.638923+0000","last_change":"2023-12-01T01:16:24.945771+0000","last_active":"2023-12-01T01:16:29.638923+0000","last_peered":"2023-12-01T01:16:29.638923+0000","last_clean":"2023-12-01T01:16:29.638923+0000","last_became_active":"2023-12-01T01:16:24.945441+0000","last_became_peered":"2023-12-01T01:16:24.945441+0000","last_unstale":"2023-12-01T01:16:29.638923+0000","last_undegraded":"2023-12-01T01:16:29.638923+0000","last_fullsized":"2023-12-01T01:16:29.638923+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:49:41.234472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.350239+0000","last_change":"2023-12-01T01:16:25.365246+0000","last_active":"2023-12-01T01:16:26.350239+0000","last_peered":"2023-12-01T01:16:26.350239+0000","last_clean":"2023-12-01T01:16:26.350239+0000","last_became_active":"2023-12-01T01:16:25.364364+0000","last_became_peered":"2023-12-01T01:16:25.364364+0000","last_unstale":"2023-12-01T01:16:26.350239+0000","last_undegraded":"2023-12-01T01:16:26.350239+0000","last_fullsized":"2023-12-01T01:16:26.350239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:56:44.815172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.375188+0000","last_change":"2023-12-01T01:16:24.944229+0000","last_active":"2023-12-01T01:16:25.375188+0000","last_peered":"2023-12-01T01:16:25.375188+0000","last_clean":"2023-12-01T01:16:25.375188+0000","last_became_active":"2023-12-01T01:16:24.943880+0000","last_became_peered":"2023-12-01T01:16:24.943880+0000","last_unstale":"2023-12-01T01:16:25.375188+0000","last_undegraded":"2023-12-01T01:16:25.375188+0000","last_fullsized":"2023-12-01T01:16:25.375188+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:44:26.769358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.350055+0000","last_change":"2023-12-01T01:16:25.361028+0000","last_active":"2023-12-01T01:16:26.350055+0000","last_peered":"2023-12-01T01:16:26.350055+0000","last_clean":"2023-12-01T01:16:26.350055+0000","last_became_active":"2023-12-01T01:16:25.360702+0000","last_became_peered":"2023-12-01T01:16:25.360702+0000","last_unstale":"2023-12-01T01:16:26.350055+0000","last_undegraded":"2023-12-01T01:16:26.350055+0000","last_fullsized":"2023-12-01T01:16:26.350055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:45:09.472497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.352416+0000","last_change":"2023-12-01T01:16:24.945567+0000","last_active":"2023-12-01T01:16:26.352416+0000","last_peered":"2023-12-01T01:16:26.352416+0000","last_clean":"2023-12-01T01:16:26.352416+0000","last_became_active":"2023-12-01T01:16:24.945256+0000","last_became_peered":"2023-12-01T01:16:24.945256+0000","last_unstale":"2023-12-01T01:16:26.352416+0000","last_undegraded":"2023-12-01T01:16:26.352416+0000","last_fullsized":"2023-12-01T01:16:26.352416+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:19:30.944573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.350244+0000","last_change":"2023-12-01T01:16:25.367247+0000","last_active":"2023-12-01T01:16:26.350244+0000","last_peered":"2023-12-01T01:16:26.350244+0000","last_clean":"2023-12-01T01:16:26.350244+0000","last_became_active":"2023-12-01T01:16:25.366872+0000","last_became_peered":"2023-12-01T01:16:25.366872+0000","last_unstale":"2023-12-01T01:16:26.350244+0000","last_undegraded":"2023-12-01T01:16:26.350244+0000","last_fullsized":"2023-12-01T01:16:26.350244+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:03:07.697233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.592768+0000","last_change":"2023-12-01T01:16:25.376005+0000","last_active":"2023-12-01T01:16:25.592768+0000","last_peered":"2023-12-01T01:16:25.592768+0000","last_clean":"2023-12-01T01:16:25.592768+0000","last_became_active":"2023-12-01T01:16:25.375702+0000","last_became_peered":"2023-12-01T01:16:25.375702+0000","last_unstale":"2023-12-01T01:16:25.592768+0000","last_undegraded":"2023-12-01T01:16:25.592768+0000","last_fullsized":"2023-12-01T01:16:25.592768+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:14:00.955730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.378145+0000","last_change":"2023-12-01T01:16:25.371023+0000","last_active":"2023-12-01T01:16:25.378145+0000","last_peered":"2023-12-01T01:16:25.378145+0000","last_clean":"2023-12-01T01:16:25.378145+0000","last_became_active":"2023-12-01T01:16:25.370694+0000","last_became_peered":"2023-12-01T01:16:25.370694+0000","last_unstale":"2023-12-01T01:16:25.378145+0000","last_undegraded":"2023-12-01T01:16:25.378145+0000","last_fullsized":"2023-12-01T01:16:25.378145+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:03:00.715413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.375644+0000","last_change":"2023-12-01T01:16:25.361795+0000","last_active":"2023-12-01T01:16:25.375644+0000","last_peered":"2023-12-01T01:16:25.375644+0000","last_clean":"2023-12-01T01:16:25.375644+0000","last_became_active":"2023-12-01T01:16:25.361459+0000","last_became_peered":"2023-12-01T01:16:25.361459+0000","last_unstale":"2023-12-01T01:16:25.375644+0000","last_undegraded":"2023-12-01T01:16:25.375644+0000","last_fullsized":"2023-12-01T01:16:25.375644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:18:57.531240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.324741+0000","last_change":"2023-12-01T01:16:25.363773+0000","last_active":"2023-12-01T01:16:27.324741+0000","last_peered":"2023-12-01T01:16:27.324741+0000","last_clean":"2023-12-01T01:16:27.324741+0000","last_became_active":"2023-12-01T01:16:25.363422+0000","last_became_peered":"2023-12-01T01:16:25.363422+0000","last_unstale":"2023-12-01T01:16:27.324741+0000","last_undegraded":"2023-12-01T01:16:27.324741+0000","last_fullsized":"2023-12-01T01:16:27.324741+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:48:33.395566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.352367+0000","last_change":"2023-12-01T01:16:25.359431+0000","last_active":"2023-12-01T01:16:26.352367+0000","last_peered":"2023-12-01T01:16:26.352367+0000","last_clean":"2023-12-01T01:16:26.352367+0000","last_became_active":"2023-12-01T01:16:25.359116+0000","last_became_peered":"2023-12-01T01:16:25.359116+0000","last_unstale":"2023-12-01T01:16:26.352367+0000","last_undegraded":"2023-12-01T01:16:26.352367+0000","last_fullsized":"2023-12-01T01:16:26.352367+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:32:26.545269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.592804+0000","last_change":"2023-12-01T01:16:25.368008+0000","last_active":"2023-12-01T01:16:25.592804+0000","last_peered":"2023-12-01T01:16:25.592804+0000","last_clean":"2023-12-01T01:16:25.592804+0000","last_became_active":"2023-12-01T01:16:25.367726+0000","last_became_peered":"2023-12-01T01:16:25.367726+0000","last_unstale":"2023-12-01T01:16:25.592804+0000","last_undegraded":"2023-12-01T01:16:25.592804+0000","last_fullsized":"2023-12-01T01:16:25.592804+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:38:33.837846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.378494+0000","last_change":"2023-12-01T01:16:25.372781+0000","last_active":"2023-12-01T01:16:25.378494+0000","last_peered":"2023-12-01T01:16:25.378494+0000","last_clean":"2023-12-01T01:16:25.378494+0000","last_became_active":"2023-12-01T01:16:25.371443+0000","last_became_peered":"2023-12-01T01:16:25.371443+0000","last_unstale":"2023-12-01T01:16:25.378494+0000","last_undegraded":"2023-12-01T01:16:25.378494+0000","last_fullsized":"2023-12-01T01:16:25.378494+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:49:06.422055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.571571+0000","last_change":"2023-12-01T01:16:25.363048+0000","last_active":"2023-12-01T01:16:26.571571+0000","last_peered":"2023-12-01T01:16:26.571571+0000","last_clean":"2023-12-01T01:16:26.571571+0000","last_became_active":"2023-12-01T01:16:25.362752+0000","last_became_peered":"2023-12-01T01:16:25.362752+0000","last_unstale":"2023-12-01T01:16:26.571571+0000","last_undegraded":"2023-12-01T01:16:26.571571+0000","last_fullsized":"2023-12-01T01:16:26.571571+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:54:16.492562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325256+0000","last_change":"2023-12-01T01:16:25.359998+0000","last_active":"2023-12-01T01:16:27.325256+0000","last_peered":"2023-12-01T01:16:27.325256+0000","last_clean":"2023-12-01T01:16:27.325256+0000","last_became_active":"2023-12-01T01:16:25.359744+0000","last_became_peered":"2023-12-01T01:16:25.359744+0000","last_unstale":"2023-12-01T01:16:27.325256+0000","last_undegraded":"2023-12-01T01:16:27.325256+0000","last_fullsized":"2023-12-01T01:16:27.325256+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:12:33.947134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.571477+0000","last_change":"2023-12-01T01:16:25.373615+0000","last_active":"2023-12-01T01:16:26.571477+0000","last_peered":"2023-12-01T01:16:26.571477+0000","last_clean":"2023-12-01T01:16:26.571477+0000","last_became_active":"2023-12-01T01:16:25.373247+0000","last_became_peered":"2023-12-01T01:16:25.373247+0000","last_unstale":"2023-12-01T01:16:26.571477+0000","last_undegraded":"2023-12-01T01:16:26.571477+0000","last_fullsized":"2023-12-01T01:16:26.571477+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:28:10.815906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.324878+0000","last_change":"2023-12-01T01:16:25.367404+0000","last_active":"2023-12-01T01:16:27.324878+0000","last_peered":"2023-12-01T01:16:27.324878+0000","last_clean":"2023-12-01T01:16:27.324878+0000","last_became_active":"2023-12-01T01:16:25.366404+0000","last_became_peered":"2023-12-01T01:16:25.366404+0000","last_unstale":"2023-12-01T01:16:27.324878+0000","last_undegraded":"2023-12-01T01:16:27.324878+0000","last_fullsized":"2023-12-01T01:16:27.324878+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:54:49.668036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.352405+0000","last_change":"2023-12-01T01:16:25.360315+0000","last_active":"2023-12-01T01:16:26.352405+0000","last_peered":"2023-12-01T01:16:26.352405+0000","last_clean":"2023-12-01T01:16:26.352405+0000","last_became_active":"2023-12-01T01:16:25.360031+0000","last_became_peered":"2023-12-01T01:16:25.360031+0000","last_unstale":"2023-12-01T01:16:26.352405+0000","last_undegraded":"2023-12-01T01:16:26.352405+0000","last_fullsized":"2023-12-01T01:16:26.352405+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:13:24.437697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.592792+0000","last_change":"2023-12-01T01:16:25.367820+0000","last_active":"2023-12-01T01:16:25.592792+0000","last_peered":"2023-12-01T01:16:25.592792+0000","last_clean":"2023-12-01T01:16:25.592792+0000","last_became_active":"2023-12-01T01:16:25.367525+0000","last_became_peered":"2023-12-01T01:16:25.367525+0000","last_unstale":"2023-12-01T01:16:25.592792+0000","last_undegraded":"2023-12-01T01:16:25.592792+0000","last_fullsized":"2023-12-01T01:16:25.592792+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:28:01.552342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4324,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":188416,"data_stored":10946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":95,"ondisk_log_size":95,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481804,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":864,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":884736,"data_stored":423396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]}]},{"osd":10,"up_from":69,"seq":296352743439,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28400,"kb_used_data":1448,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716752,"statfs":{"total":95995035648,"available":95965954048,"internally_reserved":0,"allocated":1482752,"data_stored":1006039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":1,"last update":"Fri Dec 1 01:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.438}]},{"osd":2,"last update":"Fri Dec 1 01:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.54000000000000004}]},{"osd":3,"last update":"Fri Dec 1 01:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":5.7240000000000002,"5min":5.7240000000000002,"15min":5.7240000000000002},"last":0.56799999999999995}]},{"osd":4,"last update":"Fri Dec 1 01:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.59899999999999998}]},{"osd":5,"last update":"Fri Dec 1 01:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.50800000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":5.7510000000000003,"5min":5.7510000000000003,"15min":5.7510000000000003},"last":0.66300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":8,"last update":"Fri Dec 1 01:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.63},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.69499999999999995}]},{"osd":9,"last update":"Fri Dec 1 01:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.71199999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]}]},{"osd":9,"up_from":64,"seq":274877906965,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":905216,"data_stored":423722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.374},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.40000000000000002}]},{"osd":1,"last update":"Fri Dec 1 01:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.64300000000000002}]},{"osd":2,"last update":"Fri Dec 1 01:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.55400000000000005}]},{"osd":3,"last update":"Fri Dec 1 01:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.42699999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.67800000000000005}]},{"osd":5,"last update":"Fri Dec 1 01:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.626},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.70999999999999996}]},{"osd":6,"last update":"Fri Dec 1 01:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.72599999999999998}]},{"osd":7,"last update":"Fri Dec 1 01:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.749}]},{"osd":8,"last update":"Fri Dec 1 01:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.76900000000000002}]},{"osd":10,"last update":"Fri Dec 1 01:16:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.80100000000000005}]}]},{"osd":8,"up_from":57,"seq":244813135896,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":901120,"data_stored":423552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:15:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.505},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.48399999999999999}]},{"osd":1,"last update":"Fri Dec 1 01:15:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.63700000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:15:42 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.71199999999999997}]},{"osd":3,"last update":"Fri Dec 1 01:15:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.66000000000000003}]},{"osd":4,"last update":"Fri Dec 1 01:15:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.73499999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:15:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.87}]},{"osd":6,"last update":"Fri Dec 1 01:15:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.83799999999999997}]},{"osd":7,"last update":"Fri Dec 1 01:15:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.82399999999999995}]},{"osd":9,"last update":"Fri Dec 1 01:16:01 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":5.9690000000000003,"5min":5.9690000000000003,"15min":5.9690000000000003},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":3.5259999999999998,"5min":3.5259999999999998,"15min":3.5259999999999998},"last":0.77800000000000002}]},{"osd":10,"last update":"Fri Dec 1 01:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":6.0469999999999997,"5min":6.0469999999999997,"15min":6.0469999999999997},"last":0.93600000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]}]},{"osd":7,"up_from":50,"seq":214748364828,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28396,"kb_used_data":1444,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716756,"statfs":{"total":95995035648,"available":95965958144,"internally_reserved":0,"allocated":1478656,"data_stored":1013764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.45000000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.40100000000000002,"15min":0.39300000000000002},"min":{"1min":0.27700000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.436},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.41099999999999998,"15min":0.40200000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.26900000000000002}]},{"osd":2,"last update":"Fri Dec 1 01:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56200000000000006,"15min":0.55800000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.64300000000000002}]},{"osd":3,"last update":"Fri Dec 1 01:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.55800000000000005,"15min":0.55000000000000004},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.77700000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61899999999999999,"15min":0.61599999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.85099999999999998,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.45000000000000001}]},{"osd":4,"last update":"Fri Dec 1 01:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.51600000000000001,"15min":0.51200000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.51600000000000001,"15min":0.51100000000000001},"min":{"1min":0.32200000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.35899999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.435,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.61499999999999999,"15min":0.60699999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.56000000000000005}]},{"osd":6,"last update":"Fri Dec 1 01:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.64800000000000002,"15min":0.64300000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.66400000000000003,"15min":0.65800000000000003},"min":{"1min":0.42099999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0600000000000001,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.42099999999999999}]},{"osd":8,"last update":"Fri Dec 1 01:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.63200000000000001}]},{"osd":9,"last update":"Fri Dec 1 01:16:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.40899999999999997}]},{"osd":10,"last update":"Fri Dec 1 01:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.52500000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]}]},{"osd":6,"up_from":45,"seq":193273528353,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":860,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":880640,"data_stored":422814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.41199999999999998,"15min":0.40699999999999997},"min":{"1min":0.29799999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.437,"5min":0.41399999999999998,"15min":0.40999999999999998},"min":{"1min":0.29899999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.45500000000000002}]},{"osd":1,"last update":"Fri Dec 1 01:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.623,"15min":0.626},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82199999999999995,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":1.5840000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.59099999999999997,"15min":0.59899999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.86899999999999999,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.68100000000000005}]},{"osd":2,"last update":"Fri Dec 1 01:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57699999999999996,"15min":0.57099999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.629,"15min":0.623},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":1.506}]},{"osd":3,"last update":"Fri Dec 1 01:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.47899999999999998,"15min":0.46800000000000003},"min":{"1min":0.35399999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":1.456},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.46700000000000003,"15min":0.46400000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.48499999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.83899999999999997,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.65600000000000003,"15min":0.65200000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":1.6830000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63800000000000001,"15min":0.64000000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.871,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.65800000000000003,"15min":0.65200000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.65200000000000002}]},{"osd":7,"last update":"Fri Dec 1 01:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.69999999999999996,"15min":0.70399999999999996},"min":{"1min":0.45600000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92000000000000004,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.93300000000000005,"5min":0.995,"15min":0.995},"last":1.6100000000000001}]},{"osd":8,"last update":"Fri Dec 1 01:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":1.635},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.76100000000000001}]},{"osd":9,"last update":"Fri Dec 1 01:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":1.653}]},{"osd":10,"last update":"Fri Dec 1 01:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":1.712}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":892928,"data_stored":424378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:16:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.39500000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.79300000000000004,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53200000000000003,"15min":0.51900000000000002},"min":{"1min":0.32800000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.69299999999999995,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.38100000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.51100000000000001,"15min":0.48499999999999999},"min":{"1min":0.52200000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.52400000000000002,"15min":0.51600000000000001},"min":{"1min":0.47099999999999997,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78400000000000003,"5min":1.115,"15min":1.115},"last":0.61099999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.57699999999999996,"15min":0.55700000000000005},"min":{"1min":0.48899999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88800000000000001,"5min":1.198,"15min":1.198},"last":0.628},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.59399999999999997,"15min":0.57199999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.47999999999999998}]},{"osd":4,"last update":"Fri Dec 1 01:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.42899999999999999,"15min":0.41499999999999998},"min":{"1min":0.30299999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.70799999999999996,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.497,"15min":0.48099999999999998},"min":{"1min":0.36599999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.91100000000000003,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.60199999999999998}]},{"osd":5,"last update":"Fri Dec 1 01:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.53400000000000003,"15min":0.51000000000000001},"min":{"1min":0.45200000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.53100000000000003,"15min":0.51100000000000001},"min":{"1min":0.53000000000000003,"5min":0.316,"15min":0.316},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.63800000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.60299999999999998,"15min":0.59499999999999997},"min":{"1min":0.43099999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.622,"15min":0.61399999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.46400000000000002}]},{"osd":7,"last update":"Fri Dec 1 01:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.55400000000000005,"15min":0.53800000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.88100000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.56399999999999995,"15min":0.54600000000000004},"min":{"1min":0.40300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.878,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.56299999999999994}]},{"osd":8,"last update":"Fri Dec 1 01:15:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.621}]},{"osd":9,"last update":"Fri Dec 1 01:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.69799999999999995}]},{"osd":10,"last update":"Fri Dec 1 01:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1,"5min":1,"15min":1},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.68799999999999994}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738427,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":864,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":884736,"data_stored":422904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 01:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.498,"15min":0.46800000000000003},"min":{"1min":0.35699999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.80600000000000005,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.53000000000000003,"15min":0.49099999999999999},"min":{"1min":0.39500000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.879,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":1.5229999999999999}]},{"osd":2,"last update":"Fri Dec 1 01:15:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.51500000000000001,"15min":0.495},"min":{"1min":0.39300000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":3.4590000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.51200000000000001,"15min":0.5},"min":{"1min":0.35899999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":3.1429999999999998}]},{"osd":3,"last update":"Fri Dec 1 01:14:57 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.36699999999999999,"15min":0.35899999999999999},"min":{"1min":0.247,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.441},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.40799999999999997,"15min":0.40200000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.41599999999999998}]},{"osd":4,"last update":"Fri Dec 1 01:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54300000000000004,"15min":0.53300000000000003},"min":{"1min":0.40300000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.81200000000000006,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.58999999999999997,"15min":0.57499999999999996},"min":{"1min":0.42299999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.92300000000000004,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":1.5649999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.41099999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.83199999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":1.2470000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.60899999999999999,"15min":0.59999999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":1.708}]},{"osd":6,"last update":"Fri Dec 1 01:15:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.54300000000000004,"15min":0.53700000000000003},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":1.679},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.55900000000000005,"15min":0.55300000000000005},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.51900000000000002}]},{"osd":7,"last update":"Fri Dec 1 01:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.623,"15min":0.621},"min":{"1min":0.43099999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67800000000000005,"15min":0.68200000000000005},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":1.6220000000000001}]},{"osd":8,"last update":"Fri Dec 1 01:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":1.431},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":1.4530000000000001}]},{"osd":9,"last update":"Fri Dec 1 01:16:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":1.595},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.54300000000000004}]},{"osd":10,"last update":"Fri Dec 1 01:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.999},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":1.6479999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8740000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411378,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":905216,"data_stored":424986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:15:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.49399999999999999,"15min":0.47899999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.76400000000000001,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.49099999999999999,"15min":0.47699999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.503}]},{"osd":1,"last update":"Fri Dec 1 01:15:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.51600000000000001,"15min":0.50600000000000001},"min":{"1min":0.39100000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.38800000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.438}]},{"osd":3,"last update":"Fri Dec 1 01:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.58599999999999997,"15min":0.52600000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":5.4710000000000001,"5min":5.4710000000000001,"15min":5.4710000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.55400000000000005,"15min":0.52400000000000002},"min":{"1min":0.40899999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.76700000000000002,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.58499999999999996}]},{"osd":4,"last update":"Fri Dec 1 01:16:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.53200000000000003,"15min":0.51800000000000002},"min":{"1min":0.378,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.623},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.53100000000000003,"15min":0.52900000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.45700000000000002}]},{"osd":5,"last update":"Fri Dec 1 01:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.41599999999999998,"15min":0.41499999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.47199999999999998,"15min":0.46400000000000002},"min":{"1min":0.29899999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.497}]},{"osd":6,"last update":"Fri Dec 1 01:16:00 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.64300000000000002,"15min":0.60399999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":5.5019999999999998,"5min":5.5019999999999998,"15min":5.5019999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.63800000000000001,"15min":0.63400000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.56799999999999995}]},{"osd":7,"last update":"Fri Dec 1 01:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.54000000000000004}]},{"osd":8,"last update":"Fri Dec 1 01:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.47899999999999998}]},{"osd":9,"last update":"Fri Dec 1 01:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.63800000000000001}]},{"osd":10,"last update":"Fri Dec 1 01:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":5.4450000000000003,"5min":5.4450000000000003,"15min":5.4450000000000003},"last":0.67400000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]}]},{"osd":3,"up_from":25,"seq":107374182446,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":905216,"data_stored":423222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:16:00 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.33400000000000002,"15min":0.316},"min":{"1min":0.24299999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.35299999999999998,"15min":0.33700000000000002},"min":{"1min":0.29199999999999998,"5min":0.224,"15min":0.224},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.35299999999999998}]},{"osd":1,"last update":"Fri Dec 1 01:16:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.52200000000000002,"15min":0.5},"min":{"1min":0.376,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.52300000000000002,"15min":0.51800000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.60599999999999998}]},{"osd":2,"last update":"Fri Dec 1 01:16:00 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53100000000000003,"15min":0.51500000000000001},"min":{"1min":0.42599999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.74099999999999999,"5min":0.88,"15min":0.88},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.54200000000000004,"15min":0.52500000000000002},"min":{"1min":0.41399999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.432}]},{"osd":4,"last update":"Fri Dec 1 01:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.57799999999999996,"15min":0.55600000000000005},"min":{"1min":0.40899999999999997,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.60399999999999998,"15min":0.58699999999999997},"min":{"1min":0.44500000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.623}]},{"osd":5,"last update":"Fri Dec 1 01:15:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56100000000000005,"15min":0.55600000000000005},"min":{"1min":0.443,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.58299999999999996,"15min":0.57599999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.52100000000000002}]},{"osd":6,"last update":"Fri Dec 1 01:16:00 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.373},{"interface":"front","average":{"1min":0.627,"5min":0.55900000000000005,"15min":0.54700000000000004},"min":{"1min":0.35499999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.50600000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.63300000000000001,"15min":0.63100000000000001},"min":{"1min":0.46300000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.82999999999999996,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.65600000000000003,"15min":0.65100000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.57299999999999995}]},{"osd":8,"last update":"Fri Dec 1 01:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.51200000000000001}]},{"osd":9,"last update":"Fri Dec 1 01:16:05 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.53900000000000003}]},{"osd":10,"last update":"Fri Dec 1 01:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.67900000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]}]},{"osd":4,"up_from":32,"seq":137438953514,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":864,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":884736,"data_stored":422836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.54300000000000004,"15min":0.50900000000000001},"min":{"1min":0.48799999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.373},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.53400000000000003,"15min":0.499},"min":{"1min":0.47199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.497}]},{"osd":1,"last update":"Fri Dec 1 01:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.40500000000000003,"15min":0.378},"min":{"1min":0.29599999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.39800000000000002,"15min":0.371},"min":{"1min":0.32100000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.40500000000000003}]},{"osd":2,"last update":"Fri Dec 1 01:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.55900000000000005,"15min":0.53000000000000003},"min":{"1min":0.48199999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.59499999999999997,"15min":0.56799999999999995},"min":{"1min":0.40400000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.751}]},{"osd":3,"last update":"Fri Dec 1 01:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.53300000000000003,"15min":0.499},"min":{"1min":0.45700000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.97899999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.52200000000000002,"15min":0.504},"min":{"1min":0.373,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.89300000000000002,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.54800000000000004}]},{"osd":5,"last update":"Fri Dec 1 01:15:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.56699999999999995,"15min":0.55500000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.745,"5min":0.61899999999999999,"15min":0.59799999999999998},"min":{"1min":0.52800000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.78600000000000003}]},{"osd":6,"last update":"Fri Dec 1 01:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.61699999999999999,"15min":0.60799999999999998},"min":{"1min":0.46300000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.747,"5min":0.67100000000000004,"15min":0.65900000000000003},"min":{"1min":0.47899999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.64900000000000002}]},{"osd":7,"last update":"Fri Dec 1 01:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.57699999999999996,"15min":0.55700000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.503},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.56599999999999995,"15min":0.55000000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.48099999999999998}]},{"osd":8,"last update":"Fri Dec 1 01:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.81200000000000006}]},{"osd":9,"last update":"Fri Dec 1 01:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.70199999999999996}]},{"osd":10,"last update":"Fri Dec 1 01:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.52800000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]}]},{"osd":5,"up_from":37,"seq":158913789989,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28416,"kb_used_data":1464,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716736,"statfs":{"total":95995035648,"available":95965937664,"internally_reserved":0,"allocated":1499136,"data_stored":1014279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:15:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.51600000000000001,"15min":0.50600000000000001},"min":{"1min":0.378,"5min":0.307,"15min":0.307},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.52700000000000002,"15min":0.52400000000000002},"min":{"1min":0.36699999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.45400000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:15:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.48999999999999999,"15min":0.47999999999999998},"min":{"1min":0.35999999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.50600000000000001,"15min":0.49199999999999999},"min":{"1min":0.439,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.43099999999999999}]},{"osd":2,"last update":"Fri Dec 1 01:15:39 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.38800000000000001,"15min":0.38400000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.377,"15min":0.372},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.34899999999999998}]},{"osd":3,"last update":"Fri Dec 1 01:15:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.53900000000000003,"15min":0.53100000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.53500000000000003,"15min":0.52700000000000002},"min":{"1min":0.39000000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.40600000000000003}]},{"osd":4,"last update":"Fri Dec 1 01:15:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.53700000000000003,"15min":0.52500000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.53100000000000003,"15min":0.52200000000000002},"min":{"1min":0.45200000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.63600000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:15:56 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.629,"15min":0.622},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.63700000000000001,"15min":0.623},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.58599999999999997}]},{"osd":7,"last update":"Fri Dec 1 01:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.64900000000000002,"15min":0.65100000000000002},"min":{"1min":0.49399999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.85299999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.61299999999999999,"15min":0.61099999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.53000000000000003}]},{"osd":8,"last update":"Fri Dec 1 01:15:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.501}]},{"osd":9,"last update":"Fri Dec 1 01:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.48199999999999998}]},{"osd":10,"last update":"Fri Dec 1 01:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.624},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.60999999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T01:16:34.263 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-01T01:16:34.263 INFO:tasks.cephadm:Setting up client nodes... 2023-12-01T01:16:34.264 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-12-01T01:16:34.843 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:16:34.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:34 smithi077 ceph-mon[98867]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-01T01:16:34.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:34 smithi077 ceph-mon[98867]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-01T01:16:34.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:34 smithi077 ceph-mon[98867]: pgmap v257: 97 pgs: 97 active+clean; 616 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 9.6 KiB/s wr, 23 op/s 2023-12-01T01:16:35.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:34 smithi203 ceph-mon[103581]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-01T01:16:35.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:34 smithi203 ceph-mon[103581]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-01T01:16:35.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:34 smithi203 ceph-mon[103581]: pgmap v257: 97 pgs: 97 active+clean; 616 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 9.6 KiB/s wr, 23 op/s 2023-12-01T01:16:35.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:34 smithi137 ceph-mon[103312]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-01T01:16:35.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:34 smithi137 ceph-mon[103312]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-01T01:16:35.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:34 smithi137 ceph-mon[103312]: pgmap v257: 97 pgs: 97 active+clean; 616 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 9.6 KiB/s wr, 23 op/s 2023-12-01T01:16:35.933 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:35.929+0000 7fc460c88700 1 -- 172.21.15.77:0/3521069757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc45c0ff120 msgr2=0x7fc45c0ff540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:35.933 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:35.929+0000 7fc460c88700 1 --2- 172.21.15.77:0/3521069757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc45c0ff120 0x7fc45c0ff540 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc450009990 tx=0x7fc450009ca0 comp rx=0 tx=0).stop 2023-12-01T01:16:35.933 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:35.930+0000 7fc460c88700 1 -- 172.21.15.77:0/3521069757 shutdown_connections 2023-12-01T01:16:35.933 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:35.930+0000 7fc460c88700 1 --2- 172.21.15.77:0/3521069757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc45c102aa0 0x7fc45c109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:35.933 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:35.930+0000 7fc460c88700 1 --2- 172.21.15.77:0/3521069757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc45c0ff120 0x7fc45c0ff540 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:35.933 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:35.930+0000 7fc460c88700 1 --2- 172.21.15.77:0/3521069757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc45c102100 0x7fc45c102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:35.933 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:35.930+0000 7fc460c88700 1 -- 172.21.15.77:0/3521069757 >> 172.21.15.77:0/3521069757 conn(0x7fc45c0faa60 msgr2=0x7fc45c0fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:35.933 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:35.930+0000 7fc460c88700 1 -- 172.21.15.77:0/3521069757 shutdown_connections 2023-12-01T01:16:35.933 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:35.930+0000 7fc460c88700 1 -- 172.21.15.77:0/3521069757 wait complete. 2023-12-01T01:16:35.934 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:35.931+0000 7fc460c88700 1 Processor -- start 2023-12-01T01:16:35.934 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:35.931+0000 7fc460c88700 1 -- start start 2023-12-01T01:16:35.934 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:35.931+0000 7fc460c88700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc45c0ff120 0x7fc45c196ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:35.934 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:35.931+0000 7fc460c88700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc45c102100 0x7fc45c196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:35.934 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:35.931+0000 7fc460c88700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc45c102aa0 0x7fc45c19e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:35.934 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:35.931+0000 7fc460c88700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45c19e640 con 0x7fc45c102aa0 2023-12-01T01:16:35.934 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:35.931+0000 7fc460c88700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45c19e780 con 0x7fc45c102100 2023-12-01T01:16:35.934 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:35.931+0000 7fc460c88700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45c19e910 con 0x7fc45c0ff120 2023-12-01T01:16:35.934 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:35.931+0000 7fc45ad9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc45c102aa0 0x7fc45c19e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:35.934 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:35.931+0000 7fc45ad9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc45c102aa0 0x7fc45c19e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57706/0 (socket says 172.21.15.77:57706) 2023-12-01T01:16:35.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:35.931+0000 7fc45ad9d700 1 -- 172.21.15.77:0/401044859 learned_addr learned my addr 172.21.15.77:0/401044859 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:35.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:35.931+0000 7fc459d9b700 1 --2- 172.21.15.77:0/401044859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc45c102100 0x7fc45c196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:35.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:35.931+0000 7fc45a59c700 1 --2- 172.21.15.77:0/401044859 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc45c0ff120 0x7fc45c196ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:35.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:35.931+0000 7fc45ad9d700 1 -- 172.21.15.77:0/401044859 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc45c0ff120 msgr2=0x7fc45c196ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:35.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:35.931+0000 7fc45ad9d700 1 --2- 172.21.15.77:0/401044859 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc45c0ff120 0x7fc45c196ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:35.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:35.931+0000 7fc45ad9d700 1 -- 172.21.15.77:0/401044859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc45c102100 msgr2=0x7fc45c196ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:35.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:35.931+0000 7fc45ad9d700 1 --2- 172.21.15.77:0/401044859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc45c102100 0x7fc45c196ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:35.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:35.931+0000 7fc45ad9d700 1 -- 172.21.15.77:0/401044859 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4500095f0 con 0x7fc45c102aa0 2023-12-01T01:16:35.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:35.931+0000 7fc45ad9d700 1 --2- 172.21.15.77:0/401044859 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc45c102aa0 0x7fc45c19e070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc45400add0 tx=0x7fc45400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:35.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:35.932+0000 7fc44b7fe700 1 -- 172.21.15.77:0/401044859 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45400f070 con 0x7fc45c102aa0 2023-12-01T01:16:35.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:35.932+0000 7fc460c88700 1 -- 172.21.15.77:0/401044859 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc45c19ebf0 con 0x7fc45c102aa0 2023-12-01T01:16:35.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:35.932+0000 7fc44b7fe700 1 -- 172.21.15.77:0/401044859 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc454007b30 con 0x7fc45c102aa0 2023-12-01T01:16:35.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:35.932+0000 7fc44b7fe700 1 -- 172.21.15.77:0/401044859 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc454005280 con 0x7fc45c102aa0 2023-12-01T01:16:35.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:35.932+0000 7fc460c88700 1 -- 172.21.15.77:0/401044859 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc45c19f0f0 con 0x7fc45c102aa0 2023-12-01T01:16:35.936 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:35.933+0000 7fc460c88700 1 -- 172.21.15.77:0/401044859 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc45c1084f0 con 0x7fc45c102aa0 2023-12-01T01:16:35.936 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:35.934+0000 7fc44b7fe700 1 -- 172.21.15.77:0/401044859 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fc45400c890 con 0x7fc45c102aa0 2023-12-01T01:16:35.936 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:35.934+0000 7fc44b7fe700 1 --2- 172.21.15.77:0/401044859 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc4440a05b0 0x7fc4440a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:35.936 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:35.934+0000 7fc45a59c700 1 --2- 172.21.15.77:0/401044859 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc4440a05b0 0x7fc4440a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:35.937 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:35.935+0000 7fc44b7fe700 1 -- 172.21.15.77:0/401044859 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc4540cae80 con 0x7fc45c102aa0 2023-12-01T01:16:35.937 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:35.935+0000 7fc45a59c700 1 --2- 172.21.15.77:0/401044859 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc4440a05b0 0x7fc4440a2a70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc450006010 tx=0x7fc450005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:35.940 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:35.939+0000 7fc44b7fe700 1 -- 172.21.15.77:0/401044859 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc454094b00 con 0x7fc45c102aa0 2023-12-01T01:16:36.028 INFO:journalctl@ceph.mds.b.smithi137.stdout:Dec 01 01:16:35 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-b[141375]: 2023-12-01T01:16:35.730+0000 7f1cc1df2700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2023-12-01T01:16:36.029 INFO:journalctl@ceph.mds.b.smithi137.stdout:Dec 01 01:16:35 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-b[141375]: 2023-12-01T01:16:35.730+0000 7f1cc1df2700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2023-12-01T01:16:36.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:36.206+0000 7fc460c88700 1 -- 172.21.15.77:0/401044859 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fc45c02cf60 con 0x7fc45c102aa0 2023-12-01T01:16:36.211 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:36.210+0000 7fc44b7fe700 1 -- 172.21.15.77:0/401044859 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7fc454098120 con 0x7fc45c102aa0 2023-12-01T01:16:36.211 INFO:teuthology.orchestra.run.smithi077.stdout:[client.0] 2023-12-01T01:16:36.211 INFO:teuthology.orchestra.run.smithi077.stdout: key = AQD0M2llCmtfDBAAiiSB2KsXX+3tqscwxgNJ6Q== 2023-12-01T01:16:36.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:36.212+0000 7fc460c88700 1 -- 172.21.15.77:0/401044859 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc4440a05b0 msgr2=0x7fc4440a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:36.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:36.212+0000 7fc460c88700 1 --2- 172.21.15.77:0/401044859 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc4440a05b0 0x7fc4440a2a70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc450006010 tx=0x7fc450005960 comp rx=0 tx=0).stop 2023-12-01T01:16:36.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:36.212+0000 7fc460c88700 1 -- 172.21.15.77:0/401044859 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc45c102aa0 msgr2=0x7fc45c19e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:36.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:36.212+0000 7fc460c88700 1 --2- 172.21.15.77:0/401044859 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc45c102aa0 0x7fc45c19e070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc45400add0 tx=0x7fc45400c330 comp rx=0 tx=0).stop 2023-12-01T01:16:36.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:36.212+0000 7fc460c88700 1 -- 172.21.15.77:0/401044859 shutdown_connections 2023-12-01T01:16:36.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:36.212+0000 7fc460c88700 1 --2- 172.21.15.77:0/401044859 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc4440a05b0 0x7fc4440a2a70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:36.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:36.212+0000 7fc460c88700 1 --2- 172.21.15.77:0/401044859 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc45c102aa0 0x7fc45c19e070 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:36.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:36.212+0000 7fc460c88700 1 --2- 172.21.15.77:0/401044859 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc45c0ff120 0x7fc45c196ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:36.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:36.212+0000 7fc460c88700 1 --2- 172.21.15.77:0/401044859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc45c102100 0x7fc45c196ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:36.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:36.212+0000 7fc460c88700 1 -- 172.21.15.77:0/401044859 >> 172.21.15.77:0/401044859 conn(0x7fc45c0faa60 msgr2=0x7fc45c105630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:36.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:36.212+0000 7fc460c88700 1 -- 172.21.15.77:0/401044859 shutdown_connections 2023-12-01T01:16:36.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:36.212+0000 7fc460c88700 1 -- 172.21.15.77:0/401044859 wait complete. 2023-12-01T01:16:36.512 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:36 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/401044859' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-12-01T01:16:36.513 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:36 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/401044859' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-12-01T01:16:36.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:36 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/401044859' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-12-01T01:16:36.529 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:36 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/401044859' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-12-01T01:16:36.598 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:36 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/401044859' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-12-01T01:16:36.599 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:36 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/401044859' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-12-01T01:16:36.600 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-12-01T01:16:36.600 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-12-01T01:16:36.600 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-12-01T01:16:36.641 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-12-01T01:16:36.641 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-12-01T01:16:36.642 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph mgr dump --format=json 2023-12-01T01:16:37.226 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:16:37.713 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:37 smithi077 ceph-mon[98867]: pgmap v258: 97 pgs: 97 active+clean; 619 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 9.8 KiB/s wr, 23 op/s 2023-12-01T01:16:38.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:37 smithi203 ceph-mon[103581]: pgmap v258: 97 pgs: 97 active+clean; 619 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 9.8 KiB/s wr, 23 op/s 2023-12-01T01:16:38.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:37 smithi137 ceph-mon[103312]: pgmap v258: 97 pgs: 97 active+clean; 619 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 9.8 KiB/s wr, 23 op/s 2023-12-01T01:16:38.409 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.405+0000 7fc7cfafc700 1 -- 172.21.15.77:0/396332639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7c81019c0 msgr2=0x7fc7c8105a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:38.409 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.405+0000 7fc7cfafc700 1 --2- 172.21.15.77:0/396332639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7c81019c0 0x7fc7c8105a10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4009990 tx=0x7fc7c4009ca0 comp rx=0 tx=0).stop 2023-12-01T01:16:38.409 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.405+0000 7fc7cfafc700 1 -- 172.21.15.77:0/396332639 shutdown_connections 2023-12-01T01:16:38.409 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.405+0000 7fc7cfafc700 1 --2- 172.21.15.77:0/396332639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7c81060f0 0x7fc7c810dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:38.409 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.405+0000 7fc7cfafc700 1 --2- 172.21.15.77:0/396332639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7c8101010 0x7fc7c81013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:38.409 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.405+0000 7fc7cfafc700 1 --2- 172.21.15.77:0/396332639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7c81019c0 0x7fc7c8105a10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:38.409 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.405+0000 7fc7cfafc700 1 -- 172.21.15.77:0/396332639 >> 172.21.15.77:0/396332639 conn(0x7fc7c80fc880 msgr2=0x7fc7c80feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:38.409 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.405+0000 7fc7cfafc700 1 -- 172.21.15.77:0/396332639 shutdown_connections 2023-12-01T01:16:38.409 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.405+0000 7fc7cfafc700 1 -- 172.21.15.77:0/396332639 wait complete. 2023-12-01T01:16:38.410 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.406+0000 7fc7cfafc700 1 Processor -- start 2023-12-01T01:16:38.410 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.406+0000 7fc7cfafc700 1 -- start start 2023-12-01T01:16:38.410 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.406+0000 7fc7cfafc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7c8101010 0x7fc7c8196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:38.410 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.406+0000 7fc7cfafc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7c81019c0 0x7fc7c8197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:38.410 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.406+0000 7fc7cfafc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7c81060f0 0x7fc7c819b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:38.410 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.406+0000 7fc7cfafc700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c819bef0 con 0x7fc7c81019c0 2023-12-01T01:16:38.410 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.406+0000 7fc7cfafc700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c819c060 con 0x7fc7c8101010 2023-12-01T01:16:38.410 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.406+0000 7fc7cfafc700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c819c220 con 0x7fc7c81060f0 2023-12-01T01:16:38.410 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.406+0000 7fc7cd097700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7c81019c0 0x7fc7c8197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:38.410 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.406+0000 7fc7cd097700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7c81019c0 0x7fc7c8197470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57718/0 (socket says 172.21.15.77:57718) 2023-12-01T01:16:38.410 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.406+0000 7fc7cd097700 1 -- 172.21.15.77:0/1822423742 learned_addr learned my addr 172.21.15.77:0/1822423742 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:38.410 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.406+0000 7fc7cd898700 1 --2- 172.21.15.77:0/1822423742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7c8101010 0x7fc7c8196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:38.410 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.406+0000 7fc7ce099700 1 --2- 172.21.15.77:0/1822423742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7c81060f0 0x7fc7c819b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:38.413 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.407+0000 7fc7cd097700 1 -- 172.21.15.77:0/1822423742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7c81060f0 msgr2=0x7fc7c819b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:38.413 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.407+0000 7fc7cd097700 1 --2- 172.21.15.77:0/1822423742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7c81060f0 0x7fc7c819b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:38.413 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.407+0000 7fc7cd097700 1 -- 172.21.15.77:0/1822423742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7c8101010 msgr2=0x7fc7c8196f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:38.413 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.407+0000 7fc7cd097700 1 --2- 172.21.15.77:0/1822423742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7c8101010 0x7fc7c8196f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:38.413 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.407+0000 7fc7cd097700 1 -- 172.21.15.77:0/1822423742 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c40095f0 con 0x7fc7c81019c0 2023-12-01T01:16:38.413 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.407+0000 7fc7cd097700 1 --2- 172.21.15.77:0/1822423742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7c81019c0 0x7fc7c8197470 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4003730 tx=0x7fc7c400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:38.413 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.407+0000 7fc7beffd700 1 -- 172.21.15.77:0/1822423742 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c401d070 con 0x7fc7c81019c0 2023-12-01T01:16:38.413 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.407+0000 7fc7cfafc700 1 -- 172.21.15.77:0/1822423742 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7c819c4a0 con 0x7fc7c81019c0 2023-12-01T01:16:38.413 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.407+0000 7fc7beffd700 1 -- 172.21.15.77:0/1822423742 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7c4022470 con 0x7fc7c81019c0 2023-12-01T01:16:38.413 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.407+0000 7fc7beffd700 1 -- 172.21.15.77:0/1822423742 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c4017650 con 0x7fc7c81019c0 2023-12-01T01:16:38.413 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.407+0000 7fc7cfafc700 1 -- 172.21.15.77:0/1822423742 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7c81a3d60 con 0x7fc7c81019c0 2023-12-01T01:16:38.414 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.408+0000 7fc7cfafc700 1 -- 172.21.15.77:0/1822423742 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7c804ea90 con 0x7fc7c81019c0 2023-12-01T01:16:38.414 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.409+0000 7fc7beffd700 1 -- 172.21.15.77:0/1822423742 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fc7c400fad0 con 0x7fc7c81019c0 2023-12-01T01:16:38.414 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.409+0000 7fc7beffd700 1 --2- 172.21.15.77:0/1822423742 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc7b40a05b0 0x7fc7b40a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:38.415 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.409+0000 7fc7cd898700 1 --2- 172.21.15.77:0/1822423742 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc7b40a05b0 0x7fc7b40a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:38.415 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.409+0000 7fc7beffd700 1 -- 172.21.15.77:0/1822423742 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc7c40ccdd0 con 0x7fc7c81019c0 2023-12-01T01:16:38.415 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.410+0000 7fc7cd898700 1 --2- 172.21.15.77:0/1822423742 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc7b40a05b0 0x7fc7b40a2a70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8005950 tx=0x7fc7b800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:38.415 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.414+0000 7fc7beffd700 1 -- 172.21.15.77:0/1822423742 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc7c4096a70 con 0x7fc7c81019c0 2023-12-01T01:16:38.665 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.663+0000 7fc7cfafc700 1 -- 172.21.15.77:0/1822423742 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fc7c81a3ef0 con 0x7fc7c81019c0 2023-12-01T01:16:38.669 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.668+0000 7fc7beffd700 1 -- 172.21.15.77:0/1822423742 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+258829 (secure 0 0 0) 0x7fc7c4027070 con 0x7fc7c81019c0 2023-12-01T01:16:38.672 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:16:38.674 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.671+0000 7fc7cfafc700 1 -- 172.21.15.77:0/1822423742 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc7b40a05b0 msgr2=0x7fc7b40a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:38.674 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.671+0000 7fc7cfafc700 1 --2- 172.21.15.77:0/1822423742 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc7b40a05b0 0x7fc7b40a2a70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8005950 tx=0x7fc7b800b410 comp rx=0 tx=0).stop 2023-12-01T01:16:38.674 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.671+0000 7fc7cfafc700 1 -- 172.21.15.77:0/1822423742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7c81019c0 msgr2=0x7fc7c8197470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:38.674 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.671+0000 7fc7cfafc700 1 --2- 172.21.15.77:0/1822423742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7c81019c0 0x7fc7c8197470 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4003730 tx=0x7fc7c400f690 comp rx=0 tx=0).stop 2023-12-01T01:16:38.674 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.671+0000 7fc7cfafc700 1 -- 172.21.15.77:0/1822423742 shutdown_connections 2023-12-01T01:16:38.675 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.671+0000 7fc7cfafc700 1 --2- 172.21.15.77:0/1822423742 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc7b40a05b0 0x7fc7b40a2a70 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:38.675 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.671+0000 7fc7cfafc700 1 --2- 172.21.15.77:0/1822423742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7c81060f0 0x7fc7c819b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:38.675 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.671+0000 7fc7cfafc700 1 --2- 172.21.15.77:0/1822423742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7c8101010 0x7fc7c8196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:38.675 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.671+0000 7fc7cfafc700 1 --2- 172.21.15.77:0/1822423742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7c81019c0 0x7fc7c8197470 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:38.675 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.671+0000 7fc7cfafc700 1 -- 172.21.15.77:0/1822423742 >> 172.21.15.77:0/1822423742 conn(0x7fc7c80fc880 msgr2=0x7fc7c80fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:38.675 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.671+0000 7fc7cfafc700 1 -- 172.21.15.77:0/1822423742 shutdown_connections 2023-12-01T01:16:38.675 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:38.671+0000 7fc7cfafc700 1 -- 172.21.15.77:0/1822423742 wait complete. 2023-12-01T01:16:38.979 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":17,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":3838941679},{"type":"v1","addr":"172.21.15.77:6801","nonce":3838941679}]},"active_addr":"172.21.15.77:6801/3838941679","active_change":"2023-12-01T01:10:24.191951+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":14223,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24106,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.77:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.77:0","nonce":1827570846}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.77:0","nonce":2170743527}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.77:0","nonce":3740941141}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.77:0","nonce":738416770}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.77:0","nonce":1778480102}]}]} 2023-12-01T01:16:38.985 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-12-01T01:16:38.985 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-12-01T01:16:38.985 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph osd dump --format=json 2023-12-01T01:16:39.535 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:16:39.892 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:39 smithi077 ceph-mon[98867]: pgmap v259: 97 pgs: 97 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.8 KiB/s wr, 23 op/s 2023-12-01T01:16:39.892 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:39 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/1822423742' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-12-01T01:16:40.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:39 smithi203 ceph-mon[103581]: pgmap v259: 97 pgs: 97 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.8 KiB/s wr, 23 op/s 2023-12-01T01:16:40.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:39 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/1822423742' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-12-01T01:16:40.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:39 smithi137 ceph-mon[103312]: pgmap v259: 97 pgs: 97 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.8 KiB/s wr, 23 op/s 2023-12-01T01:16:40.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:39 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/1822423742' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-12-01T01:16:40.599 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.596+0000 7fe5f0cb3700 1 -- 172.21.15.77:0/1678842909 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5ec0688b0 msgr2=0x7fe5ec068d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:40.599 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.596+0000 7fe5f0cb3700 1 --2- 172.21.15.77:0/1678842909 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5ec0688b0 0x7fe5ec068d30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0009990 tx=0x7fe5e0009ca0 comp rx=0 tx=0).stop 2023-12-01T01:16:40.600 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.596+0000 7fe5f0cb3700 1 -- 172.21.15.77:0/1678842909 shutdown_connections 2023-12-01T01:16:40.600 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.596+0000 7fe5f0cb3700 1 --2- 172.21.15.77:0/1678842909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5ec069270 0x7fe5ec10dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:40.600 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.596+0000 7fe5f0cb3700 1 --2- 172.21.15.77:0/1678842909 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5ec106f20 0x7fe5ec107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:40.600 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.596+0000 7fe5f0cb3700 1 --2- 172.21.15.77:0/1678842909 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5ec0688b0 0x7fe5ec068d30 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:40.600 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.596+0000 7fe5f0cb3700 1 -- 172.21.15.77:0/1678842909 >> 172.21.15.77:0/1678842909 conn(0x7fe5ec076d90 msgr2=0x7fe5ec0771a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:40.600 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.596+0000 7fe5f0cb3700 1 -- 172.21.15.77:0/1678842909 shutdown_connections 2023-12-01T01:16:40.600 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.596+0000 7fe5f0cb3700 1 -- 172.21.15.77:0/1678842909 wait complete. 2023-12-01T01:16:40.600 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.597+0000 7fe5f0cb3700 1 Processor -- start 2023-12-01T01:16:40.600 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.597+0000 7fe5f0cb3700 1 -- start start 2023-12-01T01:16:40.600 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.597+0000 7fe5f0cb3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5ec0688b0 0x7fe5ec0ffd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:40.600 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.597+0000 7fe5f0cb3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5ec069270 0x7fe5ec100280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:40.600 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.597+0000 7fe5f0cb3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5ec106f20 0x7fe5ec100800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:40.600 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.597+0000 7fe5f0cb3700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ec1a5280 con 0x7fe5ec0688b0 2023-12-01T01:16:40.601 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.597+0000 7fe5f0cb3700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ec1a53f0 con 0x7fe5ec069270 2023-12-01T01:16:40.601 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.597+0000 7fe5f0cb3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ec1a55b0 con 0x7fe5ec106f20 2023-12-01T01:16:40.601 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.597+0000 7fe5ea59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5ec0688b0 0x7fe5ec0ffd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:40.601 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.597+0000 7fe5ea59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5ec0688b0 0x7fe5ec0ffd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57736/0 (socket says 172.21.15.77:57736) 2023-12-01T01:16:40.601 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.597+0000 7fe5ea59c700 1 -- 172.21.15.77:0/4086609598 learned_addr learned my addr 172.21.15.77:0/4086609598 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:40.601 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.597+0000 7fe5ead9d700 1 --2- 172.21.15.77:0/4086609598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5ec106f20 0x7fe5ec100800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:40.601 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.597+0000 7fe5e9d9b700 1 --2- 172.21.15.77:0/4086609598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5ec069270 0x7fe5ec100280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:40.601 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.598+0000 7fe5ea59c700 1 -- 172.21.15.77:0/4086609598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5ec106f20 msgr2=0x7fe5ec100800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:40.601 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.598+0000 7fe5ea59c700 1 --2- 172.21.15.77:0/4086609598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5ec106f20 0x7fe5ec100800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:40.601 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.598+0000 7fe5ea59c700 1 -- 172.21.15.77:0/4086609598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5ec069270 msgr2=0x7fe5ec100280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:40.601 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.598+0000 7fe5ea59c700 1 --2- 172.21.15.77:0/4086609598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5ec069270 0x7fe5ec100280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:40.601 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.598+0000 7fe5ea59c700 1 -- 172.21.15.77:0/4086609598 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e00095f0 con 0x7fe5ec0688b0 2023-12-01T01:16:40.601 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.598+0000 7fe5ea59c700 1 --2- 172.21.15.77:0/4086609598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5ec0688b0 0x7fe5ec0ffd40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc00d980 tx=0x7fe5dc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:40.601 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.598+0000 7fe5db7fe700 1 -- 172.21.15.77:0/4086609598 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc004220 con 0x7fe5ec0688b0 2023-12-01T01:16:40.602 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.598+0000 7fe5db7fe700 1 -- 172.21.15.77:0/4086609598 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5dc007640 con 0x7fe5ec0688b0 2023-12-01T01:16:40.602 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.598+0000 7fe5f0cb3700 1 -- 172.21.15.77:0/4086609598 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5ec1a5890 con 0x7fe5ec0688b0 2023-12-01T01:16:40.602 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.598+0000 7fe5db7fe700 1 -- 172.21.15.77:0/4086609598 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc005690 con 0x7fe5ec0688b0 2023-12-01T01:16:40.602 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.598+0000 7fe5f0cb3700 1 -- 172.21.15.77:0/4086609598 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5ec1a5d60 con 0x7fe5ec0688b0 2023-12-01T01:16:40.603 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.599+0000 7fe5f0cb3700 1 -- 172.21.15.77:0/4086609598 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5ec04ea90 con 0x7fe5ec0688b0 2023-12-01T01:16:40.603 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.600+0000 7fe5db7fe700 1 -- 172.21.15.77:0/4086609598 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fe5dc004750 con 0x7fe5ec0688b0 2023-12-01T01:16:40.603 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.600+0000 7fe5db7fe700 1 --2- 172.21.15.77:0/4086609598 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fe5d40a05b0 0x7fe5d40a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:40.603 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.601+0000 7fe5e9d9b700 1 --2- 172.21.15.77:0/4086609598 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fe5d40a05b0 0x7fe5d40a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:40.603 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.601+0000 7fe5db7fe700 1 -- 172.21.15.77:0/4086609598 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe5dc0cb7e0 con 0x7fe5ec0688b0 2023-12-01T01:16:40.603 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.601+0000 7fe5e9d9b700 1 --2- 172.21.15.77:0/4086609598 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fe5d40a05b0 0x7fe5d40a2a70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe5e000b5c0 tx=0x7fe5e0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:40.606 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.605+0000 7fe5db7fe700 1 -- 172.21.15.77:0/4086609598 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe5dc0952d0 con 0x7fe5ec0688b0 2023-12-01T01:16:40.796 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.795+0000 7fe5f0cb3700 1 -- 172.21.15.77:0/4086609598 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe5ec1a5f80 con 0x7fe5ec0688b0 2023-12-01T01:16:40.798 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.796+0000 7fe5db7fe700 1 -- 172.21.15.77:0/4086609598 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22131 (secure 0 0 0) 0x7fe5dc016090 con 0x7fe5ec0688b0 2023-12-01T01:16:40.798 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:16:40.798 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":82,"fsid":"2a06e29a-8fe6-11ee-95a2-87774f69a715","created":"2023-12-01T01:09:30.210398+0000","modified":"2023-12-01T01:16:26.346791+0000","last_up_change":"2023-12-01T01:15:44.586092+0000","last_in_change":"2023-12-01T01:15:26.305439+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:12:30.514229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:16:23.568860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-01T01:16:24.531137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fc651e62-a103-4ab8-a76d-d3708ea6aeec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6803","nonce":2244799094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6805","nonce":2244799094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6809","nonce":2244799094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6807","nonce":2244799094}]},"public_addr":"172.21.15.77:6803/2244799094","cluster_addr":"172.21.15.77:6805/2244799094","heartbeat_back_addr":"172.21.15.77:6809/2244799094","heartbeat_front_addr":"172.21.15.77:6807/2244799094","state":["exists","up"]},{"osd":1,"uuid":"0adf8c54-49ec-49a0-8bf1-af1f20e60729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6801","nonce":224194890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6803","nonce":224194890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6807","nonce":224194890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6805","nonce":224194890}]},"public_addr":"172.21.15.137:6801/224194890","cluster_addr":"172.21.15.137:6803/224194890","heartbeat_back_addr":"172.21.15.137:6807/224194890","heartbeat_front_addr":"172.21.15.137:6805/224194890","state":["exists","up"]},{"osd":2,"uuid":"5a69802c-dede-4544-9a12-436f5c1c0038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6801","nonce":4078459110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6803","nonce":4078459110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6807","nonce":4078459110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6805","nonce":4078459110}]},"public_addr":"172.21.15.203:6801/4078459110","cluster_addr":"172.21.15.203:6803/4078459110","heartbeat_back_addr":"172.21.15.203:6807/4078459110","heartbeat_front_addr":"172.21.15.203:6805/4078459110","state":["exists","up"]},{"osd":3,"uuid":"ad325a97-580b-4c99-bf12-cbb0d92a6d0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6811","nonce":1454440477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6813","nonce":1454440477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6817","nonce":1454440477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6815","nonce":1454440477}]},"public_addr":"172.21.15.77:6811/1454440477","cluster_addr":"172.21.15.77:6813/1454440477","heartbeat_back_addr":"172.21.15.77:6817/1454440477","heartbeat_front_addr":"172.21.15.77:6815/1454440477","state":["exists","up"]},{"osd":4,"uuid":"1e3ac1ba-98bb-4b8b-8e26-50064c66bf2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6809","nonce":1103241451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6811","nonce":1103241451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6815","nonce":1103241451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6813","nonce":1103241451}]},"public_addr":"172.21.15.137:6809/1103241451","cluster_addr":"172.21.15.137:6811/1103241451","heartbeat_back_addr":"172.21.15.137:6815/1103241451","heartbeat_front_addr":"172.21.15.137:6813/1103241451","state":["exists","up"]},{"osd":5,"uuid":"eb32c016-abeb-44ab-abf0-19dd09235736","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6809","nonce":2648569482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6811","nonce":2648569482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6815","nonce":2648569482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6813","nonce":2648569482}]},"public_addr":"172.21.15.203:6809/2648569482","cluster_addr":"172.21.15.203:6811/2648569482","heartbeat_back_addr":"172.21.15.203:6815/2648569482","heartbeat_front_addr":"172.21.15.203:6813/2648569482","state":["exists","up"]},{"osd":6,"uuid":"0e3fa97e-2ca6-418c-9e8d-a6fe3c56adb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6819","nonce":2463414651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6821","nonce":2463414651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6825","nonce":2463414651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6823","nonce":2463414651}]},"public_addr":"172.21.15.77:6819/2463414651","cluster_addr":"172.21.15.77:6821/2463414651","heartbeat_back_addr":"172.21.15.77:6825/2463414651","heartbeat_front_addr":"172.21.15.77:6823/2463414651","state":["exists","up"]},{"osd":7,"uuid":"d7398c01-0f98-4138-8a49-c3ff0aae7e88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6817","nonce":3108330766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6819","nonce":3108330766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6823","nonce":3108330766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6821","nonce":3108330766}]},"public_addr":"172.21.15.137:6817/3108330766","cluster_addr":"172.21.15.137:6819/3108330766","heartbeat_back_addr":"172.21.15.137:6823/3108330766","heartbeat_front_addr":"172.21.15.137:6821/3108330766","state":["exists","up"]},{"osd":8,"uuid":"0c09050d-3e43-410c-8cd7-bcb891af56b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6817","nonce":1161158776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6819","nonce":1161158776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6823","nonce":1161158776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6821","nonce":1161158776}]},"public_addr":"172.21.15.203:6817/1161158776","cluster_addr":"172.21.15.203:6819/1161158776","heartbeat_back_addr":"172.21.15.203:6823/1161158776","heartbeat_front_addr":"172.21.15.203:6821/1161158776","state":["exists","up"]},{"osd":9,"uuid":"527ecb49-95ec-48bd-9872-3b07fae5c5ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6827","nonce":2409880685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6829","nonce":2409880685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6833","nonce":2409880685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6831","nonce":2409880685}]},"public_addr":"172.21.15.77:6827/2409880685","cluster_addr":"172.21.15.77:6829/2409880685","heartbeat_back_addr":"172.21.15.77:6833/2409880685","heartbeat_front_addr":"172.21.15.77:6831/2409880685","state":["exists","up"]},{"osd":10,"uuid":"1680bb3e-a3c8-427b-af68-e2214f32bed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6825","nonce":374920146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6827","nonce":374920146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6831","nonce":374920146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6829","nonce":374920146}]},"public_addr":"172.21.15.137:6825/374920146","cluster_addr":"172.21.15.137:6827/374920146","heartbeat_back_addr":"172.21.15.137:6831/374920146","heartbeat_front_addr":"172.21.15.137:6829/374920146","state":["exists","up"]},{"osd":11,"uuid":"8a533466-023d-4803-ae36-9a9f1715e49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6825","nonce":2161588962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6827","nonce":2161588962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6831","nonce":2161588962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6829","nonce":2161588962}]},"public_addr":"172.21.15.203:6825/2161588962","cluster_addr":"172.21.15.203:6827/2161588962","heartbeat_back_addr":"172.21.15.203:6831/2161588962","heartbeat_front_addr":"172.21.15.203:6829/2161588962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:11:45.420509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:06.210115+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:28.191307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:49.594116+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:11.261689+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:33.376541+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:54.968336+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:16.775310+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:38.733792+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:59.456699+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:15:20.924059+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:15:43.094773+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/2260035254":"2023-12-02T01:10:24.191835+0000","172.21.15.77:6800/1456621747":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/219838828":"2023-12-02T01:10:24.191835+0000","172.21.15.77:6801/1456621747":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/2937149266":"2023-12-02T01:10:02.232948+0000","172.21.15.77:0/807520018":"2023-12-02T01:10:02.232948+0000","172.21.15.77:6801/1661788026":"2023-12-02T01:10:02.232948+0000","172.21.15.77:6800/1661788026":"2023-12-02T01:10:02.232948+0000","172.21.15.77:0/3551282473":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/1202222558":"2023-12-02T01:10:02.232948+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:16:40.800 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.798+0000 7fe5f0cb3700 1 -- 172.21.15.77:0/4086609598 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fe5d40a05b0 msgr2=0x7fe5d40a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:40.800 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.798+0000 7fe5f0cb3700 1 --2- 172.21.15.77:0/4086609598 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fe5d40a05b0 0x7fe5d40a2a70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe5e000b5c0 tx=0x7fe5e0005960 comp rx=0 tx=0).stop 2023-12-01T01:16:40.800 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.798+0000 7fe5f0cb3700 1 -- 172.21.15.77:0/4086609598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5ec0688b0 msgr2=0x7fe5ec0ffd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:40.800 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.798+0000 7fe5f0cb3700 1 --2- 172.21.15.77:0/4086609598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5ec0688b0 0x7fe5ec0ffd40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc00d980 tx=0x7fe5dc00dc90 comp rx=0 tx=0).stop 2023-12-01T01:16:40.800 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.799+0000 7fe5f0cb3700 1 -- 172.21.15.77:0/4086609598 shutdown_connections 2023-12-01T01:16:40.800 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.799+0000 7fe5f0cb3700 1 --2- 172.21.15.77:0/4086609598 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fe5d40a05b0 0x7fe5d40a2a70 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:40.800 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.799+0000 7fe5f0cb3700 1 --2- 172.21.15.77:0/4086609598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5ec106f20 0x7fe5ec100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:40.800 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.799+0000 7fe5f0cb3700 1 --2- 172.21.15.77:0/4086609598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5ec0688b0 0x7fe5ec0ffd40 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:40.800 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.799+0000 7fe5f0cb3700 1 --2- 172.21.15.77:0/4086609598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5ec069270 0x7fe5ec100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:40.800 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.799+0000 7fe5f0cb3700 1 -- 172.21.15.77:0/4086609598 >> 172.21.15.77:0/4086609598 conn(0x7fe5ec076d90 msgr2=0x7fe5ec0fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:40.801 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.799+0000 7fe5f0cb3700 1 -- 172.21.15.77:0/4086609598 shutdown_connections 2023-12-01T01:16:40.801 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:40.799+0000 7fe5f0cb3700 1 -- 172.21.15.77:0/4086609598 wait complete. 2023-12-01T01:16:41.091 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-12-01T01:16:41.092 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph osd dump --format=json 2023-12-01T01:16:41.643 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:16:41.850 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:41 smithi077 ceph-mon[98867]: pgmap v260: 97 pgs: 97 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.4 KiB/s wr, 23 op/s 2023-12-01T01:16:41.851 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:41 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/4086609598' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:16:42.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:41 smithi203 ceph-mon[103581]: pgmap v260: 97 pgs: 97 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.4 KiB/s wr, 23 op/s 2023-12-01T01:16:42.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:41 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/4086609598' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:16:42.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:41 smithi137 ceph-mon[103312]: pgmap v260: 97 pgs: 97 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.4 KiB/s wr, 23 op/s 2023-12-01T01:16:42.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:41 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/4086609598' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:16:42.828 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:42.824+0000 7f0bfedd6700 1 -- 172.21.15.77:0/579650233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bf8104fe0 msgr2=0x7f0bf810cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:42.828 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:42.824+0000 7f0bfedd6700 1 --2- 172.21.15.77:0/579650233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bf8104fe0 0x7f0bf810cbb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0bf000b0a0 tx=0x7f0bf000b3b0 comp rx=0 tx=0).stop 2023-12-01T01:16:42.828 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:42.825+0000 7f0bfedd6700 1 -- 172.21.15.77:0/579650233 shutdown_connections 2023-12-01T01:16:42.828 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:42.825+0000 7f0bfedd6700 1 --2- 172.21.15.77:0/579650233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bf8104fe0 0x7f0bf810cbb0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:42.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:42.825+0000 7f0bfedd6700 1 --2- 172.21.15.77:0/579650233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bf80fff00 0x7f0bf81002e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:42.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:42.825+0000 7f0bfedd6700 1 --2- 172.21.15.77:0/579650233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0bf81008b0 0x7f0bf8104900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:42.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:42.825+0000 7f0bfedd6700 1 -- 172.21.15.77:0/579650233 >> 172.21.15.77:0/579650233 conn(0x7f0bf80fb790 msgr2=0x7f0bf80fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:42.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:42.825+0000 7f0bfedd6700 1 -- 172.21.15.77:0/579650233 shutdown_connections 2023-12-01T01:16:42.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:42.825+0000 7f0bfedd6700 1 -- 172.21.15.77:0/579650233 wait complete. 2023-12-01T01:16:42.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:42.825+0000 7f0bfedd6700 1 Processor -- start 2023-12-01T01:16:42.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:42.825+0000 7f0bfedd6700 1 -- start start 2023-12-01T01:16:42.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:42.826+0000 7f0bfedd6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0bf80fff00 0x7f0bf8197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:42.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:42.826+0000 7f0bfedd6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bf81008b0 0x7f0bf81983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:42.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:42.826+0000 7f0bfedd6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bf8104fe0 0x7f0bf81a2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:42.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:42.826+0000 7f0bfedd6700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf8198c60 con 0x7f0bf8104fe0 2023-12-01T01:16:42.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:42.826+0000 7f0bfedd6700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf81a3460 con 0x7f0bf81008b0 2023-12-01T01:16:42.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:42.826+0000 7f0bfedd6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf81a35a0 con 0x7f0bf80fff00 2023-12-01T01:16:42.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:42.826+0000 7f0bfd373700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bf8104fe0 0x7f0bf81a2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:42.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:42.826+0000 7f0bfd373700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bf8104fe0 0x7f0bf81a2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57752/0 (socket says 172.21.15.77:57752) 2023-12-01T01:16:42.830 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:42.826+0000 7f0bfd373700 1 -- 172.21.15.77:0/3195276508 learned_addr learned my addr 172.21.15.77:0/3195276508 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:42.830 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:42.826+0000 7f0bf7fff700 1 --2- 172.21.15.77:0/3195276508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bf81008b0 0x7f0bf81983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:42.830 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:42.826+0000 7f0bfcb72700 1 --2- 172.21.15.77:0/3195276508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0bf80fff00 0x7f0bf8197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:42.830 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:42.826+0000 7f0bfd373700 1 -- 172.21.15.77:0/3195276508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0bf80fff00 msgr2=0x7f0bf8197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:42.830 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:42.826+0000 7f0bfd373700 1 --2- 172.21.15.77:0/3195276508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0bf80fff00 0x7f0bf8197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:42.830 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:42.826+0000 7f0bfd373700 1 -- 172.21.15.77:0/3195276508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bf81008b0 msgr2=0x7f0bf81983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:42.830 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:42.826+0000 7f0bfd373700 1 --2- 172.21.15.77:0/3195276508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bf81008b0 0x7f0bf81983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:42.830 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:42.826+0000 7f0bfd373700 1 -- 172.21.15.77:0/3195276508 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf0009d00 con 0x7f0bf8104fe0 2023-12-01T01:16:42.830 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:42.826+0000 7f0bfcb72700 1 --2- 172.21.15.77:0/3195276508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0bf80fff00 0x7f0bf8197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:16:42.830 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:42.826+0000 7f0bfd373700 1 --2- 172.21.15.77:0/3195276508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bf8104fe0 0x7f0bf81a2f20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0bf000b830 tx=0x7f0bf00079b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:42.830 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:42.826+0000 7f0bf5ffb700 1 -- 172.21.15.77:0/3195276508 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf000e040 con 0x7f0bf8104fe0 2023-12-01T01:16:42.830 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:42.826+0000 7f0bf5ffb700 1 -- 172.21.15.77:0/3195276508 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bf0004d60 con 0x7f0bf8104fe0 2023-12-01T01:16:42.830 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:42.827+0000 7f0bfedd6700 1 -- 172.21.15.77:0/3195276508 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bf81a37d0 con 0x7f0bf8104fe0 2023-12-01T01:16:42.830 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:42.827+0000 7f0bf5ffb700 1 -- 172.21.15.77:0/3195276508 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf001d480 con 0x7f0bf8104fe0 2023-12-01T01:16:42.831 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:42.827+0000 7f0bfedd6700 1 -- 172.21.15.77:0/3195276508 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bf81a3b90 con 0x7f0bf8104fe0 2023-12-01T01:16:42.831 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:42.828+0000 7f0bfedd6700 1 -- 172.21.15.77:0/3195276508 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bf804ea90 con 0x7f0bf8104fe0 2023-12-01T01:16:42.831 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:42.829+0000 7f0bf5ffb700 1 -- 172.21.15.77:0/3195276508 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f0bf0003a00 con 0x7f0bf8104fe0 2023-12-01T01:16:42.832 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:42.829+0000 7f0bf5ffb700 1 --2- 172.21.15.77:0/3195276508 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f0be80a05b0 0x7f0be80a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:42.832 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:42.829+0000 7f0bfcb72700 1 --2- 172.21.15.77:0/3195276508 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f0be80a05b0 0x7f0be80a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:42.832 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:42.829+0000 7f0bf5ffb700 1 -- 172.21.15.77:0/3195276508 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0bf00cd7d0 con 0x7f0bf8104fe0 2023-12-01T01:16:42.832 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:42.830+0000 7f0bfcb72700 1 --2- 172.21.15.77:0/3195276508 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f0be80a05b0 0x7f0be80a2a70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0bec005fd0 tx=0x7f0bec005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:42.834 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:42.833+0000 7f0bf5ffb700 1 -- 172.21.15.77:0/3195276508 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0bf0097450 con 0x7f0bf8104fe0 2023-12-01T01:16:43.024 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:43.023+0000 7f0bfedd6700 1 -- 172.21.15.77:0/3195276508 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0bf81a3db0 con 0x7f0bf8104fe0 2023-12-01T01:16:43.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:43.024+0000 7f0bf5ffb700 1 -- 172.21.15.77:0/3195276508 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22131 (secure 0 0 0) 0x7f0bf009aa70 con 0x7f0bf8104fe0 2023-12-01T01:16:43.026 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:16:43.026 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":82,"fsid":"2a06e29a-8fe6-11ee-95a2-87774f69a715","created":"2023-12-01T01:09:30.210398+0000","modified":"2023-12-01T01:16:26.346791+0000","last_up_change":"2023-12-01T01:15:44.586092+0000","last_in_change":"2023-12-01T01:15:26.305439+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:12:30.514229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:16:23.568860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-01T01:16:24.531137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fc651e62-a103-4ab8-a76d-d3708ea6aeec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6803","nonce":2244799094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6805","nonce":2244799094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6809","nonce":2244799094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6807","nonce":2244799094}]},"public_addr":"172.21.15.77:6803/2244799094","cluster_addr":"172.21.15.77:6805/2244799094","heartbeat_back_addr":"172.21.15.77:6809/2244799094","heartbeat_front_addr":"172.21.15.77:6807/2244799094","state":["exists","up"]},{"osd":1,"uuid":"0adf8c54-49ec-49a0-8bf1-af1f20e60729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6801","nonce":224194890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6803","nonce":224194890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6807","nonce":224194890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6805","nonce":224194890}]},"public_addr":"172.21.15.137:6801/224194890","cluster_addr":"172.21.15.137:6803/224194890","heartbeat_back_addr":"172.21.15.137:6807/224194890","heartbeat_front_addr":"172.21.15.137:6805/224194890","state":["exists","up"]},{"osd":2,"uuid":"5a69802c-dede-4544-9a12-436f5c1c0038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6801","nonce":4078459110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6803","nonce":4078459110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6807","nonce":4078459110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6805","nonce":4078459110}]},"public_addr":"172.21.15.203:6801/4078459110","cluster_addr":"172.21.15.203:6803/4078459110","heartbeat_back_addr":"172.21.15.203:6807/4078459110","heartbeat_front_addr":"172.21.15.203:6805/4078459110","state":["exists","up"]},{"osd":3,"uuid":"ad325a97-580b-4c99-bf12-cbb0d92a6d0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6811","nonce":1454440477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6813","nonce":1454440477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6817","nonce":1454440477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6815","nonce":1454440477}]},"public_addr":"172.21.15.77:6811/1454440477","cluster_addr":"172.21.15.77:6813/1454440477","heartbeat_back_addr":"172.21.15.77:6817/1454440477","heartbeat_front_addr":"172.21.15.77:6815/1454440477","state":["exists","up"]},{"osd":4,"uuid":"1e3ac1ba-98bb-4b8b-8e26-50064c66bf2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6809","nonce":1103241451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6811","nonce":1103241451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6815","nonce":1103241451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6813","nonce":1103241451}]},"public_addr":"172.21.15.137:6809/1103241451","cluster_addr":"172.21.15.137:6811/1103241451","heartbeat_back_addr":"172.21.15.137:6815/1103241451","heartbeat_front_addr":"172.21.15.137:6813/1103241451","state":["exists","up"]},{"osd":5,"uuid":"eb32c016-abeb-44ab-abf0-19dd09235736","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6809","nonce":2648569482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6811","nonce":2648569482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6815","nonce":2648569482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6813","nonce":2648569482}]},"public_addr":"172.21.15.203:6809/2648569482","cluster_addr":"172.21.15.203:6811/2648569482","heartbeat_back_addr":"172.21.15.203:6815/2648569482","heartbeat_front_addr":"172.21.15.203:6813/2648569482","state":["exists","up"]},{"osd":6,"uuid":"0e3fa97e-2ca6-418c-9e8d-a6fe3c56adb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6819","nonce":2463414651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6821","nonce":2463414651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6825","nonce":2463414651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6823","nonce":2463414651}]},"public_addr":"172.21.15.77:6819/2463414651","cluster_addr":"172.21.15.77:6821/2463414651","heartbeat_back_addr":"172.21.15.77:6825/2463414651","heartbeat_front_addr":"172.21.15.77:6823/2463414651","state":["exists","up"]},{"osd":7,"uuid":"d7398c01-0f98-4138-8a49-c3ff0aae7e88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6817","nonce":3108330766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6819","nonce":3108330766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6823","nonce":3108330766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6821","nonce":3108330766}]},"public_addr":"172.21.15.137:6817/3108330766","cluster_addr":"172.21.15.137:6819/3108330766","heartbeat_back_addr":"172.21.15.137:6823/3108330766","heartbeat_front_addr":"172.21.15.137:6821/3108330766","state":["exists","up"]},{"osd":8,"uuid":"0c09050d-3e43-410c-8cd7-bcb891af56b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6817","nonce":1161158776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6819","nonce":1161158776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6823","nonce":1161158776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6821","nonce":1161158776}]},"public_addr":"172.21.15.203:6817/1161158776","cluster_addr":"172.21.15.203:6819/1161158776","heartbeat_back_addr":"172.21.15.203:6823/1161158776","heartbeat_front_addr":"172.21.15.203:6821/1161158776","state":["exists","up"]},{"osd":9,"uuid":"527ecb49-95ec-48bd-9872-3b07fae5c5ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6827","nonce":2409880685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6829","nonce":2409880685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6833","nonce":2409880685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6831","nonce":2409880685}]},"public_addr":"172.21.15.77:6827/2409880685","cluster_addr":"172.21.15.77:6829/2409880685","heartbeat_back_addr":"172.21.15.77:6833/2409880685","heartbeat_front_addr":"172.21.15.77:6831/2409880685","state":["exists","up"]},{"osd":10,"uuid":"1680bb3e-a3c8-427b-af68-e2214f32bed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6825","nonce":374920146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6827","nonce":374920146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6831","nonce":374920146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6829","nonce":374920146}]},"public_addr":"172.21.15.137:6825/374920146","cluster_addr":"172.21.15.137:6827/374920146","heartbeat_back_addr":"172.21.15.137:6831/374920146","heartbeat_front_addr":"172.21.15.137:6829/374920146","state":["exists","up"]},{"osd":11,"uuid":"8a533466-023d-4803-ae36-9a9f1715e49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6825","nonce":2161588962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6827","nonce":2161588962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6831","nonce":2161588962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6829","nonce":2161588962}]},"public_addr":"172.21.15.203:6825/2161588962","cluster_addr":"172.21.15.203:6827/2161588962","heartbeat_back_addr":"172.21.15.203:6831/2161588962","heartbeat_front_addr":"172.21.15.203:6829/2161588962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:11:45.420509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:06.210115+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:28.191307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:49.594116+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:11.261689+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:33.376541+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:54.968336+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:16.775310+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:38.733792+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:59.456699+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:15:20.924059+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:15:43.094773+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/2260035254":"2023-12-02T01:10:24.191835+0000","172.21.15.77:6800/1456621747":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/219838828":"2023-12-02T01:10:24.191835+0000","172.21.15.77:6801/1456621747":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/2937149266":"2023-12-02T01:10:02.232948+0000","172.21.15.77:0/807520018":"2023-12-02T01:10:02.232948+0000","172.21.15.77:6801/1661788026":"2023-12-02T01:10:02.232948+0000","172.21.15.77:6800/1661788026":"2023-12-02T01:10:02.232948+0000","172.21.15.77:0/3551282473":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/1202222558":"2023-12-02T01:10:02.232948+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:16:43.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:43.026+0000 7f0bfedd6700 1 -- 172.21.15.77:0/3195276508 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f0be80a05b0 msgr2=0x7f0be80a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:43.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:43.026+0000 7f0bfedd6700 1 --2- 172.21.15.77:0/3195276508 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f0be80a05b0 0x7f0be80a2a70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0bec005fd0 tx=0x7f0bec005ee0 comp rx=0 tx=0).stop 2023-12-01T01:16:43.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:43.026+0000 7f0bfedd6700 1 -- 172.21.15.77:0/3195276508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bf8104fe0 msgr2=0x7f0bf81a2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:43.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:43.026+0000 7f0bfedd6700 1 --2- 172.21.15.77:0/3195276508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bf8104fe0 0x7f0bf81a2f20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0bf000b830 tx=0x7f0bf00079b0 comp rx=0 tx=0).stop 2023-12-01T01:16:43.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:43.027+0000 7f0bfedd6700 1 -- 172.21.15.77:0/3195276508 shutdown_connections 2023-12-01T01:16:43.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:43.027+0000 7f0bfedd6700 1 --2- 172.21.15.77:0/3195276508 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f0be80a05b0 0x7f0be80a2a70 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:43.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:43.027+0000 7f0bfedd6700 1 --2- 172.21.15.77:0/3195276508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bf8104fe0 0x7f0bf81a2f20 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:43.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:43.027+0000 7f0bfedd6700 1 --2- 172.21.15.77:0/3195276508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0bf80fff00 0x7f0bf8197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:43.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:43.027+0000 7f0bfedd6700 1 --2- 172.21.15.77:0/3195276508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bf81008b0 0x7f0bf81983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:43.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:43.027+0000 7f0bfedd6700 1 -- 172.21.15.77:0/3195276508 >> 172.21.15.77:0/3195276508 conn(0x7f0bf80fb790 msgr2=0x7f0bf80fc200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:43.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:43.027+0000 7f0bfedd6700 1 -- 172.21.15.77:0/3195276508 shutdown_connections 2023-12-01T01:16:43.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:43.027+0000 7f0bfedd6700 1 -- 172.21.15.77:0/3195276508 wait complete. 2023-12-01T01:16:43.482 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2023-12-01T01:16:43.483 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2023-12-01T01:16:43.483 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2023-12-01T01:16:43.484 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2023-12-01T01:16:43.484 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2023-12-01T01:16:43.484 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2023-12-01T01:16:43.485 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2023-12-01T01:16:43.485 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2023-12-01T01:16:43.486 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2023-12-01T01:16:43.486 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2023-12-01T01:16:43.486 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2023-12-01T01:16:43.487 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2023-12-01T01:16:43.631 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:43 smithi077 ceph-mon[98867]: pgmap v261: 97 pgs: 97 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.3 KiB/s wr, 21 op/s 2023-12-01T01:16:43.631 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:43 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/3195276508' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:16:44.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:43 smithi203 ceph-mon[103581]: pgmap v261: 97 pgs: 97 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.3 KiB/s wr, 21 op/s 2023-12-01T01:16:44.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:43 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/3195276508' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:16:44.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:43 smithi137 ceph-mon[103312]: pgmap v261: 97 pgs: 97 active+clean; 623 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.3 KiB/s wr, 21 op/s 2023-12-01T01:16:44.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:43 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/3195276508' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:16:44.959 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:16:44.991 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:16:45.001 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:16:45.092 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:16:45.092 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:16:45.092 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:16:45.217 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:16:45.222 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:16:45.295 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:16:45.326 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:16:45.458 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:16:45.458 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:16:45.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:45 smithi077 ceph-mon[98867]: pgmap v262: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 KiB/s wr, 21 op/s 2023-12-01T01:16:46.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:45 smithi203 ceph-mon[103581]: pgmap v262: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 KiB/s wr, 21 op/s 2023-12-01T01:16:46.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:45 smithi137 ceph-mon[103312]: pgmap v262: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 KiB/s wr, 21 op/s 2023-12-01T01:16:47.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:47 smithi077 ceph-mon[98867]: pgmap v263: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.9 KiB/s wr, 14 op/s 2023-12-01T01:16:48.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:47 smithi203 ceph-mon[103581]: pgmap v263: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.9 KiB/s wr, 14 op/s 2023-12-01T01:16:48.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:47 smithi137 ceph-mon[103312]: pgmap v263: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.9 KiB/s wr, 14 op/s 2023-12-01T01:16:49.760 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:49 smithi077 ceph-mon[98867]: pgmap v264: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2023-12-01T01:16:50.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:49 smithi203 ceph-mon[103581]: pgmap v264: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2023-12-01T01:16:50.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:49 smithi137 ceph-mon[103312]: pgmap v264: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2023-12-01T01:16:50.176 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.602+0000 7f46fbbd8700 1 -- 172.21.15.77:0/880032290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46f41022b0 msgr2=0x7f46f410cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:50.176 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.602+0000 7f46fbbd8700 1 --2- 172.21.15.77:0/880032290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46f41022b0 0x7f46f410cc90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f46ec00b0d0 tx=0x7f46ec00b3e0 comp rx=0 tx=0).stop 2023-12-01T01:16:50.176 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.602+0000 7f46fbbd8700 1 -- 172.21.15.77:0/880032290 shutdown_connections 2023-12-01T01:16:50.176 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.602+0000 7f46fbbd8700 1 --2- 172.21.15.77:0/880032290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46f41022b0 0x7f46f410cc90 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:50.176 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.602+0000 7f46fbbd8700 1 --2- 172.21.15.77:0/880032290 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46f4100010 0x7f46f41003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:50.176 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.602+0000 7f46fbbd8700 1 --2- 172.21.15.77:0/880032290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46f41018f0 0x7f46f4101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:50.176 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.602+0000 7f46fbbd8700 1 -- 172.21.15.77:0/880032290 >> 172.21.15.77:0/880032290 conn(0x7f46f4076d80 msgr2=0x7f46f4077190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:50.177 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.602+0000 7f46fbbd8700 1 -- 172.21.15.77:0/880032290 shutdown_connections 2023-12-01T01:16:50.177 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.602+0000 7f46fbbd8700 1 -- 172.21.15.77:0/880032290 wait complete. 2023-12-01T01:16:50.177 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.602+0000 7f46fbbd8700 1 Processor -- start 2023-12-01T01:16:50.177 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.602+0000 7f46fbbd8700 1 -- start start 2023-12-01T01:16:50.177 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.602+0000 7f46fbbd8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46f4100010 0x7f46f4197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:50.177 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.603+0000 7f46fbbd8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46f41018f0 0x7f46f41983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:50.177 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.603+0000 7f46fbbd8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46f41022b0 0x7f46f41a2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:50.177 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.603+0000 7f46fbbd8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f4198ce0 con 0x7f46f4100010 2023-12-01T01:16:50.177 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.603+0000 7f46fbbd8700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f41a3450 con 0x7f46f41022b0 2023-12-01T01:16:50.177 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.603+0000 7f46fbbd8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f41a35c0 con 0x7f46f41018f0 2023-12-01T01:16:50.177 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.603+0000 7f46f9974700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46f4100010 0x7f46f4197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:50.177 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.603+0000 7f46f9974700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46f4100010 0x7f46f4197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57762/0 (socket says 172.21.15.77:57762) 2023-12-01T01:16:50.177 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.603+0000 7f46f9974700 1 -- 172.21.15.77:0/964238225 learned_addr learned my addr 172.21.15.77:0/964238225 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:50.177 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.603+0000 7f46fa175700 1 --2- 172.21.15.77:0/964238225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46f41022b0 0x7f46f41a2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:50.177 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.603+0000 7f46f9173700 1 --2- 172.21.15.77:0/964238225 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46f41018f0 0x7f46f41983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:50.178 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.603+0000 7f46f9974700 1 -- 172.21.15.77:0/964238225 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46f41018f0 msgr2=0x7f46f41983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:50.178 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.603+0000 7f46f9974700 1 --2- 172.21.15.77:0/964238225 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46f41018f0 0x7f46f41983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:50.178 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.603+0000 7f46f9974700 1 -- 172.21.15.77:0/964238225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46f41022b0 msgr2=0x7f46f41a2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:50.178 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.603+0000 7f46f9974700 1 --2- 172.21.15.77:0/964238225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46f41022b0 0x7f46f41a2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:50.178 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.603+0000 7f46f9974700 1 -- 172.21.15.77:0/964238225 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46ec009d00 con 0x7f46f4100010 2023-12-01T01:16:50.178 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.603+0000 7f46f9974700 1 --2- 172.21.15.77:0/964238225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46f4100010 0x7f46f4197e90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f46f000d980 tx=0x7f46f000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:50.178 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.603+0000 7f46eaffd700 1 -- 172.21.15.77:0/964238225 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f0004a30 con 0x7f46f4100010 2023-12-01T01:16:50.178 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.603+0000 7f46eaffd700 1 -- 172.21.15.77:0/964238225 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46f00054e0 con 0x7f46f4100010 2023-12-01T01:16:50.178 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.603+0000 7f46fbbd8700 1 -- 172.21.15.77:0/964238225 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46f41a38a0 con 0x7f46f4100010 2023-12-01T01:16:50.178 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.603+0000 7f46eaffd700 1 -- 172.21.15.77:0/964238225 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f0009c90 con 0x7f46f4100010 2023-12-01T01:16:50.178 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.603+0000 7f46fbbd8700 1 -- 172.21.15.77:0/964238225 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46f41a3da0 con 0x7f46f4100010 2023-12-01T01:16:50.178 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.604+0000 7f46fbbd8700 1 -- 172.21.15.77:0/964238225 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f46f404ea90 con 0x7f46f4100010 2023-12-01T01:16:50.178 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.605+0000 7f46eaffd700 1 -- 172.21.15.77:0/964238225 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f46f0004c20 con 0x7f46f4100010 2023-12-01T01:16:50.179 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.606+0000 7f46eaffd700 1 --2- 172.21.15.77:0/964238225 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f46e00a06b0 0x7f46e00a2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:50.179 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.606+0000 7f46f9173700 1 --2- 172.21.15.77:0/964238225 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f46e00a06b0 0x7f46e00a2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:50.179 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.606+0000 7f46eaffd700 1 -- 172.21.15.77:0/964238225 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f46f00cbe00 con 0x7f46f4100010 2023-12-01T01:16:50.179 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.606+0000 7f46eaffd700 1 --2- 172.21.15.77:0/964238225 >> [v2:172.21.15.137:6808/1103241451,v1:172.21.15.137:6809/1103241451] conn(0x7f46e00a9e90 0x7f46e00ac2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:50.179 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.606+0000 7f46eaffd700 1 -- 172.21.15.77:0/964238225 --> [v2:172.21.15.137:6808/1103241451,v1:172.21.15.137:6809/1103241451] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f46e00ac960 con 0x7f46e00a9e90 2023-12-01T01:16:50.179 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.606+0000 7f46eaffd700 1 -- 172.21.15.77:0/964238225 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46f00cc1b0 con 0x7f46f4100010 2023-12-01T01:16:50.179 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.606+0000 7f46f9173700 1 --2- 172.21.15.77:0/964238225 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f46e00a06b0 0x7f46e00a2b70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f46e4005950 tx=0x7f46e40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:50.179 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.606+0000 7f46fa175700 1 --2- 172.21.15.77:0/964238225 >> [v2:172.21.15.137:6808/1103241451,v1:172.21.15.137:6809/1103241451] conn(0x7f46e00a9e90 0x7f46e00ac2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:50.180 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.607+0000 7f46fa175700 1 --2- 172.21.15.77:0/964238225 >> [v2:172.21.15.137:6808/1103241451,v1:172.21.15.137:6809/1103241451] conn(0x7f46e00a9e90 0x7f46e00ac2b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:50.180 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.607+0000 7f46eaffd700 1 -- 172.21.15.77:0/964238225 <== osd.4 v2:172.21.15.137:6808/1103241451 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f46e00ac960 con 0x7f46e00a9e90 2023-12-01T01:16:50.180 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.626+0000 7f46fbbd8700 1 -- 172.21.15.77:0/964238225 --> [v2:172.21.15.137:6808/1103241451,v1:172.21.15.137:6809/1103241451] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f46f4110010 con 0x7f46e00a9e90 2023-12-01T01:16:50.180 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.627+0000 7f46eaffd700 1 -- 172.21.15.77:0/964238225 <== osd.4 v2:172.21.15.137:6808/1103241451 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f46ec009b80 con 0x7f46e00a9e90 2023-12-01T01:16:50.180 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.627+0000 7f46fbbd8700 1 -- 172.21.15.77:0/964238225 >> [v2:172.21.15.137:6808/1103241451,v1:172.21.15.137:6809/1103241451] conn(0x7f46e00a9e90 msgr2=0x7f46e00ac2b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:50.180 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.627+0000 7f46fbbd8700 1 --2- 172.21.15.77:0/964238225 >> [v2:172.21.15.137:6808/1103241451,v1:172.21.15.137:6809/1103241451] conn(0x7f46e00a9e90 0x7f46e00ac2b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:50.180 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.627+0000 7f46fbbd8700 1 -- 172.21.15.77:0/964238225 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f46e00a06b0 msgr2=0x7f46e00a2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:50.180 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.627+0000 7f46fbbd8700 1 --2- 172.21.15.77:0/964238225 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f46e00a06b0 0x7f46e00a2b70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f46e4005950 tx=0x7f46e40058e0 comp rx=0 tx=0).stop 2023-12-01T01:16:50.180 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.627+0000 7f46fbbd8700 1 -- 172.21.15.77:0/964238225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46f4100010 msgr2=0x7f46f4197e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:50.180 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.627+0000 7f46fbbd8700 1 --2- 172.21.15.77:0/964238225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46f4100010 0x7f46f4197e90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f46f000d980 tx=0x7f46f000dc90 comp rx=0 tx=0).stop 2023-12-01T01:16:50.180 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.627+0000 7f46fa175700 1 -- 172.21.15.77:0/964238225 reap_dead start 2023-12-01T01:16:50.181 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.627+0000 7f46fbbd8700 1 -- 172.21.15.77:0/964238225 shutdown_connections 2023-12-01T01:16:50.181 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.627+0000 7f46fbbd8700 1 -- 172.21.15.77:0/964238225 >> 172.21.15.77:0/964238225 conn(0x7f46f4076d80 msgr2=0x7f46f40fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:50.181 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.627+0000 7f46fbbd8700 1 -- 172.21.15.77:0/964238225 shutdown_connections 2023-12-01T01:16:50.181 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:48.627+0000 7f46fbbd8700 1 -- 172.21.15.77:0/964238225 wait complete. 2023-12-01T01:16:50.876 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.434+0000 7f59f46a7700 1 -- 172.21.15.77:0/2152615263 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59ec1022b0 msgr2=0x7f59ec10cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:50.877 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.434+0000 7f59f46a7700 1 --2- 172.21.15.77:0/2152615263 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59ec1022b0 0x7f59ec10cc90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f59e400b0d0 tx=0x7f59e400b3e0 comp rx=0 tx=0).stop 2023-12-01T01:16:50.877 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.434+0000 7f59f46a7700 1 -- 172.21.15.77:0/2152615263 shutdown_connections 2023-12-01T01:16:50.877 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.434+0000 7f59f46a7700 1 --2- 172.21.15.77:0/2152615263 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59ec1022b0 0x7f59ec10cc90 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:50.877 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.434+0000 7f59f46a7700 1 --2- 172.21.15.77:0/2152615263 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59ec100010 0x7f59ec1003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:50.877 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.434+0000 7f59f46a7700 1 --2- 172.21.15.77:0/2152615263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59ec1018f0 0x7f59ec101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:50.877 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.434+0000 7f59f46a7700 1 -- 172.21.15.77:0/2152615263 >> 172.21.15.77:0/2152615263 conn(0x7f59ec076d80 msgr2=0x7f59ec077190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:50.877 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.434+0000 7f59f46a7700 1 -- 172.21.15.77:0/2152615263 shutdown_connections 2023-12-01T01:16:50.877 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.434+0000 7f59f46a7700 1 -- 172.21.15.77:0/2152615263 wait complete. 2023-12-01T01:16:50.877 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.435+0000 7f59f46a7700 1 Processor -- start 2023-12-01T01:16:50.877 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.435+0000 7f59f46a7700 1 -- start start 2023-12-01T01:16:50.877 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.435+0000 7f59f46a7700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59ec100010 0x7f59ec195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:50.877 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.435+0000 7f59f46a7700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59ec1018f0 0x7f59ec1963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:50.878 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.435+0000 7f59f46a7700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59ec1022b0 0x7f59ec19a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:50.878 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.435+0000 7f59f46a7700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ec19ae70 con 0x7f59ec1022b0 2023-12-01T01:16:50.878 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.435+0000 7f59f46a7700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ec19afe0 con 0x7f59ec100010 2023-12-01T01:16:50.878 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.435+0000 7f59f46a7700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ec19b1a0 con 0x7f59ec1018f0 2023-12-01T01:16:50.878 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.435+0000 7f59f2c44700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59ec1022b0 0x7f59ec19a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:50.878 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.435+0000 7f59f2c44700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59ec1022b0 0x7f59ec19a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57780/0 (socket says 172.21.15.77:57780) 2023-12-01T01:16:50.878 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.435+0000 7f59f2c44700 1 -- 172.21.15.77:0/154240733 learned_addr learned my addr 172.21.15.77:0/154240733 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:50.878 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.435+0000 7f59f2443700 1 --2- 172.21.15.77:0/154240733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59ec100010 0x7f59ec195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:50.878 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.435+0000 7f59f1c42700 1 --2- 172.21.15.77:0/154240733 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59ec1018f0 0x7f59ec1963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:50.878 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.435+0000 7f59f2c44700 1 -- 172.21.15.77:0/154240733 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59ec1018f0 msgr2=0x7f59ec1963f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:50.878 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.435+0000 7f59f2c44700 1 --2- 172.21.15.77:0/154240733 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59ec1018f0 0x7f59ec1963f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:50.878 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.436+0000 7f59f2c44700 1 -- 172.21.15.77:0/154240733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59ec100010 msgr2=0x7f59ec195eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:50.878 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.436+0000 7f59f2c44700 1 --2- 172.21.15.77:0/154240733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59ec100010 0x7f59ec195eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:50.878 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.436+0000 7f59f2c44700 1 -- 172.21.15.77:0/154240733 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e4009d00 con 0x7f59ec1022b0 2023-12-01T01:16:50.879 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.436+0000 7f59f1c42700 1 --2- 172.21.15.77:0/154240733 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59ec1018f0 0x7f59ec1963f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:16:50.879 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.436+0000 7f59f2c44700 1 --2- 172.21.15.77:0/154240733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59ec1022b0 0x7f59ec19a860 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f59e4007870 tx=0x7f59e40078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:50.879 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.436+0000 7f59e37fe700 1 -- 172.21.15.77:0/154240733 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e400e040 con 0x7f59ec1022b0 2023-12-01T01:16:50.879 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.436+0000 7f59f46a7700 1 -- 172.21.15.77:0/154240733 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59ec19b420 con 0x7f59ec1022b0 2023-12-01T01:16:50.879 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.436+0000 7f59e37fe700 1 -- 172.21.15.77:0/154240733 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59e4003e10 con 0x7f59ec1022b0 2023-12-01T01:16:50.879 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.436+0000 7f59e37fe700 1 -- 172.21.15.77:0/154240733 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e401b980 con 0x7f59ec1022b0 2023-12-01T01:16:50.879 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.436+0000 7f59f46a7700 1 -- 172.21.15.77:0/154240733 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59ec1a2b10 con 0x7f59ec1022b0 2023-12-01T01:16:50.879 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.437+0000 7f59f46a7700 1 -- 172.21.15.77:0/154240733 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f59ec04ea90 con 0x7f59ec1022b0 2023-12-01T01:16:50.880 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.438+0000 7f59e37fe700 1 -- 172.21.15.77:0/154240733 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f59e4024400 con 0x7f59ec1022b0 2023-12-01T01:16:50.880 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.438+0000 7f59e37fe700 1 --2- 172.21.15.77:0/154240733 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f59d80a05b0 0x7f59d80a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:50.880 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.439+0000 7f59f2443700 1 --2- 172.21.15.77:0/154240733 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f59d80a05b0 0x7f59d80a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:50.880 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.439+0000 7f59e37fe700 1 -- 172.21.15.77:0/154240733 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f59e40cd1f0 con 0x7f59ec1022b0 2023-12-01T01:16:50.880 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.439+0000 7f59e37fe700 1 --2- 172.21.15.77:0/154240733 >> [v2:172.21.15.77:6818/2463414651,v1:172.21.15.77:6819/2463414651] conn(0x7f59d80a9d90 0x7f59d80ac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:50.880 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.439+0000 7f59e37fe700 1 -- 172.21.15.77:0/154240733 --> [v2:172.21.15.77:6818/2463414651,v1:172.21.15.77:6819/2463414651] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f59d80ac860 con 0x7f59d80a9d90 2023-12-01T01:16:50.880 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.439+0000 7f59e37fe700 1 -- 172.21.15.77:0/154240733 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59e40cf7d0 con 0x7f59ec1022b0 2023-12-01T01:16:50.880 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.439+0000 7f59f1c42700 1 --2- 172.21.15.77:0/154240733 >> [v2:172.21.15.77:6818/2463414651,v1:172.21.15.77:6819/2463414651] conn(0x7f59d80a9d90 0x7f59d80ac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:50.880 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.439+0000 7f59f2443700 1 --2- 172.21.15.77:0/154240733 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f59d80a05b0 0x7f59d80a2a70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f59dc00ac50 tx=0x7f59dc00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:50.880 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.439+0000 7f59f1c42700 1 --2- 172.21.15.77:0/154240733 >> [v2:172.21.15.77:6818/2463414651,v1:172.21.15.77:6819/2463414651] conn(0x7f59d80a9d90 0x7f59d80ac1b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:50.880 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.440+0000 7f59e37fe700 1 -- 172.21.15.77:0/154240733 <== osd.6 v2:172.21.15.77:6818/2463414651 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f59d80ac860 con 0x7f59d80a9d90 2023-12-01T01:16:50.880 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.456+0000 7f59f46a7700 1 -- 172.21.15.77:0/154240733 --> [v2:172.21.15.77:6818/2463414651,v1:172.21.15.77:6819/2463414651] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f59ec110000 con 0x7f59d80a9d90 2023-12-01T01:16:50.880 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.456+0000 7f59e37fe700 1 -- 172.21.15.77:0/154240733 <== osd.6 v2:172.21.15.77:6818/2463414651 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f59e80097b0 con 0x7f59d80a9d90 2023-12-01T01:16:50.880 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.456+0000 7f59f46a7700 1 -- 172.21.15.77:0/154240733 >> [v2:172.21.15.77:6818/2463414651,v1:172.21.15.77:6819/2463414651] conn(0x7f59d80a9d90 msgr2=0x7f59d80ac1b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:50.880 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.456+0000 7f59f46a7700 1 --2- 172.21.15.77:0/154240733 >> [v2:172.21.15.77:6818/2463414651,v1:172.21.15.77:6819/2463414651] conn(0x7f59d80a9d90 0x7f59d80ac1b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:50.881 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.456+0000 7f59f46a7700 1 -- 172.21.15.77:0/154240733 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f59d80a05b0 msgr2=0x7f59d80a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:50.881 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.456+0000 7f59f46a7700 1 --2- 172.21.15.77:0/154240733 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f59d80a05b0 0x7f59d80a2a70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f59dc00ac50 tx=0x7f59dc00a380 comp rx=0 tx=0).stop 2023-12-01T01:16:50.881 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.456+0000 7f59f46a7700 1 -- 172.21.15.77:0/154240733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59ec1022b0 msgr2=0x7f59ec19a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:50.881 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.456+0000 7f59f46a7700 1 --2- 172.21.15.77:0/154240733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59ec1022b0 0x7f59ec19a860 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f59e4007870 tx=0x7f59e40078a0 comp rx=0 tx=0).stop 2023-12-01T01:16:50.881 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.456+0000 7f59f2c44700 1 -- 172.21.15.77:0/154240733 reap_dead start 2023-12-01T01:16:50.881 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.456+0000 7f59f46a7700 1 -- 172.21.15.77:0/154240733 shutdown_connections 2023-12-01T01:16:50.881 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.456+0000 7f59f46a7700 1 -- 172.21.15.77:0/154240733 >> 172.21.15.77:0/154240733 conn(0x7f59ec076d80 msgr2=0x7f59ec0fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:50.881 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.456+0000 7f59f46a7700 1 -- 172.21.15.77:0/154240733 shutdown_connections 2023-12-01T01:16:50.881 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:50.456+0000 7f59f46a7700 1 -- 172.21.15.77:0/154240733 wait complete. 2023-12-01T01:16:51.733 INFO:teuthology.orchestra.run.smithi077.stdout:193273528358 2023-12-01T01:16:51.733 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.6 2023-12-01T01:16:51.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:51 smithi077 ceph-mon[98867]: pgmap v265: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-12-01T01:16:52.001 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.958+0000 7fdf25c34700 1 -- 172.21.15.77:0/3534382178 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf20100660 msgr2=0x7fdf2010cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:52.001 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.958+0000 7fdf25c34700 1 --2- 172.21.15.77:0/3534382178 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf20100660 0x7fdf2010cb00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800b0a0 tx=0x7fdf1800b3b0 comp rx=0 tx=0).stop 2023-12-01T01:16:52.001 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.958+0000 7fdf25c34700 1 -- 172.21.15.77:0/3534382178 shutdown_connections 2023-12-01T01:16:52.001 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.958+0000 7fdf25c34700 1 --2- 172.21.15.77:0/3534382178 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf20100660 0x7fdf2010cb00 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:52.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.958+0000 7fdf25c34700 1 --2- 172.21.15.77:0/3534382178 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf20105d50 0x7fdf20106130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:52.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.958+0000 7fdf25c34700 1 --2- 172.21.15.77:0/3534382178 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf200ffca0 0x7fdf20100120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:52.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.958+0000 7fdf25c34700 1 -- 172.21.15.77:0/3534382178 >> 172.21.15.77:0/3534382178 conn(0x7fdf20076de0 msgr2=0x7fdf200771f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:52.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.958+0000 7fdf25c34700 1 -- 172.21.15.77:0/3534382178 shutdown_connections 2023-12-01T01:16:52.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.958+0000 7fdf25c34700 1 -- 172.21.15.77:0/3534382178 wait complete. 2023-12-01T01:16:52.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.959+0000 7fdf25c34700 1 Processor -- start 2023-12-01T01:16:52.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.959+0000 7fdf25c34700 1 -- start start 2023-12-01T01:16:52.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.960+0000 7fdf25c34700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf200ffca0 0x7fdf20195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:52.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.960+0000 7fdf25c34700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf20100660 0x7fdf201963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:52.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.960+0000 7fdf25c34700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf20105d50 0x7fdf2019a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:52.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.960+0000 7fdf25c34700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2019ae40 con 0x7fdf200ffca0 2023-12-01T01:16:52.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.960+0000 7fdf25c34700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2019afb0 con 0x7fdf20105d50 2023-12-01T01:16:52.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.960+0000 7fdf25c34700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2019b170 con 0x7fdf20100660 2023-12-01T01:16:52.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.960+0000 7fdf1f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf200ffca0 0x7fdf20195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:52.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.960+0000 7fdf1f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf200ffca0 0x7fdf20195e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57788/0 (socket says 172.21.15.77:57788) 2023-12-01T01:16:52.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.960+0000 7fdf1f7fe700 1 -- 172.21.15.77:0/597833948 learned_addr learned my addr 172.21.15.77:0/597833948 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:52.004 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.960+0000 7fdf1ffff700 1 --2- 172.21.15.77:0/597833948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf20105d50 0x7fdf2019a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:52.004 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.960+0000 7fdf1effd700 1 --2- 172.21.15.77:0/597833948 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf20100660 0x7fdf201963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:52.004 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.960+0000 7fdf1f7fe700 1 -- 172.21.15.77:0/597833948 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf20100660 msgr2=0x7fdf201963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:52.004 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.960+0000 7fdf1f7fe700 1 --2- 172.21.15.77:0/597833948 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf20100660 0x7fdf201963c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:52.004 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.960+0000 7fdf1f7fe700 1 -- 172.21.15.77:0/597833948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf20105d50 msgr2=0x7fdf2019a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:52.004 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.960+0000 7fdf1f7fe700 1 --2- 172.21.15.77:0/597833948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf20105d50 0x7fdf2019a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:52.004 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.960+0000 7fdf1f7fe700 1 -- 172.21.15.77:0/597833948 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf18009d00 con 0x7fdf200ffca0 2023-12-01T01:16:52.004 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.960+0000 7fdf1f7fe700 1 --2- 172.21.15.77:0/597833948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf200ffca0 0x7fdf20195e80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdf1000e9a0 tx=0x7fdf1000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:52.005 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.960+0000 7fdf1cff9700 1 -- 172.21.15.77:0/597833948 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf10009790 con 0x7fdf200ffca0 2023-12-01T01:16:52.005 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.960+0000 7fdf25c34700 1 -- 172.21.15.77:0/597833948 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf2019b450 con 0x7fdf200ffca0 2023-12-01T01:16:52.005 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.960+0000 7fdf1cff9700 1 -- 172.21.15.77:0/597833948 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf1000cbd0 con 0x7fdf200ffca0 2023-12-01T01:16:52.005 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.960+0000 7fdf1cff9700 1 -- 172.21.15.77:0/597833948 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf10018680 con 0x7fdf200ffca0 2023-12-01T01:16:52.005 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.960+0000 7fdf25c34700 1 -- 172.21.15.77:0/597833948 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf201a2cb0 con 0x7fdf200ffca0 2023-12-01T01:16:52.005 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.961+0000 7fdf25c34700 1 -- 172.21.15.77:0/597833948 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdf2010fe70 con 0x7fdf200ffca0 2023-12-01T01:16:52.005 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.962+0000 7fdf1cff9700 1 -- 172.21.15.77:0/597833948 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fdf1000cd40 con 0x7fdf200ffca0 2023-12-01T01:16:52.005 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.963+0000 7fdf1cff9700 1 --2- 172.21.15.77:0/597833948 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fdf080a05b0 0x7fdf080a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:52.005 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.963+0000 7fdf1effd700 1 --2- 172.21.15.77:0/597833948 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fdf080a05b0 0x7fdf080a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:52.006 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.963+0000 7fdf1cff9700 1 -- 172.21.15.77:0/597833948 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fdf10014070 con 0x7fdf200ffca0 2023-12-01T01:16:52.006 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.963+0000 7fdf1cff9700 1 --2- 172.21.15.77:0/597833948 >> [v2:172.21.15.77:6810/1454440477,v1:172.21.15.77:6811/1454440477] conn(0x7fdf080a9d90 0x7fdf080ac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:52.006 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.963+0000 7fdf1cff9700 1 -- 172.21.15.77:0/597833948 --> [v2:172.21.15.77:6810/1454440477,v1:172.21.15.77:6811/1454440477] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdf080ac860 con 0x7fdf080a9d90 2023-12-01T01:16:52.006 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.963+0000 7fdf1cff9700 1 -- 172.21.15.77:0/597833948 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf100cc740 con 0x7fdf200ffca0 2023-12-01T01:16:52.006 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.963+0000 7fdf1ffff700 1 --2- 172.21.15.77:0/597833948 >> [v2:172.21.15.77:6810/1454440477,v1:172.21.15.77:6811/1454440477] conn(0x7fdf080a9d90 0x7fdf080ac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:52.006 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.963+0000 7fdf1effd700 1 --2- 172.21.15.77:0/597833948 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fdf080a05b0 0x7fdf080a2a70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdf14005950 tx=0x7fdf140058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:52.006 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.964+0000 7fdf1ffff700 1 --2- 172.21.15.77:0/597833948 >> [v2:172.21.15.77:6810/1454440477,v1:172.21.15.77:6811/1454440477] conn(0x7fdf080a9d90 0x7fdf080ac1b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:52.006 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.964+0000 7fdf1cff9700 1 -- 172.21.15.77:0/597833948 <== osd.3 v2:172.21.15.77:6810/1454440477 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fdf080ac860 con 0x7fdf080a9d90 2023-12-01T01:16:52.007 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.980+0000 7fdf25c34700 1 -- 172.21.15.77:0/597833948 --> [v2:172.21.15.77:6810/1454440477,v1:172.21.15.77:6811/1454440477] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdf2004ea90 con 0x7fdf080a9d90 2023-12-01T01:16:52.007 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.980+0000 7fdf1cff9700 1 -- 172.21.15.77:0/597833948 <== osd.3 v2:172.21.15.77:6810/1454440477 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fdf2004ea90 con 0x7fdf080a9d90 2023-12-01T01:16:52.007 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.980+0000 7fdf25c34700 1 -- 172.21.15.77:0/597833948 >> [v2:172.21.15.77:6810/1454440477,v1:172.21.15.77:6811/1454440477] conn(0x7fdf080a9d90 msgr2=0x7fdf080ac1b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:52.007 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.980+0000 7fdf25c34700 1 --2- 172.21.15.77:0/597833948 >> [v2:172.21.15.77:6810/1454440477,v1:172.21.15.77:6811/1454440477] conn(0x7fdf080a9d90 0x7fdf080ac1b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:52.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.980+0000 7fdf25c34700 1 -- 172.21.15.77:0/597833948 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fdf080a05b0 msgr2=0x7fdf080a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:52.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.980+0000 7fdf25c34700 1 --2- 172.21.15.77:0/597833948 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fdf080a05b0 0x7fdf080a2a70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdf14005950 tx=0x7fdf140058e0 comp rx=0 tx=0).stop 2023-12-01T01:16:52.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.980+0000 7fdf25c34700 1 -- 172.21.15.77:0/597833948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf200ffca0 msgr2=0x7fdf20195e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:52.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.980+0000 7fdf25c34700 1 --2- 172.21.15.77:0/597833948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf200ffca0 0x7fdf20195e80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdf1000e9a0 tx=0x7fdf1000ed60 comp rx=0 tx=0).stop 2023-12-01T01:16:52.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.980+0000 7fdf1ffff700 1 -- 172.21.15.77:0/597833948 reap_dead start 2023-12-01T01:16:52.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.980+0000 7fdf25c34700 1 -- 172.21.15.77:0/597833948 shutdown_connections 2023-12-01T01:16:52.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.980+0000 7fdf25c34700 1 -- 172.21.15.77:0/597833948 >> 172.21.15.77:0/597833948 conn(0x7fdf20076de0 msgr2=0x7fdf200fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:52.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.980+0000 7fdf25c34700 1 -- 172.21.15.77:0/597833948 shutdown_connections 2023-12-01T01:16:52.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:51.980+0000 7fdf25c34700 1 -- 172.21.15.77:0/597833948 wait complete. 2023-12-01T01:16:52.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:51 smithi203 ceph-mon[103581]: pgmap v265: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-12-01T01:16:52.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:51 smithi137 ceph-mon[103312]: pgmap v265: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-12-01T01:16:52.248 INFO:teuthology.orchestra.run.smithi077.stdout:137438953519 2023-12-01T01:16:52.248 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.4 2023-12-01T01:16:52.626 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.623+0000 7fc9a768a700 1 -- 172.21.15.77:0/725583573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9a00694a0 msgr2=0x7fc9a0069880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:52.626 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.623+0000 7fc9a768a700 1 --2- 172.21.15.77:0/725583573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9a00694a0 0x7fc9a0069880 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc99c0099c0 tx=0x7fc99c009cd0 comp rx=0 tx=0).stop 2023-12-01T01:16:52.626 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.624+0000 7fc9a768a700 1 -- 172.21.15.77:0/725583573 shutdown_connections 2023-12-01T01:16:52.626 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.624+0000 7fc9a768a700 1 --2- 172.21.15.77:0/725583573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9a0103160 0x7fc9a010dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:52.626 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.624+0000 7fc9a768a700 1 --2- 172.21.15.77:0/725583573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9a00694a0 0x7fc9a0069880 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:52.626 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.624+0000 7fc9a768a700 1 --2- 172.21.15.77:0/725583573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9a0069dc0 0x7fc9a0102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:52.626 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.624+0000 7fc9a768a700 1 -- 172.21.15.77:0/725583573 >> 172.21.15.77:0/725583573 conn(0x7fc9a0076d80 msgr2=0x7fc9a0077190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:52.627 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.625+0000 7fc9a768a700 1 -- 172.21.15.77:0/725583573 shutdown_connections 2023-12-01T01:16:52.628 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.625+0000 7fc9a768a700 1 -- 172.21.15.77:0/725583573 wait complete. 2023-12-01T01:16:52.628 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.625+0000 7fc9a768a700 1 Processor -- start 2023-12-01T01:16:52.628 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.625+0000 7fc9a768a700 1 -- start start 2023-12-01T01:16:52.628 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.625+0000 7fc9a768a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9a00694a0 0x7fc9a0072200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:52.628 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.626+0000 7fc9a768a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9a0069dc0 0x7fc9a0072740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:52.628 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.626+0000 7fc9a768a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9a0103160 0x7fc9a006b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:52.628 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.626+0000 7fc9a768a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a006b840 con 0x7fc9a0103160 2023-12-01T01:16:52.628 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.626+0000 7fc9a768a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a006b9b0 con 0x7fc9a0069dc0 2023-12-01T01:16:52.628 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.626+0000 7fc9a768a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a006bb70 con 0x7fc9a00694a0 2023-12-01T01:16:52.629 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.626+0000 7fc9a5c27700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9a0103160 0x7fc9a006b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:52.629 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.626+0000 7fc9a5c27700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9a0103160 0x7fc9a006b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57808/0 (socket says 172.21.15.77:57808) 2023-12-01T01:16:52.629 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.626+0000 7fc9a5c27700 1 -- 172.21.15.77:0/4032145154 learned_addr learned my addr 172.21.15.77:0/4032145154 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:52.629 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.626+0000 7fc9a4c25700 1 --2- 172.21.15.77:0/4032145154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9a0069dc0 0x7fc9a0072740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:52.629 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.626+0000 7fc9a5426700 1 --2- 172.21.15.77:0/4032145154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9a00694a0 0x7fc9a0072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:52.629 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.626+0000 7fc9a5c27700 1 -- 172.21.15.77:0/4032145154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9a00694a0 msgr2=0x7fc9a0072200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:52.629 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.626+0000 7fc9a5c27700 1 --2- 172.21.15.77:0/4032145154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc9a00694a0 0x7fc9a0072200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:52.629 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.626+0000 7fc9a5c27700 1 -- 172.21.15.77:0/4032145154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9a0069dc0 msgr2=0x7fc9a0072740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:52.629 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.626+0000 7fc9a5c27700 1 --2- 172.21.15.77:0/4032145154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9a0069dc0 0x7fc9a0072740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:52.629 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.626+0000 7fc9a5c27700 1 -- 172.21.15.77:0/4032145154 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc99c0095f0 con 0x7fc9a0103160 2023-12-01T01:16:52.629 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.626+0000 7fc9a5c27700 1 --2- 172.21.15.77:0/4032145154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9a0103160 0x7fc9a006b1b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc99800ef30 tx=0x7fc99800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:52.629 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.627+0000 7fc9967fc700 1 -- 172.21.15.77:0/4032145154 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc998004bb0 con 0x7fc9a0103160 2023-12-01T01:16:52.630 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.627+0000 7fc9a768a700 1 -- 172.21.15.77:0/4032145154 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9a006be50 con 0x7fc9a0103160 2023-12-01T01:16:52.630 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.627+0000 7fc9967fc700 1 -- 172.21.15.77:0/4032145154 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc99800f040 con 0x7fc9a0103160 2023-12-01T01:16:52.630 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.627+0000 7fc9967fc700 1 -- 172.21.15.77:0/4032145154 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99800cae0 con 0x7fc9a0103160 2023-12-01T01:16:52.630 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.627+0000 7fc9a768a700 1 -- 172.21.15.77:0/4032145154 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9a006ff00 con 0x7fc9a0103160 2023-12-01T01:16:52.630 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.628+0000 7fc9a768a700 1 -- 172.21.15.77:0/4032145154 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc9a0111120 con 0x7fc9a0103160 2023-12-01T01:16:52.630 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.630+0000 7fc9967fc700 1 -- 172.21.15.77:0/4032145154 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fc99800cc40 con 0x7fc9a0103160 2023-12-01T01:16:52.631 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.630+0000 7fc9967fc700 1 --2- 172.21.15.77:0/4032145154 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc98c0a0630 0x7fc98c0a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:52.631 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.630+0000 7fc9967fc700 1 -- 172.21.15.77:0/4032145154 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc9980d3500 con 0x7fc9a0103160 2023-12-01T01:16:52.632 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.631+0000 7fc9967fc700 1 --2- 172.21.15.77:0/4032145154 >> [v2:172.21.15.203:6824/2161588962,v1:172.21.15.203:6825/2161588962] conn(0x7fc98c0a9e10 0x7fc98c0ac230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:52.632 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.631+0000 7fc9967fc700 1 -- 172.21.15.77:0/4032145154 --> [v2:172.21.15.203:6824/2161588962,v1:172.21.15.203:6825/2161588962] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc98c0ac8e0 con 0x7fc98c0a9e10 2023-12-01T01:16:52.632 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.631+0000 7fc9967fc700 1 -- 172.21.15.77:0/4032145154 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9980d39a0 con 0x7fc9a0103160 2023-12-01T01:16:52.632 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.631+0000 7fc9a5426700 1 --2- 172.21.15.77:0/4032145154 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc98c0a0630 0x7fc98c0a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:52.632 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.631+0000 7fc9a4c25700 1 --2- 172.21.15.77:0/4032145154 >> [v2:172.21.15.203:6824/2161588962,v1:172.21.15.203:6825/2161588962] conn(0x7fc98c0a9e10 0x7fc98c0ac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:52.632 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.631+0000 7fc9a5426700 1 --2- 172.21.15.77:0/4032145154 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc98c0a0630 0x7fc98c0a2af0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc99c009990 tx=0x7fc99c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:52.633 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.631+0000 7fc9a4c25700 1 --2- 172.21.15.77:0/4032145154 >> [v2:172.21.15.203:6824/2161588962,v1:172.21.15.203:6825/2161588962] conn(0x7fc98c0a9e10 0x7fc98c0ac230 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:52.633 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.632+0000 7fc9967fc700 1 -- 172.21.15.77:0/4032145154 <== osd.11 v2:172.21.15.203:6824/2161588962 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fc98c0ac8e0 con 0x7fc98c0a9e10 2023-12-01T01:16:52.656 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.654+0000 7fc9a768a700 1 -- 172.21.15.77:0/4032145154 --> [v2:172.21.15.203:6824/2161588962,v1:172.21.15.203:6825/2161588962] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc9a004ea90 con 0x7fc98c0a9e10 2023-12-01T01:16:52.656 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.655+0000 7fc9967fc700 1 -- 172.21.15.77:0/4032145154 <== osd.11 v2:172.21.15.203:6824/2161588962 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc9a004ea90 con 0x7fc98c0a9e10 2023-12-01T01:16:52.657 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.655+0000 7fc9a768a700 1 -- 172.21.15.77:0/4032145154 >> [v2:172.21.15.203:6824/2161588962,v1:172.21.15.203:6825/2161588962] conn(0x7fc98c0a9e10 msgr2=0x7fc98c0ac230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:52.657 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.655+0000 7fc9a768a700 1 --2- 172.21.15.77:0/4032145154 >> [v2:172.21.15.203:6824/2161588962,v1:172.21.15.203:6825/2161588962] conn(0x7fc98c0a9e10 0x7fc98c0ac230 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:52.657 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.655+0000 7fc9a768a700 1 -- 172.21.15.77:0/4032145154 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc98c0a0630 msgr2=0x7fc98c0a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:52.657 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.655+0000 7fc9a768a700 1 --2- 172.21.15.77:0/4032145154 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc98c0a0630 0x7fc98c0a2af0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc99c009990 tx=0x7fc99c01a040 comp rx=0 tx=0).stop 2023-12-01T01:16:52.657 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.655+0000 7fc9a768a700 1 -- 172.21.15.77:0/4032145154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9a0103160 msgr2=0x7fc9a006b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:52.657 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.655+0000 7fc9a768a700 1 --2- 172.21.15.77:0/4032145154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9a0103160 0x7fc9a006b1b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc99800ef30 tx=0x7fc99800c3a0 comp rx=0 tx=0).stop 2023-12-01T01:16:52.657 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.656+0000 7fc9a5c27700 1 -- 172.21.15.77:0/4032145154 reap_dead start 2023-12-01T01:16:52.658 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.656+0000 7fc9a768a700 1 -- 172.21.15.77:0/4032145154 shutdown_connections 2023-12-01T01:16:52.658 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.656+0000 7fc9a768a700 1 -- 172.21.15.77:0/4032145154 >> 172.21.15.77:0/4032145154 conn(0x7fc9a0076d80 msgr2=0x7fc9a00fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:52.658 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.656+0000 7fc9a768a700 1 -- 172.21.15.77:0/4032145154 shutdown_connections 2023-12-01T01:16:52.658 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:52.656+0000 7fc9a768a700 1 -- 172.21.15.77:0/4032145154 wait complete. 2023-12-01T01:16:53.091 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.090+0000 7f0ec4ad9700 1 -- 172.21.15.77:0/1098728702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ec0105d80 msgr2=0x7f0ec0106160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:53.092 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.090+0000 7f0ec4ad9700 1 --2- 172.21.15.77:0/1098728702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ec0105d80 0x7f0ec0106160 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0009990 tx=0x7f0eb0009ca0 comp rx=0 tx=0).stop 2023-12-01T01:16:53.092 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.090+0000 7f0ec4ad9700 1 -- 172.21.15.77:0/1098728702 shutdown_connections 2023-12-01T01:16:53.092 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.090+0000 7f0ec4ad9700 1 --2- 172.21.15.77:0/1098728702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ec0100690 0x7f0ec010cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:53.092 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.090+0000 7f0ec4ad9700 1 --2- 172.21.15.77:0/1098728702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ec0105d80 0x7f0ec0106160 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:53.092 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.090+0000 7f0ec4ad9700 1 --2- 172.21.15.77:0/1098728702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ec00ffcd0 0x7f0ec0100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:53.092 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.090+0000 7f0ec4ad9700 1 -- 172.21.15.77:0/1098728702 >> 172.21.15.77:0/1098728702 conn(0x7f0ec0076d90 msgr2=0x7f0ec00771a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:53.092 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.090+0000 7f0ec4ad9700 1 -- 172.21.15.77:0/1098728702 shutdown_connections 2023-12-01T01:16:53.092 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.090+0000 7f0ec4ad9700 1 -- 172.21.15.77:0/1098728702 wait complete. 2023-12-01T01:16:53.094 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.091+0000 7f0ec4ad9700 1 Processor -- start 2023-12-01T01:16:53.094 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.091+0000 7f0ec4ad9700 1 -- start start 2023-12-01T01:16:53.094 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.091+0000 7f0ec4ad9700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ec00ffcd0 0x7f0ec0196e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:53.094 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.091+0000 7f0ec4ad9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ec0100690 0x7f0ec0197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:53.094 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.091+0000 7f0ec4ad9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ec0105d80 0x7f0ec019b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:53.094 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.091+0000 7f0ec4ad9700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec019bdf0 con 0x7f0ec0100690 2023-12-01T01:16:53.094 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.091+0000 7f0ec4ad9700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec019bf60 con 0x7f0ec00ffcd0 2023-12-01T01:16:53.094 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.091+0000 7f0ec4ad9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec019c120 con 0x7f0ec0105d80 2023-12-01T01:16:53.094 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.091+0000 7f0ebdd9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ec0100690 0x7f0ec0197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:53.094 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.091+0000 7f0ebdd9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ec0100690 0x7f0ec0197370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57838/0 (socket says 172.21.15.77:57838) 2023-12-01T01:16:53.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.091+0000 7f0ebdd9b700 1 -- 172.21.15.77:0/170065134 learned_addr learned my addr 172.21.15.77:0/170065134 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:53.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.091+0000 7f0ebed9d700 1 --2- 172.21.15.77:0/170065134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ec0105d80 0x7f0ec019b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:53.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.091+0000 7f0ebe59c700 1 --2- 172.21.15.77:0/170065134 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ec00ffcd0 0x7f0ec0196e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:53.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.091+0000 7f0ebdd9b700 1 -- 172.21.15.77:0/170065134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ec0105d80 msgr2=0x7f0ec019b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:53.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.091+0000 7f0ebdd9b700 1 --2- 172.21.15.77:0/170065134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ec0105d80 0x7f0ec019b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:53.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.091+0000 7f0ebdd9b700 1 -- 172.21.15.77:0/170065134 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ec00ffcd0 msgr2=0x7f0ec0196e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:53.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.091+0000 7f0ebdd9b700 1 --2- 172.21.15.77:0/170065134 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ec00ffcd0 0x7f0ec0196e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:53.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.091+0000 7f0ebdd9b700 1 -- 172.21.15.77:0/170065134 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb00095f0 con 0x7f0ec0100690 2023-12-01T01:16:53.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.092+0000 7f0ebdd9b700 1 --2- 172.21.15.77:0/170065134 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ec0100690 0x7f0ec0197370 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0eb400d980 tx=0x7f0eb400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:53.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.092+0000 7f0eaf7fe700 1 -- 172.21.15.77:0/170065134 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb400bb60 con 0x7f0ec0100690 2023-12-01T01:16:53.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.092+0000 7f0ec4ad9700 1 -- 172.21.15.77:0/170065134 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ec019c400 con 0x7f0ec0100690 2023-12-01T01:16:53.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.092+0000 7f0ec4ad9700 1 -- 172.21.15.77:0/170065134 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ec01a3c60 con 0x7f0ec0100690 2023-12-01T01:16:53.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.092+0000 7f0eaf7fe700 1 -- 172.21.15.77:0/170065134 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0eb4007640 con 0x7f0ec0100690 2023-12-01T01:16:53.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.092+0000 7f0eaf7fe700 1 -- 172.21.15.77:0/170065134 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb4004da0 con 0x7f0ec0100690 2023-12-01T01:16:53.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.093+0000 7f0ec4ad9700 1 -- 172.21.15.77:0/170065134 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0ea0000ff0 con 0x7f0ec0100690 2023-12-01T01:16:53.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.094+0000 7f0eaf7fe700 1 -- 172.21.15.77:0/170065134 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f0eb4004020 con 0x7f0ec0100690 2023-12-01T01:16:53.097 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.094+0000 7f0eaf7fe700 1 --2- 172.21.15.77:0/170065134 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f0ea80a0630 0x7f0ea80a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:53.097 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.094+0000 7f0ebe59c700 1 --2- 172.21.15.77:0/170065134 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f0ea80a0630 0x7f0ea80a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:53.097 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.094+0000 7f0eaf7fe700 1 -- 172.21.15.77:0/170065134 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0eb40cb840 con 0x7f0ec0100690 2023-12-01T01:16:53.097 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.095+0000 7f0eaf7fe700 1 --2- 172.21.15.77:0/170065134 >> [v2:172.21.15.137:6824/374920146,v1:172.21.15.137:6825/374920146] conn(0x7f0ea80a9e00 0x7f0ea80ac220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:53.097 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.095+0000 7f0eaf7fe700 1 -- 172.21.15.77:0/170065134 --> [v2:172.21.15.137:6824/374920146,v1:172.21.15.137:6825/374920146] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0ea80ac8d0 con 0x7f0ea80a9e00 2023-12-01T01:16:53.097 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.095+0000 7f0eaf7fe700 1 -- 172.21.15.77:0/170065134 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0eb40d02c0 con 0x7f0ec0100690 2023-12-01T01:16:53.097 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.095+0000 7f0ebe59c700 1 --2- 172.21.15.77:0/170065134 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f0ea80a0630 0x7f0ea80a2af0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0006010 tx=0x7f0eb000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:53.097 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.095+0000 7f0ebed9d700 1 --2- 172.21.15.77:0/170065134 >> [v2:172.21.15.137:6824/374920146,v1:172.21.15.137:6825/374920146] conn(0x7f0ea80a9e00 0x7f0ea80ac220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:53.097 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.095+0000 7f0ebed9d700 1 --2- 172.21.15.77:0/170065134 >> [v2:172.21.15.137:6824/374920146,v1:172.21.15.137:6825/374920146] conn(0x7f0ea80a9e00 0x7f0ea80ac220 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:53.098 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.096+0000 7f0eaf7fe700 1 -- 172.21.15.77:0/170065134 <== osd.10 v2:172.21.15.137:6824/374920146 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f0ea80ac8d0 con 0x7f0ea80a9e00 2023-12-01T01:16:53.115 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.113+0000 7f0ec4ad9700 1 -- 172.21.15.77:0/170065134 --> [v2:172.21.15.137:6824/374920146,v1:172.21.15.137:6825/374920146] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0ea0002d70 con 0x7f0ea80a9e00 2023-12-01T01:16:53.115 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.114+0000 7f0eaf7fe700 1 -- 172.21.15.77:0/170065134 <== osd.10 v2:172.21.15.137:6824/374920146 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0ea0002d70 con 0x7f0ea80a9e00 2023-12-01T01:16:53.116 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.115+0000 7f0ec4ad9700 1 -- 172.21.15.77:0/170065134 >> [v2:172.21.15.137:6824/374920146,v1:172.21.15.137:6825/374920146] conn(0x7f0ea80a9e00 msgr2=0x7f0ea80ac220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:53.116 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.115+0000 7f0ec4ad9700 1 --2- 172.21.15.77:0/170065134 >> [v2:172.21.15.137:6824/374920146,v1:172.21.15.137:6825/374920146] conn(0x7f0ea80a9e00 0x7f0ea80ac220 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:53.116 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.115+0000 7f0ec4ad9700 1 -- 172.21.15.77:0/170065134 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f0ea80a0630 msgr2=0x7f0ea80a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:53.116 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.115+0000 7f0ec4ad9700 1 --2- 172.21.15.77:0/170065134 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f0ea80a0630 0x7f0ea80a2af0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0006010 tx=0x7f0eb000b260 comp rx=0 tx=0).stop 2023-12-01T01:16:53.116 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.115+0000 7f0ec4ad9700 1 -- 172.21.15.77:0/170065134 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ec0100690 msgr2=0x7f0ec0197370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:53.116 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.115+0000 7f0ec4ad9700 1 --2- 172.21.15.77:0/170065134 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ec0100690 0x7f0ec0197370 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0eb400d980 tx=0x7f0eb400dd40 comp rx=0 tx=0).stop 2023-12-01T01:16:53.117 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.115+0000 7f0ebed9d700 1 -- 172.21.15.77:0/170065134 reap_dead start 2023-12-01T01:16:53.117 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.115+0000 7f0ec4ad9700 1 -- 172.21.15.77:0/170065134 shutdown_connections 2023-12-01T01:16:53.117 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.115+0000 7f0ec4ad9700 1 -- 172.21.15.77:0/170065134 >> 172.21.15.77:0/170065134 conn(0x7f0ec0076d90 msgr2=0x7f0ec00fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:53.117 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.115+0000 7f0ec4ad9700 1 -- 172.21.15.77:0/170065134 shutdown_connections 2023-12-01T01:16:53.117 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.115+0000 7f0ec4ad9700 1 -- 172.21.15.77:0/170065134 wait complete. 2023-12-01T01:16:53.217 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.048+0000 7fbb85915700 1 -- 172.21.15.77:0/17891740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb801060e0 msgr2=0x7fbb8010dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:53.217 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.048+0000 7fbb85915700 1 --2- 172.21.15.77:0/17891740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb801060e0 0x7fbb8010dcb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbb7800b0a0 tx=0x7fbb7800b3b0 comp rx=0 tx=0).stop 2023-12-01T01:16:53.218 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.048+0000 7fbb85915700 1 -- 172.21.15.77:0/17891740 shutdown_connections 2023-12-01T01:16:53.218 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.048+0000 7fbb85915700 1 --2- 172.21.15.77:0/17891740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb801060e0 0x7fbb8010dcb0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:53.218 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.048+0000 7fbb85915700 1 --2- 172.21.15.77:0/17891740 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb800ff580 0x7fbb800ff960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:53.218 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.048+0000 7fbb85915700 1 --2- 172.21.15.77:0/17891740 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb800fff30 0x7fbb80105a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:53.218 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.048+0000 7fbb85915700 1 -- 172.21.15.77:0/17891740 >> 172.21.15.77:0/17891740 conn(0x7fbb80076200 msgr2=0x7fbb800fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:53.218 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.048+0000 7fbb85915700 1 -- 172.21.15.77:0/17891740 shutdown_connections 2023-12-01T01:16:53.218 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.048+0000 7fbb85915700 1 -- 172.21.15.77:0/17891740 wait complete. 2023-12-01T01:16:53.218 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.049+0000 7fbb85915700 1 Processor -- start 2023-12-01T01:16:53.218 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.049+0000 7fbb85915700 1 -- start start 2023-12-01T01:16:53.218 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.049+0000 7fbb85915700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb800ff580 0x7fbb80196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:53.218 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.049+0000 7fbb85915700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb800fff30 0x7fbb80197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:53.218 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.049+0000 7fbb85915700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb801060e0 0x7fbb8019b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:53.218 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.049+0000 7fbb85915700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8019be80 con 0x7fbb800fff30 2023-12-01T01:16:53.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.049+0000 7fbb85915700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8019bff0 con 0x7fbb800ff580 2023-12-01T01:16:53.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.049+0000 7fbb85915700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8019c1b0 con 0x7fbb801060e0 2023-12-01T01:16:53.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.050+0000 7fbb7e7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb800fff30 0x7fbb80197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:53.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.050+0000 7fbb7e7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb800fff30 0x7fbb80197400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57822/0 (socket says 172.21.15.77:57822) 2023-12-01T01:16:53.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.050+0000 7fbb7e7fc700 1 -- 172.21.15.77:0/931359013 learned_addr learned my addr 172.21.15.77:0/931359013 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:53.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.050+0000 7fbb7f7fe700 1 --2- 172.21.15.77:0/931359013 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb801060e0 0x7fbb8019b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:53.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.050+0000 7fbb7effd700 1 --2- 172.21.15.77:0/931359013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb800ff580 0x7fbb80196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:53.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.050+0000 7fbb7e7fc700 1 -- 172.21.15.77:0/931359013 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb801060e0 msgr2=0x7fbb8019b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:53.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.050+0000 7fbb7e7fc700 1 --2- 172.21.15.77:0/931359013 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb801060e0 0x7fbb8019b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:53.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.050+0000 7fbb7e7fc700 1 -- 172.21.15.77:0/931359013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb800ff580 msgr2=0x7fbb80196ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:53.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.050+0000 7fbb7e7fc700 1 --2- 172.21.15.77:0/931359013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb800ff580 0x7fbb80196ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:53.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.050+0000 7fbb7e7fc700 1 -- 172.21.15.77:0/931359013 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb78009d00 con 0x7fbb800fff30 2023-12-01T01:16:53.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.050+0000 7fbb7e7fc700 1 --2- 172.21.15.77:0/931359013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb800fff30 0x7fbb80197400 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbb7400b550 tx=0x7fbb7400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:53.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.050+0000 7fbb84913700 1 -- 172.21.15.77:0/931359013 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb74011830 con 0x7fbb800fff30 2023-12-01T01:16:53.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.050+0000 7fbb85915700 1 -- 172.21.15.77:0/931359013 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb8019c490 con 0x7fbb800fff30 2023-12-01T01:16:53.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.050+0000 7fbb84913700 1 -- 172.21.15.77:0/931359013 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb74011e50 con 0x7fbb800fff30 2023-12-01T01:16:53.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.050+0000 7fbb84913700 1 -- 172.21.15.77:0/931359013 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb74010de0 con 0x7fbb800fff30 2023-12-01T01:16:53.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.050+0000 7fbb85915700 1 -- 172.21.15.77:0/931359013 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb801a3cf0 con 0x7fbb800fff30 2023-12-01T01:16:53.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.051+0000 7fbb85915700 1 -- 172.21.15.77:0/931359013 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbb8004ea90 con 0x7fbb800fff30 2023-12-01T01:16:53.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.053+0000 7fbb84913700 1 -- 172.21.15.77:0/931359013 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fbb74010480 con 0x7fbb800fff30 2023-12-01T01:16:53.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.053+0000 7fbb84913700 1 --2- 172.21.15.77:0/931359013 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fbb680a05b0 0x7fbb680a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:53.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.054+0000 7fbb7effd700 1 --2- 172.21.15.77:0/931359013 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fbb680a05b0 0x7fbb680a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:53.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.054+0000 7fbb84913700 1 -- 172.21.15.77:0/931359013 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fbb740cb810 con 0x7fbb800fff30 2023-12-01T01:16:53.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.054+0000 7fbb84913700 1 --2- 172.21.15.77:0/931359013 >> [v2:172.21.15.203:6808/2648569482,v1:172.21.15.203:6809/2648569482] conn(0x7fbb680a9d90 0x7fbb680ac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:53.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.054+0000 7fbb84913700 1 -- 172.21.15.77:0/931359013 --> [v2:172.21.15.203:6808/2648569482,v1:172.21.15.203:6809/2648569482] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbb680ac860 con 0x7fbb680a9d90 2023-12-01T01:16:53.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.054+0000 7fbb84913700 1 -- 172.21.15.77:0/931359013 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb740cbcb0 con 0x7fbb800fff30 2023-12-01T01:16:53.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.054+0000 7fbb7effd700 1 --2- 172.21.15.77:0/931359013 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fbb680a05b0 0x7fbb680a2a70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbb70009c60 tx=0x7fbb70009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:53.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.054+0000 7fbb7f7fe700 1 --2- 172.21.15.77:0/931359013 >> [v2:172.21.15.203:6808/2648569482,v1:172.21.15.203:6809/2648569482] conn(0x7fbb680a9d90 0x7fbb680ac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:53.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.055+0000 7fbb7f7fe700 1 --2- 172.21.15.77:0/931359013 >> [v2:172.21.15.203:6808/2648569482,v1:172.21.15.203:6809/2648569482] conn(0x7fbb680a9d90 0x7fbb680ac1b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:53.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.056+0000 7fbb84913700 1 -- 172.21.15.77:0/931359013 <== osd.5 v2:172.21.15.203:6808/2648569482 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fbb680ac860 con 0x7fbb680a9d90 2023-12-01T01:16:53.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.072+0000 7fbb85915700 1 -- 172.21.15.77:0/931359013 --> [v2:172.21.15.203:6808/2648569482,v1:172.21.15.203:6809/2648569482] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbb801110c0 con 0x7fbb680a9d90 2023-12-01T01:16:53.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.073+0000 7fbb84913700 1 -- 172.21.15.77:0/931359013 <== osd.5 v2:172.21.15.203:6808/2648569482 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fbb7800bd90 con 0x7fbb680a9d90 2023-12-01T01:16:53.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.073+0000 7fbb85915700 1 -- 172.21.15.77:0/931359013 >> [v2:172.21.15.203:6808/2648569482,v1:172.21.15.203:6809/2648569482] conn(0x7fbb680a9d90 msgr2=0x7fbb680ac1b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:53.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.073+0000 7fbb85915700 1 --2- 172.21.15.77:0/931359013 >> [v2:172.21.15.203:6808/2648569482,v1:172.21.15.203:6809/2648569482] conn(0x7fbb680a9d90 0x7fbb680ac1b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:53.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.074+0000 7fbb85915700 1 -- 172.21.15.77:0/931359013 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fbb680a05b0 msgr2=0x7fbb680a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:53.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.074+0000 7fbb85915700 1 --2- 172.21.15.77:0/931359013 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fbb680a05b0 0x7fbb680a2a70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbb70009c60 tx=0x7fbb70009260 comp rx=0 tx=0).stop 2023-12-01T01:16:53.222 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.074+0000 7fbb85915700 1 -- 172.21.15.77:0/931359013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb800fff30 msgr2=0x7fbb80197400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:53.222 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.074+0000 7fbb85915700 1 --2- 172.21.15.77:0/931359013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbb800fff30 0x7fbb80197400 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbb7400b550 tx=0x7fbb7400b860 comp rx=0 tx=0).stop 2023-12-01T01:16:53.222 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.074+0000 7fbb7f7fe700 1 -- 172.21.15.77:0/931359013 reap_dead start 2023-12-01T01:16:53.222 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.074+0000 7fbb85915700 1 -- 172.21.15.77:0/931359013 shutdown_connections 2023-12-01T01:16:53.222 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.074+0000 7fbb85915700 1 -- 172.21.15.77:0/931359013 >> 172.21.15.77:0/931359013 conn(0x7fbb80076200 msgr2=0x7fbb80104020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:53.222 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.074+0000 7fbb85915700 1 -- 172.21.15.77:0/931359013 shutdown_connections 2023-12-01T01:16:53.222 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.074+0000 7fbb85915700 1 -- 172.21.15.77:0/931359013 wait complete. 2023-12-01T01:16:53.404 INFO:teuthology.orchestra.run.smithi077.stdout:107374182451 2023-12-01T01:16:53.405 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.3 2023-12-01T01:16:53.476 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.475+0000 7f1471bbd700 1 -- 172.21.15.77:0/2759067390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f146c106f20 msgr2=0x7f146c107300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:53.476 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.475+0000 7f1471bbd700 1 --2- 172.21.15.77:0/2759067390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f146c106f20 0x7f146c107300 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f145c009990 tx=0x7f145c009ca0 comp rx=0 tx=0).stop 2023-12-01T01:16:53.476 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.475+0000 7f1471bbd700 1 -- 172.21.15.77:0/2759067390 shutdown_connections 2023-12-01T01:16:53.477 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.475+0000 7f1471bbd700 1 --2- 172.21.15.77:0/2759067390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f146c069270 0x7f146c10dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:53.477 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.475+0000 7f1471bbd700 1 --2- 172.21.15.77:0/2759067390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f146c106f20 0x7f146c107300 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:53.477 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.475+0000 7f1471bbd700 1 --2- 172.21.15.77:0/2759067390 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f146c0688b0 0x7f146c068d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:53.477 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.475+0000 7f1471bbd700 1 -- 172.21.15.77:0/2759067390 >> 172.21.15.77:0/2759067390 conn(0x7f146c076d90 msgr2=0x7f146c0771a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:53.477 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.475+0000 7f1471bbd700 1 -- 172.21.15.77:0/2759067390 shutdown_connections 2023-12-01T01:16:53.477 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.475+0000 7f1471bbd700 1 -- 172.21.15.77:0/2759067390 wait complete. 2023-12-01T01:16:53.478 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.476+0000 7f1471bbd700 1 Processor -- start 2023-12-01T01:16:53.478 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.476+0000 7f1471bbd700 1 -- start start 2023-12-01T01:16:53.478 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.476+0000 7f1471bbd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f146c0688b0 0x7f146c196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:53.479 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.476+0000 7f1471bbd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f146c069270 0x7f146c197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:53.479 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.476+0000 7f1471bbd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f146c106f20 0x7f146c19b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:53.479 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.476+0000 7f1471bbd700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146c19bef0 con 0x7f146c0688b0 2023-12-01T01:16:53.479 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.476+0000 7f1471bbd700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146c19c060 con 0x7f146c069270 2023-12-01T01:16:53.479 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.476+0000 7f1471bbd700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146c19c220 con 0x7f146c106f20 2023-12-01T01:16:53.479 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.476+0000 7f146b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f146c0688b0 0x7f146c196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:53.479 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.476+0000 7f146b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f146c0688b0 0x7f146c196f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57852/0 (socket says 172.21.15.77:57852) 2023-12-01T01:16:53.479 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.476+0000 7f146b7fe700 1 -- 172.21.15.77:0/3748567494 learned_addr learned my addr 172.21.15.77:0/3748567494 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:53.479 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.477+0000 7f146bfff700 1 --2- 172.21.15.77:0/3748567494 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f146c106f20 0x7f146c19b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:53.479 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.477+0000 7f146affd700 1 --2- 172.21.15.77:0/3748567494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f146c069270 0x7f146c197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:53.479 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.477+0000 7f146b7fe700 1 -- 172.21.15.77:0/3748567494 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f146c106f20 msgr2=0x7f146c19b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:53.479 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.477+0000 7f146b7fe700 1 --2- 172.21.15.77:0/3748567494 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f146c106f20 0x7f146c19b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:53.479 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.477+0000 7f146b7fe700 1 -- 172.21.15.77:0/3748567494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f146c069270 msgr2=0x7f146c197470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:53.479 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.477+0000 7f146b7fe700 1 --2- 172.21.15.77:0/3748567494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f146c069270 0x7f146c197470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:53.480 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.477+0000 7f146b7fe700 1 -- 172.21.15.77:0/3748567494 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f145c0095f0 con 0x7f146c0688b0 2023-12-01T01:16:53.480 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.477+0000 7f146b7fe700 1 --2- 172.21.15.77:0/3748567494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f146c0688b0 0x7f146c196f30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f145c005910 tx=0x7f145c00b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:53.480 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.477+0000 7f1468ff9700 1 -- 172.21.15.77:0/3748567494 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c01d070 con 0x7f146c0688b0 2023-12-01T01:16:53.480 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.477+0000 7f1468ff9700 1 -- 172.21.15.77:0/3748567494 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f145c00fb80 con 0x7f146c0688b0 2023-12-01T01:16:53.480 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.477+0000 7f1471bbd700 1 -- 172.21.15.77:0/3748567494 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f146c19c4a0 con 0x7f146c0688b0 2023-12-01T01:16:53.480 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.477+0000 7f1468ff9700 1 -- 172.21.15.77:0/3748567494 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c017650 con 0x7f146c0688b0 2023-12-01T01:16:53.480 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.477+0000 7f1471bbd700 1 -- 172.21.15.77:0/3748567494 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f146c0ffe40 con 0x7f146c0688b0 2023-12-01T01:16:53.480 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.478+0000 7f1471bbd700 1 -- 172.21.15.77:0/3748567494 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f146c110fc0 con 0x7f146c0688b0 2023-12-01T01:16:53.480 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.479+0000 7f1468ff9700 1 -- 172.21.15.77:0/3748567494 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f145c022470 con 0x7f146c0688b0 2023-12-01T01:16:53.481 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.480+0000 7f1468ff9700 1 --2- 172.21.15.77:0/3748567494 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f14540a05b0 0x7f14540a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:53.481 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.480+0000 7f146affd700 1 --2- 172.21.15.77:0/3748567494 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f14540a05b0 0x7f14540a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:53.481 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.480+0000 7f1468ff9700 1 -- 172.21.15.77:0/3748567494 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f145c0cd0c0 con 0x7f146c0688b0 2023-12-01T01:16:53.481 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.480+0000 7f1468ff9700 1 --2- 172.21.15.77:0/3748567494 >> [v2:172.21.15.203:6800/4078459110,v1:172.21.15.203:6801/4078459110] conn(0x7f14540a9d90 0x7f14540ac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:53.481 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.480+0000 7f1468ff9700 1 -- 172.21.15.77:0/3748567494 --> [v2:172.21.15.203:6800/4078459110,v1:172.21.15.203:6801/4078459110] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f14540ac860 con 0x7f14540a9d90 2023-12-01T01:16:53.481 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.480+0000 7f1468ff9700 1 -- 172.21.15.77:0/3748567494 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f145c0cf7d0 con 0x7f146c0688b0 2023-12-01T01:16:53.482 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.480+0000 7f146affd700 1 --2- 172.21.15.77:0/3748567494 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f14540a05b0 0x7f14540a2a70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1460005fd0 tx=0x7f1460005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:53.482 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.480+0000 7f146bfff700 1 --2- 172.21.15.77:0/3748567494 >> [v2:172.21.15.203:6800/4078459110,v1:172.21.15.203:6801/4078459110] conn(0x7f14540a9d90 0x7f14540ac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:53.482 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.481+0000 7f146bfff700 1 --2- 172.21.15.77:0/3748567494 >> [v2:172.21.15.203:6800/4078459110,v1:172.21.15.203:6801/4078459110] conn(0x7f14540a9d90 0x7f14540ac1b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:53.483 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.482+0000 7f1468ff9700 1 -- 172.21.15.77:0/3748567494 <== osd.2 v2:172.21.15.203:6800/4078459110 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f14540ac860 con 0x7f14540a9d90 2023-12-01T01:16:53.512 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.510+0000 7f1471bbd700 1 -- 172.21.15.77:0/3748567494 --> [v2:172.21.15.203:6800/4078459110,v1:172.21.15.203:6801/4078459110] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f146c19c770 con 0x7f14540a9d90 2023-12-01T01:16:53.512 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.511+0000 7f1468ff9700 1 -- 172.21.15.77:0/3748567494 <== osd.2 v2:172.21.15.203:6800/4078459110 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f146400f040 con 0x7f14540a9d90 2023-12-01T01:16:53.513 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.512+0000 7f1471bbd700 1 -- 172.21.15.77:0/3748567494 >> [v2:172.21.15.203:6800/4078459110,v1:172.21.15.203:6801/4078459110] conn(0x7f14540a9d90 msgr2=0x7f14540ac1b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:53.513 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.512+0000 7f1471bbd700 1 --2- 172.21.15.77:0/3748567494 >> [v2:172.21.15.203:6800/4078459110,v1:172.21.15.203:6801/4078459110] conn(0x7f14540a9d90 0x7f14540ac1b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:53.513 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.512+0000 7f1471bbd700 1 -- 172.21.15.77:0/3748567494 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f14540a05b0 msgr2=0x7f14540a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:53.513 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.512+0000 7f1471bbd700 1 --2- 172.21.15.77:0/3748567494 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f14540a05b0 0x7f14540a2a70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1460005fd0 tx=0x7f1460005f60 comp rx=0 tx=0).stop 2023-12-01T01:16:53.513 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.512+0000 7f1471bbd700 1 -- 172.21.15.77:0/3748567494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f146c0688b0 msgr2=0x7f146c196f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:53.513 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.512+0000 7f1471bbd700 1 --2- 172.21.15.77:0/3748567494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f146c0688b0 0x7f146c196f30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f145c005910 tx=0x7f145c00b610 comp rx=0 tx=0).stop 2023-12-01T01:16:53.513 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.512+0000 7f146bfff700 1 -- 172.21.15.77:0/3748567494 reap_dead start 2023-12-01T01:16:53.514 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.513+0000 7f1471bbd700 1 -- 172.21.15.77:0/3748567494 shutdown_connections 2023-12-01T01:16:53.514 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.513+0000 7f1471bbd700 1 -- 172.21.15.77:0/3748567494 >> 172.21.15.77:0/3748567494 conn(0x7f146c076d90 msgr2=0x7f146c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:53.515 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.513+0000 7f1471bbd700 1 -- 172.21.15.77:0/3748567494 shutdown_connections 2023-12-01T01:16:53.515 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.513+0000 7f1471bbd700 1 -- 172.21.15.77:0/3748567494 wait complete. 2023-12-01T01:16:53.640 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:53 smithi077 ceph-mon[98867]: pgmap v266: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2023-12-01T01:16:53.734 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.608+0000 7f7d67a46700 1 -- 172.21.15.77:0/288538905 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d60100690 msgr2=0x7f7d6010cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:53.734 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.608+0000 7f7d67a46700 1 --2- 172.21.15.77:0/288538905 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d60100690 0x7f7d6010cb30 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7d5800b0a0 tx=0x7f7d5800b3b0 comp rx=0 tx=0).stop 2023-12-01T01:16:53.734 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.613+0000 7f7d67a46700 1 -- 172.21.15.77:0/288538905 shutdown_connections 2023-12-01T01:16:53.734 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.613+0000 7f7d67a46700 1 --2- 172.21.15.77:0/288538905 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d60100690 0x7f7d6010cb30 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:53.735 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.613+0000 7f7d67a46700 1 --2- 172.21.15.77:0/288538905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d60105d80 0x7f7d60106160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:53.735 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.613+0000 7f7d67a46700 1 --2- 172.21.15.77:0/288538905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d600ffcd0 0x7f7d60100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:53.735 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.613+0000 7f7d67a46700 1 -- 172.21.15.77:0/288538905 >> 172.21.15.77:0/288538905 conn(0x7f7d60076d90 msgr2=0x7f7d600771a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:53.735 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.613+0000 7f7d67a46700 1 -- 172.21.15.77:0/288538905 shutdown_connections 2023-12-01T01:16:53.735 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.613+0000 7f7d67a46700 1 -- 172.21.15.77:0/288538905 wait complete. 2023-12-01T01:16:53.735 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.614+0000 7f7d67a46700 1 Processor -- start 2023-12-01T01:16:53.735 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.614+0000 7f7d67a46700 1 -- start start 2023-12-01T01:16:53.735 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.614+0000 7f7d67a46700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d600ffcd0 0x7f7d6010c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:53.735 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.614+0000 7f7d67a46700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d60100690 0x7f7d6010ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:53.735 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.614+0000 7f7d67a46700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d60105d80 0x7f7d601096a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:53.735 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.614+0000 7f7d67a46700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d60109e70 con 0x7f7d60100690 2023-12-01T01:16:53.735 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.614+0000 7f7d67a46700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d60109fe0 con 0x7f7d60105d80 2023-12-01T01:16:53.735 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.614+0000 7f7d67a46700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6010a1a0 con 0x7f7d600ffcd0 2023-12-01T01:16:53.735 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.614+0000 7f7d64fe1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d60100690 0x7f7d6010ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:53.736 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.615+0000 7f7d64fe1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d60100690 0x7f7d6010ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57872/0 (socket says 172.21.15.77:57872) 2023-12-01T01:16:53.736 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.615+0000 7f7d64fe1700 1 -- 172.21.15.77:0/3270020085 learned_addr learned my addr 172.21.15.77:0/3270020085 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:53.736 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.615+0000 7f7d65fe3700 1 --2- 172.21.15.77:0/3270020085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d60105d80 0x7f7d601096a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:53.736 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.615+0000 7f7d64fe1700 1 -- 172.21.15.77:0/3270020085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d600ffcd0 msgr2=0x7f7d6010c530 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-01T01:16:53.736 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.615+0000 7f7d64fe1700 1 --2- 172.21.15.77:0/3270020085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d600ffcd0 0x7f7d6010c530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:53.736 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.615+0000 7f7d64fe1700 1 -- 172.21.15.77:0/3270020085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d60105d80 msgr2=0x7f7d601096a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:53.736 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.615+0000 7f7d64fe1700 1 --2- 172.21.15.77:0/3270020085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d60105d80 0x7f7d601096a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:53.736 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.615+0000 7f7d64fe1700 1 -- 172.21.15.77:0/3270020085 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d5c009520 con 0x7f7d60100690 2023-12-01T01:16:53.736 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.615+0000 7f7d64fe1700 1 --2- 172.21.15.77:0/3270020085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d60100690 0x7f7d6010ca70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7d5000b530 tx=0x7f7d5000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:53.736 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.615+0000 7f7d567fc700 1 -- 172.21.15.77:0/3270020085 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d50011830 con 0x7f7d60100690 2023-12-01T01:16:53.736 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.616+0000 7f7d67a46700 1 -- 172.21.15.77:0/3270020085 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d58009d00 con 0x7f7d60100690 2023-12-01T01:16:53.736 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.616+0000 7f7d567fc700 1 -- 172.21.15.77:0/3270020085 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d50011e50 con 0x7f7d60100690 2023-12-01T01:16:53.736 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.616+0000 7f7d567fc700 1 -- 172.21.15.77:0/3270020085 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5000f2e0 con 0x7f7d60100690 2023-12-01T01:16:53.736 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.616+0000 7f7d67a46700 1 -- 172.21.15.77:0/3270020085 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d6019f880 con 0x7f7d60100690 2023-12-01T01:16:53.737 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.619+0000 7f7d567fc700 1 -- 172.21.15.77:0/3270020085 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f7d50010380 con 0x7f7d60100690 2023-12-01T01:16:53.737 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.620+0000 7f7d67a46700 1 -- 172.21.15.77:0/3270020085 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7d6010ed00 con 0x7f7d60100690 2023-12-01T01:16:53.737 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.620+0000 7f7d567fc700 1 --2- 172.21.15.77:0/3270020085 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7d4c0a05b0 0x7f7d4c0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:53.737 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.620+0000 7f7d567fc700 1 -- 172.21.15.77:0/3270020085 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7d500cb420 con 0x7f7d60100690 2023-12-01T01:16:53.737 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.621+0000 7f7d567fc700 1 --2- 172.21.15.77:0/3270020085 >> [v2:172.21.15.77:6802/2244799094,v1:172.21.15.77:6803/2244799094] conn(0x7f7d4c0a9d90 0x7f7d4c0ac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:53.737 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.621+0000 7f7d567fc700 1 -- 172.21.15.77:0/3270020085 --> [v2:172.21.15.77:6802/2244799094,v1:172.21.15.77:6803/2244799094] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7d4c0ac860 con 0x7f7d4c0a9d90 2023-12-01T01:16:53.737 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.621+0000 7f7d567fc700 1 -- 172.21.15.77:0/3270020085 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d50094f50 con 0x7f7d60100690 2023-12-01T01:16:53.737 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.621+0000 7f7d657e2700 1 --2- 172.21.15.77:0/3270020085 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7d4c0a05b0 0x7f7d4c0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:53.737 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.621+0000 7f7d65fe3700 1 --2- 172.21.15.77:0/3270020085 >> [v2:172.21.15.77:6802/2244799094,v1:172.21.15.77:6803/2244799094] conn(0x7f7d4c0a9d90 0x7f7d4c0ac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:53.738 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.621+0000 7f7d65fe3700 1 --2- 172.21.15.77:0/3270020085 >> [v2:172.21.15.77:6802/2244799094,v1:172.21.15.77:6803/2244799094] conn(0x7f7d4c0a9d90 0x7f7d4c0ac1b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:53.738 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.621+0000 7f7d657e2700 1 --2- 172.21.15.77:0/3270020085 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7d4c0a05b0 0x7f7d4c0a2a70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c009ca0 tx=0x7f7d5c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:53.738 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.622+0000 7f7d567fc700 1 -- 172.21.15.77:0/3270020085 <== osd.0 v2:172.21.15.77:6802/2244799094 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7d4c0ac860 con 0x7f7d4c0a9d90 2023-12-01T01:16:53.738 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.645+0000 7f7d67a46700 1 -- 172.21.15.77:0/3270020085 --> [v2:172.21.15.77:6802/2244799094,v1:172.21.15.77:6803/2244799094] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7d6004ea90 con 0x7f7d4c0a9d90 2023-12-01T01:16:53.738 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.646+0000 7f7d567fc700 1 -- 172.21.15.77:0/3270020085 <== osd.0 v2:172.21.15.77:6802/2244799094 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7d6004ea90 con 0x7f7d4c0a9d90 2023-12-01T01:16:53.738 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.646+0000 7f7d67a46700 1 -- 172.21.15.77:0/3270020085 >> [v2:172.21.15.77:6802/2244799094,v1:172.21.15.77:6803/2244799094] conn(0x7f7d4c0a9d90 msgr2=0x7f7d4c0ac1b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:53.738 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.646+0000 7f7d67a46700 1 --2- 172.21.15.77:0/3270020085 >> [v2:172.21.15.77:6802/2244799094,v1:172.21.15.77:6803/2244799094] conn(0x7f7d4c0a9d90 0x7f7d4c0ac1b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:53.738 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.646+0000 7f7d67a46700 1 -- 172.21.15.77:0/3270020085 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7d4c0a05b0 msgr2=0x7f7d4c0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:53.738 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.646+0000 7f7d67a46700 1 --2- 172.21.15.77:0/3270020085 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7d4c0a05b0 0x7f7d4c0a2a70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c009ca0 tx=0x7f7d5c009260 comp rx=0 tx=0).stop 2023-12-01T01:16:53.738 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.646+0000 7f7d67a46700 1 -- 172.21.15.77:0/3270020085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d60100690 msgr2=0x7f7d6010ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:53.738 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.646+0000 7f7d67a46700 1 --2- 172.21.15.77:0/3270020085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d60100690 0x7f7d6010ca70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7d5000b530 tx=0x7f7d5000b840 comp rx=0 tx=0).stop 2023-12-01T01:16:53.738 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.646+0000 7f7d65fe3700 1 -- 172.21.15.77:0/3270020085 reap_dead start 2023-12-01T01:16:53.738 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.646+0000 7f7d67a46700 1 -- 172.21.15.77:0/3270020085 shutdown_connections 2023-12-01T01:16:53.738 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.646+0000 7f7d67a46700 1 -- 172.21.15.77:0/3270020085 >> 172.21.15.77:0/3270020085 conn(0x7f7d60076d90 msgr2=0x7f7d60103930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:53.739 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.646+0000 7f7d67a46700 1 -- 172.21.15.77:0/3270020085 shutdown_connections 2023-12-01T01:16:53.739 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.646+0000 7f7d67a46700 1 -- 172.21.15.77:0/3270020085 wait complete. 2023-12-01T01:16:53.967 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.712+0000 7f0bb6bb8700 1 -- 172.21.15.77:0/1349493346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bb00ff2c0 msgr2=0x7f0bb00ff740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:53.967 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.712+0000 7f0bb6bb8700 1 --2- 172.21.15.77:0/1349493346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bb00ff2c0 0x7f0bb00ff740 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c009990 tx=0x7f0b9c009ca0 comp rx=0 tx=0).stop 2023-12-01T01:16:53.967 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.712+0000 7f0bb6bb8700 1 -- 172.21.15.77:0/1349493346 shutdown_connections 2023-12-01T01:16:53.967 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.712+0000 7f0bb6bb8700 1 --2- 172.21.15.77:0/1349493346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0bb00ffc80 0x7f0bb010dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:53.968 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.712+0000 7f0bb6bb8700 1 --2- 172.21.15.77:0/1349493346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bb0105d90 0x7f0bb0106170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:53.968 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.712+0000 7f0bb6bb8700 1 --2- 172.21.15.77:0/1349493346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bb00ff2c0 0x7f0bb00ff740 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:53.968 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.712+0000 7f0bb6bb8700 1 -- 172.21.15.77:0/1349493346 >> 172.21.15.77:0/1349493346 conn(0x7f0bb0076200 msgr2=0x7f0bb00fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:53.968 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.712+0000 7f0bb6bb8700 1 -- 172.21.15.77:0/1349493346 shutdown_connections 2023-12-01T01:16:53.968 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.712+0000 7f0bb6bb8700 1 -- 172.21.15.77:0/1349493346 wait complete. 2023-12-01T01:16:53.968 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.713+0000 7f0bb6bb8700 1 Processor -- start 2023-12-01T01:16:53.968 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.713+0000 7f0bb6bb8700 1 -- start start 2023-12-01T01:16:53.968 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.713+0000 7f0bb6bb8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bb00ff2c0 0x7f0bb0196f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:53.968 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.713+0000 7f0bb6bb8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bb00ffc80 0x7f0bb0197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:53.968 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.713+0000 7f0bb6bb8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0bb0105d90 0x7f0bb019b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:53.968 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.713+0000 7f0bb6bb8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb019bf00 con 0x7f0bb00ff2c0 2023-12-01T01:16:53.968 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.713+0000 7f0bb6bb8700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb019c070 con 0x7f0bb00ffc80 2023-12-01T01:16:53.968 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.713+0000 7f0bb6bb8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb019c230 con 0x7f0bb0105d90 2023-12-01T01:16:53.968 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.713+0000 7f0bb4954700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bb00ff2c0 0x7f0bb0196f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:53.968 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.713+0000 7f0bb4954700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bb00ff2c0 0x7f0bb0196f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57896/0 (socket says 172.21.15.77:57896) 2023-12-01T01:16:53.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.713+0000 7f0bb4954700 1 -- 172.21.15.77:0/2609506593 learned_addr learned my addr 172.21.15.77:0/2609506593 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:53.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.713+0000 7f0baffff700 1 --2- 172.21.15.77:0/2609506593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bb00ffc80 0x7f0bb0197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:53.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.713+0000 7f0bb4954700 1 -- 172.21.15.77:0/2609506593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0bb0105d90 msgr2=0x7f0bb019b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:53.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.713+0000 7f0bb5155700 1 --2- 172.21.15.77:0/2609506593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0bb0105d90 0x7f0bb019b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:53.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.713+0000 7f0bb4954700 1 --2- 172.21.15.77:0/2609506593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0bb0105d90 0x7f0bb019b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:53.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.713+0000 7f0bb4954700 1 -- 172.21.15.77:0/2609506593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bb00ffc80 msgr2=0x7f0bb0197480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:53.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.713+0000 7f0bb4954700 1 --2- 172.21.15.77:0/2609506593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bb00ffc80 0x7f0bb0197480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:53.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.713+0000 7f0bb4954700 1 -- 172.21.15.77:0/2609506593 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b9c0095f0 con 0x7f0bb00ff2c0 2023-12-01T01:16:53.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.713+0000 7f0bb4954700 1 --2- 172.21.15.77:0/2609506593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bb00ff2c0 0x7f0bb0196f40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0ba400e9a0 tx=0x7f0ba400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:53.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.714+0000 7f0badffb700 1 -- 172.21.15.77:0/2609506593 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba4009790 con 0x7f0bb00ff2c0 2023-12-01T01:16:53.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.714+0000 7f0badffb700 1 -- 172.21.15.77:0/2609506593 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ba400cbd0 con 0x7f0bb00ff2c0 2023-12-01T01:16:53.970 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.714+0000 7f0bb6bb8700 1 -- 172.21.15.77:0/2609506593 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bb019c510 con 0x7f0bb00ff2c0 2023-12-01T01:16:53.970 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.714+0000 7f0badffb700 1 -- 172.21.15.77:0/2609506593 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba4018680 con 0x7f0bb00ff2c0 2023-12-01T01:16:53.970 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.714+0000 7f0bb6bb8700 1 -- 172.21.15.77:0/2609506593 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bb01a3d70 con 0x7f0bb00ff2c0 2023-12-01T01:16:53.970 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.715+0000 7f0bb6bb8700 1 -- 172.21.15.77:0/2609506593 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0bb0111150 con 0x7f0bb00ff2c0 2023-12-01T01:16:53.970 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.716+0000 7f0badffb700 1 -- 172.21.15.77:0/2609506593 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f0ba400cd40 con 0x7f0bb00ff2c0 2023-12-01T01:16:53.970 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.716+0000 7f0badffb700 1 --2- 172.21.15.77:0/2609506593 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f0ba00a05b0 0x7f0ba00a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:53.970 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.717+0000 7f0baffff700 1 --2- 172.21.15.77:0/2609506593 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f0ba00a05b0 0x7f0ba00a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:53.970 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.717+0000 7f0badffb700 1 -- 172.21.15.77:0/2609506593 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0ba4014070 con 0x7f0bb00ff2c0 2023-12-01T01:16:53.971 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.717+0000 7f0badffb700 1 --2- 172.21.15.77:0/2609506593 >> [v2:172.21.15.77:6826/2409880685,v1:172.21.15.77:6827/2409880685] conn(0x7f0ba00a9d90 0x7f0ba00ac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:53.971 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.717+0000 7f0badffb700 1 -- 172.21.15.77:0/2609506593 --> [v2:172.21.15.77:6826/2409880685,v1:172.21.15.77:6827/2409880685] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0ba00ac860 con 0x7f0ba00a9d90 2023-12-01T01:16:53.971 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.717+0000 7f0badffb700 1 -- 172.21.15.77:0/2609506593 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ba40cc7d0 con 0x7f0bb00ff2c0 2023-12-01T01:16:53.971 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.717+0000 7f0bb5155700 1 --2- 172.21.15.77:0/2609506593 >> [v2:172.21.15.77:6826/2409880685,v1:172.21.15.77:6827/2409880685] conn(0x7f0ba00a9d90 0x7f0ba00ac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:53.971 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.717+0000 7f0baffff700 1 --2- 172.21.15.77:0/2609506593 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f0ba00a05b0 0x7f0ba00a2a70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0198560 tx=0x7f0b9c00b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:53.971 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.717+0000 7f0bb5155700 1 --2- 172.21.15.77:0/2609506593 >> [v2:172.21.15.77:6826/2409880685,v1:172.21.15.77:6827/2409880685] conn(0x7f0ba00a9d90 0x7f0ba00ac1b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:53.971 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.718+0000 7f0badffb700 1 -- 172.21.15.77:0/2609506593 <== osd.9 v2:172.21.15.77:6826/2409880685 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f0ba00ac860 con 0x7f0ba00a9d90 2023-12-01T01:16:53.971 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.734+0000 7f0bb6bb8700 1 -- 172.21.15.77:0/2609506593 --> [v2:172.21.15.77:6826/2409880685,v1:172.21.15.77:6827/2409880685] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0bb004ea90 con 0x7f0ba00a9d90 2023-12-01T01:16:53.971 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.734+0000 7f0badffb700 1 -- 172.21.15.77:0/2609506593 <== osd.9 v2:172.21.15.77:6826/2409880685 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0bb004ea90 con 0x7f0ba00a9d90 2023-12-01T01:16:53.971 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.735+0000 7f0bb6bb8700 1 -- 172.21.15.77:0/2609506593 >> [v2:172.21.15.77:6826/2409880685,v1:172.21.15.77:6827/2409880685] conn(0x7f0ba00a9d90 msgr2=0x7f0ba00ac1b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:53.971 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.735+0000 7f0bb6bb8700 1 --2- 172.21.15.77:0/2609506593 >> [v2:172.21.15.77:6826/2409880685,v1:172.21.15.77:6827/2409880685] conn(0x7f0ba00a9d90 0x7f0ba00ac1b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:53.971 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.735+0000 7f0bb6bb8700 1 -- 172.21.15.77:0/2609506593 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f0ba00a05b0 msgr2=0x7f0ba00a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:53.971 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.735+0000 7f0bb6bb8700 1 --2- 172.21.15.77:0/2609506593 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f0ba00a05b0 0x7f0ba00a2a70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0198560 tx=0x7f0b9c00b280 comp rx=0 tx=0).stop 2023-12-01T01:16:53.972 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.735+0000 7f0bb6bb8700 1 -- 172.21.15.77:0/2609506593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bb00ff2c0 msgr2=0x7f0bb0196f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:53.972 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.735+0000 7f0bb6bb8700 1 --2- 172.21.15.77:0/2609506593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bb00ff2c0 0x7f0bb0196f40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0ba400e9a0 tx=0x7f0ba400ed60 comp rx=0 tx=0).stop 2023-12-01T01:16:53.972 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.735+0000 7f0bb5155700 1 -- 172.21.15.77:0/2609506593 reap_dead start 2023-12-01T01:16:53.972 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.735+0000 7f0bb6bb8700 1 -- 172.21.15.77:0/2609506593 shutdown_connections 2023-12-01T01:16:53.972 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.735+0000 7f0bb6bb8700 1 -- 172.21.15.77:0/2609506593 >> 172.21.15.77:0/2609506593 conn(0x7f0bb0076200 msgr2=0x7f0bb0104570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:53.972 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.735+0000 7f0bb6bb8700 1 -- 172.21.15.77:0/2609506593 shutdown_connections 2023-12-01T01:16:53.972 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.735+0000 7f0bb6bb8700 1 -- 172.21.15.77:0/2609506593 wait complete. 2023-12-01T01:16:53.998 INFO:teuthology.orchestra.run.smithi077.stdout:296352743445 2023-12-01T01:16:53.999 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.10 2023-12-01T01:16:54.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:53 smithi203 ceph-mon[103581]: pgmap v266: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2023-12-01T01:16:54.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:53 smithi137 ceph-mon[103312]: pgmap v266: 97 pgs: 97 active+clean; 635 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2023-12-01T01:16:54.175 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.174+0000 7fd4d576d700 1 -- 172.21.15.77:0/752558537 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4d0105d80 msgr2=0x7fd4d0106160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:54.176 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.174+0000 7fd4d576d700 1 --2- 172.21.15.77:0/752558537 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4d0105d80 0x7fd4d0106160 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0009990 tx=0x7fd4c0009ca0 comp rx=0 tx=0).stop 2023-12-01T01:16:54.176 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.174+0000 7fd4d576d700 1 -- 172.21.15.77:0/752558537 shutdown_connections 2023-12-01T01:16:54.176 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.174+0000 7fd4d576d700 1 --2- 172.21.15.77:0/752558537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4d0100690 0x7fd4d010cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:54.176 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.174+0000 7fd4d576d700 1 --2- 172.21.15.77:0/752558537 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4d0105d80 0x7fd4d0106160 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:54.176 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.174+0000 7fd4d576d700 1 --2- 172.21.15.77:0/752558537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4d00ffcd0 0x7fd4d0100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:54.176 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.174+0000 7fd4d576d700 1 -- 172.21.15.77:0/752558537 >> 172.21.15.77:0/752558537 conn(0x7fd4d0076d90 msgr2=0x7fd4d00771a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:54.176 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.174+0000 7fd4d576d700 1 -- 172.21.15.77:0/752558537 shutdown_connections 2023-12-01T01:16:54.176 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.174+0000 7fd4d576d700 1 -- 172.21.15.77:0/752558537 wait complete. 2023-12-01T01:16:54.177 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.175+0000 7fd4d576d700 1 Processor -- start 2023-12-01T01:16:54.177 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.176+0000 7fd4d576d700 1 -- start start 2023-12-01T01:16:54.177 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.176+0000 7fd4d576d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4d00ffcd0 0x7fd4d0197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:54.177 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.176+0000 7fd4d576d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4d0100690 0x7fd4d0198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:54.179 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.176+0000 7fd4d576d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4d0105d80 0x7fd4d01a2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:54.179 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.176+0000 7fd4d576d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d0198aa0 con 0x7fd4d00ffcd0 2023-12-01T01:16:54.179 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.176+0000 7fd4d576d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d01a3210 con 0x7fd4d0105d80 2023-12-01T01:16:54.179 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.176+0000 7fd4d576d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d01a3380 con 0x7fd4d0100690 2023-12-01T01:16:54.179 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.176+0000 7fd4ceffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4d00ffcd0 0x7fd4d0197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:54.179 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.176+0000 7fd4ceffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4d00ffcd0 0x7fd4d0197c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57912/0 (socket says 172.21.15.77:57912) 2023-12-01T01:16:54.179 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.176+0000 7fd4ceffd700 1 -- 172.21.15.77:0/43078387 learned_addr learned my addr 172.21.15.77:0/43078387 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:54.179 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.176+0000 7fd4ce7fc700 1 --2- 172.21.15.77:0/43078387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4d0100690 0x7fd4d0198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:54.179 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.176+0000 7fd4cf7fe700 1 --2- 172.21.15.77:0/43078387 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4d0105d80 0x7fd4d01a2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:54.179 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.176+0000 7fd4ceffd700 1 -- 172.21.15.77:0/43078387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4d0100690 msgr2=0x7fd4d0198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:54.179 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.176+0000 7fd4ceffd700 1 --2- 172.21.15.77:0/43078387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4d0100690 0x7fd4d0198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:54.179 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.176+0000 7fd4ceffd700 1 -- 172.21.15.77:0/43078387 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4d0105d80 msgr2=0x7fd4d01a2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:54.180 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.176+0000 7fd4ceffd700 1 --2- 172.21.15.77:0/43078387 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4d0105d80 0x7fd4d01a2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:54.180 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.177+0000 7fd4ceffd700 1 -- 172.21.15.77:0/43078387 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c00095f0 con 0x7fd4d00ffcd0 2023-12-01T01:16:54.180 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.177+0000 7fd4cf7fe700 1 --2- 172.21.15.77:0/43078387 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4d0105d80 0x7fd4d01a2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:16:54.180 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.177+0000 7fd4ceffd700 1 --2- 172.21.15.77:0/43078387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4d00ffcd0 0x7fd4d0197c50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0000c00 tx=0x7fd4c000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:54.180 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.177+0000 7fd4b7fff700 1 -- 172.21.15.77:0/43078387 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c001d070 con 0x7fd4d00ffcd0 2023-12-01T01:16:54.180 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.177+0000 7fd4b7fff700 1 -- 172.21.15.77:0/43078387 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4c000be40 con 0x7fd4d00ffcd0 2023-12-01T01:16:54.180 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.177+0000 7fd4d576d700 1 -- 172.21.15.77:0/43078387 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4d01a3600 con 0x7fd4d00ffcd0 2023-12-01T01:16:54.180 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.177+0000 7fd4b7fff700 1 -- 172.21.15.77:0/43078387 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0017650 con 0x7fd4d00ffcd0 2023-12-01T01:16:54.180 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.177+0000 7fd4d576d700 1 -- 172.21.15.77:0/43078387 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4d01a3aa0 con 0x7fd4d00ffcd0 2023-12-01T01:16:54.180 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.178+0000 7fd4d576d700 1 -- 172.21.15.77:0/43078387 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd4d004ea90 con 0x7fd4d00ffcd0 2023-12-01T01:16:54.181 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.180+0000 7fd4b7fff700 1 -- 172.21.15.77:0/43078387 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fd4c0022470 con 0x7fd4d00ffcd0 2023-12-01T01:16:54.183 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.181+0000 7fd4b7fff700 1 --2- 172.21.15.77:0/43078387 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd4b80a05b0 0x7fd4b80a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:54.183 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.181+0000 7fd4b7fff700 1 -- 172.21.15.77:0/43078387 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd4c00cd790 con 0x7fd4d00ffcd0 2023-12-01T01:16:54.183 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.181+0000 7fd4b7fff700 1 --2- 172.21.15.77:0/43078387 >> [v2:172.21.15.137:6800/224194890,v1:172.21.15.137:6801/224194890] conn(0x7fd4b80a9d90 0x7fd4b80ac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:54.183 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.181+0000 7fd4b7fff700 1 -- 172.21.15.77:0/43078387 --> [v2:172.21.15.137:6800/224194890,v1:172.21.15.137:6801/224194890] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd4b80ac860 con 0x7fd4b80a9d90 2023-12-01T01:16:54.183 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.181+0000 7fd4b7fff700 1 -- 172.21.15.77:0/43078387 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4c00cdc30 con 0x7fd4d00ffcd0 2023-12-01T01:16:54.183 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.182+0000 7fd4ce7fc700 1 --2- 172.21.15.77:0/43078387 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd4b80a05b0 0x7fd4b80a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:54.184 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.182+0000 7fd4cf7fe700 1 --2- 172.21.15.77:0/43078387 >> [v2:172.21.15.137:6800/224194890,v1:172.21.15.137:6801/224194890] conn(0x7fd4b80a9d90 0x7fd4b80ac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:54.184 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.182+0000 7fd4ce7fc700 1 --2- 172.21.15.77:0/43078387 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd4b80a05b0 0x7fd4b80a2a70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0069100 tx=0x7fd4c40073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:54.184 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.182+0000 7fd4cf7fe700 1 --2- 172.21.15.77:0/43078387 >> [v2:172.21.15.137:6800/224194890,v1:172.21.15.137:6801/224194890] conn(0x7fd4b80a9d90 0x7fd4b80ac1b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:54.184 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.183+0000 7fd4b7fff700 1 -- 172.21.15.77:0/43078387 <== osd.1 v2:172.21.15.137:6800/224194890 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fd4b80ac860 con 0x7fd4b80a9d90 2023-12-01T01:16:54.200 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.199+0000 7fd4d576d700 1 -- 172.21.15.77:0/43078387 --> [v2:172.21.15.137:6800/224194890,v1:172.21.15.137:6801/224194890] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd4d010fee0 con 0x7fd4b80a9d90 2023-12-01T01:16:54.201 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.200+0000 7fd4b7fff700 1 -- 172.21.15.77:0/43078387 <== osd.1 v2:172.21.15.137:6800/224194890 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd4c8011040 con 0x7fd4b80a9d90 2023-12-01T01:16:54.201 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.200+0000 7fd4d576d700 1 -- 172.21.15.77:0/43078387 >> [v2:172.21.15.137:6800/224194890,v1:172.21.15.137:6801/224194890] conn(0x7fd4b80a9d90 msgr2=0x7fd4b80ac1b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:54.201 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.200+0000 7fd4d576d700 1 --2- 172.21.15.77:0/43078387 >> [v2:172.21.15.137:6800/224194890,v1:172.21.15.137:6801/224194890] conn(0x7fd4b80a9d90 0x7fd4b80ac1b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:54.201 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.200+0000 7fd4d576d700 1 -- 172.21.15.77:0/43078387 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd4b80a05b0 msgr2=0x7fd4b80a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:54.201 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.200+0000 7fd4d576d700 1 --2- 172.21.15.77:0/43078387 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd4b80a05b0 0x7fd4b80a2a70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0069100 tx=0x7fd4c40073d0 comp rx=0 tx=0).stop 2023-12-01T01:16:54.201 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.200+0000 7fd4d576d700 1 -- 172.21.15.77:0/43078387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4d00ffcd0 msgr2=0x7fd4d0197c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:54.202 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.200+0000 7fd4d576d700 1 --2- 172.21.15.77:0/43078387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4d00ffcd0 0x7fd4d0197c50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0000c00 tx=0x7fd4c000f690 comp rx=0 tx=0).stop 2023-12-01T01:16:54.202 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.200+0000 7fd4cf7fe700 1 -- 172.21.15.77:0/43078387 reap_dead start 2023-12-01T01:16:54.202 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.200+0000 7fd4d576d700 1 -- 172.21.15.77:0/43078387 shutdown_connections 2023-12-01T01:16:54.202 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.200+0000 7fd4d576d700 1 -- 172.21.15.77:0/43078387 >> 172.21.15.77:0/43078387 conn(0x7fd4d0076d90 msgr2=0x7fd4d00fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:54.202 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.200+0000 7fd4d576d700 1 -- 172.21.15.77:0/43078387 shutdown_connections 2023-12-01T01:16:54.202 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:54.200+0000 7fd4d576d700 1 -- 172.21.15.77:0/43078387 wait complete. 2023-12-01T01:16:54.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:54 smithi077 ceph-mon[98867]: pgmap v267: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2023-12-01T01:16:54.983 INFO:teuthology.orchestra.run.smithi077.stdout:274877906970 2023-12-01T01:16:54.984 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.9 2023-12-01T01:16:55.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:54 smithi203 ceph-mon[103581]: pgmap v267: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2023-12-01T01:16:55.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:54 smithi137 ceph-mon[103312]: pgmap v267: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2023-12-01T01:16:55.185 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.677+0000 7f7ccaf12700 1 -- 172.21.15.77:0/2503236697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cc4100690 msgr2=0x7f7cc410cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:55.185 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.677+0000 7f7ccaf12700 1 --2- 172.21.15.77:0/2503236697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cc4100690 0x7f7cc410cb30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc00b0a0 tx=0x7f7cbc00b3b0 comp rx=0 tx=0).stop 2023-12-01T01:16:55.185 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.677+0000 7f7ccaf12700 1 -- 172.21.15.77:0/2503236697 shutdown_connections 2023-12-01T01:16:55.185 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.677+0000 7f7ccaf12700 1 --2- 172.21.15.77:0/2503236697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cc4100690 0x7f7cc410cb30 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:55.185 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.677+0000 7f7ccaf12700 1 --2- 172.21.15.77:0/2503236697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cc4105d80 0x7f7cc4106160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:55.185 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.677+0000 7f7ccaf12700 1 --2- 172.21.15.77:0/2503236697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7cc40ffcd0 0x7f7cc4100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:55.185 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.677+0000 7f7ccaf12700 1 -- 172.21.15.77:0/2503236697 >> 172.21.15.77:0/2503236697 conn(0x7f7cc4076d90 msgr2=0x7f7cc40771a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:55.185 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.677+0000 7f7ccaf12700 1 -- 172.21.15.77:0/2503236697 shutdown_connections 2023-12-01T01:16:55.185 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.677+0000 7f7ccaf12700 1 -- 172.21.15.77:0/2503236697 wait complete. 2023-12-01T01:16:55.185 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.678+0000 7f7ccaf12700 1 Processor -- start 2023-12-01T01:16:55.185 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.678+0000 7f7ccaf12700 1 -- start start 2023-12-01T01:16:55.185 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.678+0000 7f7ccaf12700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cc40ffcd0 0x7f7cc4197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:55.186 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.678+0000 7f7ccaf12700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cc4100690 0x7f7cc4198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:55.186 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.678+0000 7f7ccaf12700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7cc4105d80 0x7f7cc41a2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:55.186 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.678+0000 7f7ccaf12700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc4198c70 con 0x7f7cc4100690 2023-12-01T01:16:55.186 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.678+0000 7f7ccaf12700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc41a33b0 con 0x7f7cc4105d80 2023-12-01T01:16:55.186 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.678+0000 7f7ccaf12700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc41a3570 con 0x7f7cc40ffcd0 2023-12-01T01:16:55.186 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.678+0000 7f7cc3fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cc4100690 0x7f7cc4198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:55.186 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.678+0000 7f7cc3fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cc4100690 0x7f7cc4198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57884/0 (socket says 172.21.15.77:57884) 2023-12-01T01:16:55.186 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.678+0000 7f7cc3fff700 1 -- 172.21.15.77:0/3126532442 learned_addr learned my addr 172.21.15.77:0/3126532442 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:55.186 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.678+0000 7f7cc8cae700 1 --2- 172.21.15.77:0/3126532442 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cc40ffcd0 0x7f7cc4197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:55.186 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.678+0000 7f7cc94af700 1 --2- 172.21.15.77:0/3126532442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7cc4105d80 0x7f7cc41a2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:55.186 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.678+0000 7f7cc3fff700 1 -- 172.21.15.77:0/3126532442 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cc40ffcd0 msgr2=0x7f7cc4197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:55.186 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.678+0000 7f7cc3fff700 1 --2- 172.21.15.77:0/3126532442 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cc40ffcd0 0x7f7cc4197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:55.186 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.678+0000 7f7cc3fff700 1 -- 172.21.15.77:0/3126532442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7cc4105d80 msgr2=0x7f7cc41a2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:55.187 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.678+0000 7f7cc3fff700 1 --2- 172.21.15.77:0/3126532442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7cc4105d80 0x7f7cc41a2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:55.187 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.678+0000 7f7cc3fff700 1 -- 172.21.15.77:0/3126532442 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cbc009d00 con 0x7f7cc4100690 2023-12-01T01:16:55.187 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.678+0000 7f7cc3fff700 1 --2- 172.21.15.77:0/3126532442 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cc4100690 0x7f7cc4198330 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7cb000e9a0 tx=0x7f7cb000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:55.187 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.679+0000 7f7cc1ffb700 1 -- 172.21.15.77:0/3126532442 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0009790 con 0x7f7cc4100690 2023-12-01T01:16:55.187 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.679+0000 7f7ccaf12700 1 -- 172.21.15.77:0/3126532442 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cc41a3850 con 0x7f7cc4100690 2023-12-01T01:16:55.187 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.679+0000 7f7cc1ffb700 1 -- 172.21.15.77:0/3126532442 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7cb0010430 con 0x7f7cc4100690 2023-12-01T01:16:55.187 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.679+0000 7f7cc1ffb700 1 -- 172.21.15.77:0/3126532442 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0018680 con 0x7f7cc4100690 2023-12-01T01:16:55.187 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.679+0000 7f7ccaf12700 1 -- 172.21.15.77:0/3126532442 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cc41a3da0 con 0x7f7cc4100690 2023-12-01T01:16:55.187 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.680+0000 7f7ccaf12700 1 -- 172.21.15.77:0/3126532442 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7cc404ea90 con 0x7f7cc4100690 2023-12-01T01:16:55.187 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.681+0000 7f7cc1ffb700 1 -- 172.21.15.77:0/3126532442 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f7cb00105a0 con 0x7f7cc4100690 2023-12-01T01:16:55.187 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.681+0000 7f7cc1ffb700 1 --2- 172.21.15.77:0/3126532442 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7cb40a05b0 0x7f7cb40a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:55.187 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.681+0000 7f7cc1ffb700 1 -- 172.21.15.77:0/3126532442 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7cb0014070 con 0x7f7cc4100690 2023-12-01T01:16:55.187 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.681+0000 7f7cc8cae700 1 --2- 172.21.15.77:0/3126532442 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7cb40a05b0 0x7f7cb40a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:55.187 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.682+0000 7f7cc1ffb700 1 --2- 172.21.15.77:0/3126532442 >> [v2:172.21.15.203:6816/1161158776,v1:172.21.15.203:6817/1161158776] conn(0x7f7cb40a9d90 0x7f7cb40ac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:55.188 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.682+0000 7f7cc1ffb700 1 -- 172.21.15.77:0/3126532442 --> [v2:172.21.15.203:6816/1161158776,v1:172.21.15.203:6817/1161158776] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7cb40ac860 con 0x7f7cb40a9d90 2023-12-01T01:16:55.188 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.682+0000 7f7cc1ffb700 1 -- 172.21.15.77:0/3126532442 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7cb00cc2b0 con 0x7f7cc4100690 2023-12-01T01:16:55.188 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.682+0000 7f7cc8cae700 1 --2- 172.21.15.77:0/3126532442 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7cb40a05b0 0x7f7cb40a2a70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8005950 tx=0x7f7cb800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:55.188 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.682+0000 7f7cc94af700 1 --2- 172.21.15.77:0/3126532442 >> [v2:172.21.15.203:6816/1161158776,v1:172.21.15.203:6817/1161158776] conn(0x7f7cb40a9d90 0x7f7cb40ac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:55.188 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.682+0000 7f7cc94af700 1 --2- 172.21.15.77:0/3126532442 >> [v2:172.21.15.203:6816/1161158776,v1:172.21.15.203:6817/1161158776] conn(0x7f7cb40a9d90 0x7f7cb40ac1b0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:55.188 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.683+0000 7f7cc1ffb700 1 -- 172.21.15.77:0/3126532442 <== osd.8 v2:172.21.15.203:6816/1161158776 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7cb40ac860 con 0x7f7cb40a9d90 2023-12-01T01:16:55.188 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.699+0000 7f7ccaf12700 1 -- 172.21.15.77:0/3126532442 --> [v2:172.21.15.203:6816/1161158776,v1:172.21.15.203:6817/1161158776] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7cc410fee0 con 0x7f7cb40a9d90 2023-12-01T01:16:55.188 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.699+0000 7f7cc1ffb700 1 -- 172.21.15.77:0/3126532442 <== osd.8 v2:172.21.15.203:6816/1161158776 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f7cbc009b80 con 0x7f7cb40a9d90 2023-12-01T01:16:55.188 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.700+0000 7f7ccaf12700 1 -- 172.21.15.77:0/3126532442 >> [v2:172.21.15.203:6816/1161158776,v1:172.21.15.203:6817/1161158776] conn(0x7f7cb40a9d90 msgr2=0x7f7cb40ac1b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:55.188 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.700+0000 7f7ccaf12700 1 --2- 172.21.15.77:0/3126532442 >> [v2:172.21.15.203:6816/1161158776,v1:172.21.15.203:6817/1161158776] conn(0x7f7cb40a9d90 0x7f7cb40ac1b0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:55.188 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.700+0000 7f7ccaf12700 1 -- 172.21.15.77:0/3126532442 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7cb40a05b0 msgr2=0x7f7cb40a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:55.188 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.700+0000 7f7ccaf12700 1 --2- 172.21.15.77:0/3126532442 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7cb40a05b0 0x7f7cb40a2a70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8005950 tx=0x7f7cb800b410 comp rx=0 tx=0).stop 2023-12-01T01:16:55.188 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.700+0000 7f7ccaf12700 1 -- 172.21.15.77:0/3126532442 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cc4100690 msgr2=0x7f7cc4198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:55.188 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.700+0000 7f7ccaf12700 1 --2- 172.21.15.77:0/3126532442 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cc4100690 0x7f7cc4198330 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7cb000e9a0 tx=0x7f7cb000ecb0 comp rx=0 tx=0).stop 2023-12-01T01:16:55.188 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.700+0000 7f7cc94af700 1 -- 172.21.15.77:0/3126532442 reap_dead start 2023-12-01T01:16:55.189 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.700+0000 7f7ccaf12700 1 -- 172.21.15.77:0/3126532442 shutdown_connections 2023-12-01T01:16:55.189 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.700+0000 7f7ccaf12700 1 -- 172.21.15.77:0/3126532442 >> 172.21.15.77:0/3126532442 conn(0x7f7cc4076d90 msgr2=0x7f7cc40fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:55.189 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.700+0000 7f7ccaf12700 1 -- 172.21.15.77:0/3126532442 shutdown_connections 2023-12-01T01:16:55.189 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:53.700+0000 7f7ccaf12700 1 -- 172.21.15.77:0/3126532442 wait complete. 2023-12-01T01:16:55.192 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.184+0000 7fba66b9c700 1 -- 172.21.15.77:0/1043399582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba60105d80 msgr2=0x7fba60106160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:55.192 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.184+0000 7fba66b9c700 1 --2- 172.21.15.77:0/1043399582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba60105d80 0x7fba60106160 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fba48009990 tx=0x7fba48009ca0 comp rx=0 tx=0).stop 2023-12-01T01:16:55.192 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.184+0000 7fba66b9c700 1 -- 172.21.15.77:0/1043399582 shutdown_connections 2023-12-01T01:16:55.192 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.184+0000 7fba66b9c700 1 --2- 172.21.15.77:0/1043399582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba60100690 0x7fba6010cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:55.192 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.184+0000 7fba66b9c700 1 --2- 172.21.15.77:0/1043399582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba60105d80 0x7fba60106160 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:55.192 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.184+0000 7fba66b9c700 1 --2- 172.21.15.77:0/1043399582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fba600ffcd0 0x7fba60100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:55.192 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.184+0000 7fba66b9c700 1 -- 172.21.15.77:0/1043399582 >> 172.21.15.77:0/1043399582 conn(0x7fba60076d90 msgr2=0x7fba600771a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:55.192 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.184+0000 7fba66b9c700 1 -- 172.21.15.77:0/1043399582 shutdown_connections 2023-12-01T01:16:55.192 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.184+0000 7fba66b9c700 1 -- 172.21.15.77:0/1043399582 wait complete. 2023-12-01T01:16:55.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.185+0000 7fba66b9c700 1 Processor -- start 2023-12-01T01:16:55.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.185+0000 7fba66b9c700 1 -- start start 2023-12-01T01:16:55.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.185+0000 7fba66b9c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fba600ffcd0 0x7fba60101980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:55.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.185+0000 7fba66b9c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba60100690 0x7fba60101ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:55.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.185+0000 7fba66b9c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba60105d80 0x7fba60102440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:55.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.185+0000 7fba66b9c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba601a52c0 con 0x7fba60100690 2023-12-01T01:16:55.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.185+0000 7fba66b9c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba601a5430 con 0x7fba600ffcd0 2023-12-01T01:16:55.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.185+0000 7fba66b9c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba601a55f0 con 0x7fba60105d80 2023-12-01T01:16:55.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.185+0000 7fba5ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba60100690 0x7fba60101ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:55.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.185+0000 7fba5ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba60100690 0x7fba60101ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57930/0 (socket says 172.21.15.77:57930) 2023-12-01T01:16:55.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.185+0000 7fba5ffff700 1 -- 172.21.15.77:0/1721511066 learned_addr learned my addr 172.21.15.77:0/1721511066 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:55.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.185+0000 7fba64938700 1 --2- 172.21.15.77:0/1721511066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fba600ffcd0 0x7fba60101980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:55.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.185+0000 7fba65139700 1 --2- 172.21.15.77:0/1721511066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba60105d80 0x7fba60102440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:55.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.185+0000 7fba5ffff700 1 -- 172.21.15.77:0/1721511066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba60105d80 msgr2=0x7fba60102440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:55.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.185+0000 7fba5ffff700 1 --2- 172.21.15.77:0/1721511066 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba60105d80 0x7fba60102440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:55.194 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.185+0000 7fba5ffff700 1 -- 172.21.15.77:0/1721511066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fba600ffcd0 msgr2=0x7fba60101980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:55.194 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.185+0000 7fba5ffff700 1 --2- 172.21.15.77:0/1721511066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fba600ffcd0 0x7fba60101980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:55.194 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.186+0000 7fba5ffff700 1 -- 172.21.15.77:0/1721511066 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba480095f0 con 0x7fba60100690 2023-12-01T01:16:55.194 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.186+0000 7fba64938700 1 --2- 172.21.15.77:0/1721511066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fba600ffcd0 0x7fba60101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:16:55.194 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.186+0000 7fba5ffff700 1 --2- 172.21.15.77:0/1721511066 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba60100690 0x7fba60101ec0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fba5000d950 tx=0x7fba5000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:55.194 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.186+0000 7fba5dffb700 1 -- 172.21.15.77:0/1721511066 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba50009b40 con 0x7fba60100690 2023-12-01T01:16:55.194 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.186+0000 7fba5dffb700 1 -- 172.21.15.77:0/1721511066 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba50009ca0 con 0x7fba60100690 2023-12-01T01:16:55.194 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.186+0000 7fba5dffb700 1 -- 172.21.15.77:0/1721511066 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5000f620 con 0x7fba60100690 2023-12-01T01:16:55.194 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.186+0000 7fba66b9c700 1 -- 172.21.15.77:0/1721511066 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba601a58d0 con 0x7fba60100690 2023-12-01T01:16:55.194 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.186+0000 7fba66b9c700 1 -- 172.21.15.77:0/1721511066 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba601a5d80 con 0x7fba60100690 2023-12-01T01:16:55.194 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.187+0000 7fba66b9c700 1 -- 172.21.15.77:0/1721511066 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fba6010fee0 con 0x7fba60100690 2023-12-01T01:16:55.194 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.188+0000 7fba5dffb700 1 -- 172.21.15.77:0/1721511066 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fba50010460 con 0x7fba60100690 2023-12-01T01:16:55.194 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.188+0000 7fba5dffb700 1 --2- 172.21.15.77:0/1721511066 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fba4c0a0630 0x7fba4c0a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:55.194 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.188+0000 7fba64938700 1 --2- 172.21.15.77:0/1721511066 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fba4c0a0630 0x7fba4c0a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:55.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.189+0000 7fba5dffb700 1 -- 172.21.15.77:0/1721511066 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fba500cbcc0 con 0x7fba60100690 2023-12-01T01:16:55.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.189+0000 7fba5dffb700 1 --2- 172.21.15.77:0/1721511066 >> [v2:172.21.15.137:6816/3108330766,v1:172.21.15.137:6817/3108330766] conn(0x7fba4c0a9e00 0x7fba4c0ac220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:55.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.189+0000 7fba5dffb700 1 -- 172.21.15.77:0/1721511066 --> [v2:172.21.15.137:6816/3108330766,v1:172.21.15.137:6817/3108330766] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fba4c0ac8d0 con 0x7fba4c0a9e00 2023-12-01T01:16:55.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.189+0000 7fba5dffb700 1 -- 172.21.15.77:0/1721511066 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba500cf020 con 0x7fba60100690 2023-12-01T01:16:55.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.189+0000 7fba64938700 1 --2- 172.21.15.77:0/1721511066 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fba4c0a0630 0x7fba4c0a2af0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fba48006010 tx=0x7fba4800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:55.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.189+0000 7fba65139700 1 --2- 172.21.15.77:0/1721511066 >> [v2:172.21.15.137:6816/3108330766,v1:172.21.15.137:6817/3108330766] conn(0x7fba4c0a9e00 0x7fba4c0ac220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:55.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.189+0000 7fba65139700 1 --2- 172.21.15.77:0/1721511066 >> [v2:172.21.15.137:6816/3108330766,v1:172.21.15.137:6817/3108330766] conn(0x7fba4c0a9e00 0x7fba4c0ac220 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:55.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.190+0000 7fba5dffb700 1 -- 172.21.15.77:0/1721511066 <== osd.7 v2:172.21.15.137:6816/3108330766 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fba4c0ac8d0 con 0x7fba4c0a9e00 2023-12-01T01:16:55.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.206+0000 7fba66b9c700 1 -- 172.21.15.77:0/1721511066 --> [v2:172.21.15.137:6816/3108330766,v1:172.21.15.137:6817/3108330766] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fba6004ea90 con 0x7fba4c0a9e00 2023-12-01T01:16:55.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.207+0000 7fba5dffb700 1 -- 172.21.15.77:0/1721511066 <== osd.7 v2:172.21.15.137:6816/3108330766 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fba6004ea90 con 0x7fba4c0a9e00 2023-12-01T01:16:55.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.207+0000 7fba66b9c700 1 -- 172.21.15.77:0/1721511066 >> [v2:172.21.15.137:6816/3108330766,v1:172.21.15.137:6817/3108330766] conn(0x7fba4c0a9e00 msgr2=0x7fba4c0ac220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:55.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.207+0000 7fba66b9c700 1 --2- 172.21.15.77:0/1721511066 >> [v2:172.21.15.137:6816/3108330766,v1:172.21.15.137:6817/3108330766] conn(0x7fba4c0a9e00 0x7fba4c0ac220 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:55.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.207+0000 7fba66b9c700 1 -- 172.21.15.77:0/1721511066 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fba4c0a0630 msgr2=0x7fba4c0a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:55.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.207+0000 7fba66b9c700 1 --2- 172.21.15.77:0/1721511066 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fba4c0a0630 0x7fba4c0a2af0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fba48006010 tx=0x7fba4800b540 comp rx=0 tx=0).stop 2023-12-01T01:16:55.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.207+0000 7fba66b9c700 1 -- 172.21.15.77:0/1721511066 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba60100690 msgr2=0x7fba60101ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:55.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.207+0000 7fba66b9c700 1 --2- 172.21.15.77:0/1721511066 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba60100690 0x7fba60101ec0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fba5000d950 tx=0x7fba5000dd10 comp rx=0 tx=0).stop 2023-12-01T01:16:55.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.207+0000 7fba65139700 1 -- 172.21.15.77:0/1721511066 reap_dead start 2023-12-01T01:16:55.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.207+0000 7fba66b9c700 1 -- 172.21.15.77:0/1721511066 shutdown_connections 2023-12-01T01:16:55.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.207+0000 7fba66b9c700 1 -- 172.21.15.77:0/1721511066 >> 172.21.15.77:0/1721511066 conn(0x7fba60076d90 msgr2=0x7fba600fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:55.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.207+0000 7fba66b9c700 1 -- 172.21.15.77:0/1721511066 shutdown_connections 2023-12-01T01:16:55.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:55.207+0000 7fba66b9c700 1 -- 172.21.15.77:0/1721511066 wait complete. 2023-12-01T01:16:55.274 INFO:teuthology.orchestra.run.smithi077.stdout:158913789994 2023-12-01T01:16:55.275 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.5 2023-12-01T01:16:55.323 INFO:teuthology.orchestra.run.smithi077.stdout:330712481809 2023-12-01T01:16:55.323 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.11 2023-12-01T01:16:55.413 INFO:teuthology.orchestra.run.smithi077.stdout:34359738433 2023-12-01T01:16:55.413 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.0 2023-12-01T01:16:55.893 INFO:teuthology.orchestra.run.smithi077.stdout:214748364834 2023-12-01T01:16:55.893 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.7 2023-12-01T01:16:55.998 INFO:teuthology.orchestra.run.smithi077.stdout:244813135901 2023-12-01T01:16:55.998 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.8 2023-12-01T01:16:56.074 INFO:teuthology.orchestra.run.smithi077.stdout:55834574907 2023-12-01T01:16:56.074 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.1 2023-12-01T01:16:56.127 INFO:teuthology.orchestra.run.smithi077.stdout:77309411383 2023-12-01T01:16:56.127 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.2 2023-12-01T01:16:56.495 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:16:56.908 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:16:56.910 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:16:56.913 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:16:57.065 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:16:57.345 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:16:57.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:57 smithi077 ceph-mon[98867]: pgmap v268: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 10 op/s 2023-12-01T01:16:57.878 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:16:58.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:57 smithi203 ceph-mon[103581]: pgmap v268: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 10 op/s 2023-12-01T01:16:58.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:57 smithi137 ceph-mon[103312]: pgmap v268: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 10 op/s 2023-12-01T01:16:58.680 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.575+0000 7fae67b00700 1 -- 172.21.15.77:0/162154425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae60100690 msgr2=0x7fae6010cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:58.681 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.575+0000 7fae67b00700 1 --2- 172.21.15.77:0/162154425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae60100690 0x7fae6010cb30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fae5800b0a0 tx=0x7fae5800b3b0 comp rx=0 tx=0).stop 2023-12-01T01:16:58.681 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.575+0000 7fae67b00700 1 -- 172.21.15.77:0/162154425 shutdown_connections 2023-12-01T01:16:58.681 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.575+0000 7fae67b00700 1 --2- 172.21.15.77:0/162154425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae60100690 0x7fae6010cb30 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:58.681 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.575+0000 7fae67b00700 1 --2- 172.21.15.77:0/162154425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae60105d80 0x7fae60106160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:58.681 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.575+0000 7fae67b00700 1 --2- 172.21.15.77:0/162154425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae600ffcd0 0x7fae60100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:58.681 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.575+0000 7fae67b00700 1 -- 172.21.15.77:0/162154425 >> 172.21.15.77:0/162154425 conn(0x7fae60076d90 msgr2=0x7fae600771a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:58.681 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.575+0000 7fae67b00700 1 -- 172.21.15.77:0/162154425 shutdown_connections 2023-12-01T01:16:58.681 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.576+0000 7fae67b00700 1 -- 172.21.15.77:0/162154425 wait complete. 2023-12-01T01:16:58.681 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.576+0000 7fae67b00700 1 Processor -- start 2023-12-01T01:16:58.681 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.576+0000 7fae67b00700 1 -- start start 2023-12-01T01:16:58.681 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.576+0000 7fae67b00700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae600ffcd0 0x7fae60197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:58.682 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.576+0000 7fae67b00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae60100690 0x7fae60198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:58.682 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.576+0000 7fae67b00700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae60105d80 0x7fae601a2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:58.682 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.576+0000 7fae67b00700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae60198c70 con 0x7fae600ffcd0 2023-12-01T01:16:58.682 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.576+0000 7fae67b00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae601a33b0 con 0x7fae60100690 2023-12-01T01:16:58.682 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.576+0000 7fae67b00700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae601a3570 con 0x7fae60105d80 2023-12-01T01:16:58.682 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.577+0000 7fae6589c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae600ffcd0 0x7fae60197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:58.682 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.577+0000 7fae6589c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae600ffcd0 0x7fae60197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57944/0 (socket says 172.21.15.77:57944) 2023-12-01T01:16:58.682 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.577+0000 7fae6589c700 1 -- 172.21.15.77:0/1556079898 learned_addr learned my addr 172.21.15.77:0/1556079898 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:16:58.682 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.577+0000 7fae6609d700 1 --2- 172.21.15.77:0/1556079898 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae60105d80 0x7fae601a2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:58.682 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.577+0000 7fae6509b700 1 --2- 172.21.15.77:0/1556079898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae60100690 0x7fae60198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:58.682 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.577+0000 7fae6589c700 1 -- 172.21.15.77:0/1556079898 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae60105d80 msgr2=0x7fae601a2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:58.682 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.577+0000 7fae6589c700 1 --2- 172.21.15.77:0/1556079898 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae60105d80 0x7fae601a2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:58.682 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.577+0000 7fae6589c700 1 -- 172.21.15.77:0/1556079898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae60100690 msgr2=0x7fae60198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:58.682 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.577+0000 7fae6589c700 1 --2- 172.21.15.77:0/1556079898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae60100690 0x7fae60198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:58.682 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.577+0000 7fae6589c700 1 -- 172.21.15.77:0/1556079898 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae58009d00 con 0x7fae600ffcd0 2023-12-01T01:16:58.683 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.577+0000 7fae6589c700 1 --2- 172.21.15.77:0/1556079898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae600ffcd0 0x7fae60197df0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fae5000d950 tx=0x7fae5000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:58.683 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.577+0000 7fae56ffd700 1 -- 172.21.15.77:0/1556079898 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5000f890 con 0x7fae600ffcd0 2023-12-01T01:16:58.683 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.577+0000 7fae67b00700 1 -- 172.21.15.77:0/1556079898 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae601a3850 con 0x7fae600ffcd0 2023-12-01T01:16:58.683 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.577+0000 7fae56ffd700 1 -- 172.21.15.77:0/1556079898 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae50010460 con 0x7fae600ffcd0 2023-12-01T01:16:58.683 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.577+0000 7fae56ffd700 1 -- 172.21.15.77:0/1556079898 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5000e610 con 0x7fae600ffcd0 2023-12-01T01:16:58.683 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.577+0000 7fae67b00700 1 -- 172.21.15.77:0/1556079898 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae601a3d50 con 0x7fae600ffcd0 2023-12-01T01:16:58.683 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.578+0000 7fae67b00700 1 -- 172.21.15.77:0/1556079898 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae6004ea90 con 0x7fae600ffcd0 2023-12-01T01:16:58.683 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.579+0000 7fae56ffd700 1 -- 172.21.15.77:0/1556079898 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fae500108d0 con 0x7fae600ffcd0 2023-12-01T01:16:58.683 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.580+0000 7fae56ffd700 1 --2- 172.21.15.77:0/1556079898 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fae4c0a05b0 0x7fae4c0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:16:58.683 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.580+0000 7fae6509b700 1 --2- 172.21.15.77:0/1556079898 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fae4c0a05b0 0x7fae4c0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:16:58.683 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.580+0000 7fae56ffd700 1 -- 172.21.15.77:0/1556079898 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fae500cba50 con 0x7fae600ffcd0 2023-12-01T01:16:58.683 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.580+0000 7fae6509b700 1 --2- 172.21.15.77:0/1556079898 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fae4c0a05b0 0x7fae4c0a2a70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fae60069120 tx=0x7fae5c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:16:58.683 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.585+0000 7fae56ffd700 1 -- 172.21.15.77:0/1556079898 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fae50095640 con 0x7fae600ffcd0 2023-12-01T01:16:58.684 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:16:58.710 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:16:58.838 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.837+0000 7fae67b00700 1 -- 172.21.15.77:0/1556079898 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fae601a4070 con 0x7fae600ffcd0 2023-12-01T01:16:58.838 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.837+0000 7fae56ffd700 1 -- 172.21.15.77:0/1556079898 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fae50098c60 con 0x7fae600ffcd0 2023-12-01T01:16:58.839 INFO:teuthology.orchestra.run.smithi077.stdout:107374182452 2023-12-01T01:16:58.841 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.839+0000 7fae67b00700 1 -- 172.21.15.77:0/1556079898 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fae4c0a05b0 msgr2=0x7fae4c0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:58.841 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.839+0000 7fae67b00700 1 --2- 172.21.15.77:0/1556079898 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fae4c0a05b0 0x7fae4c0a2a70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fae60069120 tx=0x7fae5c0073d0 comp rx=0 tx=0).stop 2023-12-01T01:16:58.841 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.839+0000 7fae67b00700 1 -- 172.21.15.77:0/1556079898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae600ffcd0 msgr2=0x7fae60197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:16:58.841 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.839+0000 7fae67b00700 1 --2- 172.21.15.77:0/1556079898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae600ffcd0 0x7fae60197df0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fae5000d950 tx=0x7fae5000dc60 comp rx=0 tx=0).stop 2023-12-01T01:16:58.841 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.839+0000 7fae67b00700 1 -- 172.21.15.77:0/1556079898 shutdown_connections 2023-12-01T01:16:58.841 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.839+0000 7fae67b00700 1 --2- 172.21.15.77:0/1556079898 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fae4c0a05b0 0x7fae4c0a2a70 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:58.841 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.839+0000 7fae67b00700 1 --2- 172.21.15.77:0/1556079898 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae60105d80 0x7fae601a2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:58.841 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.839+0000 7fae67b00700 1 --2- 172.21.15.77:0/1556079898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae600ffcd0 0x7fae60197df0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:58.841 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.839+0000 7fae67b00700 1 --2- 172.21.15.77:0/1556079898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae60100690 0x7fae60198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:16:58.841 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.839+0000 7fae67b00700 1 -- 172.21.15.77:0/1556079898 >> 172.21.15.77:0/1556079898 conn(0x7fae60076d90 msgr2=0x7fae600fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:16:58.842 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.840+0000 7fae67b00700 1 -- 172.21.15.77:0/1556079898 shutdown_connections 2023-12-01T01:16:58.842 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:16:58.840+0000 7fae67b00700 1 -- 172.21.15.77:0/1556079898 wait complete. 2023-12-01T01:16:59.698 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:16:59.832 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:16:59.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:59 smithi077 ceph-mon[98867]: pgmap v269: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 426 B/s wr, 11 op/s 2023-12-01T01:16:59.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:16:59 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/1556079898' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-12-01T01:17:00.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:59 smithi203 ceph-mon[103581]: pgmap v269: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 426 B/s wr, 11 op/s 2023-12-01T01:17:00.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:16:59 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/1556079898' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-12-01T01:17:00.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:59 smithi137 ceph-mon[103312]: pgmap v269: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 426 B/s wr, 11 op/s 2023-12-01T01:17:00.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:16:59 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/1556079898' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-12-01T01:17:00.155 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182451 got 107374182452 for osd.3 2023-12-01T01:17:00.155 DEBUG:teuthology.parallel:result is None 2023-12-01T01:17:00.487 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:17:00.999 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.609+0000 7fa4e135d700 1 -- 172.21.15.77:0/4056271875 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4dc0ffcd0 msgr2=0x7fa4dc100150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:01.000 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.609+0000 7fa4e135d700 1 --2- 172.21.15.77:0/4056271875 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4dc0ffcd0 0x7fa4dc100150 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0009990 tx=0x7fa4d0009ca0 comp rx=0 tx=0).stop 2023-12-01T01:17:01.000 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.609+0000 7fa4e135d700 1 -- 172.21.15.77:0/4056271875 shutdown_connections 2023-12-01T01:17:01.000 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.610+0000 7fa4e135d700 1 --2- 172.21.15.77:0/4056271875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4dc100690 0x7fa4dc10cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:01.000 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.610+0000 7fa4e135d700 1 --2- 172.21.15.77:0/4056271875 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4dc105d80 0x7fa4dc106160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:01.000 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.610+0000 7fa4e135d700 1 --2- 172.21.15.77:0/4056271875 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4dc0ffcd0 0x7fa4dc100150 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:01.000 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.610+0000 7fa4e135d700 1 -- 172.21.15.77:0/4056271875 >> 172.21.15.77:0/4056271875 conn(0x7fa4dc076d90 msgr2=0x7fa4dc0771a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:01.000 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.610+0000 7fa4e135d700 1 -- 172.21.15.77:0/4056271875 shutdown_connections 2023-12-01T01:17:01.000 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.610+0000 7fa4e135d700 1 -- 172.21.15.77:0/4056271875 wait complete. 2023-12-01T01:17:01.000 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.611+0000 7fa4e135d700 1 Processor -- start 2023-12-01T01:17:01.000 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.611+0000 7fa4e135d700 1 -- start start 2023-12-01T01:17:01.000 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.611+0000 7fa4e135d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4dc0ffcd0 0x7fa4dc197ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:01.001 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.611+0000 7fa4e135d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4dc100690 0x7fa4dc198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:01.001 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.611+0000 7fa4e135d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4dc105d80 0x7fa4dc1a2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:01.001 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.611+0000 7fa4e135d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4dc1a32a0 con 0x7fa4dc105d80 2023-12-01T01:17:01.001 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.611+0000 7fa4e135d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4dc1a33e0 con 0x7fa4dc100690 2023-12-01T01:17:01.001 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.611+0000 7fa4e135d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4dc1a3520 con 0x7fa4dc0ffcd0 2023-12-01T01:17:01.001 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.611+0000 7fa4db7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4dc105d80 0x7fa4dc1a2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:01.001 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.611+0000 7fa4db7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4dc105d80 0x7fa4dc1a2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57962/0 (socket says 172.21.15.77:57962) 2023-12-01T01:17:01.001 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.611+0000 7fa4db7fe700 1 -- 172.21.15.77:0/2094191636 learned_addr learned my addr 172.21.15.77:0/2094191636 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:01.001 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.611+0000 7fa4daffd700 1 --2- 172.21.15.77:0/2094191636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4dc0ffcd0 0x7fa4dc197ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:01.001 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.611+0000 7fa4da7fc700 1 --2- 172.21.15.77:0/2094191636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4dc100690 0x7fa4dc198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:01.001 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.612+0000 7fa4db7fe700 1 -- 172.21.15.77:0/2094191636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4dc0ffcd0 msgr2=0x7fa4dc197ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:01.001 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.612+0000 7fa4db7fe700 1 --2- 172.21.15.77:0/2094191636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4dc0ffcd0 0x7fa4dc197ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:01.001 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.612+0000 7fa4db7fe700 1 -- 172.21.15.77:0/2094191636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4dc100690 msgr2=0x7fa4dc198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:01.001 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.612+0000 7fa4db7fe700 1 --2- 172.21.15.77:0/2094191636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4dc100690 0x7fa4dc198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:01.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.612+0000 7fa4db7fe700 1 -- 172.21.15.77:0/2094191636 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d00095f0 con 0x7fa4dc105d80 2023-12-01T01:17:01.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.612+0000 7fa4db7fe700 1 --2- 172.21.15.77:0/2094191636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4dc105d80 0x7fa4dc1a2d60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4007f10 tx=0x7fa4d400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:01.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.612+0000 7fa4c3fff700 1 -- 172.21.15.77:0/2094191636 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4017070 con 0x7fa4dc105d80 2023-12-01T01:17:01.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.612+0000 7fa4e135d700 1 -- 172.21.15.77:0/2094191636 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4dc1a3780 con 0x7fa4dc105d80 2023-12-01T01:17:01.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.612+0000 7fa4c3fff700 1 -- 172.21.15.77:0/2094191636 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4d400f040 con 0x7fa4dc105d80 2023-12-01T01:17:01.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.612+0000 7fa4c3fff700 1 -- 172.21.15.77:0/2094191636 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d40054b0 con 0x7fa4dc105d80 2023-12-01T01:17:01.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.612+0000 7fa4e135d700 1 -- 172.21.15.77:0/2094191636 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4dc1a3c50 con 0x7fa4dc105d80 2023-12-01T01:17:01.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.613+0000 7fa4e135d700 1 -- 172.21.15.77:0/2094191636 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4dc04ea90 con 0x7fa4dc105d80 2023-12-01T01:17:01.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.614+0000 7fa4c3fff700 1 -- 172.21.15.77:0/2094191636 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fa4d4004ad0 con 0x7fa4dc105d80 2023-12-01T01:17:01.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.615+0000 7fa4c3fff700 1 --2- 172.21.15.77:0/2094191636 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa4c40a05b0 0x7fa4c40a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:01.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.615+0000 7fa4daffd700 1 --2- 172.21.15.77:0/2094191636 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa4c40a05b0 0x7fa4c40a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:01.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.615+0000 7fa4c3fff700 1 -- 172.21.15.77:0/2094191636 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa4d40cb040 con 0x7fa4dc105d80 2023-12-01T01:17:01.003 INFO:teuthology.orchestra.run.smithi077.stdout:214748364835 2023-12-01T01:17:01.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.615+0000 7fa4daffd700 1 --2- 172.21.15.77:0/2094191636 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa4c40a05b0 0x7fa4c40a2a70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc009be0 tx=0x7fa4cc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:01.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.619+0000 7fa4c3fff700 1 -- 172.21.15.77:0/2094191636 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa4d4094c30 con 0x7fa4dc105d80 2023-12-01T01:17:01.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.848+0000 7fa4e135d700 1 -- 172.21.15.77:0/2094191636 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fa4dc1a3f10 con 0x7fa4dc105d80 2023-12-01T01:17:01.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.848+0000 7fa4c3fff700 1 -- 172.21.15.77:0/2094191636 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa4d4098250 con 0x7fa4dc105d80 2023-12-01T01:17:01.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.850+0000 7fa4e135d700 1 -- 172.21.15.77:0/2094191636 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa4c40a05b0 msgr2=0x7fa4c40a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:01.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.850+0000 7fa4e135d700 1 --2- 172.21.15.77:0/2094191636 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa4c40a05b0 0x7fa4c40a2a70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc009be0 tx=0x7fa4cc009260 comp rx=0 tx=0).stop 2023-12-01T01:17:01.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.850+0000 7fa4e135d700 1 -- 172.21.15.77:0/2094191636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4dc105d80 msgr2=0x7fa4dc1a2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:01.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.850+0000 7fa4e135d700 1 --2- 172.21.15.77:0/2094191636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4dc105d80 0x7fa4dc1a2d60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4007f10 tx=0x7fa4d400d3b0 comp rx=0 tx=0).stop 2023-12-01T01:17:01.004 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.850+0000 7fa4e135d700 1 -- 172.21.15.77:0/2094191636 shutdown_connections 2023-12-01T01:17:01.004 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.850+0000 7fa4e135d700 1 --2- 172.21.15.77:0/2094191636 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa4c40a05b0 0x7fa4c40a2a70 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:01.004 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.850+0000 7fa4e135d700 1 --2- 172.21.15.77:0/2094191636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa4dc105d80 0x7fa4dc1a2d60 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:01.004 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.850+0000 7fa4e135d700 1 --2- 172.21.15.77:0/2094191636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa4dc0ffcd0 0x7fa4dc197ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:01.004 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.850+0000 7fa4e135d700 1 --2- 172.21.15.77:0/2094191636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4dc100690 0x7fa4dc198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:01.004 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.850+0000 7fa4e135d700 1 -- 172.21.15.77:0/2094191636 >> 172.21.15.77:0/2094191636 conn(0x7fa4dc076d90 msgr2=0x7fa4dc0fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:01.004 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.851+0000 7fa4e135d700 1 -- 172.21.15.77:0/2094191636 shutdown_connections 2023-12-01T01:17:01.004 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:00.851+0000 7fa4e135d700 1 -- 172.21.15.77:0/2094191636 wait complete. 2023-12-01T01:17:01.716 INFO:tasks.cephadm.ceph_manager.ceph:need seq 214748364834 got 214748364835 for osd.7 2023-12-01T01:17:01.716 DEBUG:teuthology.parallel:result is None 2023-12-01T01:17:01.850 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:01 smithi077 ceph-mon[98867]: pgmap v270: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 10 op/s 2023-12-01T01:17:01.850 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:01 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/2094191636' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-12-01T01:17:02.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:01 smithi203 ceph-mon[103581]: pgmap v270: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 10 op/s 2023-12-01T01:17:02.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:01 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/2094191636' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-12-01T01:17:02.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:01 smithi137 ceph-mon[103312]: pgmap v270: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 10 op/s 2023-12-01T01:17:02.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:01 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/2094191636' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-12-01T01:17:02.185 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.149+0000 7f59580b3700 1 -- 172.21.15.77:0/2082256014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5950100690 msgr2=0x7f595010cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:02.185 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.149+0000 7f59580b3700 1 --2- 172.21.15.77:0/2082256014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5950100690 0x7f595010cb30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f594800b0a0 tx=0x7f594800b3b0 comp rx=0 tx=0).stop 2023-12-01T01:17:02.185 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.149+0000 7f59580b3700 1 -- 172.21.15.77:0/2082256014 shutdown_connections 2023-12-01T01:17:02.185 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.149+0000 7f59580b3700 1 --2- 172.21.15.77:0/2082256014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5950100690 0x7f595010cb30 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:02.185 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.149+0000 7f59580b3700 1 --2- 172.21.15.77:0/2082256014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5950105d80 0x7f5950106160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:02.185 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.149+0000 7f59580b3700 1 --2- 172.21.15.77:0/2082256014 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59500ffcd0 0x7f5950100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:02.185 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.149+0000 7f59580b3700 1 -- 172.21.15.77:0/2082256014 >> 172.21.15.77:0/2082256014 conn(0x7f5950076d90 msgr2=0x7f59500771a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:02.185 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.150+0000 7f59580b3700 1 -- 172.21.15.77:0/2082256014 shutdown_connections 2023-12-01T01:17:02.186 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.150+0000 7f59580b3700 1 -- 172.21.15.77:0/2082256014 wait complete. 2023-12-01T01:17:02.186 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.150+0000 7f59580b3700 1 Processor -- start 2023-12-01T01:17:02.186 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.150+0000 7f59580b3700 1 -- start start 2023-12-01T01:17:02.186 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.151+0000 7f59580b3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59500ffcd0 0x7f595006ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:02.186 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.151+0000 7f59580b3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5950100690 0x7f59500704b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:02.186 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.151+0000 7f59580b3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5950105d80 0x7f595006b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:02.186 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.151+0000 7f59580b3700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595006b780 con 0x7f59500ffcd0 2023-12-01T01:17:02.186 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.151+0000 7f59580b3700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595006b8c0 con 0x7f5950100690 2023-12-01T01:17:02.186 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.151+0000 7f59580b3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595006ba80 con 0x7f5950105d80 2023-12-01T01:17:02.186 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.151+0000 7f595564e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5950100690 0x7f59500704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:02.186 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.151+0000 7f595564e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5950100690 0x7f59500704b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:36894/0 (socket says 172.21.15.77:36894) 2023-12-01T01:17:02.186 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.151+0000 7f595564e700 1 -- 172.21.15.77:0/1659019410 learned_addr learned my addr 172.21.15.77:0/1659019410 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:02.186 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.151+0000 7f5956650700 1 --2- 172.21.15.77:0/1659019410 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5950105d80 0x7f595006b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:02.187 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.151+0000 7f595564e700 1 -- 172.21.15.77:0/1659019410 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5950105d80 msgr2=0x7f595006b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:02.187 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.151+0000 7f595564e700 1 --2- 172.21.15.77:0/1659019410 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5950105d80 0x7f595006b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:02.189 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.151+0000 7f595564e700 1 -- 172.21.15.77:0/1659019410 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59500ffcd0 msgr2=0x7f595006ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:02.189 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.151+0000 7f5955e4f700 1 --2- 172.21.15.77:0/1659019410 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59500ffcd0 0x7f595006ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:02.189 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.151+0000 7f595564e700 1 --2- 172.21.15.77:0/1659019410 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59500ffcd0 0x7f595006ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:02.189 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.151+0000 7f595564e700 1 -- 172.21.15.77:0/1659019410 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5948009d00 con 0x7f5950100690 2023-12-01T01:17:02.189 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.151+0000 7f5955e4f700 1 --2- 172.21.15.77:0/1659019410 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59500ffcd0 0x7f595006ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:17:02.189 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.152+0000 7f595564e700 1 --2- 172.21.15.77:0/1659019410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5950100690 0x7f59500704b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f594000d980 tx=0x7f594000dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:02.189 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.152+0000 7f5946ffd700 1 -- 172.21.15.77:0/1659019410 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5940004220 con 0x7f5950100690 2023-12-01T01:17:02.189 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.152+0000 7f59580b3700 1 -- 172.21.15.77:0/1659019410 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f595006bd60 con 0x7f5950100690 2023-12-01T01:17:02.189 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.152+0000 7f5946ffd700 1 -- 172.21.15.77:0/1659019410 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5940007640 con 0x7f5950100690 2023-12-01T01:17:02.189 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.152+0000 7f5946ffd700 1 -- 172.21.15.77:0/1659019410 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5940005610 con 0x7f5950100690 2023-12-01T01:17:02.190 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.152+0000 7f59580b3700 1 -- 172.21.15.77:0/1659019410 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f595006eeb0 con 0x7f5950100690 2023-12-01T01:17:02.190 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.153+0000 7f59580b3700 1 -- 172.21.15.77:0/1659019410 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f595004ea90 con 0x7f5950100690 2023-12-01T01:17:02.190 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.154+0000 7f5946ffd700 1 -- 172.21.15.77:0/1659019410 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f5940004750 con 0x7f5950100690 2023-12-01T01:17:02.190 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.155+0000 7f5946ffd700 1 --2- 172.21.15.77:0/1659019410 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f593c0a05b0 0x7f593c0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:02.190 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.155+0000 7f5955e4f700 1 --2- 172.21.15.77:0/1659019410 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f593c0a05b0 0x7f593c0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:02.190 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.155+0000 7f5946ffd700 1 -- 172.21.15.77:0/1659019410 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f59400cbd30 con 0x7f5950100690 2023-12-01T01:17:02.190 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.155+0000 7f5955e4f700 1 --2- 172.21.15.77:0/1659019410 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f593c0a05b0 0x7f593c0a2a70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f594c009b50 tx=0x7f594c009550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:02.190 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.158+0000 7f5946ffd700 1 -- 172.21.15.77:0/1659019410 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5940095920 con 0x7f5950100690 2023-12-01T01:17:02.363 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.362+0000 7f59580b3700 1 -- 172.21.15.77:0/1659019410 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f595006f170 con 0x7f5950100690 2023-12-01T01:17:02.364 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.363+0000 7f5946ffd700 1 -- 172.21.15.77:0/1659019410 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5940016090 con 0x7f5950100690 2023-12-01T01:17:02.364 INFO:teuthology.orchestra.run.smithi077.stdout:274877906971 2023-12-01T01:17:02.366 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.365+0000 7f59580b3700 1 -- 172.21.15.77:0/1659019410 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f593c0a05b0 msgr2=0x7f593c0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:02.366 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.365+0000 7f59580b3700 1 --2- 172.21.15.77:0/1659019410 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f593c0a05b0 0x7f593c0a2a70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f594c009b50 tx=0x7f594c009550 comp rx=0 tx=0).stop 2023-12-01T01:17:02.366 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.365+0000 7f59580b3700 1 -- 172.21.15.77:0/1659019410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5950100690 msgr2=0x7f59500704b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:02.366 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.365+0000 7f59580b3700 1 --2- 172.21.15.77:0/1659019410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5950100690 0x7f59500704b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f594000d980 tx=0x7f594000dc90 comp rx=0 tx=0).stop 2023-12-01T01:17:02.366 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.365+0000 7f59580b3700 1 -- 172.21.15.77:0/1659019410 shutdown_connections 2023-12-01T01:17:02.366 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.365+0000 7f59580b3700 1 --2- 172.21.15.77:0/1659019410 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f593c0a05b0 0x7f593c0a2a70 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:02.366 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.365+0000 7f59580b3700 1 --2- 172.21.15.77:0/1659019410 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5950105d80 0x7f595006b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:02.367 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.365+0000 7f59580b3700 1 --2- 172.21.15.77:0/1659019410 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59500ffcd0 0x7f595006ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:02.367 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.365+0000 7f59580b3700 1 --2- 172.21.15.77:0/1659019410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5950100690 0x7f59500704b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:02.367 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.365+0000 7f59580b3700 1 -- 172.21.15.77:0/1659019410 >> 172.21.15.77:0/1659019410 conn(0x7f5950076d90 msgr2=0x7f59500fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:02.367 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.365+0000 7f59580b3700 1 -- 172.21.15.77:0/1659019410 shutdown_connections 2023-12-01T01:17:02.367 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:02.365+0000 7f59580b3700 1 -- 172.21.15.77:0/1659019410 wait complete. 2023-12-01T01:17:02.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:02 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/1659019410' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-12-01T01:17:03.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:02 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/1659019410' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-12-01T01:17:03.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:02 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/1659019410' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-12-01T01:17:03.247 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.244+0000 7f076e88f700 1 -- 172.21.15.77:0/3275395626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07681050d0 msgr2=0x7f076810cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:03.247 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.244+0000 7f076e88f700 1 --2- 172.21.15.77:0/3275395626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07681050d0 0x7f076810cc50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f076000b0a0 tx=0x7f076000b3b0 comp rx=0 tx=0).stop 2023-12-01T01:17:03.247 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.244+0000 7f076e88f700 1 -- 172.21.15.77:0/3275395626 shutdown_connections 2023-12-01T01:17:03.247 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.244+0000 7f076e88f700 1 --2- 172.21.15.77:0/3275395626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07681050d0 0x7f076810cc50 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:03.247 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.244+0000 7f076e88f700 1 --2- 172.21.15.77:0/3275395626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0768100180 0x7f0768100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:03.248 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.244+0000 7f076e88f700 1 --2- 172.21.15.77:0/3275395626 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0768100b30 0x7f07681049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:03.248 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.244+0000 7f076e88f700 1 -- 172.21.15.77:0/3275395626 >> 172.21.15.77:0/3275395626 conn(0x7f0768076bb0 msgr2=0x7f0768076fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:03.248 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.244+0000 7f076e88f700 1 -- 172.21.15.77:0/3275395626 shutdown_connections 2023-12-01T01:17:03.248 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.244+0000 7f076e88f700 1 -- 172.21.15.77:0/3275395626 wait complete. 2023-12-01T01:17:03.248 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.245+0000 7f076e88f700 1 Processor -- start 2023-12-01T01:17:03.248 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.245+0000 7f076e88f700 1 -- start start 2023-12-01T01:17:03.248 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.245+0000 7f076e88f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0768100180 0x7f0768197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:03.248 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.245+0000 7f076e88f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0768100b30 0x7f0768197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:03.248 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.245+0000 7f076e88f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07681050d0 0x7f076819b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:03.248 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.245+0000 7f076e88f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076819bfe0 con 0x7f0768100b30 2023-12-01T01:17:03.248 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.245+0000 7f076e88f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076819c150 con 0x7f07681050d0 2023-12-01T01:17:03.248 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.245+0000 7f076e88f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076819c310 con 0x7f0768100180 2023-12-01T01:17:03.248 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.245+0000 7f07677fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0768100b30 0x7f0768197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:03.248 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.245+0000 7f07677fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0768100b30 0x7f0768197560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57986/0 (socket says 172.21.15.77:57986) 2023-12-01T01:17:03.248 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.245+0000 7f07677fe700 1 -- 172.21.15.77:0/3696802999 learned_addr learned my addr 172.21.15.77:0/3696802999 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:03.249 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.245+0000 7f0767fff700 1 --2- 172.21.15.77:0/3696802999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0768100180 0x7f0768197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:03.249 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.245+0000 7f076ce2c700 1 --2- 172.21.15.77:0/3696802999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07681050d0 0x7f076819b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:03.249 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.245+0000 7f07677fe700 1 -- 172.21.15.77:0/3696802999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0768100180 msgr2=0x7f0768197020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:03.249 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.245+0000 7f07677fe700 1 --2- 172.21.15.77:0/3696802999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0768100180 0x7f0768197020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:03.249 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.245+0000 7f07677fe700 1 -- 172.21.15.77:0/3696802999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07681050d0 msgr2=0x7f076819b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:03.249 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.246+0000 7f07677fe700 1 --2- 172.21.15.77:0/3696802999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07681050d0 0x7f076819b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:03.249 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.246+0000 7f07677fe700 1 -- 172.21.15.77:0/3696802999 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0760009d00 con 0x7f0768100b30 2023-12-01T01:17:03.249 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.246+0000 7f07677fe700 1 --2- 172.21.15.77:0/3696802999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0768100b30 0x7f0768197560 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f075000d950 tx=0x7f075000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:03.249 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.246+0000 7f07657fa700 1 -- 172.21.15.77:0/3696802999 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0750004220 con 0x7f0768100b30 2023-12-01T01:17:03.249 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.246+0000 7f076e88f700 1 -- 172.21.15.77:0/3696802999 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f076819c5f0 con 0x7f0768100b30 2023-12-01T01:17:03.249 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.246+0000 7f07657fa700 1 -- 172.21.15.77:0/3696802999 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0750004cf0 con 0x7f0768100b30 2023-12-01T01:17:03.249 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.246+0000 7f07657fa700 1 -- 172.21.15.77:0/3696802999 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0750005620 con 0x7f0768100b30 2023-12-01T01:17:03.249 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.246+0000 7f076e88f700 1 -- 172.21.15.77:0/3696802999 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07681a3e50 con 0x7f0768100b30 2023-12-01T01:17:03.250 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.247+0000 7f076e88f700 1 -- 172.21.15.77:0/3696802999 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f076804ea90 con 0x7f0768100b30 2023-12-01T01:17:03.250 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.248+0000 7f07657fa700 1 -- 172.21.15.77:0/3696802999 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f0750004750 con 0x7f0768100b30 2023-12-01T01:17:03.250 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.249+0000 7f07657fa700 1 --2- 172.21.15.77:0/3696802999 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f07540a05f0 0x7f07540a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:03.251 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.249+0000 7f0767fff700 1 --2- 172.21.15.77:0/3696802999 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f07540a05f0 0x7f07540a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:03.251 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.249+0000 7f07657fa700 1 -- 172.21.15.77:0/3696802999 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f07500cbc20 con 0x7f0768100b30 2023-12-01T01:17:03.251 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.249+0000 7f0767fff700 1 --2- 172.21.15.77:0/3696802999 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f07540a05f0 0x7f07540a2ab0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0758005950 tx=0x7f075800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:03.254 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.253+0000 7f07657fa700 1 -- 172.21.15.77:0/3696802999 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0750095810 con 0x7f0768100b30 2023-12-01T01:17:03.426 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.424+0000 7f7f28ec5700 1 -- 172.21.15.77:0/229832535 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f24112ce0 msgr2=0x7f7f241150d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:03.426 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.424+0000 7f7f28ec5700 1 --2- 172.21.15.77:0/229832535 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f24112ce0 0x7f7f241150d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c00b0a0 tx=0x7f7f1c00b3b0 comp rx=0 tx=0).stop 2023-12-01T01:17:03.426 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.424+0000 7f7f28ec5700 1 -- 172.21.15.77:0/229832535 shutdown_connections 2023-12-01T01:17:03.426 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.424+0000 7f7f28ec5700 1 --2- 172.21.15.77:0/229832535 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f24112ce0 0x7f7f241150d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:03.426 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.424+0000 7f7f28ec5700 1 --2- 172.21.15.77:0/229832535 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f24109730 0x7f7f24109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:03.426 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.424+0000 7f7f28ec5700 1 --2- 172.21.15.77:0/229832535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f2410a050 0x7f7f241127a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:03.426 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.424+0000 7f7f28ec5700 1 -- 172.21.15.77:0/229832535 >> 172.21.15.77:0/229832535 conn(0x7f7f2406cf90 msgr2=0x7f7f2406d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:03.426 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.424+0000 7f7f28ec5700 1 -- 172.21.15.77:0/229832535 shutdown_connections 2023-12-01T01:17:03.427 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.424+0000 7f7f28ec5700 1 -- 172.21.15.77:0/229832535 wait complete. 2023-12-01T01:17:03.428 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.425+0000 7f7f28ec5700 1 Processor -- start 2023-12-01T01:17:03.428 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.425+0000 7f7f28ec5700 1 -- start start 2023-12-01T01:17:03.428 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.425+0000 7f7f28ec5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f24109730 0x7f7f24115b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:03.428 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.426+0000 7f7f28ec5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f2410a050 0x7f7f24116040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:03.428 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.426+0000 7f7f28ec5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f24112ce0 0x7f7f241165c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:03.428 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.426+0000 7f7f28ec5700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f241adbe0 con 0x7f7f24109730 2023-12-01T01:17:03.428 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.426+0000 7f7f28ec5700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f241add50 con 0x7f7f24112ce0 2023-12-01T01:17:03.428 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.426+0000 7f7f28ec5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f241adf10 con 0x7f7f2410a050 2023-12-01T01:17:03.428 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.426+0000 7f7f22d9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f24112ce0 0x7f7f241165c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:03.429 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.426+0000 7f7f22d9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f24112ce0 0x7f7f241165c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:36922/0 (socket says 172.21.15.77:36922) 2023-12-01T01:17:03.429 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.426+0000 7f7f22d9d700 1 -- 172.21.15.77:0/1941262794 learned_addr learned my addr 172.21.15.77:0/1941262794 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:03.429 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.426+0000 7f7f21d9b700 1 --2- 172.21.15.77:0/1941262794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f2410a050 0x7f7f24116040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:03.429 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.426+0000 7f7f2259c700 1 --2- 172.21.15.77:0/1941262794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f24109730 0x7f7f24115b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:03.429 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.426+0000 7f7f22d9d700 1 -- 172.21.15.77:0/1941262794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f2410a050 msgr2=0x7f7f24116040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:03.429 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.426+0000 7f7f22d9d700 1 --2- 172.21.15.77:0/1941262794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f2410a050 0x7f7f24116040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:03.429 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.426+0000 7f7f22d9d700 1 -- 172.21.15.77:0/1941262794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f24109730 msgr2=0x7f7f24115b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:03.429 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.426+0000 7f7f22d9d700 1 --2- 172.21.15.77:0/1941262794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f24109730 0x7f7f24115b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:03.429 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.426+0000 7f7f22d9d700 1 -- 172.21.15.77:0/1941262794 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f1c009d00 con 0x7f7f24112ce0 2023-12-01T01:17:03.429 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.426+0000 7f7f2259c700 1 --2- 172.21.15.77:0/1941262794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f24109730 0x7f7f24115b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:17:03.429 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.426+0000 7f7f21d9b700 1 --2- 172.21.15.77:0/1941262794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f2410a050 0x7f7f24116040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:17:03.429 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.427+0000 7f7f22d9d700 1 --2- 172.21.15.77:0/1941262794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f24112ce0 0x7f7f241165c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c00b780 tx=0x7f7f1c007990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:03.429 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.427+0000 7f7f137fe700 1 -- 172.21.15.77:0/1941262794 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c00e040 con 0x7f7f24112ce0 2023-12-01T01:17:03.430 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.427+0000 7f7f28ec5700 1 -- 172.21.15.77:0/1941262794 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f241ae190 con 0x7f7f24112ce0 2023-12-01T01:17:03.430 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.427+0000 7f7f137fe700 1 -- 172.21.15.77:0/1941262794 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f1c00be40 con 0x7f7f24112ce0 2023-12-01T01:17:03.430 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.427+0000 7f7f137fe700 1 -- 172.21.15.77:0/1941262794 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c0052a0 con 0x7f7f24112ce0 2023-12-01T01:17:03.430 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.427+0000 7f7f28ec5700 1 -- 172.21.15.77:0/1941262794 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f241ae650 con 0x7f7f24112ce0 2023-12-01T01:17:03.430 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.428+0000 7f7f28ec5700 1 -- 172.21.15.77:0/1941262794 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f2404ea90 con 0x7f7f24112ce0 2023-12-01T01:17:03.430 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.429+0000 7f7f137fe700 1 -- 172.21.15.77:0/1941262794 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f7f1c01d470 con 0x7f7f24112ce0 2023-12-01T01:17:03.431 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.430+0000 7f7f137fe700 1 --2- 172.21.15.77:0/1941262794 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7f0c0a05b0 0x7f7f0c0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:03.432 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.430+0000 7f7f2259c700 1 --2- 172.21.15.77:0/1941262794 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7f0c0a05b0 0x7f7f0c0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:03.432 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.430+0000 7f7f137fe700 1 -- 172.21.15.77:0/1941262794 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7f1c0ccf60 con 0x7f7f24112ce0 2023-12-01T01:17:03.432 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.430+0000 7f7f2259c700 1 --2- 172.21.15.77:0/1941262794 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7f0c0a05b0 0x7f7f0c0a2a70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7f14009cb0 tx=0x7f7f14009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:03.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.434+0000 7f7f137fe700 1 -- 172.21.15.77:0/1941262794 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7f1c096ae0 con 0x7f7f24112ce0 2023-12-01T01:17:03.463 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.462+0000 7f076e88f700 1 -- 172.21.15.77:0/3696802999 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f07681a40b0 con 0x7f0768100b30 2023-12-01T01:17:03.464 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.462+0000 7f07657fa700 1 -- 172.21.15.77:0/3696802999 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0750016090 con 0x7f0768100b30 2023-12-01T01:17:03.464 INFO:teuthology.orchestra.run.smithi077.stdout:193273528360 2023-12-01T01:17:03.467 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.465+0000 7f076e88f700 1 -- 172.21.15.77:0/3696802999 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f07540a05f0 msgr2=0x7f07540a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:03.467 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.465+0000 7f076e88f700 1 --2- 172.21.15.77:0/3696802999 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f07540a05f0 0x7f07540a2ab0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0758005950 tx=0x7f075800a300 comp rx=0 tx=0).stop 2023-12-01T01:17:03.467 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.465+0000 7f076e88f700 1 -- 172.21.15.77:0/3696802999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0768100b30 msgr2=0x7f0768197560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:03.467 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.465+0000 7f076e88f700 1 --2- 172.21.15.77:0/3696802999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0768100b30 0x7f0768197560 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f075000d950 tx=0x7f075000dc60 comp rx=0 tx=0).stop 2023-12-01T01:17:03.467 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.466+0000 7f076e88f700 1 -- 172.21.15.77:0/3696802999 shutdown_connections 2023-12-01T01:17:03.467 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.466+0000 7f076e88f700 1 --2- 172.21.15.77:0/3696802999 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f07540a05f0 0x7f07540a2ab0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:03.467 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.466+0000 7f076e88f700 1 --2- 172.21.15.77:0/3696802999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07681050d0 0x7f076819b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:03.467 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.466+0000 7f076e88f700 1 --2- 172.21.15.77:0/3696802999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0768100180 0x7f0768197020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:03.467 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.466+0000 7f076e88f700 1 --2- 172.21.15.77:0/3696802999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0768100b30 0x7f0768197560 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:03.468 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.466+0000 7f076e88f700 1 -- 172.21.15.77:0/3696802999 >> 172.21.15.77:0/3696802999 conn(0x7f0768076bb0 msgr2=0x7f07680692f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:03.468 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.466+0000 7f076e88f700 1 -- 172.21.15.77:0/3696802999 shutdown_connections 2023-12-01T01:17:03.468 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.466+0000 7f076e88f700 1 -- 172.21.15.77:0/3696802999 wait complete. 2023-12-01T01:17:03.660 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.659+0000 7f7f28ec5700 1 -- 172.21.15.77:0/1941262794 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f7f241ae970 con 0x7f7f24112ce0 2023-12-01T01:17:03.678 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906970 got 274877906971 for osd.9 2023-12-01T01:17:03.679 DEBUG:teuthology.parallel:result is None 2023-12-01T01:17:03.692 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.660+0000 7f7f137fe700 1 -- 172.21.15.77:0/1941262794 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7f1c09a100 con 0x7f7f24112ce0 2023-12-01T01:17:03.692 INFO:teuthology.orchestra.run.smithi077.stdout:137438953521 2023-12-01T01:17:03.692 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.662+0000 7f7f28ec5700 1 -- 172.21.15.77:0/1941262794 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7f0c0a05b0 msgr2=0x7f7f0c0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:03.692 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.662+0000 7f7f28ec5700 1 --2- 172.21.15.77:0/1941262794 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7f0c0a05b0 0x7f7f0c0a2a70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7f14009cb0 tx=0x7f7f14009260 comp rx=0 tx=0).stop 2023-12-01T01:17:03.692 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.662+0000 7f7f28ec5700 1 -- 172.21.15.77:0/1941262794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f24112ce0 msgr2=0x7f7f241165c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:03.693 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.662+0000 7f7f28ec5700 1 --2- 172.21.15.77:0/1941262794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f24112ce0 0x7f7f241165c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c00b780 tx=0x7f7f1c007990 comp rx=0 tx=0).stop 2023-12-01T01:17:03.693 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.662+0000 7f7f28ec5700 1 -- 172.21.15.77:0/1941262794 shutdown_connections 2023-12-01T01:17:03.693 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.662+0000 7f7f28ec5700 1 --2- 172.21.15.77:0/1941262794 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7f0c0a05b0 0x7f7f0c0a2a70 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:03.693 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.662+0000 7f7f28ec5700 1 --2- 172.21.15.77:0/1941262794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f24112ce0 0x7f7f241165c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:03.693 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.662+0000 7f7f28ec5700 1 --2- 172.21.15.77:0/1941262794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f24109730 0x7f7f24115b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:03.693 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.662+0000 7f7f28ec5700 1 --2- 172.21.15.77:0/1941262794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f2410a050 0x7f7f24116040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:03.693 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.662+0000 7f7f28ec5700 1 -- 172.21.15.77:0/1941262794 >> 172.21.15.77:0/1941262794 conn(0x7f7f2406cf90 msgr2=0x7f7f2406f870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:03.693 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.662+0000 7f7f28ec5700 1 -- 172.21.15.77:0/1941262794 shutdown_connections 2023-12-01T01:17:03.693 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:03.662+0000 7f7f28ec5700 1 -- 172.21.15.77:0/1941262794 wait complete. 2023-12-01T01:17:03.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:03 smithi077 ceph-mon[98867]: pgmap v271: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2023-12-01T01:17:03.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:03 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/3696802999' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-12-01T01:17:04.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:03 smithi203 ceph-mon[103581]: pgmap v271: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2023-12-01T01:17:04.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:03 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/3696802999' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-12-01T01:17:04.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:03 smithi137 ceph-mon[103312]: pgmap v271: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2023-12-01T01:17:04.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:03 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/3696802999' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-12-01T01:17:04.108 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528358 got 193273528360 for osd.6 2023-12-01T01:17:04.108 DEBUG:teuthology.parallel:result is None 2023-12-01T01:17:04.136 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.135+0000 7f6a77c46700 1 -- 172.21.15.77:0/1208717945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a700ff6b0 msgr2=0x7f6a700ffb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:04.136 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.135+0000 7f6a77c46700 1 --2- 172.21.15.77:0/1208717945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a700ff6b0 0x7f6a700ffb30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6a60009990 tx=0x7f6a60009ca0 comp rx=0 tx=0).stop 2023-12-01T01:17:04.136 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.135+0000 7f6a77c46700 1 -- 172.21.15.77:0/1208717945 shutdown_connections 2023-12-01T01:17:04.136 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.135+0000 7f6a77c46700 1 --2- 172.21.15.77:0/1208717945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a70100070 0x7f6a7010dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:04.136 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.135+0000 7f6a77c46700 1 --2- 172.21.15.77:0/1208717945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a70105e40 0x7f6a70106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:04.136 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.135+0000 7f6a77c46700 1 --2- 172.21.15.77:0/1208717945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a700ff6b0 0x7f6a700ffb30 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:04.136 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.135+0000 7f6a77c46700 1 -- 172.21.15.77:0/1208717945 >> 172.21.15.77:0/1208717945 conn(0x7f6a70074fb0 msgr2=0x7f6a700753c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:04.137 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.135+0000 7f6a77c46700 1 -- 172.21.15.77:0/1208717945 shutdown_connections 2023-12-01T01:17:04.137 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.135+0000 7f6a77c46700 1 -- 172.21.15.77:0/1208717945 wait complete. 2023-12-01T01:17:04.138 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.135+0000 7f6a77c46700 1 Processor -- start 2023-12-01T01:17:04.138 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.135+0000 7f6a77c46700 1 -- start start 2023-12-01T01:17:04.138 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.136+0000 7f6a77c46700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a700ff6b0 0x7f6a70196ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:04.138 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.136+0000 7f6a77c46700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a70100070 0x7f6a70197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:04.138 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.136+0000 7f6a77c46700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a70105e40 0x7f6a7019b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:04.138 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.136+0000 7f6a77c46700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7019beb0 con 0x7f6a700ff6b0 2023-12-01T01:17:04.138 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.136+0000 7f6a77c46700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7019c020 con 0x7f6a70100070 2023-12-01T01:17:04.138 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.136+0000 7f6a77c46700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7019c1e0 con 0x7f6a70105e40 2023-12-01T01:17:04.138 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.136+0000 7f6a759e2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a700ff6b0 0x7f6a70196ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:04.139 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.136+0000 7f6a759e2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a700ff6b0 0x7f6a70196ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58012/0 (socket says 172.21.15.77:58012) 2023-12-01T01:17:04.139 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.136+0000 7f6a759e2700 1 -- 172.21.15.77:0/2515907820 learned_addr learned my addr 172.21.15.77:0/2515907820 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:04.139 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.136+0000 7f6a761e3700 1 --2- 172.21.15.77:0/2515907820 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a70105e40 0x7f6a7019b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:04.139 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.136+0000 7f6a751e1700 1 --2- 172.21.15.77:0/2515907820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a70100070 0x7f6a70197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:04.139 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.136+0000 7f6a759e2700 1 -- 172.21.15.77:0/2515907820 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a70105e40 msgr2=0x7f6a7019b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:04.139 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.136+0000 7f6a759e2700 1 --2- 172.21.15.77:0/2515907820 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a70105e40 0x7f6a7019b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:04.139 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.136+0000 7f6a759e2700 1 -- 172.21.15.77:0/2515907820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a70100070 msgr2=0x7f6a70197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:04.139 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.136+0000 7f6a759e2700 1 --2- 172.21.15.77:0/2515907820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a70100070 0x7f6a70197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:04.139 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.136+0000 7f6a759e2700 1 -- 172.21.15.77:0/2515907820 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a600095f0 con 0x7f6a700ff6b0 2023-12-01T01:17:04.139 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.136+0000 7f6a751e1700 1 --2- 172.21.15.77:0/2515907820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a70100070 0x7f6a70197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:17:04.139 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.136+0000 7f6a761e3700 1 --2- 172.21.15.77:0/2515907820 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a70105e40 0x7f6a7019b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:17:04.139 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.136+0000 7f6a759e2700 1 --2- 172.21.15.77:0/2515907820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a700ff6b0 0x7f6a70196ef0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c00d950 tx=0x7f6a6c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:04.139 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.136+0000 7f6a66ffd700 1 -- 172.21.15.77:0/2515907820 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c009b40 con 0x7f6a700ff6b0 2023-12-01T01:17:04.139 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.136+0000 7f6a77c46700 1 -- 172.21.15.77:0/2515907820 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a7019c4c0 con 0x7f6a700ff6b0 2023-12-01T01:17:04.140 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.136+0000 7f6a66ffd700 1 -- 172.21.15.77:0/2515907820 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a6c009ca0 con 0x7f6a700ff6b0 2023-12-01T01:17:04.140 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.136+0000 7f6a66ffd700 1 -- 172.21.15.77:0/2515907820 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c00f620 con 0x7f6a700ff6b0 2023-12-01T01:17:04.140 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.136+0000 7f6a77c46700 1 -- 172.21.15.77:0/2515907820 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a701a3d20 con 0x7f6a700ff6b0 2023-12-01T01:17:04.140 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.137+0000 7f6a77c46700 1 -- 172.21.15.77:0/2515907820 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a7004ea90 con 0x7f6a700ff6b0 2023-12-01T01:17:04.140 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.139+0000 7f6a66ffd700 1 -- 172.21.15.77:0/2515907820 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f6a6c010460 con 0x7f6a700ff6b0 2023-12-01T01:17:04.141 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.140+0000 7f6a66ffd700 1 --2- 172.21.15.77:0/2515907820 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f6a5c0a05b0 0x7f6a5c0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:04.141 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.140+0000 7f6a751e1700 1 --2- 172.21.15.77:0/2515907820 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f6a5c0a05b0 0x7f6a5c0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:04.141 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.140+0000 7f6a66ffd700 1 -- 172.21.15.77:0/2515907820 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6a6c0cbdb0 con 0x7f6a700ff6b0 2023-12-01T01:17:04.141 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.140+0000 7f6a751e1700 1 --2- 172.21.15.77:0/2515907820 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f6a5c0a05b0 0x7f6a5c0a2a70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6a6000b5c0 tx=0x7f6a60005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:04.144 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.143+0000 7f6a66ffd700 1 -- 172.21.15.77:0/2515907820 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6a6c0958a0 con 0x7f6a700ff6b0 2023-12-01T01:17:04.346 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.345+0000 7f6a77c46700 1 -- 172.21.15.77:0/2515907820 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6a70198250 con 0x7f6a700ff6b0 2023-12-01T01:17:04.347 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.345+0000 7f6a66ffd700 1 -- 172.21.15.77:0/2515907820 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6a6c020020 con 0x7f6a700ff6b0 2023-12-01T01:17:04.347 INFO:teuthology.orchestra.run.smithi077.stdout:34359738434 2023-12-01T01:17:04.349 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.347+0000 7f6a77c46700 1 -- 172.21.15.77:0/2515907820 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f6a5c0a05b0 msgr2=0x7f6a5c0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:04.349 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.347+0000 7f6a77c46700 1 --2- 172.21.15.77:0/2515907820 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f6a5c0a05b0 0x7f6a5c0a2a70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6a6000b5c0 tx=0x7f6a60005fb0 comp rx=0 tx=0).stop 2023-12-01T01:17:04.349 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.347+0000 7f6a77c46700 1 -- 172.21.15.77:0/2515907820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a700ff6b0 msgr2=0x7f6a70196ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:04.349 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.347+0000 7f6a77c46700 1 --2- 172.21.15.77:0/2515907820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a700ff6b0 0x7f6a70196ef0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c00d950 tx=0x7f6a6c00dd10 comp rx=0 tx=0).stop 2023-12-01T01:17:04.349 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.348+0000 7f6a77c46700 1 -- 172.21.15.77:0/2515907820 shutdown_connections 2023-12-01T01:17:04.349 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.348+0000 7f6a77c46700 1 --2- 172.21.15.77:0/2515907820 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f6a5c0a05b0 0x7f6a5c0a2a70 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:04.349 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.348+0000 7f6a77c46700 1 --2- 172.21.15.77:0/2515907820 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a70105e40 0x7f6a7019b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:04.349 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.348+0000 7f6a77c46700 1 --2- 172.21.15.77:0/2515907820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a700ff6b0 0x7f6a70196ef0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:04.349 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.348+0000 7f6a77c46700 1 --2- 172.21.15.77:0/2515907820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a70100070 0x7f6a70197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:04.349 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.348+0000 7f6a77c46700 1 -- 172.21.15.77:0/2515907820 >> 172.21.15.77:0/2515907820 conn(0x7f6a70074fb0 msgr2=0x7f6a700fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:04.350 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.348+0000 7f6a77c46700 1 -- 172.21.15.77:0/2515907820 shutdown_connections 2023-12-01T01:17:04.350 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.348+0000 7f6a77c46700 1 -- 172.21.15.77:0/2515907820 wait complete. 2023-12-01T01:17:04.458 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.226+0000 7fc155c43700 1 -- 172.21.15.77:0/1790914344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1501061b0 msgr2=0x7fc15010dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:04.458 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.226+0000 7fc155c43700 1 --2- 172.21.15.77:0/1790914344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1501061b0 0x7fc15010dd30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc14800b0a0 tx=0x7fc14800b3b0 comp rx=0 tx=0).stop 2023-12-01T01:17:04.458 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.226+0000 7fc155c43700 1 -- 172.21.15.77:0/1790914344 shutdown_connections 2023-12-01T01:17:04.458 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.226+0000 7fc155c43700 1 --2- 172.21.15.77:0/1790914344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1501061b0 0x7fc15010dd30 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:04.458 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.226+0000 7fc155c43700 1 --2- 172.21.15.77:0/1790914344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc150101280 0x7fc150101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:04.458 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.226+0000 7fc155c43700 1 --2- 172.21.15.77:0/1790914344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc150101c30 0x7fc150105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:04.458 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.226+0000 7fc155c43700 1 -- 172.21.15.77:0/1790914344 >> 172.21.15.77:0/1790914344 conn(0x7fc150078d60 msgr2=0x7fc150079170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:04.458 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.226+0000 7fc155c43700 1 -- 172.21.15.77:0/1790914344 shutdown_connections 2023-12-01T01:17:04.458 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.226+0000 7fc155c43700 1 -- 172.21.15.77:0/1790914344 wait complete. 2023-12-01T01:17:04.458 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.226+0000 7fc155c43700 1 Processor -- start 2023-12-01T01:17:04.458 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.226+0000 7fc155c43700 1 -- start start 2023-12-01T01:17:04.459 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.227+0000 7fc155c43700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc150101280 0x7fc15019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:04.459 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.227+0000 7fc155c43700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc150101c30 0x7fc15019f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:04.459 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.227+0000 7fc155c43700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1501061b0 0x7fc150198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:04.459 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.227+0000 7fc155c43700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc150199610 con 0x7fc1501061b0 2023-12-01T01:17:04.459 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.227+0000 7fc155c43700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc150199780 con 0x7fc150101c30 2023-12-01T01:17:04.459 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.227+0000 7fc155c43700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc150199940 con 0x7fc150101280 2023-12-01T01:17:04.459 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.227+0000 7fc14ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1501061b0 0x7fc150198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:04.459 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.227+0000 7fc14ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1501061b0 0x7fc150198f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58030/0 (socket says 172.21.15.77:58030) 2023-12-01T01:17:04.459 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.227+0000 7fc14ffff700 1 -- 172.21.15.77:0/3457809512 learned_addr learned my addr 172.21.15.77:0/3457809512 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:04.459 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.227+0000 7fc14f7fe700 1 --2- 172.21.15.77:0/3457809512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc150101280 0x7fc15019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:04.459 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.227+0000 7fc14effd700 1 --2- 172.21.15.77:0/3457809512 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc150101c30 0x7fc15019f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:04.459 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.227+0000 7fc14ffff700 1 -- 172.21.15.77:0/3457809512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc150101280 msgr2=0x7fc15019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:04.459 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.227+0000 7fc14ffff700 1 --2- 172.21.15.77:0/3457809512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc150101280 0x7fc15019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:04.460 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.227+0000 7fc14ffff700 1 -- 172.21.15.77:0/3457809512 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc150101c30 msgr2=0x7fc15019f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:04.460 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.227+0000 7fc14ffff700 1 --2- 172.21.15.77:0/3457809512 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc150101c30 0x7fc15019f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:04.460 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.227+0000 7fc14ffff700 1 -- 172.21.15.77:0/3457809512 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc148009d00 con 0x7fc1501061b0 2023-12-01T01:17:04.460 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.227+0000 7fc14effd700 1 --2- 172.21.15.77:0/3457809512 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc150101c30 0x7fc15019f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:17:04.460 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.227+0000 7fc14ffff700 1 --2- 172.21.15.77:0/3457809512 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1501061b0 0x7fc150198f80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc14800d010 tx=0x7fc148009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:04.460 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.227+0000 7fc14cff9700 1 -- 172.21.15.77:0/3457809512 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14800e040 con 0x7fc1501061b0 2023-12-01T01:17:04.460 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.227+0000 7fc155c43700 1 -- 172.21.15.77:0/3457809512 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc150199bc0 con 0x7fc1501061b0 2023-12-01T01:17:04.460 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.227+0000 7fc14cff9700 1 -- 172.21.15.77:0/3457809512 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc148004e50 con 0x7fc1501061b0 2023-12-01T01:17:04.460 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.227+0000 7fc14cff9700 1 -- 172.21.15.77:0/3457809512 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14801bb20 con 0x7fc1501061b0 2023-12-01T01:17:04.460 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.227+0000 7fc155c43700 1 -- 172.21.15.77:0/3457809512 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1501a5ef0 con 0x7fc1501061b0 2023-12-01T01:17:04.460 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.228+0000 7fc155c43700 1 -- 172.21.15.77:0/3457809512 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc150108350 con 0x7fc1501061b0 2023-12-01T01:17:04.460 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.229+0000 7fc14cff9700 1 -- 172.21.15.77:0/3457809512 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fc14801bc80 con 0x7fc1501061b0 2023-12-01T01:17:04.460 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.230+0000 7fc14cff9700 1 --2- 172.21.15.77:0/3457809512 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc1380a05b0 0x7fc1380a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:04.460 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.230+0000 7fc14f7fe700 1 --2- 172.21.15.77:0/3457809512 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc1380a05b0 0x7fc1380a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:04.461 INFO:teuthology.orchestra.run.smithi077.stdout:330712481811 2023-12-01T01:17:04.461 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.230+0000 7fc14cff9700 1 -- 172.21.15.77:0/3457809512 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc1480cdd50 con 0x7fc1501061b0 2023-12-01T01:17:04.461 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.231+0000 7fc14f7fe700 1 --2- 172.21.15.77:0/3457809512 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc1380a05b0 0x7fc1380a2a70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc140005950 tx=0x7fc1400058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:04.461 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.234+0000 7fc14cff9700 1 -- 172.21.15.77:0/3457809512 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc148097910 con 0x7fc1501061b0 2023-12-01T01:17:04.461 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.436+0000 7fc155c43700 1 -- 172.21.15.77:0/3457809512 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7fc15002cc70 con 0x7fc1501061b0 2023-12-01T01:17:04.461 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.437+0000 7fc14cff9700 1 -- 172.21.15.77:0/3457809512 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fc148021020 con 0x7fc1501061b0 2023-12-01T01:17:04.461 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.440+0000 7fc155c43700 1 -- 172.21.15.77:0/3457809512 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc1380a05b0 msgr2=0x7fc1380a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:04.461 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.440+0000 7fc155c43700 1 --2- 172.21.15.77:0/3457809512 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc1380a05b0 0x7fc1380a2a70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc140005950 tx=0x7fc1400058e0 comp rx=0 tx=0).stop 2023-12-01T01:17:04.461 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.440+0000 7fc155c43700 1 -- 172.21.15.77:0/3457809512 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1501061b0 msgr2=0x7fc150198f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:04.462 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.440+0000 7fc155c43700 1 --2- 172.21.15.77:0/3457809512 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1501061b0 0x7fc150198f80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc14800d010 tx=0x7fc148009500 comp rx=0 tx=0).stop 2023-12-01T01:17:04.462 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.440+0000 7fc155c43700 1 -- 172.21.15.77:0/3457809512 shutdown_connections 2023-12-01T01:17:04.462 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.440+0000 7fc155c43700 1 --2- 172.21.15.77:0/3457809512 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc1380a05b0 0x7fc1380a2a70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:04.462 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.440+0000 7fc155c43700 1 --2- 172.21.15.77:0/3457809512 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1501061b0 0x7fc150198f80 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:04.462 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.440+0000 7fc155c43700 1 --2- 172.21.15.77:0/3457809512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc150101280 0x7fc15019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:04.462 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.440+0000 7fc155c43700 1 --2- 172.21.15.77:0/3457809512 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc150101c30 0x7fc15019f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:04.462 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.440+0000 7fc155c43700 1 -- 172.21.15.77:0/3457809512 >> 172.21.15.77:0/3457809512 conn(0x7fc150078d60 msgr2=0x7fc1500ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:04.462 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.440+0000 7fc155c43700 1 -- 172.21.15.77:0/3457809512 shutdown_connections 2023-12-01T01:17:04.462 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:04.440+0000 7fc155c43700 1 -- 172.21.15.77:0/3457809512 wait complete. 2023-12-01T01:17:04.676 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953519 got 137438953521 for osd.4 2023-12-01T01:17:04.676 DEBUG:teuthology.parallel:result is None 2023-12-01T01:17:04.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:04 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/1941262794' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-12-01T01:17:04.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:04 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/2515907820' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-12-01T01:17:04.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:04 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/3457809512' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-12-01T01:17:05.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:04 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/1941262794' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-12-01T01:17:05.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:04 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/2515907820' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-12-01T01:17:05.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:04 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/3457809512' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-12-01T01:17:05.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:04 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/1941262794' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-12-01T01:17:05.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:04 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/2515907820' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-12-01T01:17:05.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:04 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/3457809512' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-12-01T01:17:05.330 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738433 got 34359738434 for osd.0 2023-12-01T01:17:05.331 DEBUG:teuthology.parallel:result is None 2023-12-01T01:17:05.564 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.563+0000 7f9c118d8700 1 -- 172.21.15.77:0/1674217214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c0c1018f0 msgr2=0x7f9c0c101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:05.564 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.563+0000 7f9c118d8700 1 --2- 172.21.15.77:0/1674217214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c0c1018f0 0x7f9c0c101d70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9c000099c0 tx=0x7f9c00009cd0 comp rx=0 tx=0).stop 2023-12-01T01:17:05.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.563+0000 7f9c118d8700 1 -- 172.21.15.77:0/1674217214 shutdown_connections 2023-12-01T01:17:05.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.563+0000 7f9c118d8700 1 --2- 172.21.15.77:0/1674217214 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c0c1022b0 0x7f9c0c10cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:05.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.563+0000 7f9c118d8700 1 --2- 172.21.15.77:0/1674217214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c0c100010 0x7f9c0c1003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:05.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.563+0000 7f9c118d8700 1 --2- 172.21.15.77:0/1674217214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c0c1018f0 0x7f9c0c101d70 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:05.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.563+0000 7f9c118d8700 1 -- 172.21.15.77:0/1674217214 >> 172.21.15.77:0/1674217214 conn(0x7f9c0c076d80 msgr2=0x7f9c0c077190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:05.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.563+0000 7f9c118d8700 1 -- 172.21.15.77:0/1674217214 shutdown_connections 2023-12-01T01:17:05.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.563+0000 7f9c118d8700 1 -- 172.21.15.77:0/1674217214 wait complete. 2023-12-01T01:17:05.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.563+0000 7f9c118d8700 1 Processor -- start 2023-12-01T01:17:05.567 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.563+0000 7f9c118d8700 1 -- start start 2023-12-01T01:17:05.567 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.564+0000 7f9c118d8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c0c100010 0x7f9c0c195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:05.567 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.564+0000 7f9c118d8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c0c1018f0 0x7f9c0c196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:05.567 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.564+0000 7f9c118d8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c0c1022b0 0x7f9c0c19a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:05.567 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.564+0000 7f9c118d8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0c19adc0 con 0x7f9c0c1018f0 2023-12-01T01:17:05.567 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.564+0000 7f9c118d8700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0c19af30 con 0x7f9c0c1022b0 2023-12-01T01:17:05.567 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.564+0000 7f9c118d8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0c19b0f0 con 0x7f9c0c100010 2023-12-01T01:17:05.567 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.564+0000 7f9c0a7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c0c1018f0 0x7f9c0c196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:05.567 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.564+0000 7f9c0a7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c0c1018f0 0x7f9c0c196340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58042/0 (socket says 172.21.15.77:58042) 2023-12-01T01:17:05.567 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.564+0000 7f9c0a7fc700 1 -- 172.21.15.77:0/4075344681 learned_addr learned my addr 172.21.15.77:0/4075344681 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:05.567 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.564+0000 7f9c0b7fe700 1 --2- 172.21.15.77:0/4075344681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c0c1022b0 0x7f9c0c19a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:05.567 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.564+0000 7f9c0affd700 1 --2- 172.21.15.77:0/4075344681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c0c100010 0x7f9c0c195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:05.568 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.564+0000 7f9c0a7fc700 1 -- 172.21.15.77:0/4075344681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c0c100010 msgr2=0x7f9c0c195e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:05.568 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.564+0000 7f9c0a7fc700 1 --2- 172.21.15.77:0/4075344681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c0c100010 0x7f9c0c195e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:05.568 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.564+0000 7f9c0a7fc700 1 -- 172.21.15.77:0/4075344681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c0c1022b0 msgr2=0x7f9c0c19a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:05.568 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.564+0000 7f9c0a7fc700 1 --2- 172.21.15.77:0/4075344681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c0c1022b0 0x7f9c0c19a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:05.568 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.564+0000 7f9c0a7fc700 1 -- 172.21.15.77:0/4075344681 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c000095f0 con 0x7f9c0c1018f0 2023-12-01T01:17:05.568 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.564+0000 7f9c0b7fe700 1 --2- 172.21.15.77:0/4075344681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c0c1022b0 0x7f9c0c19a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:17:05.568 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.564+0000 7f9c0a7fc700 1 --2- 172.21.15.77:0/4075344681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c0c1018f0 0x7f9c0c196340 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9c00009990 tx=0x7f9c00009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:05.568 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.564+0000 7f9c108d6700 1 -- 172.21.15.77:0/4075344681 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0001f070 con 0x7f9c0c1018f0 2023-12-01T01:17:05.568 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.564+0000 7f9c108d6700 1 -- 172.21.15.77:0/4075344681 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c0001ab50 con 0x7f9c0c1018f0 2023-12-01T01:17:05.568 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.564+0000 7f9c118d8700 1 -- 172.21.15.77:0/4075344681 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c0c19b370 con 0x7f9c0c1018f0 2023-12-01T01:17:05.568 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.565+0000 7f9c118d8700 1 -- 172.21.15.77:0/4075344681 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c0c0692b0 con 0x7f9c0c1018f0 2023-12-01T01:17:05.568 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.565+0000 7f9c108d6700 1 -- 172.21.15.77:0/4075344681 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c00004120 con 0x7f9c0c1018f0 2023-12-01T01:17:05.568 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.565+0000 7f9c118d8700 1 -- 172.21.15.77:0/4075344681 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf8005320 con 0x7f9c0c1018f0 2023-12-01T01:17:05.569 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.567+0000 7f9c108d6700 1 -- 172.21.15.77:0/4075344681 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f9c00023070 con 0x7f9c0c1018f0 2023-12-01T01:17:05.569 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.567+0000 7f9c108d6700 1 --2- 172.21.15.77:0/4075344681 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f9bf40a05b0 0x7f9bf40a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:05.569 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.567+0000 7f9c0affd700 1 --2- 172.21.15.77:0/4075344681 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f9bf40a05b0 0x7f9bf40a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:05.569 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.567+0000 7f9c108d6700 1 -- 172.21.15.77:0/4075344681 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9c000cef70 con 0x7f9c0c1018f0 2023-12-01T01:17:05.569 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.568+0000 7f9c0affd700 1 --2- 172.21.15.77:0/4075344681 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f9bf40a05b0 0x7f9bf40a2a70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc005950 tx=0x7f9bfc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:05.572 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.571+0000 7f9c108d6700 1 -- 172.21.15.77:0/4075344681 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9c00098bf0 con 0x7f9c0c1018f0 2023-12-01T01:17:05.731 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.729+0000 7fa72ce59700 1 -- 172.21.15.77:0/3186558855 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7281007b0 msgr2=0x7fa728100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:05.731 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.729+0000 7fa72ce59700 1 --2- 172.21.15.77:0/3186558855 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7281007b0 0x7fa728100b90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa718009990 tx=0x7fa718009ca0 comp rx=0 tx=0).stop 2023-12-01T01:17:05.731 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.729+0000 7fa72ce59700 1 -- 172.21.15.77:0/3186558855 shutdown_connections 2023-12-01T01:17:05.731 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.729+0000 7fa72ce59700 1 --2- 172.21.15.77:0/3186558855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa72810d820 0x7fa72810fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:05.731 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.729+0000 7fa72ce59700 1 --2- 172.21.15.77:0/3186558855 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7281007b0 0x7fa728100b90 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:05.731 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.729+0000 7fa72ce59700 1 --2- 172.21.15.77:0/3186558855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7281010d0 0x7fa72810d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:05.731 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.729+0000 7fa72ce59700 1 -- 172.21.15.77:0/3186558855 >> 172.21.15.77:0/3186558855 conn(0x7fa728076d90 msgr2=0x7fa7280771a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:05.731 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.729+0000 7fa72ce59700 1 -- 172.21.15.77:0/3186558855 shutdown_connections 2023-12-01T01:17:05.731 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.730+0000 7fa72ce59700 1 -- 172.21.15.77:0/3186558855 wait complete. 2023-12-01T01:17:05.733 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.730+0000 7fa72ce59700 1 Processor -- start 2023-12-01T01:17:05.733 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.730+0000 7fa72ce59700 1 -- start start 2023-12-01T01:17:05.733 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.731+0000 7fa72ce59700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7281007b0 0x7fa728199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:05.733 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.731+0000 7fa72ce59700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7281010d0 0x7fa728199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:05.733 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.731+0000 7fa72ce59700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa72810d820 0x7fa72819dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:05.733 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.731+0000 7fa72ce59700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72819e0c0 con 0x7fa7281007b0 2023-12-01T01:17:05.733 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.731+0000 7fa72ce59700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72819e230 con 0x7fa72810d820 2023-12-01T01:17:05.733 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.731+0000 7fa72ce59700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72819e3f0 con 0x7fa7281010d0 2023-12-01T01:17:05.733 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.731+0000 7fa72659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7281007b0 0x7fa728199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:05.733 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.731+0000 7fa72659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7281007b0 0x7fa728199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58054/0 (socket says 172.21.15.77:58054) 2023-12-01T01:17:05.734 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.731+0000 7fa72659c700 1 -- 172.21.15.77:0/2654345656 learned_addr learned my addr 172.21.15.77:0/2654345656 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:05.734 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.731+0000 7fa726d9d700 1 --2- 172.21.15.77:0/2654345656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa72810d820 0x7fa72819dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:05.734 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.731+0000 7fa725d9b700 1 --2- 172.21.15.77:0/2654345656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7281010d0 0x7fa728199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:05.734 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.731+0000 7fa72659c700 1 -- 172.21.15.77:0/2654345656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7281010d0 msgr2=0x7fa728199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:05.734 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.731+0000 7fa72659c700 1 --2- 172.21.15.77:0/2654345656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7281010d0 0x7fa728199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:05.734 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.731+0000 7fa72659c700 1 -- 172.21.15.77:0/2654345656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa72810d820 msgr2=0x7fa72819dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:05.734 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.731+0000 7fa72659c700 1 --2- 172.21.15.77:0/2654345656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa72810d820 0x7fa72819dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:05.734 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.731+0000 7fa72659c700 1 -- 172.21.15.77:0/2654345656 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7180095f0 con 0x7fa7281007b0 2023-12-01T01:17:05.734 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.731+0000 7fa72659c700 1 --2- 172.21.15.77:0/2654345656 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7281007b0 0x7fa728199100 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa7180095c0 tx=0x7fa71800f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:05.734 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.731+0000 7fa7177fe700 1 -- 172.21.15.77:0/2654345656 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71801d070 con 0x7fa7281007b0 2023-12-01T01:17:05.734 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.732+0000 7fa72ce59700 1 -- 172.21.15.77:0/2654345656 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa72819e670 con 0x7fa7281007b0 2023-12-01T01:17:05.734 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.732+0000 7fa7177fe700 1 -- 172.21.15.77:0/2654345656 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7180054f0 con 0x7fa7281007b0 2023-12-01T01:17:05.734 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.732+0000 7fa7177fe700 1 -- 172.21.15.77:0/2654345656 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa718004a70 con 0x7fa7281007b0 2023-12-01T01:17:05.735 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.732+0000 7fa72ce59700 1 -- 172.21.15.77:0/2654345656 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7281a5f30 con 0x7fa7281007b0 2023-12-01T01:17:05.735 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.733+0000 7fa72ce59700 1 -- 172.21.15.77:0/2654345656 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa72804ea90 con 0x7fa7281007b0 2023-12-01T01:17:05.735 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.734+0000 7fa7177fe700 1 -- 172.21.15.77:0/2654345656 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fa718004bd0 con 0x7fa7281007b0 2023-12-01T01:17:05.735 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.734+0000 7fa7177fe700 1 --2- 172.21.15.77:0/2654345656 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa7100a05b0 0x7fa7100a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:05.736 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.734+0000 7fa725d9b700 1 --2- 172.21.15.77:0/2654345656 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa7100a05b0 0x7fa7100a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:05.736 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.734+0000 7fa7177fe700 1 -- 172.21.15.77:0/2654345656 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa7180cc670 con 0x7fa7281007b0 2023-12-01T01:17:05.736 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.735+0000 7fa725d9b700 1 --2- 172.21.15.77:0/2654345656 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa7100a05b0 0x7fa7100a2a70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa72819a720 tx=0x7fa71c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:05.740 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.738+0000 7fa7177fe700 1 -- 172.21.15.77:0/2654345656 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa7180962f0 con 0x7fa7281007b0 2023-12-01T01:17:05.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.778+0000 7f9c118d8700 1 -- 172.21.15.77:0/4075344681 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f9bf8005cc0 con 0x7f9c0c1018f0 2023-12-01T01:17:05.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.778+0000 7f9c108d6700 1 -- 172.21.15.77:0/4075344681 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9c0009c210 con 0x7f9c0c1018f0 2023-12-01T01:17:05.779 INFO:teuthology.orchestra.run.smithi077.stdout:244813135903 2023-12-01T01:17:05.782 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.781+0000 7f9c118d8700 1 -- 172.21.15.77:0/4075344681 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f9bf40a05b0 msgr2=0x7f9bf40a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:05.782 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.781+0000 7f9c118d8700 1 --2- 172.21.15.77:0/4075344681 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f9bf40a05b0 0x7f9bf40a2a70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc005950 tx=0x7f9bfc0058e0 comp rx=0 tx=0).stop 2023-12-01T01:17:05.782 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.781+0000 7f9c118d8700 1 -- 172.21.15.77:0/4075344681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c0c1018f0 msgr2=0x7f9c0c196340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:05.782 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.781+0000 7f9c118d8700 1 --2- 172.21.15.77:0/4075344681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c0c1018f0 0x7f9c0c196340 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9c00009990 tx=0x7f9c00009580 comp rx=0 tx=0).stop 2023-12-01T01:17:05.785 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.784+0000 7f9c118d8700 1 -- 172.21.15.77:0/4075344681 shutdown_connections 2023-12-01T01:17:05.785 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.784+0000 7f9c118d8700 1 --2- 172.21.15.77:0/4075344681 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f9bf40a05b0 0x7f9bf40a2a70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:05.785 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.784+0000 7f9c118d8700 1 --2- 172.21.15.77:0/4075344681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c0c1022b0 0x7f9c0c19a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:05.785 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.784+0000 7f9c118d8700 1 --2- 172.21.15.77:0/4075344681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9c0c100010 0x7f9c0c195e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:05.785 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.784+0000 7f9c118d8700 1 --2- 172.21.15.77:0/4075344681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c0c1018f0 0x7f9c0c196340 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:05.785 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.784+0000 7f9c118d8700 1 -- 172.21.15.77:0/4075344681 >> 172.21.15.77:0/4075344681 conn(0x7f9c0c076d80 msgr2=0x7f9c0c0fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:05.785 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.784+0000 7f9c118d8700 1 -- 172.21.15.77:0/4075344681 shutdown_connections 2023-12-01T01:17:05.785 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.784+0000 7f9c118d8700 1 -- 172.21.15.77:0/4075344681 wait complete. 2023-12-01T01:17:05.898 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:05 smithi077 ceph-mon[98867]: pgmap v272: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2023-12-01T01:17:05.899 INFO:tasks.cephadm.ceph_manager.ceph:need seq 330712481809 got 330712481811 for osd.11 2023-12-01T01:17:05.899 DEBUG:teuthology.parallel:result is None 2023-12-01T01:17:05.952 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.951+0000 7fa72ce59700 1 -- 172.21.15.77:0/2654345656 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa7281a60c0 con 0x7fa7281007b0 2023-12-01T01:17:05.952 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.951+0000 7fa7177fe700 1 -- 172.21.15.77:0/2654345656 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa718099910 con 0x7fa7281007b0 2023-12-01T01:17:05.953 INFO:teuthology.orchestra.run.smithi077.stdout:77309411385 2023-12-01T01:17:05.954 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.953+0000 7fa72ce59700 1 -- 172.21.15.77:0/2654345656 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa7100a05b0 msgr2=0x7fa7100a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:05.955 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.953+0000 7fa72ce59700 1 --2- 172.21.15.77:0/2654345656 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa7100a05b0 0x7fa7100a2a70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa72819a720 tx=0x7fa71c00b410 comp rx=0 tx=0).stop 2023-12-01T01:17:05.955 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.953+0000 7fa72ce59700 1 -- 172.21.15.77:0/2654345656 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7281007b0 msgr2=0x7fa728199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:05.955 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.953+0000 7fa72ce59700 1 --2- 172.21.15.77:0/2654345656 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7281007b0 0x7fa728199100 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa7180095c0 tx=0x7fa71800f710 comp rx=0 tx=0).stop 2023-12-01T01:17:05.955 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.953+0000 7fa72ce59700 1 -- 172.21.15.77:0/2654345656 shutdown_connections 2023-12-01T01:17:05.955 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.953+0000 7fa72ce59700 1 --2- 172.21.15.77:0/2654345656 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa7100a05b0 0x7fa7100a2a70 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:05.955 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.953+0000 7fa72ce59700 1 --2- 172.21.15.77:0/2654345656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa72810d820 0x7fa72819dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:05.955 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.953+0000 7fa72ce59700 1 --2- 172.21.15.77:0/2654345656 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7281007b0 0x7fa728199100 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:05.955 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.953+0000 7fa72ce59700 1 --2- 172.21.15.77:0/2654345656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7281010d0 0x7fa728199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:05.955 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.953+0000 7fa72ce59700 1 -- 172.21.15.77:0/2654345656 >> 172.21.15.77:0/2654345656 conn(0x7fa728076d90 msgr2=0x7fa72810ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:05.955 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.954+0000 7fa72ce59700 1 -- 172.21.15.77:0/2654345656 shutdown_connections 2023-12-01T01:17:05.955 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.954+0000 7fa72ce59700 1 -- 172.21.15.77:0/2654345656 wait complete. 2023-12-01T01:17:05.978 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.977+0000 7f639dd1a700 1 -- 172.21.15.77:0/3120181576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6398069270 msgr2=0x7f639810dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:05.979 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.977+0000 7f639dd1a700 1 --2- 172.21.15.77:0/3120181576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6398069270 0x7f639810dbd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f639000b0a0 tx=0x7f639000b3b0 comp rx=0 tx=0).stop 2023-12-01T01:17:05.979 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.977+0000 7f639dd1a700 1 -- 172.21.15.77:0/3120181576 shutdown_connections 2023-12-01T01:17:05.979 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.977+0000 7f639dd1a700 1 --2- 172.21.15.77:0/3120181576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6398069270 0x7f639810dbd0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:05.979 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.977+0000 7f639dd1a700 1 --2- 172.21.15.77:0/3120181576 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6398106f20 0x7f6398107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:05.979 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.977+0000 7f639dd1a700 1 --2- 172.21.15.77:0/3120181576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63980688b0 0x7f6398068d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:05.979 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.977+0000 7f639dd1a700 1 -- 172.21.15.77:0/3120181576 >> 172.21.15.77:0/3120181576 conn(0x7f6398076d90 msgr2=0x7f63980771a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:05.979 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.977+0000 7f639dd1a700 1 -- 172.21.15.77:0/3120181576 shutdown_connections 2023-12-01T01:17:05.979 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.977+0000 7f639dd1a700 1 -- 172.21.15.77:0/3120181576 wait complete. 2023-12-01T01:17:05.981 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.978+0000 7f639dd1a700 1 Processor -- start 2023-12-01T01:17:05.981 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.978+0000 7f639dd1a700 1 -- start start 2023-12-01T01:17:05.981 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.978+0000 7f639dd1a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63980688b0 0x7f6398196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:05.981 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.978+0000 7f639dd1a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6398069270 0x7f6398197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:05.981 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.978+0000 7f639dd1a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6398106f20 0x7f639819b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:05.981 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.978+0000 7f639dd1a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639819be90 con 0x7f63980688b0 2023-12-01T01:17:05.981 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.978+0000 7f639dd1a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639819c000 con 0x7f6398106f20 2023-12-01T01:17:05.982 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.978+0000 7f639dd1a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639819c1c0 con 0x7f6398069270 2023-12-01T01:17:05.982 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.978+0000 7f63977fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63980688b0 0x7f6398196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:05.982 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.978+0000 7f63977fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63980688b0 0x7f6398196ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58068/0 (socket says 172.21.15.77:58068) 2023-12-01T01:17:05.982 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.978+0000 7f63977fe700 1 -- 172.21.15.77:0/3712629891 learned_addr learned my addr 172.21.15.77:0/3712629891 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:05.982 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.978+0000 7f6397fff700 1 --2- 172.21.15.77:0/3712629891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6398106f20 0x7f639819b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:05.982 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.978+0000 7f6396ffd700 1 --2- 172.21.15.77:0/3712629891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6398069270 0x7f6398197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:05.982 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.979+0000 7f63977fe700 1 -- 172.21.15.77:0/3712629891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6398069270 msgr2=0x7f6398197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:05.982 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.979+0000 7f63977fe700 1 --2- 172.21.15.77:0/3712629891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6398069270 0x7f6398197410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:05.982 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.979+0000 7f63977fe700 1 -- 172.21.15.77:0/3712629891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6398106f20 msgr2=0x7f639819b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:05.982 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.979+0000 7f63977fe700 1 --2- 172.21.15.77:0/3712629891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6398106f20 0x7f639819b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:05.982 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.979+0000 7f63977fe700 1 -- 172.21.15.77:0/3712629891 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6390009d00 con 0x7f63980688b0 2023-12-01T01:17:05.982 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.979+0000 7f6397fff700 1 --2- 172.21.15.77:0/3712629891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6398106f20 0x7f639819b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:17:05.983 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.979+0000 7f63977fe700 1 --2- 172.21.15.77:0/3712629891 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63980688b0 0x7f6398196ed0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f638800d950 tx=0x7f638800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:05.983 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.979+0000 7f6394ff9700 1 -- 172.21.15.77:0/3712629891 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6388009b40 con 0x7f63980688b0 2023-12-01T01:17:05.983 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.979+0000 7f639dd1a700 1 -- 172.21.15.77:0/3712629891 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f639819c4a0 con 0x7f63980688b0 2023-12-01T01:17:05.983 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.979+0000 7f6394ff9700 1 -- 172.21.15.77:0/3712629891 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6388009ca0 con 0x7f63980688b0 2023-12-01T01:17:05.983 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.979+0000 7f6394ff9700 1 -- 172.21.15.77:0/3712629891 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638800f620 con 0x7f63980688b0 2023-12-01T01:17:05.983 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.979+0000 7f639dd1a700 1 -- 172.21.15.77:0/3712629891 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63981a3d00 con 0x7f63980688b0 2023-12-01T01:17:05.983 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.980+0000 7f639dd1a700 1 -- 172.21.15.77:0/3712629891 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f639804ea90 con 0x7f63980688b0 2023-12-01T01:17:05.984 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.981+0000 7f6394ff9700 1 -- 172.21.15.77:0/3712629891 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f6388010460 con 0x7f63980688b0 2023-12-01T01:17:05.984 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.981+0000 7f6394ff9700 1 --2- 172.21.15.77:0/3712629891 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f63800a05b0 0x7f63800a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:05.984 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.982+0000 7f6396ffd700 1 --2- 172.21.15.77:0/3712629891 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f63800a05b0 0x7f63800a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:05.984 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.982+0000 7f6394ff9700 1 -- 172.21.15.77:0/3712629891 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f63880cbd60 con 0x7f63980688b0 2023-12-01T01:17:05.984 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.982+0000 7f6396ffd700 1 --2- 172.21.15.77:0/3712629891 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f63800a05b0 0x7f63800a2a70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f63981984f0 tx=0x7f638c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:05.986 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:05.985+0000 7f6394ff9700 1 -- 172.21.15.77:0/3712629891 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6388095950 con 0x7f63980688b0 2023-12-01T01:17:06.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:05 smithi203 ceph-mon[103581]: pgmap v272: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2023-12-01T01:17:06.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:05 smithi137 ceph-mon[103312]: pgmap v272: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2023-12-01T01:17:06.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.193+0000 7f639dd1a700 1 -- 172.21.15.77:0/3712629891 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f63980ffd10 con 0x7f63980688b0 2023-12-01T01:17:06.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.193+0000 7f6394ff9700 1 -- 172.21.15.77:0/3712629891 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6388020020 con 0x7f63980688b0 2023-12-01T01:17:06.195 INFO:teuthology.orchestra.run.smithi077.stdout:158913789996 2023-12-01T01:17:06.197 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.195+0000 7f639dd1a700 1 -- 172.21.15.77:0/3712629891 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f63800a05b0 msgr2=0x7f63800a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:06.197 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.195+0000 7f639dd1a700 1 --2- 172.21.15.77:0/3712629891 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f63800a05b0 0x7f63800a2a70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f63981984f0 tx=0x7f638c009260 comp rx=0 tx=0).stop 2023-12-01T01:17:06.197 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.195+0000 7f639dd1a700 1 -- 172.21.15.77:0/3712629891 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63980688b0 msgr2=0x7f6398196ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:06.197 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.195+0000 7f639dd1a700 1 --2- 172.21.15.77:0/3712629891 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63980688b0 0x7f6398196ed0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f638800d950 tx=0x7f638800dd10 comp rx=0 tx=0).stop 2023-12-01T01:17:06.197 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.195+0000 7f639dd1a700 1 -- 172.21.15.77:0/3712629891 shutdown_connections 2023-12-01T01:17:06.197 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.195+0000 7f639dd1a700 1 --2- 172.21.15.77:0/3712629891 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f63800a05b0 0x7f63800a2a70 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:06.197 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.195+0000 7f639dd1a700 1 --2- 172.21.15.77:0/3712629891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6398106f20 0x7f639819b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:06.197 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.195+0000 7f639dd1a700 1 --2- 172.21.15.77:0/3712629891 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63980688b0 0x7f6398196ed0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:06.197 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.195+0000 7f639dd1a700 1 --2- 172.21.15.77:0/3712629891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6398069270 0x7f6398197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:06.197 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.195+0000 7f639dd1a700 1 -- 172.21.15.77:0/3712629891 >> 172.21.15.77:0/3712629891 conn(0x7f6398076d90 msgr2=0x7f6398104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:06.197 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.196+0000 7f639dd1a700 1 -- 172.21.15.77:0/3712629891 shutdown_connections 2023-12-01T01:17:06.197 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.196+0000 7f639dd1a700 1 -- 172.21.15.77:0/3712629891 wait complete. 2023-12-01T01:17:06.406 INFO:tasks.cephadm.ceph_manager.ceph:need seq 244813135901 got 244813135903 for osd.8 2023-12-01T01:17:06.406 DEBUG:teuthology.parallel:result is None 2023-12-01T01:17:06.896 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:06 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/4075344681' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-12-01T01:17:06.896 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:06 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/2654345656' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-12-01T01:17:06.896 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:06 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/3712629891' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-12-01T01:17:06.897 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411383 got 77309411385 for osd.2 2023-12-01T01:17:06.897 DEBUG:teuthology.parallel:result is None 2023-12-01T01:17:06.898 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.896+0000 7f51d599b700 1 -- 172.21.15.77:0/1708906118 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51d0102000 msgr2=0x7f51d01023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:06.898 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.897+0000 7f51d599b700 1 --2- 172.21.15.77:0/1708906118 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51d0102000 0x7f51d01023e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f51c0009990 tx=0x7f51c0009ca0 comp rx=0 tx=0).stop 2023-12-01T01:17:06.898 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.897+0000 7f51d599b700 1 -- 172.21.15.77:0/1708906118 shutdown_connections 2023-12-01T01:17:06.898 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.897+0000 7f51d599b700 1 --2- 172.21.15.77:0/1708906118 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51d010b3e0 0x7f51d010d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:06.898 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.897+0000 7f51d599b700 1 --2- 172.21.15.77:0/1708906118 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51d0102000 0x7f51d01023e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:06.898 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.897+0000 7f51d599b700 1 --2- 172.21.15.77:0/1708906118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51d0102920 0x7f51d010ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:06.898 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.897+0000 7f51d599b700 1 -- 172.21.15.77:0/1708906118 >> 172.21.15.77:0/1708906118 conn(0x7f51d00fb750 msgr2=0x7f51d00fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:06.898 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.897+0000 7f51d599b700 1 -- 172.21.15.77:0/1708906118 shutdown_connections 2023-12-01T01:17:06.898 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.897+0000 7f51d599b700 1 -- 172.21.15.77:0/1708906118 wait complete. 2023-12-01T01:17:06.901 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.897+0000 7f51d599b700 1 Processor -- start 2023-12-01T01:17:06.901 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.897+0000 7f51d599b700 1 -- start start 2023-12-01T01:17:06.901 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.898+0000 7f51d599b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51d0102000 0x7f51d0196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:06.901 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.898+0000 7f51d599b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51d0102920 0x7f51d0197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:06.901 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.898+0000 7f51d599b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51d010b3e0 0x7f51d019b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:06.901 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.898+0000 7f51d599b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d019bef0 con 0x7f51d0102920 2023-12-01T01:17:06.901 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.898+0000 7f51d599b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d019c060 con 0x7f51d010b3e0 2023-12-01T01:17:06.901 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.898+0000 7f51d599b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d019c220 con 0x7f51d0102000 2023-12-01T01:17:06.901 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.898+0000 7f51ce7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51d0102920 0x7f51d0197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:06.902 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.898+0000 7f51ce7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51d0102920 0x7f51d0197470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58084/0 (socket says 172.21.15.77:58084) 2023-12-01T01:17:06.902 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.898+0000 7f51ce7fc700 1 -- 172.21.15.77:0/398636634 learned_addr learned my addr 172.21.15.77:0/398636634 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:06.902 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.898+0000 7f51ceffd700 1 --2- 172.21.15.77:0/398636634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51d0102000 0x7f51d0196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:06.902 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.898+0000 7f51cf7fe700 1 --2- 172.21.15.77:0/398636634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51d010b3e0 0x7f51d019b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:06.902 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.898+0000 7f51ce7fc700 1 -- 172.21.15.77:0/398636634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51d0102000 msgr2=0x7f51d0196f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:06.902 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.898+0000 7f51ce7fc700 1 --2- 172.21.15.77:0/398636634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51d0102000 0x7f51d0196f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:06.902 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.898+0000 7f51ce7fc700 1 -- 172.21.15.77:0/398636634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51d010b3e0 msgr2=0x7f51d019b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:06.902 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.898+0000 7f51ce7fc700 1 --2- 172.21.15.77:0/398636634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51d010b3e0 0x7f51d019b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:06.902 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.898+0000 7f51ce7fc700 1 -- 172.21.15.77:0/398636634 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c00095f0 con 0x7f51d0102920 2023-12-01T01:17:06.902 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.898+0000 7f51ce7fc700 1 --2- 172.21.15.77:0/398636634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51d0102920 0x7f51d0197470 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f51c400d980 tx=0x7f51c400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:06.902 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.898+0000 7f51d4999700 1 -- 172.21.15.77:0/398636634 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c4009930 con 0x7f51d0102920 2023-12-01T01:17:06.902 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.898+0000 7f51d4999700 1 -- 172.21.15.77:0/398636634 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51c4010460 con 0x7f51d0102920 2023-12-01T01:17:06.902 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.898+0000 7f51d4999700 1 -- 172.21.15.77:0/398636634 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c400f620 con 0x7f51d0102920 2023-12-01T01:17:06.903 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.898+0000 7f51d599b700 1 -- 172.21.15.77:0/398636634 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51d019c500 con 0x7f51d0102920 2023-12-01T01:17:06.903 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.898+0000 7f51d599b700 1 -- 172.21.15.77:0/398636634 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d01a3d60 con 0x7f51d0102920 2023-12-01T01:17:06.903 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.899+0000 7f51d599b700 1 -- 172.21.15.77:0/398636634 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51d004ea90 con 0x7f51d0102920 2023-12-01T01:17:06.903 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.900+0000 7f51d4999700 1 -- 172.21.15.77:0/398636634 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f51c40108d0 con 0x7f51d0102920 2023-12-01T01:17:06.903 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.901+0000 7f51d4999700 1 --2- 172.21.15.77:0/398636634 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f51b80a0630 0x7f51b80a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:06.904 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.901+0000 7f51ceffd700 1 --2- 172.21.15.77:0/398636634 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f51b80a0630 0x7f51b80a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:06.904 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.901+0000 7f51d4999700 1 -- 172.21.15.77:0/398636634 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f51c40cb3c0 con 0x7f51d0102920 2023-12-01T01:17:06.904 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.902+0000 7f51ceffd700 1 --2- 172.21.15.77:0/398636634 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f51b80a0630 0x7f51b80a2af0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f51c000b5c0 tx=0x7f51c0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:06.906 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:06.905+0000 7f51d4999700 1 -- 172.21.15.77:0/398636634 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f51c4095030 con 0x7f51d0102920 2023-12-01T01:17:06.982 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913789994 got 158913789996 for osd.5 2023-12-01T01:17:06.983 DEBUG:teuthology.parallel:result is None 2023-12-01T01:17:07.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:06 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/4075344681' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-12-01T01:17:07.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:06 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/2654345656' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-12-01T01:17:07.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:06 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/3712629891' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-12-01T01:17:07.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:06 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/4075344681' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-12-01T01:17:07.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:06 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/2654345656' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-12-01T01:17:07.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:06 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/3712629891' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-12-01T01:17:07.124 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.122+0000 7f51d599b700 1 -- 172.21.15.77:0/398636634 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f51d0198290 con 0x7f51d0102920 2023-12-01T01:17:07.124 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.123+0000 7f51d4999700 1 -- 172.21.15.77:0/398636634 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f51c4017020 con 0x7f51d0102920 2023-12-01T01:17:07.124 INFO:teuthology.orchestra.run.smithi077.stdout:55834574909 2023-12-01T01:17:07.126 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.125+0000 7f51d599b700 1 -- 172.21.15.77:0/398636634 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f51b80a0630 msgr2=0x7f51b80a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:07.126 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.125+0000 7f51d599b700 1 --2- 172.21.15.77:0/398636634 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f51b80a0630 0x7f51b80a2af0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f51c000b5c0 tx=0x7f51c0005fd0 comp rx=0 tx=0).stop 2023-12-01T01:17:07.126 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.125+0000 7f51d599b700 1 -- 172.21.15.77:0/398636634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51d0102920 msgr2=0x7f51d0197470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:07.126 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.125+0000 7f51d599b700 1 --2- 172.21.15.77:0/398636634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51d0102920 0x7f51d0197470 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f51c400d980 tx=0x7f51c400dc90 comp rx=0 tx=0).stop 2023-12-01T01:17:07.126 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.125+0000 7f51d599b700 1 -- 172.21.15.77:0/398636634 shutdown_connections 2023-12-01T01:17:07.127 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.125+0000 7f51d599b700 1 --2- 172.21.15.77:0/398636634 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f51b80a0630 0x7f51b80a2af0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:07.127 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.125+0000 7f51d599b700 1 --2- 172.21.15.77:0/398636634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51d010b3e0 0x7f51d019b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:07.127 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.125+0000 7f51d599b700 1 --2- 172.21.15.77:0/398636634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51d0102000 0x7f51d0196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:07.127 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.125+0000 7f51d599b700 1 --2- 172.21.15.77:0/398636634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51d0102920 0x7f51d0197470 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:07.127 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.125+0000 7f51d599b700 1 -- 172.21.15.77:0/398636634 >> 172.21.15.77:0/398636634 conn(0x7f51d00fb750 msgr2=0x7f51d010d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:07.127 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.125+0000 7f51d599b700 1 -- 172.21.15.77:0/398636634 shutdown_connections 2023-12-01T01:17:07.127 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.125+0000 7f51d599b700 1 -- 172.21.15.77:0/398636634 wait complete. 2023-12-01T01:17:07.182 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.178+0000 7f36c0627700 1 -- 172.21.15.77:0/1070662415 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36b810d9d0 msgr2=0x7f36b810fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:07.183 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.178+0000 7f36c0627700 1 --2- 172.21.15.77:0/1070662415 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36b810d9d0 0x7f36b810fdc0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f36b000b0a0 tx=0x7f36b000b3b0 comp rx=0 tx=0).stop 2023-12-01T01:17:07.183 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.178+0000 7f36c0627700 1 -- 172.21.15.77:0/1070662415 shutdown_connections 2023-12-01T01:17:07.183 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.178+0000 7f36c0627700 1 --2- 172.21.15.77:0/1070662415 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36b810d9d0 0x7f36b810fdc0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:07.183 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.178+0000 7f36c0627700 1 --2- 172.21.15.77:0/1070662415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36b8072fc0 0x7f36b80733a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:07.183 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.178+0000 7f36c0627700 1 --2- 172.21.15.77:0/1070662415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36b80738e0 0x7f36b810d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:07.183 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.178+0000 7f36c0627700 1 -- 172.21.15.77:0/1070662415 >> 172.21.15.77:0/1070662415 conn(0x7f36b8078db0 msgr2=0x7f36b80791c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:07.183 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.178+0000 7f36c0627700 1 -- 172.21.15.77:0/1070662415 shutdown_connections 2023-12-01T01:17:07.184 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.178+0000 7f36c0627700 1 -- 172.21.15.77:0/1070662415 wait complete. 2023-12-01T01:17:07.184 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.179+0000 7f36c0627700 1 Processor -- start 2023-12-01T01:17:07.184 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.179+0000 7f36c0627700 1 -- start start 2023-12-01T01:17:07.184 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.179+0000 7f36c0627700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36b8072fc0 0x7f36b81991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:07.184 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.179+0000 7f36c0627700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36b80738e0 0x7f36b81996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:07.184 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.179+0000 7f36c0627700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36b810d9d0 0x7f36b819db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:07.184 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.179+0000 7f36c0627700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b819e160 con 0x7f36b80738e0 2023-12-01T01:17:07.184 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.179+0000 7f36c0627700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b819e2d0 con 0x7f36b8072fc0 2023-12-01T01:17:07.184 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.179+0000 7f36c0627700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b819e490 con 0x7f36b810d9d0 2023-12-01T01:17:07.184 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.179+0000 7f36bdbc2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36b80738e0 0x7f36b81996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:07.185 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.179+0000 7f36bdbc2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36b80738e0 0x7f36b81996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58098/0 (socket says 172.21.15.77:58098) 2023-12-01T01:17:07.185 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.179+0000 7f36bdbc2700 1 -- 172.21.15.77:0/1629832984 learned_addr learned my addr 172.21.15.77:0/1629832984 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:07.185 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.180+0000 7f36be3c3700 1 --2- 172.21.15.77:0/1629832984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36b8072fc0 0x7f36b81991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:07.185 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.180+0000 7f36bebc4700 1 --2- 172.21.15.77:0/1629832984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36b810d9d0 0x7f36b819db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:07.185 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.180+0000 7f36bdbc2700 1 -- 172.21.15.77:0/1629832984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36b810d9d0 msgr2=0x7f36b819db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:07.185 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.180+0000 7f36bdbc2700 1 --2- 172.21.15.77:0/1629832984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36b810d9d0 0x7f36b819db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:07.185 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.180+0000 7f36bdbc2700 1 -- 172.21.15.77:0/1629832984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36b8072fc0 msgr2=0x7f36b81991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:07.185 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.180+0000 7f36bdbc2700 1 --2- 172.21.15.77:0/1629832984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36b8072fc0 0x7f36b81991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:07.186 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.180+0000 7f36bdbc2700 1 -- 172.21.15.77:0/1629832984 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b0009d00 con 0x7f36b80738e0 2023-12-01T01:17:07.187 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.180+0000 7f36bdbc2700 1 --2- 172.21.15.77:0/1629832984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36b80738e0 0x7f36b81996e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f36a800e9c0 tx=0x7f36a800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:07.187 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.180+0000 7f36af7fe700 1 -- 172.21.15.77:0/1629832984 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a80097b0 con 0x7f36b80738e0 2023-12-01T01:17:07.187 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.180+0000 7f36c0627700 1 -- 172.21.15.77:0/1629832984 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36b819e770 con 0x7f36b80738e0 2023-12-01T01:17:07.187 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.180+0000 7f36af7fe700 1 -- 172.21.15.77:0/1629832984 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36a8004cf0 con 0x7f36b80738e0 2023-12-01T01:17:07.187 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.180+0000 7f36af7fe700 1 -- 172.21.15.77:0/1629832984 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a8005600 con 0x7f36b80738e0 2023-12-01T01:17:07.188 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.180+0000 7f36c0627700 1 -- 172.21.15.77:0/1629832984 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36b81a5fd0 con 0x7f36b80738e0 2023-12-01T01:17:07.188 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.181+0000 7f36c0627700 1 -- 172.21.15.77:0/1629832984 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36b804ea90 con 0x7f36b80738e0 2023-12-01T01:17:07.188 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.182+0000 7f36af7fe700 1 -- 172.21.15.77:0/1629832984 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f36a800cc60 con 0x7f36b80738e0 2023-12-01T01:17:07.188 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.183+0000 7f36af7fe700 1 --2- 172.21.15.77:0/1629832984 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f36a40a05b0 0x7f36a40a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:07.188 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.183+0000 7f36af7fe700 1 -- 172.21.15.77:0/1629832984 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f36a8014070 con 0x7f36b80738e0 2023-12-01T01:17:07.188 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.183+0000 7f36be3c3700 1 --2- 172.21.15.77:0/1629832984 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f36a40a05b0 0x7f36a40a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:07.188 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.183+0000 7f36be3c3700 1 --2- 172.21.15.77:0/1629832984 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f36a40a05b0 0x7f36a40a2a70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f36b4007be0 tx=0x7f36b40073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:07.191 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.190+0000 7f36af7fe700 1 -- 172.21.15.77:0/1629832984 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f36a80959b0 con 0x7f36b80738e0 2023-12-01T01:17:07.394 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.393+0000 7f36c0627700 1 -- 172.21.15.77:0/1629832984 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f36b819a500 con 0x7f36b80738e0 2023-12-01T01:17:07.394 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.393+0000 7f36af7fe700 1 -- 172.21.15.77:0/1629832984 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f36a8098fd0 con 0x7f36b80738e0 2023-12-01T01:17:07.395 INFO:teuthology.orchestra.run.smithi077.stdout:296352743447 2023-12-01T01:17:07.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.395+0000 7f36c0627700 1 -- 172.21.15.77:0/1629832984 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f36a40a05b0 msgr2=0x7f36a40a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:07.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.395+0000 7f36c0627700 1 --2- 172.21.15.77:0/1629832984 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f36a40a05b0 0x7f36a40a2a70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f36b4007be0 tx=0x7f36b40073d0 comp rx=0 tx=0).stop 2023-12-01T01:17:07.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.395+0000 7f36c0627700 1 -- 172.21.15.77:0/1629832984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36b80738e0 msgr2=0x7f36b81996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:07.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.395+0000 7f36c0627700 1 --2- 172.21.15.77:0/1629832984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36b80738e0 0x7f36b81996e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f36a800e9c0 tx=0x7f36a800ecd0 comp rx=0 tx=0).stop 2023-12-01T01:17:07.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.395+0000 7f36c0627700 1 -- 172.21.15.77:0/1629832984 shutdown_connections 2023-12-01T01:17:07.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.395+0000 7f36c0627700 1 --2- 172.21.15.77:0/1629832984 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f36a40a05b0 0x7f36a40a2a70 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:07.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.395+0000 7f36c0627700 1 --2- 172.21.15.77:0/1629832984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36b810d9d0 0x7f36b819db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:07.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.395+0000 7f36c0627700 1 --2- 172.21.15.77:0/1629832984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36b8072fc0 0x7f36b81991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:07.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.395+0000 7f36c0627700 1 --2- 172.21.15.77:0/1629832984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36b80738e0 0x7f36b81996e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:07.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.395+0000 7f36c0627700 1 -- 172.21.15.77:0/1629832984 >> 172.21.15.77:0/1629832984 conn(0x7f36b8078db0 msgr2=0x7f36b80fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:07.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.395+0000 7f36c0627700 1 -- 172.21.15.77:0/1629832984 shutdown_connections 2023-12-01T01:17:07.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:07.395+0000 7f36c0627700 1 -- 172.21.15.77:0/1629832984 wait complete. 2023-12-01T01:17:07.486 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574907 got 55834574909 for osd.1 2023-12-01T01:17:07.486 DEBUG:teuthology.parallel:result is None 2023-12-01T01:17:07.718 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:07 smithi077 ceph-mon[98867]: pgmap v273: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 8 op/s 2023-12-01T01:17:07.719 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:07 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/398636634' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-12-01T01:17:07.719 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:07 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/1629832984' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-12-01T01:17:07.720 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743445 got 296352743447 for osd.10 2023-12-01T01:17:07.720 DEBUG:teuthology.parallel:result is None 2023-12-01T01:17:07.720 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-12-01T01:17:07.720 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph pg dump --format=json 2023-12-01T01:17:08.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:07 smithi203 ceph-mon[103581]: pgmap v273: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 8 op/s 2023-12-01T01:17:08.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:07 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/398636634' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-12-01T01:17:08.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:07 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/1629832984' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-12-01T01:17:08.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:07 smithi137 ceph-mon[103312]: pgmap v273: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 8 op/s 2023-12-01T01:17:08.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:07 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/398636634' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-12-01T01:17:08.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:07 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/1629832984' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-12-01T01:17:08.275 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:17:09.479 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.476+0000 7f6ae3300700 1 -- 172.21.15.77:0/3535297548 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6adc1019b0 msgr2=0x7f6adc105a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:09.479 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.476+0000 7f6ae3300700 1 --2- 172.21.15.77:0/3535297548 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6adc1019b0 0x7f6adc105a00 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8009990 tx=0x7f6ad8009ca0 comp rx=0 tx=0).stop 2023-12-01T01:17:09.480 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.476+0000 7f6ae3300700 1 -- 172.21.15.77:0/3535297548 shutdown_connections 2023-12-01T01:17:09.480 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.476+0000 7f6ae3300700 1 --2- 172.21.15.77:0/3535297548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6adc1060e0 0x7f6adc10dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:09.480 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.476+0000 7f6ae3300700 1 --2- 172.21.15.77:0/3535297548 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6adc101000 0x7f6adc1013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:09.480 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.476+0000 7f6ae3300700 1 --2- 172.21.15.77:0/3535297548 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6adc1019b0 0x7f6adc105a00 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:09.480 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.476+0000 7f6ae3300700 1 -- 172.21.15.77:0/3535297548 >> 172.21.15.77:0/3535297548 conn(0x7f6adc0fc830 msgr2=0x7f6adc0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:09.480 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.476+0000 7f6ae3300700 1 -- 172.21.15.77:0/3535297548 shutdown_connections 2023-12-01T01:17:09.480 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.476+0000 7f6ae3300700 1 -- 172.21.15.77:0/3535297548 wait complete. 2023-12-01T01:17:09.480 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.477+0000 7f6ae3300700 1 Processor -- start 2023-12-01T01:17:09.480 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.477+0000 7f6ae3300700 1 -- start start 2023-12-01T01:17:09.480 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.477+0000 7f6ae3300700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6adc101000 0x7f6adc196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:09.480 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.477+0000 7f6ae3300700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6adc1019b0 0x7f6adc197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:09.480 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.477+0000 7f6ae3300700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6adc1060e0 0x7f6adc19b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:09.480 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.477+0000 7f6ae3300700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc19bec0 con 0x7f6adc101000 2023-12-01T01:17:09.480 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.477+0000 7f6ae3300700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc19c030 con 0x7f6adc1060e0 2023-12-01T01:17:09.481 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.477+0000 7f6ae3300700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc19c1f0 con 0x7f6adc1019b0 2023-12-01T01:17:09.481 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.478+0000 7f6ae109c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6adc101000 0x7f6adc196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:09.481 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.478+0000 7f6ae109c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6adc101000 0x7f6adc196f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58110/0 (socket says 172.21.15.77:58110) 2023-12-01T01:17:09.481 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.478+0000 7f6ae109c700 1 -- 172.21.15.77:0/1074155006 learned_addr learned my addr 172.21.15.77:0/1074155006 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:09.481 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.478+0000 7f6ae089b700 1 --2- 172.21.15.77:0/1074155006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6adc1019b0 0x7f6adc197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:09.481 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.478+0000 7f6ae189d700 1 --2- 172.21.15.77:0/1074155006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6adc1060e0 0x7f6adc19b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:09.481 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.478+0000 7f6ae109c700 1 -- 172.21.15.77:0/1074155006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6adc1019b0 msgr2=0x7f6adc197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:09.481 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.478+0000 7f6ae109c700 1 --2- 172.21.15.77:0/1074155006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6adc1019b0 0x7f6adc197440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:09.481 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.478+0000 7f6ae109c700 1 -- 172.21.15.77:0/1074155006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6adc1060e0 msgr2=0x7f6adc19b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:09.481 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.478+0000 7f6ae109c700 1 --2- 172.21.15.77:0/1074155006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6adc1060e0 0x7f6adc19b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:09.481 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.478+0000 7f6ae109c700 1 -- 172.21.15.77:0/1074155006 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad80095f0 con 0x7f6adc101000 2023-12-01T01:17:09.481 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.478+0000 7f6ae109c700 1 --2- 172.21.15.77:0/1074155006 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6adc101000 0x7f6adc196f00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6ad000c920 tx=0x7f6ad000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:09.481 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.478+0000 7f6ace7fc700 1 -- 172.21.15.77:0/1074155006 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0007b00 con 0x7f6adc101000 2023-12-01T01:17:09.481 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.478+0000 7f6ace7fc700 1 -- 172.21.15.77:0/1074155006 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ad000ce70 con 0x7f6adc101000 2023-12-01T01:17:09.481 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.478+0000 7f6ae3300700 1 -- 172.21.15.77:0/1074155006 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6adc19c4d0 con 0x7f6adc101000 2023-12-01T01:17:09.482 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.478+0000 7f6ace7fc700 1 -- 172.21.15.77:0/1074155006 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad00054e0 con 0x7f6adc101000 2023-12-01T01:17:09.482 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.478+0000 7f6ae3300700 1 -- 172.21.15.77:0/1074155006 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6adc1a3d30 con 0x7f6adc101000 2023-12-01T01:17:09.482 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.479+0000 7f6ae3300700 1 -- 172.21.15.77:0/1074155006 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6adc04ea90 con 0x7f6adc101000 2023-12-01T01:17:09.483 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.480+0000 7f6ace7fc700 1 -- 172.21.15.77:0/1074155006 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f6ad0004750 con 0x7f6adc101000 2023-12-01T01:17:09.483 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.481+0000 7f6ace7fc700 1 --2- 172.21.15.77:0/1074155006 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f6ac80a0630 0x7f6ac80a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:09.483 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.481+0000 7f6ae089b700 1 --2- 172.21.15.77:0/1074155006 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f6ac80a0630 0x7f6ac80a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:09.483 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.481+0000 7f6ace7fc700 1 -- 172.21.15.77:0/1074155006 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6ad00cb9d0 con 0x7f6adc101000 2023-12-01T01:17:09.483 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.481+0000 7f6ae089b700 1 --2- 172.21.15.77:0/1074155006 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f6ac80a0630 0x7f6ac80a2af0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6adc198520 tx=0x7f6ad8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:09.485 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.484+0000 7f6ace7fc700 1 -- 172.21.15.77:0/1074155006 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6ad0095540 con 0x7f6adc101000 2023-12-01T01:17:09.670 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.669+0000 7f6ae3300700 1 -- 172.21.15.77:0/1074155006 --> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6adc0fda40 con 0x7f6ac80a0630 2023-12-01T01:17:09.680 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.679+0000 7f6ace7fc700 1 -- 172.21.15.77:0/1074155006 <== mgr.14152 v2:172.21.15.77:6800/3838941679 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340613 (secure 0 0 0) 0x7f6adc0fda40 con 0x7f6ac80a0630 2023-12-01T01:17:09.684 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:17:09.687 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.682+0000 7f6ae3300700 1 -- 172.21.15.77:0/1074155006 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f6ac80a0630 msgr2=0x7f6ac80a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:09.687 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.682+0000 7f6ae3300700 1 --2- 172.21.15.77:0/1074155006 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f6ac80a0630 0x7f6ac80a2af0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6adc198520 tx=0x7f6ad8005fd0 comp rx=0 tx=0).stop 2023-12-01T01:17:09.687 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.682+0000 7f6ae3300700 1 -- 172.21.15.77:0/1074155006 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6adc101000 msgr2=0x7f6adc196f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:09.687 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.682+0000 7f6ae3300700 1 --2- 172.21.15.77:0/1074155006 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6adc101000 0x7f6adc196f00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6ad000c920 tx=0x7f6ad000cc30 comp rx=0 tx=0).stop 2023-12-01T01:17:09.687 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.682+0000 7f6ae3300700 1 -- 172.21.15.77:0/1074155006 shutdown_connections 2023-12-01T01:17:09.687 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.682+0000 7f6ae3300700 1 --2- 172.21.15.77:0/1074155006 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f6ac80a0630 0x7f6ac80a2af0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:09.687 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.682+0000 7f6ae3300700 1 --2- 172.21.15.77:0/1074155006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6adc1060e0 0x7f6adc19b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:09.687 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.682+0000 7f6ae3300700 1 --2- 172.21.15.77:0/1074155006 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6adc101000 0x7f6adc196f00 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:09.688 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.682+0000 7f6ae3300700 1 --2- 172.21.15.77:0/1074155006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6adc1019b0 0x7f6adc197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:09.688 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.682+0000 7f6ae3300700 1 -- 172.21.15.77:0/1074155006 >> 172.21.15.77:0/1074155006 conn(0x7f6adc0fc830 msgr2=0x7f6adc0fd330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:09.688 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.682+0000 7f6ae3300700 1 -- 172.21.15.77:0/1074155006 shutdown_connections 2023-12-01T01:17:09.688 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:09.682+0000 7f6ae3300700 1 -- 172.21.15.77:0/1074155006 wait complete. 2023-12-01T01:17:09.688 INFO:teuthology.orchestra.run.smithi077.stderr:dumped all 2023-12-01T01:17:09.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:09 smithi077 ceph-mon[98867]: pgmap v274: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 10 op/s 2023-12-01T01:17:10.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:09 smithi203 ceph-mon[103581]: pgmap v274: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 10 op/s 2023-12-01T01:17:10.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:09 smithi137 ceph-mon[103312]: pgmap v274: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 10 op/s 2023-12-01T01:17:10.031 INFO:teuthology.orchestra.run.smithi077.stdout:{"pg_ready":true,"pg_map":{"version":274,"stamp":"2023-12-01T01:17:08.276619+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655767,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":507,"num_read_kb":377,"num_write":412,"num_write_kb":3066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":340,"ondisk_log_size":340,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336052,"kb_used_data":12628,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124605772,"statfs":{"total":1151940427776,"available":1151596310528,"internally_reserved":0,"allocated":12931072,"data_stored":7040407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1652,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":122,"num_read_kb":71,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003248"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.592825+0000","last_change":"2023-12-01T01:16:25.373817+0000","last_active":"2023-12-01T01:16:25.592825+0000","last_peered":"2023-12-01T01:16:25.592825+0000","last_clean":"2023-12-01T01:16:25.592825+0000","last_became_active":"2023-12-01T01:16:25.373154+0000","last_became_peered":"2023-12-01T01:16:25.373154+0000","last_unstale":"2023-12-01T01:16:25.592825+0000","last_undegraded":"2023-12-01T01:16:25.592825+0000","last_fullsized":"2023-12-01T01:16:25.592825+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:33:43.284106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.324984+0000","last_change":"2023-12-01T01:16:25.364855+0000","last_active":"2023-12-01T01:16:27.324984+0000","last_peered":"2023-12-01T01:16:27.324984+0000","last_clean":"2023-12-01T01:16:27.324984+0000","last_became_active":"2023-12-01T01:16:25.364536+0000","last_became_peered":"2023-12-01T01:16:25.364536+0000","last_unstale":"2023-12-01T01:16:27.324984+0000","last_undegraded":"2023-12-01T01:16:27.324984+0000","last_fullsized":"2023-12-01T01:16:27.324984+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:45:23.471557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.324702+0000","last_change":"2023-12-01T01:16:25.372443+0000","last_active":"2023-12-01T01:16:27.324702+0000","last_peered":"2023-12-01T01:16:27.324702+0000","last_clean":"2023-12-01T01:16:27.324702+0000","last_became_active":"2023-12-01T01:16:25.372133+0000","last_became_peered":"2023-12-01T01:16:25.372133+0000","last_unstale":"2023-12-01T01:16:27.324702+0000","last_undegraded":"2023-12-01T01:16:27.324702+0000","last_fullsized":"2023-12-01T01:16:27.324702+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:54:19.439408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.554816+0000","last_change":"2023-12-01T01:16:25.365857+0000","last_active":"2023-12-01T01:16:26.554816+0000","last_peered":"2023-12-01T01:16:26.554816+0000","last_clean":"2023-12-01T01:16:26.554816+0000","last_became_active":"2023-12-01T01:16:25.364992+0000","last_became_peered":"2023-12-01T01:16:25.364992+0000","last_unstale":"2023-12-01T01:16:26.554816+0000","last_undegraded":"2023-12-01T01:16:26.554816+0000","last_fullsized":"2023-12-01T01:16:26.554816+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:01:11.064587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.071460+0000","last_change":"2023-12-01T01:16:25.364081+0000","last_active":"2023-12-01T01:16:26.071460+0000","last_peered":"2023-12-01T01:16:26.071460+0000","last_clean":"2023-12-01T01:16:26.071460+0000","last_became_active":"2023-12-01T01:16:25.363187+0000","last_became_peered":"2023-12-01T01:16:25.363187+0000","last_unstale":"2023-12-01T01:16:26.071460+0000","last_undegraded":"2023-12-01T01:16:26.071460+0000","last_fullsized":"2023-12-01T01:16:26.071460+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:05:42.852128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325515+0000","last_change":"2023-12-01T01:16:25.366104+0000","last_active":"2023-12-01T01:16:27.325515+0000","last_peered":"2023-12-01T01:16:27.325515+0000","last_clean":"2023-12-01T01:16:27.325515+0000","last_became_active":"2023-12-01T01:16:25.365778+0000","last_became_peered":"2023-12-01T01:16:25.365778+0000","last_unstale":"2023-12-01T01:16:27.325515+0000","last_undegraded":"2023-12-01T01:16:27.325515+0000","last_fullsized":"2023-12-01T01:16:27.325515+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:30:09.485609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.349881+0000","last_change":"2023-12-01T01:16:25.371354+0000","last_active":"2023-12-01T01:16:26.349881+0000","last_peered":"2023-12-01T01:16:26.349881+0000","last_clean":"2023-12-01T01:16:26.349881+0000","last_became_active":"2023-12-01T01:16:25.370955+0000","last_became_peered":"2023-12-01T01:16:25.370955+0000","last_unstale":"2023-12-01T01:16:26.349881+0000","last_undegraded":"2023-12-01T01:16:26.349881+0000","last_fullsized":"2023-12-01T01:16:26.349881+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:38:53.508285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.571448+0000","last_change":"2023-12-01T01:16:25.368187+0000","last_active":"2023-12-01T01:16:26.571448+0000","last_peered":"2023-12-01T01:16:26.571448+0000","last_clean":"2023-12-01T01:16:26.571448+0000","last_became_active":"2023-12-01T01:16:25.367854+0000","last_became_peered":"2023-12-01T01:16:25.367854+0000","last_unstale":"2023-12-01T01:16:26.571448+0000","last_undegraded":"2023-12-01T01:16:26.571448+0000","last_fullsized":"2023-12-01T01:16:26.571448+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:22:35.159185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325090+0000","last_change":"2023-12-01T01:16:25.372829+0000","last_active":"2023-12-01T01:16:27.325090+0000","last_peered":"2023-12-01T01:16:27.325090+0000","last_clean":"2023-12-01T01:16:27.325090+0000","last_became_active":"2023-12-01T01:16:25.372469+0000","last_became_peered":"2023-12-01T01:16:25.372469+0000","last_unstale":"2023-12-01T01:16:27.325090+0000","last_undegraded":"2023-12-01T01:16:27.325090+0000","last_fullsized":"2023-12-01T01:16:27.325090+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:25:10.300138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325829+0000","last_change":"2023-12-01T01:16:25.379135+0000","last_active":"2023-12-01T01:16:27.325829+0000","last_peered":"2023-12-01T01:16:27.325829+0000","last_clean":"2023-12-01T01:16:27.325829+0000","last_became_active":"2023-12-01T01:16:25.378852+0000","last_became_peered":"2023-12-01T01:16:25.378852+0000","last_unstale":"2023-12-01T01:16:27.325829+0000","last_undegraded":"2023-12-01T01:16:27.325829+0000","last_fullsized":"2023-12-01T01:16:27.325829+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:42:16.803389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.350179+0000","last_change":"2023-12-01T01:16:25.367602+0000","last_active":"2023-12-01T01:16:26.350179+0000","last_peered":"2023-12-01T01:16:26.350179+0000","last_clean":"2023-12-01T01:16:26.350179+0000","last_became_active":"2023-12-01T01:16:25.366565+0000","last_became_peered":"2023-12-01T01:16:25.366565+0000","last_unstale":"2023-12-01T01:16:26.350179+0000","last_undegraded":"2023-12-01T01:16:26.350179+0000","last_fullsized":"2023-12-01T01:16:26.350179+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:28:52.576838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.554806+0000","last_change":"2023-12-01T01:16:25.375004+0000","last_active":"2023-12-01T01:16:26.554806+0000","last_peered":"2023-12-01T01:16:26.554806+0000","last_clean":"2023-12-01T01:16:26.554806+0000","last_became_active":"2023-12-01T01:16:25.374320+0000","last_became_peered":"2023-12-01T01:16:25.374320+0000","last_unstale":"2023-12-01T01:16:26.554806+0000","last_undegraded":"2023-12-01T01:16:26.554806+0000","last_fullsized":"2023-12-01T01:16:26.554806+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:40:49.482061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.352492+0000","last_change":"2023-12-01T01:16:25.367858+0000","last_active":"2023-12-01T01:16:26.352492+0000","last_peered":"2023-12-01T01:16:26.352492+0000","last_clean":"2023-12-01T01:16:26.352492+0000","last_became_active":"2023-12-01T01:16:25.367560+0000","last_became_peered":"2023-12-01T01:16:25.367560+0000","last_unstale":"2023-12-01T01:16:26.352492+0000","last_undegraded":"2023-12-01T01:16:26.352492+0000","last_fullsized":"2023-12-01T01:16:26.352492+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:33:21.048672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.324783+0000","last_change":"2023-12-01T01:16:25.375654+0000","last_active":"2023-12-01T01:16:27.324783+0000","last_peered":"2023-12-01T01:16:27.324783+0000","last_clean":"2023-12-01T01:16:27.324783+0000","last_became_active":"2023-12-01T01:16:25.375379+0000","last_became_peered":"2023-12-01T01:16:25.375379+0000","last_unstale":"2023-12-01T01:16:27.324783+0000","last_undegraded":"2023-12-01T01:16:27.324783+0000","last_fullsized":"2023-12-01T01:16:27.324783+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:33:42.335365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325759+0000","last_change":"2023-12-01T01:16:25.372546+0000","last_active":"2023-12-01T01:16:27.325759+0000","last_peered":"2023-12-01T01:16:27.325759+0000","last_clean":"2023-12-01T01:16:27.325759+0000","last_became_active":"2023-12-01T01:16:25.372327+0000","last_became_peered":"2023-12-01T01:16:25.372327+0000","last_unstale":"2023-12-01T01:16:27.325759+0000","last_undegraded":"2023-12-01T01:16:27.325759+0000","last_fullsized":"2023-12-01T01:16:27.325759+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:12:34.171834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.324599+0000","last_change":"2023-12-01T01:16:25.366449+0000","last_active":"2023-12-01T01:16:27.324599+0000","last_peered":"2023-12-01T01:16:27.324599+0000","last_clean":"2023-12-01T01:16:27.324599+0000","last_became_active":"2023-12-01T01:16:25.366147+0000","last_became_peered":"2023-12-01T01:16:25.366147+0000","last_unstale":"2023-12-01T01:16:27.324599+0000","last_undegraded":"2023-12-01T01:16:27.324599+0000","last_fullsized":"2023-12-01T01:16:27.324599+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:01:29.670329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.592834+0000","last_change":"2023-12-01T01:16:25.374016+0000","last_active":"2023-12-01T01:16:25.592834+0000","last_peered":"2023-12-01T01:16:25.592834+0000","last_clean":"2023-12-01T01:16:25.592834+0000","last_became_active":"2023-12-01T01:16:25.373543+0000","last_became_peered":"2023-12-01T01:16:25.373543+0000","last_unstale":"2023-12-01T01:16:25.592834+0000","last_undegraded":"2023-12-01T01:16:25.592834+0000","last_fullsized":"2023-12-01T01:16:25.592834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:36:22.858446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.584312+0000","last_change":"2023-12-01T01:16:25.363598+0000","last_active":"2023-12-01T01:16:26.584312+0000","last_peered":"2023-12-01T01:16:26.584312+0000","last_clean":"2023-12-01T01:16:26.584312+0000","last_became_active":"2023-12-01T01:16:25.363111+0000","last_became_peered":"2023-12-01T01:16:25.363111+0000","last_unstale":"2023-12-01T01:16:26.584312+0000","last_undegraded":"2023-12-01T01:16:26.584312+0000","last_fullsized":"2023-12-01T01:16:26.584312+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:42:08.171789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.375829+0000","last_change":"2023-12-01T01:16:25.368446+0000","last_active":"2023-12-01T01:16:25.375829+0000","last_peered":"2023-12-01T01:16:25.375829+0000","last_clean":"2023-12-01T01:16:25.375829+0000","last_became_active":"2023-12-01T01:16:25.361893+0000","last_became_peered":"2023-12-01T01:16:25.361893+0000","last_unstale":"2023-12-01T01:16:25.375829+0000","last_undegraded":"2023-12-01T01:16:25.375829+0000","last_fullsized":"2023-12-01T01:16:25.375829+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:05:45.697338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'13","reported_seq":60,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:17:04.664374+0000","last_change":"2023-12-01T01:16:24.934939+0000","last_active":"2023-12-01T01:17:04.664374+0000","last_peered":"2023-12-01T01:17:04.664374+0000","last_clean":"2023-12-01T01:17:04.664374+0000","last_became_active":"2023-12-01T01:16:24.934481+0000","last_became_peered":"2023-12-01T01:16:24.934481+0000","last_unstale":"2023-12-01T01:17:04.664374+0000","last_undegraded":"2023-12-01T01:17:04.664374+0000","last_fullsized":"2023-12-01T01:17:04.664374+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:39:49.804567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.571536+0000","last_change":"2023-12-01T01:16:25.364061+0000","last_active":"2023-12-01T01:16:26.571536+0000","last_peered":"2023-12-01T01:16:26.571536+0000","last_clean":"2023-12-01T01:16:26.571536+0000","last_became_active":"2023-12-01T01:16:25.363678+0000","last_became_peered":"2023-12-01T01:16:25.363678+0000","last_unstale":"2023-12-01T01:16:26.571536+0000","last_undegraded":"2023-12-01T01:16:26.571536+0000","last_fullsized":"2023-12-01T01:16:26.571536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:24:07.107604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325949+0000","last_change":"2023-12-01T01:16:24.939338+0000","last_active":"2023-12-01T01:16:27.325949+0000","last_peered":"2023-12-01T01:16:27.325949+0000","last_clean":"2023-12-01T01:16:27.325949+0000","last_became_active":"2023-12-01T01:16:24.939007+0000","last_became_peered":"2023-12-01T01:16:24.939007+0000","last_unstale":"2023-12-01T01:16:27.325949+0000","last_undegraded":"2023-12-01T01:16:27.325949+0000","last_fullsized":"2023-12-01T01:16:27.325949+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:56:50.732188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325303+0000","last_change":"2023-12-01T01:16:25.362231+0000","last_active":"2023-12-01T01:16:27.325303+0000","last_peered":"2023-12-01T01:16:27.325303+0000","last_clean":"2023-12-01T01:16:27.325303+0000","last_became_active":"2023-12-01T01:16:25.361920+0000","last_became_peered":"2023-12-01T01:16:25.361920+0000","last_unstale":"2023-12-01T01:16:27.325303+0000","last_undegraded":"2023-12-01T01:16:27.325303+0000","last_fullsized":"2023-12-01T01:16:27.325303+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:22:22.104480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'12","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:31.605443+0000","last_change":"2023-12-01T01:16:24.931801+0000","last_active":"2023-12-01T01:16:31.605443+0000","last_peered":"2023-12-01T01:16:31.605443+0000","last_clean":"2023-12-01T01:16:31.605443+0000","last_became_active":"2023-12-01T01:16:24.931451+0000","last_became_peered":"2023-12-01T01:16:24.931451+0000","last_unstale":"2023-12-01T01:16:31.605443+0000","last_undegraded":"2023-12-01T01:16:31.605443+0000","last_fullsized":"2023-12-01T01:16:31.605443+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:22:41.666502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.324789+0000","last_change":"2023-12-01T01:16:25.361789+0000","last_active":"2023-12-01T01:16:27.324789+0000","last_peered":"2023-12-01T01:16:27.324789+0000","last_clean":"2023-12-01T01:16:27.324789+0000","last_became_active":"2023-12-01T01:16:25.361480+0000","last_became_peered":"2023-12-01T01:16:25.361480+0000","last_unstale":"2023-12-01T01:16:27.324789+0000","last_undegraded":"2023-12-01T01:16:27.324789+0000","last_fullsized":"2023-12-01T01:16:27.324789+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:07:00.742167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'5","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:31.604682+0000","last_change":"2023-12-01T01:16:24.933362+0000","last_active":"2023-12-01T01:16:31.604682+0000","last_peered":"2023-12-01T01:16:31.604682+0000","last_clean":"2023-12-01T01:16:31.604682+0000","last_became_active":"2023-12-01T01:16:24.933049+0000","last_became_peered":"2023-12-01T01:16:24.933049+0000","last_unstale":"2023-12-01T01:16:31.604682+0000","last_undegraded":"2023-12-01T01:16:31.604682+0000","last_fullsized":"2023-12-01T01:16:31.604682+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:47:37.153344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325647+0000","last_change":"2023-12-01T01:16:25.364419+0000","last_active":"2023-12-01T01:16:27.325647+0000","last_peered":"2023-12-01T01:16:27.325647+0000","last_clean":"2023-12-01T01:16:27.325647+0000","last_became_active":"2023-12-01T01:16:25.363022+0000","last_became_peered":"2023-12-01T01:16:25.363022+0000","last_unstale":"2023-12-01T01:16:27.325647+0000","last_undegraded":"2023-12-01T01:16:27.325647+0000","last_fullsized":"2023-12-01T01:16:27.325647+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:58:45.660625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'6","reported_seq":53,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:17:02.645443+0000","last_change":"2023-12-01T01:16:24.935947+0000","last_active":"2023-12-01T01:17:02.645443+0000","last_peered":"2023-12-01T01:17:02.645443+0000","last_clean":"2023-12-01T01:17:02.645443+0000","last_became_active":"2023-12-01T01:16:24.935061+0000","last_became_peered":"2023-12-01T01:16:24.935061+0000","last_unstale":"2023-12-01T01:17:02.645443+0000","last_undegraded":"2023-12-01T01:17:02.645443+0000","last_fullsized":"2023-12-01T01:17:02.645443+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:00:31.551362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":967,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.071450+0000","last_change":"2023-12-01T01:16:25.363229+0000","last_active":"2023-12-01T01:16:26.071450+0000","last_peered":"2023-12-01T01:16:26.071450+0000","last_clean":"2023-12-01T01:16:26.071450+0000","last_became_active":"2023-12-01T01:16:25.362770+0000","last_became_peered":"2023-12-01T01:16:25.362770+0000","last_unstale":"2023-12-01T01:16:26.071450+0000","last_undegraded":"2023-12-01T01:16:26.071450+0000","last_fullsized":"2023-12-01T01:16:26.071450+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:45:02.782201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:30.654762+0000","last_change":"2023-12-01T01:16:24.930509+0000","last_active":"2023-12-01T01:16:30.654762+0000","last_peered":"2023-12-01T01:16:30.654762+0000","last_clean":"2023-12-01T01:16:30.654762+0000","last_became_active":"2023-12-01T01:16:24.930188+0000","last_became_peered":"2023-12-01T01:16:24.930188+0000","last_unstale":"2023-12-01T01:16:30.654762+0000","last_undegraded":"2023-12-01T01:16:30.654762+0000","last_fullsized":"2023-12-01T01:16:30.654762+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:04:17.748363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325530+0000","last_change":"2023-12-01T01:16:25.363383+0000","last_active":"2023-12-01T01:16:27.325530+0000","last_peered":"2023-12-01T01:16:27.325530+0000","last_clean":"2023-12-01T01:16:27.325530+0000","last_became_active":"2023-12-01T01:16:25.363085+0000","last_became_peered":"2023-12-01T01:16:25.363085+0000","last_unstale":"2023-12-01T01:16:27.325530+0000","last_undegraded":"2023-12-01T01:16:27.325530+0000","last_fullsized":"2023-12-01T01:16:27.325530+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:34:36.921706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"82'3","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:29.637968+0000","last_change":"2023-12-01T01:16:24.929081+0000","last_active":"2023-12-01T01:16:29.637968+0000","last_peered":"2023-12-01T01:16:29.637968+0000","last_clean":"2023-12-01T01:16:29.637968+0000","last_became_active":"2023-12-01T01:16:24.928747+0000","last_became_peered":"2023-12-01T01:16:24.928747+0000","last_unstale":"2023-12-01T01:16:29.637968+0000","last_undegraded":"2023-12-01T01:16:29.637968+0000","last_fullsized":"2023-12-01T01:16:29.637968+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:02:24.727835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.571467+0000","last_change":"2023-12-01T01:16:25.360959+0000","last_active":"2023-12-01T01:16:26.571467+0000","last_peered":"2023-12-01T01:16:26.571467+0000","last_clean":"2023-12-01T01:16:26.571467+0000","last_became_active":"2023-12-01T01:16:25.360647+0000","last_became_peered":"2023-12-01T01:16:25.360647+0000","last_unstale":"2023-12-01T01:16:26.571467+0000","last_undegraded":"2023-12-01T01:16:26.571467+0000","last_fullsized":"2023-12-01T01:16:26.571467+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:50:24.151273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'3","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:31.603750+0000","last_change":"2023-12-01T01:16:24.938322+0000","last_active":"2023-12-01T01:16:31.603750+0000","last_peered":"2023-12-01T01:16:31.603750+0000","last_clean":"2023-12-01T01:16:31.603750+0000","last_became_active":"2023-12-01T01:16:24.937747+0000","last_became_peered":"2023-12-01T01:16:24.937747+0000","last_unstale":"2023-12-01T01:16:31.603750+0000","last_undegraded":"2023-12-01T01:16:31.603750+0000","last_fullsized":"2023-12-01T01:16:31.603750+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:37:59.999748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.584270+0000","last_change":"2023-12-01T01:16:25.362779+0000","last_active":"2023-12-01T01:16:26.584270+0000","last_peered":"2023-12-01T01:16:26.584270+0000","last_clean":"2023-12-01T01:16:26.584270+0000","last_became_active":"2023-12-01T01:16:25.362487+0000","last_became_peered":"2023-12-01T01:16:25.362487+0000","last_unstale":"2023-12-01T01:16:26.584270+0000","last_undegraded":"2023-12-01T01:16:26.584270+0000","last_fullsized":"2023-12-01T01:16:26.584270+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:04:42.409498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":88,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:17:06.651263+0000","last_change":"2023-12-01T01:16:24.933187+0000","last_active":"2023-12-01T01:17:06.651263+0000","last_peered":"2023-12-01T01:17:06.651263+0000","last_clean":"2023-12-01T01:17:06.651263+0000","last_became_active":"2023-12-01T01:16:24.932819+0000","last_became_peered":"2023-12-01T01:16:24.932819+0000","last_unstale":"2023-12-01T01:17:06.651263+0000","last_undegraded":"2023-12-01T01:17:06.651263+0000","last_fullsized":"2023-12-01T01:17:06.651263+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:54:34.739815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.324904+0000","last_change":"2023-12-01T01:16:25.361183+0000","last_active":"2023-12-01T01:16:27.324904+0000","last_peered":"2023-12-01T01:16:27.324904+0000","last_clean":"2023-12-01T01:16:27.324904+0000","last_became_active":"2023-12-01T01:16:25.360871+0000","last_became_peered":"2023-12-01T01:16:25.360871+0000","last_unstale":"2023-12-01T01:16:27.324904+0000","last_undegraded":"2023-12-01T01:16:27.324904+0000","last_fullsized":"2023-12-01T01:16:27.324904+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:06:04.748159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'4","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:31.601992+0000","last_change":"2023-12-01T01:16:24.939355+0000","last_active":"2023-12-01T01:16:31.601992+0000","last_peered":"2023-12-01T01:16:31.601992+0000","last_clean":"2023-12-01T01:16:31.601992+0000","last_became_active":"2023-12-01T01:16:24.938967+0000","last_became_peered":"2023-12-01T01:16:24.938967+0000","last_unstale":"2023-12-01T01:16:31.601992+0000","last_undegraded":"2023-12-01T01:16:31.601992+0000","last_fullsized":"2023-12-01T01:16:31.601992+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:20:51.172753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325217+0000","last_change":"2023-12-01T01:16:25.366752+0000","last_active":"2023-12-01T01:16:27.325217+0000","last_peered":"2023-12-01T01:16:27.325217+0000","last_clean":"2023-12-01T01:16:27.325217+0000","last_became_active":"2023-12-01T01:16:25.366466+0000","last_became_peered":"2023-12-01T01:16:25.366466+0000","last_unstale":"2023-12-01T01:16:27.325217+0000","last_undegraded":"2023-12-01T01:16:27.325217+0000","last_fullsized":"2023-12-01T01:16:27.325217+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:09:53.716127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:30.650645+0000","last_change":"2023-12-01T01:16:24.940827+0000","last_active":"2023-12-01T01:16:30.650645+0000","last_peered":"2023-12-01T01:16:30.650645+0000","last_clean":"2023-12-01T01:16:30.650645+0000","last_became_active":"2023-12-01T01:16:24.940515+0000","last_became_peered":"2023-12-01T01:16:24.940515+0000","last_unstale":"2023-12-01T01:16:30.650645+0000","last_undegraded":"2023-12-01T01:16:30.650645+0000","last_fullsized":"2023-12-01T01:16:30.650645+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:08:26.409014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325032+0000","last_change":"2023-12-01T01:16:25.369662+0000","last_active":"2023-12-01T01:16:27.325032+0000","last_peered":"2023-12-01T01:16:27.325032+0000","last_clean":"2023-12-01T01:16:27.325032+0000","last_became_active":"2023-12-01T01:16:25.369178+0000","last_became_peered":"2023-12-01T01:16:25.369178+0000","last_unstale":"2023-12-01T01:16:27.325032+0000","last_undegraded":"2023-12-01T01:16:27.325032+0000","last_fullsized":"2023-12-01T01:16:27.325032+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:40:39.919414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'29","reported_seq":120,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:17:06.654122+0000","last_change":"2023-12-01T01:16:24.936733+0000","last_active":"2023-12-01T01:17:06.654122+0000","last_peered":"2023-12-01T01:17:06.654122+0000","last_clean":"2023-12-01T01:17:06.654122+0000","last_became_active":"2023-12-01T01:16:24.936459+0000","last_became_peered":"2023-12-01T01:16:24.936459+0000","last_unstale":"2023-12-01T01:17:06.654122+0000","last_undegraded":"2023-12-01T01:17:06.654122+0000","last_fullsized":"2023-12-01T01:17:06.654122+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:19:58.387650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58530,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":79,"num_read_kb":96,"num_write":31,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.071505+0000","last_change":"2023-12-01T01:16:25.364292+0000","last_active":"2023-12-01T01:16:26.071505+0000","last_peered":"2023-12-01T01:16:26.071505+0000","last_clean":"2023-12-01T01:16:26.071505+0000","last_became_active":"2023-12-01T01:16:25.362615+0000","last_became_peered":"2023-12-01T01:16:25.362615+0000","last_unstale":"2023-12-01T01:16:26.071505+0000","last_undegraded":"2023-12-01T01:16:26.071505+0000","last_fullsized":"2023-12-01T01:16:26.071505+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:06:07.350607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'9","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:31.602567+0000","last_change":"2023-12-01T01:16:24.941007+0000","last_active":"2023-12-01T01:16:31.602567+0000","last_peered":"2023-12-01T01:16:31.602567+0000","last_clean":"2023-12-01T01:16:31.602567+0000","last_became_active":"2023-12-01T01:16:24.940669+0000","last_became_peered":"2023-12-01T01:16:24.940669+0000","last_unstale":"2023-12-01T01:16:31.602567+0000","last_undegraded":"2023-12-01T01:16:31.602567+0000","last_fullsized":"2023-12-01T01:16:31.602567+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:26:21.907942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.352205+0000","last_change":"2023-12-01T01:16:25.360104+0000","last_active":"2023-12-01T01:16:26.352205+0000","last_peered":"2023-12-01T01:16:26.352205+0000","last_clean":"2023-12-01T01:16:26.352205+0000","last_became_active":"2023-12-01T01:16:25.359807+0000","last_became_peered":"2023-12-01T01:16:25.359807+0000","last_unstale":"2023-12-01T01:16:26.352205+0000","last_undegraded":"2023-12-01T01:16:26.352205+0000","last_fullsized":"2023-12-01T01:16:26.352205+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:04:05.100258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'5","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:31.604782+0000","last_change":"2023-12-01T01:16:24.945074+0000","last_active":"2023-12-01T01:16:31.604782+0000","last_peered":"2023-12-01T01:16:31.604782+0000","last_clean":"2023-12-01T01:16:31.604782+0000","last_became_active":"2023-12-01T01:16:24.944737+0000","last_became_peered":"2023-12-01T01:16:24.944737+0000","last_unstale":"2023-12-01T01:16:31.604782+0000","last_undegraded":"2023-12-01T01:16:31.604782+0000","last_fullsized":"2023-12-01T01:16:31.604782+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:44:53.931454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.350184+0000","last_change":"2023-12-01T01:16:25.364995+0000","last_active":"2023-12-01T01:16:26.350184+0000","last_peered":"2023-12-01T01:16:26.350184+0000","last_clean":"2023-12-01T01:16:26.350184+0000","last_became_active":"2023-12-01T01:16:25.364684+0000","last_became_peered":"2023-12-01T01:16:25.364684+0000","last_unstale":"2023-12-01T01:16:26.350184+0000","last_undegraded":"2023-12-01T01:16:26.350184+0000","last_fullsized":"2023-12-01T01:16:26.350184+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:41:35.422238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":51,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:17:01.660724+0000","last_change":"2023-12-01T01:16:24.929103+0000","last_active":"2023-12-01T01:17:01.660724+0000","last_peered":"2023-12-01T01:17:01.660724+0000","last_clean":"2023-12-01T01:17:01.660724+0000","last_became_active":"2023-12-01T01:16:24.928769+0000","last_became_peered":"2023-12-01T01:16:24.928769+0000","last_unstale":"2023-12-01T01:17:01.660724+0000","last_undegraded":"2023-12-01T01:17:01.660724+0000","last_fullsized":"2023-12-01T01:17:01.660724+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:45:24.542610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":35,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.349997+0000","last_change":"2023-12-01T01:16:25.370816+0000","last_active":"2023-12-01T01:16:26.349997+0000","last_peered":"2023-12-01T01:16:26.349997+0000","last_clean":"2023-12-01T01:16:26.349997+0000","last_became_active":"2023-12-01T01:16:25.370465+0000","last_became_peered":"2023-12-01T01:16:25.370465+0000","last_unstale":"2023-12-01T01:16:26.349997+0000","last_undegraded":"2023-12-01T01:16:26.349997+0000","last_fullsized":"2023-12-01T01:16:26.349997+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:52:28.990416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'17","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:31.594625+0000","last_change":"2023-12-01T01:16:24.933022+0000","last_active":"2023-12-01T01:16:31.594625+0000","last_peered":"2023-12-01T01:16:31.594625+0000","last_clean":"2023-12-01T01:16:31.594625+0000","last_became_active":"2023-12-01T01:16:24.932705+0000","last_became_peered":"2023-12-01T01:16:24.932705+0000","last_unstale":"2023-12-01T01:16:31.594625+0000","last_undegraded":"2023-12-01T01:16:31.594625+0000","last_fullsized":"2023-12-01T01:16:31.594625+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:36:44.747984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325386+0000","last_change":"2023-12-01T01:16:25.370881+0000","last_active":"2023-12-01T01:16:27.325386+0000","last_peered":"2023-12-01T01:16:27.325386+0000","last_clean":"2023-12-01T01:16:27.325386+0000","last_became_active":"2023-12-01T01:16:25.368242+0000","last_became_peered":"2023-12-01T01:16:25.368242+0000","last_unstale":"2023-12-01T01:16:27.325386+0000","last_undegraded":"2023-12-01T01:16:27.325386+0000","last_fullsized":"2023-12-01T01:16:27.325386+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:35:41.128841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:31.594803+0000","last_change":"2023-12-01T01:16:24.933579+0000","last_active":"2023-12-01T01:16:31.594803+0000","last_peered":"2023-12-01T01:16:31.594803+0000","last_clean":"2023-12-01T01:16:31.594803+0000","last_became_active":"2023-12-01T01:16:24.933207+0000","last_became_peered":"2023-12-01T01:16:24.933207+0000","last_unstale":"2023-12-01T01:16:31.594803+0000","last_undegraded":"2023-12-01T01:16:31.594803+0000","last_fullsized":"2023-12-01T01:16:31.594803+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:28.309551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.592816+0000","last_change":"2023-12-01T01:16:25.368611+0000","last_active":"2023-12-01T01:16:25.592816+0000","last_peered":"2023-12-01T01:16:25.592816+0000","last_clean":"2023-12-01T01:16:25.592816+0000","last_became_active":"2023-12-01T01:16:25.368291+0000","last_became_peered":"2023-12-01T01:16:25.368291+0000","last_unstale":"2023-12-01T01:16:25.592816+0000","last_undegraded":"2023-12-01T01:16:25.592816+0000","last_fullsized":"2023-12-01T01:16:25.592816+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:17:21.824737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325140+0000","last_change":"2023-12-01T01:16:24.941410+0000","last_active":"2023-12-01T01:16:27.325140+0000","last_peered":"2023-12-01T01:16:27.325140+0000","last_clean":"2023-12-01T01:16:27.325140+0000","last_became_active":"2023-12-01T01:16:24.941127+0000","last_became_peered":"2023-12-01T01:16:24.941127+0000","last_unstale":"2023-12-01T01:16:27.325140+0000","last_undegraded":"2023-12-01T01:16:27.325140+0000","last_fullsized":"2023-12-01T01:16:27.325140+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:25:20.911892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.476088+0000","last_change":"2023-12-01T01:16:25.368235+0000","last_active":"2023-12-01T01:16:27.476088+0000","last_peered":"2023-12-01T01:16:27.476088+0000","last_clean":"2023-12-01T01:16:27.476088+0000","last_became_active":"2023-12-01T01:16:25.367918+0000","last_became_peered":"2023-12-01T01:16:25.367918+0000","last_unstale":"2023-12-01T01:16:27.476088+0000","last_undegraded":"2023-12-01T01:16:27.476088+0000","last_fullsized":"2023-12-01T01:16:27.476088+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:12:26.665404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'5","reported_seq":49,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:17:01.661313+0000","last_change":"2023-12-01T01:16:24.932057+0000","last_active":"2023-12-01T01:17:01.661313+0000","last_peered":"2023-12-01T01:17:01.661313+0000","last_clean":"2023-12-01T01:17:01.661313+0000","last_became_active":"2023-12-01T01:16:24.931817+0000","last_became_peered":"2023-12-01T01:16:24.931817+0000","last_unstale":"2023-12-01T01:17:01.661313+0000","last_undegraded":"2023-12-01T01:17:01.661313+0000","last_fullsized":"2023-12-01T01:17:01.661313+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:19:02.039334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":32,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325595+0000","last_change":"2023-12-01T01:16:25.367431+0000","last_active":"2023-12-01T01:16:27.325595+0000","last_peered":"2023-12-01T01:16:27.325595+0000","last_clean":"2023-12-01T01:16:27.325595+0000","last_became_active":"2023-12-01T01:16:25.367184+0000","last_became_peered":"2023-12-01T01:16:25.367184+0000","last_unstale":"2023-12-01T01:16:27.325595+0000","last_undegraded":"2023-12-01T01:16:27.325595+0000","last_fullsized":"2023-12-01T01:16:27.325595+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:21:19.132725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325383+0000","last_change":"2023-12-01T01:16:25.370593+0000","last_active":"2023-12-01T01:16:27.325383+0000","last_peered":"2023-12-01T01:16:27.325383+0000","last_clean":"2023-12-01T01:16:27.325383+0000","last_became_active":"2023-12-01T01:16:25.367614+0000","last_became_peered":"2023-12-01T01:16:25.367614+0000","last_unstale":"2023-12-01T01:16:27.325383+0000","last_undegraded":"2023-12-01T01:16:27.325383+0000","last_fullsized":"2023-12-01T01:16:27.325383+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:56:34.388180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":54,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:17:05.666244+0000","last_change":"2023-12-01T01:16:24.931188+0000","last_active":"2023-12-01T01:17:05.666244+0000","last_peered":"2023-12-01T01:17:05.666244+0000","last_clean":"2023-12-01T01:17:05.666244+0000","last_became_active":"2023-12-01T01:16:24.930875+0000","last_became_peered":"2023-12-01T01:16:24.930875+0000","last_unstale":"2023-12-01T01:17:05.666244+0000","last_undegraded":"2023-12-01T01:17:05.666244+0000","last_fullsized":"2023-12-01T01:17:05.666244+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:17.729286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.571499+0000","last_change":"2023-12-01T01:16:25.361359+0000","last_active":"2023-12-01T01:16:26.571499+0000","last_peered":"2023-12-01T01:16:26.571499+0000","last_clean":"2023-12-01T01:16:26.571499+0000","last_became_active":"2023-12-01T01:16:25.361083+0000","last_became_peered":"2023-12-01T01:16:25.361083+0000","last_unstale":"2023-12-01T01:16:26.571499+0000","last_undegraded":"2023-12-01T01:16:26.571499+0000","last_fullsized":"2023-12-01T01:16:26.571499+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:42:09.417021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:31.603727+0000","last_change":"2023-12-01T01:16:24.927614+0000","last_active":"2023-12-01T01:16:31.603727+0000","last_peered":"2023-12-01T01:16:31.603727+0000","last_clean":"2023-12-01T01:16:31.603727+0000","last_became_active":"2023-12-01T01:16:24.926459+0000","last_became_peered":"2023-12-01T01:16:24.926459+0000","last_unstale":"2023-12-01T01:16:31.603727+0000","last_undegraded":"2023-12-01T01:16:31.603727+0000","last_fullsized":"2023-12-01T01:16:31.603727+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:41:25.435261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":323,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.358040+0000","last_change":"2023-12-01T01:15:25.599551+0000","last_active":"2023-12-01T01:16:25.358040+0000","last_peered":"2023-12-01T01:16:25.358040+0000","last_clean":"2023-12-01T01:16:25.358040+0000","last_became_active":"2023-12-01T01:15:25.598951+0000","last_became_peered":"2023-12-01T01:15:25.598951+0000","last_unstale":"2023-12-01T01:16:25.358040+0000","last_undegraded":"2023-12-01T01:16:25.358040+0000","last_fullsized":"2023-12-01T01:16:25.358040+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:12:31.173756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:12:31.173756+0000","last_clean_scrub_stamp":"2023-12-01T01:12:31.173756+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:00:11.343823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.352149+0000","last_change":"2023-12-01T01:16:25.358617+0000","last_active":"2023-12-01T01:16:26.352149+0000","last_peered":"2023-12-01T01:16:26.352149+0000","last_clean":"2023-12-01T01:16:26.352149+0000","last_became_active":"2023-12-01T01:16:25.358325+0000","last_became_peered":"2023-12-01T01:16:25.358325+0000","last_unstale":"2023-12-01T01:16:26.352149+0000","last_undegraded":"2023-12-01T01:16:26.352149+0000","last_fullsized":"2023-12-01T01:16:26.352149+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:40:01.111777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'4","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:29.637779+0000","last_change":"2023-12-01T01:16:24.935103+0000","last_active":"2023-12-01T01:16:29.637779+0000","last_peered":"2023-12-01T01:16:29.637779+0000","last_clean":"2023-12-01T01:16:29.637779+0000","last_became_active":"2023-12-01T01:16:24.934900+0000","last_became_peered":"2023-12-01T01:16:24.934900+0000","last_unstale":"2023-12-01T01:16:29.637779+0000","last_undegraded":"2023-12-01T01:16:29.637779+0000","last_fullsized":"2023-12-01T01:16:29.637779+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:59:37.138090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"82'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:31.594317+0000","last_change":"2023-12-01T01:16:24.938260+0000","last_active":"2023-12-01T01:16:31.594317+0000","last_peered":"2023-12-01T01:16:31.594317+0000","last_clean":"2023-12-01T01:16:31.594317+0000","last_became_active":"2023-12-01T01:16:24.937939+0000","last_became_peered":"2023-12-01T01:16:24.937939+0000","last_unstale":"2023-12-01T01:16:31.594317+0000","last_undegraded":"2023-12-01T01:16:31.594317+0000","last_fullsized":"2023-12-01T01:16:31.594317+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:07:05.090877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.352470+0000","last_change":"2023-12-01T01:16:25.362137+0000","last_active":"2023-12-01T01:16:26.352470+0000","last_peered":"2023-12-01T01:16:26.352470+0000","last_clean":"2023-12-01T01:16:26.352470+0000","last_became_active":"2023-12-01T01:16:25.361563+0000","last_became_peered":"2023-12-01T01:16:25.361563+0000","last_unstale":"2023-12-01T01:16:26.352470+0000","last_undegraded":"2023-12-01T01:16:26.352470+0000","last_fullsized":"2023-12-01T01:16:26.352470+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:37:56.473920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'9","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:34.873654+0000","last_change":"2023-12-01T01:16:24.935692+0000","last_active":"2023-12-01T01:16:34.873654+0000","last_peered":"2023-12-01T01:16:34.873654+0000","last_clean":"2023-12-01T01:16:34.873654+0000","last_became_active":"2023-12-01T01:16:24.934799+0000","last_became_peered":"2023-12-01T01:16:24.934799+0000","last_unstale":"2023-12-01T01:16:34.873654+0000","last_undegraded":"2023-12-01T01:16:34.873654+0000","last_fullsized":"2023-12-01T01:16:34.873654+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:22:51.586089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.571589+0000","last_change":"2023-12-01T01:16:25.368672+0000","last_active":"2023-12-01T01:16:26.571589+0000","last_peered":"2023-12-01T01:16:26.571589+0000","last_clean":"2023-12-01T01:16:26.571589+0000","last_became_active":"2023-12-01T01:16:25.368309+0000","last_became_peered":"2023-12-01T01:16:25.368309+0000","last_unstale":"2023-12-01T01:16:26.571589+0000","last_undegraded":"2023-12-01T01:16:26.571589+0000","last_fullsized":"2023-12-01T01:16:26.571589+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:30:59.029250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:31.603713+0000","last_change":"2023-12-01T01:16:24.931322+0000","last_active":"2023-12-01T01:16:31.603713+0000","last_peered":"2023-12-01T01:16:31.603713+0000","last_clean":"2023-12-01T01:16:31.603713+0000","last_became_active":"2023-12-01T01:16:24.930961+0000","last_became_peered":"2023-12-01T01:16:24.930961+0000","last_unstale":"2023-12-01T01:16:31.603713+0000","last_undegraded":"2023-12-01T01:16:31.603713+0000","last_fullsized":"2023-12-01T01:16:31.603713+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:47:16.317035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325104+0000","last_change":"2023-12-01T01:16:25.364065+0000","last_active":"2023-12-01T01:16:27.325104+0000","last_peered":"2023-12-01T01:16:27.325104+0000","last_clean":"2023-12-01T01:16:27.325104+0000","last_became_active":"2023-12-01T01:16:25.363694+0000","last_became_peered":"2023-12-01T01:16:25.363694+0000","last_unstale":"2023-12-01T01:16:27.325104+0000","last_undegraded":"2023-12-01T01:16:27.325104+0000","last_fullsized":"2023-12-01T01:16:27.325104+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:49:39.091587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:28.633891+0000","last_change":"2023-12-01T01:16:24.932730+0000","last_active":"2023-12-01T01:16:28.633891+0000","last_peered":"2023-12-01T01:16:28.633891+0000","last_clean":"2023-12-01T01:16:28.633891+0000","last_became_active":"2023-12-01T01:16:24.932404+0000","last_became_peered":"2023-12-01T01:16:24.932404+0000","last_unstale":"2023-12-01T01:16:28.633891+0000","last_undegraded":"2023-12-01T01:16:28.633891+0000","last_fullsized":"2023-12-01T01:16:28.633891+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:41:33.104418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.071437+0000","last_change":"2023-12-01T01:16:25.375795+0000","last_active":"2023-12-01T01:16:26.071437+0000","last_peered":"2023-12-01T01:16:26.071437+0000","last_clean":"2023-12-01T01:16:26.071437+0000","last_became_active":"2023-12-01T01:16:25.375485+0000","last_became_peered":"2023-12-01T01:16:25.375485+0000","last_unstale":"2023-12-01T01:16:26.071437+0000","last_undegraded":"2023-12-01T01:16:26.071437+0000","last_fullsized":"2023-12-01T01:16:26.071437+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:10:04.009511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'7","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:31.602193+0000","last_change":"2023-12-01T01:16:24.936770+0000","last_active":"2023-12-01T01:16:31.602193+0000","last_peered":"2023-12-01T01:16:31.602193+0000","last_clean":"2023-12-01T01:16:31.602193+0000","last_became_active":"2023-12-01T01:16:24.936058+0000","last_became_peered":"2023-12-01T01:16:24.936058+0000","last_unstale":"2023-12-01T01:16:31.602193+0000","last_undegraded":"2023-12-01T01:16:31.602193+0000","last_fullsized":"2023-12-01T01:16:31.602193+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:22:21.414131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.592744+0000","last_change":"2023-12-01T01:16:25.372183+0000","last_active":"2023-12-01T01:16:25.592744+0000","last_peered":"2023-12-01T01:16:25.592744+0000","last_clean":"2023-12-01T01:16:25.592744+0000","last_became_active":"2023-12-01T01:16:25.371865+0000","last_became_peered":"2023-12-01T01:16:25.371865+0000","last_unstale":"2023-12-01T01:16:25.592744+0000","last_undegraded":"2023-12-01T01:16:25.592744+0000","last_fullsized":"2023-12-01T01:16:25.592744+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:19:54.675425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'3","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:33.355776+0000","last_change":"2023-12-01T01:16:24.939397+0000","last_active":"2023-12-01T01:16:33.355776+0000","last_peered":"2023-12-01T01:16:33.355776+0000","last_clean":"2023-12-01T01:16:33.355776+0000","last_became_active":"2023-12-01T01:16:24.939081+0000","last_became_peered":"2023-12-01T01:16:24.939081+0000","last_unstale":"2023-12-01T01:16:33.355776+0000","last_undegraded":"2023-12-01T01:16:33.355776+0000","last_fullsized":"2023-12-01T01:16:33.355776+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:21:34.967255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.571515+0000","last_change":"2023-12-01T01:16:25.368484+0000","last_active":"2023-12-01T01:16:26.571515+0000","last_peered":"2023-12-01T01:16:26.571515+0000","last_clean":"2023-12-01T01:16:26.571515+0000","last_became_active":"2023-12-01T01:16:25.367638+0000","last_became_peered":"2023-12-01T01:16:25.367638+0000","last_unstale":"2023-12-01T01:16:26.571515+0000","last_undegraded":"2023-12-01T01:16:26.571515+0000","last_fullsized":"2023-12-01T01:16:26.571515+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:06:05.354443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:28.633709+0000","last_change":"2023-12-01T01:16:24.933304+0000","last_active":"2023-12-01T01:16:28.633709+0000","last_peered":"2023-12-01T01:16:28.633709+0000","last_clean":"2023-12-01T01:16:28.633709+0000","last_became_active":"2023-12-01T01:16:24.933028+0000","last_became_peered":"2023-12-01T01:16:24.933028+0000","last_unstale":"2023-12-01T01:16:28.633709+0000","last_undegraded":"2023-12-01T01:16:28.633709+0000","last_fullsized":"2023-12-01T01:16:28.633709+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:26:51.810995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.350012+0000","last_change":"2023-12-01T01:16:25.364888+0000","last_active":"2023-12-01T01:16:26.350012+0000","last_peered":"2023-12-01T01:16:26.350012+0000","last_clean":"2023-12-01T01:16:26.350012+0000","last_became_active":"2023-12-01T01:16:25.364595+0000","last_became_peered":"2023-12-01T01:16:25.364595+0000","last_unstale":"2023-12-01T01:16:26.350012+0000","last_undegraded":"2023-12-01T01:16:26.350012+0000","last_fullsized":"2023-12-01T01:16:26.350012+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:54:23.178824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'5","reported_seq":53,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:17:03.647081+0000","last_change":"2023-12-01T01:16:24.945771+0000","last_active":"2023-12-01T01:17:03.647081+0000","last_peered":"2023-12-01T01:17:03.647081+0000","last_clean":"2023-12-01T01:17:03.647081+0000","last_became_active":"2023-12-01T01:16:24.945441+0000","last_became_peered":"2023-12-01T01:16:24.945441+0000","last_unstale":"2023-12-01T01:17:03.647081+0000","last_undegraded":"2023-12-01T01:17:03.647081+0000","last_fullsized":"2023-12-01T01:17:03.647081+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:49:41.234472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":36,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.350239+0000","last_change":"2023-12-01T01:16:25.365246+0000","last_active":"2023-12-01T01:16:26.350239+0000","last_peered":"2023-12-01T01:16:26.350239+0000","last_clean":"2023-12-01T01:16:26.350239+0000","last_became_active":"2023-12-01T01:16:25.364364+0000","last_became_peered":"2023-12-01T01:16:25.364364+0000","last_unstale":"2023-12-01T01:16:26.350239+0000","last_undegraded":"2023-12-01T01:16:26.350239+0000","last_fullsized":"2023-12-01T01:16:26.350239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:56:44.815172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'3","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:31.594729+0000","last_change":"2023-12-01T01:16:24.944229+0000","last_active":"2023-12-01T01:16:31.594729+0000","last_peered":"2023-12-01T01:16:31.594729+0000","last_clean":"2023-12-01T01:16:31.594729+0000","last_became_active":"2023-12-01T01:16:24.943880+0000","last_became_peered":"2023-12-01T01:16:24.943880+0000","last_unstale":"2023-12-01T01:16:31.594729+0000","last_undegraded":"2023-12-01T01:16:31.594729+0000","last_fullsized":"2023-12-01T01:16:31.594729+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:44:26.769358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:28.318887+0000","last_change":"2023-12-01T01:16:25.361028+0000","last_active":"2023-12-01T01:16:28.318887+0000","last_peered":"2023-12-01T01:16:28.318887+0000","last_clean":"2023-12-01T01:16:28.318887+0000","last_became_active":"2023-12-01T01:16:25.360702+0000","last_became_peered":"2023-12-01T01:16:25.360702+0000","last_unstale":"2023-12-01T01:16:28.318887+0000","last_undegraded":"2023-12-01T01:16:28.318887+0000","last_fullsized":"2023-12-01T01:16:28.318887+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:45:09.472497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'5","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:31.601241+0000","last_change":"2023-12-01T01:16:24.945567+0000","last_active":"2023-12-01T01:16:31.601241+0000","last_peered":"2023-12-01T01:16:31.601241+0000","last_clean":"2023-12-01T01:16:31.601241+0000","last_became_active":"2023-12-01T01:16:24.945256+0000","last_became_peered":"2023-12-01T01:16:24.945256+0000","last_unstale":"2023-12-01T01:16:31.601241+0000","last_undegraded":"2023-12-01T01:16:31.601241+0000","last_fullsized":"2023-12-01T01:16:31.601241+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:19:30.944573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.350244+0000","last_change":"2023-12-01T01:16:25.367247+0000","last_active":"2023-12-01T01:16:26.350244+0000","last_peered":"2023-12-01T01:16:26.350244+0000","last_clean":"2023-12-01T01:16:26.350244+0000","last_became_active":"2023-12-01T01:16:25.366872+0000","last_became_peered":"2023-12-01T01:16:25.366872+0000","last_unstale":"2023-12-01T01:16:26.350244+0000","last_undegraded":"2023-12-01T01:16:26.350244+0000","last_fullsized":"2023-12-01T01:16:26.350244+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:03:07.697233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.592768+0000","last_change":"2023-12-01T01:16:25.376005+0000","last_active":"2023-12-01T01:16:25.592768+0000","last_peered":"2023-12-01T01:16:25.592768+0000","last_clean":"2023-12-01T01:16:25.592768+0000","last_became_active":"2023-12-01T01:16:25.375702+0000","last_became_peered":"2023-12-01T01:16:25.375702+0000","last_unstale":"2023-12-01T01:16:25.592768+0000","last_undegraded":"2023-12-01T01:16:25.592768+0000","last_fullsized":"2023-12-01T01:16:25.592768+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:14:00.955730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.378145+0000","last_change":"2023-12-01T01:16:25.371023+0000","last_active":"2023-12-01T01:16:25.378145+0000","last_peered":"2023-12-01T01:16:25.378145+0000","last_clean":"2023-12-01T01:16:25.378145+0000","last_became_active":"2023-12-01T01:16:25.370694+0000","last_became_peered":"2023-12-01T01:16:25.370694+0000","last_unstale":"2023-12-01T01:16:25.378145+0000","last_undegraded":"2023-12-01T01:16:25.378145+0000","last_fullsized":"2023-12-01T01:16:25.378145+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:03:00.715413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.375644+0000","last_change":"2023-12-01T01:16:25.361795+0000","last_active":"2023-12-01T01:16:25.375644+0000","last_peered":"2023-12-01T01:16:25.375644+0000","last_clean":"2023-12-01T01:16:25.375644+0000","last_became_active":"2023-12-01T01:16:25.361459+0000","last_became_peered":"2023-12-01T01:16:25.361459+0000","last_unstale":"2023-12-01T01:16:25.375644+0000","last_undegraded":"2023-12-01T01:16:25.375644+0000","last_fullsized":"2023-12-01T01:16:25.375644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:18:57.531240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.324741+0000","last_change":"2023-12-01T01:16:25.363773+0000","last_active":"2023-12-01T01:16:27.324741+0000","last_peered":"2023-12-01T01:16:27.324741+0000","last_clean":"2023-12-01T01:16:27.324741+0000","last_became_active":"2023-12-01T01:16:25.363422+0000","last_became_peered":"2023-12-01T01:16:25.363422+0000","last_unstale":"2023-12-01T01:16:27.324741+0000","last_undegraded":"2023-12-01T01:16:27.324741+0000","last_fullsized":"2023-12-01T01:16:27.324741+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:48:33.395566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.352367+0000","last_change":"2023-12-01T01:16:25.359431+0000","last_active":"2023-12-01T01:16:26.352367+0000","last_peered":"2023-12-01T01:16:26.352367+0000","last_clean":"2023-12-01T01:16:26.352367+0000","last_became_active":"2023-12-01T01:16:25.359116+0000","last_became_peered":"2023-12-01T01:16:25.359116+0000","last_unstale":"2023-12-01T01:16:26.352367+0000","last_undegraded":"2023-12-01T01:16:26.352367+0000","last_fullsized":"2023-12-01T01:16:26.352367+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:32:26.545269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.592804+0000","last_change":"2023-12-01T01:16:25.368008+0000","last_active":"2023-12-01T01:16:25.592804+0000","last_peered":"2023-12-01T01:16:25.592804+0000","last_clean":"2023-12-01T01:16:25.592804+0000","last_became_active":"2023-12-01T01:16:25.367726+0000","last_became_peered":"2023-12-01T01:16:25.367726+0000","last_unstale":"2023-12-01T01:16:25.592804+0000","last_undegraded":"2023-12-01T01:16:25.592804+0000","last_fullsized":"2023-12-01T01:16:25.592804+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:38:33.837846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.378494+0000","last_change":"2023-12-01T01:16:25.372781+0000","last_active":"2023-12-01T01:16:25.378494+0000","last_peered":"2023-12-01T01:16:25.378494+0000","last_clean":"2023-12-01T01:16:25.378494+0000","last_became_active":"2023-12-01T01:16:25.371443+0000","last_became_peered":"2023-12-01T01:16:25.371443+0000","last_unstale":"2023-12-01T01:16:25.378494+0000","last_undegraded":"2023-12-01T01:16:25.378494+0000","last_fullsized":"2023-12-01T01:16:25.378494+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:49:06.422055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.571571+0000","last_change":"2023-12-01T01:16:25.363048+0000","last_active":"2023-12-01T01:16:26.571571+0000","last_peered":"2023-12-01T01:16:26.571571+0000","last_clean":"2023-12-01T01:16:26.571571+0000","last_became_active":"2023-12-01T01:16:25.362752+0000","last_became_peered":"2023-12-01T01:16:25.362752+0000","last_unstale":"2023-12-01T01:16:26.571571+0000","last_undegraded":"2023-12-01T01:16:26.571571+0000","last_fullsized":"2023-12-01T01:16:26.571571+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:54:16.492562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325256+0000","last_change":"2023-12-01T01:16:25.359998+0000","last_active":"2023-12-01T01:16:27.325256+0000","last_peered":"2023-12-01T01:16:27.325256+0000","last_clean":"2023-12-01T01:16:27.325256+0000","last_became_active":"2023-12-01T01:16:25.359744+0000","last_became_peered":"2023-12-01T01:16:25.359744+0000","last_unstale":"2023-12-01T01:16:27.325256+0000","last_undegraded":"2023-12-01T01:16:27.325256+0000","last_fullsized":"2023-12-01T01:16:27.325256+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:12:33.947134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.571477+0000","last_change":"2023-12-01T01:16:25.373615+0000","last_active":"2023-12-01T01:16:26.571477+0000","last_peered":"2023-12-01T01:16:26.571477+0000","last_clean":"2023-12-01T01:16:26.571477+0000","last_became_active":"2023-12-01T01:16:25.373247+0000","last_became_peered":"2023-12-01T01:16:25.373247+0000","last_unstale":"2023-12-01T01:16:26.571477+0000","last_undegraded":"2023-12-01T01:16:26.571477+0000","last_fullsized":"2023-12-01T01:16:26.571477+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:28:10.815906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.324878+0000","last_change":"2023-12-01T01:16:25.367404+0000","last_active":"2023-12-01T01:16:27.324878+0000","last_peered":"2023-12-01T01:16:27.324878+0000","last_clean":"2023-12-01T01:16:27.324878+0000","last_became_active":"2023-12-01T01:16:25.366404+0000","last_became_peered":"2023-12-01T01:16:25.366404+0000","last_unstale":"2023-12-01T01:16:27.324878+0000","last_undegraded":"2023-12-01T01:16:27.324878+0000","last_fullsized":"2023-12-01T01:16:27.324878+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:54:49.668036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.352405+0000","last_change":"2023-12-01T01:16:25.360315+0000","last_active":"2023-12-01T01:16:26.352405+0000","last_peered":"2023-12-01T01:16:26.352405+0000","last_clean":"2023-12-01T01:16:26.352405+0000","last_became_active":"2023-12-01T01:16:25.360031+0000","last_became_peered":"2023-12-01T01:16:25.360031+0000","last_unstale":"2023-12-01T01:16:26.352405+0000","last_undegraded":"2023-12-01T01:16:26.352405+0000","last_fullsized":"2023-12-01T01:16:26.352405+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:13:24.437697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.592792+0000","last_change":"2023-12-01T01:16:25.367820+0000","last_active":"2023-12-01T01:16:25.592792+0000","last_peered":"2023-12-01T01:16:25.592792+0000","last_clean":"2023-12-01T01:16:25.592792+0000","last_became_active":"2023-12-01T01:16:25.367525+0000","last_became_peered":"2023-12-01T01:16:25.367525+0000","last_unstale":"2023-12-01T01:16:25.592792+0000","last_undegraded":"2023-12-01T01:16:25.592792+0000","last_fullsized":"2023-12-01T01:16:25.592792+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:28:01.552342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65153,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":381,"num_read_kb":268,"num_write":229,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":195459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":201,"ondisk_log_size":201,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481812,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":892928,"data_stored":423958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.442},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.61199999999999999}]},{"osd":1,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.68100000000000005}]},{"osd":2,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.49199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.72999999999999998}]},{"osd":5,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.57899999999999996}]},{"osd":6,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.70099999999999996}]},{"osd":7,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.78800000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":9,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.71799999999999997}]},{"osd":10,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.649,"5min":1.649,"15min":1.649},"last":0.83899999999999997}]}]},{"osd":10,"up_from":69,"seq":296352743448,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28504,"kb_used_data":1552,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716648,"statfs":{"total":95995035648,"available":95965847552,"internally_reserved":0,"allocated":1589248,"data_stored":1073192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":1,"last update":"Fri Dec 1 01:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.44800000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.60099999999999998}]},{"osd":3,"last update":"Fri Dec 1 01:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":5.7240000000000002,"5min":5.7240000000000002,"15min":5.7240000000000002},"last":0.61199999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.64000000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.63}]},{"osd":6,"last update":"Fri Dec 1 01:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":5.7510000000000003,"5min":5.7510000000000003,"15min":5.7510000000000003},"last":0.68100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":8,"last update":"Fri Dec 1 01:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.81699999999999995}]},{"osd":9,"last update":"Fri Dec 1 01:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.79400000000000004}]},{"osd":11,"last update":"Fri Dec 1 01:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.746},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.77900000000000003}]}]},{"osd":9,"up_from":64,"seq":274877906973,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":917504,"data_stored":423936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.46700000000000003,"15min":0.44900000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":3.6589999999999998,"5min":3.6589999999999998,"15min":3.6589999999999998},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.45600000000000002,"15min":0.46100000000000002},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.24199999999999999}]},{"osd":1,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.58199999999999996,"15min":0.59199999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.65100000000000002,"15min":0.63200000000000001},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":3.4969999999999999,"5min":3.4969999999999999,"15min":3.4969999999999999},"last":0.46999999999999997}]},{"osd":2,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57999999999999996,"15min":0.58499999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.72299999999999998,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.64200000000000002,"15min":0.625},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":3.5470000000000002,"5min":3.5470000000000002,"15min":3.5470000000000002},"last":0.52900000000000003}]},{"osd":3,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.54500000000000004,"15min":0.55700000000000005},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.53300000000000003,"15min":0.54500000000000004},"min":{"1min":0.33800000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.317}]},{"osd":4,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.71299999999999997,"15min":0.69899999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":3.6019999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":0.5},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.69499999999999995,"15min":0.71299999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.872,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.40699999999999997}]},{"osd":5,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.67400000000000004,"15min":0.68999999999999995},"min":{"1min":0.46500000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.79100000000000004,"5min":1.099,"15min":1.099},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.66600000000000004,"15min":0.68200000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.70999999999999996,"5min":1.052,"15min":1.052},"last":0.42999999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.626,"15min":0.64400000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.69799999999999995,"15min":0.69199999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":3.8410000000000002,"5min":3.8410000000000002,"15min":3.8410000000000002},"last":0.29699999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.76600000000000001,"15min":0.79000000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.83199999999999996,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.749,"15min":0.77200000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81299999999999994,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.50700000000000001}]},{"osd":8,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.77400000000000002,"15min":0.76600000000000001},"min":{"1min":0.50900000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":3.5760000000000001,"5min":3.5760000000000001,"15min":3.5760000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.751,"15min":0.77100000000000002},"min":{"1min":0.5,"5min":0.495,"15min":0.495},"max":{"1min":0.76900000000000002,"5min":1.157,"15min":1.157},"last":0.49199999999999999}]},{"osd":10,"last update":"Fri Dec 1 01:16:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.443},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.54800000000000004}]}]},{"osd":8,"up_from":57,"seq":244813135904,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":892,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":913408,"data_stored":424314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.57899999999999996,"15min":0.55700000000000005},"min":{"1min":0.39500000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":3.4089999999999998,"5min":3.4089999999999998,"15min":3.4089999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54000000000000004,"15min":0.53700000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.58099999999999996}]},{"osd":1,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.626,"15min":0.58399999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":6.0960000000000001,"5min":6.0960000000000001,"15min":6.0960000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.747,"5min":0.61099999999999999,"15min":0.58899999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":3.556,"5min":3.556,"15min":3.556},"last":0.69899999999999995}]},{"osd":2,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.50900000000000001,"15min":0.47099999999999997},"min":{"1min":0.28399999999999997,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":5.9199999999999999,"5min":5.9199999999999999,"15min":5.9199999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.48499999999999999,"15min":0.45400000000000001},"min":{"1min":0.311,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":3.6099999999999999,"5min":3.6099999999999999,"15min":3.6099999999999999},"last":0.46100000000000002}]},{"osd":3,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.496},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.627,"15min":0.58199999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":6.0209999999999999,"5min":6.0209999999999999,"15min":6.0209999999999999},"last":0.65400000000000003}]},{"osd":4,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.60999999999999999,"15min":0.60399999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.66400000000000003,"15min":0.61899999999999999},"min":{"1min":0.39100000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":6.0709999999999997,"5min":6.0709999999999997,"15min":6.0709999999999997},"last":0.86499999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.58399999999999996,"15min":0.55700000000000005},"min":{"1min":0.35799999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":3.4580000000000002,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.55500000000000005,"15min":0.54000000000000004},"min":{"1min":0.38200000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.61599999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.73399999999999999,"15min":0.69499999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":5.9969999999999999,"5min":5.9969999999999999,"15min":5.9969999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.71499999999999997,"15min":0.68899999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":3.4980000000000002,"5min":3.4980000000000002,"15min":3.4980000000000002},"last":0.77500000000000002}]},{"osd":7,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.72299999999999998,"15min":0.69999999999999996},"min":{"1min":0.498,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":3.5800000000000001,"5min":3.5800000000000001,"15min":3.5800000000000001},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.69399999999999995,"15min":0.68600000000000005},"min":{"1min":0.51800000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0229999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.67700000000000005}]},{"osd":9,"last update":"Fri Dec 1 01:17:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":1.0409999999999999,"15min":1.085},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.647,"5min":5.9690000000000003,"15min":5.9690000000000003},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.91300000000000003,"15min":0.93500000000000005},"min":{"1min":0.51700000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.532,"5min":3.5259999999999998,"15min":3.5259999999999998},"last":0.85199999999999998}]},{"osd":10,"last update":"Fri Dec 1 01:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":6.0469999999999997,"5min":6.0469999999999997,"15min":6.0469999999999997},"last":0.89100000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]}]},{"osd":7,"up_from":50,"seq":214748364836,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28428,"kb_used_data":1476,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716724,"statfs":{"total":95995035648,"available":95965925376,"internally_reserved":0,"allocated":1511424,"data_stored":1016696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.45000000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.40100000000000002,"15min":0.39300000000000002},"min":{"1min":0.27700000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.41099999999999998,"15min":0.40200000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.51000000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56200000000000006,"15min":0.55800000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.65000000000000002}]},{"osd":3,"last update":"Fri Dec 1 01:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.55800000000000005,"15min":0.55000000000000004},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.77700000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61899999999999999,"15min":0.61599999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.85099999999999998,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.58499999999999996}]},{"osd":4,"last update":"Fri Dec 1 01:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.51600000000000001,"15min":0.51200000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.51600000000000001,"15min":0.51100000000000001},"min":{"1min":0.32200000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.47399999999999998}]},{"osd":5,"last update":"Fri Dec 1 01:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.435,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.61499999999999999,"15min":0.60699999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.56200000000000006}]},{"osd":6,"last update":"Fri Dec 1 01:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.64800000000000002,"15min":0.64300000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.66400000000000003,"15min":0.65800000000000003},"min":{"1min":0.42099999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0600000000000001,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.76800000000000002}]},{"osd":8,"last update":"Fri Dec 1 01:16:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.71699999999999997,"15min":0.72099999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.89400000000000002,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.74199999999999999,"15min":0.74099999999999999},"min":{"1min":0.49199999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.66500000000000004}]},{"osd":9,"last update":"Fri Dec 1 01:16:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.81899999999999995}]},{"osd":10,"last update":"Fri Dec 1 01:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.753}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]}]},{"osd":6,"up_from":45,"seq":193273528362,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":901120,"data_stored":423611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.27600000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.502,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.42299999999999999,"15min":0.41299999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":0.30399999999999999}]},{"osd":1,"last update":"Fri Dec 1 01:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61899999999999999,"15min":0.625},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.57499999999999996,"15min":0.59399999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.68100000000000005,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.41699999999999998}]},{"osd":2,"last update":"Fri Dec 1 01:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.59599999999999997,"15min":0.57699999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":2.9870000000000001,"5min":2.9870000000000001,"15min":2.9870000000000001},"last":0.443},{"interface":"front","average":{"1min":0.623,"5min":0.629,"15min":0.623},"min":{"1min":0.42199999999999999,"5min":0.38,"15min":0.38},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.55000000000000004}]},{"osd":3,"last update":"Fri Dec 1 01:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48399999999999999,"15min":0.46999999999999997},"min":{"1min":0.318,"5min":0.308,"15min":0.308},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.377},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.45100000000000001,"15min":0.45900000000000002},"min":{"1min":0.29999999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.501,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.35099999999999998}]},{"osd":4,"last update":"Fri Dec 1 01:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.61899999999999999,"15min":0.61099999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":2.919,"5min":2.919,"15min":2.919},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.65600000000000003,"15min":0.65200000000000002},"min":{"1min":0.45400000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.50800000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.61699999999999999,"15min":0.63300000000000001},"min":{"1min":0.40300000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.67000000000000004,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.67100000000000004,"15min":0.65700000000000003},"min":{"1min":0.437,"5min":0.373,"15min":0.373},"max":{"1min":3.0179999999999998,"5min":3.0179999999999998,"15min":3.0179999999999998},"last":0.52400000000000002}]},{"osd":7,"last update":"Fri Dec 1 01:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.69999999999999996,"15min":0.70399999999999996},"min":{"1min":0.45600000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92000000000000004,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.93300000000000005,"5min":0.995,"15min":0.995},"last":0.58099999999999996}]},{"osd":8,"last update":"Fri Dec 1 01:16:40 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.77100000000000002,"15min":0.77400000000000002},"min":{"1min":0.54500000000000004,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":0.622},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.68400000000000005,"15min":0.69199999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.89000000000000001,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.49299999999999999}]},{"osd":9,"last update":"Fri Dec 1 01:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.69699999999999995,"15min":0.70399999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":2.1320000000000001,"5min":2.1320000000000001,"15min":2.1320000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.71999999999999997,"15min":0.73099999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":0.60599999999999998}]},{"osd":10,"last update":"Fri Dec 1 01:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.63800000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574909,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":962560,"data_stored":483048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:16:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.39500000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.79300000000000004,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.378},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53200000000000003,"15min":0.51900000000000002},"min":{"1min":0.32800000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.69299999999999995,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.41599999999999998}]},{"osd":2,"last update":"Fri Dec 1 01:16:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.53900000000000003,"15min":0.495},"min":{"1min":0.38400000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.878,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.75,"5min":0.57199999999999995,"15min":0.53200000000000003},"min":{"1min":0.40300000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.9399999999999999,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":0.73199999999999998}]},{"osd":3,"last update":"Fri Dec 1 01:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58399999999999996,"15min":0.55900000000000005},"min":{"1min":0.44700000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72099999999999997,"5min":1.198,"15min":1.198},"last":0.503},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59999999999999998,"15min":0.57399999999999995},"min":{"1min":0.46700000000000003,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.77400000000000002,"5min":1.282,"15min":1.282},"last":0.54600000000000004}]},{"osd":4,"last update":"Fri Dec 1 01:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.42899999999999999,"15min":0.41499999999999998},"min":{"1min":0.30299999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.70799999999999996,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.497,"15min":0.48099999999999998},"min":{"1min":0.36599999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.91100000000000003,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.67200000000000004}]},{"osd":5,"last update":"Fri Dec 1 01:16:43 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.56599999999999995,"15min":0.52100000000000002},"min":{"1min":0.437,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.92600000000000005,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.56799999999999995,"15min":0.52400000000000002},"min":{"1min":0.46000000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.754}]},{"osd":6,"last update":"Fri Dec 1 01:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.60299999999999998,"15min":0.59499999999999997},"min":{"1min":0.43099999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.622,"15min":0.61399999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.64200000000000002}]},{"osd":7,"last update":"Fri Dec 1 01:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.55400000000000005,"15min":0.53800000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.88100000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.628},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.56399999999999995,"15min":0.54600000000000004},"min":{"1min":0.40300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.878,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.61399999999999999}]},{"osd":8,"last update":"Fri Dec 1 01:16:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.71499999999999997,"15min":0.71599999999999997},"min":{"1min":0.44800000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.73499999999999999,"15min":0.74099999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.94799999999999995,"5min":1.252,"15min":1.252},"last":0.74399999999999999}]},{"osd":9,"last update":"Fri Dec 1 01:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.71999999999999997}]},{"osd":10,"last update":"Fri Dec 1 01:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1,"5min":1,"15min":1},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.70799999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":900,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":921600,"data_stored":424944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 01:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.498,"15min":0.46800000000000003},"min":{"1min":0.35699999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.80600000000000005,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.53000000000000003,"15min":0.49099999999999999},"min":{"1min":0.39500000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.879,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.52300000000000002}]},{"osd":2,"last update":"Fri Dec 1 01:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.56799999999999995,"15min":0.51200000000000001},"min":{"1min":0.39000000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":3.4590000000000001,"5min":3.4590000000000001,"15min":3.4590000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.54800000000000004,"15min":0.51200000000000001},"min":{"1min":0.35499999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":3.1429999999999998,"5min":3.1429999999999998,"15min":3.1429999999999998},"last":0.59999999999999998}]},{"osd":3,"last update":"Fri Dec 1 01:14:57 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.36699999999999999,"15min":0.35899999999999999},"min":{"1min":0.247,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.318},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.40799999999999997,"15min":0.40200000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.34699999999999998}]},{"osd":4,"last update":"Fri Dec 1 01:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54300000000000004,"15min":0.53300000000000003},"min":{"1min":0.40300000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.81200000000000006,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.58999999999999997,"15min":0.57499999999999996},"min":{"1min":0.42299999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.92300000000000004,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.49299999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56299999999999994,"15min":0.55500000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.61899999999999999,"15min":0.60299999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.708,"5min":1.708,"15min":1.708},"last":0.60999999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:17:00 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.59599999999999997,"15min":0.55500000000000005},"min":{"1min":0.27900000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":3.7189999999999999,"5min":3.7189999999999999,"15min":3.7189999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.60199999999999998,"15min":0.56699999999999995},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":3.9340000000000002,"5min":3.9340000000000002,"15min":3.9340000000000002},"last":0.436}]},{"osd":7,"last update":"Fri Dec 1 01:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.623,"15min":0.621},"min":{"1min":0.43099999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67800000000000005,"15min":0.68200000000000005},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":1.6220000000000001}]},{"osd":8,"last update":"Fri Dec 1 01:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.65400000000000003,"15min":0.66000000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.629,"5min":0.66700000000000004,"15min":0.67300000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.57199999999999995}]},{"osd":9,"last update":"Fri Dec 1 01:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.67600000000000005,"15min":0.67200000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":3.694,"5min":3.694,"15min":3.694},"last":0.58699999999999997},{"interface":"front","average":{"1min":1.1910000000000001,"5min":0.751,"15min":0.67700000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":14.702,"5min":14.702,"15min":14.702},"last":0.503}]},{"osd":10,"last update":"Fri Dec 1 01:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.64300000000000002}]},{"osd":11,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":3.9689999999999999,"5min":3.9689999999999999,"15min":3.9689999999999999},"last":0.625},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":0.61699999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":983040,"data_stored":483613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.497,"15min":0.47999999999999998},"min":{"1min":0.32700000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.79800000000000004,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.504,"5min":0.497,"15min":0.47899999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70399999999999996,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.441}]},{"osd":1,"last update":"Fri Dec 1 01:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.52700000000000002,"15min":0.51000000000000001},"min":{"1min":0.36199999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.72899999999999998,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.58999999999999997,"15min":0.54200000000000004},"min":{"1min":0.35899999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":5.4169999999999998,"5min":5.4169999999999998,"15min":5.4169999999999998},"last":0.69499999999999995}]},{"osd":3,"last update":"Fri Dec 1 01:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.59799999999999998,"15min":0.53000000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.91400000000000003,"5min":5.4710000000000001,"15min":5.4710000000000001},"last":0.622},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56999999999999995,"15min":0.53000000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.78500000000000003,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.59899999999999998}]},{"osd":4,"last update":"Fri Dec 1 01:16:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.53200000000000003,"15min":0.51800000000000002},"min":{"1min":0.378,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.53100000000000003,"15min":0.52900000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.73299999999999998}]},{"osd":5,"last update":"Fri Dec 1 01:16:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.437,"15min":0.42199999999999999},"min":{"1min":0.312,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.49399999999999999,"15min":0.47199999999999998},"min":{"1min":0.36599999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.39300000000000002}]},{"osd":6,"last update":"Fri Dec 1 01:17:00 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.64500000000000002,"15min":0.60499999999999998},"min":{"1min":0.43099999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.89300000000000002,"5min":5.5019999999999998,"15min":5.5019999999999998},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.63800000000000001,"15min":0.63400000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77100000000000002,"5min":1.429,"15min":1.429},"last":0.51300000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.54000000000000004}]},{"osd":8,"last update":"Fri Dec 1 01:16:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.53700000000000003,"15min":0.52600000000000002},"min":{"1min":0.34799999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.58899999999999997,"15min":0.58299999999999996},"min":{"1min":0.39900000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.65200000000000002}]},{"osd":9,"last update":"Fri Dec 1 01:17:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.69799999999999995,"15min":0.70099999999999996},"min":{"1min":0.52000000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.004,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.68700000000000006,"15min":0.68200000000000005},"min":{"1min":0.51700000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94299999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.66600000000000004}]},{"osd":10,"last update":"Fri Dec 1 01:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":5.4450000000000003,"5min":5.4450000000000003,"15min":5.4450000000000003},"last":0.70599999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]}]},{"osd":3,"up_from":25,"seq":107374182454,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":917504,"data_stored":423490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:17:02 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.34300000000000003,"15min":0.31900000000000001},"min":{"1min":0.26800000000000002,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.51300000000000001,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.35699999999999998,"15min":0.33800000000000002},"min":{"1min":0.26600000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.48299999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.39400000000000002}]},{"osd":1,"last update":"Fri Dec 1 01:17:02 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.53500000000000003,"15min":0.505},"min":{"1min":0.38200000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73099999999999998,"5min":0.996,"15min":0.996},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.375,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.69599999999999995,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.54100000000000004}]},{"osd":2,"last update":"Fri Dec 1 01:17:02 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.51700000000000002},"min":{"1min":0.39700000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.749,"5min":0.88,"15min":0.88},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55100000000000005,"15min":0.52800000000000002},"min":{"1min":0.41599999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.78600000000000003,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.59399999999999997}]},{"osd":4,"last update":"Fri Dec 1 01:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.57799999999999996,"15min":0.55600000000000005},"min":{"1min":0.40899999999999997,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.60399999999999998,"15min":0.58699999999999997},"min":{"1min":0.44500000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.629}]},{"osd":5,"last update":"Fri Dec 1 01:16:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.55800000000000005,"15min":0.55500000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.65100000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.57399999999999995,"15min":0.57299999999999995},"min":{"1min":0.40100000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.69299999999999995,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.80500000000000005}]},{"osd":6,"last update":"Fri Dec 1 01:17:02 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.46500000000000002,"15min":0.47299999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.55100000000000005,"15min":0.54500000000000004},"min":{"1min":0.34999999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.69899999999999995,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.52700000000000002}]},{"osd":7,"last update":"Fri Dec 1 01:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.63300000000000001,"15min":0.63100000000000001},"min":{"1min":0.46300000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.82999999999999996,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.65600000000000003,"15min":0.65100000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.56799999999999995}]},{"osd":8,"last update":"Fri Dec 1 01:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.51200000000000001}]},{"osd":9,"last update":"Fri Dec 1 01:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.71199999999999997,"15min":0.73399999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77300000000000002,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.63600000000000001,"15min":0.65100000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.75,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.57799999999999996}]},{"osd":10,"last update":"Fri Dec 1 01:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.60299999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]}]},{"osd":4,"up_from":32,"seq":137438953522,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":901120,"data_stored":423466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.54300000000000004,"15min":0.50900000000000001},"min":{"1min":0.48799999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.373},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.53400000000000003,"15min":0.499},"min":{"1min":0.47199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.497}]},{"osd":1,"last update":"Fri Dec 1 01:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.40500000000000003,"15min":0.378},"min":{"1min":0.29599999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.39800000000000002,"15min":0.371},"min":{"1min":0.32100000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.54300000000000004}]},{"osd":2,"last update":"Fri Dec 1 01:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.55900000000000005,"15min":0.53000000000000003},"min":{"1min":0.48199999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.59499999999999997,"15min":0.56799999999999995},"min":{"1min":0.40400000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.63500000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.53300000000000003,"15min":0.499},"min":{"1min":0.45700000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.97899999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.52200000000000002,"15min":0.504},"min":{"1min":0.373,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.89300000000000002,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.504}]},{"osd":5,"last update":"Fri Dec 1 01:16:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57399999999999995,"15min":0.55800000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.64300000000000002,"15min":0.60599999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.67700000000000005}]},{"osd":6,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.629,"15min":0.61199999999999999},"min":{"1min":0.42999999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.94699999999999995,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.66000000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.80700000000000005}]},{"osd":7,"last update":"Fri Dec 1 01:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.57699999999999996,"15min":0.55700000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.56599999999999995,"15min":0.55000000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.872}]},{"osd":8,"last update":"Fri Dec 1 01:16:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.63100000000000001,"15min":0.63600000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.621},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76000000000000001,"15min":0.75900000000000001},"min":{"1min":0.47699999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.88400000000000001}]},{"osd":9,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.72999999999999998,"15min":0.73499999999999999},"min":{"1min":0.46600000000000003,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.79000000000000004,"15min":0.80000000000000004},"min":{"1min":0.51600000000000001,"5min":0.503,"15min":0.503},"max":{"1min":1.0900000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.91200000000000003}]},{"osd":10,"last update":"Fri Dec 1 01:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.89600000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94899999999999995}]}]},{"osd":5,"up_from":37,"seq":158913789997,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28436,"kb_used_data":1484,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716716,"statfs":{"total":95995035648,"available":95965917184,"internally_reserved":0,"allocated":1519616,"data_stored":1016139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:16:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.52700000000000002,"15min":0.50900000000000001},"min":{"1min":0.39400000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52900000000000003,"15min":0.52500000000000002},"min":{"1min":0.38400000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.37}]},{"osd":1,"last update":"Fri Dec 1 01:16:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.51100000000000001,"15min":0.48699999999999999},"min":{"1min":0.38300000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.68300000000000005,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.53200000000000003,"15min":0.501},"min":{"1min":0.39800000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.495}]},{"osd":2,"last update":"Fri Dec 1 01:16:39 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.39400000000000002,"15min":0.38600000000000001},"min":{"1min":0.31,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.38800000000000001,"15min":0.376},"min":{"1min":0.28799999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.28499999999999998}]},{"osd":3,"last update":"Fri Dec 1 01:16:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.54100000000000004,"15min":0.53100000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.55300000000000005,"15min":0.53300000000000003},"min":{"1min":0.40600000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.41299999999999998}]},{"osd":4,"last update":"Fri Dec 1 01:16:39 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.55600000000000005,"15min":0.53200000000000003},"min":{"1min":0.45000000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.55200000000000005,"15min":0.52900000000000003},"min":{"1min":0.40600000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.78200000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.53000000000000003}]},{"osd":6,"last update":"Fri Dec 1 01:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.64200000000000002,"15min":0.626},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":2.391,"5min":2.391,"15min":2.391},"last":0.501},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.66000000000000003,"15min":0.63},"min":{"1min":0.42899999999999999,"5min":0.37,"15min":0.37},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":0.39000000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.64900000000000002,"15min":0.65100000000000002},"min":{"1min":0.49399999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.85299999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.624,"5min":0.61299999999999999,"15min":0.61099999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.48599999999999999}]},{"osd":8,"last update":"Fri Dec 1 01:16:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58099999999999996,"15min":0.57999999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.754,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.58999999999999997}]},{"osd":9,"last update":"Fri Dec 1 01:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.55200000000000005}]},{"osd":10,"last update":"Fri Dec 1 01:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.57699999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T01:17:10.038 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph pg dump --format=json 2023-12-01T01:17:10.591 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:17:11.832 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:11.829+0000 7f0b126ed700 1 -- 172.21.15.77:0/3523133316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b0c100690 msgr2=0x7f0b0c10cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:11.832 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:11.829+0000 7f0b126ed700 1 --2- 172.21.15.77:0/3523133316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b0c100690 0x7f0b0c10cb30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0b0400b0a0 tx=0x7f0b0400b3b0 comp rx=0 tx=0).stop 2023-12-01T01:17:11.832 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:11.829+0000 7f0b126ed700 1 -- 172.21.15.77:0/3523133316 shutdown_connections 2023-12-01T01:17:11.832 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:11.829+0000 7f0b126ed700 1 --2- 172.21.15.77:0/3523133316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b0c100690 0x7f0b0c10cb30 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:11.832 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:11.829+0000 7f0b126ed700 1 --2- 172.21.15.77:0/3523133316 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b0c105d80 0x7f0b0c106160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:11.833 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:11.829+0000 7f0b126ed700 1 --2- 172.21.15.77:0/3523133316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b0c0ffcd0 0x7f0b0c100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:11.833 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:11.829+0000 7f0b126ed700 1 -- 172.21.15.77:0/3523133316 >> 172.21.15.77:0/3523133316 conn(0x7f0b0c076d90 msgr2=0x7f0b0c0771a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:11.833 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:11.829+0000 7f0b126ed700 1 -- 172.21.15.77:0/3523133316 shutdown_connections 2023-12-01T01:17:11.833 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:11.829+0000 7f0b126ed700 1 -- 172.21.15.77:0/3523133316 wait complete. 2023-12-01T01:17:11.833 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:11.829+0000 7f0b126ed700 1 Processor -- start 2023-12-01T01:17:11.833 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:11.829+0000 7f0b126ed700 1 -- start start 2023-12-01T01:17:11.833 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:11.830+0000 7f0b126ed700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b0c0ffcd0 0x7f0b0c197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:11.833 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:11.830+0000 7f0b126ed700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b0c100690 0x7f0b0c1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:11.833 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:11.830+0000 7f0b126ed700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b0c105d80 0x7f0b0c1a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:11.833 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:11.830+0000 7f0b126ed700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c1a3270 con 0x7f0b0c100690 2023-12-01T01:17:11.833 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:11.830+0000 7f0b126ed700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c1a33b0 con 0x7f0b0c0ffcd0 2023-12-01T01:17:11.833 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:11.830+0000 7f0b126ed700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c1a34f0 con 0x7f0b0c105d80 2023-12-01T01:17:11.833 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:11.830+0000 7f0b0b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b0c100690 0x7f0b0c1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:11.833 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:11.830+0000 7f0b0b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b0c100690 0x7f0b0c1981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58124/0 (socket says 172.21.15.77:58124) 2023-12-01T01:17:11.834 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:11.830+0000 7f0b0b7fe700 1 -- 172.21.15.77:0/2373871044 learned_addr learned my addr 172.21.15.77:0/2373871044 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:11.834 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:11.830+0000 7f0b0bfff700 1 --2- 172.21.15.77:0/2373871044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b0c0ffcd0 0x7f0b0c197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:11.834 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:11.830+0000 7f0b10c8a700 1 --2- 172.21.15.77:0/2373871044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b0c105d80 0x7f0b0c1a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:11.834 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:11.830+0000 7f0b0b7fe700 1 -- 172.21.15.77:0/2373871044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b0c105d80 msgr2=0x7f0b0c1a2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:11.834 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:11.830+0000 7f0b0b7fe700 1 --2- 172.21.15.77:0/2373871044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b0c105d80 0x7f0b0c1a2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:11.834 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:11.830+0000 7f0b0b7fe700 1 -- 172.21.15.77:0/2373871044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b0c0ffcd0 msgr2=0x7f0b0c197cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:11.834 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:11.830+0000 7f0b0b7fe700 1 --2- 172.21.15.77:0/2373871044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b0c0ffcd0 0x7f0b0c197cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:11.834 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:11.830+0000 7f0b0b7fe700 1 -- 172.21.15.77:0/2373871044 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b04009d00 con 0x7f0b0c100690 2023-12-01T01:17:11.834 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:11.830+0000 7f0b0b7fe700 1 --2- 172.21.15.77:0/2373871044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b0c100690 0x7f0b0c1981f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0afc00c9e0 tx=0x7f0afc00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:11.834 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:11.831+0000 7f0b097fa700 1 -- 172.21.15.77:0/2373871044 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0afc007ac0 con 0x7f0b0c100690 2023-12-01T01:17:11.834 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:11.831+0000 7f0b126ed700 1 -- 172.21.15.77:0/2373871044 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b0c1a3750 con 0x7f0b0c100690 2023-12-01T01:17:11.834 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:11.831+0000 7f0b097fa700 1 -- 172.21.15.77:0/2373871044 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0afc007c20 con 0x7f0b0c100690 2023-12-01T01:17:11.834 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:11.831+0000 7f0b097fa700 1 -- 172.21.15.77:0/2373871044 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0afc018660 con 0x7f0b0c100690 2023-12-01T01:17:11.835 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:11.831+0000 7f0b126ed700 1 -- 172.21.15.77:0/2373871044 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b0c1a3c50 con 0x7f0b0c100690 2023-12-01T01:17:11.835 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:11.832+0000 7f0b126ed700 1 -- 172.21.15.77:0/2373871044 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b0c04ea90 con 0x7f0b0c100690 2023-12-01T01:17:11.835 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:11.833+0000 7f0b097fa700 1 -- 172.21.15.77:0/2373871044 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f0afc00f450 con 0x7f0b0c100690 2023-12-01T01:17:11.836 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:11.833+0000 7f0b097fa700 1 --2- 172.21.15.77:0/2373871044 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f0af40a05b0 0x7f0af40a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:11.836 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:11.833+0000 7f0b0bfff700 1 --2- 172.21.15.77:0/2373871044 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f0af40a05b0 0x7f0af40a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:11.836 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:11.833+0000 7f0b097fa700 1 -- 172.21.15.77:0/2373871044 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0afc0cbb90 con 0x7f0b0c100690 2023-12-01T01:17:11.836 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:11.833+0000 7f0b0bfff700 1 --2- 172.21.15.77:0/2373871044 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f0af40a05b0 0x7f0af40a2a70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0b00005950 tx=0x7f0b0000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:11.838 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:11.837+0000 7f0b097fa700 1 -- 172.21.15.77:0/2373871044 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0afc095810 con 0x7f0b0c100690 2023-12-01T01:17:11.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:11 smithi077 ceph-mon[98867]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-01T01:17:11.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:11 smithi077 ceph-mon[98867]: pgmap v275: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2023-12-01T01:17:12.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:11 smithi203 ceph-mon[103581]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-01T01:17:12.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:11 smithi203 ceph-mon[103581]: pgmap v275: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2023-12-01T01:17:12.023 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:12.022+0000 7f0b126ed700 1 -- 172.21.15.77:0/2373871044 --> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0b0c0feeb0 con 0x7f0af40a05b0 2023-12-01T01:17:12.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:11 smithi137 ceph-mon[103312]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-01T01:17:12.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:11 smithi137 ceph-mon[103312]: pgmap v275: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2023-12-01T01:17:12.032 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:12.031+0000 7f0b097fa700 1 -- 172.21.15.77:0/2373871044 <== mgr.14152 v2:172.21.15.77:6800/3838941679 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340666 (secure 0 0 0) 0x7f0b0c0feeb0 con 0x7f0af40a05b0 2023-12-01T01:17:12.036 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:17:12.039 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:12.034+0000 7f0b126ed700 1 -- 172.21.15.77:0/2373871044 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f0af40a05b0 msgr2=0x7f0af40a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:12.039 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:12.034+0000 7f0b126ed700 1 --2- 172.21.15.77:0/2373871044 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f0af40a05b0 0x7f0af40a2a70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0b00005950 tx=0x7f0b0000a300 comp rx=0 tx=0).stop 2023-12-01T01:17:12.039 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:12.034+0000 7f0b126ed700 1 -- 172.21.15.77:0/2373871044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b0c100690 msgr2=0x7f0b0c1981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:12.039 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:12.034+0000 7f0b126ed700 1 --2- 172.21.15.77:0/2373871044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b0c100690 0x7f0b0c1981f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0afc00c9e0 tx=0x7f0afc00ccf0 comp rx=0 tx=0).stop 2023-12-01T01:17:12.039 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:12.034+0000 7f0b126ed700 1 -- 172.21.15.77:0/2373871044 shutdown_connections 2023-12-01T01:17:12.039 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:12.034+0000 7f0b126ed700 1 --2- 172.21.15.77:0/2373871044 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f0af40a05b0 0x7f0af40a2a70 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:12.039 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:12.034+0000 7f0b126ed700 1 --2- 172.21.15.77:0/2373871044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b0c105d80 0x7f0b0c1a2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:12.039 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:12.034+0000 7f0b126ed700 1 --2- 172.21.15.77:0/2373871044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b0c0ffcd0 0x7f0b0c197cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:12.039 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:12.034+0000 7f0b126ed700 1 --2- 172.21.15.77:0/2373871044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b0c100690 0x7f0b0c1981f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:12.039 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:12.034+0000 7f0b126ed700 1 -- 172.21.15.77:0/2373871044 >> 172.21.15.77:0/2373871044 conn(0x7f0b0c076d90 msgr2=0x7f0b0c0fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:12.040 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:12.034+0000 7f0b126ed700 1 -- 172.21.15.77:0/2373871044 shutdown_connections 2023-12-01T01:17:12.040 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:12.034+0000 7f0b126ed700 1 -- 172.21.15.77:0/2373871044 wait complete. 2023-12-01T01:17:12.040 INFO:teuthology.orchestra.run.smithi077.stderr:dumped all 2023-12-01T01:17:12.447 INFO:teuthology.orchestra.run.smithi077.stdout:{"pg_ready":true,"pg_map":{"version":275,"stamp":"2023-12-01T01:17:10.277174+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655767,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":527,"num_read_kb":377,"num_write":412,"num_write_kb":3066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":340,"ondisk_log_size":340,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336052,"kb_used_data":12628,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124605772,"statfs":{"total":1151940427776,"available":1151596310528,"internally_reserved":0,"allocated":12931072,"data_stored":7040407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1652,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":119,"num_read_kb":52,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003242"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.592825+0000","last_change":"2023-12-01T01:16:25.373817+0000","last_active":"2023-12-01T01:16:25.592825+0000","last_peered":"2023-12-01T01:16:25.592825+0000","last_clean":"2023-12-01T01:16:25.592825+0000","last_became_active":"2023-12-01T01:16:25.373154+0000","last_became_peered":"2023-12-01T01:16:25.373154+0000","last_unstale":"2023-12-01T01:16:25.592825+0000","last_undegraded":"2023-12-01T01:16:25.592825+0000","last_fullsized":"2023-12-01T01:16:25.592825+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:33:43.284106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.324984+0000","last_change":"2023-12-01T01:16:25.364855+0000","last_active":"2023-12-01T01:16:27.324984+0000","last_peered":"2023-12-01T01:16:27.324984+0000","last_clean":"2023-12-01T01:16:27.324984+0000","last_became_active":"2023-12-01T01:16:25.364536+0000","last_became_peered":"2023-12-01T01:16:25.364536+0000","last_unstale":"2023-12-01T01:16:27.324984+0000","last_undegraded":"2023-12-01T01:16:27.324984+0000","last_fullsized":"2023-12-01T01:16:27.324984+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:45:23.471557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.324702+0000","last_change":"2023-12-01T01:16:25.372443+0000","last_active":"2023-12-01T01:16:27.324702+0000","last_peered":"2023-12-01T01:16:27.324702+0000","last_clean":"2023-12-01T01:16:27.324702+0000","last_became_active":"2023-12-01T01:16:25.372133+0000","last_became_peered":"2023-12-01T01:16:25.372133+0000","last_unstale":"2023-12-01T01:16:27.324702+0000","last_undegraded":"2023-12-01T01:16:27.324702+0000","last_fullsized":"2023-12-01T01:16:27.324702+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:54:19.439408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.554816+0000","last_change":"2023-12-01T01:16:25.365857+0000","last_active":"2023-12-01T01:16:26.554816+0000","last_peered":"2023-12-01T01:16:26.554816+0000","last_clean":"2023-12-01T01:16:26.554816+0000","last_became_active":"2023-12-01T01:16:25.364992+0000","last_became_peered":"2023-12-01T01:16:25.364992+0000","last_unstale":"2023-12-01T01:16:26.554816+0000","last_undegraded":"2023-12-01T01:16:26.554816+0000","last_fullsized":"2023-12-01T01:16:26.554816+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:01:11.064587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.071460+0000","last_change":"2023-12-01T01:16:25.364081+0000","last_active":"2023-12-01T01:16:26.071460+0000","last_peered":"2023-12-01T01:16:26.071460+0000","last_clean":"2023-12-01T01:16:26.071460+0000","last_became_active":"2023-12-01T01:16:25.363187+0000","last_became_peered":"2023-12-01T01:16:25.363187+0000","last_unstale":"2023-12-01T01:16:26.071460+0000","last_undegraded":"2023-12-01T01:16:26.071460+0000","last_fullsized":"2023-12-01T01:16:26.071460+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:05:42.852128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325515+0000","last_change":"2023-12-01T01:16:25.366104+0000","last_active":"2023-12-01T01:16:27.325515+0000","last_peered":"2023-12-01T01:16:27.325515+0000","last_clean":"2023-12-01T01:16:27.325515+0000","last_became_active":"2023-12-01T01:16:25.365778+0000","last_became_peered":"2023-12-01T01:16:25.365778+0000","last_unstale":"2023-12-01T01:16:27.325515+0000","last_undegraded":"2023-12-01T01:16:27.325515+0000","last_fullsized":"2023-12-01T01:16:27.325515+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:30:09.485609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.349881+0000","last_change":"2023-12-01T01:16:25.371354+0000","last_active":"2023-12-01T01:16:26.349881+0000","last_peered":"2023-12-01T01:16:26.349881+0000","last_clean":"2023-12-01T01:16:26.349881+0000","last_became_active":"2023-12-01T01:16:25.370955+0000","last_became_peered":"2023-12-01T01:16:25.370955+0000","last_unstale":"2023-12-01T01:16:26.349881+0000","last_undegraded":"2023-12-01T01:16:26.349881+0000","last_fullsized":"2023-12-01T01:16:26.349881+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:38:53.508285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.571448+0000","last_change":"2023-12-01T01:16:25.368187+0000","last_active":"2023-12-01T01:16:26.571448+0000","last_peered":"2023-12-01T01:16:26.571448+0000","last_clean":"2023-12-01T01:16:26.571448+0000","last_became_active":"2023-12-01T01:16:25.367854+0000","last_became_peered":"2023-12-01T01:16:25.367854+0000","last_unstale":"2023-12-01T01:16:26.571448+0000","last_undegraded":"2023-12-01T01:16:26.571448+0000","last_fullsized":"2023-12-01T01:16:26.571448+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:22:35.159185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325090+0000","last_change":"2023-12-01T01:16:25.372829+0000","last_active":"2023-12-01T01:16:27.325090+0000","last_peered":"2023-12-01T01:16:27.325090+0000","last_clean":"2023-12-01T01:16:27.325090+0000","last_became_active":"2023-12-01T01:16:25.372469+0000","last_became_peered":"2023-12-01T01:16:25.372469+0000","last_unstale":"2023-12-01T01:16:27.325090+0000","last_undegraded":"2023-12-01T01:16:27.325090+0000","last_fullsized":"2023-12-01T01:16:27.325090+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:25:10.300138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325829+0000","last_change":"2023-12-01T01:16:25.379135+0000","last_active":"2023-12-01T01:16:27.325829+0000","last_peered":"2023-12-01T01:16:27.325829+0000","last_clean":"2023-12-01T01:16:27.325829+0000","last_became_active":"2023-12-01T01:16:25.378852+0000","last_became_peered":"2023-12-01T01:16:25.378852+0000","last_unstale":"2023-12-01T01:16:27.325829+0000","last_undegraded":"2023-12-01T01:16:27.325829+0000","last_fullsized":"2023-12-01T01:16:27.325829+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:42:16.803389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.350179+0000","last_change":"2023-12-01T01:16:25.367602+0000","last_active":"2023-12-01T01:16:26.350179+0000","last_peered":"2023-12-01T01:16:26.350179+0000","last_clean":"2023-12-01T01:16:26.350179+0000","last_became_active":"2023-12-01T01:16:25.366565+0000","last_became_peered":"2023-12-01T01:16:25.366565+0000","last_unstale":"2023-12-01T01:16:26.350179+0000","last_undegraded":"2023-12-01T01:16:26.350179+0000","last_fullsized":"2023-12-01T01:16:26.350179+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:28:52.576838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.554806+0000","last_change":"2023-12-01T01:16:25.375004+0000","last_active":"2023-12-01T01:16:26.554806+0000","last_peered":"2023-12-01T01:16:26.554806+0000","last_clean":"2023-12-01T01:16:26.554806+0000","last_became_active":"2023-12-01T01:16:25.374320+0000","last_became_peered":"2023-12-01T01:16:25.374320+0000","last_unstale":"2023-12-01T01:16:26.554806+0000","last_undegraded":"2023-12-01T01:16:26.554806+0000","last_fullsized":"2023-12-01T01:16:26.554806+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:40:49.482061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.352492+0000","last_change":"2023-12-01T01:16:25.367858+0000","last_active":"2023-12-01T01:16:26.352492+0000","last_peered":"2023-12-01T01:16:26.352492+0000","last_clean":"2023-12-01T01:16:26.352492+0000","last_became_active":"2023-12-01T01:16:25.367560+0000","last_became_peered":"2023-12-01T01:16:25.367560+0000","last_unstale":"2023-12-01T01:16:26.352492+0000","last_undegraded":"2023-12-01T01:16:26.352492+0000","last_fullsized":"2023-12-01T01:16:26.352492+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:33:21.048672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.324783+0000","last_change":"2023-12-01T01:16:25.375654+0000","last_active":"2023-12-01T01:16:27.324783+0000","last_peered":"2023-12-01T01:16:27.324783+0000","last_clean":"2023-12-01T01:16:27.324783+0000","last_became_active":"2023-12-01T01:16:25.375379+0000","last_became_peered":"2023-12-01T01:16:25.375379+0000","last_unstale":"2023-12-01T01:16:27.324783+0000","last_undegraded":"2023-12-01T01:16:27.324783+0000","last_fullsized":"2023-12-01T01:16:27.324783+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:33:42.335365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325759+0000","last_change":"2023-12-01T01:16:25.372546+0000","last_active":"2023-12-01T01:16:27.325759+0000","last_peered":"2023-12-01T01:16:27.325759+0000","last_clean":"2023-12-01T01:16:27.325759+0000","last_became_active":"2023-12-01T01:16:25.372327+0000","last_became_peered":"2023-12-01T01:16:25.372327+0000","last_unstale":"2023-12-01T01:16:27.325759+0000","last_undegraded":"2023-12-01T01:16:27.325759+0000","last_fullsized":"2023-12-01T01:16:27.325759+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:12:34.171834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.324599+0000","last_change":"2023-12-01T01:16:25.366449+0000","last_active":"2023-12-01T01:16:27.324599+0000","last_peered":"2023-12-01T01:16:27.324599+0000","last_clean":"2023-12-01T01:16:27.324599+0000","last_became_active":"2023-12-01T01:16:25.366147+0000","last_became_peered":"2023-12-01T01:16:25.366147+0000","last_unstale":"2023-12-01T01:16:27.324599+0000","last_undegraded":"2023-12-01T01:16:27.324599+0000","last_fullsized":"2023-12-01T01:16:27.324599+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:01:29.670329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.592834+0000","last_change":"2023-12-01T01:16:25.374016+0000","last_active":"2023-12-01T01:16:25.592834+0000","last_peered":"2023-12-01T01:16:25.592834+0000","last_clean":"2023-12-01T01:16:25.592834+0000","last_became_active":"2023-12-01T01:16:25.373543+0000","last_became_peered":"2023-12-01T01:16:25.373543+0000","last_unstale":"2023-12-01T01:16:25.592834+0000","last_undegraded":"2023-12-01T01:16:25.592834+0000","last_fullsized":"2023-12-01T01:16:25.592834+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:36:22.858446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.584312+0000","last_change":"2023-12-01T01:16:25.363598+0000","last_active":"2023-12-01T01:16:26.584312+0000","last_peered":"2023-12-01T01:16:26.584312+0000","last_clean":"2023-12-01T01:16:26.584312+0000","last_became_active":"2023-12-01T01:16:25.363111+0000","last_became_peered":"2023-12-01T01:16:25.363111+0000","last_unstale":"2023-12-01T01:16:26.584312+0000","last_undegraded":"2023-12-01T01:16:26.584312+0000","last_fullsized":"2023-12-01T01:16:26.584312+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:42:08.171789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.375829+0000","last_change":"2023-12-01T01:16:25.368446+0000","last_active":"2023-12-01T01:16:25.375829+0000","last_peered":"2023-12-01T01:16:25.375829+0000","last_clean":"2023-12-01T01:16:25.375829+0000","last_became_active":"2023-12-01T01:16:25.361893+0000","last_became_peered":"2023-12-01T01:16:25.361893+0000","last_unstale":"2023-12-01T01:16:25.375829+0000","last_undegraded":"2023-12-01T01:16:25.375829+0000","last_fullsized":"2023-12-01T01:16:25.375829+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:05:45.697338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'13","reported_seq":60,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:17:04.664374+0000","last_change":"2023-12-01T01:16:24.934939+0000","last_active":"2023-12-01T01:17:04.664374+0000","last_peered":"2023-12-01T01:17:04.664374+0000","last_clean":"2023-12-01T01:17:04.664374+0000","last_became_active":"2023-12-01T01:16:24.934481+0000","last_became_peered":"2023-12-01T01:16:24.934481+0000","last_unstale":"2023-12-01T01:17:04.664374+0000","last_undegraded":"2023-12-01T01:17:04.664374+0000","last_fullsized":"2023-12-01T01:17:04.664374+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:39:49.804567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.571536+0000","last_change":"2023-12-01T01:16:25.364061+0000","last_active":"2023-12-01T01:16:26.571536+0000","last_peered":"2023-12-01T01:16:26.571536+0000","last_clean":"2023-12-01T01:16:26.571536+0000","last_became_active":"2023-12-01T01:16:25.363678+0000","last_became_peered":"2023-12-01T01:16:25.363678+0000","last_unstale":"2023-12-01T01:16:26.571536+0000","last_undegraded":"2023-12-01T01:16:26.571536+0000","last_fullsized":"2023-12-01T01:16:26.571536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:24:07.107604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325949+0000","last_change":"2023-12-01T01:16:24.939338+0000","last_active":"2023-12-01T01:16:27.325949+0000","last_peered":"2023-12-01T01:16:27.325949+0000","last_clean":"2023-12-01T01:16:27.325949+0000","last_became_active":"2023-12-01T01:16:24.939007+0000","last_became_peered":"2023-12-01T01:16:24.939007+0000","last_unstale":"2023-12-01T01:16:27.325949+0000","last_undegraded":"2023-12-01T01:16:27.325949+0000","last_fullsized":"2023-12-01T01:16:27.325949+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:56:50.732188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325303+0000","last_change":"2023-12-01T01:16:25.362231+0000","last_active":"2023-12-01T01:16:27.325303+0000","last_peered":"2023-12-01T01:16:27.325303+0000","last_clean":"2023-12-01T01:16:27.325303+0000","last_became_active":"2023-12-01T01:16:25.361920+0000","last_became_peered":"2023-12-01T01:16:25.361920+0000","last_unstale":"2023-12-01T01:16:27.325303+0000","last_undegraded":"2023-12-01T01:16:27.325303+0000","last_fullsized":"2023-12-01T01:16:27.325303+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:22:22.104480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'12","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:31.605443+0000","last_change":"2023-12-01T01:16:24.931801+0000","last_active":"2023-12-01T01:16:31.605443+0000","last_peered":"2023-12-01T01:16:31.605443+0000","last_clean":"2023-12-01T01:16:31.605443+0000","last_became_active":"2023-12-01T01:16:24.931451+0000","last_became_peered":"2023-12-01T01:16:24.931451+0000","last_unstale":"2023-12-01T01:16:31.605443+0000","last_undegraded":"2023-12-01T01:16:31.605443+0000","last_fullsized":"2023-12-01T01:16:31.605443+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:22:41.666502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.324789+0000","last_change":"2023-12-01T01:16:25.361789+0000","last_active":"2023-12-01T01:16:27.324789+0000","last_peered":"2023-12-01T01:16:27.324789+0000","last_clean":"2023-12-01T01:16:27.324789+0000","last_became_active":"2023-12-01T01:16:25.361480+0000","last_became_peered":"2023-12-01T01:16:25.361480+0000","last_unstale":"2023-12-01T01:16:27.324789+0000","last_undegraded":"2023-12-01T01:16:27.324789+0000","last_fullsized":"2023-12-01T01:16:27.324789+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:07:00.742167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'5","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:31.604682+0000","last_change":"2023-12-01T01:16:24.933362+0000","last_active":"2023-12-01T01:16:31.604682+0000","last_peered":"2023-12-01T01:16:31.604682+0000","last_clean":"2023-12-01T01:16:31.604682+0000","last_became_active":"2023-12-01T01:16:24.933049+0000","last_became_peered":"2023-12-01T01:16:24.933049+0000","last_unstale":"2023-12-01T01:16:31.604682+0000","last_undegraded":"2023-12-01T01:16:31.604682+0000","last_fullsized":"2023-12-01T01:16:31.604682+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:47:37.153344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325647+0000","last_change":"2023-12-01T01:16:25.364419+0000","last_active":"2023-12-01T01:16:27.325647+0000","last_peered":"2023-12-01T01:16:27.325647+0000","last_clean":"2023-12-01T01:16:27.325647+0000","last_became_active":"2023-12-01T01:16:25.363022+0000","last_became_peered":"2023-12-01T01:16:25.363022+0000","last_unstale":"2023-12-01T01:16:27.325647+0000","last_undegraded":"2023-12-01T01:16:27.325647+0000","last_fullsized":"2023-12-01T01:16:27.325647+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:58:45.660625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'6","reported_seq":58,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:17:07.653533+0000","last_change":"2023-12-01T01:16:24.935947+0000","last_active":"2023-12-01T01:17:07.653533+0000","last_peered":"2023-12-01T01:17:07.653533+0000","last_clean":"2023-12-01T01:17:07.653533+0000","last_became_active":"2023-12-01T01:16:24.935061+0000","last_became_peered":"2023-12-01T01:16:24.935061+0000","last_unstale":"2023-12-01T01:17:07.653533+0000","last_undegraded":"2023-12-01T01:17:07.653533+0000","last_fullsized":"2023-12-01T01:17:07.653533+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:00:31.551362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":967,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.071450+0000","last_change":"2023-12-01T01:16:25.363229+0000","last_active":"2023-12-01T01:16:26.071450+0000","last_peered":"2023-12-01T01:16:26.071450+0000","last_clean":"2023-12-01T01:16:26.071450+0000","last_became_active":"2023-12-01T01:16:25.362770+0000","last_became_peered":"2023-12-01T01:16:25.362770+0000","last_unstale":"2023-12-01T01:16:26.071450+0000","last_undegraded":"2023-12-01T01:16:26.071450+0000","last_fullsized":"2023-12-01T01:16:26.071450+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:45:02.782201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:30.654762+0000","last_change":"2023-12-01T01:16:24.930509+0000","last_active":"2023-12-01T01:16:30.654762+0000","last_peered":"2023-12-01T01:16:30.654762+0000","last_clean":"2023-12-01T01:16:30.654762+0000","last_became_active":"2023-12-01T01:16:24.930188+0000","last_became_peered":"2023-12-01T01:16:24.930188+0000","last_unstale":"2023-12-01T01:16:30.654762+0000","last_undegraded":"2023-12-01T01:16:30.654762+0000","last_fullsized":"2023-12-01T01:16:30.654762+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:04:17.748363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325530+0000","last_change":"2023-12-01T01:16:25.363383+0000","last_active":"2023-12-01T01:16:27.325530+0000","last_peered":"2023-12-01T01:16:27.325530+0000","last_clean":"2023-12-01T01:16:27.325530+0000","last_became_active":"2023-12-01T01:16:25.363085+0000","last_became_peered":"2023-12-01T01:16:25.363085+0000","last_unstale":"2023-12-01T01:16:27.325530+0000","last_undegraded":"2023-12-01T01:16:27.325530+0000","last_fullsized":"2023-12-01T01:16:27.325530+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:34:36.921706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"82'3","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:29.637968+0000","last_change":"2023-12-01T01:16:24.929081+0000","last_active":"2023-12-01T01:16:29.637968+0000","last_peered":"2023-12-01T01:16:29.637968+0000","last_clean":"2023-12-01T01:16:29.637968+0000","last_became_active":"2023-12-01T01:16:24.928747+0000","last_became_peered":"2023-12-01T01:16:24.928747+0000","last_unstale":"2023-12-01T01:16:29.637968+0000","last_undegraded":"2023-12-01T01:16:29.637968+0000","last_fullsized":"2023-12-01T01:16:29.637968+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:02:24.727835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.571467+0000","last_change":"2023-12-01T01:16:25.360959+0000","last_active":"2023-12-01T01:16:26.571467+0000","last_peered":"2023-12-01T01:16:26.571467+0000","last_clean":"2023-12-01T01:16:26.571467+0000","last_became_active":"2023-12-01T01:16:25.360647+0000","last_became_peered":"2023-12-01T01:16:25.360647+0000","last_unstale":"2023-12-01T01:16:26.571467+0000","last_undegraded":"2023-12-01T01:16:26.571467+0000","last_fullsized":"2023-12-01T01:16:26.571467+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:50:24.151273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'3","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:31.603750+0000","last_change":"2023-12-01T01:16:24.938322+0000","last_active":"2023-12-01T01:16:31.603750+0000","last_peered":"2023-12-01T01:16:31.603750+0000","last_clean":"2023-12-01T01:16:31.603750+0000","last_became_active":"2023-12-01T01:16:24.937747+0000","last_became_peered":"2023-12-01T01:16:24.937747+0000","last_unstale":"2023-12-01T01:16:31.603750+0000","last_undegraded":"2023-12-01T01:16:31.603750+0000","last_fullsized":"2023-12-01T01:16:31.603750+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:37:59.999748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.584270+0000","last_change":"2023-12-01T01:16:25.362779+0000","last_active":"2023-12-01T01:16:26.584270+0000","last_peered":"2023-12-01T01:16:26.584270+0000","last_clean":"2023-12-01T01:16:26.584270+0000","last_became_active":"2023-12-01T01:16:25.362487+0000","last_became_peered":"2023-12-01T01:16:25.362487+0000","last_unstale":"2023-12-01T01:16:26.584270+0000","last_undegraded":"2023-12-01T01:16:26.584270+0000","last_fullsized":"2023-12-01T01:16:26.584270+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:04:42.409498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":88,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:17:06.651263+0000","last_change":"2023-12-01T01:16:24.933187+0000","last_active":"2023-12-01T01:17:06.651263+0000","last_peered":"2023-12-01T01:17:06.651263+0000","last_clean":"2023-12-01T01:17:06.651263+0000","last_became_active":"2023-12-01T01:16:24.932819+0000","last_became_peered":"2023-12-01T01:16:24.932819+0000","last_unstale":"2023-12-01T01:17:06.651263+0000","last_undegraded":"2023-12-01T01:17:06.651263+0000","last_fullsized":"2023-12-01T01:17:06.651263+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:54:34.739815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.324904+0000","last_change":"2023-12-01T01:16:25.361183+0000","last_active":"2023-12-01T01:16:27.324904+0000","last_peered":"2023-12-01T01:16:27.324904+0000","last_clean":"2023-12-01T01:16:27.324904+0000","last_became_active":"2023-12-01T01:16:25.360871+0000","last_became_peered":"2023-12-01T01:16:25.360871+0000","last_unstale":"2023-12-01T01:16:27.324904+0000","last_undegraded":"2023-12-01T01:16:27.324904+0000","last_fullsized":"2023-12-01T01:16:27.324904+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:06:04.748159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'4","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:31.601992+0000","last_change":"2023-12-01T01:16:24.939355+0000","last_active":"2023-12-01T01:16:31.601992+0000","last_peered":"2023-12-01T01:16:31.601992+0000","last_clean":"2023-12-01T01:16:31.601992+0000","last_became_active":"2023-12-01T01:16:24.938967+0000","last_became_peered":"2023-12-01T01:16:24.938967+0000","last_unstale":"2023-12-01T01:16:31.601992+0000","last_undegraded":"2023-12-01T01:16:31.601992+0000","last_fullsized":"2023-12-01T01:16:31.601992+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:20:51.172753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325217+0000","last_change":"2023-12-01T01:16:25.366752+0000","last_active":"2023-12-01T01:16:27.325217+0000","last_peered":"2023-12-01T01:16:27.325217+0000","last_clean":"2023-12-01T01:16:27.325217+0000","last_became_active":"2023-12-01T01:16:25.366466+0000","last_became_peered":"2023-12-01T01:16:25.366466+0000","last_unstale":"2023-12-01T01:16:27.325217+0000","last_undegraded":"2023-12-01T01:16:27.325217+0000","last_fullsized":"2023-12-01T01:16:27.325217+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:09:53.716127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:30.650645+0000","last_change":"2023-12-01T01:16:24.940827+0000","last_active":"2023-12-01T01:16:30.650645+0000","last_peered":"2023-12-01T01:16:30.650645+0000","last_clean":"2023-12-01T01:16:30.650645+0000","last_became_active":"2023-12-01T01:16:24.940515+0000","last_became_peered":"2023-12-01T01:16:24.940515+0000","last_unstale":"2023-12-01T01:16:30.650645+0000","last_undegraded":"2023-12-01T01:16:30.650645+0000","last_fullsized":"2023-12-01T01:16:30.650645+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:08:26.409014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325032+0000","last_change":"2023-12-01T01:16:25.369662+0000","last_active":"2023-12-01T01:16:27.325032+0000","last_peered":"2023-12-01T01:16:27.325032+0000","last_clean":"2023-12-01T01:16:27.325032+0000","last_became_active":"2023-12-01T01:16:25.369178+0000","last_became_peered":"2023-12-01T01:16:25.369178+0000","last_unstale":"2023-12-01T01:16:27.325032+0000","last_undegraded":"2023-12-01T01:16:27.325032+0000","last_fullsized":"2023-12-01T01:16:27.325032+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:40:39.919414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'29","reported_seq":120,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:17:06.654122+0000","last_change":"2023-12-01T01:16:24.936733+0000","last_active":"2023-12-01T01:17:06.654122+0000","last_peered":"2023-12-01T01:17:06.654122+0000","last_clean":"2023-12-01T01:17:06.654122+0000","last_became_active":"2023-12-01T01:16:24.936459+0000","last_became_peered":"2023-12-01T01:16:24.936459+0000","last_unstale":"2023-12-01T01:17:06.654122+0000","last_undegraded":"2023-12-01T01:17:06.654122+0000","last_fullsized":"2023-12-01T01:17:06.654122+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:19:58.387650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58530,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":79,"num_read_kb":96,"num_write":31,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.071505+0000","last_change":"2023-12-01T01:16:25.364292+0000","last_active":"2023-12-01T01:16:26.071505+0000","last_peered":"2023-12-01T01:16:26.071505+0000","last_clean":"2023-12-01T01:16:26.071505+0000","last_became_active":"2023-12-01T01:16:25.362615+0000","last_became_peered":"2023-12-01T01:16:25.362615+0000","last_unstale":"2023-12-01T01:16:26.071505+0000","last_undegraded":"2023-12-01T01:16:26.071505+0000","last_fullsized":"2023-12-01T01:16:26.071505+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:06:07.350607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'9","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:31.602567+0000","last_change":"2023-12-01T01:16:24.941007+0000","last_active":"2023-12-01T01:16:31.602567+0000","last_peered":"2023-12-01T01:16:31.602567+0000","last_clean":"2023-12-01T01:16:31.602567+0000","last_became_active":"2023-12-01T01:16:24.940669+0000","last_became_peered":"2023-12-01T01:16:24.940669+0000","last_unstale":"2023-12-01T01:16:31.602567+0000","last_undegraded":"2023-12-01T01:16:31.602567+0000","last_fullsized":"2023-12-01T01:16:31.602567+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:26:21.907942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.352205+0000","last_change":"2023-12-01T01:16:25.360104+0000","last_active":"2023-12-01T01:16:26.352205+0000","last_peered":"2023-12-01T01:16:26.352205+0000","last_clean":"2023-12-01T01:16:26.352205+0000","last_became_active":"2023-12-01T01:16:25.359807+0000","last_became_peered":"2023-12-01T01:16:25.359807+0000","last_unstale":"2023-12-01T01:16:26.352205+0000","last_undegraded":"2023-12-01T01:16:26.352205+0000","last_fullsized":"2023-12-01T01:16:26.352205+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:04:05.100258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'5","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:31.604782+0000","last_change":"2023-12-01T01:16:24.945074+0000","last_active":"2023-12-01T01:16:31.604782+0000","last_peered":"2023-12-01T01:16:31.604782+0000","last_clean":"2023-12-01T01:16:31.604782+0000","last_became_active":"2023-12-01T01:16:24.944737+0000","last_became_peered":"2023-12-01T01:16:24.944737+0000","last_unstale":"2023-12-01T01:16:31.604782+0000","last_undegraded":"2023-12-01T01:16:31.604782+0000","last_fullsized":"2023-12-01T01:16:31.604782+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:44:53.931454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.350184+0000","last_change":"2023-12-01T01:16:25.364995+0000","last_active":"2023-12-01T01:16:26.350184+0000","last_peered":"2023-12-01T01:16:26.350184+0000","last_clean":"2023-12-01T01:16:26.350184+0000","last_became_active":"2023-12-01T01:16:25.364684+0000","last_became_peered":"2023-12-01T01:16:25.364684+0000","last_unstale":"2023-12-01T01:16:26.350184+0000","last_undegraded":"2023-12-01T01:16:26.350184+0000","last_fullsized":"2023-12-01T01:16:26.350184+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:41:35.422238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":56,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:17:06.668544+0000","last_change":"2023-12-01T01:16:24.929103+0000","last_active":"2023-12-01T01:17:06.668544+0000","last_peered":"2023-12-01T01:17:06.668544+0000","last_clean":"2023-12-01T01:17:06.668544+0000","last_became_active":"2023-12-01T01:16:24.928769+0000","last_became_peered":"2023-12-01T01:16:24.928769+0000","last_unstale":"2023-12-01T01:17:06.668544+0000","last_undegraded":"2023-12-01T01:17:06.668544+0000","last_fullsized":"2023-12-01T01:17:06.668544+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:45:24.542610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.349997+0000","last_change":"2023-12-01T01:16:25.370816+0000","last_active":"2023-12-01T01:16:26.349997+0000","last_peered":"2023-12-01T01:16:26.349997+0000","last_clean":"2023-12-01T01:16:26.349997+0000","last_became_active":"2023-12-01T01:16:25.370465+0000","last_became_peered":"2023-12-01T01:16:25.370465+0000","last_unstale":"2023-12-01T01:16:26.349997+0000","last_undegraded":"2023-12-01T01:16:26.349997+0000","last_fullsized":"2023-12-01T01:16:26.349997+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:52:28.990416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'17","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:31.594625+0000","last_change":"2023-12-01T01:16:24.933022+0000","last_active":"2023-12-01T01:16:31.594625+0000","last_peered":"2023-12-01T01:16:31.594625+0000","last_clean":"2023-12-01T01:16:31.594625+0000","last_became_active":"2023-12-01T01:16:24.932705+0000","last_became_peered":"2023-12-01T01:16:24.932705+0000","last_unstale":"2023-12-01T01:16:31.594625+0000","last_undegraded":"2023-12-01T01:16:31.594625+0000","last_fullsized":"2023-12-01T01:16:31.594625+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:36:44.747984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325386+0000","last_change":"2023-12-01T01:16:25.370881+0000","last_active":"2023-12-01T01:16:27.325386+0000","last_peered":"2023-12-01T01:16:27.325386+0000","last_clean":"2023-12-01T01:16:27.325386+0000","last_became_active":"2023-12-01T01:16:25.368242+0000","last_became_peered":"2023-12-01T01:16:25.368242+0000","last_unstale":"2023-12-01T01:16:27.325386+0000","last_undegraded":"2023-12-01T01:16:27.325386+0000","last_fullsized":"2023-12-01T01:16:27.325386+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:35:41.128841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:31.594803+0000","last_change":"2023-12-01T01:16:24.933579+0000","last_active":"2023-12-01T01:16:31.594803+0000","last_peered":"2023-12-01T01:16:31.594803+0000","last_clean":"2023-12-01T01:16:31.594803+0000","last_became_active":"2023-12-01T01:16:24.933207+0000","last_became_peered":"2023-12-01T01:16:24.933207+0000","last_unstale":"2023-12-01T01:16:31.594803+0000","last_undegraded":"2023-12-01T01:16:31.594803+0000","last_fullsized":"2023-12-01T01:16:31.594803+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:28.309551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.592816+0000","last_change":"2023-12-01T01:16:25.368611+0000","last_active":"2023-12-01T01:16:25.592816+0000","last_peered":"2023-12-01T01:16:25.592816+0000","last_clean":"2023-12-01T01:16:25.592816+0000","last_became_active":"2023-12-01T01:16:25.368291+0000","last_became_peered":"2023-12-01T01:16:25.368291+0000","last_unstale":"2023-12-01T01:16:25.592816+0000","last_undegraded":"2023-12-01T01:16:25.592816+0000","last_fullsized":"2023-12-01T01:16:25.592816+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:17:21.824737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325140+0000","last_change":"2023-12-01T01:16:24.941410+0000","last_active":"2023-12-01T01:16:27.325140+0000","last_peered":"2023-12-01T01:16:27.325140+0000","last_clean":"2023-12-01T01:16:27.325140+0000","last_became_active":"2023-12-01T01:16:24.941127+0000","last_became_peered":"2023-12-01T01:16:24.941127+0000","last_unstale":"2023-12-01T01:16:27.325140+0000","last_undegraded":"2023-12-01T01:16:27.325140+0000","last_fullsized":"2023-12-01T01:16:27.325140+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:25:20.911892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.476088+0000","last_change":"2023-12-01T01:16:25.368235+0000","last_active":"2023-12-01T01:16:27.476088+0000","last_peered":"2023-12-01T01:16:27.476088+0000","last_clean":"2023-12-01T01:16:27.476088+0000","last_became_active":"2023-12-01T01:16:25.367918+0000","last_became_peered":"2023-12-01T01:16:25.367918+0000","last_unstale":"2023-12-01T01:16:27.476088+0000","last_undegraded":"2023-12-01T01:16:27.476088+0000","last_fullsized":"2023-12-01T01:16:27.476088+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:12:26.665404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'5","reported_seq":54,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:17:06.667790+0000","last_change":"2023-12-01T01:16:24.932057+0000","last_active":"2023-12-01T01:17:06.667790+0000","last_peered":"2023-12-01T01:17:06.667790+0000","last_clean":"2023-12-01T01:17:06.667790+0000","last_became_active":"2023-12-01T01:16:24.931817+0000","last_became_peered":"2023-12-01T01:16:24.931817+0000","last_unstale":"2023-12-01T01:17:06.667790+0000","last_undegraded":"2023-12-01T01:17:06.667790+0000","last_fullsized":"2023-12-01T01:17:06.667790+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:19:02.039334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325595+0000","last_change":"2023-12-01T01:16:25.367431+0000","last_active":"2023-12-01T01:16:27.325595+0000","last_peered":"2023-12-01T01:16:27.325595+0000","last_clean":"2023-12-01T01:16:27.325595+0000","last_became_active":"2023-12-01T01:16:25.367184+0000","last_became_peered":"2023-12-01T01:16:25.367184+0000","last_unstale":"2023-12-01T01:16:27.325595+0000","last_undegraded":"2023-12-01T01:16:27.325595+0000","last_fullsized":"2023-12-01T01:16:27.325595+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:21:19.132725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325383+0000","last_change":"2023-12-01T01:16:25.370593+0000","last_active":"2023-12-01T01:16:27.325383+0000","last_peered":"2023-12-01T01:16:27.325383+0000","last_clean":"2023-12-01T01:16:27.325383+0000","last_became_active":"2023-12-01T01:16:25.367614+0000","last_became_peered":"2023-12-01T01:16:25.367614+0000","last_unstale":"2023-12-01T01:16:27.325383+0000","last_undegraded":"2023-12-01T01:16:27.325383+0000","last_fullsized":"2023-12-01T01:16:27.325383+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:56:34.388180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":54,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:17:05.666244+0000","last_change":"2023-12-01T01:16:24.931188+0000","last_active":"2023-12-01T01:17:05.666244+0000","last_peered":"2023-12-01T01:17:05.666244+0000","last_clean":"2023-12-01T01:17:05.666244+0000","last_became_active":"2023-12-01T01:16:24.930875+0000","last_became_peered":"2023-12-01T01:16:24.930875+0000","last_unstale":"2023-12-01T01:17:05.666244+0000","last_undegraded":"2023-12-01T01:17:05.666244+0000","last_fullsized":"2023-12-01T01:17:05.666244+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:17.729286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.571499+0000","last_change":"2023-12-01T01:16:25.361359+0000","last_active":"2023-12-01T01:16:26.571499+0000","last_peered":"2023-12-01T01:16:26.571499+0000","last_clean":"2023-12-01T01:16:26.571499+0000","last_became_active":"2023-12-01T01:16:25.361083+0000","last_became_peered":"2023-12-01T01:16:25.361083+0000","last_unstale":"2023-12-01T01:16:26.571499+0000","last_undegraded":"2023-12-01T01:16:26.571499+0000","last_fullsized":"2023-12-01T01:16:26.571499+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:42:09.417021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:31.603727+0000","last_change":"2023-12-01T01:16:24.927614+0000","last_active":"2023-12-01T01:16:31.603727+0000","last_peered":"2023-12-01T01:16:31.603727+0000","last_clean":"2023-12-01T01:16:31.603727+0000","last_became_active":"2023-12-01T01:16:24.926459+0000","last_became_peered":"2023-12-01T01:16:24.926459+0000","last_unstale":"2023-12-01T01:16:31.603727+0000","last_undegraded":"2023-12-01T01:16:31.603727+0000","last_fullsized":"2023-12-01T01:16:31.603727+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:41:25.435261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":323,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.358040+0000","last_change":"2023-12-01T01:15:25.599551+0000","last_active":"2023-12-01T01:16:25.358040+0000","last_peered":"2023-12-01T01:16:25.358040+0000","last_clean":"2023-12-01T01:16:25.358040+0000","last_became_active":"2023-12-01T01:15:25.598951+0000","last_became_peered":"2023-12-01T01:15:25.598951+0000","last_unstale":"2023-12-01T01:16:25.358040+0000","last_undegraded":"2023-12-01T01:16:25.358040+0000","last_fullsized":"2023-12-01T01:16:25.358040+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:12:31.173756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:12:31.173756+0000","last_clean_scrub_stamp":"2023-12-01T01:12:31.173756+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:00:11.343823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.352149+0000","last_change":"2023-12-01T01:16:25.358617+0000","last_active":"2023-12-01T01:16:26.352149+0000","last_peered":"2023-12-01T01:16:26.352149+0000","last_clean":"2023-12-01T01:16:26.352149+0000","last_became_active":"2023-12-01T01:16:25.358325+0000","last_became_peered":"2023-12-01T01:16:25.358325+0000","last_unstale":"2023-12-01T01:16:26.352149+0000","last_undegraded":"2023-12-01T01:16:26.352149+0000","last_fullsized":"2023-12-01T01:16:26.352149+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:40:01.111777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'4","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:29.637779+0000","last_change":"2023-12-01T01:16:24.935103+0000","last_active":"2023-12-01T01:16:29.637779+0000","last_peered":"2023-12-01T01:16:29.637779+0000","last_clean":"2023-12-01T01:16:29.637779+0000","last_became_active":"2023-12-01T01:16:24.934900+0000","last_became_peered":"2023-12-01T01:16:24.934900+0000","last_unstale":"2023-12-01T01:16:29.637779+0000","last_undegraded":"2023-12-01T01:16:29.637779+0000","last_fullsized":"2023-12-01T01:16:29.637779+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:59:37.138090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"82'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:31.594317+0000","last_change":"2023-12-01T01:16:24.938260+0000","last_active":"2023-12-01T01:16:31.594317+0000","last_peered":"2023-12-01T01:16:31.594317+0000","last_clean":"2023-12-01T01:16:31.594317+0000","last_became_active":"2023-12-01T01:16:24.937939+0000","last_became_peered":"2023-12-01T01:16:24.937939+0000","last_unstale":"2023-12-01T01:16:31.594317+0000","last_undegraded":"2023-12-01T01:16:31.594317+0000","last_fullsized":"2023-12-01T01:16:31.594317+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:07:05.090877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.352470+0000","last_change":"2023-12-01T01:16:25.362137+0000","last_active":"2023-12-01T01:16:26.352470+0000","last_peered":"2023-12-01T01:16:26.352470+0000","last_clean":"2023-12-01T01:16:26.352470+0000","last_became_active":"2023-12-01T01:16:25.361563+0000","last_became_peered":"2023-12-01T01:16:25.361563+0000","last_unstale":"2023-12-01T01:16:26.352470+0000","last_undegraded":"2023-12-01T01:16:26.352470+0000","last_fullsized":"2023-12-01T01:16:26.352470+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:37:56.473920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'9","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:34.873654+0000","last_change":"2023-12-01T01:16:24.935692+0000","last_active":"2023-12-01T01:16:34.873654+0000","last_peered":"2023-12-01T01:16:34.873654+0000","last_clean":"2023-12-01T01:16:34.873654+0000","last_became_active":"2023-12-01T01:16:24.934799+0000","last_became_peered":"2023-12-01T01:16:24.934799+0000","last_unstale":"2023-12-01T01:16:34.873654+0000","last_undegraded":"2023-12-01T01:16:34.873654+0000","last_fullsized":"2023-12-01T01:16:34.873654+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:22:51.586089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.571589+0000","last_change":"2023-12-01T01:16:25.368672+0000","last_active":"2023-12-01T01:16:26.571589+0000","last_peered":"2023-12-01T01:16:26.571589+0000","last_clean":"2023-12-01T01:16:26.571589+0000","last_became_active":"2023-12-01T01:16:25.368309+0000","last_became_peered":"2023-12-01T01:16:25.368309+0000","last_unstale":"2023-12-01T01:16:26.571589+0000","last_undegraded":"2023-12-01T01:16:26.571589+0000","last_fullsized":"2023-12-01T01:16:26.571589+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:30:59.029250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:31.603713+0000","last_change":"2023-12-01T01:16:24.931322+0000","last_active":"2023-12-01T01:16:31.603713+0000","last_peered":"2023-12-01T01:16:31.603713+0000","last_clean":"2023-12-01T01:16:31.603713+0000","last_became_active":"2023-12-01T01:16:24.930961+0000","last_became_peered":"2023-12-01T01:16:24.930961+0000","last_unstale":"2023-12-01T01:16:31.603713+0000","last_undegraded":"2023-12-01T01:16:31.603713+0000","last_fullsized":"2023-12-01T01:16:31.603713+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:47:16.317035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325104+0000","last_change":"2023-12-01T01:16:25.364065+0000","last_active":"2023-12-01T01:16:27.325104+0000","last_peered":"2023-12-01T01:16:27.325104+0000","last_clean":"2023-12-01T01:16:27.325104+0000","last_became_active":"2023-12-01T01:16:25.363694+0000","last_became_peered":"2023-12-01T01:16:25.363694+0000","last_unstale":"2023-12-01T01:16:27.325104+0000","last_undegraded":"2023-12-01T01:16:27.325104+0000","last_fullsized":"2023-12-01T01:16:27.325104+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:49:39.091587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:28.633891+0000","last_change":"2023-12-01T01:16:24.932730+0000","last_active":"2023-12-01T01:16:28.633891+0000","last_peered":"2023-12-01T01:16:28.633891+0000","last_clean":"2023-12-01T01:16:28.633891+0000","last_became_active":"2023-12-01T01:16:24.932404+0000","last_became_peered":"2023-12-01T01:16:24.932404+0000","last_unstale":"2023-12-01T01:16:28.633891+0000","last_undegraded":"2023-12-01T01:16:28.633891+0000","last_fullsized":"2023-12-01T01:16:28.633891+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:41:33.104418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.071437+0000","last_change":"2023-12-01T01:16:25.375795+0000","last_active":"2023-12-01T01:16:26.071437+0000","last_peered":"2023-12-01T01:16:26.071437+0000","last_clean":"2023-12-01T01:16:26.071437+0000","last_became_active":"2023-12-01T01:16:25.375485+0000","last_became_peered":"2023-12-01T01:16:25.375485+0000","last_unstale":"2023-12-01T01:16:26.071437+0000","last_undegraded":"2023-12-01T01:16:26.071437+0000","last_fullsized":"2023-12-01T01:16:26.071437+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:10:04.009511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'7","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:31.602193+0000","last_change":"2023-12-01T01:16:24.936770+0000","last_active":"2023-12-01T01:16:31.602193+0000","last_peered":"2023-12-01T01:16:31.602193+0000","last_clean":"2023-12-01T01:16:31.602193+0000","last_became_active":"2023-12-01T01:16:24.936058+0000","last_became_peered":"2023-12-01T01:16:24.936058+0000","last_unstale":"2023-12-01T01:16:31.602193+0000","last_undegraded":"2023-12-01T01:16:31.602193+0000","last_fullsized":"2023-12-01T01:16:31.602193+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:22:21.414131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.592744+0000","last_change":"2023-12-01T01:16:25.372183+0000","last_active":"2023-12-01T01:16:25.592744+0000","last_peered":"2023-12-01T01:16:25.592744+0000","last_clean":"2023-12-01T01:16:25.592744+0000","last_became_active":"2023-12-01T01:16:25.371865+0000","last_became_peered":"2023-12-01T01:16:25.371865+0000","last_unstale":"2023-12-01T01:16:25.592744+0000","last_undegraded":"2023-12-01T01:16:25.592744+0000","last_fullsized":"2023-12-01T01:16:25.592744+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:19:54.675425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'3","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:33.355776+0000","last_change":"2023-12-01T01:16:24.939397+0000","last_active":"2023-12-01T01:16:33.355776+0000","last_peered":"2023-12-01T01:16:33.355776+0000","last_clean":"2023-12-01T01:16:33.355776+0000","last_became_active":"2023-12-01T01:16:24.939081+0000","last_became_peered":"2023-12-01T01:16:24.939081+0000","last_unstale":"2023-12-01T01:16:33.355776+0000","last_undegraded":"2023-12-01T01:16:33.355776+0000","last_fullsized":"2023-12-01T01:16:33.355776+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:21:34.967255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.571515+0000","last_change":"2023-12-01T01:16:25.368484+0000","last_active":"2023-12-01T01:16:26.571515+0000","last_peered":"2023-12-01T01:16:26.571515+0000","last_clean":"2023-12-01T01:16:26.571515+0000","last_became_active":"2023-12-01T01:16:25.367638+0000","last_became_peered":"2023-12-01T01:16:25.367638+0000","last_unstale":"2023-12-01T01:16:26.571515+0000","last_undegraded":"2023-12-01T01:16:26.571515+0000","last_fullsized":"2023-12-01T01:16:26.571515+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:06:05.354443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:28.633709+0000","last_change":"2023-12-01T01:16:24.933304+0000","last_active":"2023-12-01T01:16:28.633709+0000","last_peered":"2023-12-01T01:16:28.633709+0000","last_clean":"2023-12-01T01:16:28.633709+0000","last_became_active":"2023-12-01T01:16:24.933028+0000","last_became_peered":"2023-12-01T01:16:24.933028+0000","last_unstale":"2023-12-01T01:16:28.633709+0000","last_undegraded":"2023-12-01T01:16:28.633709+0000","last_fullsized":"2023-12-01T01:16:28.633709+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:26:51.810995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.350012+0000","last_change":"2023-12-01T01:16:25.364888+0000","last_active":"2023-12-01T01:16:26.350012+0000","last_peered":"2023-12-01T01:16:26.350012+0000","last_clean":"2023-12-01T01:16:26.350012+0000","last_became_active":"2023-12-01T01:16:25.364595+0000","last_became_peered":"2023-12-01T01:16:25.364595+0000","last_unstale":"2023-12-01T01:16:26.350012+0000","last_undegraded":"2023-12-01T01:16:26.350012+0000","last_fullsized":"2023-12-01T01:16:26.350012+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:54:23.178824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'5","reported_seq":58,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:17:08.654905+0000","last_change":"2023-12-01T01:16:24.945771+0000","last_active":"2023-12-01T01:17:08.654905+0000","last_peered":"2023-12-01T01:17:08.654905+0000","last_clean":"2023-12-01T01:17:08.654905+0000","last_became_active":"2023-12-01T01:16:24.945441+0000","last_became_peered":"2023-12-01T01:16:24.945441+0000","last_unstale":"2023-12-01T01:17:08.654905+0000","last_undegraded":"2023-12-01T01:17:08.654905+0000","last_fullsized":"2023-12-01T01:17:08.654905+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:49:41.234472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":41,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.350239+0000","last_change":"2023-12-01T01:16:25.365246+0000","last_active":"2023-12-01T01:16:26.350239+0000","last_peered":"2023-12-01T01:16:26.350239+0000","last_clean":"2023-12-01T01:16:26.350239+0000","last_became_active":"2023-12-01T01:16:25.364364+0000","last_became_peered":"2023-12-01T01:16:25.364364+0000","last_unstale":"2023-12-01T01:16:26.350239+0000","last_undegraded":"2023-12-01T01:16:26.350239+0000","last_fullsized":"2023-12-01T01:16:26.350239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:56:44.815172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'3","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:31.594729+0000","last_change":"2023-12-01T01:16:24.944229+0000","last_active":"2023-12-01T01:16:31.594729+0000","last_peered":"2023-12-01T01:16:31.594729+0000","last_clean":"2023-12-01T01:16:31.594729+0000","last_became_active":"2023-12-01T01:16:24.943880+0000","last_became_peered":"2023-12-01T01:16:24.943880+0000","last_unstale":"2023-12-01T01:16:31.594729+0000","last_undegraded":"2023-12-01T01:16:31.594729+0000","last_fullsized":"2023-12-01T01:16:31.594729+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:44:26.769358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:28.318887+0000","last_change":"2023-12-01T01:16:25.361028+0000","last_active":"2023-12-01T01:16:28.318887+0000","last_peered":"2023-12-01T01:16:28.318887+0000","last_clean":"2023-12-01T01:16:28.318887+0000","last_became_active":"2023-12-01T01:16:25.360702+0000","last_became_peered":"2023-12-01T01:16:25.360702+0000","last_unstale":"2023-12-01T01:16:28.318887+0000","last_undegraded":"2023-12-01T01:16:28.318887+0000","last_fullsized":"2023-12-01T01:16:28.318887+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:45:09.472497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'5","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:31.601241+0000","last_change":"2023-12-01T01:16:24.945567+0000","last_active":"2023-12-01T01:16:31.601241+0000","last_peered":"2023-12-01T01:16:31.601241+0000","last_clean":"2023-12-01T01:16:31.601241+0000","last_became_active":"2023-12-01T01:16:24.945256+0000","last_became_peered":"2023-12-01T01:16:24.945256+0000","last_unstale":"2023-12-01T01:16:31.601241+0000","last_undegraded":"2023-12-01T01:16:31.601241+0000","last_fullsized":"2023-12-01T01:16:31.601241+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:19:30.944573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.350244+0000","last_change":"2023-12-01T01:16:25.367247+0000","last_active":"2023-12-01T01:16:26.350244+0000","last_peered":"2023-12-01T01:16:26.350244+0000","last_clean":"2023-12-01T01:16:26.350244+0000","last_became_active":"2023-12-01T01:16:25.366872+0000","last_became_peered":"2023-12-01T01:16:25.366872+0000","last_unstale":"2023-12-01T01:16:26.350244+0000","last_undegraded":"2023-12-01T01:16:26.350244+0000","last_fullsized":"2023-12-01T01:16:26.350244+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:03:07.697233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.592768+0000","last_change":"2023-12-01T01:16:25.376005+0000","last_active":"2023-12-01T01:16:25.592768+0000","last_peered":"2023-12-01T01:16:25.592768+0000","last_clean":"2023-12-01T01:16:25.592768+0000","last_became_active":"2023-12-01T01:16:25.375702+0000","last_became_peered":"2023-12-01T01:16:25.375702+0000","last_unstale":"2023-12-01T01:16:25.592768+0000","last_undegraded":"2023-12-01T01:16:25.592768+0000","last_fullsized":"2023-12-01T01:16:25.592768+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:14:00.955730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.378145+0000","last_change":"2023-12-01T01:16:25.371023+0000","last_active":"2023-12-01T01:16:25.378145+0000","last_peered":"2023-12-01T01:16:25.378145+0000","last_clean":"2023-12-01T01:16:25.378145+0000","last_became_active":"2023-12-01T01:16:25.370694+0000","last_became_peered":"2023-12-01T01:16:25.370694+0000","last_unstale":"2023-12-01T01:16:25.378145+0000","last_undegraded":"2023-12-01T01:16:25.378145+0000","last_fullsized":"2023-12-01T01:16:25.378145+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:03:00.715413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.375644+0000","last_change":"2023-12-01T01:16:25.361795+0000","last_active":"2023-12-01T01:16:25.375644+0000","last_peered":"2023-12-01T01:16:25.375644+0000","last_clean":"2023-12-01T01:16:25.375644+0000","last_became_active":"2023-12-01T01:16:25.361459+0000","last_became_peered":"2023-12-01T01:16:25.361459+0000","last_unstale":"2023-12-01T01:16:25.375644+0000","last_undegraded":"2023-12-01T01:16:25.375644+0000","last_fullsized":"2023-12-01T01:16:25.375644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:18:57.531240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.324741+0000","last_change":"2023-12-01T01:16:25.363773+0000","last_active":"2023-12-01T01:16:27.324741+0000","last_peered":"2023-12-01T01:16:27.324741+0000","last_clean":"2023-12-01T01:16:27.324741+0000","last_became_active":"2023-12-01T01:16:25.363422+0000","last_became_peered":"2023-12-01T01:16:25.363422+0000","last_unstale":"2023-12-01T01:16:27.324741+0000","last_undegraded":"2023-12-01T01:16:27.324741+0000","last_fullsized":"2023-12-01T01:16:27.324741+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:48:33.395566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.352367+0000","last_change":"2023-12-01T01:16:25.359431+0000","last_active":"2023-12-01T01:16:26.352367+0000","last_peered":"2023-12-01T01:16:26.352367+0000","last_clean":"2023-12-01T01:16:26.352367+0000","last_became_active":"2023-12-01T01:16:25.359116+0000","last_became_peered":"2023-12-01T01:16:25.359116+0000","last_unstale":"2023-12-01T01:16:26.352367+0000","last_undegraded":"2023-12-01T01:16:26.352367+0000","last_fullsized":"2023-12-01T01:16:26.352367+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:32:26.545269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.592804+0000","last_change":"2023-12-01T01:16:25.368008+0000","last_active":"2023-12-01T01:16:25.592804+0000","last_peered":"2023-12-01T01:16:25.592804+0000","last_clean":"2023-12-01T01:16:25.592804+0000","last_became_active":"2023-12-01T01:16:25.367726+0000","last_became_peered":"2023-12-01T01:16:25.367726+0000","last_unstale":"2023-12-01T01:16:25.592804+0000","last_undegraded":"2023-12-01T01:16:25.592804+0000","last_fullsized":"2023-12-01T01:16:25.592804+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:38:33.837846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.378494+0000","last_change":"2023-12-01T01:16:25.372781+0000","last_active":"2023-12-01T01:16:25.378494+0000","last_peered":"2023-12-01T01:16:25.378494+0000","last_clean":"2023-12-01T01:16:25.378494+0000","last_became_active":"2023-12-01T01:16:25.371443+0000","last_became_peered":"2023-12-01T01:16:25.371443+0000","last_unstale":"2023-12-01T01:16:25.378494+0000","last_undegraded":"2023-12-01T01:16:25.378494+0000","last_fullsized":"2023-12-01T01:16:25.378494+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:49:06.422055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.571571+0000","last_change":"2023-12-01T01:16:25.363048+0000","last_active":"2023-12-01T01:16:26.571571+0000","last_peered":"2023-12-01T01:16:26.571571+0000","last_clean":"2023-12-01T01:16:26.571571+0000","last_became_active":"2023-12-01T01:16:25.362752+0000","last_became_peered":"2023-12-01T01:16:25.362752+0000","last_unstale":"2023-12-01T01:16:26.571571+0000","last_undegraded":"2023-12-01T01:16:26.571571+0000","last_fullsized":"2023-12-01T01:16:26.571571+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:54:16.492562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.325256+0000","last_change":"2023-12-01T01:16:25.359998+0000","last_active":"2023-12-01T01:16:27.325256+0000","last_peered":"2023-12-01T01:16:27.325256+0000","last_clean":"2023-12-01T01:16:27.325256+0000","last_became_active":"2023-12-01T01:16:25.359744+0000","last_became_peered":"2023-12-01T01:16:25.359744+0000","last_unstale":"2023-12-01T01:16:27.325256+0000","last_undegraded":"2023-12-01T01:16:27.325256+0000","last_fullsized":"2023-12-01T01:16:27.325256+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:12:33.947134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.571477+0000","last_change":"2023-12-01T01:16:25.373615+0000","last_active":"2023-12-01T01:16:26.571477+0000","last_peered":"2023-12-01T01:16:26.571477+0000","last_clean":"2023-12-01T01:16:26.571477+0000","last_became_active":"2023-12-01T01:16:25.373247+0000","last_became_peered":"2023-12-01T01:16:25.373247+0000","last_unstale":"2023-12-01T01:16:26.571477+0000","last_undegraded":"2023-12-01T01:16:26.571477+0000","last_fullsized":"2023-12-01T01:16:26.571477+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:28:10.815906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:27.324878+0000","last_change":"2023-12-01T01:16:25.367404+0000","last_active":"2023-12-01T01:16:27.324878+0000","last_peered":"2023-12-01T01:16:27.324878+0000","last_clean":"2023-12-01T01:16:27.324878+0000","last_became_active":"2023-12-01T01:16:25.366404+0000","last_became_peered":"2023-12-01T01:16:25.366404+0000","last_unstale":"2023-12-01T01:16:27.324878+0000","last_undegraded":"2023-12-01T01:16:27.324878+0000","last_fullsized":"2023-12-01T01:16:27.324878+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:54:49.668036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-12-01T01:16:26.352405+0000","last_change":"2023-12-01T01:16:25.360315+0000","last_active":"2023-12-01T01:16:26.352405+0000","last_peered":"2023-12-01T01:16:26.352405+0000","last_clean":"2023-12-01T01:16:26.352405+0000","last_became_active":"2023-12-01T01:16:25.360031+0000","last_became_peered":"2023-12-01T01:16:25.360031+0000","last_unstale":"2023-12-01T01:16:26.352405+0000","last_undegraded":"2023-12-01T01:16:26.352405+0000","last_fullsized":"2023-12-01T01:16:26.352405+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:13:24.437697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-12-01T01:16:25.592792+0000","last_change":"2023-12-01T01:16:25.367820+0000","last_active":"2023-12-01T01:16:25.592792+0000","last_peered":"2023-12-01T01:16:25.592792+0000","last_clean":"2023-12-01T01:16:25.592792+0000","last_became_active":"2023-12-01T01:16:25.367525+0000","last_became_peered":"2023-12-01T01:16:25.367525+0000","last_unstale":"2023-12-01T01:16:25.592792+0000","last_undegraded":"2023-12-01T01:16:25.592792+0000","last_fullsized":"2023-12-01T01:16:25.592792+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:28:01.552342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65153,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":401,"num_read_kb":268,"num_write":229,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":195459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":201,"ondisk_log_size":201,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481812,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":892928,"data_stored":423958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.442},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.61199999999999999}]},{"osd":1,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.68100000000000005}]},{"osd":2,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.49199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.72999999999999998}]},{"osd":5,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.57899999999999996}]},{"osd":6,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.70099999999999996}]},{"osd":7,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.78800000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":9,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.71799999999999997}]},{"osd":10,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.649,"5min":1.649,"15min":1.649},"last":0.83899999999999997}]}]},{"osd":10,"up_from":69,"seq":296352743448,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28504,"kb_used_data":1552,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716648,"statfs":{"total":95995035648,"available":95965847552,"internally_reserved":0,"allocated":1589248,"data_stored":1073192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":1,"last update":"Fri Dec 1 01:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.44800000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.60099999999999998}]},{"osd":3,"last update":"Fri Dec 1 01:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":5.7240000000000002,"5min":5.7240000000000002,"15min":5.7240000000000002},"last":0.61199999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.64000000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.63}]},{"osd":6,"last update":"Fri Dec 1 01:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":5.7510000000000003,"5min":5.7510000000000003,"15min":5.7510000000000003},"last":0.68100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":8,"last update":"Fri Dec 1 01:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.81699999999999995}]},{"osd":9,"last update":"Fri Dec 1 01:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.79400000000000004}]},{"osd":11,"last update":"Fri Dec 1 01:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.746},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.77900000000000003}]}]},{"osd":9,"up_from":64,"seq":274877906973,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":917504,"data_stored":423936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.46700000000000003,"15min":0.44900000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":3.6589999999999998,"5min":3.6589999999999998,"15min":3.6589999999999998},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.45600000000000002,"15min":0.46100000000000002},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.24199999999999999}]},{"osd":1,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.58199999999999996,"15min":0.59199999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.65100000000000002,"15min":0.63200000000000001},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":3.4969999999999999,"5min":3.4969999999999999,"15min":3.4969999999999999},"last":0.46999999999999997}]},{"osd":2,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57999999999999996,"15min":0.58499999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.72299999999999998,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.64200000000000002,"15min":0.625},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":3.5470000000000002,"5min":3.5470000000000002,"15min":3.5470000000000002},"last":0.52900000000000003}]},{"osd":3,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.54500000000000004,"15min":0.55700000000000005},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.53300000000000003,"15min":0.54500000000000004},"min":{"1min":0.33800000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.317}]},{"osd":4,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.71299999999999997,"15min":0.69899999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":3.6019999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":0.5},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.69499999999999995,"15min":0.71299999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.872,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.40699999999999997}]},{"osd":5,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.67400000000000004,"15min":0.68999999999999995},"min":{"1min":0.46500000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.79100000000000004,"5min":1.099,"15min":1.099},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.66600000000000004,"15min":0.68200000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.70999999999999996,"5min":1.052,"15min":1.052},"last":0.42999999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.626,"15min":0.64400000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.69799999999999995,"15min":0.69199999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":3.8410000000000002,"5min":3.8410000000000002,"15min":3.8410000000000002},"last":0.29699999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.76600000000000001,"15min":0.79000000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.83199999999999996,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.749,"15min":0.77200000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81299999999999994,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.50700000000000001}]},{"osd":8,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.77400000000000002,"15min":0.76600000000000001},"min":{"1min":0.50900000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":3.5760000000000001,"5min":3.5760000000000001,"15min":3.5760000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.751,"15min":0.77100000000000002},"min":{"1min":0.5,"5min":0.495,"15min":0.495},"max":{"1min":0.76900000000000002,"5min":1.157,"15min":1.157},"last":0.49199999999999999}]},{"osd":10,"last update":"Fri Dec 1 01:16:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.443},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.54800000000000004}]}]},{"osd":8,"up_from":57,"seq":244813135904,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":892,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":913408,"data_stored":424314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.57899999999999996,"15min":0.55700000000000005},"min":{"1min":0.39500000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":3.4089999999999998,"5min":3.4089999999999998,"15min":3.4089999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54000000000000004,"15min":0.53700000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.58099999999999996}]},{"osd":1,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.626,"15min":0.58399999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":6.0960000000000001,"5min":6.0960000000000001,"15min":6.0960000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.747,"5min":0.61099999999999999,"15min":0.58899999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":3.556,"5min":3.556,"15min":3.556},"last":0.69899999999999995}]},{"osd":2,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.50900000000000001,"15min":0.47099999999999997},"min":{"1min":0.28399999999999997,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":5.9199999999999999,"5min":5.9199999999999999,"15min":5.9199999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.48499999999999999,"15min":0.45400000000000001},"min":{"1min":0.311,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":3.6099999999999999,"5min":3.6099999999999999,"15min":3.6099999999999999},"last":0.46100000000000002}]},{"osd":3,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.496},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.627,"15min":0.58199999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":6.0209999999999999,"5min":6.0209999999999999,"15min":6.0209999999999999},"last":0.65400000000000003}]},{"osd":4,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.60999999999999999,"15min":0.60399999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.66400000000000003,"15min":0.61899999999999999},"min":{"1min":0.39100000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":6.0709999999999997,"5min":6.0709999999999997,"15min":6.0709999999999997},"last":0.86499999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.58399999999999996,"15min":0.55700000000000005},"min":{"1min":0.35799999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":3.4580000000000002,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.55500000000000005,"15min":0.54000000000000004},"min":{"1min":0.38200000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.61599999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.73399999999999999,"15min":0.69499999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":5.9969999999999999,"5min":5.9969999999999999,"15min":5.9969999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.71499999999999997,"15min":0.68899999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":3.4980000000000002,"5min":3.4980000000000002,"15min":3.4980000000000002},"last":0.77500000000000002}]},{"osd":7,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.72299999999999998,"15min":0.69999999999999996},"min":{"1min":0.498,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":3.5800000000000001,"5min":3.5800000000000001,"15min":3.5800000000000001},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.69399999999999995,"15min":0.68600000000000005},"min":{"1min":0.51800000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0229999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.67700000000000005}]},{"osd":9,"last update":"Fri Dec 1 01:17:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":1.0409999999999999,"15min":1.085},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.647,"5min":5.9690000000000003,"15min":5.9690000000000003},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.91300000000000003,"15min":0.93500000000000005},"min":{"1min":0.51700000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.532,"5min":3.5259999999999998,"15min":3.5259999999999998},"last":0.85199999999999998}]},{"osd":10,"last update":"Fri Dec 1 01:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":6.0469999999999997,"5min":6.0469999999999997,"15min":6.0469999999999997},"last":0.89100000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]}]},{"osd":7,"up_from":50,"seq":214748364837,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28428,"kb_used_data":1476,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716724,"statfs":{"total":95995035648,"available":95965925376,"internally_reserved":0,"allocated":1511424,"data_stored":1016696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.45000000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.40100000000000002,"15min":0.39300000000000002},"min":{"1min":0.27700000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.41099999999999998,"15min":0.40200000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.54400000000000004}]},{"osd":2,"last update":"Fri Dec 1 01:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56200000000000006,"15min":0.55800000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.628}]},{"osd":3,"last update":"Fri Dec 1 01:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.55800000000000005,"15min":0.55000000000000004},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.77700000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61899999999999999,"15min":0.61599999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.85099999999999998,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.59599999999999997}]},{"osd":4,"last update":"Fri Dec 1 01:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.51600000000000001,"15min":0.51200000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.622},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.51600000000000001,"15min":0.51100000000000001},"min":{"1min":0.32200000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.66900000000000004}]},{"osd":5,"last update":"Fri Dec 1 01:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.435,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.61499999999999999,"15min":0.60699999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.75}]},{"osd":6,"last update":"Fri Dec 1 01:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.64800000000000002,"15min":0.64300000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.66400000000000003,"15min":0.65800000000000003},"min":{"1min":0.42099999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0600000000000001,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.79600000000000004}]},{"osd":8,"last update":"Fri Dec 1 01:16:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.71699999999999997,"15min":0.72099999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.89400000000000002,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.74199999999999999,"15min":0.74099999999999999},"min":{"1min":0.49199999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.88500000000000001}]},{"osd":9,"last update":"Fri Dec 1 01:17:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.72099999999999997,"15min":0.72399999999999998},"min":{"1min":0.51200000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.93600000000000005,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.71899999999999997,"15min":0.72099999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.874,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.80700000000000005}]},{"osd":10,"last update":"Fri Dec 1 01:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.78200000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]}]},{"osd":6,"up_from":45,"seq":193273528362,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":901120,"data_stored":423611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.27600000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.502,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.42299999999999999,"15min":0.41299999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":0.30399999999999999}]},{"osd":1,"last update":"Fri Dec 1 01:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61899999999999999,"15min":0.625},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.57499999999999996,"15min":0.59399999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.68100000000000005,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.41699999999999998}]},{"osd":2,"last update":"Fri Dec 1 01:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.59599999999999997,"15min":0.57699999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":2.9870000000000001,"5min":2.9870000000000001,"15min":2.9870000000000001},"last":0.443},{"interface":"front","average":{"1min":0.623,"5min":0.629,"15min":0.623},"min":{"1min":0.42199999999999999,"5min":0.38,"15min":0.38},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.55000000000000004}]},{"osd":3,"last update":"Fri Dec 1 01:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48399999999999999,"15min":0.46999999999999997},"min":{"1min":0.318,"5min":0.308,"15min":0.308},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.377},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.45100000000000001,"15min":0.45900000000000002},"min":{"1min":0.29999999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.501,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.35099999999999998}]},{"osd":4,"last update":"Fri Dec 1 01:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.61899999999999999,"15min":0.61099999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":2.919,"5min":2.919,"15min":2.919},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.65600000000000003,"15min":0.65200000000000002},"min":{"1min":0.45400000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.50800000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.61699999999999999,"15min":0.63300000000000001},"min":{"1min":0.40300000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.67000000000000004,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.67100000000000004,"15min":0.65700000000000003},"min":{"1min":0.437,"5min":0.373,"15min":0.373},"max":{"1min":3.0179999999999998,"5min":3.0179999999999998,"15min":3.0179999999999998},"last":0.52400000000000002}]},{"osd":7,"last update":"Fri Dec 1 01:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.69999999999999996,"15min":0.70399999999999996},"min":{"1min":0.45600000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92000000000000004,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.93300000000000005,"5min":0.995,"15min":0.995},"last":0.58099999999999996}]},{"osd":8,"last update":"Fri Dec 1 01:16:40 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.77100000000000002,"15min":0.77400000000000002},"min":{"1min":0.54500000000000004,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":0.622},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.68400000000000005,"15min":0.69199999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.89000000000000001,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.49299999999999999}]},{"osd":9,"last update":"Fri Dec 1 01:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.69699999999999995,"15min":0.70399999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":2.1320000000000001,"5min":2.1320000000000001,"15min":2.1320000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.71999999999999997,"15min":0.73099999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":0.60599999999999998}]},{"osd":10,"last update":"Fri Dec 1 01:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.63800000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":962560,"data_stored":483048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:16:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.39500000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.79300000000000004,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53200000000000003,"15min":0.51900000000000002},"min":{"1min":0.32800000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.69299999999999995,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.48599999999999999}]},{"osd":2,"last update":"Fri Dec 1 01:16:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.53900000000000003,"15min":0.495},"min":{"1min":0.38400000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.878,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.75,"5min":0.57199999999999995,"15min":0.53200000000000003},"min":{"1min":0.40300000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.9399999999999999,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":0.76300000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58399999999999996,"15min":0.55900000000000005},"min":{"1min":0.44700000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72099999999999997,"5min":1.198,"15min":1.198},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59999999999999998,"15min":0.57399999999999995},"min":{"1min":0.46700000000000003,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.77400000000000002,"5min":1.282,"15min":1.282},"last":0.57199999999999995}]},{"osd":4,"last update":"Fri Dec 1 01:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.42899999999999999,"15min":0.41499999999999998},"min":{"1min":0.30299999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.70799999999999996,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.497,"15min":0.48099999999999998},"min":{"1min":0.36599999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.91100000000000003,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.74299999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:16:43 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.56599999999999995,"15min":0.52100000000000002},"min":{"1min":0.437,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.92600000000000005,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.56799999999999995,"15min":0.52400000000000002},"min":{"1min":0.46000000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.69299999999999995}]},{"osd":6,"last update":"Fri Dec 1 01:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60599999999999998,"15min":0.59599999999999997},"min":{"1min":0.47699999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79900000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.628,"15min":0.61599999999999999},"min":{"1min":0.46400000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.89600000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.78700000000000003}]},{"osd":7,"last update":"Fri Dec 1 01:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.55400000000000005,"15min":0.53800000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.88100000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.56399999999999995,"15min":0.54600000000000004},"min":{"1min":0.40300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.878,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.82999999999999996}]},{"osd":8,"last update":"Fri Dec 1 01:16:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.71499999999999997,"15min":0.71599999999999997},"min":{"1min":0.44800000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.73499999999999999,"15min":0.74099999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.94799999999999995,"5min":1.252,"15min":1.252},"last":0.77700000000000002}]},{"osd":9,"last update":"Fri Dec 1 01:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.73099999999999998,"15min":0.73499999999999999},"min":{"1min":0.503,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88500000000000001,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.76200000000000001,"15min":0.76600000000000001},"min":{"1min":0.58699999999999997,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.93999999999999995,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.86499999999999999}]},{"osd":10,"last update":"Fri Dec 1 01:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1,"5min":1,"15min":1},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.88200000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":900,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":921600,"data_stored":424944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 01:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.498,"15min":0.46800000000000003},"min":{"1min":0.35699999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.80600000000000005,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.53000000000000003,"15min":0.49099999999999999},"min":{"1min":0.39500000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.879,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.52300000000000002}]},{"osd":2,"last update":"Fri Dec 1 01:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.56799999999999995,"15min":0.51200000000000001},"min":{"1min":0.39000000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":3.4590000000000001,"5min":3.4590000000000001,"15min":3.4590000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.54800000000000004,"15min":0.51200000000000001},"min":{"1min":0.35499999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":3.1429999999999998,"5min":3.1429999999999998,"15min":3.1429999999999998},"last":0.59999999999999998}]},{"osd":3,"last update":"Fri Dec 1 01:14:57 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.36699999999999999,"15min":0.35899999999999999},"min":{"1min":0.247,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.318},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.40799999999999997,"15min":0.40200000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.34699999999999998}]},{"osd":4,"last update":"Fri Dec 1 01:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54300000000000004,"15min":0.53300000000000003},"min":{"1min":0.40300000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.81200000000000006,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.58999999999999997,"15min":0.57499999999999996},"min":{"1min":0.42299999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.92300000000000004,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.49299999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.56299999999999994,"15min":0.55500000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.61899999999999999,"15min":0.60299999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.708,"5min":1.708,"15min":1.708},"last":0.60999999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:17:00 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.59599999999999997,"15min":0.55500000000000005},"min":{"1min":0.27900000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":3.7189999999999999,"5min":3.7189999999999999,"15min":3.7189999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.60199999999999998,"15min":0.56699999999999995},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":3.9340000000000002,"5min":3.9340000000000002,"15min":3.9340000000000002},"last":0.436}]},{"osd":7,"last update":"Fri Dec 1 01:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.623,"15min":0.621},"min":{"1min":0.43099999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67800000000000005,"15min":0.68200000000000005},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":1.6220000000000001}]},{"osd":8,"last update":"Fri Dec 1 01:16:41 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.65400000000000003,"15min":0.66000000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.629,"5min":0.66700000000000004,"15min":0.67300000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.57199999999999995}]},{"osd":9,"last update":"Fri Dec 1 01:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.67600000000000005,"15min":0.67200000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":3.694,"5min":3.694,"15min":3.694},"last":0.58699999999999997},{"interface":"front","average":{"1min":1.1910000000000001,"5min":0.751,"15min":0.67700000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":14.702,"5min":14.702,"15min":14.702},"last":0.503}]},{"osd":10,"last update":"Fri Dec 1 01:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.64300000000000002}]},{"osd":11,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":3.9689999999999999,"5min":3.9689999999999999,"15min":3.9689999999999999},"last":0.625},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":0.61699999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":983040,"data_stored":483613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.497,"15min":0.47999999999999998},"min":{"1min":0.32700000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.79800000000000004,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.504,"5min":0.497,"15min":0.47899999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70399999999999996,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.443}]},{"osd":1,"last update":"Fri Dec 1 01:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.52700000000000002,"15min":0.51000000000000001},"min":{"1min":0.36199999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.72899999999999998,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.58999999999999997,"15min":0.54200000000000004},"min":{"1min":0.35899999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":5.4169999999999998,"5min":5.4169999999999998,"15min":5.4169999999999998},"last":0.74299999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.59799999999999998,"15min":0.53000000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.91400000000000003,"5min":5.4710000000000001,"15min":5.4710000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56999999999999995,"15min":0.53000000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.78500000000000003,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.55500000000000005}]},{"osd":4,"last update":"Fri Dec 1 01:16:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.53200000000000003,"15min":0.51800000000000002},"min":{"1min":0.378,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.53100000000000003,"15min":0.52900000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.70199999999999996}]},{"osd":5,"last update":"Fri Dec 1 01:16:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.437,"15min":0.42199999999999999},"min":{"1min":0.312,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.49399999999999999,"15min":0.47199999999999998},"min":{"1min":0.36599999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.76000000000000001}]},{"osd":6,"last update":"Fri Dec 1 01:17:00 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.64500000000000002,"15min":0.60499999999999998},"min":{"1min":0.43099999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.89300000000000002,"5min":5.5019999999999998,"15min":5.5019999999999998},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.63800000000000001,"15min":0.63400000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77100000000000002,"5min":1.429,"15min":1.429},"last":0.79700000000000004}]},{"osd":7,"last update":"Fri Dec 1 01:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.54000000000000004}]},{"osd":8,"last update":"Fri Dec 1 01:16:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.53700000000000003,"15min":0.52600000000000002},"min":{"1min":0.34799999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.58899999999999997,"15min":0.58299999999999996},"min":{"1min":0.39900000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.62}]},{"osd":9,"last update":"Fri Dec 1 01:17:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.69799999999999995,"15min":0.70099999999999996},"min":{"1min":0.52000000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.004,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.68700000000000006,"15min":0.68200000000000005},"min":{"1min":0.51700000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94299999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.871}]},{"osd":10,"last update":"Fri Dec 1 01:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":5.4450000000000003,"5min":5.4450000000000003,"15min":5.4450000000000003},"last":0.81299999999999994}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]}]},{"osd":3,"up_from":25,"seq":107374182454,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":917504,"data_stored":423490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:17:02 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.34300000000000003,"15min":0.31900000000000001},"min":{"1min":0.26800000000000002,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.51300000000000001,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.35699999999999998,"15min":0.33800000000000002},"min":{"1min":0.26600000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.48299999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.39400000000000002}]},{"osd":1,"last update":"Fri Dec 1 01:17:02 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.53500000000000003,"15min":0.505},"min":{"1min":0.38200000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73099999999999998,"5min":0.996,"15min":0.996},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.375,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.69599999999999995,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.54100000000000004}]},{"osd":2,"last update":"Fri Dec 1 01:17:02 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.51700000000000002},"min":{"1min":0.39700000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.749,"5min":0.88,"15min":0.88},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55100000000000005,"15min":0.52800000000000002},"min":{"1min":0.41599999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.78600000000000003,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.59399999999999997}]},{"osd":4,"last update":"Fri Dec 1 01:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.57799999999999996,"15min":0.55600000000000005},"min":{"1min":0.40899999999999997,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.60399999999999998,"15min":0.58699999999999997},"min":{"1min":0.44500000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.629}]},{"osd":5,"last update":"Fri Dec 1 01:16:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.55800000000000005,"15min":0.55500000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.65100000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.57399999999999995,"15min":0.57299999999999995},"min":{"1min":0.40100000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.69299999999999995,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.80500000000000005}]},{"osd":6,"last update":"Fri Dec 1 01:17:02 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.46500000000000002,"15min":0.47299999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.55100000000000005,"15min":0.54500000000000004},"min":{"1min":0.34999999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.69899999999999995,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.52700000000000002}]},{"osd":7,"last update":"Fri Dec 1 01:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.63300000000000001,"15min":0.63100000000000001},"min":{"1min":0.46300000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.82999999999999996,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.65600000000000003,"15min":0.65100000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.56799999999999995}]},{"osd":8,"last update":"Fri Dec 1 01:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.51200000000000001}]},{"osd":9,"last update":"Fri Dec 1 01:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.71199999999999997,"15min":0.73399999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77300000000000002,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.63600000000000001,"15min":0.65100000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.75,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.57799999999999996}]},{"osd":10,"last update":"Fri Dec 1 01:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.60299999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]}]},{"osd":4,"up_from":32,"seq":137438953523,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":901120,"data_stored":423466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.54300000000000004,"15min":0.50900000000000001},"min":{"1min":0.48799999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.373},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.53400000000000003,"15min":0.499},"min":{"1min":0.47199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.497}]},{"osd":1,"last update":"Fri Dec 1 01:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.40500000000000003,"15min":0.378},"min":{"1min":0.29599999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.495,"5min":0.39800000000000002,"15min":0.371},"min":{"1min":0.32100000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.505}]},{"osd":2,"last update":"Fri Dec 1 01:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.55900000000000005,"15min":0.53000000000000003},"min":{"1min":0.48199999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.59499999999999997,"15min":0.56799999999999995},"min":{"1min":0.40400000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.78200000000000003}]},{"osd":3,"last update":"Fri Dec 1 01:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.53300000000000003,"15min":0.499},"min":{"1min":0.45700000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.97899999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.52200000000000002,"15min":0.504},"min":{"1min":0.373,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.89300000000000002,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.56399999999999995}]},{"osd":5,"last update":"Fri Dec 1 01:16:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57399999999999995,"15min":0.55800000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.64300000000000002,"15min":0.60599999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.85399999999999998}]},{"osd":6,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.629,"15min":0.61199999999999999},"min":{"1min":0.42999999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.94699999999999995,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.66000000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.65500000000000003}]},{"osd":7,"last update":"Fri Dec 1 01:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.57699999999999996,"15min":0.55700000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.56599999999999995,"15min":0.55000000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.68000000000000005}]},{"osd":8,"last update":"Fri Dec 1 01:16:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.63100000000000001,"15min":0.63600000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76000000000000001,"15min":0.75900000000000001},"min":{"1min":0.47699999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.88200000000000001}]},{"osd":9,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.72999999999999998,"15min":0.73499999999999999},"min":{"1min":0.46600000000000003,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.87},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.79000000000000004,"15min":0.80000000000000004},"min":{"1min":0.51600000000000001,"5min":0.503,"15min":0.503},"max":{"1min":1.0900000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.747}]},{"osd":10,"last update":"Fri Dec 1 01:16:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.84099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]}]},{"osd":5,"up_from":37,"seq":158913789998,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28436,"kb_used_data":1484,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716716,"statfs":{"total":95995035648,"available":95965917184,"internally_reserved":0,"allocated":1519616,"data_stored":1016139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:16:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.52700000000000002,"15min":0.50900000000000001},"min":{"1min":0.39400000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52900000000000003,"15min":0.52500000000000002},"min":{"1min":0.38400000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.433}]},{"osd":1,"last update":"Fri Dec 1 01:16:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.51100000000000001,"15min":0.48699999999999999},"min":{"1min":0.38300000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.68300000000000005,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.53200000000000003,"15min":0.501},"min":{"1min":0.39800000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.63700000000000001}]},{"osd":2,"last update":"Fri Dec 1 01:16:39 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.39400000000000002,"15min":0.38600000000000001},"min":{"1min":0.31,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.38800000000000001,"15min":0.376},"min":{"1min":0.28799999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.34200000000000003}]},{"osd":3,"last update":"Fri Dec 1 01:16:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.54100000000000004,"15min":0.53100000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.55300000000000005,"15min":0.53300000000000003},"min":{"1min":0.40600000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.501}]},{"osd":4,"last update":"Fri Dec 1 01:16:39 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.55600000000000005,"15min":0.53200000000000003},"min":{"1min":0.45000000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.55200000000000005,"15min":0.52900000000000003},"min":{"1min":0.40600000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.78200000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.72899999999999998}]},{"osd":6,"last update":"Fri Dec 1 01:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.64200000000000002,"15min":0.626},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":2.391,"5min":2.391,"15min":2.391},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.66000000000000003,"15min":0.63},"min":{"1min":0.42899999999999999,"5min":0.37,"15min":0.37},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":0.48799999999999999}]},{"osd":7,"last update":"Fri Dec 1 01:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.64900000000000002,"15min":0.65100000000000002},"min":{"1min":0.49399999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.85299999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.624,"5min":0.61299999999999999,"15min":0.61099999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.55100000000000005}]},{"osd":8,"last update":"Fri Dec 1 01:16:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58099999999999996,"15min":0.57999999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.754,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.77000000000000002}]},{"osd":9,"last update":"Fri Dec 1 01:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.70999999999999996,"15min":0.70199999999999996},"min":{"1min":0.47299999999999998,"5min":0.443,"15min":0.443},"max":{"1min":2.8069999999999999,"5min":2.8069999999999999,"15min":2.8069999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.71799999999999997,"15min":0.70199999999999996},"min":{"1min":0.48199999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":3.9049999999999998,"5min":3.9049999999999998,"15min":3.9049999999999998},"last":0.59699999999999998}]},{"osd":10,"last update":"Fri Dec 1 01:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.78600000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T01:17:12.456 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-12-01T01:17:12.456 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-12-01T01:17:12.457 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-12-01T01:17:12.457 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph health --format=json 2023-12-01T01:17:13.007 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:17:13.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:13 smithi077 ceph-mon[98867]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-01T01:17:13.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:13 smithi077 ceph-mon[98867]: pgmap v276: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 8 op/s 2023-12-01T01:17:14.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:13 smithi203 ceph-mon[103581]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-01T01:17:14.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:13 smithi203 ceph-mon[103581]: pgmap v276: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 8 op/s 2023-12-01T01:17:14.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:13 smithi137 ceph-mon[103312]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-01T01:17:14.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:13 smithi137 ceph-mon[103312]: pgmap v276: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 8 op/s 2023-12-01T01:17:14.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.216+0000 7f0358458700 1 -- 172.21.15.77:0/2715224707 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03500ffcd0 msgr2=0x7f0350100150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:14.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.216+0000 7f0358458700 1 --2- 172.21.15.77:0/2715224707 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03500ffcd0 0x7f0350100150 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f034c009990 tx=0x7f034c009ca0 comp rx=0 tx=0).stop 2023-12-01T01:17:14.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.216+0000 7f0358458700 1 -- 172.21.15.77:0/2715224707 shutdown_connections 2023-12-01T01:17:14.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.216+0000 7f0358458700 1 --2- 172.21.15.77:0/2715224707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0350100690 0x7f035010cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:14.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.216+0000 7f0358458700 1 --2- 172.21.15.77:0/2715224707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0350105d80 0x7f0350106160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:14.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.216+0000 7f0358458700 1 --2- 172.21.15.77:0/2715224707 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03500ffcd0 0x7f0350100150 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:14.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.216+0000 7f0358458700 1 -- 172.21.15.77:0/2715224707 >> 172.21.15.77:0/2715224707 conn(0x7f0350076d90 msgr2=0x7f03500771a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:14.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.216+0000 7f0358458700 1 -- 172.21.15.77:0/2715224707 shutdown_connections 2023-12-01T01:17:14.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.216+0000 7f0358458700 1 -- 172.21.15.77:0/2715224707 wait complete. 2023-12-01T01:17:14.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.217+0000 7f0358458700 1 Processor -- start 2023-12-01T01:17:14.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.217+0000 7f0358458700 1 -- start start 2023-12-01T01:17:14.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.217+0000 7f0358458700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03500ffcd0 0x7f0350196ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:14.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.217+0000 7f0358458700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0350100690 0x7f0350197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:14.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.217+0000 7f0358458700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0350105d80 0x7f035019b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:14.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.217+0000 7f0358458700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035019be00 con 0x7f03500ffcd0 2023-12-01T01:17:14.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.217+0000 7f0358458700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035019bf70 con 0x7f0350100690 2023-12-01T01:17:14.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.217+0000 7f0358458700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035019c130 con 0x7f0350105d80 2023-12-01T01:17:14.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.217+0000 7f03561f4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03500ffcd0 0x7f0350196ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:14.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.217+0000 7f03561f4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03500ffcd0 0x7f0350196ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58136/0 (socket says 172.21.15.77:58136) 2023-12-01T01:17:14.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.217+0000 7f03561f4700 1 -- 172.21.15.77:0/639593059 learned_addr learned my addr 172.21.15.77:0/639593059 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:14.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.217+0000 7f03569f5700 1 --2- 172.21.15.77:0/639593059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0350105d80 0x7f035019b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:14.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.218+0000 7f03559f3700 1 --2- 172.21.15.77:0/639593059 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0350100690 0x7f0350197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:14.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.218+0000 7f03561f4700 1 -- 172.21.15.77:0/639593059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0350105d80 msgr2=0x7f035019b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:14.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.218+0000 7f03561f4700 1 --2- 172.21.15.77:0/639593059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0350105d80 0x7f035019b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:14.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.218+0000 7f03561f4700 1 -- 172.21.15.77:0/639593059 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0350100690 msgr2=0x7f0350197420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:14.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.218+0000 7f03561f4700 1 --2- 172.21.15.77:0/639593059 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0350100690 0x7f0350197420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:14.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.218+0000 7f03561f4700 1 -- 172.21.15.77:0/639593059 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f034c0095f0 con 0x7f03500ffcd0 2023-12-01T01:17:14.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.218+0000 7f03561f4700 1 --2- 172.21.15.77:0/639593059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03500ffcd0 0x7f0350196ee0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f034000d980 tx=0x7f034000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:14.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.218+0000 7f03477fe700 1 -- 172.21.15.77:0/639593059 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0340004a30 con 0x7f03500ffcd0 2023-12-01T01:17:14.222 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.218+0000 7f0358458700 1 -- 172.21.15.77:0/639593059 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f035019c410 con 0x7f03500ffcd0 2023-12-01T01:17:14.222 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.218+0000 7f03477fe700 1 -- 172.21.15.77:0/639593059 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03400054e0 con 0x7f03500ffcd0 2023-12-01T01:17:14.222 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.218+0000 7f03477fe700 1 -- 172.21.15.77:0/639593059 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0340009c90 con 0x7f03500ffcd0 2023-12-01T01:17:14.222 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.218+0000 7f0358458700 1 -- 172.21.15.77:0/639593059 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03501a3c70 con 0x7f03500ffcd0 2023-12-01T01:17:14.223 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.219+0000 7f0358458700 1 -- 172.21.15.77:0/639593059 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f035004ea90 con 0x7f03500ffcd0 2023-12-01T01:17:14.223 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.220+0000 7f03477fe700 1 -- 172.21.15.77:0/639593059 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f0340004c20 con 0x7f03500ffcd0 2023-12-01T01:17:14.223 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.221+0000 7f03477fe700 1 --2- 172.21.15.77:0/639593059 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f033c0a05b0 0x7f033c0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:14.223 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.221+0000 7f03559f3700 1 --2- 172.21.15.77:0/639593059 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f033c0a05b0 0x7f033c0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:14.223 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.221+0000 7f03477fe700 1 -- 172.21.15.77:0/639593059 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f03400cbc60 con 0x7f03500ffcd0 2023-12-01T01:17:14.223 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.221+0000 7f03559f3700 1 --2- 172.21.15.77:0/639593059 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f033c0a05b0 0x7f033c0a2a70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f034c006010 tx=0x7f034c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:14.225 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.224+0000 7f03477fe700 1 -- 172.21.15.77:0/639593059 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0340095850 con 0x7f03500ffcd0 2023-12-01T01:17:14.458 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.457+0000 7f0358458700 1 -- 172.21.15.77:0/639593059 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f0350198240 con 0x7f03500ffcd0 2023-12-01T01:17:14.458 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.457+0000 7f03477fe700 1 -- 172.21.15.77:0/639593059 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f0340098e70 con 0x7f03500ffcd0 2023-12-01T01:17:14.459 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:17:14.459 INFO:teuthology.orchestra.run.smithi077.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-12-01T01:17:14.460 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.459+0000 7f0358458700 1 -- 172.21.15.77:0/639593059 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f033c0a05b0 msgr2=0x7f033c0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:14.460 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.459+0000 7f0358458700 1 --2- 172.21.15.77:0/639593059 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f033c0a05b0 0x7f033c0a2a70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f034c006010 tx=0x7f034c00b540 comp rx=0 tx=0).stop 2023-12-01T01:17:14.461 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.459+0000 7f0358458700 1 -- 172.21.15.77:0/639593059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03500ffcd0 msgr2=0x7f0350196ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:14.461 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.459+0000 7f0358458700 1 --2- 172.21.15.77:0/639593059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03500ffcd0 0x7f0350196ee0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f034000d980 tx=0x7f034000dc90 comp rx=0 tx=0).stop 2023-12-01T01:17:14.461 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.459+0000 7f0358458700 1 -- 172.21.15.77:0/639593059 shutdown_connections 2023-12-01T01:17:14.461 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.459+0000 7f0358458700 1 --2- 172.21.15.77:0/639593059 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f033c0a05b0 0x7f033c0a2a70 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:14.461 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.459+0000 7f0358458700 1 --2- 172.21.15.77:0/639593059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0350105d80 0x7f035019b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:14.461 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.459+0000 7f0358458700 1 --2- 172.21.15.77:0/639593059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03500ffcd0 0x7f0350196ee0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:14.461 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.459+0000 7f0358458700 1 --2- 172.21.15.77:0/639593059 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0350100690 0x7f0350197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:14.461 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.459+0000 7f0358458700 1 -- 172.21.15.77:0/639593059 >> 172.21.15.77:0/639593059 conn(0x7f0350076d90 msgr2=0x7f0350068850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:14.461 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.459+0000 7f0358458700 1 -- 172.21.15.77:0/639593059 shutdown_connections 2023-12-01T01:17:14.461 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.459+0000 7f0358458700 1 -- 172.21.15.77:0/639593059 wait complete. 2023-12-01T01:17:14.810 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:14 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/639593059' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-01T01:17:14.861 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-12-01T01:17:14.862 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:17:14.987 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.985+0000 7f644c6fb700 1 -- 172.21.15.77:0/314741307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64440ffc70 msgr2=0x7f6444100090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:14.987 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.985+0000 7f644c6fb700 1 --2- 172.21.15.77:0/314741307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64440ffc70 0x7f6444100090 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f64380099c0 tx=0x7f6438009cd0 comp rx=0 tx=0).stop 2023-12-01T01:17:14.987 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.985+0000 7f644c6fb700 1 -- 172.21.15.77:0/314741307 shutdown_connections 2023-12-01T01:17:14.987 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.985+0000 7f644c6fb700 1 --2- 172.21.15.77:0/314741307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6444101800 0x7f64441080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:14.987 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.985+0000 7f644c6fb700 1 --2- 172.21.15.77:0/314741307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64440ffc70 0x7f6444100090 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:14.987 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.985+0000 7f644c6fb700 1 --2- 172.21.15.77:0/314741307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6444100e60 0x7f64441012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:14.987 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.985+0000 7f644c6fb700 1 -- 172.21.15.77:0/314741307 >> 172.21.15.77:0/314741307 conn(0x7f64440fb1c0 msgr2=0x7f64440fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:14.987 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.985+0000 7f644c6fb700 1 -- 172.21.15.77:0/314741307 shutdown_connections 2023-12-01T01:17:14.987 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.985+0000 7f644c6fb700 1 -- 172.21.15.77:0/314741307 wait complete. 2023-12-01T01:17:14.989 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.986+0000 7f644c6fb700 1 Processor -- start 2023-12-01T01:17:14.989 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.986+0000 7f644c6fb700 1 -- start start 2023-12-01T01:17:14.990 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.986+0000 7f644c6fb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f64440ffc70 0x7f6444195910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:14.990 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.986+0000 7f644c6fb700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6444100e60 0x7f6444195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:14.990 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.986+0000 7f644c6fb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6444101800 0x7f644419cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:14.990 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.986+0000 7f644c6fb700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f644419d490 con 0x7f6444101800 2023-12-01T01:17:14.990 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.986+0000 7f644c6fb700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f644419d5d0 con 0x7f64440ffc70 2023-12-01T01:17:14.990 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.986+0000 7f644c6fb700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f644419d760 con 0x7f6444100e60 2023-12-01T01:17:14.990 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.986+0000 7f644ac98700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6444101800 0x7f644419cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:14.990 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.987+0000 7f644ac98700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6444101800 0x7f644419cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58152/0 (socket says 172.21.15.77:58152) 2023-12-01T01:17:14.990 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.987+0000 7f644ac98700 1 -- 172.21.15.77:0/799212511 learned_addr learned my addr 172.21.15.77:0/799212511 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:14.990 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.987+0000 7f644a497700 1 --2- 172.21.15.77:0/799212511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f64440ffc70 0x7f6444195910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:14.990 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.987+0000 7f6449c96700 1 --2- 172.21.15.77:0/799212511 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6444100e60 0x7f6444195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:14.990 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.987+0000 7f644ac98700 1 -- 172.21.15.77:0/799212511 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6444100e60 msgr2=0x7f6444195e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:14.990 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.987+0000 7f644ac98700 1 --2- 172.21.15.77:0/799212511 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6444100e60 0x7f6444195e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:14.991 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.987+0000 7f644ac98700 1 -- 172.21.15.77:0/799212511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f64440ffc70 msgr2=0x7f6444195910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:14.991 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.987+0000 7f644ac98700 1 --2- 172.21.15.77:0/799212511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f64440ffc70 0x7f6444195910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:14.991 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.987+0000 7f644ac98700 1 -- 172.21.15.77:0/799212511 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64380095f0 con 0x7f6444101800 2023-12-01T01:17:14.991 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.987+0000 7f644ac98700 1 --2- 172.21.15.77:0/799212511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6444101800 0x7f644419cec0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f643c007ee0 tx=0x7f643c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:14.991 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.987+0000 7f64377fe700 1 -- 172.21.15.77:0/799212511 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643c017070 con 0x7f6444101800 2023-12-01T01:17:14.991 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.987+0000 7f64377fe700 1 -- 172.21.15.77:0/799212511 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f643c00f040 con 0x7f6444101800 2023-12-01T01:17:14.991 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.987+0000 7f644c6fb700 1 -- 172.21.15.77:0/799212511 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f644419da40 con 0x7f6444101800 2023-12-01T01:17:14.991 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.987+0000 7f64377fe700 1 -- 172.21.15.77:0/799212511 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643c005740 con 0x7f6444101800 2023-12-01T01:17:14.991 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.987+0000 7f644c6fb700 1 -- 172.21.15.77:0/799212511 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f644419df00 con 0x7f6444101800 2023-12-01T01:17:14.991 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.988+0000 7f644c6fb700 1 -- 172.21.15.77:0/799212511 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f644410b480 con 0x7f6444101800 2023-12-01T01:17:14.992 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.989+0000 7f64377fe700 1 -- 172.21.15.77:0/799212511 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f643c0058a0 con 0x7f6444101800 2023-12-01T01:17:14.992 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.990+0000 7f64377fe700 1 --2- 172.21.15.77:0/799212511 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f64300a05f0 0x7f64300a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:14.992 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.990+0000 7f644a497700 1 --2- 172.21.15.77:0/799212511 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f64300a05f0 0x7f64300a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:14.992 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.990+0000 7f64377fe700 1 -- 172.21.15.77:0/799212511 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f643c0cb980 con 0x7f6444101800 2023-12-01T01:17:14.992 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.990+0000 7f644a497700 1 --2- 172.21.15.77:0/799212511 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f64300a05f0 0x7f64300a2ab0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f643800b580 tx=0x7f6438005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:14.994 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:14.993+0000 7f64377fe700 1 -- 172.21.15.77:0/799212511 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f643c095470 con 0x7f6444101800 2023-12-01T01:17:15.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:14 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/639593059' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-01T01:17:15.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:14 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/639593059' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-01T01:17:15.188 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.187+0000 7f644c6fb700 1 -- 172.21.15.77:0/799212511 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f644410b640 con 0x7f6444101800 2023-12-01T01:17:15.190 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.188+0000 7f64377fe700 1 -- 172.21.15.77:0/799212511 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22131 (secure 0 0 0) 0x7f643c098a90 con 0x7f6444101800 2023-12-01T01:17:15.190 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:17:15.190 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":82,"fsid":"2a06e29a-8fe6-11ee-95a2-87774f69a715","created":"2023-12-01T01:09:30.210398+0000","modified":"2023-12-01T01:16:26.346791+0000","last_up_change":"2023-12-01T01:15:44.586092+0000","last_in_change":"2023-12-01T01:15:26.305439+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:12:30.514229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:16:23.568860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-01T01:16:24.531137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fc651e62-a103-4ab8-a76d-d3708ea6aeec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6803","nonce":2244799094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6805","nonce":2244799094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6809","nonce":2244799094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6807","nonce":2244799094}]},"public_addr":"172.21.15.77:6803/2244799094","cluster_addr":"172.21.15.77:6805/2244799094","heartbeat_back_addr":"172.21.15.77:6809/2244799094","heartbeat_front_addr":"172.21.15.77:6807/2244799094","state":["exists","up"]},{"osd":1,"uuid":"0adf8c54-49ec-49a0-8bf1-af1f20e60729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6801","nonce":224194890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6803","nonce":224194890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6807","nonce":224194890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6805","nonce":224194890}]},"public_addr":"172.21.15.137:6801/224194890","cluster_addr":"172.21.15.137:6803/224194890","heartbeat_back_addr":"172.21.15.137:6807/224194890","heartbeat_front_addr":"172.21.15.137:6805/224194890","state":["exists","up"]},{"osd":2,"uuid":"5a69802c-dede-4544-9a12-436f5c1c0038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6801","nonce":4078459110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6803","nonce":4078459110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6807","nonce":4078459110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6805","nonce":4078459110}]},"public_addr":"172.21.15.203:6801/4078459110","cluster_addr":"172.21.15.203:6803/4078459110","heartbeat_back_addr":"172.21.15.203:6807/4078459110","heartbeat_front_addr":"172.21.15.203:6805/4078459110","state":["exists","up"]},{"osd":3,"uuid":"ad325a97-580b-4c99-bf12-cbb0d92a6d0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6811","nonce":1454440477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6813","nonce":1454440477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6817","nonce":1454440477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6815","nonce":1454440477}]},"public_addr":"172.21.15.77:6811/1454440477","cluster_addr":"172.21.15.77:6813/1454440477","heartbeat_back_addr":"172.21.15.77:6817/1454440477","heartbeat_front_addr":"172.21.15.77:6815/1454440477","state":["exists","up"]},{"osd":4,"uuid":"1e3ac1ba-98bb-4b8b-8e26-50064c66bf2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6809","nonce":1103241451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6811","nonce":1103241451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6815","nonce":1103241451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6813","nonce":1103241451}]},"public_addr":"172.21.15.137:6809/1103241451","cluster_addr":"172.21.15.137:6811/1103241451","heartbeat_back_addr":"172.21.15.137:6815/1103241451","heartbeat_front_addr":"172.21.15.137:6813/1103241451","state":["exists","up"]},{"osd":5,"uuid":"eb32c016-abeb-44ab-abf0-19dd09235736","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6809","nonce":2648569482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6811","nonce":2648569482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6815","nonce":2648569482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6813","nonce":2648569482}]},"public_addr":"172.21.15.203:6809/2648569482","cluster_addr":"172.21.15.203:6811/2648569482","heartbeat_back_addr":"172.21.15.203:6815/2648569482","heartbeat_front_addr":"172.21.15.203:6813/2648569482","state":["exists","up"]},{"osd":6,"uuid":"0e3fa97e-2ca6-418c-9e8d-a6fe3c56adb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6819","nonce":2463414651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6821","nonce":2463414651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6825","nonce":2463414651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6823","nonce":2463414651}]},"public_addr":"172.21.15.77:6819/2463414651","cluster_addr":"172.21.15.77:6821/2463414651","heartbeat_back_addr":"172.21.15.77:6825/2463414651","heartbeat_front_addr":"172.21.15.77:6823/2463414651","state":["exists","up"]},{"osd":7,"uuid":"d7398c01-0f98-4138-8a49-c3ff0aae7e88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6817","nonce":3108330766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6819","nonce":3108330766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6823","nonce":3108330766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6821","nonce":3108330766}]},"public_addr":"172.21.15.137:6817/3108330766","cluster_addr":"172.21.15.137:6819/3108330766","heartbeat_back_addr":"172.21.15.137:6823/3108330766","heartbeat_front_addr":"172.21.15.137:6821/3108330766","state":["exists","up"]},{"osd":8,"uuid":"0c09050d-3e43-410c-8cd7-bcb891af56b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6817","nonce":1161158776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6819","nonce":1161158776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6823","nonce":1161158776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6821","nonce":1161158776}]},"public_addr":"172.21.15.203:6817/1161158776","cluster_addr":"172.21.15.203:6819/1161158776","heartbeat_back_addr":"172.21.15.203:6823/1161158776","heartbeat_front_addr":"172.21.15.203:6821/1161158776","state":["exists","up"]},{"osd":9,"uuid":"527ecb49-95ec-48bd-9872-3b07fae5c5ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6827","nonce":2409880685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6829","nonce":2409880685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6833","nonce":2409880685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6831","nonce":2409880685}]},"public_addr":"172.21.15.77:6827/2409880685","cluster_addr":"172.21.15.77:6829/2409880685","heartbeat_back_addr":"172.21.15.77:6833/2409880685","heartbeat_front_addr":"172.21.15.77:6831/2409880685","state":["exists","up"]},{"osd":10,"uuid":"1680bb3e-a3c8-427b-af68-e2214f32bed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6825","nonce":374920146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6827","nonce":374920146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6831","nonce":374920146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6829","nonce":374920146}]},"public_addr":"172.21.15.137:6825/374920146","cluster_addr":"172.21.15.137:6827/374920146","heartbeat_back_addr":"172.21.15.137:6831/374920146","heartbeat_front_addr":"172.21.15.137:6829/374920146","state":["exists","up"]},{"osd":11,"uuid":"8a533466-023d-4803-ae36-9a9f1715e49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6825","nonce":2161588962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6827","nonce":2161588962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6831","nonce":2161588962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6829","nonce":2161588962}]},"public_addr":"172.21.15.203:6825/2161588962","cluster_addr":"172.21.15.203:6827/2161588962","heartbeat_back_addr":"172.21.15.203:6831/2161588962","heartbeat_front_addr":"172.21.15.203:6829/2161588962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:11:45.420509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:06.210115+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:28.191307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:49.594116+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:11.261689+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:33.376541+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:54.968336+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:16.775310+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:38.733792+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:59.456699+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:15:20.924059+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:15:43.094773+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/2260035254":"2023-12-02T01:10:24.191835+0000","172.21.15.77:6800/1456621747":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/219838828":"2023-12-02T01:10:24.191835+0000","172.21.15.77:6801/1456621747":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/2937149266":"2023-12-02T01:10:02.232948+0000","172.21.15.77:0/807520018":"2023-12-02T01:10:02.232948+0000","172.21.15.77:6801/1661788026":"2023-12-02T01:10:02.232948+0000","172.21.15.77:6800/1661788026":"2023-12-02T01:10:02.232948+0000","172.21.15.77:0/3551282473":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/1202222558":"2023-12-02T01:10:02.232948+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:17:15.192 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.191+0000 7f644c6fb700 1 -- 172.21.15.77:0/799212511 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f64300a05f0 msgr2=0x7f64300a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:15.192 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.191+0000 7f644c6fb700 1 --2- 172.21.15.77:0/799212511 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f64300a05f0 0x7f64300a2ab0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f643800b580 tx=0x7f6438005fb0 comp rx=0 tx=0).stop 2023-12-01T01:17:15.192 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.191+0000 7f644c6fb700 1 -- 172.21.15.77:0/799212511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6444101800 msgr2=0x7f644419cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:15.192 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.191+0000 7f644c6fb700 1 --2- 172.21.15.77:0/799212511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6444101800 0x7f644419cec0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f643c007ee0 tx=0x7f643c00d3b0 comp rx=0 tx=0).stop 2023-12-01T01:17:15.192 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.191+0000 7f644c6fb700 1 -- 172.21.15.77:0/799212511 shutdown_connections 2023-12-01T01:17:15.192 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.191+0000 7f644c6fb700 1 --2- 172.21.15.77:0/799212511 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f64300a05f0 0x7f64300a2ab0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:15.192 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.191+0000 7f644c6fb700 1 --2- 172.21.15.77:0/799212511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6444101800 0x7f644419cec0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:15.192 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.191+0000 7f644c6fb700 1 --2- 172.21.15.77:0/799212511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f64440ffc70 0x7f6444195910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:15.192 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.191+0000 7f644c6fb700 1 --2- 172.21.15.77:0/799212511 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6444100e60 0x7f6444195e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:15.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.191+0000 7f644c6fb700 1 -- 172.21.15.77:0/799212511 >> 172.21.15.77:0/799212511 conn(0x7f64440fb1c0 msgr2=0x7f64440fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:15.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.191+0000 7f644c6fb700 1 -- 172.21.15.77:0/799212511 shutdown_connections 2023-12-01T01:17:15.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.191+0000 7f644c6fb700 1 -- 172.21.15.77:0/799212511 wait complete. 2023-12-01T01:17:15.204 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:12:30.514229+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T01:16:23.568860+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T01:16:24.531137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:17:15.205 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:17:15.332 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.329+0000 7f8e0d3e4700 1 -- 172.21.15.77:0/960936227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e080ff030 msgr2=0x7f8e080ff450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:15.332 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.329+0000 7f8e0d3e4700 1 --2- 172.21.15.77:0/960936227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e080ff030 0x7f8e080ff450 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc009990 tx=0x7f8dfc009ca0 comp rx=0 tx=0).stop 2023-12-01T01:17:15.332 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.329+0000 7f8e0d3e4700 1 -- 172.21.15.77:0/960936227 shutdown_connections 2023-12-01T01:17:15.332 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.329+0000 7f8e0d3e4700 1 --2- 172.21.15.77:0/960936227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e081029a0 0x7f8e08109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:15.332 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.329+0000 7f8e0d3e4700 1 --2- 172.21.15.77:0/960936227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e080ff030 0x7f8e080ff450 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:15.333 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.329+0000 7f8e0d3e4700 1 --2- 172.21.15.77:0/960936227 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e08102000 0x7f8e08102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:15.333 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.329+0000 7f8e0d3e4700 1 -- 172.21.15.77:0/960936227 >> 172.21.15.77:0/960936227 conn(0x7f8e080fa960 msgr2=0x7f8e080fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:15.333 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.329+0000 7f8e0d3e4700 1 -- 172.21.15.77:0/960936227 shutdown_connections 2023-12-01T01:17:15.333 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.329+0000 7f8e0d3e4700 1 -- 172.21.15.77:0/960936227 wait complete. 2023-12-01T01:17:15.333 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.329+0000 7f8e0d3e4700 1 Processor -- start 2023-12-01T01:17:15.333 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.329+0000 7f8e0d3e4700 1 -- start start 2023-12-01T01:17:15.333 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.330+0000 7f8e0d3e4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e080ff030 0x7f8e08101970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:15.333 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.330+0000 7f8e0d3e4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e08102000 0x7f8e080fffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:15.333 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.330+0000 7f8e0d3e4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e081029a0 0x7f8e081004f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:15.333 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.330+0000 7f8e0d3e4700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e08100c90 con 0x7f8e080ff030 2023-12-01T01:17:15.333 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.330+0000 7f8e0d3e4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e08100e00 con 0x7f8e08102000 2023-12-01T01:17:15.333 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.330+0000 7f8e0d3e4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e081a0af0 con 0x7f8e081029a0 2023-12-01T01:17:15.333 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.330+0000 7f8e06ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e080ff030 0x7f8e08101970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:15.333 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.330+0000 7f8e06ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e080ff030 0x7f8e08101970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58162/0 (socket says 172.21.15.77:58162) 2023-12-01T01:17:15.333 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.330+0000 7f8e06ffd700 1 -- 172.21.15.77:0/3410945778 learned_addr learned my addr 172.21.15.77:0/3410945778 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:15.334 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.330+0000 7f8e067fc700 1 --2- 172.21.15.77:0/3410945778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e08102000 0x7f8e080fffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:15.334 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.330+0000 7f8e077fe700 1 --2- 172.21.15.77:0/3410945778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e081029a0 0x7f8e081004f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:15.334 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.330+0000 7f8e06ffd700 1 -- 172.21.15.77:0/3410945778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e081029a0 msgr2=0x7f8e081004f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:15.334 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.330+0000 7f8e06ffd700 1 --2- 172.21.15.77:0/3410945778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e081029a0 0x7f8e081004f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:15.334 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.330+0000 7f8e06ffd700 1 -- 172.21.15.77:0/3410945778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e08102000 msgr2=0x7f8e080fffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:15.334 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.330+0000 7f8e06ffd700 1 --2- 172.21.15.77:0/3410945778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e08102000 0x7f8e080fffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:15.334 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.330+0000 7f8e06ffd700 1 -- 172.21.15.77:0/3410945778 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dfc0095f0 con 0x7f8e080ff030 2023-12-01T01:17:15.334 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.330+0000 7f8e077fe700 1 --2- 172.21.15.77:0/3410945778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e081029a0 0x7f8e081004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:17:15.334 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.330+0000 7f8e06ffd700 1 --2- 172.21.15.77:0/3410945778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e080ff030 0x7f8e08101970 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc000c00 tx=0x7f8dfc00f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:15.334 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.330+0000 7f8deffff700 1 -- 172.21.15.77:0/3410945778 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc01d070 con 0x7f8e080ff030 2023-12-01T01:17:15.334 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.330+0000 7f8e0d3e4700 1 -- 172.21.15.77:0/3410945778 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e081a0d70 con 0x7f8e080ff030 2023-12-01T01:17:15.334 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.330+0000 7f8deffff700 1 -- 172.21.15.77:0/3410945778 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8dfc00fb50 con 0x7f8e080ff030 2023-12-01T01:17:15.334 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.330+0000 7f8deffff700 1 -- 172.21.15.77:0/3410945778 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc017650 con 0x7f8e080ff030 2023-12-01T01:17:15.335 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.330+0000 7f8e0d3e4700 1 -- 172.21.15.77:0/3410945778 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e081a11c0 con 0x7f8e080ff030 2023-12-01T01:17:15.335 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.331+0000 7f8e0d3e4700 1 -- 172.21.15.77:0/3410945778 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e08107df0 con 0x7f8e080ff030 2023-12-01T01:17:15.336 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.332+0000 7f8deffff700 1 -- 172.21.15.77:0/3410945778 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f8dfc00fcc0 con 0x7f8e080ff030 2023-12-01T01:17:15.336 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.333+0000 7f8deffff700 1 --2- 172.21.15.77:0/3410945778 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8df00a05b0 0x7f8df00a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:15.336 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.333+0000 7f8e067fc700 1 --2- 172.21.15.77:0/3410945778 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8df00a05b0 0x7f8df00a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:15.336 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.333+0000 7f8deffff700 1 -- 172.21.15.77:0/3410945778 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8dfc0ccf50 con 0x7f8e080ff030 2023-12-01T01:17:15.336 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.333+0000 7f8e067fc700 1 --2- 172.21.15.77:0/3410945778 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8df00a05b0 0x7f8df00a2a70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8df8005950 tx=0x7f8df800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:15.337 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.336+0000 7f8deffff700 1 -- 172.21.15.77:0/3410945778 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8dfc096bd0 con 0x7f8e080ff030 2023-12-01T01:17:15.529 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.528+0000 7f8e0d3e4700 1 -- 172.21.15.77:0/3410945778 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8e0802cc70 con 0x7f8e080ff030 2023-12-01T01:17:15.530 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.528+0000 7f8deffff700 1 -- 172.21.15.77:0/3410945778 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8dfc09a1f0 con 0x7f8e080ff030 2023-12-01T01:17:15.530 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 1 2023-12-01T01:17:15.532 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.530+0000 7f8e0d3e4700 1 -- 172.21.15.77:0/3410945778 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8df00a05b0 msgr2=0x7f8df00a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:15.532 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.530+0000 7f8e0d3e4700 1 --2- 172.21.15.77:0/3410945778 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8df00a05b0 0x7f8df00a2a70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8df8005950 tx=0x7f8df800b410 comp rx=0 tx=0).stop 2023-12-01T01:17:15.532 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.530+0000 7f8e0d3e4700 1 -- 172.21.15.77:0/3410945778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e080ff030 msgr2=0x7f8e08101970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:15.532 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.530+0000 7f8e0d3e4700 1 --2- 172.21.15.77:0/3410945778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e080ff030 0x7f8e08101970 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc000c00 tx=0x7f8dfc00f710 comp rx=0 tx=0).stop 2023-12-01T01:17:15.532 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.530+0000 7f8e0d3e4700 1 -- 172.21.15.77:0/3410945778 shutdown_connections 2023-12-01T01:17:15.532 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.530+0000 7f8e0d3e4700 1 --2- 172.21.15.77:0/3410945778 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8df00a05b0 0x7f8df00a2a70 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:15.532 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.530+0000 7f8e0d3e4700 1 --2- 172.21.15.77:0/3410945778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e081029a0 0x7f8e081004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:15.532 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.530+0000 7f8e0d3e4700 1 --2- 172.21.15.77:0/3410945778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8e080ff030 0x7f8e08101970 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:15.532 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.530+0000 7f8e0d3e4700 1 --2- 172.21.15.77:0/3410945778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e08102000 0x7f8e080fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:15.533 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.530+0000 7f8e0d3e4700 1 -- 172.21.15.77:0/3410945778 >> 172.21.15.77:0/3410945778 conn(0x7f8e080fa960 msgr2=0x7f8e080fcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:15.533 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.531+0000 7f8e0d3e4700 1 -- 172.21.15.77:0/3410945778 shutdown_connections 2023-12-01T01:17:15.533 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.531+0000 7f8e0d3e4700 1 -- 172.21.15.77:0/3410945778 wait complete. 2023-12-01T01:17:15.544 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T01:17:15.672 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.668+0000 7f39be1dd700 1 -- 172.21.15.77:0/1504635128 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39b8101810 msgr2=0x7f39b81080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:15.672 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.668+0000 7f39be1dd700 1 --2- 172.21.15.77:0/1504635128 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39b8101810 0x7f39b81080c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f39b000b0d0 tx=0x7f39b000b3e0 comp rx=0 tx=0).stop 2023-12-01T01:17:15.672 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.668+0000 7f39be1dd700 1 -- 172.21.15.77:0/1504635128 shutdown_connections 2023-12-01T01:17:15.672 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.668+0000 7f39be1dd700 1 --2- 172.21.15.77:0/1504635128 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39b8101810 0x7f39b81080c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:15.672 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.668+0000 7f39be1dd700 1 --2- 172.21.15.77:0/1504635128 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39b80ffc80 0x7f39b81000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:15.672 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.668+0000 7f39be1dd700 1 --2- 172.21.15.77:0/1504635128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39b8100e70 0x7f39b81012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:15.672 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.668+0000 7f39be1dd700 1 -- 172.21.15.77:0/1504635128 >> 172.21.15.77:0/1504635128 conn(0x7f39b80fb1f0 msgr2=0x7f39b80fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:15.672 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.668+0000 7f39be1dd700 1 -- 172.21.15.77:0/1504635128 shutdown_connections 2023-12-01T01:17:15.672 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.668+0000 7f39be1dd700 1 -- 172.21.15.77:0/1504635128 wait complete. 2023-12-01T01:17:15.672 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.668+0000 7f39be1dd700 1 Processor -- start 2023-12-01T01:17:15.672 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.669+0000 7f39be1dd700 1 -- start start 2023-12-01T01:17:15.672 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.669+0000 7f39be1dd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39b80ffc80 0x7f39b8196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:15.673 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.669+0000 7f39be1dd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39b8100e70 0x7f39b81970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:15.673 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.669+0000 7f39be1dd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39b8101810 0x7f39b819e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:15.673 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.669+0000 7f39be1dd700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b819e720 con 0x7f39b80ffc80 2023-12-01T01:17:15.673 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.669+0000 7f39be1dd700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b819e860 con 0x7f39b8101810 2023-12-01T01:17:15.673 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.669+0000 7f39be1dd700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b819e9f0 con 0x7f39b8100e70 2023-12-01T01:17:15.673 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.669+0000 7f39b77fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39b80ffc80 0x7f39b8196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:15.673 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.669+0000 7f39b77fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39b80ffc80 0x7f39b8196ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58176/0 (socket says 172.21.15.77:58176) 2023-12-01T01:17:15.673 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.669+0000 7f39b77fe700 1 -- 172.21.15.77:0/2362082880 learned_addr learned my addr 172.21.15.77:0/2362082880 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:15.673 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.669+0000 7f39b7fff700 1 --2- 172.21.15.77:0/2362082880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39b8101810 0x7f39b819e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:15.673 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.669+0000 7f39b6ffd700 1 --2- 172.21.15.77:0/2362082880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39b8100e70 0x7f39b81970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:15.673 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.669+0000 7f39b77fe700 1 -- 172.21.15.77:0/2362082880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39b8100e70 msgr2=0x7f39b81970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:15.673 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.669+0000 7f39b77fe700 1 --2- 172.21.15.77:0/2362082880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39b8100e70 0x7f39b81970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:15.673 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.669+0000 7f39b77fe700 1 -- 172.21.15.77:0/2362082880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39b8101810 msgr2=0x7f39b819e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:15.673 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.669+0000 7f39b77fe700 1 --2- 172.21.15.77:0/2362082880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39b8101810 0x7f39b819e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:15.673 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.669+0000 7f39b77fe700 1 -- 172.21.15.77:0/2362082880 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b0009d00 con 0x7f39b80ffc80 2023-12-01T01:17:15.673 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.669+0000 7f39b6ffd700 1 --2- 172.21.15.77:0/2362082880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39b8100e70 0x7f39b81970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:17:15.674 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.669+0000 7f39b77fe700 1 --2- 172.21.15.77:0/2362082880 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39b80ffc80 0x7f39b8196ba0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f39a800e970 tx=0x7f39a800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:15.674 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.669+0000 7f39b4ff9700 1 -- 172.21.15.77:0/2362082880 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a8009790 con 0x7f39b80ffc80 2023-12-01T01:17:15.674 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.670+0000 7f39be1dd700 1 -- 172.21.15.77:0/2362082880 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39b819ecd0 con 0x7f39b80ffc80 2023-12-01T01:17:15.674 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.670+0000 7f39b4ff9700 1 -- 172.21.15.77:0/2362082880 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39a80044e0 con 0x7f39b80ffc80 2023-12-01T01:17:15.674 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.670+0000 7f39b4ff9700 1 -- 172.21.15.77:0/2362082880 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a8003e30 con 0x7f39b80ffc80 2023-12-01T01:17:15.674 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.670+0000 7f39be1dd700 1 -- 172.21.15.77:0/2362082880 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39b819f180 con 0x7f39b80ffc80 2023-12-01T01:17:15.674 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.671+0000 7f39be1dd700 1 -- 172.21.15.77:0/2362082880 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39b810b560 con 0x7f39b80ffc80 2023-12-01T01:17:15.675 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.671+0000 7f39b4ff9700 1 -- 172.21.15.77:0/2362082880 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f39a8003f90 con 0x7f39b80ffc80 2023-12-01T01:17:15.675 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.672+0000 7f39b4ff9700 1 --2- 172.21.15.77:0/2362082880 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f39a40a0630 0x7f39a40a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:15.675 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.672+0000 7f39b6ffd700 1 --2- 172.21.15.77:0/2362082880 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f39a40a0630 0x7f39a40a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:15.675 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.672+0000 7f39b4ff9700 1 -- 172.21.15.77:0/2362082880 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f39a8014070 con 0x7f39b80ffc80 2023-12-01T01:17:15.675 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.672+0000 7f39b6ffd700 1 --2- 172.21.15.77:0/2362082880 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f39a40a0630 0x7f39a40a2af0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f39a0005fd0 tx=0x7f39a0005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:15.677 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.676+0000 7f39b4ff9700 1 -- 172.21.15.77:0/2362082880 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f39a8095850 con 0x7f39b80ffc80 2023-12-01T01:17:15.870 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.868+0000 7f39be1dd700 1 -- 172.21.15.77:0/2362082880 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f39b810b720 con 0x7f39b80ffc80 2023-12-01T01:17:15.870 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.869+0000 7f39b4ff9700 1 -- 172.21.15.77:0/2362082880 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7f39a8098e70 con 0x7f39b80ffc80 2023-12-01T01:17:15.870 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 32 2023-12-01T01:17:15.873 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.871+0000 7f39be1dd700 1 -- 172.21.15.77:0/2362082880 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f39a40a0630 msgr2=0x7f39a40a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:15.873 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.871+0000 7f39be1dd700 1 --2- 172.21.15.77:0/2362082880 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f39a40a0630 0x7f39a40a2af0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f39a0005fd0 tx=0x7f39a0005ee0 comp rx=0 tx=0).stop 2023-12-01T01:17:15.873 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.871+0000 7f39be1dd700 1 -- 172.21.15.77:0/2362082880 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39b80ffc80 msgr2=0x7f39b8196ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:15.873 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.871+0000 7f39be1dd700 1 --2- 172.21.15.77:0/2362082880 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39b80ffc80 0x7f39b8196ba0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f39a800e970 tx=0x7f39a800ed30 comp rx=0 tx=0).stop 2023-12-01T01:17:15.873 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.871+0000 7f39be1dd700 1 -- 172.21.15.77:0/2362082880 shutdown_connections 2023-12-01T01:17:15.873 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.871+0000 7f39be1dd700 1 --2- 172.21.15.77:0/2362082880 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f39a40a0630 0x7f39a40a2af0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:15.873 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.871+0000 7f39be1dd700 1 --2- 172.21.15.77:0/2362082880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39b8101810 0x7f39b819e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:15.873 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.871+0000 7f39be1dd700 1 --2- 172.21.15.77:0/2362082880 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39b80ffc80 0x7f39b8196ba0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:15.873 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.871+0000 7f39be1dd700 1 --2- 172.21.15.77:0/2362082880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39b8100e70 0x7f39b81970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:15.873 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.871+0000 7f39be1dd700 1 -- 172.21.15.77:0/2362082880 >> 172.21.15.77:0/2362082880 conn(0x7f39b80fb1f0 msgr2=0x7f39b8067eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:15.873 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.871+0000 7f39be1dd700 1 -- 172.21.15.77:0/2362082880 shutdown_connections 2023-12-01T01:17:15.873 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:15.871+0000 7f39be1dd700 1 -- 172.21.15.77:0/2362082880 wait complete. 2023-12-01T01:17:15.884 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:15 smithi077 ceph-mon[98867]: pgmap v277: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2023-12-01T01:17:15.884 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:15 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/799212511' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:17:15.884 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:15 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/3410945778' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-01T01:17:15.885 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T01:17:16.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.007+0000 7f3454646700 1 -- 172.21.15.77:0/2597983954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f344c101f90 msgr2=0x7f344c1023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:16.094 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.007+0000 7f3454646700 1 --2- 172.21.15.77:0/2597983954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f344c101f90 0x7f344c1023f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f343c0099c0 tx=0x7f343c009cd0 comp rx=0 tx=0).stop 2023-12-01T01:17:16.094 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.008+0000 7f3454646700 1 -- 172.21.15.77:0/2597983954 shutdown_connections 2023-12-01T01:17:16.094 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.008+0000 7f3454646700 1 --2- 172.21.15.77:0/2597983954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f344c102930 0x7f344c1091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:16.094 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.008+0000 7f3454646700 1 --2- 172.21.15.77:0/2597983954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f344c100da0 0x7f344c1011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:16.094 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.008+0000 7f3454646700 1 --2- 172.21.15.77:0/2597983954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f344c101f90 0x7f344c1023f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:16.094 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.008+0000 7f3454646700 1 -- 172.21.15.77:0/2597983954 >> 172.21.15.77:0/2597983954 conn(0x7f344c0fc310 msgr2=0x7f344c0fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:16.094 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.008+0000 7f3454646700 1 -- 172.21.15.77:0/2597983954 shutdown_connections 2023-12-01T01:17:16.094 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.008+0000 7f3454646700 1 -- 172.21.15.77:0/2597983954 wait complete. 2023-12-01T01:17:16.094 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.008+0000 7f3454646700 1 Processor -- start 2023-12-01T01:17:16.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.008+0000 7f3454646700 1 -- start start 2023-12-01T01:17:16.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.008+0000 7f3454646700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f344c100da0 0x7f344c196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:16.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.008+0000 7f3454646700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f344c101f90 0x7f344c197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:16.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.008+0000 7f3454646700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f344c102930 0x7f344c19e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:16.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.008+0000 7f3454646700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344c19e6b0 con 0x7f344c101f90 2023-12-01T01:17:16.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.008+0000 7f3454646700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344c19e7f0 con 0x7f344c100da0 2023-12-01T01:17:16.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.008+0000 7f3454646700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344c19e980 con 0x7f344c102930 2023-12-01T01:17:16.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.008+0000 7f3451be1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f344c101f90 0x7f344c197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:16.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.009+0000 7f3451be1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f344c101f90 0x7f344c197070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58192/0 (socket says 172.21.15.77:58192) 2023-12-01T01:17:16.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.009+0000 7f3451be1700 1 -- 172.21.15.77:0/184822007 learned_addr learned my addr 172.21.15.77:0/184822007 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:16.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.009+0000 7f3452be3700 1 --2- 172.21.15.77:0/184822007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f344c102930 0x7f344c19e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:16.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.009+0000 7f34523e2700 1 --2- 172.21.15.77:0/184822007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f344c100da0 0x7f344c196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:16.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.009+0000 7f3451be1700 1 -- 172.21.15.77:0/184822007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f344c102930 msgr2=0x7f344c19e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:16.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.009+0000 7f3451be1700 1 --2- 172.21.15.77:0/184822007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f344c102930 0x7f344c19e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:16.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.009+0000 7f3451be1700 1 -- 172.21.15.77:0/184822007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f344c100da0 msgr2=0x7f344c196b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:16.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.009+0000 7f3451be1700 1 --2- 172.21.15.77:0/184822007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f344c100da0 0x7f344c196b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:16.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.009+0000 7f3451be1700 1 -- 172.21.15.77:0/184822007 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f343c0095f0 con 0x7f344c101f90 2023-12-01T01:17:16.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.009+0000 7f34523e2700 1 --2- 172.21.15.77:0/184822007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f344c100da0 0x7f344c196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:17:16.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.009+0000 7f3451be1700 1 --2- 172.21.15.77:0/184822007 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f344c101f90 0x7f344c197070 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f343c00c010 tx=0x7f343c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:16.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.009+0000 7f34437fe700 1 -- 172.21.15.77:0/184822007 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343c01d070 con 0x7f344c101f90 2023-12-01T01:17:16.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.009+0000 7f34437fe700 1 -- 172.21.15.77:0/184822007 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f343c0040d0 con 0x7f344c101f90 2023-12-01T01:17:16.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.009+0000 7f3454646700 1 -- 172.21.15.77:0/184822007 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f344c19ec00 con 0x7f344c101f90 2023-12-01T01:17:16.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.009+0000 7f34437fe700 1 -- 172.21.15.77:0/184822007 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343c005290 con 0x7f344c101f90 2023-12-01T01:17:16.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.009+0000 7f3454646700 1 -- 172.21.15.77:0/184822007 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f344c19f020 con 0x7f344c101f90 2023-12-01T01:17:16.097 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.010+0000 7f3454646700 1 -- 172.21.15.77:0/184822007 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f344c107d20 con 0x7f344c101f90 2023-12-01T01:17:16.097 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:15 smithi137 ceph-mon[103312]: pgmap v277: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2023-12-01T01:17:16.098 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:15 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/799212511' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:17:16.098 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:15 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/3410945778' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-01T01:17:16.098 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:15 smithi203 ceph-mon[103581]: pgmap v277: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2023-12-01T01:17:16.098 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:15 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/799212511' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:17:16.098 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:15 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/3410945778' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-01T01:17:16.099 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.011+0000 7f34437fe700 1 -- 172.21.15.77:0/184822007 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f343c004810 con 0x7f344c101f90 2023-12-01T01:17:16.099 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.012+0000 7f34437fe700 1 --2- 172.21.15.77:0/184822007 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f34380a05b0 0x7f34380a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:16.099 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.012+0000 7f34523e2700 1 --2- 172.21.15.77:0/184822007 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f34380a05b0 0x7f34380a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:16.099 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.012+0000 7f34437fe700 1 -- 172.21.15.77:0/184822007 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f343c0cc9a0 con 0x7f344c101f90 2023-12-01T01:17:16.099 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.012+0000 7f34523e2700 1 --2- 172.21.15.77:0/184822007 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f34380a05b0 0x7f34380a2a70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f344c101df0 tx=0x7f3448008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:16.099 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.016+0000 7f34437fe700 1 -- 172.21.15.77:0/184822007 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f343c096620 con 0x7f344c101f90 2023-12-01T01:17:16.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.207+0000 7f3454646700 1 -- 172.21.15.77:0/184822007 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f344c10c710 con 0x7f344c101f90 2023-12-01T01:17:16.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.207+0000 7f34437fe700 1 -- 172.21.15.77:0/184822007 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7f343c099c40 con 0x7f344c101f90 2023-12-01T01:17:16.209 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 64 2023-12-01T01:17:16.211 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.209+0000 7f3454646700 1 -- 172.21.15.77:0/184822007 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f34380a05b0 msgr2=0x7f34380a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:16.211 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.209+0000 7f3454646700 1 --2- 172.21.15.77:0/184822007 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f34380a05b0 0x7f34380a2a70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f344c101df0 tx=0x7f3448008040 comp rx=0 tx=0).stop 2023-12-01T01:17:16.211 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.210+0000 7f3454646700 1 -- 172.21.15.77:0/184822007 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f344c101f90 msgr2=0x7f344c197070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:16.211 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.210+0000 7f3454646700 1 --2- 172.21.15.77:0/184822007 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f344c101f90 0x7f344c197070 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f343c00c010 tx=0x7f343c009580 comp rx=0 tx=0).stop 2023-12-01T01:17:16.211 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.210+0000 7f3454646700 1 -- 172.21.15.77:0/184822007 shutdown_connections 2023-12-01T01:17:16.212 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.210+0000 7f3454646700 1 --2- 172.21.15.77:0/184822007 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f34380a05b0 0x7f34380a2a70 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:16.212 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.210+0000 7f3454646700 1 --2- 172.21.15.77:0/184822007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f344c102930 0x7f344c19e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:16.212 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.210+0000 7f3454646700 1 --2- 172.21.15.77:0/184822007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f344c100da0 0x7f344c196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:16.212 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.210+0000 7f3454646700 1 --2- 172.21.15.77:0/184822007 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f344c101f90 0x7f344c197070 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:16.212 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.210+0000 7f3454646700 1 -- 172.21.15.77:0/184822007 >> 172.21.15.77:0/184822007 conn(0x7f344c0fc310 msgr2=0x7f344c0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:16.212 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.210+0000 7f3454646700 1 -- 172.21.15.77:0/184822007 shutdown_connections 2023-12-01T01:17:16.212 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.210+0000 7f3454646700 1 -- 172.21.15.77:0/184822007 wait complete. 2023-12-01T01:17:16.223 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:17:16.348 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.343+0000 7f9e0fe20700 1 -- 172.21.15.77:0/517907917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e08101810 msgr2=0x7f9e081080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:16.348 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.343+0000 7f9e0fe20700 1 --2- 172.21.15.77:0/517907917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e08101810 0x7f9e081080c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9e0000b0d0 tx=0x7f9e0000b3e0 comp rx=0 tx=0).stop 2023-12-01T01:17:16.348 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.344+0000 7f9e0fe20700 1 -- 172.21.15.77:0/517907917 shutdown_connections 2023-12-01T01:17:16.348 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.344+0000 7f9e0fe20700 1 --2- 172.21.15.77:0/517907917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e08101810 0x7f9e081080c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:16.348 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.344+0000 7f9e0fe20700 1 --2- 172.21.15.77:0/517907917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e080ffc80 0x7f9e081000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:16.349 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.344+0000 7f9e0fe20700 1 --2- 172.21.15.77:0/517907917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e08100e70 0x7f9e081012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:16.349 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.344+0000 7f9e0fe20700 1 -- 172.21.15.77:0/517907917 >> 172.21.15.77:0/517907917 conn(0x7f9e080fb1f0 msgr2=0x7f9e080fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:16.349 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.344+0000 7f9e0fe20700 1 -- 172.21.15.77:0/517907917 shutdown_connections 2023-12-01T01:17:16.349 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.344+0000 7f9e0fe20700 1 -- 172.21.15.77:0/517907917 wait complete. 2023-12-01T01:17:16.349 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.344+0000 7f9e0fe20700 1 Processor -- start 2023-12-01T01:17:16.349 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.344+0000 7f9e0fe20700 1 -- start start 2023-12-01T01:17:16.349 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.344+0000 7f9e0fe20700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e080ffc80 0x7f9e08069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:16.349 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.344+0000 7f9e0fe20700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e08100e70 0x7f9e0819b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:16.349 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.344+0000 7f9e0fe20700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e08101810 0x7f9e0819de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:16.349 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.344+0000 7f9e0fe20700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0819e640 con 0x7f9e08101810 2023-12-01T01:17:16.349 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.344+0000 7f9e0fe20700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0819e7b0 con 0x7f9e080ffc80 2023-12-01T01:17:16.349 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.344+0000 7f9e0fe20700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0819e970 con 0x7f9e08100e70 2023-12-01T01:17:16.349 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.345+0000 7f9e0e3bd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e08101810 0x7f9e0819de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:16.350 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.345+0000 7f9e0e3bd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e08101810 0x7f9e0819de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58208/0 (socket says 172.21.15.77:58208) 2023-12-01T01:17:16.350 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.345+0000 7f9e0e3bd700 1 -- 172.21.15.77:0/4246386090 learned_addr learned my addr 172.21.15.77:0/4246386090 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:16.350 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.345+0000 7f9e0d3bb700 1 --2- 172.21.15.77:0/4246386090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e08100e70 0x7f9e0819b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:16.350 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.345+0000 7f9e0dbbc700 1 --2- 172.21.15.77:0/4246386090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e080ffc80 0x7f9e08069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:16.350 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.345+0000 7f9e0e3bd700 1 -- 172.21.15.77:0/4246386090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e08100e70 msgr2=0x7f9e0819b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:16.350 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.345+0000 7f9e0e3bd700 1 --2- 172.21.15.77:0/4246386090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e08100e70 0x7f9e0819b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:16.350 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.345+0000 7f9e0e3bd700 1 -- 172.21.15.77:0/4246386090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e080ffc80 msgr2=0x7f9e08069ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:16.350 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.345+0000 7f9e0e3bd700 1 --2- 172.21.15.77:0/4246386090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e080ffc80 0x7f9e08069ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:16.350 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.345+0000 7f9e0e3bd700 1 -- 172.21.15.77:0/4246386090 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e00009d00 con 0x7f9e08101810 2023-12-01T01:17:16.350 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.345+0000 7f9e0d3bb700 1 --2- 172.21.15.77:0/4246386090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e08100e70 0x7f9e0819b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:17:16.350 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.345+0000 7f9e0e3bd700 1 --2- 172.21.15.77:0/4246386090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e08101810 0x7f9e0819de70 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9e00007a00 tx=0x7f9e00009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:16.350 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.345+0000 7f9dfeffd700 1 -- 172.21.15.77:0/4246386090 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0000e040 con 0x7f9e08101810 2023-12-01T01:17:16.351 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.345+0000 7f9e0fe20700 1 -- 172.21.15.77:0/4246386090 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e0819ebf0 con 0x7f9e08101810 2023-12-01T01:17:16.351 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.345+0000 7f9dfeffd700 1 -- 172.21.15.77:0/4246386090 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e00003e10 con 0x7f9e08101810 2023-12-01T01:17:16.351 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.345+0000 7f9dfeffd700 1 -- 172.21.15.77:0/4246386090 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e00005610 con 0x7f9e08101810 2023-12-01T01:17:16.351 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.345+0000 7f9e0fe20700 1 -- 172.21.15.77:0/4246386090 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e0819efc0 con 0x7f9e08101810 2023-12-01T01:17:16.351 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.346+0000 7f9e0fe20700 1 -- 172.21.15.77:0/4246386090 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e08107180 con 0x7f9e08101810 2023-12-01T01:17:16.351 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.347+0000 7f9dfeffd700 1 -- 172.21.15.77:0/4246386090 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f9e00005770 con 0x7f9e08101810 2023-12-01T01:17:16.352 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.348+0000 7f9dfeffd700 1 --2- 172.21.15.77:0/4246386090 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f9df40a05f0 0x7f9df40a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:16.352 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.348+0000 7f9e0dbbc700 1 --2- 172.21.15.77:0/4246386090 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f9df40a05f0 0x7f9df40a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:16.352 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.348+0000 7f9dfeffd700 1 -- 172.21.15.77:0/4246386090 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9e000cd240 con 0x7f9e08101810 2023-12-01T01:17:16.352 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.348+0000 7f9e0dbbc700 1 --2- 172.21.15.77:0/4246386090 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f9df40a05f0 0x7f9df40a2ab0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9e04005950 tx=0x7f9e040058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:16.353 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.352+0000 7f9dfeffd700 1 -- 172.21.15.77:0/4246386090 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9e00096ec0 con 0x7f9e08101810 2023-12-01T01:17:16.587 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.586+0000 7f9e0fe20700 1 -- 172.21.15.77:0/4246386090 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e0802d090 con 0x7f9e08101810 2023-12-01T01:17:16.588 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.587+0000 7f9dfeffd700 1 -- 172.21.15.77:0/4246386090 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10838 (secure 0 0 0) 0x7f9e0009a4e0 con 0x7f9e08101810 2023-12-01T01:17:16.588 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:17:16.588 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1435145620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1435145620},{"type":"v1","addr":"172.21.15.77:6835","nonce":1435145620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/841838908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":841838908},{"type":"v1","addr":"172.21.15.77:6837","nonce":841838908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-01T01:16:25.345276+0000","modified":"2023-12-01T01:16:32.596617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24496,"mds_2":24475,"mds_3":24473,"mds_4":14649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/3726913446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3726913446},{"type":"v1","addr":"172.21.15.77:6839","nonce":3726913446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14649":{"gid":14649,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":5,"addr":"172.21.15.77:6841/3023184965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3023184965},{"type":"v1","addr":"172.21.15.77:6841","nonce":3023184965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/2249638445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2249638445},{"type":"v1","addr":"172.21.15.137:6833","nonce":2249638445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6833/1201599774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1201599774},{"type":"v1","addr":"172.21.15.203:6833","nonce":1201599774}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3935354878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3935354878},{"type":"v1","addr":"172.21.15.137:6835","nonce":3935354878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3237538176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3237538176},{"type":"v1","addr":"172.21.15.203:6835","nonce":3237538176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3518881867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3518881867},{"type":"v1","addr":"172.21.15.137:6837","nonce":3518881867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.203:6837/2887080977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2887080977},{"type":"v1","addr":"172.21.15.203:6837","nonce":2887080977}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.203:6839/2888221613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2888221613},{"type":"v1","addr":"172.21.15.203:6839","nonce":2888221613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/1099966660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1099966660},{"type":"v1","addr":"172.21.15.137:6839","nonce":1099966660}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-01T01:17:16.591 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.589+0000 7f9e0fe20700 1 -- 172.21.15.77:0/4246386090 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f9df40a05f0 msgr2=0x7f9df40a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:16.591 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.589+0000 7f9e0fe20700 1 --2- 172.21.15.77:0/4246386090 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f9df40a05f0 0x7f9df40a2ab0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9e04005950 tx=0x7f9e040058e0 comp rx=0 tx=0).stop 2023-12-01T01:17:16.591 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.589+0000 7f9e0fe20700 1 -- 172.21.15.77:0/4246386090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e08101810 msgr2=0x7f9e0819de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:16.591 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.589+0000 7f9e0fe20700 1 --2- 172.21.15.77:0/4246386090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e08101810 0x7f9e0819de70 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9e00007a00 tx=0x7f9e00009570 comp rx=0 tx=0).stop 2023-12-01T01:17:16.591 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.589+0000 7f9e0fe20700 1 -- 172.21.15.77:0/4246386090 shutdown_connections 2023-12-01T01:17:16.591 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.589+0000 7f9e0fe20700 1 --2- 172.21.15.77:0/4246386090 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f9df40a05f0 0x7f9df40a2ab0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:16.591 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.589+0000 7f9e0fe20700 1 --2- 172.21.15.77:0/4246386090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e08101810 0x7f9e0819de70 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:16.591 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.589+0000 7f9e0fe20700 1 --2- 172.21.15.77:0/4246386090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e080ffc80 0x7f9e08069ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:16.591 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.589+0000 7f9e0fe20700 1 --2- 172.21.15.77:0/4246386090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e08100e70 0x7f9e0819b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:16.591 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.589+0000 7f9e0fe20700 1 -- 172.21.15.77:0/4246386090 >> 172.21.15.77:0/4246386090 conn(0x7f9e080fb1f0 msgr2=0x7f9e081042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:16.591 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.589+0000 7f9e0fe20700 1 -- 172.21.15.77:0/4246386090 shutdown_connections 2023-12-01T01:17:16.592 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.589+0000 7f9e0fe20700 1 -- 172.21.15.77:0/4246386090 wait complete. 2023-12-01T01:17:16.592 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 27 2023-12-01T01:17:16.603 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:17:16.728 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.726+0000 7f59448de700 1 -- 172.21.15.77:0/1389419584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59401020d0 msgr2=0x7f5940102530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:16.728 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.726+0000 7f59448de700 1 --2- 172.21.15.77:0/1389419584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59401020d0 0x7f5940102530 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5934009990 tx=0x7f5934009ca0 comp rx=0 tx=0).stop 2023-12-01T01:17:16.728 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.727+0000 7f59448de700 1 -- 172.21.15.77:0/1389419584 shutdown_connections 2023-12-01T01:17:16.728 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.727+0000 7f59448de700 1 --2- 172.21.15.77:0/1389419584 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5940102a70 0x7f5940109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:16.728 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.727+0000 7f59448de700 1 --2- 172.21.15.77:0/1389419584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59400ff060 0x7f59400ff480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:16.728 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.727+0000 7f59448de700 1 --2- 172.21.15.77:0/1389419584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59401020d0 0x7f5940102530 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:16.728 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.727+0000 7f59448de700 1 -- 172.21.15.77:0/1389419584 >> 172.21.15.77:0/1389419584 conn(0x7f59400fa950 msgr2=0x7f59400fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:16.729 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.727+0000 7f59448de700 1 -- 172.21.15.77:0/1389419584 shutdown_connections 2023-12-01T01:17:16.729 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.727+0000 7f59448de700 1 -- 172.21.15.77:0/1389419584 wait complete. 2023-12-01T01:17:16.731 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.727+0000 7f59448de700 1 Processor -- start 2023-12-01T01:17:16.731 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.727+0000 7f59448de700 1 -- start start 2023-12-01T01:17:16.731 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.727+0000 7f59448de700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59400ff060 0x7f5940196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:16.731 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.727+0000 7f59448de700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59401020d0 0x7f5940196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:16.731 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.727+0000 7f59448de700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5940102a70 0x7f594019dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:16.731 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.728+0000 7f59448de700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594019e5b0 con 0x7f59400ff060 2023-12-01T01:17:16.731 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.728+0000 7f59448de700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594019e6f0 con 0x7f5940102a70 2023-12-01T01:17:16.731 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.728+0000 7f59448de700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594019e880 con 0x7f59401020d0 2023-12-01T01:17:16.731 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.728+0000 7f593e59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59400ff060 0x7f5940196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:16.731 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.728+0000 7f593e59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59400ff060 0x7f5940196a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58218/0 (socket says 172.21.15.77:58218) 2023-12-01T01:17:16.731 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.728+0000 7f593e59c700 1 -- 172.21.15.77:0/259309703 learned_addr learned my addr 172.21.15.77:0/259309703 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:16.731 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.728+0000 7f593ed9d700 1 --2- 172.21.15.77:0/259309703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5940102a70 0x7f594019dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:16.732 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.728+0000 7f593dd9b700 1 --2- 172.21.15.77:0/259309703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59401020d0 0x7f5940196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:16.732 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.728+0000 7f593e59c700 1 -- 172.21.15.77:0/259309703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59401020d0 msgr2=0x7f5940196f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:16.732 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.728+0000 7f593e59c700 1 --2- 172.21.15.77:0/259309703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59401020d0 0x7f5940196f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:16.732 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.728+0000 7f593e59c700 1 -- 172.21.15.77:0/259309703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5940102a70 msgr2=0x7f594019dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:16.732 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.728+0000 7f593e59c700 1 --2- 172.21.15.77:0/259309703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5940102a70 0x7f594019dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:16.732 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.728+0000 7f593e59c700 1 -- 172.21.15.77:0/259309703 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59340095f0 con 0x7f59400ff060 2023-12-01T01:17:16.732 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.728+0000 7f593e59c700 1 --2- 172.21.15.77:0/259309703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59400ff060 0x7f5940196a30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f593000d980 tx=0x7f593000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:16.732 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.728+0000 7f592f7fe700 1 -- 172.21.15.77:0/259309703 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593000f890 con 0x7f59400ff060 2023-12-01T01:17:16.732 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.728+0000 7f592f7fe700 1 -- 172.21.15.77:0/259309703 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5930010460 con 0x7f59400ff060 2023-12-01T01:17:16.732 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.728+0000 7f59448de700 1 -- 172.21.15.77:0/259309703 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f594019eb60 con 0x7f59400ff060 2023-12-01T01:17:16.732 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.728+0000 7f592f7fe700 1 -- 172.21.15.77:0/259309703 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59300195c0 con 0x7f59400ff060 2023-12-01T01:17:16.732 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.728+0000 7f59448de700 1 -- 172.21.15.77:0/259309703 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f594019f070 con 0x7f59400ff060 2023-12-01T01:17:16.732 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.729+0000 7f59448de700 1 -- 172.21.15.77:0/259309703 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5940107e60 con 0x7f59400ff060 2023-12-01T01:17:16.733 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.730+0000 7f592f7fe700 1 -- 172.21.15.77:0/259309703 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f59300105d0 con 0x7f59400ff060 2023-12-01T01:17:16.733 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.731+0000 7f592f7fe700 1 --2- 172.21.15.77:0/259309703 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f59280a05f0 0x7f59280a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:16.733 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.731+0000 7f592f7fe700 1 -- 172.21.15.77:0/259309703 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f59300cbbf0 con 0x7f59400ff060 2023-12-01T01:17:16.733 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.731+0000 7f593dd9b700 1 --2- 172.21.15.77:0/259309703 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f59280a05f0 0x7f59280a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:16.733 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.731+0000 7f593dd9b700 1 --2- 172.21.15.77:0/259309703 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f59280a05f0 0x7f59280a2ab0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f593400b5c0 tx=0x7f5934005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:16.736 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.735+0000 7f592f7fe700 1 -- 172.21.15.77:0/259309703 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f59300957e0 con 0x7f59400ff060 2023-12-01T01:17:16.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:16 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/2362082880' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-01T01:17:16.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:16 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/184822007' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-01T01:17:16.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:16 smithi077 ceph-mon[98867]: pgmap v278: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-01T01:17:16.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:16 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/4246386090' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:17:16.932 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.931+0000 7f59448de700 1 -- 172.21.15.77:0/259309703 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f594002cc70 con 0x7f59400ff060 2023-12-01T01:17:16.934 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.933+0000 7f592f7fe700 1 -- 172.21.15.77:0/259309703 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22131 (secure 0 0 0) 0x7f5930098e00 con 0x7f59400ff060 2023-12-01T01:17:16.934 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:17:16.934 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":82,"fsid":"2a06e29a-8fe6-11ee-95a2-87774f69a715","created":"2023-12-01T01:09:30.210398+0000","modified":"2023-12-01T01:16:26.346791+0000","last_up_change":"2023-12-01T01:15:44.586092+0000","last_in_change":"2023-12-01T01:15:26.305439+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:12:30.514229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:16:23.568860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-01T01:16:24.531137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fc651e62-a103-4ab8-a76d-d3708ea6aeec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6803","nonce":2244799094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6805","nonce":2244799094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6809","nonce":2244799094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6807","nonce":2244799094}]},"public_addr":"172.21.15.77:6803/2244799094","cluster_addr":"172.21.15.77:6805/2244799094","heartbeat_back_addr":"172.21.15.77:6809/2244799094","heartbeat_front_addr":"172.21.15.77:6807/2244799094","state":["exists","up"]},{"osd":1,"uuid":"0adf8c54-49ec-49a0-8bf1-af1f20e60729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6801","nonce":224194890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6803","nonce":224194890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6807","nonce":224194890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6805","nonce":224194890}]},"public_addr":"172.21.15.137:6801/224194890","cluster_addr":"172.21.15.137:6803/224194890","heartbeat_back_addr":"172.21.15.137:6807/224194890","heartbeat_front_addr":"172.21.15.137:6805/224194890","state":["exists","up"]},{"osd":2,"uuid":"5a69802c-dede-4544-9a12-436f5c1c0038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6801","nonce":4078459110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6803","nonce":4078459110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6807","nonce":4078459110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6805","nonce":4078459110}]},"public_addr":"172.21.15.203:6801/4078459110","cluster_addr":"172.21.15.203:6803/4078459110","heartbeat_back_addr":"172.21.15.203:6807/4078459110","heartbeat_front_addr":"172.21.15.203:6805/4078459110","state":["exists","up"]},{"osd":3,"uuid":"ad325a97-580b-4c99-bf12-cbb0d92a6d0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6811","nonce":1454440477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6813","nonce":1454440477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6817","nonce":1454440477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6815","nonce":1454440477}]},"public_addr":"172.21.15.77:6811/1454440477","cluster_addr":"172.21.15.77:6813/1454440477","heartbeat_back_addr":"172.21.15.77:6817/1454440477","heartbeat_front_addr":"172.21.15.77:6815/1454440477","state":["exists","up"]},{"osd":4,"uuid":"1e3ac1ba-98bb-4b8b-8e26-50064c66bf2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6809","nonce":1103241451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6811","nonce":1103241451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6815","nonce":1103241451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6813","nonce":1103241451}]},"public_addr":"172.21.15.137:6809/1103241451","cluster_addr":"172.21.15.137:6811/1103241451","heartbeat_back_addr":"172.21.15.137:6815/1103241451","heartbeat_front_addr":"172.21.15.137:6813/1103241451","state":["exists","up"]},{"osd":5,"uuid":"eb32c016-abeb-44ab-abf0-19dd09235736","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6809","nonce":2648569482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6811","nonce":2648569482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6815","nonce":2648569482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6813","nonce":2648569482}]},"public_addr":"172.21.15.203:6809/2648569482","cluster_addr":"172.21.15.203:6811/2648569482","heartbeat_back_addr":"172.21.15.203:6815/2648569482","heartbeat_front_addr":"172.21.15.203:6813/2648569482","state":["exists","up"]},{"osd":6,"uuid":"0e3fa97e-2ca6-418c-9e8d-a6fe3c56adb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6819","nonce":2463414651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6821","nonce":2463414651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6825","nonce":2463414651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6823","nonce":2463414651}]},"public_addr":"172.21.15.77:6819/2463414651","cluster_addr":"172.21.15.77:6821/2463414651","heartbeat_back_addr":"172.21.15.77:6825/2463414651","heartbeat_front_addr":"172.21.15.77:6823/2463414651","state":["exists","up"]},{"osd":7,"uuid":"d7398c01-0f98-4138-8a49-c3ff0aae7e88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6817","nonce":3108330766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6819","nonce":3108330766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6823","nonce":3108330766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6821","nonce":3108330766}]},"public_addr":"172.21.15.137:6817/3108330766","cluster_addr":"172.21.15.137:6819/3108330766","heartbeat_back_addr":"172.21.15.137:6823/3108330766","heartbeat_front_addr":"172.21.15.137:6821/3108330766","state":["exists","up"]},{"osd":8,"uuid":"0c09050d-3e43-410c-8cd7-bcb891af56b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6817","nonce":1161158776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6819","nonce":1161158776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6823","nonce":1161158776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6821","nonce":1161158776}]},"public_addr":"172.21.15.203:6817/1161158776","cluster_addr":"172.21.15.203:6819/1161158776","heartbeat_back_addr":"172.21.15.203:6823/1161158776","heartbeat_front_addr":"172.21.15.203:6821/1161158776","state":["exists","up"]},{"osd":9,"uuid":"527ecb49-95ec-48bd-9872-3b07fae5c5ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6827","nonce":2409880685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6829","nonce":2409880685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6833","nonce":2409880685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6831","nonce":2409880685}]},"public_addr":"172.21.15.77:6827/2409880685","cluster_addr":"172.21.15.77:6829/2409880685","heartbeat_back_addr":"172.21.15.77:6833/2409880685","heartbeat_front_addr":"172.21.15.77:6831/2409880685","state":["exists","up"]},{"osd":10,"uuid":"1680bb3e-a3c8-427b-af68-e2214f32bed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6825","nonce":374920146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6827","nonce":374920146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6831","nonce":374920146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6829","nonce":374920146}]},"public_addr":"172.21.15.137:6825/374920146","cluster_addr":"172.21.15.137:6827/374920146","heartbeat_back_addr":"172.21.15.137:6831/374920146","heartbeat_front_addr":"172.21.15.137:6829/374920146","state":["exists","up"]},{"osd":11,"uuid":"8a533466-023d-4803-ae36-9a9f1715e49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6825","nonce":2161588962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6827","nonce":2161588962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6831","nonce":2161588962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6829","nonce":2161588962}]},"public_addr":"172.21.15.203:6825/2161588962","cluster_addr":"172.21.15.203:6827/2161588962","heartbeat_back_addr":"172.21.15.203:6831/2161588962","heartbeat_front_addr":"172.21.15.203:6829/2161588962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:11:45.420509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:06.210115+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:28.191307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:49.594116+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:11.261689+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:33.376541+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:54.968336+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:16.775310+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:38.733792+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:59.456699+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:15:20.924059+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:15:43.094773+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/2260035254":"2023-12-02T01:10:24.191835+0000","172.21.15.77:6800/1456621747":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/219838828":"2023-12-02T01:10:24.191835+0000","172.21.15.77:6801/1456621747":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/2937149266":"2023-12-02T01:10:02.232948+0000","172.21.15.77:0/807520018":"2023-12-02T01:10:02.232948+0000","172.21.15.77:6801/1661788026":"2023-12-02T01:10:02.232948+0000","172.21.15.77:6800/1661788026":"2023-12-02T01:10:02.232948+0000","172.21.15.77:0/3551282473":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/1202222558":"2023-12-02T01:10:02.232948+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:17:16.937 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.935+0000 7f59448de700 1 -- 172.21.15.77:0/259309703 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f59280a05f0 msgr2=0x7f59280a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:16.937 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.935+0000 7f59448de700 1 --2- 172.21.15.77:0/259309703 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f59280a05f0 0x7f59280a2ab0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f593400b5c0 tx=0x7f5934005fb0 comp rx=0 tx=0).stop 2023-12-01T01:17:16.937 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.935+0000 7f59448de700 1 -- 172.21.15.77:0/259309703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59400ff060 msgr2=0x7f5940196a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:16.937 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.935+0000 7f59448de700 1 --2- 172.21.15.77:0/259309703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59400ff060 0x7f5940196a30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f593000d980 tx=0x7f593000dc90 comp rx=0 tx=0).stop 2023-12-01T01:17:16.937 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.935+0000 7f59448de700 1 -- 172.21.15.77:0/259309703 shutdown_connections 2023-12-01T01:17:16.937 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.935+0000 7f59448de700 1 --2- 172.21.15.77:0/259309703 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f59280a05f0 0x7f59280a2ab0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:16.937 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.935+0000 7f59448de700 1 --2- 172.21.15.77:0/259309703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5940102a70 0x7f594019dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:16.937 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.935+0000 7f59448de700 1 --2- 172.21.15.77:0/259309703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59400ff060 0x7f5940196a30 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:16.937 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.935+0000 7f59448de700 1 --2- 172.21.15.77:0/259309703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59401020d0 0x7f5940196f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:16.937 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.935+0000 7f59448de700 1 -- 172.21.15.77:0/259309703 >> 172.21.15.77:0/259309703 conn(0x7f59400fa950 msgr2=0x7f59400fcc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:16.937 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.935+0000 7f59448de700 1 -- 172.21.15.77:0/259309703 shutdown_connections 2023-12-01T01:17:16.937 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:16.935+0000 7f59448de700 1 -- 172.21.15.77:0/259309703 wait complete. 2023-12-01T01:17:16.951 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:12:30.514229+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T01:16:23.568860+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T01:16:24.531137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:17:16.952 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:17:17.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:16 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/2362082880' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-01T01:17:17.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:16 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/184822007' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-01T01:17:17.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:16 smithi203 ceph-mon[103581]: pgmap v278: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-01T01:17:17.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:16 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/4246386090' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:17:17.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:16 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/2362082880' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-01T01:17:17.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:16 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/184822007' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-01T01:17:17.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:16 smithi137 ceph-mon[103312]: pgmap v278: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-01T01:17:17.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:16 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/4246386090' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:17:17.078 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.074+0000 7fa67b96c700 1 -- 172.21.15.77:0/2603422628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa674102910 msgr2=0x7fa6741091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:17.078 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.074+0000 7fa67b96c700 1 --2- 172.21.15.77:0/2603422628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa674102910 0x7fa6741091c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa66c00b0d0 tx=0x7fa66c00b3e0 comp rx=0 tx=0).stop 2023-12-01T01:17:17.078 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.074+0000 7fa67b96c700 1 -- 172.21.15.77:0/2603422628 shutdown_connections 2023-12-01T01:17:17.078 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.074+0000 7fa67b96c700 1 --2- 172.21.15.77:0/2603422628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa674102910 0x7fa6741091c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:17.078 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.074+0000 7fa67b96c700 1 --2- 172.21.15.77:0/2603422628 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa674100d80 0x7fa6741011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:17.079 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.074+0000 7fa67b96c700 1 --2- 172.21.15.77:0/2603422628 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa674101f70 0x7fa6741023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:17.079 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.074+0000 7fa67b96c700 1 -- 172.21.15.77:0/2603422628 >> 172.21.15.77:0/2603422628 conn(0x7fa6740fc2d0 msgr2=0x7fa6740fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:17.079 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.075+0000 7fa67b96c700 1 -- 172.21.15.77:0/2603422628 shutdown_connections 2023-12-01T01:17:17.079 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.075+0000 7fa67b96c700 1 -- 172.21.15.77:0/2603422628 wait complete. 2023-12-01T01:17:17.079 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.075+0000 7fa67b96c700 1 Processor -- start 2023-12-01T01:17:17.079 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.075+0000 7fa67b96c700 1 -- start start 2023-12-01T01:17:17.079 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.075+0000 7fa67b96c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa674100d80 0x7fa674196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:17.079 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.075+0000 7fa67b96c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa674101f70 0x7fa674197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:17.079 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.075+0000 7fa67b96c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa674102910 0x7fa67419e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:17.079 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.075+0000 7fa67b96c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67419e6c0 con 0x7fa674102910 2023-12-01T01:17:17.079 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.075+0000 7fa67b96c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67419e800 con 0x7fa674101f70 2023-12-01T01:17:17.079 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.075+0000 7fa67b96c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67419e990 con 0x7fa674100d80 2023-12-01T01:17:17.079 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.075+0000 7fa679f09700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa674102910 0x7fa67419e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:17.080 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.075+0000 7fa679f09700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa674102910 0x7fa67419e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58236/0 (socket says 172.21.15.77:58236) 2023-12-01T01:17:17.080 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.075+0000 7fa679f09700 1 -- 172.21.15.77:0/656486396 learned_addr learned my addr 172.21.15.77:0/656486396 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:17.080 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.076+0000 7fa679708700 1 --2- 172.21.15.77:0/656486396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa674100d80 0x7fa674196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:17.080 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.076+0000 7fa679f09700 1 -- 172.21.15.77:0/656486396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa674100d80 msgr2=0x7fa674196b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:17.080 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.076+0000 7fa679f09700 1 --2- 172.21.15.77:0/656486396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa674100d80 0x7fa674196b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:17.080 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.076+0000 7fa678f07700 1 --2- 172.21.15.77:0/656486396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa674101f70 0x7fa674197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:17.080 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.076+0000 7fa679f09700 1 -- 172.21.15.77:0/656486396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa674101f70 msgr2=0x7fa674197080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:17.080 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.076+0000 7fa679f09700 1 --2- 172.21.15.77:0/656486396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa674101f70 0x7fa674197080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:17.080 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.076+0000 7fa679f09700 1 -- 172.21.15.77:0/656486396 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa66c009d00 con 0x7fa674102910 2023-12-01T01:17:17.080 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.076+0000 7fa679f09700 1 --2- 172.21.15.77:0/656486396 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa674102910 0x7fa67419e0f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa66c007a00 tx=0x7fa66c0094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:17.080 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.076+0000 7fa66a7fc700 1 -- 172.21.15.77:0/656486396 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66c00e040 con 0x7fa674102910 2023-12-01T01:17:17.081 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.076+0000 7fa67b96c700 1 -- 172.21.15.77:0/656486396 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa67419ec10 con 0x7fa674102910 2023-12-01T01:17:17.081 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.076+0000 7fa66a7fc700 1 -- 172.21.15.77:0/656486396 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa66c003e10 con 0x7fa674102910 2023-12-01T01:17:17.081 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.076+0000 7fa66a7fc700 1 -- 172.21.15.77:0/656486396 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66c012a90 con 0x7fa674102910 2023-12-01T01:17:17.081 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.076+0000 7fa67b96c700 1 -- 172.21.15.77:0/656486396 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa67419f030 con 0x7fa674102910 2023-12-01T01:17:17.081 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.077+0000 7fa67b96c700 1 -- 172.21.15.77:0/656486396 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6741082d0 con 0x7fa674102910 2023-12-01T01:17:17.082 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.079+0000 7fa66a7fc700 1 -- 172.21.15.77:0/656486396 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fa66c0056c0 con 0x7fa674102910 2023-12-01T01:17:17.082 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.079+0000 7fa66a7fc700 1 --2- 172.21.15.77:0/656486396 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa6600a05b0 0x7fa6600a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:17.082 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.079+0000 7fa679708700 1 --2- 172.21.15.77:0/656486396 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa6600a05b0 0x7fa6600a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:17.082 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.079+0000 7fa66a7fc700 1 -- 172.21.15.77:0/656486396 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa66c0cd210 con 0x7fa674102910 2023-12-01T01:17:17.082 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.080+0000 7fa679708700 1 --2- 172.21.15.77:0/656486396 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa6600a05b0 0x7fa6600a2a70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa674101dd0 tx=0x7fa664007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:17.083 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.082+0000 7fa66a7fc700 1 -- 172.21.15.77:0/656486396 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa66c096e90 con 0x7fa674102910 2023-12-01T01:17:17.275 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.274+0000 7fa67b96c700 1 -- 172.21.15.77:0/656486396 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa67410c610 con 0x7fa674102910 2023-12-01T01:17:17.276 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.275+0000 7fa66a7fc700 1 -- 172.21.15.77:0/656486396 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa66c09a4b0 con 0x7fa674102910 2023-12-01T01:17:17.276 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 1 2023-12-01T01:17:17.278 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.276+0000 7fa67b96c700 1 -- 172.21.15.77:0/656486396 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa6600a05b0 msgr2=0x7fa6600a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:17.278 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.276+0000 7fa67b96c700 1 --2- 172.21.15.77:0/656486396 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa6600a05b0 0x7fa6600a2a70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa674101dd0 tx=0x7fa664007400 comp rx=0 tx=0).stop 2023-12-01T01:17:17.278 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.276+0000 7fa67b96c700 1 -- 172.21.15.77:0/656486396 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa674102910 msgr2=0x7fa67419e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:17.278 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.276+0000 7fa67b96c700 1 --2- 172.21.15.77:0/656486396 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa674102910 0x7fa67419e0f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa66c007a00 tx=0x7fa66c0094c0 comp rx=0 tx=0).stop 2023-12-01T01:17:17.278 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.277+0000 7fa67b96c700 1 -- 172.21.15.77:0/656486396 shutdown_connections 2023-12-01T01:17:17.278 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.277+0000 7fa67b96c700 1 --2- 172.21.15.77:0/656486396 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa6600a05b0 0x7fa6600a2a70 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:17.279 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.277+0000 7fa67b96c700 1 --2- 172.21.15.77:0/656486396 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa674102910 0x7fa67419e0f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:17.279 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.277+0000 7fa67b96c700 1 --2- 172.21.15.77:0/656486396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa674100d80 0x7fa674196b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:17.279 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.277+0000 7fa67b96c700 1 --2- 172.21.15.77:0/656486396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa674101f70 0x7fa674197080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:17.279 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.277+0000 7fa67b96c700 1 -- 172.21.15.77:0/656486396 >> 172.21.15.77:0/656486396 conn(0x7fa6740fc2d0 msgr2=0x7fa674105410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:17.279 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.277+0000 7fa67b96c700 1 -- 172.21.15.77:0/656486396 shutdown_connections 2023-12-01T01:17:17.279 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.277+0000 7fa67b96c700 1 -- 172.21.15.77:0/656486396 wait complete. 2023-12-01T01:17:17.290 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T01:17:17.416 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.413+0000 7fbb02c35700 1 -- 172.21.15.77:0/3219367873 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbafc106ac0 msgr2=0x7fbafc108ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:17.417 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.413+0000 7fbb02c35700 1 --2- 172.21.15.77:0/3219367873 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbafc106ac0 0x7fbafc108ed0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbaf400b0a0 tx=0x7fbaf400b3b0 comp rx=0 tx=0).stop 2023-12-01T01:17:17.417 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.413+0000 7fbb02c35700 1 -- 172.21.15.77:0/3219367873 shutdown_connections 2023-12-01T01:17:17.417 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.413+0000 7fbb02c35700 1 --2- 172.21.15.77:0/3219367873 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbafc106ac0 0x7fbafc108ed0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:17.417 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.413+0000 7fbb02c35700 1 --2- 172.21.15.77:0/3219367873 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbafc101860 0x7fbafc103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:17.417 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.413+0000 7fbb02c35700 1 --2- 172.21.15.77:0/3219367873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbafc104190 0x7fbafc106580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:17.417 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.413+0000 7fbb02c35700 1 -- 172.21.15.77:0/3219367873 >> 172.21.15.77:0/3219367873 conn(0x7fbafc0fb1f0 msgr2=0x7fbafc0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:17.417 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.413+0000 7fbb02c35700 1 -- 172.21.15.77:0/3219367873 shutdown_connections 2023-12-01T01:17:17.417 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.413+0000 7fbb02c35700 1 -- 172.21.15.77:0/3219367873 wait complete. 2023-12-01T01:17:17.417 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.414+0000 7fbb02c35700 1 Processor -- start 2023-12-01T01:17:17.417 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.414+0000 7fbb02c35700 1 -- start start 2023-12-01T01:17:17.418 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.414+0000 7fbb02c35700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbafc101860 0x7fbafc196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:17.418 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.414+0000 7fbb02c35700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbafc104190 0x7fbafc197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:17.418 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.414+0000 7fbb02c35700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbafc106ac0 0x7fbafc19e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:17.418 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.414+0000 7fbb02c35700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc19e6b0 con 0x7fbafc104190 2023-12-01T01:17:17.418 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.414+0000 7fbb02c35700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc19e7f0 con 0x7fbafc101860 2023-12-01T01:17:17.418 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.414+0000 7fbb02c35700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc19e980 con 0x7fbafc106ac0 2023-12-01T01:17:17.418 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.414+0000 7fbafbfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbafc104190 0x7fbafc197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:17.418 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.415+0000 7fbafbfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbafc104190 0x7fbafc197070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58248/0 (socket says 172.21.15.77:58248) 2023-12-01T01:17:17.418 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.415+0000 7fbafbfff700 1 -- 172.21.15.77:0/110083528 learned_addr learned my addr 172.21.15.77:0/110083528 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:17.418 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.415+0000 7fbafbfff700 1 -- 172.21.15.77:0/110083528 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbafc106ac0 msgr2=0x7fbafc19e0e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-01T01:17:17.418 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.415+0000 7fbafbfff700 1 --2- 172.21.15.77:0/110083528 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbafc106ac0 0x7fbafc19e0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:17.418 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.415+0000 7fbafbfff700 1 -- 172.21.15.77:0/110083528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbafc101860 msgr2=0x7fbafc196b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:17.418 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.415+0000 7fbafbfff700 1 --2- 172.21.15.77:0/110083528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbafc101860 0x7fbafc196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:17.419 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.415+0000 7fbafbfff700 1 -- 172.21.15.77:0/110083528 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf4009d00 con 0x7fbafc104190 2023-12-01T01:17:17.419 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.415+0000 7fbafbfff700 1 --2- 172.21.15.77:0/110083528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbafc104190 0x7fbafc197070 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbae800c9b0 tx=0x7fbae800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:17.419 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.415+0000 7fbaf9ffb700 1 -- 172.21.15.77:0/110083528 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae8007b00 con 0x7fbafc104190 2023-12-01T01:17:17.419 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.415+0000 7fbaf9ffb700 1 -- 172.21.15.77:0/110083528 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbae8007c60 con 0x7fbafc104190 2023-12-01T01:17:17.419 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.415+0000 7fbb02c35700 1 -- 172.21.15.77:0/110083528 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbafc19ec60 con 0x7fbafc104190 2023-12-01T01:17:17.419 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.415+0000 7fbaf9ffb700 1 -- 172.21.15.77:0/110083528 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae8018660 con 0x7fbafc104190 2023-12-01T01:17:17.419 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.415+0000 7fbb02c35700 1 -- 172.21.15.77:0/110083528 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbafc19f170 con 0x7fbafc104190 2023-12-01T01:17:17.420 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.416+0000 7fbb02c35700 1 -- 172.21.15.77:0/110083528 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbafc0fcd70 con 0x7fbafc104190 2023-12-01T01:17:17.420 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.417+0000 7fbaf9ffb700 1 -- 172.21.15.77:0/110083528 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fbae8018880 con 0x7fbafc104190 2023-12-01T01:17:17.420 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.418+0000 7fbaf9ffb700 1 --2- 172.21.15.77:0/110083528 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fbaec0a05b0 0x7fbaec0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:17.420 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.418+0000 7fbb009d1700 1 --2- 172.21.15.77:0/110083528 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fbaec0a05b0 0x7fbaec0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:17.420 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.418+0000 7fbaf9ffb700 1 -- 172.21.15.77:0/110083528 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fbae80cbaf0 con 0x7fbafc104190 2023-12-01T01:17:17.420 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.418+0000 7fbb009d1700 1 --2- 172.21.15.77:0/110083528 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fbaec0a05b0 0x7fbaec0a2a70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0005950 tx=0x7fbaf00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:17.423 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.422+0000 7fbaf9ffb700 1 -- 172.21.15.77:0/110083528 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fbae80955e0 con 0x7fbafc104190 2023-12-01T01:17:17.616 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.615+0000 7fbb02c35700 1 -- 172.21.15.77:0/110083528 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbafc02cef0 con 0x7fbafc104190 2023-12-01T01:17:17.617 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.615+0000 7fbaf9ffb700 1 -- 172.21.15.77:0/110083528 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbae8098c00 con 0x7fbafc104190 2023-12-01T01:17:17.617 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 32 2023-12-01T01:17:17.619 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.618+0000 7fbb02c35700 1 -- 172.21.15.77:0/110083528 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fbaec0a05b0 msgr2=0x7fbaec0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:17.619 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.618+0000 7fbb02c35700 1 --2- 172.21.15.77:0/110083528 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fbaec0a05b0 0x7fbaec0a2a70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0005950 tx=0x7fbaf00058e0 comp rx=0 tx=0).stop 2023-12-01T01:17:17.620 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.618+0000 7fbb02c35700 1 -- 172.21.15.77:0/110083528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbafc104190 msgr2=0x7fbafc197070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:17.620 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.618+0000 7fbb02c35700 1 --2- 172.21.15.77:0/110083528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbafc104190 0x7fbafc197070 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbae800c9b0 tx=0x7fbae800cd70 comp rx=0 tx=0).stop 2023-12-01T01:17:17.620 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.618+0000 7fbb02c35700 1 -- 172.21.15.77:0/110083528 shutdown_connections 2023-12-01T01:17:17.620 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.618+0000 7fbb02c35700 1 --2- 172.21.15.77:0/110083528 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fbaec0a05b0 0x7fbaec0a2a70 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:17.620 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.618+0000 7fbb02c35700 1 --2- 172.21.15.77:0/110083528 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbafc106ac0 0x7fbafc19e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:17.620 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.618+0000 7fbb02c35700 1 --2- 172.21.15.77:0/110083528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbafc101860 0x7fbafc196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:17.620 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.618+0000 7fbb02c35700 1 --2- 172.21.15.77:0/110083528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbafc104190 0x7fbafc197070 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:17.620 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.618+0000 7fbb02c35700 1 -- 172.21.15.77:0/110083528 >> 172.21.15.77:0/110083528 conn(0x7fbafc0fb1f0 msgr2=0x7fbafc1024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:17.620 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.618+0000 7fbb02c35700 1 -- 172.21.15.77:0/110083528 shutdown_connections 2023-12-01T01:17:17.620 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.618+0000 7fbb02c35700 1 -- 172.21.15.77:0/110083528 wait complete. 2023-12-01T01:17:17.631 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T01:17:17.755 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.754+0000 7f867a787700 1 -- 172.21.15.77:0/2884227311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86740ffc80 msgr2=0x7f86741000a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:17.756 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.754+0000 7f867a787700 1 --2- 172.21.15.77:0/2884227311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86740ffc80 0x7f86741000a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f86640099c0 tx=0x7f8664009cd0 comp rx=0 tx=0).stop 2023-12-01T01:17:17.756 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.754+0000 7f867a787700 1 -- 172.21.15.77:0/2884227311 shutdown_connections 2023-12-01T01:17:17.756 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.754+0000 7f867a787700 1 --2- 172.21.15.77:0/2884227311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8674101810 0x7f86741080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:17.756 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.754+0000 7f867a787700 1 --2- 172.21.15.77:0/2884227311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86740ffc80 0x7f86741000a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:17.756 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.754+0000 7f867a787700 1 --2- 172.21.15.77:0/2884227311 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8674100e70 0x7f86741012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:17.756 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.754+0000 7f867a787700 1 -- 172.21.15.77:0/2884227311 >> 172.21.15.77:0/2884227311 conn(0x7f86740fb1f0 msgr2=0x7f86740fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:17.756 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.754+0000 7f867a787700 1 -- 172.21.15.77:0/2884227311 shutdown_connections 2023-12-01T01:17:17.756 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.754+0000 7f867a787700 1 -- 172.21.15.77:0/2884227311 wait complete. 2023-12-01T01:17:17.756 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.754+0000 7f867a787700 1 Processor -- start 2023-12-01T01:17:17.756 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.754+0000 7f867a787700 1 -- start start 2023-12-01T01:17:17.758 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.755+0000 7f867a787700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86740ffc80 0x7f8674068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:17.758 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.755+0000 7f867a787700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8674100e70 0x7f867419b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:17.758 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.755+0000 7f867a787700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8674101810 0x7f867419de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:17.758 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.755+0000 7f867a787700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867419e630 con 0x7f86740ffc80 2023-12-01T01:17:17.758 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.755+0000 7f867a787700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867419e7a0 con 0x7f8674100e70 2023-12-01T01:17:17.759 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.755+0000 7f867a787700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867419e960 con 0x7f8674101810 2023-12-01T01:17:17.759 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.755+0000 7f8673fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86740ffc80 0x7f8674068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:17.759 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.755+0000 7f8673fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86740ffc80 0x7f8674068a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58260/0 (socket says 172.21.15.77:58260) 2023-12-01T01:17:17.759 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.755+0000 7f8673fff700 1 -- 172.21.15.77:0/2267347781 learned_addr learned my addr 172.21.15.77:0/2267347781 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:17.759 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.755+0000 7f86737fe700 1 --2- 172.21.15.77:0/2267347781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8674100e70 0x7f867419b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:17.759 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.755+0000 7f8673fff700 1 -- 172.21.15.77:0/2267347781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8674101810 msgr2=0x7f867419de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:17.759 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.755+0000 7f8673fff700 1 --2- 172.21.15.77:0/2267347781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8674101810 0x7f867419de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:17.759 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.755+0000 7f8673fff700 1 -- 172.21.15.77:0/2267347781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8674100e70 msgr2=0x7f867419b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:17.759 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.755+0000 7f8673fff700 1 --2- 172.21.15.77:0/2267347781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8674100e70 0x7f867419b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:17.759 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.755+0000 7f8673fff700 1 -- 172.21.15.77:0/2267347781 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86640095f0 con 0x7f86740ffc80 2023-12-01T01:17:17.759 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.755+0000 7f86737fe700 1 --2- 172.21.15.77:0/2267347781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8674100e70 0x7f867419b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:17:17.759 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.755+0000 7f8673fff700 1 --2- 172.21.15.77:0/2267347781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86740ffc80 0x7f8674068a80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f86640036f0 tx=0x7f86640038c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:17.759 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.755+0000 7f86717fa700 1 -- 172.21.15.77:0/2267347781 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866401d070 con 0x7f86740ffc80 2023-12-01T01:17:17.760 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.756+0000 7f86717fa700 1 -- 172.21.15.77:0/2267347781 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8664022470 con 0x7f86740ffc80 2023-12-01T01:17:17.760 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.756+0000 7f867a787700 1 -- 172.21.15.77:0/2267347781 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f867419ebe0 con 0x7f86740ffc80 2023-12-01T01:17:17.760 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.756+0000 7f86717fa700 1 -- 172.21.15.77:0/2267347781 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866400f6c0 con 0x7f86740ffc80 2023-12-01T01:17:17.760 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.756+0000 7f867a787700 1 -- 172.21.15.77:0/2267347781 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f867419f000 con 0x7f86740ffc80 2023-12-01T01:17:17.760 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.756+0000 7f867a787700 1 -- 172.21.15.77:0/2267347781 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f867410b550 con 0x7f86740ffc80 2023-12-01T01:17:17.760 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.758+0000 7f86717fa700 1 -- 172.21.15.77:0/2267347781 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f86640225e0 con 0x7f86740ffc80 2023-12-01T01:17:17.761 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.758+0000 7f86717fa700 1 --2- 172.21.15.77:0/2267347781 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f865c0a05b0 0x7f865c0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:17.761 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.758+0000 7f86717fa700 1 -- 172.21.15.77:0/2267347781 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f86640cd680 con 0x7f86740ffc80 2023-12-01T01:17:17.761 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.758+0000 7f86737fe700 1 --2- 172.21.15.77:0/2267347781 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f865c0a05b0 0x7f865c0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:17.761 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.759+0000 7f86737fe700 1 --2- 172.21.15.77:0/2267347781 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f865c0a05b0 0x7f865c0a2a70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f86740687f0 tx=0x7f8668005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:17.762 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.761+0000 7f86717fa700 1 -- 172.21.15.77:0/2267347781 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8664097300 con 0x7f86740ffc80 2023-12-01T01:17:17.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:17 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/259309703' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:17:17.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:17 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/656486396' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-01T01:17:17.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:17 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/110083528' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-01T01:17:17.954 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.953+0000 7f867a787700 1 -- 172.21.15.77:0/2267347781 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8674106bf0 con 0x7f86740ffc80 2023-12-01T01:17:17.954 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.953+0000 7f86717fa700 1 -- 172.21.15.77:0/2267347781 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7f866409a920 con 0x7f86740ffc80 2023-12-01T01:17:17.954 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 64 2023-12-01T01:17:17.957 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.955+0000 7f867a787700 1 -- 172.21.15.77:0/2267347781 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f865c0a05b0 msgr2=0x7f865c0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:17.957 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.955+0000 7f867a787700 1 --2- 172.21.15.77:0/2267347781 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f865c0a05b0 0x7f865c0a2a70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f86740687f0 tx=0x7f8668005f50 comp rx=0 tx=0).stop 2023-12-01T01:17:17.957 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.955+0000 7f867a787700 1 -- 172.21.15.77:0/2267347781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86740ffc80 msgr2=0x7f8674068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:17.957 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.955+0000 7f867a787700 1 --2- 172.21.15.77:0/2267347781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86740ffc80 0x7f8674068a80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f86640036f0 tx=0x7f86640038c0 comp rx=0 tx=0).stop 2023-12-01T01:17:17.957 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.955+0000 7f867a787700 1 -- 172.21.15.77:0/2267347781 shutdown_connections 2023-12-01T01:17:17.957 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.955+0000 7f867a787700 1 --2- 172.21.15.77:0/2267347781 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f865c0a05b0 0x7f865c0a2a70 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:17.957 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.955+0000 7f867a787700 1 --2- 172.21.15.77:0/2267347781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8674101810 0x7f867419de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:17.958 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.955+0000 7f867a787700 1 --2- 172.21.15.77:0/2267347781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86740ffc80 0x7f8674068a80 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:17.958 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.955+0000 7f867a787700 1 --2- 172.21.15.77:0/2267347781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8674100e70 0x7f867419b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:17.958 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.955+0000 7f867a787700 1 -- 172.21.15.77:0/2267347781 >> 172.21.15.77:0/2267347781 conn(0x7f86740fb1f0 msgr2=0x7f86740fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:17.958 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.955+0000 7f867a787700 1 -- 172.21.15.77:0/2267347781 shutdown_connections 2023-12-01T01:17:17.958 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:17.955+0000 7f867a787700 1 -- 172.21.15.77:0/2267347781 wait complete. 2023-12-01T01:17:17.969 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:17:18.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:17 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/259309703' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:17:18.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:17 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/656486396' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-01T01:17:18.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:17 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/110083528' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-01T01:17:18.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:17 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/259309703' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:17:18.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:17 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/656486396' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-01T01:17:18.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:17 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/110083528' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-01T01:17:18.093 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.089+0000 7f8f2b528700 1 -- 172.21.15.77:0/4186201308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f24074dd0 msgr2=0x7f8f24073220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:18.093 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.089+0000 7f8f2b528700 1 --2- 172.21.15.77:0/4186201308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f24074dd0 0x7f8f24073220 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8f20009990 tx=0x7f8f20009ca0 comp rx=0 tx=0).stop 2023-12-01T01:17:18.093 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.089+0000 7f8f2b528700 1 -- 172.21.15.77:0/4186201308 shutdown_connections 2023-12-01T01:17:18.093 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.089+0000 7f8f2b528700 1 --2- 172.21.15.77:0/4186201308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f2410d390 0x7f8f2410f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:18.093 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.089+0000 7f8f2b528700 1 --2- 172.21.15.77:0/4186201308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f24074dd0 0x7f8f24073220 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:18.093 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.089+0000 7f8f2b528700 1 --2- 172.21.15.77:0/4186201308 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f240737f0 0x7f8f24073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:18.093 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.089+0000 7f8f2b528700 1 -- 172.21.15.77:0/4186201308 >> 172.21.15.77:0/4186201308 conn(0x7f8f240fc310 msgr2=0x7f8f240fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:18.093 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.090+0000 7f8f2b528700 1 -- 172.21.15.77:0/4186201308 shutdown_connections 2023-12-01T01:17:18.093 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.090+0000 7f8f2b528700 1 -- 172.21.15.77:0/4186201308 wait complete. 2023-12-01T01:17:18.093 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.090+0000 7f8f2b528700 1 Processor -- start 2023-12-01T01:17:18.093 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.090+0000 7f8f2b528700 1 -- start start 2023-12-01T01:17:18.094 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.090+0000 7f8f2b528700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f240737f0 0x7f8f2419d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:18.094 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.090+0000 7f8f2b528700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f24074dd0 0x7f8f2419d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:18.094 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.090+0000 7f8f2b528700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f2410d390 0x7f8f241a45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:18.094 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.090+0000 7f8f2b528700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f241a4bb0 con 0x7f8f2410d390 2023-12-01T01:17:18.094 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.090+0000 7f8f2b528700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f241a4cf0 con 0x7f8f240737f0 2023-12-01T01:17:18.094 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.090+0000 7f8f2b528700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f241a4e80 con 0x7f8f24074dd0 2023-12-01T01:17:18.094 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.090+0000 7f8f29ac5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f2410d390 0x7f8f241a45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:18.094 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.090+0000 7f8f29ac5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f2410d390 0x7f8f241a45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58278/0 (socket says 172.21.15.77:58278) 2023-12-01T01:17:18.094 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.090+0000 7f8f29ac5700 1 -- 172.21.15.77:0/3696168281 learned_addr learned my addr 172.21.15.77:0/3696168281 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:18.094 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.090+0000 7f8f292c4700 1 --2- 172.21.15.77:0/3696168281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f240737f0 0x7f8f2419d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:18.094 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.090+0000 7f8f28ac3700 1 --2- 172.21.15.77:0/3696168281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f24074dd0 0x7f8f2419d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:18.094 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.091+0000 7f8f29ac5700 1 -- 172.21.15.77:0/3696168281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f24074dd0 msgr2=0x7f8f2419d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:18.094 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.091+0000 7f8f29ac5700 1 --2- 172.21.15.77:0/3696168281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f24074dd0 0x7f8f2419d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:18.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.091+0000 7f8f29ac5700 1 -- 172.21.15.77:0/3696168281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f240737f0 msgr2=0x7f8f2419d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:18.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.091+0000 7f8f29ac5700 1 --2- 172.21.15.77:0/3696168281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f240737f0 0x7f8f2419d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:18.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.091+0000 7f8f29ac5700 1 -- 172.21.15.77:0/3696168281 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f200095f0 con 0x7f8f2410d390 2023-12-01T01:17:18.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.091+0000 7f8f29ac5700 1 --2- 172.21.15.77:0/3696168281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f2410d390 0x7f8f241a45e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c007ee0 tx=0x7f8f1c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:18.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.091+0000 7f8f1a7fc700 1 -- 172.21.15.77:0/3696168281 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c017070 con 0x7f8f2410d390 2023-12-01T01:17:18.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.091+0000 7f8f2b528700 1 -- 172.21.15.77:0/3696168281 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f241a5160 con 0x7f8f2410d390 2023-12-01T01:17:18.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.091+0000 7f8f1a7fc700 1 -- 172.21.15.77:0/3696168281 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f1c00f040 con 0x7f8f2410d390 2023-12-01T01:17:18.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.091+0000 7f8f1a7fc700 1 -- 172.21.15.77:0/3696168281 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c005580 con 0x7f8f2410d390 2023-12-01T01:17:18.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.091+0000 7f8f2b528700 1 -- 172.21.15.77:0/3696168281 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f241a5620 con 0x7f8f2410d390 2023-12-01T01:17:18.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.092+0000 7f8f2b528700 1 -- 172.21.15.77:0/3696168281 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f24102ee0 con 0x7f8f2410d390 2023-12-01T01:17:18.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.093+0000 7f8f1a7fc700 1 -- 172.21.15.77:0/3696168281 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f8f1c00db20 con 0x7f8f2410d390 2023-12-01T01:17:18.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.094+0000 7f8f1a7fc700 1 --2- 172.21.15.77:0/3696168281 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8f100a05f0 0x7f8f100a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:18.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.094+0000 7f8f292c4700 1 --2- 172.21.15.77:0/3696168281 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8f100a05f0 0x7f8f100a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:18.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.094+0000 7f8f1a7fc700 1 -- 172.21.15.77:0/3696168281 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8f1c0cb7f0 con 0x7f8f2410d390 2023-12-01T01:17:18.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.094+0000 7f8f292c4700 1 --2- 172.21.15.77:0/3696168281 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8f100a05f0 0x7f8f100a2ab0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8f20006010 tx=0x7f8f20003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:18.098 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.097+0000 7f8f1a7fc700 1 -- 172.21.15.77:0/3696168281 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8f1c0952e0 con 0x7f8f2410d390 2023-12-01T01:17:18.332 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.331+0000 7f8f2b528700 1 -- 172.21.15.77:0/3696168281 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f2402cc70 con 0x7f8f2410d390 2023-12-01T01:17:18.333 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.332+0000 7f8f1a7fc700 1 -- 172.21.15.77:0/3696168281 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10838 (secure 0 0 0) 0x7f8f1c098900 con 0x7f8f2410d390 2023-12-01T01:17:18.333 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:17:18.333 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1435145620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1435145620},{"type":"v1","addr":"172.21.15.77:6835","nonce":1435145620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/841838908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":841838908},{"type":"v1","addr":"172.21.15.77:6837","nonce":841838908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-01T01:16:25.345276+0000","modified":"2023-12-01T01:16:32.596617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24496,"mds_2":24475,"mds_3":24473,"mds_4":14649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/3726913446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3726913446},{"type":"v1","addr":"172.21.15.77:6839","nonce":3726913446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14649":{"gid":14649,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":5,"addr":"172.21.15.77:6841/3023184965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3023184965},{"type":"v1","addr":"172.21.15.77:6841","nonce":3023184965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/2249638445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2249638445},{"type":"v1","addr":"172.21.15.137:6833","nonce":2249638445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6833/1201599774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1201599774},{"type":"v1","addr":"172.21.15.203:6833","nonce":1201599774}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3935354878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3935354878},{"type":"v1","addr":"172.21.15.137:6835","nonce":3935354878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3237538176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3237538176},{"type":"v1","addr":"172.21.15.203:6835","nonce":3237538176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3518881867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3518881867},{"type":"v1","addr":"172.21.15.137:6837","nonce":3518881867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.203:6837/2887080977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2887080977},{"type":"v1","addr":"172.21.15.203:6837","nonce":2887080977}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.203:6839/2888221613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2888221613},{"type":"v1","addr":"172.21.15.203:6839","nonce":2888221613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/1099966660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1099966660},{"type":"v1","addr":"172.21.15.137:6839","nonce":1099966660}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-01T01:17:18.335 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.334+0000 7f8f2b528700 1 -- 172.21.15.77:0/3696168281 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8f100a05f0 msgr2=0x7f8f100a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:18.335 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.334+0000 7f8f2b528700 1 --2- 172.21.15.77:0/3696168281 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8f100a05f0 0x7f8f100a2ab0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8f20006010 tx=0x7f8f20003680 comp rx=0 tx=0).stop 2023-12-01T01:17:18.335 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.334+0000 7f8f2b528700 1 -- 172.21.15.77:0/3696168281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f2410d390 msgr2=0x7f8f241a45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:18.336 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.334+0000 7f8f2b528700 1 --2- 172.21.15.77:0/3696168281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f2410d390 0x7f8f241a45e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c007ee0 tx=0x7f8f1c00d3b0 comp rx=0 tx=0).stop 2023-12-01T01:17:18.336 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.334+0000 7f8f2b528700 1 -- 172.21.15.77:0/3696168281 shutdown_connections 2023-12-01T01:17:18.336 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.334+0000 7f8f2b528700 1 --2- 172.21.15.77:0/3696168281 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8f100a05f0 0x7f8f100a2ab0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:18.336 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.334+0000 7f8f2b528700 1 --2- 172.21.15.77:0/3696168281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f2410d390 0x7f8f241a45e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:18.336 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.334+0000 7f8f2b528700 1 --2- 172.21.15.77:0/3696168281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f240737f0 0x7f8f2419d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:18.336 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.334+0000 7f8f2b528700 1 --2- 172.21.15.77:0/3696168281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f24074dd0 0x7f8f2419d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:18.336 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.334+0000 7f8f2b528700 1 -- 172.21.15.77:0/3696168281 >> 172.21.15.77:0/3696168281 conn(0x7f8f240fc310 msgr2=0x7f8f240fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:18.336 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.334+0000 7f8f2b528700 1 -- 172.21.15.77:0/3696168281 shutdown_connections 2023-12-01T01:17:18.336 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.334+0000 7f8f2b528700 1 -- 172.21.15.77:0/3696168281 wait complete. 2023-12-01T01:17:18.336 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 27 2023-12-01T01:17:18.348 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:17:18.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.467+0000 7f565dd2a700 1 -- 172.21.15.77:0/2315155594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5658102010 msgr2=0x7f5658102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:18.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.467+0000 7f565dd2a700 1 --2- 172.21.15.77:0/2315155594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5658102010 0x7f5658102470 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5648009990 tx=0x7f5648009ca0 comp rx=0 tx=0).stop 2023-12-01T01:17:18.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.467+0000 7f565dd2a700 1 -- 172.21.15.77:0/2315155594 shutdown_connections 2023-12-01T01:17:18.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.467+0000 7f565dd2a700 1 --2- 172.21.15.77:0/2315155594 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56581029b0 0x7f5658109260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:18.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.467+0000 7f565dd2a700 1 --2- 172.21.15.77:0/2315155594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56580ff040 0x7f56580ff460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:18.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.467+0000 7f565dd2a700 1 --2- 172.21.15.77:0/2315155594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5658102010 0x7f5658102470 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:18.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.467+0000 7f565dd2a700 1 -- 172.21.15.77:0/2315155594 >> 172.21.15.77:0/2315155594 conn(0x7f56580fa950 msgr2=0x7f56580fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:18.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.467+0000 7f565dd2a700 1 -- 172.21.15.77:0/2315155594 shutdown_connections 2023-12-01T01:17:18.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.467+0000 7f565dd2a700 1 -- 172.21.15.77:0/2315155594 wait complete. 2023-12-01T01:17:18.472 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.468+0000 7f565dd2a700 1 Processor -- start 2023-12-01T01:17:18.472 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.468+0000 7f565dd2a700 1 -- start start 2023-12-01T01:17:18.472 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.468+0000 7f565dd2a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56580ff040 0x7f5658196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:18.472 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.468+0000 7f565dd2a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5658102010 0x7f5658196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:18.472 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.468+0000 7f565dd2a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56581029b0 0x7f565819dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:18.472 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.468+0000 7f565dd2a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565819e5c0 con 0x7f5658102010 2023-12-01T01:17:18.472 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.468+0000 7f565dd2a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565819e700 con 0x7f56581029b0 2023-12-01T01:17:18.472 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.468+0000 7f565dd2a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565819e890 con 0x7f56580ff040 2023-12-01T01:17:18.472 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.468+0000 7f5656ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5658102010 0x7f5658196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:18.472 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.468+0000 7f5656ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5658102010 0x7f5658196f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58290/0 (socket says 172.21.15.77:58290) 2023-12-01T01:17:18.472 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.468+0000 7f5656ffd700 1 -- 172.21.15.77:0/2629260852 learned_addr learned my addr 172.21.15.77:0/2629260852 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:18.472 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.468+0000 7f5657fff700 1 --2- 172.21.15.77:0/2629260852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56581029b0 0x7f565819dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:18.472 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.468+0000 7f5656ffd700 1 -- 172.21.15.77:0/2629260852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56580ff040 msgr2=0x7f5658196a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:18.473 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.468+0000 7f56577fe700 1 --2- 172.21.15.77:0/2629260852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56580ff040 0x7f5658196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:18.473 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.468+0000 7f5656ffd700 1 --2- 172.21.15.77:0/2629260852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56580ff040 0x7f5658196a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:18.473 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.468+0000 7f5656ffd700 1 -- 172.21.15.77:0/2629260852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56581029b0 msgr2=0x7f565819dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:18.473 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.468+0000 7f5656ffd700 1 --2- 172.21.15.77:0/2629260852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56581029b0 0x7f565819dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:18.473 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.468+0000 7f5656ffd700 1 -- 172.21.15.77:0/2629260852 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56480095f0 con 0x7f5658102010 2023-12-01T01:17:18.473 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.468+0000 7f5657fff700 1 --2- 172.21.15.77:0/2629260852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56581029b0 0x7f565819dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:17:18.473 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.468+0000 7f56577fe700 1 --2- 172.21.15.77:0/2629260852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56580ff040 0x7f5658196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:17:18.473 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.469+0000 7f5656ffd700 1 --2- 172.21.15.77:0/2629260852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5658102010 0x7f5658196f80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5648000c00 tx=0x7f564800f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:18.473 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.469+0000 7f5654ff9700 1 -- 172.21.15.77:0/2629260852 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564801d070 con 0x7f5658102010 2023-12-01T01:17:18.473 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.469+0000 7f565dd2a700 1 -- 172.21.15.77:0/2629260852 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f565819eb10 con 0x7f5658102010 2023-12-01T01:17:18.473 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.469+0000 7f5654ff9700 1 -- 172.21.15.77:0/2629260852 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f564800fc00 con 0x7f5658102010 2023-12-01T01:17:18.473 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.469+0000 7f5654ff9700 1 -- 172.21.15.77:0/2629260852 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5648017650 con 0x7f5658102010 2023-12-01T01:17:18.473 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.469+0000 7f565dd2a700 1 -- 172.21.15.77:0/2629260852 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f565819efc0 con 0x7f5658102010 2023-12-01T01:17:18.473 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.470+0000 7f565dd2a700 1 -- 172.21.15.77:0/2629260852 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f565810c600 con 0x7f5658102010 2023-12-01T01:17:18.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.471+0000 7f5654ff9700 1 -- 172.21.15.77:0/2629260852 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f5648022470 con 0x7f5658102010 2023-12-01T01:17:18.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.471+0000 7f5654ff9700 1 --2- 172.21.15.77:0/2629260852 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f56400a05f0 0x7f56400a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:18.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.472+0000 7f56577fe700 1 --2- 172.21.15.77:0/2629260852 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f56400a05f0 0x7f56400a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:18.475 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.472+0000 7f5654ff9700 1 -- 172.21.15.77:0/2629260852 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f56480ccf50 con 0x7f5658102010 2023-12-01T01:17:18.475 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.472+0000 7f56577fe700 1 --2- 172.21.15.77:0/2629260852 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f56400a05f0 0x7f56400a2ab0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f564c005fd0 tx=0x7f564c005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:18.476 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.475+0000 7f5654ff9700 1 -- 172.21.15.77:0/2629260852 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5648096bd0 con 0x7f5658102010 2023-12-01T01:17:18.671 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.670+0000 7f565dd2a700 1 -- 172.21.15.77:0/2629260852 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5658107cb0 con 0x7f5658102010 2023-12-01T01:17:18.673 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.672+0000 7f5654ff9700 1 -- 172.21.15.77:0/2629260852 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22131 (secure 0 0 0) 0x7f564809a1f0 con 0x7f5658102010 2023-12-01T01:17:18.673 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:17:18.673 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":82,"fsid":"2a06e29a-8fe6-11ee-95a2-87774f69a715","created":"2023-12-01T01:09:30.210398+0000","modified":"2023-12-01T01:16:26.346791+0000","last_up_change":"2023-12-01T01:15:44.586092+0000","last_in_change":"2023-12-01T01:15:26.305439+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:12:30.514229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:16:23.568860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-01T01:16:24.531137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fc651e62-a103-4ab8-a76d-d3708ea6aeec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6803","nonce":2244799094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6805","nonce":2244799094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6809","nonce":2244799094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6807","nonce":2244799094}]},"public_addr":"172.21.15.77:6803/2244799094","cluster_addr":"172.21.15.77:6805/2244799094","heartbeat_back_addr":"172.21.15.77:6809/2244799094","heartbeat_front_addr":"172.21.15.77:6807/2244799094","state":["exists","up"]},{"osd":1,"uuid":"0adf8c54-49ec-49a0-8bf1-af1f20e60729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6801","nonce":224194890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6803","nonce":224194890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6807","nonce":224194890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6805","nonce":224194890}]},"public_addr":"172.21.15.137:6801/224194890","cluster_addr":"172.21.15.137:6803/224194890","heartbeat_back_addr":"172.21.15.137:6807/224194890","heartbeat_front_addr":"172.21.15.137:6805/224194890","state":["exists","up"]},{"osd":2,"uuid":"5a69802c-dede-4544-9a12-436f5c1c0038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6801","nonce":4078459110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6803","nonce":4078459110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6807","nonce":4078459110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6805","nonce":4078459110}]},"public_addr":"172.21.15.203:6801/4078459110","cluster_addr":"172.21.15.203:6803/4078459110","heartbeat_back_addr":"172.21.15.203:6807/4078459110","heartbeat_front_addr":"172.21.15.203:6805/4078459110","state":["exists","up"]},{"osd":3,"uuid":"ad325a97-580b-4c99-bf12-cbb0d92a6d0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6811","nonce":1454440477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6813","nonce":1454440477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6817","nonce":1454440477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6815","nonce":1454440477}]},"public_addr":"172.21.15.77:6811/1454440477","cluster_addr":"172.21.15.77:6813/1454440477","heartbeat_back_addr":"172.21.15.77:6817/1454440477","heartbeat_front_addr":"172.21.15.77:6815/1454440477","state":["exists","up"]},{"osd":4,"uuid":"1e3ac1ba-98bb-4b8b-8e26-50064c66bf2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6809","nonce":1103241451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6811","nonce":1103241451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6815","nonce":1103241451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6813","nonce":1103241451}]},"public_addr":"172.21.15.137:6809/1103241451","cluster_addr":"172.21.15.137:6811/1103241451","heartbeat_back_addr":"172.21.15.137:6815/1103241451","heartbeat_front_addr":"172.21.15.137:6813/1103241451","state":["exists","up"]},{"osd":5,"uuid":"eb32c016-abeb-44ab-abf0-19dd09235736","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6809","nonce":2648569482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6811","nonce":2648569482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6815","nonce":2648569482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6813","nonce":2648569482}]},"public_addr":"172.21.15.203:6809/2648569482","cluster_addr":"172.21.15.203:6811/2648569482","heartbeat_back_addr":"172.21.15.203:6815/2648569482","heartbeat_front_addr":"172.21.15.203:6813/2648569482","state":["exists","up"]},{"osd":6,"uuid":"0e3fa97e-2ca6-418c-9e8d-a6fe3c56adb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6819","nonce":2463414651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6821","nonce":2463414651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6825","nonce":2463414651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6823","nonce":2463414651}]},"public_addr":"172.21.15.77:6819/2463414651","cluster_addr":"172.21.15.77:6821/2463414651","heartbeat_back_addr":"172.21.15.77:6825/2463414651","heartbeat_front_addr":"172.21.15.77:6823/2463414651","state":["exists","up"]},{"osd":7,"uuid":"d7398c01-0f98-4138-8a49-c3ff0aae7e88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6817","nonce":3108330766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6819","nonce":3108330766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6823","nonce":3108330766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6821","nonce":3108330766}]},"public_addr":"172.21.15.137:6817/3108330766","cluster_addr":"172.21.15.137:6819/3108330766","heartbeat_back_addr":"172.21.15.137:6823/3108330766","heartbeat_front_addr":"172.21.15.137:6821/3108330766","state":["exists","up"]},{"osd":8,"uuid":"0c09050d-3e43-410c-8cd7-bcb891af56b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6817","nonce":1161158776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6819","nonce":1161158776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6823","nonce":1161158776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6821","nonce":1161158776}]},"public_addr":"172.21.15.203:6817/1161158776","cluster_addr":"172.21.15.203:6819/1161158776","heartbeat_back_addr":"172.21.15.203:6823/1161158776","heartbeat_front_addr":"172.21.15.203:6821/1161158776","state":["exists","up"]},{"osd":9,"uuid":"527ecb49-95ec-48bd-9872-3b07fae5c5ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6827","nonce":2409880685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6829","nonce":2409880685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6833","nonce":2409880685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6831","nonce":2409880685}]},"public_addr":"172.21.15.77:6827/2409880685","cluster_addr":"172.21.15.77:6829/2409880685","heartbeat_back_addr":"172.21.15.77:6833/2409880685","heartbeat_front_addr":"172.21.15.77:6831/2409880685","state":["exists","up"]},{"osd":10,"uuid":"1680bb3e-a3c8-427b-af68-e2214f32bed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6825","nonce":374920146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6827","nonce":374920146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6831","nonce":374920146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6829","nonce":374920146}]},"public_addr":"172.21.15.137:6825/374920146","cluster_addr":"172.21.15.137:6827/374920146","heartbeat_back_addr":"172.21.15.137:6831/374920146","heartbeat_front_addr":"172.21.15.137:6829/374920146","state":["exists","up"]},{"osd":11,"uuid":"8a533466-023d-4803-ae36-9a9f1715e49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6825","nonce":2161588962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6827","nonce":2161588962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6831","nonce":2161588962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6829","nonce":2161588962}]},"public_addr":"172.21.15.203:6825/2161588962","cluster_addr":"172.21.15.203:6827/2161588962","heartbeat_back_addr":"172.21.15.203:6831/2161588962","heartbeat_front_addr":"172.21.15.203:6829/2161588962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:11:45.420509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:06.210115+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:28.191307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:49.594116+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:11.261689+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:33.376541+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:54.968336+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:16.775310+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:38.733792+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:59.456699+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:15:20.924059+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:15:43.094773+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/2260035254":"2023-12-02T01:10:24.191835+0000","172.21.15.77:6800/1456621747":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/219838828":"2023-12-02T01:10:24.191835+0000","172.21.15.77:6801/1456621747":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/2937149266":"2023-12-02T01:10:02.232948+0000","172.21.15.77:0/807520018":"2023-12-02T01:10:02.232948+0000","172.21.15.77:6801/1661788026":"2023-12-02T01:10:02.232948+0000","172.21.15.77:6800/1661788026":"2023-12-02T01:10:02.232948+0000","172.21.15.77:0/3551282473":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/1202222558":"2023-12-02T01:10:02.232948+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:17:18.675 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.674+0000 7f565dd2a700 1 -- 172.21.15.77:0/2629260852 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f56400a05f0 msgr2=0x7f56400a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:18.675 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.674+0000 7f565dd2a700 1 --2- 172.21.15.77:0/2629260852 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f56400a05f0 0x7f56400a2ab0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f564c005fd0 tx=0x7f564c005f00 comp rx=0 tx=0).stop 2023-12-01T01:17:18.675 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.674+0000 7f565dd2a700 1 -- 172.21.15.77:0/2629260852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5658102010 msgr2=0x7f5658196f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:18.675 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.674+0000 7f565dd2a700 1 --2- 172.21.15.77:0/2629260852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5658102010 0x7f5658196f80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5648000c00 tx=0x7f564800f710 comp rx=0 tx=0).stop 2023-12-01T01:17:18.675 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.674+0000 7f565dd2a700 1 -- 172.21.15.77:0/2629260852 shutdown_connections 2023-12-01T01:17:18.676 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.674+0000 7f565dd2a700 1 --2- 172.21.15.77:0/2629260852 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f56400a05f0 0x7f56400a2ab0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:18.676 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.674+0000 7f565dd2a700 1 --2- 172.21.15.77:0/2629260852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56581029b0 0x7f565819dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:18.676 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.674+0000 7f565dd2a700 1 --2- 172.21.15.77:0/2629260852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56580ff040 0x7f5658196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:18.676 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.674+0000 7f565dd2a700 1 --2- 172.21.15.77:0/2629260852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5658102010 0x7f5658196f80 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:18.676 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.674+0000 7f565dd2a700 1 -- 172.21.15.77:0/2629260852 >> 172.21.15.77:0/2629260852 conn(0x7f56580fa950 msgr2=0x7f56580fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:18.676 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.674+0000 7f565dd2a700 1 -- 172.21.15.77:0/2629260852 shutdown_connections 2023-12-01T01:17:18.676 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.674+0000 7f565dd2a700 1 -- 172.21.15.77:0/2629260852 wait complete. 2023-12-01T01:17:18.688 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:17:18.811 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.807+0000 7f999150f700 1 -- 172.21.15.77:0/249374642 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f998c101810 msgr2=0x7f998c1080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:18.811 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.807+0000 7f999150f700 1 --2- 172.21.15.77:0/249374642 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f998c101810 0x7f998c1080c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f998400b0d0 tx=0x7f998400b3e0 comp rx=0 tx=0).stop 2023-12-01T01:17:18.811 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.808+0000 7f999150f700 1 -- 172.21.15.77:0/249374642 shutdown_connections 2023-12-01T01:17:18.812 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.808+0000 7f999150f700 1 --2- 172.21.15.77:0/249374642 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f998c101810 0x7f998c1080c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:18.812 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.808+0000 7f999150f700 1 --2- 172.21.15.77:0/249374642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f998c0ffc80 0x7f998c1000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:18.812 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.808+0000 7f999150f700 1 --2- 172.21.15.77:0/249374642 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f998c100e70 0x7f998c1012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:18.812 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.808+0000 7f999150f700 1 -- 172.21.15.77:0/249374642 >> 172.21.15.77:0/249374642 conn(0x7f998c0fb1f0 msgr2=0x7f998c0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:18.812 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.808+0000 7f999150f700 1 -- 172.21.15.77:0/249374642 shutdown_connections 2023-12-01T01:17:18.812 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.808+0000 7f999150f700 1 -- 172.21.15.77:0/249374642 wait complete. 2023-12-01T01:17:18.812 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.808+0000 7f999150f700 1 Processor -- start 2023-12-01T01:17:18.812 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.808+0000 7f999150f700 1 -- start start 2023-12-01T01:17:18.812 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.808+0000 7f999150f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f998c0ffc80 0x7f998c103c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:18.812 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.808+0000 7f999150f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f998c100e70 0x7f998c102250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:18.812 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.808+0000 7f999150f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f998c101810 0x7f998c102790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:18.812 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.808+0000 7f999150f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998c102f30 con 0x7f998c0ffc80 2023-12-01T01:17:18.812 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.808+0000 7f999150f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998c1030a0 con 0x7f998c100e70 2023-12-01T01:17:18.813 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.808+0000 7f999150f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998c1a0b40 con 0x7f998c101810 2023-12-01T01:17:18.813 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.809+0000 7f998affd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f998c0ffc80 0x7f998c103c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:18.813 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.809+0000 7f998affd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f998c0ffc80 0x7f998c103c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58302/0 (socket says 172.21.15.77:58302) 2023-12-01T01:17:18.813 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.809+0000 7f998affd700 1 -- 172.21.15.77:0/2966966068 learned_addr learned my addr 172.21.15.77:0/2966966068 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:18.813 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.809+0000 7f998a7fc700 1 --2- 172.21.15.77:0/2966966068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f998c100e70 0x7f998c102250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:18.813 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.809+0000 7f998b7fe700 1 --2- 172.21.15.77:0/2966966068 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f998c101810 0x7f998c102790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:18.813 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.809+0000 7f998affd700 1 -- 172.21.15.77:0/2966966068 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f998c101810 msgr2=0x7f998c102790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:18.813 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.809+0000 7f998affd700 1 --2- 172.21.15.77:0/2966966068 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f998c101810 0x7f998c102790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:18.813 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.809+0000 7f998affd700 1 -- 172.21.15.77:0/2966966068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f998c100e70 msgr2=0x7f998c102250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:18.813 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.809+0000 7f998affd700 1 --2- 172.21.15.77:0/2966966068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f998c100e70 0x7f998c102250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:18.813 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.809+0000 7f998affd700 1 -- 172.21.15.77:0/2966966068 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9984009d00 con 0x7f998c0ffc80 2023-12-01T01:17:18.813 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.809+0000 7f998affd700 1 --2- 172.21.15.77:0/2966966068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f998c0ffc80 0x7f998c103c10 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f997c00d950 tx=0x7f997c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:18.813 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.809+0000 7f9983fff700 1 -- 172.21.15.77:0/2966966068 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997c004a30 con 0x7f998c0ffc80 2023-12-01T01:17:18.813 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.809+0000 7f9983fff700 1 -- 172.21.15.77:0/2966966068 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f997c0054e0 con 0x7f998c0ffc80 2023-12-01T01:17:18.814 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.809+0000 7f999150f700 1 -- 172.21.15.77:0/2966966068 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f998c1a0e20 con 0x7f998c0ffc80 2023-12-01T01:17:18.814 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.809+0000 7f9983fff700 1 -- 172.21.15.77:0/2966966068 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997c009bc0 con 0x7f998c0ffc80 2023-12-01T01:17:18.814 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.809+0000 7f999150f700 1 -- 172.21.15.77:0/2966966068 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f998c1a11f0 con 0x7f998c0ffc80 2023-12-01T01:17:18.814 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.810+0000 7f999150f700 1 -- 172.21.15.77:0/2966966068 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f998c10b530 con 0x7f998c0ffc80 2023-12-01T01:17:18.814 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.811+0000 7f9983fff700 1 -- 172.21.15.77:0/2966966068 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f997c004b90 con 0x7f998c0ffc80 2023-12-01T01:17:18.815 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.812+0000 7f9983fff700 1 --2- 172.21.15.77:0/2966966068 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f99780a05f0 0x7f99780a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:18.815 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.812+0000 7f9983fff700 1 -- 172.21.15.77:0/2966966068 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f997c0cbc20 con 0x7f998c0ffc80 2023-12-01T01:17:18.815 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.812+0000 7f998a7fc700 1 --2- 172.21.15.77:0/2966966068 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f99780a05f0 0x7f99780a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:18.815 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.812+0000 7f998a7fc700 1 --2- 172.21.15.77:0/2966966068 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f99780a05f0 0x7f99780a2ab0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9974007be0 tx=0x7f99740073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:18.816 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:18.815+0000 7f9983fff700 1 -- 172.21.15.77:0/2966966068 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f997c095810 con 0x7f998c0ffc80 2023-12-01T01:17:19.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:18 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/2267347781' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-01T01:17:19.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:18 smithi203 ceph-mon[103581]: pgmap v279: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2023-12-01T01:17:19.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:18 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/3696168281' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:17:19.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:18 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/2267347781' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-01T01:17:19.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:18 smithi137 ceph-mon[103312]: pgmap v279: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2023-12-01T01:17:19.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:18 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/3696168281' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:17:19.052 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.051+0000 7f999150f700 1 -- 172.21.15.77:0/2966966068 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f998c1a1830 con 0x7f998c0ffc80 2023-12-01T01:17:19.053 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.052+0000 7f9983fff700 1 -- 172.21.15.77:0/2966966068 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10838 (secure 0 0 0) 0x7f997c017070 con 0x7f998c0ffc80 2023-12-01T01:17:19.053 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:17:19.053 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1435145620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1435145620},{"type":"v1","addr":"172.21.15.77:6835","nonce":1435145620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/841838908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":841838908},{"type":"v1","addr":"172.21.15.77:6837","nonce":841838908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-01T01:16:25.345276+0000","modified":"2023-12-01T01:16:32.596617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24496,"mds_2":24475,"mds_3":24473,"mds_4":14649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/3726913446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3726913446},{"type":"v1","addr":"172.21.15.77:6839","nonce":3726913446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14649":{"gid":14649,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":5,"addr":"172.21.15.77:6841/3023184965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3023184965},{"type":"v1","addr":"172.21.15.77:6841","nonce":3023184965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/2249638445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2249638445},{"type":"v1","addr":"172.21.15.137:6833","nonce":2249638445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6833/1201599774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1201599774},{"type":"v1","addr":"172.21.15.203:6833","nonce":1201599774}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3935354878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3935354878},{"type":"v1","addr":"172.21.15.137:6835","nonce":3935354878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3237538176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3237538176},{"type":"v1","addr":"172.21.15.203:6835","nonce":3237538176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3518881867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3518881867},{"type":"v1","addr":"172.21.15.137:6837","nonce":3518881867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.203:6837/2887080977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2887080977},{"type":"v1","addr":"172.21.15.203:6837","nonce":2887080977}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.203:6839/2888221613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2888221613},{"type":"v1","addr":"172.21.15.203:6839","nonce":2888221613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/1099966660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1099966660},{"type":"v1","addr":"172.21.15.137:6839","nonce":1099966660}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-01T01:17:19.055 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.054+0000 7f999150f700 1 -- 172.21.15.77:0/2966966068 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f99780a05f0 msgr2=0x7f99780a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.055 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.054+0000 7f999150f700 1 --2- 172.21.15.77:0/2966966068 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f99780a05f0 0x7f99780a2ab0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9974007be0 tx=0x7f99740073d0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.055 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.054+0000 7f999150f700 1 -- 172.21.15.77:0/2966966068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f998c0ffc80 msgr2=0x7f998c103c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.056 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.054+0000 7f999150f700 1 --2- 172.21.15.77:0/2966966068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f998c0ffc80 0x7f998c103c10 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f997c00d950 tx=0x7f997c00dc60 comp rx=0 tx=0).stop 2023-12-01T01:17:19.056 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.054+0000 7f999150f700 1 -- 172.21.15.77:0/2966966068 shutdown_connections 2023-12-01T01:17:19.056 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.054+0000 7f999150f700 1 --2- 172.21.15.77:0/2966966068 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f99780a05f0 0x7f99780a2ab0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.056 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.054+0000 7f999150f700 1 --2- 172.21.15.77:0/2966966068 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f998c101810 0x7f998c102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.056 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.054+0000 7f999150f700 1 --2- 172.21.15.77:0/2966966068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f998c0ffc80 0x7f998c103c10 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.056 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.054+0000 7f999150f700 1 --2- 172.21.15.77:0/2966966068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f998c100e70 0x7f998c102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.056 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.054+0000 7f999150f700 1 -- 172.21.15.77:0/2966966068 >> 172.21.15.77:0/2966966068 conn(0x7f998c0fb1f0 msgr2=0x7f998c069130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:19.056 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.054+0000 7f999150f700 1 -- 172.21.15.77:0/2966966068 shutdown_connections 2023-12-01T01:17:19.056 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.054+0000 7f999150f700 1 -- 172.21.15.77:0/2966966068 wait complete. 2023-12-01T01:17:19.057 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 27 2023-12-01T01:17:19.066 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:18 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/2267347781' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-01T01:17:19.066 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:18 smithi077 ceph-mon[98867]: pgmap v279: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2023-12-01T01:17:19.066 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:18 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/3696168281' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:17:19.068 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-12-01T01:16:25.345276+0000', 'modified': '2023-12-01T01:16:32.596617+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24500, 'mds_1': 24496, 'mds_2': 24475, 'mds_3': 24473, 'mds_4': 14649}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14637': {'gid': 14637, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.77:6839/3726913446', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6838', 'nonce': 3726913446}, {'type': 'v1', 'addr': '172.21.15.77:6839', 'nonce': 3726913446}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_14649': {'gid': 14649, 'name': 'a', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.77:6841/3023184965', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6840', 'nonce': 3023184965}, {'type': 'v1', 'addr': '172.21.15.77:6841', 'nonce': 3023184965}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24428': {'gid': 24428, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.137:6833/2249638445', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6832', 'nonce': 2249638445}, {'type': 'v1', 'addr': '172.21.15.137:6833', 'nonce': 2249638445}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24430': {'gid': 24430, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.203:6833/1201599774', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6832', 'nonce': 1201599774}, {'type': 'v1', 'addr': '172.21.15.203:6833', 'nonce': 1201599774}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24449': {'gid': 24449, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.137:6835/3935354878', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6834', 'nonce': 3935354878}, {'type': 'v1', 'addr': '172.21.15.137:6835', 'nonce': 3935354878}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24463': {'gid': 24463, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.203:6835/3237538176', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6834', 'nonce': 3237538176}, {'type': 'v1', 'addr': '172.21.15.203:6835', 'nonce': 3237538176}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24473': {'gid': 24473, 'name': 'h', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.137:6837/3518881867', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6836', 'nonce': 3518881867}, {'type': 'v1', 'addr': '172.21.15.137:6837', 'nonce': 3518881867}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24475': {'gid': 24475, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.203:6837/2887080977', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6836', 'nonce': 2887080977}, {'type': 'v1', 'addr': '172.21.15.203:6837', 'nonce': 2887080977}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24496': {'gid': 24496, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.203:6839/2888221613', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6838', 'nonce': 2888221613}, {'type': 'v1', 'addr': '172.21.15.203:6839', 'nonce': 2888221613}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24500': {'gid': 24500, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.137:6839/1099966660', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6838', 'nonce': 1099966660}, {'type': 'v1', 'addr': '172.21.15.137:6839', 'nonce': 1099966660}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-01T01:17:19.069 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-12-01T01:17:19.069 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-12-01T01:17:19.192 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.189+0000 7f0ebdccf700 1 -- 172.21.15.77:0/724011653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0eb8100df0 msgr2=0x7f0eb8101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.192 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.189+0000 7f0ebdccf700 1 --2- 172.21.15.77:0/724011653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0eb8100df0 0x7f0eb8101210 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0eac0099c0 tx=0x7f0eac009cd0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.192 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.189+0000 7f0ebdccf700 1 -- 172.21.15.77:0/724011653 shutdown_connections 2023-12-01T01:17:19.192 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.189+0000 7f0ebdccf700 1 --2- 172.21.15.77:0/724011653 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb8102980 0x7f0eb8109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.189+0000 7f0ebdccf700 1 --2- 172.21.15.77:0/724011653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0eb8100df0 0x7f0eb8101210 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.189+0000 7f0ebdccf700 1 --2- 172.21.15.77:0/724011653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0eb8101fe0 0x7f0eb8102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.189+0000 7f0ebdccf700 1 -- 172.21.15.77:0/724011653 >> 172.21.15.77:0/724011653 conn(0x7f0eb80fc360 msgr2=0x7f0eb80fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:19.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.189+0000 7f0ebdccf700 1 -- 172.21.15.77:0/724011653 shutdown_connections 2023-12-01T01:17:19.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.189+0000 7f0ebdccf700 1 -- 172.21.15.77:0/724011653 wait complete. 2023-12-01T01:17:19.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.189+0000 7f0ebdccf700 1 Processor -- start 2023-12-01T01:17:19.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.190+0000 7f0ebdccf700 1 -- start start 2023-12-01T01:17:19.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.190+0000 7f0ebdccf700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0eb8100df0 0x7f0eb8074b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.190+0000 7f0ebdccf700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0eb8101fe0 0x7f0eb8073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.190+0000 7f0ebdccf700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb8102980 0x7f0eb80736c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.190+0000 7f0ebdccf700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb8073e60 con 0x7f0eb8101fe0 2023-12-01T01:17:19.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.190+0000 7f0ebdccf700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb8073fd0 con 0x7f0eb8102980 2023-12-01T01:17:19.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.190+0000 7f0ebdccf700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb81a0b70 con 0x7f0eb8100df0 2023-12-01T01:17:19.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.190+0000 7f0eb6ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0eb8101fe0 0x7f0eb8073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.190+0000 7f0eb6ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0eb8101fe0 0x7f0eb8073180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58318/0 (socket says 172.21.15.77:58318) 2023-12-01T01:17:19.194 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.190+0000 7f0eb6ffd700 1 -- 172.21.15.77:0/2204230169 learned_addr learned my addr 172.21.15.77:0/2204230169 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:19.194 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.190+0000 7f0eb7fff700 1 --2- 172.21.15.77:0/2204230169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb8102980 0x7f0eb80736c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.194 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.190+0000 7f0eb77fe700 1 --2- 172.21.15.77:0/2204230169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0eb8100df0 0x7f0eb8074b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.194 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.190+0000 7f0eb6ffd700 1 -- 172.21.15.77:0/2204230169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0eb8100df0 msgr2=0x7f0eb8074b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.194 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.190+0000 7f0eb6ffd700 1 --2- 172.21.15.77:0/2204230169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0eb8100df0 0x7f0eb8074b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.194 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.190+0000 7f0eb6ffd700 1 -- 172.21.15.77:0/2204230169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb8102980 msgr2=0x7f0eb80736c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.194 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.190+0000 7f0eb6ffd700 1 --2- 172.21.15.77:0/2204230169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb8102980 0x7f0eb80736c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.194 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.190+0000 7f0eb6ffd700 1 -- 172.21.15.77:0/2204230169 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eac0095f0 con 0x7f0eb8101fe0 2023-12-01T01:17:19.194 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.190+0000 7f0eb7fff700 1 --2- 172.21.15.77:0/2204230169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb8102980 0x7f0eb80736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:17:19.194 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.190+0000 7f0eb77fe700 1 --2- 172.21.15.77:0/2204230169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0eb8100df0 0x7f0eb8074b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:17:19.194 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.190+0000 7f0eb6ffd700 1 --2- 172.21.15.77:0/2204230169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0eb8101fe0 0x7f0eb8073180 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0ea800d980 tx=0x7f0ea800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:19.194 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.190+0000 7f0eb4ff9700 1 -- 172.21.15.77:0/2204230169 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea800bb60 con 0x7f0eb8101fe0 2023-12-01T01:17:19.194 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.190+0000 7f0eb4ff9700 1 -- 172.21.15.77:0/2204230169 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ea8007640 con 0x7f0eb8101fe0 2023-12-01T01:17:19.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.191+0000 7f0ebdccf700 1 -- 172.21.15.77:0/2204230169 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eb81a0e50 con 0x7f0eb8101fe0 2023-12-01T01:17:19.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.191+0000 7f0eb4ff9700 1 -- 172.21.15.77:0/2204230169 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea8004e90 con 0x7f0eb8101fe0 2023-12-01T01:17:19.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.191+0000 7f0ebdccf700 1 -- 172.21.15.77:0/2204230169 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eb81a12d0 con 0x7f0eb8101fe0 2023-12-01T01:17:19.201 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.192+0000 7f0eb4ff9700 1 -- 172.21.15.77:0/2204230169 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f0ea800bcc0 con 0x7f0eb8101fe0 2023-12-01T01:17:19.201 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.193+0000 7f0eb4ff9700 1 --2- 172.21.15.77:0/2204230169 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f0ea00a05b0 0x7f0ea00a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.201 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.193+0000 7f0eb77fe700 1 --2- 172.21.15.77:0/2204230169 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f0ea00a05b0 0x7f0ea00a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.201 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.193+0000 7f0eb4ff9700 1 -- 172.21.15.77:0/2204230169 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0ea80cb820 con 0x7f0eb8101fe0 2023-12-01T01:17:19.201 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.193+0000 7f0eb77fe700 1 --2- 172.21.15.77:0/2204230169 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f0ea00a05b0 0x7f0ea00a2a70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0eac0098e0 tx=0x7f0eac000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:19.201 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.194+0000 7f0ec9e5fb80 1 Processor -- start 2023-12-01T01:17:19.201 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.194+0000 7f0ec9e5fb80 1 -- start start 2023-12-01T01:17:19.201 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.194+0000 7f0ec9e5fb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ae97bd0a0 0x562ae97bd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.201 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.194+0000 7f0ec9e5fb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ae97bda20 0x562ae97bbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.202 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.194+0000 7f0ec9e5fb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ae97bc3a0 0x562ae97bc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.202 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.194+0000 7f0ec9e5fb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ae97139c0 con 0x562ae97bda20 2023-12-01T01:17:19.202 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.194+0000 7f0ec9e5fb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ae9711380 con 0x562ae97bd0a0 2023-12-01T01:17:19.202 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.194+0000 7f0ec9e5fb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ae970cc70 con 0x562ae97bc3a0 2023-12-01T01:17:19.202 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.194+0000 7f0eb6ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ae97bda20 0x562ae97bbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.202 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.194+0000 7f0eb6ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ae97bda20 0x562ae97bbe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58326/0 (socket says 172.21.15.77:58326) 2023-12-01T01:17:19.202 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.194+0000 7f0eb6ffd700 1 -- 172.21.15.77:0/2474945962 learned_addr learned my addr 172.21.15.77:0/2474945962 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:19.202 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.195+0000 7f0eb7fff700 1 --2- 172.21.15.77:0/2474945962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ae97bc3a0 0x562ae97bc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.202 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.195+0000 7f0eb77fe700 1 --2- 172.21.15.77:0/2474945962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ae97bd0a0 0x562ae97bd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.202 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.195+0000 7f0eb6ffd700 1 -- 172.21.15.77:0/2474945962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ae97bc3a0 msgr2=0x562ae97bc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:19.202 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.195+0000 7f0eb6ffd700 1 --2- 172.21.15.77:0/2474945962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ae97bc3a0 0x562ae97bc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.202 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.195+0000 7f0eb6ffd700 1 -- 172.21.15.77:0/2474945962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ae97bd0a0 msgr2=0x562ae97bd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:19.202 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.195+0000 7f0eb6ffd700 1 --2- 172.21.15.77:0/2474945962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ae97bd0a0 0x562ae97bd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.202 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.195+0000 7f0eb6ffd700 1 -- 172.21.15.77:0/2474945962 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ae953ad90 con 0x562ae97bda20 2023-12-01T01:17:19.203 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.195+0000 7f0eb77fe700 1 --2- 172.21.15.77:0/2474945962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ae97bd0a0 0x562ae97bd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:17:19.203 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.195+0000 7f0eb6ffd700 1 --2- 172.21.15.77:0/2474945962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ae97bda20 0x562ae97bbe10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0ea80c93d0 tx=0x7f0ea80047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee904b4ccc41f21a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:19.203 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.195+0000 7f0e9dffb700 1 -- 172.21.15.77:0/2474945962 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea80990d0 con 0x562ae97bda20 2023-12-01T01:17:19.203 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.195+0000 7f0e9dffb700 1 -- 172.21.15.77:0/2474945962 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ea809cf00 con 0x562ae97bda20 2023-12-01T01:17:19.203 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.195+0000 7f0e9dffb700 1 -- 172.21.15.77:0/2474945962 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea80a0eb0 con 0x562ae97bda20 2023-12-01T01:17:19.203 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.196+0000 7f0ec9e5fb80 1 -- 172.21.15.77:0/2474945962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ae97bda20 msgr2=0x562ae97bbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.203 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.196+0000 7f0ec9e5fb80 1 --2- 172.21.15.77:0/2474945962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ae97bda20 0x562ae97bbe10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0ea80c93d0 tx=0x7f0ea80047d0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.203 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.196+0000 7f0ec9e5fb80 1 -- 172.21.15.77:0/2474945962 shutdown_connections 2023-12-01T01:17:19.203 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.196+0000 7f0ec9e5fb80 1 --2- 172.21.15.77:0/2474945962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ae97bc3a0 0x562ae97bc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.203 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.196+0000 7f0ec9e5fb80 1 --2- 172.21.15.77:0/2474945962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ae97bd0a0 0x562ae97bd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.203 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.196+0000 7f0ec9e5fb80 1 --2- 172.21.15.77:0/2474945962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ae97bda20 0x562ae97bbe10 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.205 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.196+0000 7f0ec9e5fb80 1 -- 172.21.15.77:0/2474945962 >> 172.21.15.77:0/2474945962 conn(0x562ae971b080 msgr2=0x562ae97b68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:19.205 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.196+0000 7f0ec9e5fb80 1 -- 172.21.15.77:0/2474945962 shutdown_connections 2023-12-01T01:17:19.205 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.196+0000 7f0ec9e5fb80 1 -- 172.21.15.77:0/2474945962 wait complete. 2023-12-01T01:17:19.205 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.196+0000 7f0ec9e5fb80 1 -- start start 2023-12-01T01:17:19.205 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.196+0000 7f0ec9e5fb80 1 Processor -- start 2023-12-01T01:17:19.205 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.197+0000 7f0ec9e5fb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ae97bc3a0 0x562ae97c5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.205 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.197+0000 7f0ec9e5fb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ae97bd0a0 0x562ae97c5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.205 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.197+0000 7f0ec9e5fb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ae97bda20 0x562ae97cce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.205 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.197+0000 7f0ec9e5fb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ae96a94c0 con 0x562ae97bc3a0 2023-12-01T01:17:19.205 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.197+0000 7f0ec9e5fb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ae96a9160 con 0x562ae97bda20 2023-12-01T01:17:19.205 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.197+0000 7f0ec9e5fb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ae95908d0 con 0x562ae97bd0a0 2023-12-01T01:17:19.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.197+0000 7f0eb77fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ae97bc3a0 0x562ae97c5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.197+0000 7f0ec9e5fb80 10 client.0 _ll_register_callbacks cb 0x562ae95f6a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:17:19.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.197+0000 7f0eb77fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ae97bc3a0 0x562ae97c5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58330/0 (socket says 172.21.15.77:58330) 2023-12-01T01:17:19.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.197+0000 7f0eb77fe700 1 -- 172.21.15.77:0/3695063269 learned_addr learned my addr 172.21.15.77:0/3695063269 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:19.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.197+0000 7f0eb7fff700 1 --2- 172.21.15.77:0/3695063269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ae97bda20 0x562ae97cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.197+0000 7f0eb6ffd700 1 --2- 172.21.15.77:0/3695063269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ae97bd0a0 0x562ae97c5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.197+0000 7f0eb77fe700 1 -- 172.21.15.77:0/3695063269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ae97bd0a0 msgr2=0x562ae97c5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:19.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.197+0000 7f0eb77fe700 1 --2- 172.21.15.77:0/3695063269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ae97bd0a0 0x562ae97c5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.197+0000 7f0eb77fe700 1 -- 172.21.15.77:0/3695063269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ae97bda20 msgr2=0x562ae97cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:19.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.197+0000 7f0eb77fe700 1 --2- 172.21.15.77:0/3695063269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ae97bda20 0x562ae97cce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.197+0000 7f0eb77fe700 1 -- 172.21.15.77:0/3695063269 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eb81a12d0 con 0x562ae97bc3a0 2023-12-01T01:17:19.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.197+0000 7f0eb77fe700 1 --2- 172.21.15.77:0/3695063269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ae97bc3a0 0x562ae97c5910 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0eac015000 tx=0x7f0eac009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b502ec8e51e8138e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:19.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.197+0000 7f0e96ffd700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:17:19.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.197+0000 7f0e96ffd700 1 -- 172.21.15.77:0/3695063269 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac02b070 con 0x562ae97bc3a0 2023-12-01T01:17:19.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.197+0000 7f0e96ffd700 1 -- 172.21.15.77:0/3695063269 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0eac003ad0 con 0x562ae97bc3a0 2023-12-01T01:17:19.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.197+0000 7f0ec9e5fb80 1 -- 172.21.15.77:0/3695063269 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ae9737cc0 con 0x562ae97bc3a0 2023-12-01T01:17:19.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.198+0000 7f0e96ffd700 1 -- 172.21.15.77:0/3695063269 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0eac01e070 con 0x562ae97bc3a0 2023-12-01T01:17:19.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.198+0000 7f0ec9e5fb80 10 client.15060 fetch_fsmap learned FSMap version 27 2023-12-01T01:17:19.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.198+0000 7f0ec9e5fb80 1 -- 172.21.15.77:0/3695063269 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562ae9523c10 con 0x562ae97bc3a0 2023-12-01T01:17:19.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.198+0000 7f0e96ffd700 1 -- 172.21.15.77:0/3695063269 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562ae9737cc0 con 0x562ae97bc3a0 2023-12-01T01:17:19.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.198+0000 7f0e96ffd700 1 -- 172.21.15.77:0/3695063269 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f0eac004b60 con 0x562ae97bc3a0 2023-12-01T01:17:19.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.198+0000 7f0ec9e5fb80 10 client.15060 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:17:19.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.198+0000 7f0ec9e5fb80 10 client.15060 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-01T01:17:19.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.198+0000 7f0ec9e5fb80 20 client.15060 populate_metadata read hostname 'smithi077' 2023-12-01T01:17:19.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.198+0000 7f0ec9e5fb80 1 --2- 172.21.15.77:0/3695063269 >> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] conn(0x562ae97d0a80 0x562ae97cfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.198+0000 7f0ec9e5fb80 4 client.15060 mds_command: new command op to 14649 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:17:19.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.198+0000 7f0ec9e5fb80 1 -- 172.21.15.77:0/3695063269 --> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ae978adc0 con 0x562ae97d0a80 2023-12-01T01:17:19.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.198+0000 7f0eb6ffd700 1 --2- 172.21.15.77:0/3695063269 >> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] conn(0x562ae97d0a80 0x562ae97cfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.198+0000 7f0e96ffd700 10 client.15060 ms_handle_connect on v2:172.21.15.77:6840/3023184965 2023-12-01T01:17:19.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.198+0000 7f0eb6ffd700 1 --2- 172.21.15.77:0/3695063269 >> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] conn(0x562ae97d0a80 0x562ae97cfeb0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=272753de7149c547 server_cookie=bed8dfaea52a068a in_seq=0 out_seq=0 2023-12-01T01:17:19.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.199+0000 7f0e96ffd700 1 -- 172.21.15.77:0/3695063269 <== mds.4 v2:172.21.15.77:6840/3023184965 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x562ae978adc0 con 0x562ae97d0a80 2023-12-01T01:17:19.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.199+0000 7f0e96ffd700 10 client.15060 handle_command_reply: tid=0 2023-12-01T01:17:19.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.199+0000 7f0ec9e5fb80 1 client.15060 shutdown 2023-12-01T01:17:19.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.199+0000 7f0ec9e5fb80 1 -- 172.21.15.77:0/3695063269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ae97bc3a0 msgr2=0x562ae97c5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.199+0000 7f0ec9e5fb80 1 --2- 172.21.15.77:0/3695063269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ae97bc3a0 0x562ae97c5910 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0eac015000 tx=0x7f0eac009310 comp rx=0 tx=0).stop 2023-12-01T01:17:19.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.199+0000 7f0ec9e5fb80 1 -- 172.21.15.77:0/3695063269 shutdown_connections 2023-12-01T01:17:19.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.199+0000 7f0ec9e5fb80 1 --2- 172.21.15.77:0/3695063269 >> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] conn(0x562ae97d0a80 0x562ae97cfeb0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.199+0000 7f0ec9e5fb80 1 --2- 172.21.15.77:0/3695063269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ae97bda20 0x562ae97cce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.199+0000 7f0ec9e5fb80 1 --2- 172.21.15.77:0/3695063269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ae97bc3a0 0x562ae97c5910 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.199+0000 7f0ec9e5fb80 1 --2- 172.21.15.77:0/3695063269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ae97bd0a0 0x562ae97c5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.199+0000 7f0ec9e5fb80 1 -- 172.21.15.77:0/3695063269 >> 172.21.15.77:0/3695063269 conn(0x562ae971b080 msgr2=0x562ae97b6060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:19.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.199+0000 7f0e96ffd700 0 client.15060 ms_handle_reset on v2:172.21.15.77:6840/3023184965 2023-12-01T01:17:19.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.199+0000 7f0ec9e5fb80 1 -- 172.21.15.77:0/3695063269 shutdown_connections 2023-12-01T01:17:19.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.199+0000 7f0ec9e5fb80 1 -- 172.21.15.77:0/3695063269 wait complete. 2023-12-01T01:17:19.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.199+0000 7f0ec9e5fb80 20 client.15060 trim_cache size 0 max 16384 2023-12-01T01:17:19.224 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.216+0000 7f0ec9e5fb80 1 Processor -- start 2023-12-01T01:17:19.224 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.216+0000 7f0ec9e5fb80 1 -- start start 2023-12-01T01:17:19.224 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.216+0000 7f0ec9e5fb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ae97bc3a0 0x562ae97bbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.224 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.216+0000 7f0ec9e5fb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ae97bd0a0 0x562ae97d9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.224 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.216+0000 7f0ec9e5fb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ae97bda20 0x562ae97c02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.224 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.216+0000 7f0ec9e5fb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ae97c54e0 con 0x562ae97bda20 2023-12-01T01:17:19.224 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.216+0000 7f0ec9e5fb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ae9737cc0 con 0x562ae97bd0a0 2023-12-01T01:17:19.224 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.216+0000 7f0ec9e5fb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ae97c6b00 con 0x562ae97bc3a0 2023-12-01T01:17:19.224 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.217+0000 7f0eb7fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ae97bda20 0x562ae97c02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.224 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.217+0000 7f0eb7fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ae97bda20 0x562ae97c02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58342/0 (socket says 172.21.15.77:58342) 2023-12-01T01:17:19.224 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.217+0000 7f0eb7fff700 1 -- 172.21.15.77:0/1367864985 learned_addr learned my addr 172.21.15.77:0/1367864985 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:19.224 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.217+0000 7f0eb6ffd700 1 --2- 172.21.15.77:0/1367864985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ae97bd0a0 0x562ae97d9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.225 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.217+0000 7f0eb7fff700 1 -- 172.21.15.77:0/1367864985 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ae97bc3a0 msgr2=0x562ae97bbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:19.225 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.217+0000 7f0eb7fff700 1 --2- 172.21.15.77:0/1367864985 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ae97bc3a0 0x562ae97bbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.225 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.217+0000 7f0eb7fff700 1 -- 172.21.15.77:0/1367864985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ae97bd0a0 msgr2=0x562ae97d9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:19.225 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.217+0000 7f0eb7fff700 1 --2- 172.21.15.77:0/1367864985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ae97bd0a0 0x562ae97d9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.225 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.217+0000 7f0eb7fff700 1 -- 172.21.15.77:0/1367864985 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ae9561200 con 0x562ae97bda20 2023-12-01T01:17:19.225 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.217+0000 7f0eb6ffd700 1 --2- 172.21.15.77:0/1367864985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ae97bd0a0 0x562ae97d9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:17:19.225 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.217+0000 7f0eb7fff700 1 --2- 172.21.15.77:0/1367864985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ae97bda20 0x562ae97c02e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0009dd0 tx=0x7f0eb00076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a9d9beeb86ff58 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:19.225 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.217+0000 7f0e9e7fc700 1 -- 172.21.15.77:0/1367864985 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0017640 con 0x562ae97bda20 2023-12-01T01:17:19.225 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.217+0000 7f0e9e7fc700 1 -- 172.21.15.77:0/1367864985 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0eb000f040 con 0x562ae97bda20 2023-12-01T01:17:19.225 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.217+0000 7f0e9e7fc700 1 -- 172.21.15.77:0/1367864985 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0016980 con 0x562ae97bda20 2023-12-01T01:17:19.225 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.218+0000 7f0ec9e5fb80 1 -- 172.21.15.77:0/1367864985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ae97bda20 msgr2=0x562ae97c02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.225 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.218+0000 7f0ec9e5fb80 1 --2- 172.21.15.77:0/1367864985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ae97bda20 0x562ae97c02e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0009dd0 tx=0x7f0eb00076c0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.225 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.218+0000 7f0ec9e5fb80 1 -- 172.21.15.77:0/1367864985 shutdown_connections 2023-12-01T01:17:19.225 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.218+0000 7f0ec9e5fb80 1 --2- 172.21.15.77:0/1367864985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ae97bda20 0x562ae97c02e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.226 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.218+0000 7f0ec9e5fb80 1 --2- 172.21.15.77:0/1367864985 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ae97bc3a0 0x562ae97bbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.226 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.218+0000 7f0ec9e5fb80 1 --2- 172.21.15.77:0/1367864985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ae97bd0a0 0x562ae97d9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.226 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.218+0000 7f0ec9e5fb80 1 -- 172.21.15.77:0/1367864985 >> 172.21.15.77:0/1367864985 conn(0x562ae971b080 msgr2=0x562ae97cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:19.226 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.218+0000 7f0ec9e5fb80 1 -- 172.21.15.77:0/1367864985 shutdown_connections 2023-12-01T01:17:19.226 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.218+0000 7f0ec9e5fb80 1 -- 172.21.15.77:0/1367864985 wait complete. 2023-12-01T01:17:19.226 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.218+0000 7f0ec9e5fb80 1 -- start start 2023-12-01T01:17:19.226 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.218+0000 7f0ec9e5fb80 1 Processor -- start 2023-12-01T01:17:19.226 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.218+0000 7f0ec9e5fb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ae97bc3a0 0x562ae97d4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.226 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.218+0000 7f0ec9e5fb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ae97bd0a0 0x562ae97d4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.226 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.218+0000 7f0ec9e5fb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ae97bda20 0x562ae9822590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.226 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.218+0000 7f0ec9e5fb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ae97d5160 con 0x562ae97bc3a0 2023-12-01T01:17:19.226 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.218+0000 7f0ec9e5fb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ae9822ad0 con 0x562ae97bd0a0 2023-12-01T01:17:19.226 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.218+0000 7f0ec9e5fb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ae9822c10 con 0x562ae97bda20 2023-12-01T01:17:19.227 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.218+0000 7f0ec9e5fb80 10 client.0 _ll_register_callbacks cb 0x7f0eac0055c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:17:19.227 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.218+0000 7f0eb77fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ae97bc3a0 0x562ae97d4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.227 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.219+0000 7f0eb77fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ae97bc3a0 0x562ae97d4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58346/0 (socket says 172.21.15.77:58346) 2023-12-01T01:17:19.227 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.219+0000 7f0eb77fe700 1 -- 172.21.15.77:0/331090599 learned_addr learned my addr 172.21.15.77:0/331090599 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:19.227 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.219+0000 7f0eb6ffd700 1 --2- 172.21.15.77:0/331090599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ae97bd0a0 0x562ae97d4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.227 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.219+0000 7f0eb7fff700 1 --2- 172.21.15.77:0/331090599 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ae97bda20 0x562ae9822590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.227 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.219+0000 7f0eb77fe700 1 -- 172.21.15.77:0/331090599 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ae97bda20 msgr2=0x562ae9822590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:19.227 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.219+0000 7f0eb77fe700 1 --2- 172.21.15.77:0/331090599 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ae97bda20 0x562ae9822590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.227 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.219+0000 7f0eb77fe700 1 -- 172.21.15.77:0/331090599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ae97bd0a0 msgr2=0x562ae97d4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:19.227 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.219+0000 7f0eb77fe700 1 --2- 172.21.15.77:0/331090599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ae97bd0a0 0x562ae97d4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.227 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.219+0000 7f0eb77fe700 1 -- 172.21.15.77:0/331090599 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eb0009a80 con 0x562ae97bc3a0 2023-12-01T01:17:19.227 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.219+0000 7f0eb6ffd700 1 --2- 172.21.15.77:0/331090599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ae97bd0a0 0x562ae97d4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:17:19.227 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.219+0000 7f0eb77fe700 1 --2- 172.21.15.77:0/331090599 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ae97bc3a0 0x562ae97d4600 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0eac0099c0 tx=0x7f0eac00f630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60cee8b85ce718bb server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:19.227 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.219+0000 7f0e96ffd700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:17:19.227 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.219+0000 7f0e96ffd700 1 -- 172.21.15.77:0/331090599 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac02b040 con 0x562ae97bc3a0 2023-12-01T01:17:19.227 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.219+0000 7f0e96ffd700 1 -- 172.21.15.77:0/331090599 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0eac037020 con 0x562ae97bc3a0 2023-12-01T01:17:19.228 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.219+0000 7f0ec9e5fb80 1 -- 172.21.15.77:0/331090599 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ea809cf00 con 0x562ae97bc3a0 2023-12-01T01:17:19.228 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.219+0000 7f0e96ffd700 1 -- 172.21.15.77:0/331090599 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0eac00f770 con 0x562ae97bc3a0 2023-12-01T01:17:19.228 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.219+0000 7f0ec9e5fb80 10 client.15066 fetch_fsmap learned FSMap version 27 2023-12-01T01:17:19.228 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.219+0000 7f0ec9e5fb80 1 -- 172.21.15.77:0/331090599 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562ae9539260 con 0x562ae97bc3a0 2023-12-01T01:17:19.228 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.219+0000 7f0e96ffd700 1 -- 172.21.15.77:0/331090599 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ea809cf00 con 0x562ae97bc3a0 2023-12-01T01:17:19.228 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.220+0000 7f0e96ffd700 1 -- 172.21.15.77:0/331090599 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f0eac044030 con 0x562ae97bc3a0 2023-12-01T01:17:19.228 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.220+0000 7f0ec9e5fb80 10 client.15066 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:17:19.228 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.220+0000 7f0ec9e5fb80 10 client.15066 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-01T01:17:19.228 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.220+0000 7f0ec9e5fb80 20 client.15066 populate_metadata read hostname 'smithi077' 2023-12-01T01:17:19.228 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.220+0000 7f0ec9e5fb80 1 --2- 172.21.15.77:0/331090599 >> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] conn(0x562ae9827560 0x562ae9823ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.228 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.220+0000 7f0ec9e5fb80 4 client.15066 mds_command: new command op to 14649 tid=0[{"prefix": "status"}] 2023-12-01T01:17:19.228 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.220+0000 7f0ec9e5fb80 1 -- 172.21.15.77:0/331090599 --> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562ae9705cf0 con 0x562ae9827560 2023-12-01T01:17:19.228 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.220+0000 7f0eb6ffd700 1 --2- 172.21.15.77:0/331090599 >> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] conn(0x562ae9827560 0x562ae9823ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.228 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.220+0000 7f0e96ffd700 10 client.15066 ms_handle_connect on v2:172.21.15.77:6840/3023184965 2023-12-01T01:17:19.228 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.220+0000 7f0eb6ffd700 1 --2- 172.21.15.77:0/331090599 >> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] conn(0x562ae9827560 0x562ae9823ed0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bf0a706e90901e3d server_cookie=554b66463d2301f8 in_seq=0 out_seq=0 2023-12-01T01:17:19.228 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.220+0000 7f0e96ffd700 1 -- 172.21.15.77:0/331090599 <== mds.4 v2:172.21.15.77:6840/3023184965 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x562ae9705cf0 con 0x562ae9827560 2023-12-01T01:17:19.229 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.220+0000 7f0e96ffd700 10 client.15066 handle_command_reply: tid=0 2023-12-01T01:17:19.229 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.220+0000 7f0ec9e5fb80 1 client.15066 shutdown 2023-12-01T01:17:19.229 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.220+0000 7f0ec9e5fb80 1 -- 172.21.15.77:0/331090599 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ae97bc3a0 msgr2=0x562ae97d4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.229 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.220+0000 7f0ec9e5fb80 1 --2- 172.21.15.77:0/331090599 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ae97bc3a0 0x562ae97d4600 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0eac0099c0 tx=0x7f0eac00f630 comp rx=0 tx=0).stop 2023-12-01T01:17:19.229 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.220+0000 7f0ec9e5fb80 1 -- 172.21.15.77:0/331090599 shutdown_connections 2023-12-01T01:17:19.229 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.220+0000 7f0ec9e5fb80 1 --2- 172.21.15.77:0/331090599 >> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] conn(0x562ae9827560 0x562ae9823ed0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.229 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.220+0000 7f0ec9e5fb80 1 --2- 172.21.15.77:0/331090599 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562ae97bda20 0x562ae9822590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.229 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.221+0000 7f0ec9e5fb80 1 --2- 172.21.15.77:0/331090599 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562ae97bc3a0 0x562ae97d4600 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.229 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.221+0000 7f0ec9e5fb80 1 --2- 172.21.15.77:0/331090599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ae97bd0a0 0x562ae97d4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.229 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.221+0000 7f0e96ffd700 0 client.15066 ms_handle_reset on v2:172.21.15.77:6840/3023184965 2023-12-01T01:17:19.229 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.221+0000 7f0ec9e5fb80 1 -- 172.21.15.77:0/331090599 >> 172.21.15.77:0/331090599 conn(0x562ae971b080 msgr2=0x562ae97c02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:19.229 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.221+0000 7f0ec9e5fb80 1 -- 172.21.15.77:0/331090599 shutdown_connections 2023-12-01T01:17:19.229 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.221+0000 7f0ec9e5fb80 1 -- 172.21.15.77:0/331090599 wait complete. 2023-12-01T01:17:19.229 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.221+0000 7f0ec9e5fb80 20 client.15066 trim_cache size 0 max 16384 2023-12-01T01:17:19.230 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2023-12-01T01:17:19.230 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "2a06e29a-8fe6-11ee-95a2-87774f69a715", 2023-12-01T01:17:19.230 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 4, 2023-12-01T01:17:19.230 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 14649, 2023-12-01T01:17:19.230 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2023-12-01T01:17:19.230 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2023-12-01T01:17:19.230 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2023-12-01T01:17:19.230 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 47.628144648000003, 2023-12-01T01:17:19.230 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 27, 2023-12-01T01:17:19.230 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 82, 2023-12-01T01:17:19.231 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 82, 2023-12-01T01:17:19.231 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 61.614879285999997 2023-12-01T01:17:19.231 INFO:teuthology.orchestra.run.smithi077.stdout:} 2023-12-01T01:17:19.231 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.221+0000 7f0ebdccf700 1 -- 172.21.15.77:0/2204230169 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f0ea00a05b0 msgr2=0x7f0ea00a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.232 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.221+0000 7f0ebdccf700 1 --2- 172.21.15.77:0/2204230169 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f0ea00a05b0 0x7f0ea00a2a70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0eac0098e0 tx=0x7f0eac000bc0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.232 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.221+0000 7f0ebdccf700 1 -- 172.21.15.77:0/2204230169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0eb8101fe0 msgr2=0x7f0eb8073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.232 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.221+0000 7f0ebdccf700 1 --2- 172.21.15.77:0/2204230169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0eb8101fe0 0x7f0eb8073180 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0ea800d980 tx=0x7f0ea800dd40 comp rx=0 tx=0).stop 2023-12-01T01:17:19.232 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.221+0000 7f0ebdccf700 1 -- 172.21.15.77:0/2204230169 shutdown_connections 2023-12-01T01:17:19.232 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.221+0000 7f0ebdccf700 1 --2- 172.21.15.77:0/2204230169 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f0ea00a05b0 0x7f0ea00a2a70 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.232 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.221+0000 7f0ebdccf700 1 --2- 172.21.15.77:0/2204230169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb8102980 0x7f0eb80736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.232 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.221+0000 7f0ebdccf700 1 --2- 172.21.15.77:0/2204230169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0eb8100df0 0x7f0eb8074b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.232 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.221+0000 7f0ebdccf700 1 --2- 172.21.15.77:0/2204230169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0eb8101fe0 0x7f0eb8073180 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.232 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.221+0000 7f0ebdccf700 1 -- 172.21.15.77:0/2204230169 >> 172.21.15.77:0/2204230169 conn(0x7f0eb80fc360 msgr2=0x7f0eb8105510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:19.232 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.221+0000 7f0ebdccf700 1 -- 172.21.15.77:0/2204230169 shutdown_connections 2023-12-01T01:17:19.232 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.221+0000 7f0ebdccf700 1 -- 172.21.15.77:0/2204230169 wait complete. 2023-12-01T01:17:19.235 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-12-01T01:17:19.359 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.355+0000 7fa13ab4c700 1 -- 172.21.15.77:0/3012797971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa134101800 msgr2=0x7fa134103bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.359 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.355+0000 7fa13ab4c700 1 --2- 172.21.15.77:0/3012797971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa134101800 0x7fa134103bf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa128009990 tx=0x7fa128009ca0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.359 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.355+0000 7fa13ab4c700 1 -- 172.21.15.77:0/3012797971 shutdown_connections 2023-12-01T01:17:19.359 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.356+0000 7fa13ab4c700 1 --2- 172.21.15.77:0/3012797971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa134106a60 0x7fa134108ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.359 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.356+0000 7fa13ab4c700 1 --2- 172.21.15.77:0/3012797971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa134101800 0x7fa134103bf0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.360 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.356+0000 7fa13ab4c700 1 --2- 172.21.15.77:0/3012797971 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa134104130 0x7fa134106520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.360 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.356+0000 7fa13ab4c700 1 -- 172.21.15.77:0/3012797971 >> 172.21.15.77:0/3012797971 conn(0x7fa1340fb1f0 msgr2=0x7fa1340fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:19.360 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.356+0000 7fa13ab4c700 1 -- 172.21.15.77:0/3012797971 shutdown_connections 2023-12-01T01:17:19.360 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.356+0000 7fa13ab4c700 1 -- 172.21.15.77:0/3012797971 wait complete. 2023-12-01T01:17:19.360 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.356+0000 7fa13ab4c700 1 Processor -- start 2023-12-01T01:17:19.360 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.356+0000 7fa13ab4c700 1 -- start start 2023-12-01T01:17:19.360 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.356+0000 7fa13ab4c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa134101800 0x7fa134198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.360 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.356+0000 7fa13ab4c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa134104130 0x7fa134199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.360 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.356+0000 7fa13ab4c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa134106a60 0x7fa1341a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.360 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.356+0000 7fa13ab4c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1341a0840 con 0x7fa134104130 2023-12-01T01:17:19.360 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.356+0000 7fa13ab4c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1341a09b0 con 0x7fa134101800 2023-12-01T01:17:19.360 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.356+0000 7fa13ab4c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1341a0b70 con 0x7fa134106a60 2023-12-01T01:17:19.360 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.356+0000 7fa133fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa134104130 0x7fa134199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.360 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.356+0000 7fa133fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa134104130 0x7fa134199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58358/0 (socket says 172.21.15.77:58358) 2023-12-01T01:17:19.361 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.356+0000 7fa133fff700 1 -- 172.21.15.77:0/2832920502 learned_addr learned my addr 172.21.15.77:0/2832920502 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:19.361 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.357+0000 7fa1390e9700 1 --2- 172.21.15.77:0/2832920502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa134106a60 0x7fa1341a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.361 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.357+0000 7fa1388e8700 1 --2- 172.21.15.77:0/2832920502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa134101800 0x7fa134198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.361 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.357+0000 7fa133fff700 1 -- 172.21.15.77:0/2832920502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa134106a60 msgr2=0x7fa1341a0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.361 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.357+0000 7fa133fff700 1 --2- 172.21.15.77:0/2832920502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa134106a60 0x7fa1341a0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.361 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.357+0000 7fa133fff700 1 -- 172.21.15.77:0/2832920502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa134101800 msgr2=0x7fa134198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.361 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.357+0000 7fa133fff700 1 --2- 172.21.15.77:0/2832920502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa134101800 0x7fa134198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.361 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.357+0000 7fa133fff700 1 -- 172.21.15.77:0/2832920502 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1280095f0 con 0x7fa134104130 2023-12-01T01:17:19.361 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.357+0000 7fa1388e8700 1 --2- 172.21.15.77:0/2832920502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa134101800 0x7fa134198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:17:19.361 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.357+0000 7fa133fff700 1 --2- 172.21.15.77:0/2832920502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa134104130 0x7fa134199240 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa12000c920 tx=0x7fa12000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:19.361 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.357+0000 7fa131ffb700 1 -- 172.21.15.77:0/2832920502 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa120007b00 con 0x7fa134104130 2023-12-01T01:17:19.361 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.357+0000 7fa131ffb700 1 -- 172.21.15.77:0/2832920502 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa12000ce70 con 0x7fa134104130 2023-12-01T01:17:19.361 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.357+0000 7fa13ab4c700 1 -- 172.21.15.77:0/2832920502 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1341a0e50 con 0x7fa134104130 2023-12-01T01:17:19.361 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.357+0000 7fa131ffb700 1 -- 172.21.15.77:0/2832920502 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1200054e0 con 0x7fa134104130 2023-12-01T01:17:19.362 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.357+0000 7fa13ab4c700 1 -- 172.21.15.77:0/2832920502 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1341a1270 con 0x7fa134104130 2023-12-01T01:17:19.368 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.359+0000 7fa131ffb700 1 -- 172.21.15.77:0/2832920502 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fa120004750 con 0x7fa134104130 2023-12-01T01:17:19.368 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.360+0000 7fa131ffb700 1 --2- 172.21.15.77:0/2832920502 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa1240a05b0 0x7fa1240a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.368 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.360+0000 7fa131ffb700 1 -- 172.21.15.77:0/2832920502 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa1200cb520 con 0x7fa134104130 2023-12-01T01:17:19.368 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.360+0000 7fa1388e8700 1 --2- 172.21.15.77:0/2832920502 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa1240a05b0 0x7fa1240a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.368 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.360+0000 7fa1388e8700 1 --2- 172.21.15.77:0/2832920502 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa1240a05b0 0x7fa1240a2a70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa12800b5c0 tx=0x7fa128005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:19.368 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.361+0000 7fa146cdcb80 1 Processor -- start 2023-12-01T01:17:19.368 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.361+0000 7fa146cdcb80 1 -- start start 2023-12-01T01:17:19.368 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.361+0000 7fa146cdcb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561426d850a0 0x561426d854e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.368 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.361+0000 7fa146cdcb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561426d85a20 0x561426d83e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.368 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.361+0000 7fa146cdcb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561426d843a0 0x561426d84820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.368 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.361+0000 7fa146cdcb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x561426cdb9c0 con 0x561426d850a0 2023-12-01T01:17:19.369 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.361+0000 7fa146cdcb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x561426cd9380 con 0x561426d85a20 2023-12-01T01:17:19.369 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.361+0000 7fa146cdcb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x561426cd4c70 con 0x561426d843a0 2023-12-01T01:17:19.369 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.361+0000 7fa1388e8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561426d850a0 0x561426d854e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.369 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.361+0000 7fa1388e8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561426d850a0 0x561426d854e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58364/0 (socket says 172.21.15.77:58364) 2023-12-01T01:17:19.369 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.361+0000 7fa1388e8700 1 -- 172.21.15.77:0/2578899618 learned_addr learned my addr 172.21.15.77:0/2578899618 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:19.369 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.361+0000 7fa1390e9700 1 --2- 172.21.15.77:0/2578899618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561426d843a0 0x561426d84820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.369 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.362+0000 7fa1388e8700 1 -- 172.21.15.77:0/2578899618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561426d843a0 msgr2=0x561426d84820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:19.369 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.362+0000 7fa1388e8700 1 --2- 172.21.15.77:0/2578899618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561426d843a0 0x561426d84820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.369 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.362+0000 7fa1388e8700 1 -- 172.21.15.77:0/2578899618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561426d85a20 msgr2=0x561426d83e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:19.369 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.362+0000 7fa1388e8700 1 --2- 172.21.15.77:0/2578899618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561426d85a20 0x561426d83e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.369 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.362+0000 7fa1388e8700 1 -- 172.21.15.77:0/2578899618 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561426b02d90 con 0x561426d850a0 2023-12-01T01:17:19.369 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.362+0000 7fa1388e8700 1 --2- 172.21.15.77:0/2578899618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561426d850a0 0x561426d854e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa128015000 tx=0x7fa12801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87b4e14c49955ab6 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:19.369 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.362+0000 7fa11effd700 1 -- 172.21.15.77:0/2578899618 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12802b070 con 0x561426d850a0 2023-12-01T01:17:19.369 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.362+0000 7fa11effd700 1 -- 172.21.15.77:0/2578899618 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa12801e040 con 0x561426d850a0 2023-12-01T01:17:19.370 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.362+0000 7fa11effd700 1 -- 172.21.15.77:0/2578899618 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa128005030 con 0x561426d850a0 2023-12-01T01:17:19.370 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.362+0000 7fa146cdcb80 1 -- 172.21.15.77:0/2578899618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561426d850a0 msgr2=0x561426d854e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.370 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.362+0000 7fa146cdcb80 1 --2- 172.21.15.77:0/2578899618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561426d850a0 0x561426d854e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa128015000 tx=0x7fa12801f000 comp rx=0 tx=0).stop 2023-12-01T01:17:19.370 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.363+0000 7fa146cdcb80 1 -- 172.21.15.77:0/2578899618 shutdown_connections 2023-12-01T01:17:19.370 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.363+0000 7fa146cdcb80 1 --2- 172.21.15.77:0/2578899618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561426d843a0 0x561426d84820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.370 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.363+0000 7fa146cdcb80 1 --2- 172.21.15.77:0/2578899618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561426d850a0 0x561426d854e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.370 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.363+0000 7fa146cdcb80 1 --2- 172.21.15.77:0/2578899618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561426d85a20 0x561426d83e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.370 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.363+0000 7fa146cdcb80 1 -- 172.21.15.77:0/2578899618 >> 172.21.15.77:0/2578899618 conn(0x561426ce3080 msgr2=0x561426d7e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:19.370 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.363+0000 7fa146cdcb80 1 -- 172.21.15.77:0/2578899618 shutdown_connections 2023-12-01T01:17:19.370 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.363+0000 7fa146cdcb80 1 -- 172.21.15.77:0/2578899618 wait complete. 2023-12-01T01:17:19.370 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.363+0000 7fa146cdcb80 1 -- start start 2023-12-01T01:17:19.370 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.363+0000 7fa146cdcb80 1 Processor -- start 2023-12-01T01:17:19.370 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.363+0000 7fa146cdcb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561426d843a0 0x561426d8d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.371 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.363+0000 7fa146cdcb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561426d850a0 0x561426d8deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.371 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.363+0000 7fa146cdcb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561426d85a20 0x561426d94ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.371 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.363+0000 7fa146cdcb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x561426c714c0 con 0x561426d85a20 2023-12-01T01:17:19.371 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.363+0000 7fa146cdcb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x561426c71160 con 0x561426d850a0 2023-12-01T01:17:19.371 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.363+0000 7fa146cdcb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x561426b588d0 con 0x561426d843a0 2023-12-01T01:17:19.371 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.363+0000 7fa146cdcb80 10 client.0 _ll_register_callbacks cb 0x561426bbea30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:17:19.371 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.363+0000 7fa1390e9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561426d85a20 0x561426d94ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.371 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.364+0000 7fa1390e9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561426d85a20 0x561426d94ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58374/0 (socket says 172.21.15.77:58374) 2023-12-01T01:17:19.371 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.364+0000 7fa1390e9700 1 -- 172.21.15.77:0/3078328215 learned_addr learned my addr 172.21.15.77:0/3078328215 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:19.371 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.364+0000 7fa1388e8700 1 --2- 172.21.15.77:0/3078328215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561426d843a0 0x561426d8d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.371 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.364+0000 7fa133fff700 1 --2- 172.21.15.77:0/3078328215 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561426d850a0 0x561426d8deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.371 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.364+0000 7fa1390e9700 1 -- 172.21.15.77:0/3078328215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561426d843a0 msgr2=0x561426d8d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:19.371 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.364+0000 7fa1390e9700 1 --2- 172.21.15.77:0/3078328215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561426d843a0 0x561426d8d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.371 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.364+0000 7fa1390e9700 1 -- 172.21.15.77:0/3078328215 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561426d850a0 msgr2=0x561426d8deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:19.372 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.364+0000 7fa1390e9700 1 --2- 172.21.15.77:0/3078328215 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561426d850a0 0x561426d8deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.372 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.364+0000 7fa1390e9700 1 -- 172.21.15.77:0/3078328215 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa134109ac0 con 0x561426d85a20 2023-12-01T01:17:19.372 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.364+0000 7fa1390e9700 1 --2- 172.21.15.77:0/3078328215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561426d85a20 0x561426d94ef0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa12c0040e0 tx=0x7fa12c0043f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f2ad3f54d15c25 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:19.372 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.364+0000 7fa113fff700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:17:19.372 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.364+0000 7fa113fff700 1 -- 172.21.15.77:0/3078328215 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12c003b90 con 0x561426d85a20 2023-12-01T01:17:19.372 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.364+0000 7fa113fff700 1 -- 172.21.15.77:0/3078328215 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa12c007910 con 0x561426d85a20 2023-12-01T01:17:19.372 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.364+0000 7fa146cdcb80 1 -- 172.21.15.77:0/3078328215 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561426cffcc0 con 0x561426d85a20 2023-12-01T01:17:19.372 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.364+0000 7fa113fff700 1 -- 172.21.15.77:0/3078328215 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa12c007ad0 con 0x561426d85a20 2023-12-01T01:17:19.372 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.364+0000 7fa146cdcb80 10 client.15078 fetch_fsmap learned FSMap version 27 2023-12-01T01:17:19.372 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.364+0000 7fa146cdcb80 1 -- 172.21.15.77:0/3078328215 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561426aebc10 con 0x561426d85a20 2023-12-01T01:17:19.372 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.364+0000 7fa113fff700 1 -- 172.21.15.77:0/3078328215 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561426cffcc0 con 0x561426d85a20 2023-12-01T01:17:19.372 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.364+0000 7fa113fff700 1 -- 172.21.15.77:0/3078328215 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fa12c014040 con 0x561426d85a20 2023-12-01T01:17:19.372 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.365+0000 7fa146cdcb80 10 client.15078 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:17:19.373 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.365+0000 7fa146cdcb80 10 client.15078 resolve_mds: resolved name 'h' to daemon mds.h 2023-12-01T01:17:19.373 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.365+0000 7fa146cdcb80 20 client.15078 populate_metadata read hostname 'smithi077' 2023-12-01T01:17:19.373 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.365+0000 7fa146cdcb80 1 --2- 172.21.15.77:0/3078328215 >> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] conn(0x561426d98ca0 0x561426d98050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.373 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.365+0000 7fa146cdcb80 4 client.15078 mds_command: new command op to 24473 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:17:19.373 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.365+0000 7fa146cdcb80 1 -- 172.21.15.77:0/3078328215 --> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561426d52dc0 con 0x561426d98ca0 2023-12-01T01:17:19.373 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.365+0000 7fa1388e8700 1 --2- 172.21.15.77:0/3078328215 >> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] conn(0x561426d98ca0 0x561426d98050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.373 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.365+0000 7fa1388e8700 1 --2- 172.21.15.77:0/3078328215 >> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] conn(0x561426d98ca0 0x561426d98050 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9ea133f158300691 server_cookie=db65ef9b151746fd in_seq=0 out_seq=0 2023-12-01T01:17:19.373 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.365+0000 7fa113fff700 10 client.15078 ms_handle_connect on v2:172.21.15.137:6836/3518881867 2023-12-01T01:17:19.373 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.366+0000 7fa113fff700 1 -- 172.21.15.77:0/3078328215 <== mds.3 v2:172.21.15.137:6836/3518881867 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x561426d52dc0 con 0x561426d98ca0 2023-12-01T01:17:19.373 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.366+0000 7fa113fff700 10 client.15078 handle_command_reply: tid=0 2023-12-01T01:17:19.373 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.366+0000 7fa146cdcb80 1 client.15078 shutdown 2023-12-01T01:17:19.373 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.366+0000 7fa146cdcb80 1 -- 172.21.15.77:0/3078328215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561426d85a20 msgr2=0x561426d94ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.373 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.366+0000 7fa146cdcb80 1 --2- 172.21.15.77:0/3078328215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561426d85a20 0x561426d94ef0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa12c0040e0 tx=0x7fa12c0043f0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.373 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.366+0000 7fa146cdcb80 1 -- 172.21.15.77:0/3078328215 shutdown_connections 2023-12-01T01:17:19.374 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.366+0000 7fa146cdcb80 1 --2- 172.21.15.77:0/3078328215 >> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] conn(0x561426d98ca0 0x561426d98050 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.374 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.366+0000 7fa146cdcb80 1 --2- 172.21.15.77:0/3078328215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561426d85a20 0x561426d94ef0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.374 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.366+0000 7fa113fff700 0 client.15078 ms_handle_reset on v2:172.21.15.137:6836/3518881867 2023-12-01T01:17:19.374 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.366+0000 7fa146cdcb80 1 --2- 172.21.15.77:0/3078328215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561426d843a0 0x561426d8d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.374 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.366+0000 7fa146cdcb80 1 --2- 172.21.15.77:0/3078328215 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561426d850a0 0x561426d8deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.374 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.366+0000 7fa146cdcb80 1 -- 172.21.15.77:0/3078328215 >> 172.21.15.77:0/3078328215 conn(0x561426ce3080 msgr2=0x561426d7e060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:19.374 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.366+0000 7fa146cdcb80 1 -- 172.21.15.77:0/3078328215 shutdown_connections 2023-12-01T01:17:19.374 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.366+0000 7fa146cdcb80 1 -- 172.21.15.77:0/3078328215 wait complete. 2023-12-01T01:17:19.374 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.367+0000 7fa146cdcb80 20 client.15078 trim_cache size 0 max 16384 2023-12-01T01:17:19.392 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.384+0000 7fa146cdcb80 1 Processor -- start 2023-12-01T01:17:19.392 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.384+0000 7fa146cdcb80 1 -- start start 2023-12-01T01:17:19.392 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.384+0000 7fa146cdcb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561426ce3080 0x561426d83e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.392 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.384+0000 7fa146cdcb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561426d843a0 0x561426da19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.392 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.384+0000 7fa146cdcb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561426d850a0 0x561426d8b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.392 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.384+0000 7fa146cdcb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x561426d9a140 con 0x561426ce3080 2023-12-01T01:17:19.392 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.384+0000 7fa146cdcb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x561426d7f810 con 0x561426d850a0 2023-12-01T01:17:19.392 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.384+0000 7fa146cdcb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x561426d79610 con 0x561426d843a0 2023-12-01T01:17:19.392 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.384+0000 7fa1388e8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561426ce3080 0x561426d83e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.392 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.384+0000 7fa1388e8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561426ce3080 0x561426d83e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58378/0 (socket says 172.21.15.77:58378) 2023-12-01T01:17:19.392 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.384+0000 7fa1388e8700 1 -- 172.21.15.77:0/1491889600 learned_addr learned my addr 172.21.15.77:0/1491889600 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:19.393 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.384+0000 7fa1390e9700 1 --2- 172.21.15.77:0/1491889600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561426d850a0 0x561426d8b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.393 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.384+0000 7fa1388e8700 1 -- 172.21.15.77:0/1491889600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561426d843a0 msgr2=0x561426da19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:19.393 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.384+0000 7fa1388e8700 1 --2- 172.21.15.77:0/1491889600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561426d843a0 0x561426da19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.393 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.384+0000 7fa1388e8700 1 -- 172.21.15.77:0/1491889600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561426d850a0 msgr2=0x561426d8b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:19.393 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.384+0000 7fa1388e8700 1 --2- 172.21.15.77:0/1491889600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561426d850a0 0x561426d8b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.393 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.384+0000 7fa1388e8700 1 -- 172.21.15.77:0/1491889600 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561426d84a50 con 0x561426ce3080 2023-12-01T01:17:19.393 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.384+0000 7fa1390e9700 1 --2- 172.21.15.77:0/1491889600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561426d850a0 0x561426d8b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:17:19.393 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.384+0000 7fa1388e8700 1 --2- 172.21.15.77:0/1491889600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561426ce3080 0x561426d83e10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa128015000 tx=0x7fa128022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aa8fb71ad866f24 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:19.393 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.384+0000 7fa11f7fe700 1 -- 172.21.15.77:0/1491889600 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12802b040 con 0x561426ce3080 2023-12-01T01:17:19.393 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.384+0000 7fa11f7fe700 1 -- 172.21.15.77:0/1491889600 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa12802c040 con 0x561426ce3080 2023-12-01T01:17:19.393 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.384+0000 7fa11f7fe700 1 -- 172.21.15.77:0/1491889600 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1280052e0 con 0x561426ce3080 2023-12-01T01:17:19.393 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.385+0000 7fa146cdcb80 1 -- 172.21.15.77:0/1491889600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561426ce3080 msgr2=0x561426d83e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.393 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.385+0000 7fa146cdcb80 1 --2- 172.21.15.77:0/1491889600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561426ce3080 0x561426d83e10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa128015000 tx=0x7fa128022000 comp rx=0 tx=0).stop 2023-12-01T01:17:19.393 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.385+0000 7fa146cdcb80 1 -- 172.21.15.77:0/1491889600 shutdown_connections 2023-12-01T01:17:19.394 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.385+0000 7fa146cdcb80 1 --2- 172.21.15.77:0/1491889600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561426d850a0 0x561426d8b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.394 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.385+0000 7fa146cdcb80 1 --2- 172.21.15.77:0/1491889600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561426ce3080 0x561426d83e10 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.394 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.385+0000 7fa146cdcb80 1 --2- 172.21.15.77:0/1491889600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561426d843a0 0x561426da19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.394 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.385+0000 7fa146cdcb80 1 -- 172.21.15.77:0/1491889600 >> 172.21.15.77:0/1491889600 conn(0x561426d7ec20 msgr2=0x561426d883c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:19.394 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.385+0000 7fa146cdcb80 1 -- 172.21.15.77:0/1491889600 shutdown_connections 2023-12-01T01:17:19.394 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.385+0000 7fa146cdcb80 1 -- 172.21.15.77:0/1491889600 wait complete. 2023-12-01T01:17:19.394 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.385+0000 7fa146cdcb80 1 -- start start 2023-12-01T01:17:19.394 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.385+0000 7fa146cdcb80 1 Processor -- start 2023-12-01T01:17:19.394 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.386+0000 7fa146cdcb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561426ce3080 0x561426d9e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.394 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.386+0000 7fa146cdcb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561426d843a0 0x561426d9ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.394 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.386+0000 7fa146cdcb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561426d850a0 0x561426dea750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.386+0000 7fa146cdcb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x561426da1b10 con 0x561426d843a0 2023-12-01T01:17:19.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.386+0000 7fa146cdcb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x561426da1c80 con 0x561426d850a0 2023-12-01T01:17:19.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.386+0000 7fa146cdcb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x561426deac90 con 0x561426ce3080 2023-12-01T01:17:19.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.386+0000 7fa146cdcb80 10 client.0 _ll_register_callbacks cb 0x7fa12c009950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:17:19.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.386+0000 7fa133fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561426d843a0 0x561426d9ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.386+0000 7fa133fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561426d843a0 0x561426d9ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58386/0 (socket says 172.21.15.77:58386) 2023-12-01T01:17:19.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.386+0000 7fa133fff700 1 -- 172.21.15.77:0/711603117 learned_addr learned my addr 172.21.15.77:0/711603117 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:19.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.386+0000 7fa1388e8700 1 --2- 172.21.15.77:0/711603117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561426ce3080 0x561426d9e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.386+0000 7fa1390e9700 1 --2- 172.21.15.77:0/711603117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561426d850a0 0x561426dea750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.386+0000 7fa133fff700 1 -- 172.21.15.77:0/711603117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561426ce3080 msgr2=0x561426d9e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:19.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.386+0000 7fa133fff700 1 --2- 172.21.15.77:0/711603117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561426ce3080 0x561426d9e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.386+0000 7fa133fff700 1 -- 172.21.15.77:0/711603117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561426d850a0 msgr2=0x561426dea750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:19.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.386+0000 7fa133fff700 1 --2- 172.21.15.77:0/711603117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561426d850a0 0x561426dea750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.386+0000 7fa133fff700 1 -- 172.21.15.77:0/711603117 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561426b02d90 con 0x561426d843a0 2023-12-01T01:17:19.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.386+0000 7fa133fff700 1 --2- 172.21.15.77:0/711603117 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561426d843a0 0x561426d9ebd0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa1200074a0 tx=0x7fa1200987a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a219fd54a8282b64 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:19.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.386+0000 7fa113fff700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:17:19.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.386+0000 7fa113fff700 1 -- 172.21.15.77:0/711603117 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa120098dd0 con 0x561426d843a0 2023-12-01T01:17:19.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.387+0000 7fa113fff700 1 -- 172.21.15.77:0/711603117 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa12009cc00 con 0x561426d843a0 2023-12-01T01:17:19.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.387+0000 7fa146cdcb80 1 -- 172.21.15.77:0/711603117 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa12801e040 con 0x561426d843a0 2023-12-01T01:17:19.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.387+0000 7fa113fff700 1 -- 172.21.15.77:0/711603117 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa120061050 con 0x561426d843a0 2023-12-01T01:17:19.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.387+0000 7fa146cdcb80 10 client.15084 fetch_fsmap learned FSMap version 27 2023-12-01T01:17:19.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.387+0000 7fa146cdcb80 1 -- 172.21.15.77:0/711603117 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561426b29200 con 0x561426d843a0 2023-12-01T01:17:19.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.387+0000 7fa113fff700 1 -- 172.21.15.77:0/711603117 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa12801e040 con 0x561426d843a0 2023-12-01T01:17:19.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.387+0000 7fa113fff700 1 -- 172.21.15.77:0/711603117 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fa1200a09b0 con 0x561426d843a0 2023-12-01T01:17:19.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.387+0000 7fa146cdcb80 10 client.15084 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:17:19.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.387+0000 7fa146cdcb80 10 client.15084 resolve_mds: resolved name 'h' to daemon mds.h 2023-12-01T01:17:19.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.387+0000 7fa146cdcb80 20 client.15084 populate_metadata read hostname 'smithi077' 2023-12-01T01:17:19.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.387+0000 7fa146cdcb80 1 --2- 172.21.15.77:0/711603117 >> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] conn(0x561426d85a20 0x561426dec100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.387+0000 7fa146cdcb80 4 client.15084 mds_command: new command op to 24473 tid=0[{"prefix": "status"}] 2023-12-01T01:17:19.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.387+0000 7fa146cdcb80 1 -- 172.21.15.77:0/711603117 --> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561426ccdcf0 con 0x561426d85a20 2023-12-01T01:17:19.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.387+0000 7fa1388e8700 1 --2- 172.21.15.77:0/711603117 >> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] conn(0x561426d85a20 0x561426dec100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.388+0000 7fa113fff700 10 client.15084 ms_handle_connect on v2:172.21.15.137:6836/3518881867 2023-12-01T01:17:19.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.388+0000 7fa1388e8700 1 --2- 172.21.15.77:0/711603117 >> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] conn(0x561426d85a20 0x561426dec100 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f69a6765cb8ee493 server_cookie=57eabcc50874c0b9 in_seq=0 out_seq=0 2023-12-01T01:17:19.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.388+0000 7fa113fff700 1 -- 172.21.15.77:0/711603117 <== mds.3 v2:172.21.15.137:6836/3518881867 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x561426ccdcf0 con 0x561426d85a20 2023-12-01T01:17:19.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.388+0000 7fa113fff700 10 client.15084 handle_command_reply: tid=0 2023-12-01T01:17:19.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.388+0000 7fa146cdcb80 1 client.15084 shutdown 2023-12-01T01:17:19.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.388+0000 7fa146cdcb80 1 -- 172.21.15.77:0/711603117 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561426d843a0 msgr2=0x561426d9ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.388+0000 7fa146cdcb80 1 --2- 172.21.15.77:0/711603117 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561426d843a0 0x561426d9ebd0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa1200074a0 tx=0x7fa1200987a0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.388+0000 7fa146cdcb80 1 -- 172.21.15.77:0/711603117 shutdown_connections 2023-12-01T01:17:19.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.388+0000 7fa146cdcb80 1 --2- 172.21.15.77:0/711603117 >> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] conn(0x561426d85a20 0x561426dec100 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.398 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.388+0000 7fa146cdcb80 1 --2- 172.21.15.77:0/711603117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561426d850a0 0x561426dea750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.398 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.388+0000 7fa113fff700 0 client.15084 ms_handle_reset on v2:172.21.15.137:6836/3518881867 2023-12-01T01:17:19.398 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.388+0000 7fa146cdcb80 1 --2- 172.21.15.77:0/711603117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561426ce3080 0x561426d9e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.398 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.388+0000 7fa146cdcb80 1 --2- 172.21.15.77:0/711603117 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561426d843a0 0x561426d9ebd0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.398 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.388+0000 7fa146cdcb80 1 -- 172.21.15.77:0/711603117 >> 172.21.15.77:0/711603117 conn(0x561426d7ec20 msgr2=0x561426d8b870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:19.398 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.388+0000 7fa146cdcb80 1 -- 172.21.15.77:0/711603117 shutdown_connections 2023-12-01T01:17:19.399 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.388+0000 7fa146cdcb80 1 -- 172.21.15.77:0/711603117 wait complete. 2023-12-01T01:17:19.399 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.388+0000 7fa146cdcb80 20 client.15084 trim_cache size 0 max 16384 2023-12-01T01:17:19.399 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2023-12-01T01:17:19.400 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "2a06e29a-8fe6-11ee-95a2-87774f69a715", 2023-12-01T01:17:19.400 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 3, 2023-12-01T01:17:19.400 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24473, 2023-12-01T01:17:19.400 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2023-12-01T01:17:19.400 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2023-12-01T01:17:19.400 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2023-12-01T01:17:19.400 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 48.744801981999998, 2023-12-01T01:17:19.400 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 27, 2023-12-01T01:17:19.401 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 82, 2023-12-01T01:17:19.401 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 82, 2023-12-01T01:17:19.401 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 68.187791700000005 2023-12-01T01:17:19.401 INFO:teuthology.orchestra.run.smithi077.stdout:} 2023-12-01T01:17:19.402 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.389+0000 7fa13ab4c700 1 -- 172.21.15.77:0/2832920502 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa1240a05b0 msgr2=0x7fa1240a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.402 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.389+0000 7fa13ab4c700 1 --2- 172.21.15.77:0/2832920502 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa1240a05b0 0x7fa1240a2a70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa12800b5c0 tx=0x7fa128005fd0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.402 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.389+0000 7fa13ab4c700 1 -- 172.21.15.77:0/2832920502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa134104130 msgr2=0x7fa134199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.402 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.389+0000 7fa13ab4c700 1 --2- 172.21.15.77:0/2832920502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa134104130 0x7fa134199240 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa12000c920 tx=0x7fa12000cc30 comp rx=0 tx=0).stop 2023-12-01T01:17:19.403 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.389+0000 7fa13ab4c700 1 -- 172.21.15.77:0/2832920502 shutdown_connections 2023-12-01T01:17:19.403 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.389+0000 7fa13ab4c700 1 --2- 172.21.15.77:0/2832920502 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa1240a05b0 0x7fa1240a2a70 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.403 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.389+0000 7fa13ab4c700 1 --2- 172.21.15.77:0/2832920502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa134106a60 0x7fa1341a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.403 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.389+0000 7fa13ab4c700 1 --2- 172.21.15.77:0/2832920502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa134101800 0x7fa134198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.403 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.389+0000 7fa13ab4c700 1 --2- 172.21.15.77:0/2832920502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa134104130 0x7fa134199240 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.403 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.389+0000 7fa13ab4c700 1 -- 172.21.15.77:0/2832920502 >> 172.21.15.77:0/2832920502 conn(0x7fa1340fb1f0 msgr2=0x7fa134100780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:19.403 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.389+0000 7fa13ab4c700 1 -- 172.21.15.77:0/2832920502 shutdown_connections 2023-12-01T01:17:19.403 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.389+0000 7fa13ab4c700 1 -- 172.21.15.77:0/2832920502 wait complete. 2023-12-01T01:17:19.403 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-12-01T01:17:19.528 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.525+0000 7fa75fe56700 1 -- 172.21.15.77:0/3090305701 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa758100e70 msgr2=0x7fa7581012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.528 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.525+0000 7fa75fe56700 1 --2- 172.21.15.77:0/3090305701 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa758100e70 0x7fa7581012d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa7540099c0 tx=0x7fa754009cd0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.529 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.525+0000 7fa75fe56700 1 -- 172.21.15.77:0/3090305701 shutdown_connections 2023-12-01T01:17:19.529 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.525+0000 7fa75fe56700 1 --2- 172.21.15.77:0/3090305701 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa758101810 0x7fa7581080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.529 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.525+0000 7fa75fe56700 1 --2- 172.21.15.77:0/3090305701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7580ffc80 0x7fa7581000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.529 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.525+0000 7fa75fe56700 1 --2- 172.21.15.77:0/3090305701 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa758100e70 0x7fa7581012d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.529 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.525+0000 7fa75fe56700 1 -- 172.21.15.77:0/3090305701 >> 172.21.15.77:0/3090305701 conn(0x7fa7580fb1f0 msgr2=0x7fa7580fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:19.529 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.525+0000 7fa75fe56700 1 -- 172.21.15.77:0/3090305701 shutdown_connections 2023-12-01T01:17:19.529 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.525+0000 7fa75fe56700 1 -- 172.21.15.77:0/3090305701 wait complete. 2023-12-01T01:17:19.529 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.526+0000 7fa75fe56700 1 Processor -- start 2023-12-01T01:17:19.529 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.526+0000 7fa75fe56700 1 -- start start 2023-12-01T01:17:19.529 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.526+0000 7fa75fe56700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7580ffc80 0x7fa758068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.529 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.526+0000 7fa75fe56700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa758100e70 0x7fa75819b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.529 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.526+0000 7fa75fe56700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa758101810 0x7fa75819ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.530 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.526+0000 7fa75fe56700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75819e5c0 con 0x7fa758100e70 2023-12-01T01:17:19.530 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.526+0000 7fa75fe56700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75819e730 con 0x7fa758101810 2023-12-01T01:17:19.530 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.526+0000 7fa75fe56700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75819e8f0 con 0x7fa7580ffc80 2023-12-01T01:17:19.530 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.526+0000 7fa75d3f1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa758100e70 0x7fa75819b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.530 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.526+0000 7fa75d3f1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa758100e70 0x7fa75819b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58400/0 (socket says 172.21.15.77:58400) 2023-12-01T01:17:19.530 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.526+0000 7fa75d3f1700 1 -- 172.21.15.77:0/2902584186 learned_addr learned my addr 172.21.15.77:0/2902584186 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:19.530 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.526+0000 7fa75dbf2700 1 --2- 172.21.15.77:0/2902584186 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7580ffc80 0x7fa758068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.530 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.526+0000 7fa75d3f1700 1 -- 172.21.15.77:0/2902584186 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7580ffc80 msgr2=0x7fa758068a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.530 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.526+0000 7fa75d3f1700 1 --2- 172.21.15.77:0/2902584186 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7580ffc80 0x7fa758068a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.530 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.526+0000 7fa75d3f1700 1 -- 172.21.15.77:0/2902584186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa758101810 msgr2=0x7fa75819ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.530 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.526+0000 7fa75e3f3700 1 --2- 172.21.15.77:0/2902584186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa758101810 0x7fa75819ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.530 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.526+0000 7fa75d3f1700 1 --2- 172.21.15.77:0/2902584186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa758101810 0x7fa75819ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.531 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.526+0000 7fa75d3f1700 1 -- 172.21.15.77:0/2902584186 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7540095f0 con 0x7fa758100e70 2023-12-01T01:17:19.531 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.527+0000 7fa75d3f1700 1 --2- 172.21.15.77:0/2902584186 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa758100e70 0x7fa75819b7a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa754009fd0 tx=0x7fa75400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:19.531 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.527+0000 7fa74effd700 1 -- 172.21.15.77:0/2902584186 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75401d070 con 0x7fa758100e70 2023-12-01T01:17:19.531 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.527+0000 7fa74effd700 1 -- 172.21.15.77:0/2902584186 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7540054f0 con 0x7fa758100e70 2023-12-01T01:17:19.531 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.527+0000 7fa75fe56700 1 -- 172.21.15.77:0/2902584186 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa75819eb70 con 0x7fa758100e70 2023-12-01T01:17:19.531 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.527+0000 7fa74effd700 1 -- 172.21.15.77:0/2902584186 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa754004bd0 con 0x7fa758100e70 2023-12-01T01:17:19.531 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.527+0000 7fa75fe56700 1 -- 172.21.15.77:0/2902584186 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa75819ef90 con 0x7fa758100e70 2023-12-01T01:17:19.538 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.529+0000 7fa74effd700 1 -- 172.21.15.77:0/2902584186 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fa75400fb80 con 0x7fa758100e70 2023-12-01T01:17:19.538 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.529+0000 7fa74effd700 1 --2- 172.21.15.77:0/2902584186 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa7440a0630 0x7fa7440a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.538 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.529+0000 7fa75dbf2700 1 --2- 172.21.15.77:0/2902584186 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa7440a0630 0x7fa7440a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.538 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.529+0000 7fa74effd700 1 -- 172.21.15.77:0/2902584186 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa7540cc920 con 0x7fa758100e70 2023-12-01T01:17:19.538 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.530+0000 7fa75dbf2700 1 --2- 172.21.15.77:0/2902584186 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa7440a0630 0x7fa7440a2af0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa758100cd0 tx=0x7fa748007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:19.538 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.530+0000 7fa76bfe6b80 1 Processor -- start 2023-12-01T01:17:19.538 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.530+0000 7fa76bfe6b80 1 -- start start 2023-12-01T01:17:19.538 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.530+0000 7fa76bfe6b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9f58f60a0 0x55a9f58f64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.538 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.530+0000 7fa76bfe6b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a9f58f6a20 0x55a9f58f4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.538 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.531+0000 7fa76bfe6b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f58f53a0 0x55a9f58f5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.538 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.531+0000 7fa76bfe6b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f584c9c0 con 0x55a9f58f60a0 2023-12-01T01:17:19.538 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.531+0000 7fa76bfe6b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f584a380 con 0x55a9f58f53a0 2023-12-01T01:17:19.539 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.531+0000 7fa76bfe6b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f5845c70 con 0x55a9f58f6a20 2023-12-01T01:17:19.539 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.531+0000 7fa75dbf2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9f58f60a0 0x55a9f58f64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.539 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.531+0000 7fa75dbf2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9f58f60a0 0x55a9f58f64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58406/0 (socket says 172.21.15.77:58406) 2023-12-01T01:17:19.539 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.531+0000 7fa75dbf2700 1 -- 172.21.15.77:0/3721488274 learned_addr learned my addr 172.21.15.77:0/3721488274 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:19.539 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.531+0000 7fa75d3f1700 1 --2- 172.21.15.77:0/3721488274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a9f58f6a20 0x55a9f58f4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.539 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.531+0000 7fa75dbf2700 1 -- 172.21.15.77:0/3721488274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a9f58f6a20 msgr2=0x55a9f58f4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:19.539 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.531+0000 7fa75dbf2700 1 --2- 172.21.15.77:0/3721488274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a9f58f6a20 0x55a9f58f4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.539 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.531+0000 7fa75dbf2700 1 -- 172.21.15.77:0/3721488274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f58f53a0 msgr2=0x55a9f58f5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:19.539 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.531+0000 7fa75e3f3700 1 --2- 172.21.15.77:0/3721488274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f58f53a0 0x55a9f58f5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.539 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.531+0000 7fa75dbf2700 1 --2- 172.21.15.77:0/3721488274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f58f53a0 0x55a9f58f5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.540 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.531+0000 7fa75dbf2700 1 -- 172.21.15.77:0/3721488274 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9f5673d90 con 0x55a9f58f60a0 2023-12-01T01:17:19.540 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.531+0000 7fa75d3f1700 1 --2- 172.21.15.77:0/3721488274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a9f58f6a20 0x55a9f58f4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:17:19.540 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.531+0000 7fa75dbf2700 1 --2- 172.21.15.77:0/3721488274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9f58f60a0 0x55a9f58f64e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa748010e50 tx=0x7fa748013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3745389a65fa2827 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:19.540 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.531+0000 7fa743fff700 1 -- 172.21.15.77:0/3721488274 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa748004220 con 0x55a9f58f60a0 2023-12-01T01:17:19.540 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.531+0000 7fa743fff700 1 -- 172.21.15.77:0/3721488274 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa748015070 con 0x55a9f58f60a0 2023-12-01T01:17:19.540 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.531+0000 7fa743fff700 1 -- 172.21.15.77:0/3721488274 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74801e660 con 0x55a9f58f60a0 2023-12-01T01:17:19.540 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.532+0000 7fa76bfe6b80 1 -- 172.21.15.77:0/3721488274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9f58f60a0 msgr2=0x55a9f58f64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.540 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.532+0000 7fa76bfe6b80 1 --2- 172.21.15.77:0/3721488274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9f58f60a0 0x55a9f58f64e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa748010e50 tx=0x7fa748013000 comp rx=0 tx=0).stop 2023-12-01T01:17:19.540 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.532+0000 7fa76bfe6b80 1 -- 172.21.15.77:0/3721488274 shutdown_connections 2023-12-01T01:17:19.540 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.532+0000 7fa76bfe6b80 1 --2- 172.21.15.77:0/3721488274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f58f53a0 0x55a9f58f5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.540 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.532+0000 7fa76bfe6b80 1 --2- 172.21.15.77:0/3721488274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9f58f60a0 0x55a9f58f64e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.540 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.532+0000 7fa76bfe6b80 1 --2- 172.21.15.77:0/3721488274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a9f58f6a20 0x55a9f58f4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.540 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.532+0000 7fa76bfe6b80 1 -- 172.21.15.77:0/3721488274 >> 172.21.15.77:0/3721488274 conn(0x55a9f5854080 msgr2=0x55a9f58ef8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:19.541 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.532+0000 7fa76bfe6b80 1 -- 172.21.15.77:0/3721488274 shutdown_connections 2023-12-01T01:17:19.541 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.532+0000 7fa76bfe6b80 1 -- 172.21.15.77:0/3721488274 wait complete. 2023-12-01T01:17:19.541 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.532+0000 7fa76bfe6b80 1 -- start start 2023-12-01T01:17:19.541 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.532+0000 7fa76bfe6b80 1 Processor -- start 2023-12-01T01:17:19.541 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.533+0000 7fa76bfe6b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9f58f53a0 0x55a9f58fe930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.541 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.533+0000 7fa76bfe6b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a9f58f60a0 0x55a9f58fee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.541 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.533+0000 7fa76bfe6b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f58f6a20 0x55a9f5905eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.541 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.533+0000 7fa76bfe6b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f57e24c0 con 0x55a9f58f53a0 2023-12-01T01:17:19.541 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.533+0000 7fa76bfe6b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f57e2160 con 0x55a9f58f6a20 2023-12-01T01:17:19.541 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.533+0000 7fa76bfe6b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f56c98d0 con 0x55a9f58f60a0 2023-12-01T01:17:19.541 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.533+0000 7fa75dbf2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9f58f53a0 0x55a9f58fe930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.541 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.533+0000 7fa76bfe6b80 10 client.0 _ll_register_callbacks cb 0x55a9f572fa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:17:19.541 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.533+0000 7fa75dbf2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9f58f53a0 0x55a9f58fe930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58412/0 (socket says 172.21.15.77:58412) 2023-12-01T01:17:19.541 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.533+0000 7fa75dbf2700 1 -- 172.21.15.77:0/2643816385 learned_addr learned my addr 172.21.15.77:0/2643816385 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:19.542 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.533+0000 7fa75e3f3700 1 --2- 172.21.15.77:0/2643816385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f58f6a20 0x55a9f5905eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.542 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.533+0000 7fa75d3f1700 1 --2- 172.21.15.77:0/2643816385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a9f58f60a0 0x55a9f58fee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.542 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.533+0000 7fa75dbf2700 1 -- 172.21.15.77:0/2643816385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a9f58f60a0 msgr2=0x55a9f58fee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:19.542 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.533+0000 7fa75dbf2700 1 --2- 172.21.15.77:0/2643816385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a9f58f60a0 0x55a9f58fee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.542 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.533+0000 7fa75dbf2700 1 -- 172.21.15.77:0/2643816385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f58f6a20 msgr2=0x55a9f5905eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:19.542 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.533+0000 7fa75dbf2700 1 --2- 172.21.15.77:0/2643816385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f58f6a20 0x55a9f5905eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.542 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.533+0000 7fa75dbf2700 1 -- 172.21.15.77:0/2643816385 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa748010b30 con 0x55a9f58f53a0 2023-12-01T01:17:19.542 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.534+0000 7fa75dbf2700 1 --2- 172.21.15.77:0/2643816385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9f58f53a0 0x55a9f58fe930 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa748010ad0 tx=0x7fa748003920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4207be928cc3582d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:19.542 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.534+0000 7fa7417fa700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:17:19.542 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.534+0000 7fa7417fa700 1 -- 172.21.15.77:0/2643816385 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa748005460 con 0x55a9f58f53a0 2023-12-01T01:17:19.542 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.534+0000 7fa7417fa700 1 -- 172.21.15.77:0/2643816385 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa748017070 con 0x55a9f58f53a0 2023-12-01T01:17:19.542 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.534+0000 7fa76bfe6b80 1 -- 172.21.15.77:0/2643816385 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9f5870cc0 con 0x55a9f58f53a0 2023-12-01T01:17:19.542 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.534+0000 7fa7417fa700 1 -- 172.21.15.77:0/2643816385 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7480055c0 con 0x55a9f58f53a0 2023-12-01T01:17:19.543 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.534+0000 7fa76bfe6b80 10 client.15096 fetch_fsmap learned FSMap version 27 2023-12-01T01:17:19.543 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.534+0000 7fa76bfe6b80 1 -- 172.21.15.77:0/2643816385 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a9f565cc10 con 0x55a9f58f53a0 2023-12-01T01:17:19.543 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.534+0000 7fa7417fa700 1 -- 172.21.15.77:0/2643816385 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a9f5870cc0 con 0x55a9f58f53a0 2023-12-01T01:17:19.543 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.534+0000 7fa7417fa700 1 -- 172.21.15.77:0/2643816385 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fa748024070 con 0x55a9f58f53a0 2023-12-01T01:17:19.543 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.534+0000 7fa76bfe6b80 10 client.15096 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:17:19.543 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.534+0000 7fa76bfe6b80 10 client.15096 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-01T01:17:19.543 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.534+0000 7fa76bfe6b80 20 client.15096 populate_metadata read hostname 'smithi077' 2023-12-01T01:17:19.543 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.534+0000 7fa76bfe6b80 1 --2- 172.21.15.77:0/2643816385 >> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] conn(0x55a9f5909b70 0x55a9f5908fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.543 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.534+0000 7fa76bfe6b80 4 client.15096 mds_command: new command op to 24475 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:17:19.543 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.534+0000 7fa76bfe6b80 1 -- 172.21.15.77:0/2643816385 --> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a9f58c3dc0 con 0x55a9f5909b70 2023-12-01T01:17:19.543 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.535+0000 7fa75d3f1700 1 --2- 172.21.15.77:0/2643816385 >> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] conn(0x55a9f5909b70 0x55a9f5908fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.543 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.535+0000 7fa75d3f1700 1 --2- 172.21.15.77:0/2643816385 >> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] conn(0x55a9f5909b70 0x55a9f5908fa0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8b4f8df37aa4fd54 server_cookie=11b525f350c8e6b4 in_seq=0 out_seq=0 2023-12-01T01:17:19.543 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.535+0000 7fa7417fa700 10 client.15096 ms_handle_connect on v2:172.21.15.203:6836/2887080977 2023-12-01T01:17:19.543 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.536+0000 7fa7417fa700 1 -- 172.21.15.77:0/2643816385 <== mds.2 v2:172.21.15.203:6836/2887080977 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55a9f58c3dc0 con 0x55a9f5909b70 2023-12-01T01:17:19.544 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.536+0000 7fa7417fa700 10 client.15096 handle_command_reply: tid=0 2023-12-01T01:17:19.544 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.536+0000 7fa76bfe6b80 1 client.15096 shutdown 2023-12-01T01:17:19.544 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.536+0000 7fa76bfe6b80 1 -- 172.21.15.77:0/2643816385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9f58f53a0 msgr2=0x55a9f58fe930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.544 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.536+0000 7fa76bfe6b80 1 --2- 172.21.15.77:0/2643816385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9f58f53a0 0x55a9f58fe930 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa748010ad0 tx=0x7fa748003920 comp rx=0 tx=0).stop 2023-12-01T01:17:19.544 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.536+0000 7fa76bfe6b80 1 -- 172.21.15.77:0/2643816385 shutdown_connections 2023-12-01T01:17:19.544 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.536+0000 7fa76bfe6b80 1 --2- 172.21.15.77:0/2643816385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f58f6a20 0x55a9f5905eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.544 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.536+0000 7fa76bfe6b80 1 --2- 172.21.15.77:0/2643816385 >> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] conn(0x55a9f5909b70 0x55a9f5908fa0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.544 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.536+0000 7fa76bfe6b80 1 --2- 172.21.15.77:0/2643816385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9f58f53a0 0x55a9f58fe930 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.544 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.536+0000 7fa76bfe6b80 1 --2- 172.21.15.77:0/2643816385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a9f58f60a0 0x55a9f58fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.544 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.536+0000 7fa7417fa700 0 client.15096 ms_handle_reset on v2:172.21.15.203:6836/2887080977 2023-12-01T01:17:19.544 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.536+0000 7fa76bfe6b80 1 -- 172.21.15.77:0/2643816385 >> 172.21.15.77:0/2643816385 conn(0x55a9f5854080 msgr2=0x55a9f58ef060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:19.544 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.536+0000 7fa76bfe6b80 1 -- 172.21.15.77:0/2643816385 shutdown_connections 2023-12-01T01:17:19.544 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.536+0000 7fa76bfe6b80 1 -- 172.21.15.77:0/2643816385 wait complete. 2023-12-01T01:17:19.544 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.536+0000 7fa76bfe6b80 20 client.15096 trim_cache size 0 max 16384 2023-12-01T01:17:19.560 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.553+0000 7fa76bfe6b80 1 Processor -- start 2023-12-01T01:17:19.560 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.553+0000 7fa76bfe6b80 1 -- start start 2023-12-01T01:17:19.560 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.553+0000 7fa76bfe6b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9f58f53a0 0x55a9f58f4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.561 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.553+0000 7fa76bfe6b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a9f58f60a0 0x55a9f59128a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.561 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.553+0000 7fa76bfe6b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f58f6a20 0x55a9f58f9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.561 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.553+0000 7fa76bfe6b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f58fe500 con 0x55a9f58f53a0 2023-12-01T01:17:19.561 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.553+0000 7fa76bfe6b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f58c3dc0 con 0x55a9f58f6a20 2023-12-01T01:17:19.561 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.553+0000 7fa76bfe6b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f5870cc0 con 0x55a9f58f60a0 2023-12-01T01:17:19.561 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.553+0000 7fa75dbf2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9f58f53a0 0x55a9f58f4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.561 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.553+0000 7fa75dbf2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9f58f53a0 0x55a9f58f4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58420/0 (socket says 172.21.15.77:58420) 2023-12-01T01:17:19.561 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.553+0000 7fa75dbf2700 1 -- 172.21.15.77:0/4082843951 learned_addr learned my addr 172.21.15.77:0/4082843951 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:19.561 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.553+0000 7fa75d3f1700 1 --2- 172.21.15.77:0/4082843951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a9f58f60a0 0x55a9f59128a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.561 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.554+0000 7fa75e3f3700 1 --2- 172.21.15.77:0/4082843951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f58f6a20 0x55a9f58f9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.561 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.554+0000 7fa75dbf2700 1 -- 172.21.15.77:0/4082843951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a9f58f60a0 msgr2=0x55a9f59128a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:19.561 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.554+0000 7fa75dbf2700 1 --2- 172.21.15.77:0/4082843951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a9f58f60a0 0x55a9f59128a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.561 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.554+0000 7fa75dbf2700 1 -- 172.21.15.77:0/4082843951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f58f6a20 msgr2=0x55a9f58f9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:19.561 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.554+0000 7fa75dbf2700 1 --2- 172.21.15.77:0/4082843951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f58f6a20 0x55a9f58f9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.562 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.554+0000 7fa75dbf2700 1 -- 172.21.15.77:0/4082843951 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9f569a200 con 0x55a9f58f53a0 2023-12-01T01:17:19.562 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.554+0000 7fa75e3f3700 1 --2- 172.21.15.77:0/4082843951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f58f6a20 0x55a9f58f9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:17:19.562 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.554+0000 7fa75dbf2700 1 --2- 172.21.15.77:0/4082843951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9f58f53a0 0x55a9f58f4e10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa748010b00 tx=0x7fa748005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b24793e8a9637ec4 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:19.562 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.554+0000 7fa74cff9700 1 -- 172.21.15.77:0/4082843951 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa748003a40 con 0x55a9f58f53a0 2023-12-01T01:17:19.562 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.554+0000 7fa74cff9700 1 -- 172.21.15.77:0/4082843951 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7480079b0 con 0x55a9f58f53a0 2023-12-01T01:17:19.562 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.554+0000 7fa74cff9700 1 -- 172.21.15.77:0/4082843951 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7480319a0 con 0x55a9f58f53a0 2023-12-01T01:17:19.562 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.555+0000 7fa76bfe6b80 1 -- 172.21.15.77:0/4082843951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9f58f53a0 msgr2=0x55a9f58f4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.562 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.555+0000 7fa76bfe6b80 1 --2- 172.21.15.77:0/4082843951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9f58f53a0 0x55a9f58f4e10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa748010b00 tx=0x7fa748005a00 comp rx=0 tx=0).stop 2023-12-01T01:17:19.562 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.555+0000 7fa76bfe6b80 1 -- 172.21.15.77:0/4082843951 shutdown_connections 2023-12-01T01:17:19.562 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.555+0000 7fa76bfe6b80 1 --2- 172.21.15.77:0/4082843951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f58f6a20 0x55a9f58f9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.562 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.555+0000 7fa76bfe6b80 1 --2- 172.21.15.77:0/4082843951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9f58f53a0 0x55a9f58f4e10 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.562 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.555+0000 7fa76bfe6b80 1 --2- 172.21.15.77:0/4082843951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a9f58f60a0 0x55a9f59128a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.562 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.555+0000 7fa76bfe6b80 1 -- 172.21.15.77:0/4082843951 >> 172.21.15.77:0/4082843951 conn(0x55a9f5854080 msgr2=0x55a9f58fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:19.563 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.555+0000 7fa76bfe6b80 1 -- 172.21.15.77:0/4082843951 shutdown_connections 2023-12-01T01:17:19.563 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.555+0000 7fa76bfe6b80 1 -- 172.21.15.77:0/4082843951 wait complete. 2023-12-01T01:17:19.563 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.555+0000 7fa76bfe6b80 1 -- start start 2023-12-01T01:17:19.563 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.555+0000 7fa76bfe6b80 1 Processor -- start 2023-12-01T01:17:19.563 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.555+0000 7fa76bfe6b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f58f53a0 0x55a9f590f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.563 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.555+0000 7fa76bfe6b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a9f58f60a0 0x55a9f590fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.563 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.555+0000 7fa76bfe6b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9f58f6a20 0x55a9f595b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.563 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.555+0000 7fa76bfe6b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f5910220 con 0x55a9f58f6a20 2023-12-01T01:17:19.563 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.555+0000 7fa76bfe6b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f595bb60 con 0x55a9f58f53a0 2023-12-01T01:17:19.563 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.555+0000 7fa76bfe6b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f595bca0 con 0x55a9f58f60a0 2023-12-01T01:17:19.563 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.555+0000 7fa76bfe6b80 10 client.0 _ll_register_callbacks cb 0x55a9f572fa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:17:19.563 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.555+0000 7fa75e3f3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9f58f6a20 0x55a9f595b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.563 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.555+0000 7fa75e3f3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9f58f6a20 0x55a9f595b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58430/0 (socket says 172.21.15.77:58430) 2023-12-01T01:17:19.564 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.555+0000 7fa75e3f3700 1 -- 172.21.15.77:0/2195919889 learned_addr learned my addr 172.21.15.77:0/2195919889 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:19.564 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.556+0000 7fa75dbf2700 1 --2- 172.21.15.77:0/2195919889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f58f53a0 0x55a9f590f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.564 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.556+0000 7fa75d3f1700 1 --2- 172.21.15.77:0/2195919889 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a9f58f60a0 0x55a9f590fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.564 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.556+0000 7fa75e3f3700 1 -- 172.21.15.77:0/2195919889 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a9f58f60a0 msgr2=0x55a9f590fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:19.564 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.556+0000 7fa75e3f3700 1 --2- 172.21.15.77:0/2195919889 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a9f58f60a0 0x55a9f590fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.564 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.556+0000 7fa75e3f3700 1 -- 172.21.15.77:0/2195919889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f58f53a0 msgr2=0x55a9f590f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:19.564 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.556+0000 7fa75e3f3700 1 --2- 172.21.15.77:0/2195919889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f58f53a0 0x55a9f590f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.564 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.556+0000 7fa75e3f3700 1 -- 172.21.15.77:0/2195919889 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a9f565cc10 con 0x55a9f58f6a20 2023-12-01T01:17:19.564 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.556+0000 7fa75dbf2700 1 --2- 172.21.15.77:0/2195919889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f58f53a0 0x55a9f590f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:17:19.564 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.556+0000 7fa75d3f1700 1 --2- 172.21.15.77:0/2195919889 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a9f58f60a0 0x55a9f590fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:17:19.564 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.556+0000 7fa75e3f3700 1 --2- 172.21.15.77:0/2195919889 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9f58f6a20 0x55a9f595b620 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa750007ab0 tx=0x7fa750007e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c866fd32a17038e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:19.564 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.556+0000 7fa7417fa700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:17:19.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.556+0000 7fa7417fa700 1 -- 172.21.15.77:0/2195919889 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa750013910 con 0x55a9f58f6a20 2023-12-01T01:17:19.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.556+0000 7fa7417fa700 1 -- 172.21.15.77:0/2195919889 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa750013a70 con 0x55a9f58f6a20 2023-12-01T01:17:19.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.556+0000 7fa76bfe6b80 1 -- 172.21.15.77:0/2195919889 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa748015070 con 0x55a9f58f6a20 2023-12-01T01:17:19.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.556+0000 7fa7417fa700 1 -- 172.21.15.77:0/2195919889 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa750013c30 con 0x55a9f58f6a20 2023-12-01T01:17:19.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.556+0000 7fa76bfe6b80 10 client.15102 fetch_fsmap learned FSMap version 27 2023-12-01T01:17:19.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.556+0000 7fa76bfe6b80 1 -- 172.21.15.77:0/2195919889 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a9f5672260 con 0x55a9f58f6a20 2023-12-01T01:17:19.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.556+0000 7fa7417fa700 1 -- 172.21.15.77:0/2195919889 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa750029080 con 0x55a9f58f6a20 2023-12-01T01:17:19.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.556+0000 7fa7417fa700 1 -- 172.21.15.77:0/2195919889 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fa750016070 con 0x55a9f58f6a20 2023-12-01T01:17:19.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.557+0000 7fa76bfe6b80 10 client.15102 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:17:19.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.557+0000 7fa76bfe6b80 10 client.15102 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-01T01:17:19.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.557+0000 7fa76bfe6b80 20 client.15102 populate_metadata read hostname 'smithi077' 2023-12-01T01:17:19.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.557+0000 7fa76bfe6b80 1 --2- 172.21.15.77:0/2195919889 >> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] conn(0x55a9f5960ab0 0x55a9f595d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.557+0000 7fa76bfe6b80 4 client.15102 mds_command: new command op to 24475 tid=0[{"prefix": "status"}] 2023-12-01T01:17:19.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.557+0000 7fa76bfe6b80 1 -- 172.21.15.77:0/2195919889 --> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a9f583ecf0 con 0x55a9f5960ab0 2023-12-01T01:17:19.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.557+0000 7fa75dbf2700 1 --2- 172.21.15.77:0/2195919889 >> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] conn(0x55a9f5960ab0 0x55a9f595d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.566 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.557+0000 7fa7417fa700 10 client.15102 ms_handle_connect on v2:172.21.15.203:6836/2887080977 2023-12-01T01:17:19.566 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.557+0000 7fa75dbf2700 1 --2- 172.21.15.77:0/2195919889 >> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] conn(0x55a9f5960ab0 0x55a9f595d430 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6ad44b7b83ffba06 server_cookie=e6da61297adc116e in_seq=0 out_seq=0 2023-12-01T01:17:19.566 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.557+0000 7fa7417fa700 1 -- 172.21.15.77:0/2195919889 <== mds.2 v2:172.21.15.203:6836/2887080977 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55a9f583ecf0 con 0x55a9f5960ab0 2023-12-01T01:17:19.566 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.557+0000 7fa7417fa700 10 client.15102 handle_command_reply: tid=0 2023-12-01T01:17:19.566 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.557+0000 7fa76bfe6b80 1 client.15102 shutdown 2023-12-01T01:17:19.566 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.557+0000 7fa76bfe6b80 1 -- 172.21.15.77:0/2195919889 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9f58f6a20 msgr2=0x55a9f595b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.566 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.558+0000 7fa76bfe6b80 1 --2- 172.21.15.77:0/2195919889 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9f58f6a20 0x55a9f595b620 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa750007ab0 tx=0x7fa750007e70 comp rx=0 tx=0).stop 2023-12-01T01:17:19.566 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.558+0000 7fa76bfe6b80 1 -- 172.21.15.77:0/2195919889 shutdown_connections 2023-12-01T01:17:19.566 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.558+0000 7fa76bfe6b80 1 --2- 172.21.15.77:0/2195919889 >> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] conn(0x55a9f5960ab0 0x55a9f595d430 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.566 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.558+0000 7fa76bfe6b80 1 --2- 172.21.15.77:0/2195919889 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a9f58f6a20 0x55a9f595b620 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.566 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.558+0000 7fa76bfe6b80 1 --2- 172.21.15.77:0/2195919889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a9f58f53a0 0x55a9f590f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.566 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.558+0000 7fa76bfe6b80 1 --2- 172.21.15.77:0/2195919889 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a9f58f60a0 0x55a9f590fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.566 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.558+0000 7fa7417fa700 0 client.15102 ms_handle_reset on v2:172.21.15.203:6836/2887080977 2023-12-01T01:17:19.567 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.558+0000 7fa76bfe6b80 1 -- 172.21.15.77:0/2195919889 >> 172.21.15.77:0/2195919889 conn(0x55a9f5854080 msgr2=0x55a9f58f9360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:19.567 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.558+0000 7fa76bfe6b80 1 -- 172.21.15.77:0/2195919889 shutdown_connections 2023-12-01T01:17:19.567 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.558+0000 7fa76bfe6b80 1 -- 172.21.15.77:0/2195919889 wait complete. 2023-12-01T01:17:19.567 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.558+0000 7fa76bfe6b80 20 client.15102 trim_cache size 0 max 16384 2023-12-01T01:17:19.567 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2023-12-01T01:17:19.567 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "2a06e29a-8fe6-11ee-95a2-87774f69a715", 2023-12-01T01:17:19.567 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 2, 2023-12-01T01:17:19.567 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24475, 2023-12-01T01:17:19.567 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2023-12-01T01:17:19.567 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2023-12-01T01:17:19.567 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2023-12-01T01:17:19.568 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 49.930106436000003, 2023-12-01T01:17:19.568 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 27, 2023-12-01T01:17:19.568 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 82, 2023-12-01T01:17:19.568 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 82, 2023-12-01T01:17:19.568 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 70.668938429999997 2023-12-01T01:17:19.568 INFO:teuthology.orchestra.run.smithi077.stdout:} 2023-12-01T01:17:19.569 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.558+0000 7fa75fe56700 1 -- 172.21.15.77:0/2902584186 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa7440a0630 msgr2=0x7fa7440a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.569 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.558+0000 7fa75fe56700 1 --2- 172.21.15.77:0/2902584186 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa7440a0630 0x7fa7440a2af0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa758100cd0 tx=0x7fa748007400 comp rx=0 tx=0).stop 2023-12-01T01:17:19.569 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.558+0000 7fa75fe56700 1 -- 172.21.15.77:0/2902584186 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa758100e70 msgr2=0x7fa75819b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.569 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.558+0000 7fa75fe56700 1 --2- 172.21.15.77:0/2902584186 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa758100e70 0x7fa75819b7a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa754009fd0 tx=0x7fa75400f690 comp rx=0 tx=0).stop 2023-12-01T01:17:19.569 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.558+0000 7fa75fe56700 1 -- 172.21.15.77:0/2902584186 shutdown_connections 2023-12-01T01:17:19.569 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.558+0000 7fa75fe56700 1 --2- 172.21.15.77:0/2902584186 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa7440a0630 0x7fa7440a2af0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.569 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.558+0000 7fa75fe56700 1 --2- 172.21.15.77:0/2902584186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa758101810 0x7fa75819ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.569 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.558+0000 7fa75fe56700 1 --2- 172.21.15.77:0/2902584186 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7580ffc80 0x7fa758068a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.569 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.558+0000 7fa75fe56700 1 --2- 172.21.15.77:0/2902584186 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa758100e70 0x7fa75819b7a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.569 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.558+0000 7fa75fe56700 1 -- 172.21.15.77:0/2902584186 >> 172.21.15.77:0/2902584186 conn(0x7fa7580fb1f0 msgr2=0x7fa7580fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:19.569 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.558+0000 7fa75fe56700 1 -- 172.21.15.77:0/2902584186 shutdown_connections 2023-12-01T01:17:19.570 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.558+0000 7fa75fe56700 1 -- 172.21.15.77:0/2902584186 wait complete. 2023-12-01T01:17:19.572 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-12-01T01:17:19.700 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.694+0000 7fb2673bc700 1 -- 172.21.15.77:0/3040559657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb260100e70 msgr2=0x7fb2601012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.700 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.694+0000 7fb2673bc700 1 --2- 172.21.15.77:0/3040559657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb260100e70 0x7fb2601012d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb2500099c0 tx=0x7fb250009cd0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.700 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.694+0000 7fb2673bc700 1 -- 172.21.15.77:0/3040559657 shutdown_connections 2023-12-01T01:17:19.700 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.694+0000 7fb2673bc700 1 --2- 172.21.15.77:0/3040559657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb260101810 0x7fb2601080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.701 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.694+0000 7fb2673bc700 1 --2- 172.21.15.77:0/3040559657 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2600ffc80 0x7fb2601000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.701 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.694+0000 7fb2673bc700 1 --2- 172.21.15.77:0/3040559657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb260100e70 0x7fb2601012d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.701 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.694+0000 7fb2673bc700 1 -- 172.21.15.77:0/3040559657 >> 172.21.15.77:0/3040559657 conn(0x7fb2600fb1f0 msgr2=0x7fb2600fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:19.701 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.695+0000 7fb2673bc700 1 -- 172.21.15.77:0/3040559657 shutdown_connections 2023-12-01T01:17:19.701 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.695+0000 7fb2673bc700 1 -- 172.21.15.77:0/3040559657 wait complete. 2023-12-01T01:17:19.701 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.695+0000 7fb2673bc700 1 Processor -- start 2023-12-01T01:17:19.701 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.695+0000 7fb2673bc700 1 -- start start 2023-12-01T01:17:19.701 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.695+0000 7fb2673bc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2600ffc80 0x7fb260196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.701 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.695+0000 7fb2673bc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb260100e70 0x7fb260197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.701 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.695+0000 7fb2673bc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb260101810 0x7fb26019e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.701 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.695+0000 7fb2673bc700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26019e6d0 con 0x7fb260100e70 2023-12-01T01:17:19.701 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.695+0000 7fb2673bc700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26019e810 con 0x7fb260101810 2023-12-01T01:17:19.701 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.695+0000 7fb2673bc700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26019e9a0 con 0x7fb2600ffc80 2023-12-01T01:17:19.701 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.695+0000 7fb264957700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb260100e70 0x7fb260197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.702 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.695+0000 7fb264957700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb260100e70 0x7fb260197090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58442/0 (socket says 172.21.15.77:58442) 2023-12-01T01:17:19.702 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.695+0000 7fb264957700 1 -- 172.21.15.77:0/3703660883 learned_addr learned my addr 172.21.15.77:0/3703660883 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:19.702 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.695+0000 7fb265158700 1 --2- 172.21.15.77:0/3703660883 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2600ffc80 0x7fb260196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.702 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.696+0000 7fb265959700 1 --2- 172.21.15.77:0/3703660883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb260101810 0x7fb26019e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.702 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.696+0000 7fb264957700 1 -- 172.21.15.77:0/3703660883 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2600ffc80 msgr2=0x7fb260196b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.702 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.696+0000 7fb264957700 1 --2- 172.21.15.77:0/3703660883 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2600ffc80 0x7fb260196b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.702 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.696+0000 7fb264957700 1 -- 172.21.15.77:0/3703660883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb260101810 msgr2=0x7fb26019e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.702 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.696+0000 7fb264957700 1 --2- 172.21.15.77:0/3703660883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb260101810 0x7fb26019e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.702 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.696+0000 7fb264957700 1 -- 172.21.15.77:0/3703660883 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2500095f0 con 0x7fb260100e70 2023-12-01T01:17:19.702 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.696+0000 7fb265959700 1 --2- 172.21.15.77:0/3703660883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb260101810 0x7fb26019e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:17:19.702 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.696+0000 7fb265158700 1 --2- 172.21.15.77:0/3703660883 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2600ffc80 0x7fb260196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:17:19.702 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.696+0000 7fb264957700 1 --2- 172.21.15.77:0/3703660883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb260100e70 0x7fb260197090 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb250009990 tx=0x7fb250003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:19.702 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.696+0000 7fb2567fc700 1 -- 172.21.15.77:0/3703660883 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25001d070 con 0x7fb260100e70 2023-12-01T01:17:19.702 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.696+0000 7fb2567fc700 1 -- 172.21.15.77:0/3703660883 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb250004640 con 0x7fb260100e70 2023-12-01T01:17:19.703 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.696+0000 7fb2673bc700 1 -- 172.21.15.77:0/3703660883 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb26019ec20 con 0x7fb260100e70 2023-12-01T01:17:19.703 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.696+0000 7fb2567fc700 1 -- 172.21.15.77:0/3703660883 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25000f460 con 0x7fb260100e70 2023-12-01T01:17:19.703 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.696+0000 7fb2673bc700 1 -- 172.21.15.77:0/3703660883 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb26019f0d0 con 0x7fb260100e70 2023-12-01T01:17:19.703 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.698+0000 7fb2567fc700 1 -- 172.21.15.77:0/3703660883 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fb250022470 con 0x7fb260100e70 2023-12-01T01:17:19.703 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.699+0000 7fb2567fc700 1 --2- 172.21.15.77:0/3703660883 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fb24c0a05b0 0x7fb24c0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.709 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.699+0000 7fb265158700 1 --2- 172.21.15.77:0/3703660883 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fb24c0a05b0 0x7fb24c0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.710 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.699+0000 7fb2567fc700 1 -- 172.21.15.77:0/3703660883 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb2500cd110 con 0x7fb260100e70 2023-12-01T01:17:19.710 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.699+0000 7fb265158700 1 --2- 172.21.15.77:0/3703660883 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fb24c0a05b0 0x7fb24c0a2a70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb260100cd0 tx=0x7fb25c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:19.710 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.700+0000 7fb27354cb80 1 Processor -- start 2023-12-01T01:17:19.710 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.700+0000 7fb27354cb80 1 -- start start 2023-12-01T01:17:19.710 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.700+0000 7fb27354cb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5b3c210a0 0x55f5b3c214e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.710 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.700+0000 7fb27354cb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5b3c21a20 0x55f5b3c1fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.710 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.700+0000 7fb27354cb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f5b3c203a0 0x55f5b3c20820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.710 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.700+0000 7fb27354cb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5b3b779c0 con 0x55f5b3c21a20 2023-12-01T01:17:19.710 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.700+0000 7fb27354cb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5b3b75380 con 0x55f5b3c210a0 2023-12-01T01:17:19.710 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.700+0000 7fb27354cb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5b3b70c70 con 0x55f5b3c203a0 2023-12-01T01:17:19.710 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.700+0000 7fb264957700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5b3c21a20 0x55f5b3c1fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.710 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.700+0000 7fb264957700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5b3c21a20 0x55f5b3c1fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58450/0 (socket says 172.21.15.77:58450) 2023-12-01T01:17:19.711 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.700+0000 7fb264957700 1 -- 172.21.15.77:0/429226918 learned_addr learned my addr 172.21.15.77:0/429226918 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:19.711 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.700+0000 7fb265158700 1 --2- 172.21.15.77:0/429226918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5b3c210a0 0x55f5b3c214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.711 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.700+0000 7fb265959700 1 --2- 172.21.15.77:0/429226918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f5b3c203a0 0x55f5b3c20820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.711 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.700+0000 7fb264957700 1 -- 172.21.15.77:0/429226918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f5b3c203a0 msgr2=0x55f5b3c20820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:19.711 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.700+0000 7fb264957700 1 --2- 172.21.15.77:0/429226918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f5b3c203a0 0x55f5b3c20820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.711 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.700+0000 7fb264957700 1 -- 172.21.15.77:0/429226918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5b3c210a0 msgr2=0x55f5b3c214e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:19.711 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.700+0000 7fb264957700 1 --2- 172.21.15.77:0/429226918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5b3c210a0 0x55f5b3c214e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.711 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.700+0000 7fb264957700 1 -- 172.21.15.77:0/429226918 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5b399ed90 con 0x55f5b3c21a20 2023-12-01T01:17:19.711 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.700+0000 7fb264957700 1 --2- 172.21.15.77:0/429226918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5b3c21a20 0x55f5b3c1fe10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb250006010 tx=0x7fb250009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a14ca55ed9323b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:19.711 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.701+0000 7fb24b7fe700 1 -- 172.21.15.77:0/429226918 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25009a950 con 0x55f5b3c21a20 2023-12-01T01:17:19.711 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.701+0000 7fb24b7fe700 1 -- 172.21.15.77:0/429226918 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb25009e780 con 0x55f5b3c21a20 2023-12-01T01:17:19.711 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.701+0000 7fb24b7fe700 1 -- 172.21.15.77:0/429226918 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25009ae10 con 0x55f5b3c21a20 2023-12-01T01:17:19.711 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.701+0000 7fb27354cb80 1 -- 172.21.15.77:0/429226918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5b3c21a20 msgr2=0x55f5b3c1fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.711 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.701+0000 7fb27354cb80 1 --2- 172.21.15.77:0/429226918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5b3c21a20 0x55f5b3c1fe10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb250006010 tx=0x7fb250009cd0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.711 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.701+0000 7fb27354cb80 1 -- 172.21.15.77:0/429226918 shutdown_connections 2023-12-01T01:17:19.712 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.701+0000 7fb27354cb80 1 --2- 172.21.15.77:0/429226918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f5b3c203a0 0x55f5b3c20820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.712 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.701+0000 7fb27354cb80 1 --2- 172.21.15.77:0/429226918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5b3c210a0 0x55f5b3c214e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.712 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.701+0000 7fb27354cb80 1 --2- 172.21.15.77:0/429226918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5b3c21a20 0x55f5b3c1fe10 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.712 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.701+0000 7fb27354cb80 1 -- 172.21.15.77:0/429226918 >> 172.21.15.77:0/429226918 conn(0x55f5b3b7f080 msgr2=0x55f5b3c1a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:19.712 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.701+0000 7fb27354cb80 1 -- 172.21.15.77:0/429226918 shutdown_connections 2023-12-01T01:17:19.712 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.701+0000 7fb27354cb80 1 -- 172.21.15.77:0/429226918 wait complete. 2023-12-01T01:17:19.712 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.702+0000 7fb27354cb80 1 -- start start 2023-12-01T01:17:19.712 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.702+0000 7fb27354cb80 1 Processor -- start 2023-12-01T01:17:19.712 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.702+0000 7fb27354cb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f5b3c203a0 0x55f5b3c295d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.712 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.702+0000 7fb27354cb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5b3c210a0 0x55f5b3c29b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.712 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.702+0000 7fb27354cb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5b3c21a20 0x55f5b3c30b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.712 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.702+0000 7fb27354cb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5b3b0d4c0 con 0x55f5b3c210a0 2023-12-01T01:17:19.713 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.702+0000 7fb27354cb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5b3b0d160 con 0x55f5b3c21a20 2023-12-01T01:17:19.713 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.702+0000 7fb27354cb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5b39f48d0 con 0x55f5b3c203a0 2023-12-01T01:17:19.713 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.702+0000 7fb27354cb80 10 client.0 _ll_register_callbacks cb 0x55f5b3a5aa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:17:19.713 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.702+0000 7fb264957700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5b3c210a0 0x55f5b3c29b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.713 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.702+0000 7fb264957700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5b3c210a0 0x55f5b3c29b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58456/0 (socket says 172.21.15.77:58456) 2023-12-01T01:17:19.713 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.702+0000 7fb264957700 1 -- 172.21.15.77:0/245066301 learned_addr learned my addr 172.21.15.77:0/245066301 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:19.713 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.702+0000 7fb265158700 1 --2- 172.21.15.77:0/245066301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f5b3c203a0 0x55f5b3c295d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.713 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.702+0000 7fb265959700 1 --2- 172.21.15.77:0/245066301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5b3c21a20 0x55f5b3c30b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.713 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.703+0000 7fb264957700 1 -- 172.21.15.77:0/245066301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f5b3c203a0 msgr2=0x55f5b3c295d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:19.713 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.703+0000 7fb264957700 1 --2- 172.21.15.77:0/245066301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f5b3c203a0 0x55f5b3c295d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.713 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.703+0000 7fb264957700 1 -- 172.21.15.77:0/245066301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5b3c21a20 msgr2=0x55f5b3c30b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:19.713 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.703+0000 7fb264957700 1 --2- 172.21.15.77:0/245066301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5b3c21a20 0x55f5b3c30b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.713 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.703+0000 7fb264957700 1 -- 172.21.15.77:0/245066301 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb26019f0d0 con 0x55f5b3c210a0 2023-12-01T01:17:19.713 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.703+0000 7fb264957700 1 --2- 172.21.15.77:0/245066301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5b3c210a0 0x55f5b3c29b10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb250096e50 tx=0x7fb2500caae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a59ca62f0878313 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:19.713 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.703+0000 7fb248ff9700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:17:19.714 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.703+0000 7fb248ff9700 1 -- 172.21.15.77:0/245066301 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2500a2930 con 0x55f5b3c210a0 2023-12-01T01:17:19.714 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.703+0000 7fb248ff9700 1 -- 172.21.15.77:0/245066301 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb25009a950 con 0x55f5b3c210a0 2023-12-01T01:17:19.714 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.703+0000 7fb27354cb80 1 -- 172.21.15.77:0/245066301 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f5b3b9bcc0 con 0x55f5b3c210a0 2023-12-01T01:17:19.714 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.703+0000 7fb248ff9700 1 -- 172.21.15.77:0/245066301 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb25009adc0 con 0x55f5b3c210a0 2023-12-01T01:17:19.714 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.703+0000 7fb27354cb80 10 client.15114 fetch_fsmap learned FSMap version 27 2023-12-01T01:17:19.714 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.703+0000 7fb27354cb80 1 -- 172.21.15.77:0/245066301 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f5b3987c10 con 0x55f5b3c210a0 2023-12-01T01:17:19.714 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.703+0000 7fb248ff9700 1 -- 172.21.15.77:0/245066301 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f5b3b9bcc0 con 0x55f5b3c210a0 2023-12-01T01:17:19.714 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.703+0000 7fb248ff9700 1 -- 172.21.15.77:0/245066301 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fb2500998b0 con 0x55f5b3c210a0 2023-12-01T01:17:19.714 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.703+0000 7fb27354cb80 10 client.15114 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:17:19.714 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.703+0000 7fb27354cb80 10 client.15114 resolve_mds: resolved name 'i' to daemon mds.i 2023-12-01T01:17:19.714 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.703+0000 7fb27354cb80 20 client.15114 populate_metadata read hostname 'smithi077' 2023-12-01T01:17:19.714 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.703+0000 7fb27354cb80 1 --2- 172.21.15.77:0/245066301 >> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] conn(0x55f5b3c34440 0x55f5b3c33a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.714 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.703+0000 7fb27354cb80 4 client.15114 mds_command: new command op to 24496 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:17:19.714 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.703+0000 7fb27354cb80 1 -- 172.21.15.77:0/245066301 --> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f5b3beedc0 con 0x55f5b3c34440 2023-12-01T01:17:19.715 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.704+0000 7fb265158700 1 --2- 172.21.15.77:0/245066301 >> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] conn(0x55f5b3c34440 0x55f5b3c33a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.715 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.704+0000 7fb248ff9700 10 client.15114 ms_handle_connect on v2:172.21.15.203:6838/2888221613 2023-12-01T01:17:19.715 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.704+0000 7fb265158700 1 --2- 172.21.15.77:0/245066301 >> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] conn(0x55f5b3c34440 0x55f5b3c33a80 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=33748b9c832228d1 server_cookie=dbbc0c1dbe924c8e in_seq=0 out_seq=0 2023-12-01T01:17:19.715 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.705+0000 7fb248ff9700 1 -- 172.21.15.77:0/245066301 <== mds.1 v2:172.21.15.203:6838/2888221613 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55f5b3beedc0 con 0x55f5b3c34440 2023-12-01T01:17:19.715 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.705+0000 7fb248ff9700 10 client.15114 handle_command_reply: tid=0 2023-12-01T01:17:19.715 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.705+0000 7fb27354cb80 1 client.15114 shutdown 2023-12-01T01:17:19.715 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.705+0000 7fb27354cb80 1 -- 172.21.15.77:0/245066301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5b3c210a0 msgr2=0x55f5b3c29b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.715 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.705+0000 7fb27354cb80 1 --2- 172.21.15.77:0/245066301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5b3c210a0 0x55f5b3c29b10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb250096e50 tx=0x7fb2500caae0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.715 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.705+0000 7fb27354cb80 1 -- 172.21.15.77:0/245066301 shutdown_connections 2023-12-01T01:17:19.715 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.705+0000 7fb27354cb80 1 --2- 172.21.15.77:0/245066301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5b3c21a20 0x55f5b3c30b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.715 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.705+0000 7fb27354cb80 1 --2- 172.21.15.77:0/245066301 >> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] conn(0x55f5b3c34440 0x55f5b3c33a80 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.715 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.705+0000 7fb27354cb80 1 --2- 172.21.15.77:0/245066301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f5b3c203a0 0x55f5b3c295d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.715 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.705+0000 7fb27354cb80 1 --2- 172.21.15.77:0/245066301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5b3c210a0 0x55f5b3c29b10 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.715 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.705+0000 7fb248ff9700 0 client.15114 ms_handle_reset on v2:172.21.15.203:6838/2888221613 2023-12-01T01:17:19.716 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.705+0000 7fb27354cb80 1 -- 172.21.15.77:0/245066301 >> 172.21.15.77:0/245066301 conn(0x55f5b3b7f080 msgr2=0x55f5b3c1a060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:19.716 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.705+0000 7fb27354cb80 1 -- 172.21.15.77:0/245066301 shutdown_connections 2023-12-01T01:17:19.716 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.705+0000 7fb27354cb80 1 -- 172.21.15.77:0/245066301 wait complete. 2023-12-01T01:17:19.716 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.705+0000 7fb27354cb80 20 client.15114 trim_cache size 0 max 16384 2023-12-01T01:17:19.729 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.722+0000 7fb27354cb80 1 Processor -- start 2023-12-01T01:17:19.729 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.722+0000 7fb27354cb80 1 -- start start 2023-12-01T01:17:19.729 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.722+0000 7fb27354cb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5b3c203a0 0x55f5b3c27710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.729 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.722+0000 7fb27354cb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f5b3c210a0 0x55f5b3c3d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.729 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.722+0000 7fb27354cb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5b3c21a20 0x55f5b3c27c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.729 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.722+0000 7fb27354cb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5b3beedc0 con 0x55f5b3c203a0 2023-12-01T01:17:19.730 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.722+0000 7fb27354cb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5b3b9bcc0 con 0x55f5b3c21a20 2023-12-01T01:17:19.730 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.722+0000 7fb27354cb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5b3b1a830 con 0x55f5b3c210a0 2023-12-01T01:17:19.730 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.723+0000 7fb265158700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5b3c203a0 0x55f5b3c27710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.730 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.723+0000 7fb265158700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5b3c203a0 0x55f5b3c27710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58462/0 (socket says 172.21.15.77:58462) 2023-12-01T01:17:19.730 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.723+0000 7fb265158700 1 -- 172.21.15.77:0/3276635060 learned_addr learned my addr 172.21.15.77:0/3276635060 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:19.730 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.723+0000 7fb264957700 1 --2- 172.21.15.77:0/3276635060 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f5b3c210a0 0x55f5b3c3d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.730 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.723+0000 7fb265158700 1 -- 172.21.15.77:0/3276635060 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f5b3c210a0 msgr2=0x55f5b3c3d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:19.730 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.723+0000 7fb265158700 1 --2- 172.21.15.77:0/3276635060 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f5b3c210a0 0x55f5b3c3d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.730 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.723+0000 7fb265158700 1 -- 172.21.15.77:0/3276635060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5b3c21a20 msgr2=0x55f5b3c27c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:19.730 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.723+0000 7fb265158700 1 --2- 172.21.15.77:0/3276635060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5b3c21a20 0x55f5b3c27c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.730 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.723+0000 7fb265158700 1 -- 172.21.15.77:0/3276635060 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5b3c20a50 con 0x55f5b3c203a0 2023-12-01T01:17:19.730 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.723+0000 7fb264957700 1 --2- 172.21.15.77:0/3276635060 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f5b3c210a0 0x55f5b3c3d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:17:19.730 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.723+0000 7fb265158700 1 --2- 172.21.15.77:0/3276635060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5b3c203a0 0x55f5b3c27710 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb25c006fd0 tx=0x7fb25c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce671fa86f2f0dd server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:19.731 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.723+0000 7fb24bfff700 1 -- 172.21.15.77:0/3276635060 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25c027070 con 0x55f5b3c203a0 2023-12-01T01:17:19.731 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.723+0000 7fb24bfff700 1 -- 172.21.15.77:0/3276635060 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb25c00d8e0 con 0x55f5b3c203a0 2023-12-01T01:17:19.731 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.723+0000 7fb24bfff700 1 -- 172.21.15.77:0/3276635060 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25c0227e0 con 0x55f5b3c203a0 2023-12-01T01:17:19.731 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.724+0000 7fb27354cb80 1 -- 172.21.15.77:0/3276635060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5b3c203a0 msgr2=0x55f5b3c27710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.731 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.724+0000 7fb27354cb80 1 --2- 172.21.15.77:0/3276635060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5b3c203a0 0x55f5b3c27710 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb25c006fd0 tx=0x7fb25c013000 comp rx=0 tx=0).stop 2023-12-01T01:17:19.731 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.724+0000 7fb27354cb80 1 -- 172.21.15.77:0/3276635060 shutdown_connections 2023-12-01T01:17:19.731 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.724+0000 7fb27354cb80 1 --2- 172.21.15.77:0/3276635060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5b3c21a20 0x55f5b3c27c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.731 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.724+0000 7fb27354cb80 1 --2- 172.21.15.77:0/3276635060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5b3c203a0 0x55f5b3c27710 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.731 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.724+0000 7fb27354cb80 1 --2- 172.21.15.77:0/3276635060 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f5b3c210a0 0x55f5b3c3d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.731 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.724+0000 7fb27354cb80 1 -- 172.21.15.77:0/3276635060 >> 172.21.15.77:0/3276635060 conn(0x55f5b3b7f080 msgr2=0x55f5b3c33a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:19.731 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.724+0000 7fb27354cb80 1 -- 172.21.15.77:0/3276635060 shutdown_connections 2023-12-01T01:17:19.731 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.724+0000 7fb27354cb80 1 -- 172.21.15.77:0/3276635060 wait complete. 2023-12-01T01:17:19.731 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.724+0000 7fb27354cb80 1 -- start start 2023-12-01T01:17:19.731 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.724+0000 7fb27354cb80 1 Processor -- start 2023-12-01T01:17:19.731 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.724+0000 7fb27354cb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5b3c203a0 0x55f5b3c38590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.732 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.724+0000 7fb27354cb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f5b3c210a0 0x55f5b3c38ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.732 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.724+0000 7fb27354cb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5b3c21a20 0x55f5b3c3baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.732 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.724+0000 7fb27354cb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5b3c3c030 con 0x55f5b3c203a0 2023-12-01T01:17:19.732 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.724+0000 7fb27354cb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5b3c3c1a0 con 0x55f5b3c21a20 2023-12-01T01:17:19.732 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.724+0000 7fb27354cb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5b3c3c360 con 0x55f5b3c210a0 2023-12-01T01:17:19.732 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.725+0000 7fb27354cb80 10 client.0 _ll_register_callbacks cb 0x7fb25c0095e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:17:19.732 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.725+0000 7fb265158700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5b3c203a0 0x55f5b3c38590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.732 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.725+0000 7fb265158700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5b3c203a0 0x55f5b3c38590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58468/0 (socket says 172.21.15.77:58468) 2023-12-01T01:17:19.732 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.725+0000 7fb265158700 1 -- 172.21.15.77:0/1917192222 learned_addr learned my addr 172.21.15.77:0/1917192222 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:19.732 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.725+0000 7fb265959700 1 --2- 172.21.15.77:0/1917192222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5b3c21a20 0x55f5b3c3baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.732 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.725+0000 7fb264957700 1 --2- 172.21.15.77:0/1917192222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f5b3c210a0 0x55f5b3c38ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.732 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.725+0000 7fb265158700 1 -- 172.21.15.77:0/1917192222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f5b3c210a0 msgr2=0x55f5b3c38ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:19.732 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.725+0000 7fb265158700 1 --2- 172.21.15.77:0/1917192222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f5b3c210a0 0x55f5b3c38ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.732 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.725+0000 7fb265158700 1 -- 172.21.15.77:0/1917192222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5b3c21a20 msgr2=0x55f5b3c3baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:19.733 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.725+0000 7fb265158700 1 --2- 172.21.15.77:0/1917192222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5b3c21a20 0x55f5b3c3baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.733 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.725+0000 7fb265158700 1 -- 172.21.15.77:0/1917192222 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb25c00e040 con 0x55f5b3c203a0 2023-12-01T01:17:19.733 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.725+0000 7fb265959700 1 --2- 172.21.15.77:0/1917192222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5b3c21a20 0x55f5b3c3baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:17:19.733 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.725+0000 7fb265158700 1 --2- 172.21.15.77:0/1917192222 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5b3c203a0 0x55f5b3c38590 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb25c009f60 tx=0x7fb25c0148d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe8c649a6c13fb66 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:19.733 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.725+0000 7fb248ff9700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:17:19.733 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.725+0000 7fb248ff9700 1 -- 172.21.15.77:0/1917192222 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25c027040 con 0x55f5b3c203a0 2023-12-01T01:17:19.733 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.725+0000 7fb248ff9700 1 -- 172.21.15.77:0/1917192222 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb25c022760 con 0x55f5b3c203a0 2023-12-01T01:17:19.733 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.725+0000 7fb27354cb80 1 -- 172.21.15.77:0/1917192222 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb25009e780 con 0x55f5b3c203a0 2023-12-01T01:17:19.733 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.725+0000 7fb248ff9700 1 -- 172.21.15.77:0/1917192222 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb25c022920 con 0x55f5b3c203a0 2023-12-01T01:17:19.733 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.725+0000 7fb27354cb80 10 client.15120 fetch_fsmap learned FSMap version 27 2023-12-01T01:17:19.733 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.725+0000 7fb27354cb80 1 -- 172.21.15.77:0/1917192222 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f5b39c5200 con 0x55f5b3c203a0 2023-12-01T01:17:19.733 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.725+0000 7fb248ff9700 1 -- 172.21.15.77:0/1917192222 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb25009e780 con 0x55f5b3c203a0 2023-12-01T01:17:19.733 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.725+0000 7fb248ff9700 1 -- 172.21.15.77:0/1917192222 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fb25c030b00 con 0x55f5b3c203a0 2023-12-01T01:17:19.734 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.726+0000 7fb27354cb80 10 client.15120 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:17:19.734 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.726+0000 7fb27354cb80 10 client.15120 resolve_mds: resolved name 'i' to daemon mds.i 2023-12-01T01:17:19.734 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.726+0000 7fb27354cb80 20 client.15120 populate_metadata read hostname 'smithi077' 2023-12-01T01:17:19.734 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.726+0000 7fb27354cb80 1 --2- 172.21.15.77:0/1917192222 >> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] conn(0x55f5b3c8b760 0x55f5b3c880e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.734 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.726+0000 7fb27354cb80 4 client.15120 mds_command: new command op to 24496 tid=0[{"prefix": "status"}] 2023-12-01T01:17:19.734 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.726+0000 7fb27354cb80 1 -- 172.21.15.77:0/1917192222 --> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f5b3b69cf0 con 0x55f5b3c8b760 2023-12-01T01:17:19.734 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.726+0000 7fb264957700 1 --2- 172.21.15.77:0/1917192222 >> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] conn(0x55f5b3c8b760 0x55f5b3c880e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.734 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.726+0000 7fb248ff9700 10 client.15120 ms_handle_connect on v2:172.21.15.203:6838/2888221613 2023-12-01T01:17:19.735 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.726+0000 7fb264957700 1 --2- 172.21.15.77:0/1917192222 >> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] conn(0x55f5b3c8b760 0x55f5b3c880e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=21e8cde1eea68dc3 server_cookie=cd9b26525378aebd in_seq=0 out_seq=0 2023-12-01T01:17:19.735 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.726+0000 7fb248ff9700 1 -- 172.21.15.77:0/1917192222 <== mds.1 v2:172.21.15.203:6838/2888221613 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55f5b3b69cf0 con 0x55f5b3c8b760 2023-12-01T01:17:19.735 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.726+0000 7fb248ff9700 10 client.15120 handle_command_reply: tid=0 2023-12-01T01:17:19.735 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.726+0000 7fb27354cb80 1 client.15120 shutdown 2023-12-01T01:17:19.735 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.727+0000 7fb27354cb80 1 -- 172.21.15.77:0/1917192222 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5b3c203a0 msgr2=0x55f5b3c38590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.735 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.727+0000 7fb27354cb80 1 --2- 172.21.15.77:0/1917192222 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5b3c203a0 0x55f5b3c38590 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb25c009f60 tx=0x7fb25c0148d0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.735 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.727+0000 7fb27354cb80 1 -- 172.21.15.77:0/1917192222 shutdown_connections 2023-12-01T01:17:19.735 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.727+0000 7fb27354cb80 1 --2- 172.21.15.77:0/1917192222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5b3c21a20 0x55f5b3c3baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.735 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.727+0000 7fb27354cb80 1 --2- 172.21.15.77:0/1917192222 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5b3c203a0 0x55f5b3c38590 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.735 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.727+0000 7fb27354cb80 1 --2- 172.21.15.77:0/1917192222 >> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] conn(0x55f5b3c8b760 0x55f5b3c880e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.735 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.727+0000 7fb27354cb80 1 --2- 172.21.15.77:0/1917192222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f5b3c210a0 0x55f5b3c38ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.735 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.727+0000 7fb248ff9700 0 client.15120 ms_handle_reset on v2:172.21.15.203:6838/2888221613 2023-12-01T01:17:19.735 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.727+0000 7fb27354cb80 1 -- 172.21.15.77:0/1917192222 >> 172.21.15.77:0/1917192222 conn(0x55f5b3b7f080 msgr2=0x55f5b3c24220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:19.736 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.727+0000 7fb27354cb80 1 -- 172.21.15.77:0/1917192222 shutdown_connections 2023-12-01T01:17:19.736 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.727+0000 7fb27354cb80 1 -- 172.21.15.77:0/1917192222 wait complete. 2023-12-01T01:17:19.736 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.727+0000 7fb27354cb80 20 client.15120 trim_cache size 0 max 16384 2023-12-01T01:17:19.736 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2023-12-01T01:17:19.736 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "2a06e29a-8fe6-11ee-95a2-87774f69a715", 2023-12-01T01:17:19.736 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 1, 2023-12-01T01:17:19.736 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24496, 2023-12-01T01:17:19.736 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2023-12-01T01:17:19.736 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2023-12-01T01:17:19.736 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2023-12-01T01:17:19.736 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 51.104054654000002, 2023-12-01T01:17:19.737 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 27, 2023-12-01T01:17:19.737 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 82, 2023-12-01T01:17:19.737 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 82, 2023-12-01T01:17:19.737 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 66.375863461999998 2023-12-01T01:17:19.737 INFO:teuthology.orchestra.run.smithi077.stdout:} 2023-12-01T01:17:19.737 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.727+0000 7fb2673bc700 1 -- 172.21.15.77:0/3703660883 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fb24c0a05b0 msgr2=0x7fb24c0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.738 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.727+0000 7fb2673bc700 1 --2- 172.21.15.77:0/3703660883 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fb24c0a05b0 0x7fb24c0a2a70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb260100cd0 tx=0x7fb25c008040 comp rx=0 tx=0).stop 2023-12-01T01:17:19.738 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.727+0000 7fb2673bc700 1 -- 172.21.15.77:0/3703660883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb260100e70 msgr2=0x7fb260197090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.738 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.727+0000 7fb2673bc700 1 --2- 172.21.15.77:0/3703660883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb260100e70 0x7fb260197090 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb250009990 tx=0x7fb250003ba0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.738 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.727+0000 7fb2673bc700 1 -- 172.21.15.77:0/3703660883 shutdown_connections 2023-12-01T01:17:19.738 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.727+0000 7fb2673bc700 1 --2- 172.21.15.77:0/3703660883 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fb24c0a05b0 0x7fb24c0a2a70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.738 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.727+0000 7fb2673bc700 1 --2- 172.21.15.77:0/3703660883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb260101810 0x7fb26019e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.738 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.727+0000 7fb2673bc700 1 --2- 172.21.15.77:0/3703660883 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2600ffc80 0x7fb260196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.738 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.727+0000 7fb2673bc700 1 --2- 172.21.15.77:0/3703660883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb260100e70 0x7fb260197090 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.738 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.727+0000 7fb2673bc700 1 -- 172.21.15.77:0/3703660883 >> 172.21.15.77:0/3703660883 conn(0x7fb2600fb1f0 msgr2=0x7fb2600fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:19.738 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.727+0000 7fb2673bc700 1 -- 172.21.15.77:0/3703660883 shutdown_connections 2023-12-01T01:17:19.738 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.727+0000 7fb2673bc700 1 -- 172.21.15.77:0/3703660883 wait complete. 2023-12-01T01:17:19.740 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-12-01T01:17:19.864 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.861+0000 7f258611a700 1 -- 172.21.15.77:0/1837588480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2580101820 msgr2=0x7f25801080d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.865 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.861+0000 7f258611a700 1 --2- 172.21.15.77:0/1837588480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2580101820 0x7f25801080d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f257800b0d0 tx=0x7f257800b3e0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.865 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.861+0000 7f258611a700 1 -- 172.21.15.77:0/1837588480 shutdown_connections 2023-12-01T01:17:19.865 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.861+0000 7f258611a700 1 --2- 172.21.15.77:0/1837588480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2580101820 0x7f25801080d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.865 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.861+0000 7f258611a700 1 --2- 172.21.15.77:0/1837588480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25800ffc90 0x7f25801000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.865 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.861+0000 7f258611a700 1 --2- 172.21.15.77:0/1837588480 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2580100e80 0x7f25801012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.865 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.861+0000 7f258611a700 1 -- 172.21.15.77:0/1837588480 >> 172.21.15.77:0/1837588480 conn(0x7f25800fb220 msgr2=0x7f25800fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:19.865 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.861+0000 7f258611a700 1 -- 172.21.15.77:0/1837588480 shutdown_connections 2023-12-01T01:17:19.865 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.861+0000 7f258611a700 1 -- 172.21.15.77:0/1837588480 wait complete. 2023-12-01T01:17:19.865 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.862+0000 7f258611a700 1 Processor -- start 2023-12-01T01:17:19.865 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.862+0000 7f258611a700 1 -- start start 2023-12-01T01:17:19.865 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.862+0000 7f258611a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25800ffc90 0x7f25801959f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.866 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.862+0000 7f258611a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2580100e80 0x7f2580195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.866 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.862+0000 7f258611a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2580101820 0x7f258019cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.866 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.862+0000 7f258611a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258019d570 con 0x7f2580100e80 2023-12-01T01:17:19.866 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.862+0000 7f258611a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258019d6b0 con 0x7f2580101820 2023-12-01T01:17:19.866 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.862+0000 7f258611a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258019d840 con 0x7f25800ffc90 2023-12-01T01:17:19.866 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.862+0000 7f257effd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2580100e80 0x7f2580195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.866 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.862+0000 7f257effd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2580100e80 0x7f2580195f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58484/0 (socket says 172.21.15.77:58484) 2023-12-01T01:17:19.866 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.862+0000 7f257f7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25800ffc90 0x7f25801959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.866 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.862+0000 7f257effd700 1 -- 172.21.15.77:0/2535368221 learned_addr learned my addr 172.21.15.77:0/2535368221 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:19.866 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.863+0000 7f257ffff700 1 --2- 172.21.15.77:0/2535368221 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2580101820 0x7f258019cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.866 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.863+0000 7f257effd700 1 -- 172.21.15.77:0/2535368221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25800ffc90 msgr2=0x7f25801959f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.866 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.863+0000 7f257effd700 1 --2- 172.21.15.77:0/2535368221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25800ffc90 0x7f25801959f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.866 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.863+0000 7f257effd700 1 -- 172.21.15.77:0/2535368221 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2580101820 msgr2=0x7f258019cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.867 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.863+0000 7f257effd700 1 --2- 172.21.15.77:0/2535368221 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2580101820 0x7f258019cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.867 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.863+0000 7f257effd700 1 -- 172.21.15.77:0/2535368221 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2578009d00 con 0x7f2580100e80 2023-12-01T01:17:19.867 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.863+0000 7f257ffff700 1 --2- 172.21.15.77:0/2535368221 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2580101820 0x7f258019cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:17:19.867 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.863+0000 7f257effd700 1 --2- 172.21.15.77:0/2535368221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2580100e80 0x7f2580195f30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f257400c9b0 tx=0x7f257400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:19.867 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.863+0000 7f257cff9700 1 -- 172.21.15.77:0/2535368221 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2574007b00 con 0x7f2580100e80 2023-12-01T01:17:19.867 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.863+0000 7f257cff9700 1 -- 172.21.15.77:0/2535368221 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2574007c60 con 0x7f2580100e80 2023-12-01T01:17:19.867 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.863+0000 7f258611a700 1 -- 172.21.15.77:0/2535368221 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f258019db20 con 0x7f2580100e80 2023-12-01T01:17:19.867 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.863+0000 7f257cff9700 1 -- 172.21.15.77:0/2535368221 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2574018660 con 0x7f2580100e80 2023-12-01T01:17:19.867 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.863+0000 7f258611a700 1 -- 172.21.15.77:0/2535368221 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f258019e030 con 0x7f2580100e80 2023-12-01T01:17:19.874 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.865+0000 7f257cff9700 1 -- 172.21.15.77:0/2535368221 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f257400f450 con 0x7f2580100e80 2023-12-01T01:17:19.874 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.865+0000 7f257cff9700 1 --2- 172.21.15.77:0/2535368221 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f25680a05b0 0x7f25680a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.874 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.866+0000 7f257cff9700 1 -- 172.21.15.77:0/2535368221 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f25740cb910 con 0x7f2580100e80 2023-12-01T01:17:19.874 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.866+0000 7f257f7fe700 1 --2- 172.21.15.77:0/2535368221 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f25680a05b0 0x7f25680a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.874 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.866+0000 7f257f7fe700 1 --2- 172.21.15.77:0/2535368221 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f25680a05b0 0x7f25680a2a70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2580100ce0 tx=0x7f257000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:19.874 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.866+0000 7f25922aab80 1 Processor -- start 2023-12-01T01:17:19.874 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.867+0000 7f25922aab80 1 -- start start 2023-12-01T01:17:19.874 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.867+0000 7f25922aab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56288669c2b0 0x56288669c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.874 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.867+0000 7f25922aab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288669cc30 0x56288669b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.874 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.867+0000 7f25922aab80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56288669b5b0 0x56288669ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.874 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.867+0000 7f25922aab80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628865f09c0 con 0x56288669b5b0 2023-12-01T01:17:19.875 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.867+0000 7f25922aab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628865ee380 con 0x56288669cc30 2023-12-01T01:17:19.875 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.867+0000 7f25922aab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628865e9c70 con 0x56288669c2b0 2023-12-01T01:17:19.875 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.867+0000 7f257ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56288669b5b0 0x56288669ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.875 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.867+0000 7f257ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56288669b5b0 0x56288669ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58494/0 (socket says 172.21.15.77:58494) 2023-12-01T01:17:19.875 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.867+0000 7f257ffff700 1 -- 172.21.15.77:0/148455360 learned_addr learned my addr 172.21.15.77:0/148455360 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:19.875 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.867+0000 7f257effd700 1 --2- 172.21.15.77:0/148455360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288669cc30 0x56288669b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.875 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.867+0000 7f257f7fe700 1 --2- 172.21.15.77:0/148455360 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56288669c2b0 0x56288669c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.875 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.867+0000 7f257ffff700 1 -- 172.21.15.77:0/148455360 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56288669c2b0 msgr2=0x56288669c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:19.875 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.867+0000 7f257ffff700 1 --2- 172.21.15.77:0/148455360 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56288669c2b0 0x56288669c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.875 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.867+0000 7f257ffff700 1 -- 172.21.15.77:0/148455360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288669cc30 msgr2=0x56288669b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:19.875 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.867+0000 7f257ffff700 1 --2- 172.21.15.77:0/148455360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288669cc30 0x56288669b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.875 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.867+0000 7f257ffff700 1 -- 172.21.15.77:0/148455360 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562886417d90 con 0x56288669b5b0 2023-12-01T01:17:19.875 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.867+0000 7f257effd700 1 --2- 172.21.15.77:0/148455360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288669cc30 0x56288669b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:17:19.876 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.867+0000 7f257ffff700 1 --2- 172.21.15.77:0/148455360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56288669b5b0 0x56288669ba30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f257800cef0 tx=0x7f257800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2206b7a94628b68 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:19.876 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.867+0000 7f2565ffb700 1 -- 172.21.15.77:0/148455360 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257800e070 con 0x56288669b5b0 2023-12-01T01:17:19.876 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.867+0000 7f2565ffb700 1 -- 172.21.15.77:0/148455360 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2578007690 con 0x56288669b5b0 2023-12-01T01:17:19.876 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.867+0000 7f2565ffb700 1 -- 172.21.15.77:0/148455360 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25780053b0 con 0x56288669b5b0 2023-12-01T01:17:19.876 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.868+0000 7f25922aab80 1 -- 172.21.15.77:0/148455360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56288669b5b0 msgr2=0x56288669ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.876 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.868+0000 7f25922aab80 1 --2- 172.21.15.77:0/148455360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56288669b5b0 0x56288669ba30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f257800cef0 tx=0x7f257800cf20 comp rx=0 tx=0).stop 2023-12-01T01:17:19.876 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.868+0000 7f25922aab80 1 -- 172.21.15.77:0/148455360 shutdown_connections 2023-12-01T01:17:19.876 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.868+0000 7f25922aab80 1 --2- 172.21.15.77:0/148455360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56288669b5b0 0x56288669ba30 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.876 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.868+0000 7f25922aab80 1 --2- 172.21.15.77:0/148455360 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56288669c2b0 0x56288669c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.876 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.868+0000 7f25922aab80 1 --2- 172.21.15.77:0/148455360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288669cc30 0x56288669b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.876 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.868+0000 7f25922aab80 1 -- 172.21.15.77:0/148455360 >> 172.21.15.77:0/148455360 conn(0x5628865f8080 msgr2=0x5628866938f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:19.876 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.868+0000 7f25922aab80 1 -- 172.21.15.77:0/148455360 shutdown_connections 2023-12-01T01:17:19.876 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.868+0000 7f25922aab80 1 -- 172.21.15.77:0/148455360 wait complete. 2023-12-01T01:17:19.876 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.868+0000 7f25922aab80 1 -- start start 2023-12-01T01:17:19.877 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.869+0000 7f25922aab80 1 Processor -- start 2023-12-01T01:17:19.877 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.869+0000 7f25922aab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56288669b5b0 0x5628866a4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.877 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.869+0000 7f25922aab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288669c2b0 0x5628866a5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.877 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.869+0000 7f25922aab80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56288669cc30 0x5628866ac040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.877 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.869+0000 7f25922aab80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628865864c0 con 0x56288669cc30 2023-12-01T01:17:19.877 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.869+0000 7f25922aab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562886586160 con 0x56288669c2b0 2023-12-01T01:17:19.877 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.869+0000 7f25922aab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56288646d8d0 con 0x56288669b5b0 2023-12-01T01:17:19.877 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.869+0000 7f25922aab80 10 client.0 _ll_register_callbacks cb 0x5628864d3a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:17:19.877 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.869+0000 7f257ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56288669cc30 0x5628866ac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.877 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.869+0000 7f257ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56288669cc30 0x5628866ac040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58500/0 (socket says 172.21.15.77:58500) 2023-12-01T01:17:19.877 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.869+0000 7f257ffff700 1 -- 172.21.15.77:0/2794972435 learned_addr learned my addr 172.21.15.77:0/2794972435 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:19.877 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.869+0000 7f257effd700 1 --2- 172.21.15.77:0/2794972435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288669c2b0 0x5628866a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.877 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.869+0000 7f257ffff700 1 -- 172.21.15.77:0/2794972435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56288669b5b0 msgr2=0x5628866a4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:19.877 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.869+0000 7f257ffff700 1 --2- 172.21.15.77:0/2794972435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56288669b5b0 0x5628866a4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.877 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.869+0000 7f257ffff700 1 -- 172.21.15.77:0/2794972435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288669c2b0 msgr2=0x5628866a5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:19.878 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.869+0000 7f257ffff700 1 --2- 172.21.15.77:0/2794972435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288669c2b0 0x5628866a5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.878 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.869+0000 7f257ffff700 1 -- 172.21.15.77:0/2794972435 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2580108a20 con 0x56288669cc30 2023-12-01T01:17:19.878 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.869+0000 7f257effd700 1 --2- 172.21.15.77:0/2794972435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288669c2b0 0x5628866a5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:17:19.878 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.869+0000 7f257ffff700 1 --2- 172.21.15.77:0/2794972435 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56288669cc30 0x5628866ac040 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f25780074e0 tx=0x7f2578004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edd1321acb2012e2 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:19.878 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.869+0000 7f255effd700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:17:19.878 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.869+0000 7f255effd700 1 -- 172.21.15.77:0/2794972435 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257800e040 con 0x56288669cc30 2023-12-01T01:17:19.878 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.870+0000 7f255effd700 1 -- 172.21.15.77:0/2794972435 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f25780054a0 con 0x56288669cc30 2023-12-01T01:17:19.878 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.870+0000 7f25922aab80 1 -- 172.21.15.77:0/2794972435 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562886614cc0 con 0x56288669cc30 2023-12-01T01:17:19.878 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.870+0000 7f255effd700 1 -- 172.21.15.77:0/2794972435 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2578005660 con 0x56288669cc30 2023-12-01T01:17:19.878 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.870+0000 7f25922aab80 10 client.15132 fetch_fsmap learned FSMap version 27 2023-12-01T01:17:19.878 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.870+0000 7f25922aab80 1 -- 172.21.15.77:0/2794972435 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562886400c10 con 0x56288669cc30 2023-12-01T01:17:19.878 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.870+0000 7f255effd700 1 -- 172.21.15.77:0/2794972435 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2578039080 con 0x56288669cc30 2023-12-01T01:17:19.878 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.870+0000 7f255effd700 1 -- 172.21.15.77:0/2794972435 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f257800b100 con 0x56288669cc30 2023-12-01T01:17:19.879 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.870+0000 7f25922aab80 10 client.15132 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:17:19.879 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.870+0000 7f25922aab80 10 client.15132 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-01T01:17:19.879 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.870+0000 7f25922aab80 20 client.15132 populate_metadata read hostname 'smithi077' 2023-12-01T01:17:19.879 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.870+0000 7f25922aab80 1 --2- 172.21.15.77:0/2794972435 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5628866af9e0 0x5628866aeee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.879 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.870+0000 7f25922aab80 4 client.15132 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:17:19.879 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.870+0000 7f25922aab80 1 -- 172.21.15.77:0/2794972435 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562886667dc0 con 0x5628866af9e0 2023-12-01T01:17:19.879 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.870+0000 7f257f7fe700 1 --2- 172.21.15.77:0/2794972435 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5628866af9e0 0x5628866aeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.879 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.871+0000 7f255effd700 10 client.15132 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:17:19.879 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.871+0000 7f257f7fe700 1 --2- 172.21.15.77:0/2794972435 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5628866af9e0 0x5628866aeee0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c304c33dcf50199 server_cookie=e1abd24fa84d3319 in_seq=0 out_seq=0 2023-12-01T01:17:19.879 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.872+0000 7f255effd700 1 -- 172.21.15.77:0/2794972435 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x562886667dc0 con 0x5628866af9e0 2023-12-01T01:17:19.879 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.872+0000 7f255effd700 10 client.15132 handle_command_reply: tid=0 2023-12-01T01:17:19.879 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.872+0000 7f25922aab80 1 client.15132 shutdown 2023-12-01T01:17:19.879 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.872+0000 7f25922aab80 1 -- 172.21.15.77:0/2794972435 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56288669cc30 msgr2=0x5628866ac040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.879 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.872+0000 7f25922aab80 1 --2- 172.21.15.77:0/2794972435 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56288669cc30 0x5628866ac040 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f25780074e0 tx=0x7f2578004d50 comp rx=0 tx=0).stop 2023-12-01T01:17:19.880 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.872+0000 7f25922aab80 1 -- 172.21.15.77:0/2794972435 shutdown_connections 2023-12-01T01:17:19.880 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.872+0000 7f25922aab80 1 --2- 172.21.15.77:0/2794972435 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56288669cc30 0x5628866ac040 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.880 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.872+0000 7f25922aab80 1 --2- 172.21.15.77:0/2794972435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56288669b5b0 0x5628866a4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.880 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.872+0000 7f25922aab80 1 --2- 172.21.15.77:0/2794972435 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5628866af9e0 0x5628866aeee0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.880 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.872+0000 7f25922aab80 1 --2- 172.21.15.77:0/2794972435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288669c2b0 0x5628866a5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.880 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.872+0000 7f25922aab80 1 -- 172.21.15.77:0/2794972435 >> 172.21.15.77:0/2794972435 conn(0x5628865f8080 msgr2=0x562886693060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:19.880 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.872+0000 7f255effd700 0 client.15132 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:17:19.880 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.872+0000 7f25922aab80 1 -- 172.21.15.77:0/2794972435 shutdown_connections 2023-12-01T01:17:19.880 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.872+0000 7f25922aab80 1 -- 172.21.15.77:0/2794972435 wait complete. 2023-12-01T01:17:19.880 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.872+0000 7f25922aab80 20 client.15132 trim_cache size 0 max 16384 2023-12-01T01:17:19.892 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.889+0000 7f25922aab80 1 Processor -- start 2023-12-01T01:17:19.892 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.889+0000 7f25922aab80 1 -- start start 2023-12-01T01:17:19.892 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.889+0000 7f25922aab80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56288669b5b0 0x5628866a2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.892 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.889+0000 7f25922aab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56288669c2b0 0x5628866a2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.892 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.889+0000 7f25922aab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288669cc30 0x5628866b8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.892 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.889+0000 7f25922aab80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x562886694670 con 0x56288669b5b0 2023-12-01T01:17:19.894 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.889+0000 7f25922aab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628865864c0 con 0x56288669cc30 2023-12-01T01:17:19.894 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.889+0000 7f25922aab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628866a3660 con 0x56288669c2b0 2023-12-01T01:17:19.894 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.889+0000 7f257f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56288669b5b0 0x5628866a2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.894 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.889+0000 7f257f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56288669b5b0 0x5628866a2980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58504/0 (socket says 172.21.15.77:58504) 2023-12-01T01:17:19.894 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.890+0000 7f257f7fe700 1 -- 172.21.15.77:0/4263129367 learned_addr learned my addr 172.21.15.77:0/4263129367 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:19.894 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.890+0000 7f257effd700 1 --2- 172.21.15.77:0/4263129367 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56288669c2b0 0x5628866a2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.894 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.890+0000 7f257f7fe700 1 -- 172.21.15.77:0/4263129367 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56288669c2b0 msgr2=0x5628866a2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:19.895 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.890+0000 7f257f7fe700 1 --2- 172.21.15.77:0/4263129367 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56288669c2b0 0x5628866a2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.895 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.890+0000 7f257f7fe700 1 -- 172.21.15.77:0/4263129367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288669cc30 msgr2=0x5628866b8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:19.895 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.890+0000 7f257f7fe700 1 --2- 172.21.15.77:0/4263129367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288669cc30 0x5628866b8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.895 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.890+0000 7f257f7fe700 1 -- 172.21.15.77:0/4263129367 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56288643e200 con 0x56288669b5b0 2023-12-01T01:17:19.895 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.890+0000 7f257f7fe700 1 --2- 172.21.15.77:0/4263129367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56288669b5b0 0x5628866a2980 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x5628866a4a90 tx=0x7f2570012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ef7dec96aa93cd1 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:19.895 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.890+0000 7f25667fc700 1 -- 172.21.15.77:0/4263129367 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2570021040 con 0x56288669b5b0 2023-12-01T01:17:19.895 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.890+0000 7f25667fc700 1 -- 172.21.15.77:0/4263129367 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2570014070 con 0x56288669b5b0 2023-12-01T01:17:19.895 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.890+0000 7f25667fc700 1 -- 172.21.15.77:0/4263129367 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2570023a10 con 0x56288669b5b0 2023-12-01T01:17:19.895 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.891+0000 7f25922aab80 1 -- 172.21.15.77:0/4263129367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56288669b5b0 msgr2=0x5628866a2980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.895 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.891+0000 7f25922aab80 1 --2- 172.21.15.77:0/4263129367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56288669b5b0 0x5628866a2980 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x5628866a4a90 tx=0x7f2570012000 comp rx=0 tx=0).stop 2023-12-01T01:17:19.895 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.891+0000 7f25922aab80 1 -- 172.21.15.77:0/4263129367 shutdown_connections 2023-12-01T01:17:19.895 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.891+0000 7f25922aab80 1 --2- 172.21.15.77:0/4263129367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288669cc30 0x5628866b8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.895 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.891+0000 7f25922aab80 1 --2- 172.21.15.77:0/4263129367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56288669b5b0 0x5628866a2980 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.895 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.891+0000 7f25922aab80 1 --2- 172.21.15.77:0/4263129367 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56288669c2b0 0x5628866a2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.896 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.891+0000 7f25922aab80 1 -- 172.21.15.77:0/4263129367 >> 172.21.15.77:0/4263129367 conn(0x5628865f8080 msgr2=0x5628866938c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:19.896 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.891+0000 7f25922aab80 1 -- 172.21.15.77:0/4263129367 shutdown_connections 2023-12-01T01:17:19.896 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.891+0000 7f25922aab80 1 -- 172.21.15.77:0/4263129367 wait complete. 2023-12-01T01:17:19.896 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.891+0000 7f25922aab80 1 -- start start 2023-12-01T01:17:19.896 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.891+0000 7f25922aab80 1 Processor -- start 2023-12-01T01:17:19.899 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.891+0000 7f25922aab80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56288669b5b0 0x5628866a1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.899 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.891+0000 7f25922aab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288669c2b0 0x5628866a1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.899 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.891+0000 7f25922aab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56288669cc30 0x5628867014f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.899 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.891+0000 7f25922aab80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x562886701a30 con 0x56288669b5b0 2023-12-01T01:17:19.899 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.891+0000 7f25922aab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562886701b70 con 0x56288669c2b0 2023-12-01T01:17:19.899 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.891+0000 7f25922aab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x562886701cb0 con 0x56288669cc30 2023-12-01T01:17:19.899 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.891+0000 7f25922aab80 10 client.0 _ll_register_callbacks cb 0x7f2578009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:17:19.899 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.892+0000 7f257f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56288669b5b0 0x5628866a1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.900 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.892+0000 7f257f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56288669b5b0 0x5628866a1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58510/0 (socket says 172.21.15.77:58510) 2023-12-01T01:17:19.900 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.892+0000 7f257f7fe700 1 -- 172.21.15.77:0/1383681894 learned_addr learned my addr 172.21.15.77:0/1383681894 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:19.900 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.892+0000 7f257f7fe700 1 -- 172.21.15.77:0/1383681894 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56288669cc30 msgr2=0x5628867014f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:19.900 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.892+0000 7f257f7fe700 1 --2- 172.21.15.77:0/1383681894 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56288669cc30 0x5628867014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.900 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.892+0000 7f257f7fe700 1 -- 172.21.15.77:0/1383681894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288669c2b0 msgr2=0x5628866a1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:19.900 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.892+0000 7f257f7fe700 1 --2- 172.21.15.77:0/1383681894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288669c2b0 0x5628866a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.900 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.892+0000 7f257f7fe700 1 -- 172.21.15.77:0/1383681894 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f257000f960 con 0x56288669b5b0 2023-12-01T01:17:19.900 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.892+0000 7f257f7fe700 1 --2- 172.21.15.77:0/1383681894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56288669b5b0 0x5628866a1430 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2570005910 tx=0x7f257000ffd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a673f161720b629d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:19.900 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.892+0000 7f255effd700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:17:19.900 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.892+0000 7f255effd700 1 -- 172.21.15.77:0/1383681894 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2570010cb0 con 0x56288669b5b0 2023-12-01T01:17:19.900 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.892+0000 7f255effd700 1 -- 172.21.15.77:0/1383681894 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2570005040 con 0x56288669b5b0 2023-12-01T01:17:19.900 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.892+0000 7f25922aab80 1 -- 172.21.15.77:0/1383681894 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2578007690 con 0x56288669b5b0 2023-12-01T01:17:19.900 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.892+0000 7f255effd700 1 -- 172.21.15.77:0/1383681894 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2570005200 con 0x56288669b5b0 2023-12-01T01:17:19.900 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.892+0000 7f25922aab80 10 client.15138 fetch_fsmap learned FSMap version 27 2023-12-01T01:17:19.901 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.892+0000 7f25922aab80 1 -- 172.21.15.77:0/1383681894 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562886416260 con 0x56288669b5b0 2023-12-01T01:17:19.901 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.892+0000 7f255effd700 1 -- 172.21.15.77:0/1383681894 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2578007690 con 0x56288669b5b0 2023-12-01T01:17:19.901 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.892+0000 7f255effd700 1 -- 172.21.15.77:0/1383681894 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f2570004780 con 0x56288669b5b0 2023-12-01T01:17:19.901 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.892+0000 7f25922aab80 10 client.15138 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:17:19.901 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.893+0000 7f25922aab80 10 client.15138 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-01T01:17:19.901 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.893+0000 7f25922aab80 20 client.15138 populate_metadata read hostname 'smithi077' 2023-12-01T01:17:19.901 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.893+0000 7f25922aab80 1 --2- 172.21.15.77:0/1383681894 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x562886706ab0 0x562886703420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:19.901 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.893+0000 7f25922aab80 4 client.15138 mds_command: new command op to 24500 tid=0[{"prefix": "status"}] 2023-12-01T01:17:19.901 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.893+0000 7f25922aab80 1 -- 172.21.15.77:0/1383681894 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5628865e2cf0 con 0x562886706ab0 2023-12-01T01:17:19.901 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.893+0000 7f257effd700 1 --2- 172.21.15.77:0/1383681894 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x562886706ab0 0x562886703420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:19.901 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.893+0000 7f255effd700 10 client.15138 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:17:19.901 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.893+0000 7f257effd700 1 --2- 172.21.15.77:0/1383681894 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x562886706ab0 0x562886703420 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49116d1109da8b76 server_cookie=a13e182d6254b697 in_seq=0 out_seq=0 2023-12-01T01:17:19.901 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.893+0000 7f255effd700 1 -- 172.21.15.77:0/1383681894 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x5628865e2cf0 con 0x562886706ab0 2023-12-01T01:17:19.901 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.893+0000 7f255effd700 10 client.15138 handle_command_reply: tid=0 2023-12-01T01:17:19.901 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.893+0000 7f25922aab80 1 client.15138 shutdown 2023-12-01T01:17:19.902 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.894+0000 7f25922aab80 1 -- 172.21.15.77:0/1383681894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56288669b5b0 msgr2=0x5628866a1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.902 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.894+0000 7f25922aab80 1 --2- 172.21.15.77:0/1383681894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56288669b5b0 0x5628866a1430 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2570005910 tx=0x7f257000ffd0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.902 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.894+0000 7f25922aab80 1 -- 172.21.15.77:0/1383681894 shutdown_connections 2023-12-01T01:17:19.902 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.894+0000 7f25922aab80 1 --2- 172.21.15.77:0/1383681894 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56288669cc30 0x5628867014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.902 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.894+0000 7f25922aab80 1 --2- 172.21.15.77:0/1383681894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56288669b5b0 0x5628866a1430 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.902 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.894+0000 7f25922aab80 1 --2- 172.21.15.77:0/1383681894 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x562886706ab0 0x562886703420 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.902 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.894+0000 7f25922aab80 1 --2- 172.21.15.77:0/1383681894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56288669c2b0 0x5628866a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.902 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.894+0000 7f25922aab80 1 -- 172.21.15.77:0/1383681894 >> 172.21.15.77:0/1383681894 conn(0x5628865f8080 msgr2=0x56288669f530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:19.902 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.894+0000 7f255effd700 0 client.15138 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:17:19.902 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.894+0000 7f25922aab80 1 -- 172.21.15.77:0/1383681894 shutdown_connections 2023-12-01T01:17:19.902 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.894+0000 7f25922aab80 1 -- 172.21.15.77:0/1383681894 wait complete. 2023-12-01T01:17:19.902 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.894+0000 7f25922aab80 20 client.15138 trim_cache size 0 max 16384 2023-12-01T01:17:19.903 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2023-12-01T01:17:19.903 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "2a06e29a-8fe6-11ee-95a2-87774f69a715", 2023-12-01T01:17:19.903 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 0, 2023-12-01T01:17:19.903 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24500, 2023-12-01T01:17:19.903 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2023-12-01T01:17:19.903 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2023-12-01T01:17:19.903 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2023-12-01T01:17:19.903 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 54.546756121999998, 2023-12-01T01:17:19.903 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 27, 2023-12-01T01:17:19.903 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 82, 2023-12-01T01:17:19.903 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 82, 2023-12-01T01:17:19.903 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 60.024728609 2023-12-01T01:17:19.903 INFO:teuthology.orchestra.run.smithi077.stdout:} 2023-12-01T01:17:19.904 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.894+0000 7f258611a700 1 -- 172.21.15.77:0/2535368221 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f25680a05b0 msgr2=0x7f25680a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.904 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.894+0000 7f258611a700 1 --2- 172.21.15.77:0/2535368221 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f25680a05b0 0x7f25680a2a70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2580100ce0 tx=0x7f257000a380 comp rx=0 tx=0).stop 2023-12-01T01:17:19.904 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.894+0000 7f258611a700 1 -- 172.21.15.77:0/2535368221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2580100e80 msgr2=0x7f2580195f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:19.904 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.894+0000 7f258611a700 1 --2- 172.21.15.77:0/2535368221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2580100e80 0x7f2580195f30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f257400c9b0 tx=0x7f257400ccc0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.904 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.894+0000 7f258611a700 1 -- 172.21.15.77:0/2535368221 shutdown_connections 2023-12-01T01:17:19.904 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.894+0000 7f258611a700 1 --2- 172.21.15.77:0/2535368221 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f25680a05b0 0x7f25680a2a70 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.905 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.894+0000 7f258611a700 1 --2- 172.21.15.77:0/2535368221 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2580101820 0x7f258019cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.905 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.894+0000 7f258611a700 1 --2- 172.21.15.77:0/2535368221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25800ffc90 0x7f25801959f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.905 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.894+0000 7f258611a700 1 --2- 172.21.15.77:0/2535368221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2580100e80 0x7f2580195f30 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:19.905 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.894+0000 7f258611a700 1 -- 172.21.15.77:0/2535368221 >> 172.21.15.77:0/2535368221 conn(0x7f25800fb220 msgr2=0x7f25800fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:19.905 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.894+0000 7f258611a700 1 -- 172.21.15.77:0/2535368221 shutdown_connections 2023-12-01T01:17:19.905 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:19.894+0000 7f258611a700 1 -- 172.21.15.77:0/2535368221 wait complete. 2023-12-01T01:17:19.907 INFO:tasks.cephadm:Setup complete, yielding 2023-12-01T01:17:19.908 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-12-01T01:17:19.917 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi077.front.sepia.ceph.com 2023-12-01T01:17:19.917 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- bash -c 'ceph orch status' 2023-12-01T01:17:19.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:19 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/2629260852' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:17:19.941 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:19 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/2966966068' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:17:20.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:19 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/2629260852' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:17:20.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:19 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/2966966068' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:17:20.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:19 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/2629260852' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:17:20.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:19 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/2966966068' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:17:20.469 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:17:20.999 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:20 smithi077 ceph-mon[98867]: pgmap v280: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-01T01:17:21.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:20 smithi203 ceph-mon[103581]: pgmap v280: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-01T01:17:21.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:20 smithi137 ceph-mon[103312]: pgmap v280: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-01T01:17:21.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.652+0000 7f73fc13d700 1 -- 172.21.15.77:0/289850697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f73f4106bb0 msgr2=0x7f73f4108fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:21.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.652+0000 7f73fc13d700 1 --2- 172.21.15.77:0/289850697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f73f4106bb0 0x7f73f4108fa0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f73ec00b0a0 tx=0x7f73ec00b3b0 comp rx=0 tx=0).stop 2023-12-01T01:17:21.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.652+0000 7f73fc13d700 1 -- 172.21.15.77:0/289850697 shutdown_connections 2023-12-01T01:17:21.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.652+0000 7f73fc13d700 1 --2- 172.21.15.77:0/289850697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f73f4106bb0 0x7f73f4108fa0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:21.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.652+0000 7f73fc13d700 1 --2- 172.21.15.77:0/289850697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f73f4101950 0x7f73f4103d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:21.656 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.652+0000 7f73fc13d700 1 --2- 172.21.15.77:0/289850697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73f4104280 0x7f73f4106670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:21.656 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.652+0000 7f73fc13d700 1 -- 172.21.15.77:0/289850697 >> 172.21.15.77:0/289850697 conn(0x7f73f40fb2a0 msgr2=0x7f73f40fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:21.656 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.652+0000 7f73fc13d700 1 -- 172.21.15.77:0/289850697 shutdown_connections 2023-12-01T01:17:21.656 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.652+0000 7f73fc13d700 1 -- 172.21.15.77:0/289850697 wait complete. 2023-12-01T01:17:21.656 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.653+0000 7f73fc13d700 1 Processor -- start 2023-12-01T01:17:21.656 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.653+0000 7f73fc13d700 1 -- start start 2023-12-01T01:17:21.656 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.653+0000 7f73fc13d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f73f4101950 0x7f73f4196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:21.656 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.653+0000 7f73fc13d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73f4104280 0x7f73f4197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:21.656 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.653+0000 7f73fc13d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f73f4106bb0 0x7f73f419e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:21.656 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.653+0000 7f73fc13d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f419e690 con 0x7f73f4106bb0 2023-12-01T01:17:21.656 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.653+0000 7f73fc13d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f419e7d0 con 0x7f73f4101950 2023-12-01T01:17:21.656 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.653+0000 7f73fc13d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f419e960 con 0x7f73f4104280 2023-12-01T01:17:21.656 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.653+0000 7f73fa6da700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f73f4106bb0 0x7f73f419e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:21.656 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.653+0000 7f73fa6da700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f73f4106bb0 0x7f73f419e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58528/0 (socket says 172.21.15.77:58528) 2023-12-01T01:17:21.656 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.653+0000 7f73fa6da700 1 -- 172.21.15.77:0/3830959024 learned_addr learned my addr 172.21.15.77:0/3830959024 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:21.657 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.653+0000 7f73f9ed9700 1 --2- 172.21.15.77:0/3830959024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f73f4101950 0x7f73f4196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:21.657 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.653+0000 7f73f96d8700 1 --2- 172.21.15.77:0/3830959024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73f4104280 0x7f73f4197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:21.657 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.653+0000 7f73fa6da700 1 -- 172.21.15.77:0/3830959024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73f4104280 msgr2=0x7f73f4197040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:21.657 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.653+0000 7f73fa6da700 1 --2- 172.21.15.77:0/3830959024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73f4104280 0x7f73f4197040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:21.657 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.653+0000 7f73fa6da700 1 -- 172.21.15.77:0/3830959024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f73f4101950 msgr2=0x7f73f4196b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:21.657 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.653+0000 7f73fa6da700 1 --2- 172.21.15.77:0/3830959024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f73f4101950 0x7f73f4196b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:21.657 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.653+0000 7f73fa6da700 1 -- 172.21.15.77:0/3830959024 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73ec009d00 con 0x7f73f4106bb0 2023-12-01T01:17:21.657 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.653+0000 7f73fa6da700 1 --2- 172.21.15.77:0/3830959024 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f73f4106bb0 0x7f73f419e0c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f73ec00b070 tx=0x7f73ec004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:21.657 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.654+0000 7f73e6ffd700 1 -- 172.21.15.77:0/3830959024 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ec00e040 con 0x7f73f4106bb0 2023-12-01T01:17:21.657 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.654+0000 7f73fc13d700 1 -- 172.21.15.77:0/3830959024 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73f419ebe0 con 0x7f73f4106bb0 2023-12-01T01:17:21.657 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.654+0000 7f73e6ffd700 1 -- 172.21.15.77:0/3830959024 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73ec009440 con 0x7f73f4106bb0 2023-12-01T01:17:21.657 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.654+0000 7f73e6ffd700 1 -- 172.21.15.77:0/3830959024 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ec004020 con 0x7f73f4106bb0 2023-12-01T01:17:21.657 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.654+0000 7f73fc13d700 1 -- 172.21.15.77:0/3830959024 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73f419f0a0 con 0x7f73f4106bb0 2023-12-01T01:17:21.658 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.655+0000 7f73fc13d700 1 -- 172.21.15.77:0/3830959024 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73f40fce20 con 0x7f73f4106bb0 2023-12-01T01:17:21.658 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.656+0000 7f73e6ffd700 1 -- 172.21.15.77:0/3830959024 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f73ec01b660 con 0x7f73f4106bb0 2023-12-01T01:17:21.658 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.656+0000 7f73e6ffd700 1 --2- 172.21.15.77:0/3830959024 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f73e00a05b0 0x7f73e00a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:21.658 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.656+0000 7f73f9ed9700 1 --2- 172.21.15.77:0/3830959024 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f73e00a05b0 0x7f73e00a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:21.658 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.657+0000 7f73e6ffd700 1 -- 172.21.15.77:0/3830959024 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f73ec0cd180 con 0x7f73f4106bb0 2023-12-01T01:17:21.659 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.657+0000 7f73f9ed9700 1 --2- 172.21.15.77:0/3830959024 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f73e00a05b0 0x7f73e00a2a70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f73e8005950 tx=0x7f73e800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:21.662 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.662+0000 7f73e6ffd700 1 -- 172.21.15.77:0/3830959024 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f73ec096d00 con 0x7f73f4106bb0 2023-12-01T01:17:21.867 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.866+0000 7f73fc13d700 1 -- 172.21.15.77:0/3830959024 --> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f73f40611d0 con 0x7f73e00a05b0 2023-12-01T01:17:21.868 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.867+0000 7f73e6ffd700 1 -- 172.21.15.77:0/3830959024 <== mgr.14152 v2:172.21.15.77:6800/3838941679 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f73f40611d0 con 0x7f73e00a05b0 2023-12-01T01:17:21.868 INFO:teuthology.orchestra.run.smithi077.stdout:Backend: cephadm 2023-12-01T01:17:21.868 INFO:teuthology.orchestra.run.smithi077.stdout:Available: Yes 2023-12-01T01:17:21.868 INFO:teuthology.orchestra.run.smithi077.stdout:Paused: No 2023-12-01T01:17:21.870 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.869+0000 7f73fc13d700 1 -- 172.21.15.77:0/3830959024 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f73e00a05b0 msgr2=0x7f73e00a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:21.870 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.869+0000 7f73fc13d700 1 --2- 172.21.15.77:0/3830959024 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f73e00a05b0 0x7f73e00a2a70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f73e8005950 tx=0x7f73e800a300 comp rx=0 tx=0).stop 2023-12-01T01:17:21.871 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.869+0000 7f73fc13d700 1 -- 172.21.15.77:0/3830959024 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f73f4106bb0 msgr2=0x7f73f419e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:21.871 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.869+0000 7f73fc13d700 1 --2- 172.21.15.77:0/3830959024 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f73f4106bb0 0x7f73f419e0c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f73ec00b070 tx=0x7f73ec004d00 comp rx=0 tx=0).stop 2023-12-01T01:17:21.871 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.869+0000 7f73fc13d700 1 -- 172.21.15.77:0/3830959024 shutdown_connections 2023-12-01T01:17:21.871 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.869+0000 7f73fc13d700 1 --2- 172.21.15.77:0/3830959024 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f73e00a05b0 0x7f73e00a2a70 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:21.871 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.869+0000 7f73fc13d700 1 --2- 172.21.15.77:0/3830959024 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f73f4106bb0 0x7f73f419e0c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:21.871 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.869+0000 7f73fc13d700 1 --2- 172.21.15.77:0/3830959024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f73f4101950 0x7f73f4196b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:21.871 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.869+0000 7f73fc13d700 1 --2- 172.21.15.77:0/3830959024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73f4104280 0x7f73f4197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:21.871 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.869+0000 7f73fc13d700 1 -- 172.21.15.77:0/3830959024 >> 172.21.15.77:0/3830959024 conn(0x7f73f40fb2a0 msgr2=0x7f73f41025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:21.871 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.869+0000 7f73fc13d700 1 -- 172.21.15.77:0/3830959024 shutdown_connections 2023-12-01T01:17:21.871 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:21.869+0000 7f73fc13d700 1 -- 172.21.15.77:0/3830959024 wait complete. 2023-12-01T01:17:22.226 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- bash -c 'ceph orch ps' 2023-12-01T01:17:22.819 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:17:23.878 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:23 smithi077 ceph-mon[98867]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:17:23.878 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:23 smithi077 ceph-mon[98867]: pgmap v281: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-12-01T01:17:24.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.000+0000 7f9014ba9700 1 -- 172.21.15.77:0/1130892134 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9010100f20 msgr2=0x7f9010101380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:24.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.000+0000 7f9014ba9700 1 --2- 172.21.15.77:0/1130892134 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9010100f20 0x7f9010101380 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f90000099c0 tx=0x7f9000009cd0 comp rx=0 tx=0).stop 2023-12-01T01:17:24.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.000+0000 7f9014ba9700 1 -- 172.21.15.77:0/1130892134 shutdown_connections 2023-12-01T01:17:24.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.000+0000 7f9014ba9700 1 --2- 172.21.15.77:0/1130892134 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90101018c0 0x7f9010108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:24.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.000+0000 7f9014ba9700 1 --2- 172.21.15.77:0/1130892134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90100ffd20 0x7f9010100140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:24.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.000+0000 7f9014ba9700 1 --2- 172.21.15.77:0/1130892134 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9010100f20 0x7f9010101380 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:24.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.000+0000 7f9014ba9700 1 -- 172.21.15.77:0/1130892134 >> 172.21.15.77:0/1130892134 conn(0x7f90100fb2a0 msgr2=0x7f90100fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:24.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.000+0000 7f9014ba9700 1 -- 172.21.15.77:0/1130892134 shutdown_connections 2023-12-01T01:17:24.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.000+0000 7f9014ba9700 1 -- 172.21.15.77:0/1130892134 wait complete. 2023-12-01T01:17:24.005 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.001+0000 7f9014ba9700 1 Processor -- start 2023-12-01T01:17:24.005 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.001+0000 7f9014ba9700 1 -- start start 2023-12-01T01:17:24.005 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.001+0000 7f9014ba9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90100ffd20 0x7f9010103cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:24.005 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.001+0000 7f9014ba9700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9010100f20 0x7f9010102300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:24.005 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.001+0000 7f9014ba9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90101018c0 0x7f9010102840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:24.005 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.001+0000 7f9014ba9700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9010102fc0 con 0x7f90101018c0 2023-12-01T01:17:24.005 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.001+0000 7f9014ba9700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9010103130 con 0x7f9010100f20 2023-12-01T01:17:24.006 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.001+0000 7f9014ba9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90101a0b80 con 0x7f90100ffd20 2023-12-01T01:17:24.006 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.001+0000 7f900ed9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90101018c0 0x7f9010102840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:24.006 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.001+0000 7f900ed9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90101018c0 0x7f9010102840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58538/0 (socket says 172.21.15.77:58538) 2023-12-01T01:17:24.006 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.001+0000 7f900ed9d700 1 -- 172.21.15.77:0/203364352 learned_addr learned my addr 172.21.15.77:0/203364352 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:24.006 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.001+0000 7f900e59c700 1 --2- 172.21.15.77:0/203364352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90100ffd20 0x7f9010103cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:24.006 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.002+0000 7f900dd9b700 1 --2- 172.21.15.77:0/203364352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9010100f20 0x7f9010102300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:24.006 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.002+0000 7f900ed9d700 1 -- 172.21.15.77:0/203364352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90100ffd20 msgr2=0x7f9010103cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:24.006 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.002+0000 7f900ed9d700 1 --2- 172.21.15.77:0/203364352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90100ffd20 0x7f9010103cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:24.007 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.002+0000 7f900ed9d700 1 -- 172.21.15.77:0/203364352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9010100f20 msgr2=0x7f9010102300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:24.007 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.002+0000 7f900ed9d700 1 --2- 172.21.15.77:0/203364352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9010100f20 0x7f9010102300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:24.007 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.002+0000 7f900ed9d700 1 -- 172.21.15.77:0/203364352 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90000095f0 con 0x7f90101018c0 2023-12-01T01:17:24.007 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.002+0000 7f900ed9d700 1 --2- 172.21.15.77:0/203364352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90101018c0 0x7f9010102840 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f900800b970 tx=0x7f900800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:24.007 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.002+0000 7f8fff7fe700 1 -- 172.21.15.77:0/203364352 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900800c930 con 0x7f90101018c0 2023-12-01T01:17:24.007 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.002+0000 7f8fff7fe700 1 -- 172.21.15.77:0/203364352 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9008014440 con 0x7f90101018c0 2023-12-01T01:17:24.007 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.002+0000 7f9014ba9700 1 -- 172.21.15.77:0/203364352 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90101a0e60 con 0x7f90101018c0 2023-12-01T01:17:24.007 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.002+0000 7f8fff7fe700 1 -- 172.21.15.77:0/203364352 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90080075d0 con 0x7f90101018c0 2023-12-01T01:17:24.007 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.002+0000 7f9014ba9700 1 -- 172.21.15.77:0/203364352 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90101a12d0 con 0x7f90101018c0 2023-12-01T01:17:24.007 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.003+0000 7f9014ba9700 1 -- 172.21.15.77:0/203364352 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f901010b5a0 con 0x7f90101018c0 2023-12-01T01:17:24.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.004+0000 7f8fff7fe700 1 -- 172.21.15.77:0/203364352 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f90080145b0 con 0x7f90101018c0 2023-12-01T01:17:24.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.005+0000 7f8fff7fe700 1 --2- 172.21.15.77:0/203364352 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8ff80a05b0 0x7f8ff80a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:24.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.005+0000 7f900e59c700 1 --2- 172.21.15.77:0/203364352 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8ff80a05b0 0x7f8ff80a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:24.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.005+0000 7f8fff7fe700 1 -- 172.21.15.77:0/203364352 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9008006e40 con 0x7f90101018c0 2023-12-01T01:17:24.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.005+0000 7f900e59c700 1 --2- 172.21.15.77:0/203364352 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8ff80a05b0 0x7f8ff80a2a70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9010100d80 tx=0x7f900400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:24.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.008+0000 7f8fff7fe700 1 -- 172.21.15.77:0/203364352 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9008094df0 con 0x7f90101018c0 2023-12-01T01:17:24.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:23 smithi203 ceph-mon[103581]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:17:24.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:23 smithi203 ceph-mon[103581]: pgmap v281: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-12-01T01:17:24.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:23 smithi137 ceph-mon[103312]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:17:24.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:23 smithi137 ceph-mon[103312]: pgmap v281: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-12-01T01:17:24.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.205+0000 7f9014ba9700 1 -- 172.21.15.77:0/203364352 --> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9010069870 con 0x7f8ff80a05b0 2023-12-01T01:17:24.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.218+0000 7f8fff7fe700 1 -- 172.21.15.77:0/203364352 <== mgr.14152 v2:172.21.15.77:6800/3838941679 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f9010069870 con 0x7f8ff80a05b0 2023-12-01T01:17:24.219 INFO:teuthology.orchestra.run.smithi077.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-12-01T01:17:24.220 INFO:teuthology.orchestra.run.smithi077.stdout:mds.a smithi077 running (66s) 57s ago 66s 16.5M - 18.2.0-1249-ga6f82997 96d837748436 93e77b2a14a9 2023-12-01T01:17:24.220 INFO:teuthology.orchestra.run.smithi077.stdout:mds.b smithi137 running (64s) 57s ago 64s 16.6M - 18.2.0-1249-ga6f82997 96d837748436 58d18dc2de8b 2023-12-01T01:17:24.220 INFO:teuthology.orchestra.run.smithi077.stdout:mds.c smithi203 running (75s) 57s ago 75s 13.5M - 18.2.0-1249-ga6f82997 96d837748436 1415fa67b514 2023-12-01T01:17:24.220 INFO:teuthology.orchestra.run.smithi077.stdout:mds.d smithi077 running (82s) 57s ago 81s 14.3M - 18.2.0-1249-ga6f82997 96d837748436 f40a9289b76c 2023-12-01T01:17:24.220 INFO:teuthology.orchestra.run.smithi077.stdout:mds.e smithi137 running (80s) 57s ago 79s 13.4M - 18.2.0-1249-ga6f82997 96d837748436 afce27568d7f 2023-12-01T01:17:24.220 INFO:teuthology.orchestra.run.smithi077.stdout:mds.f smithi203 running (77s) 57s ago 77s 15.7M - 18.2.0-1249-ga6f82997 96d837748436 a068deb4a29d 2023-12-01T01:17:24.220 INFO:teuthology.orchestra.run.smithi077.stdout:mds.g smithi077 running (69s) 57s ago 68s 13.3M - 18.2.0-1249-ga6f82997 96d837748436 6472af40a783 2023-12-01T01:17:24.220 INFO:teuthology.orchestra.run.smithi077.stdout:mds.h smithi137 running (73s) 57s ago 72s 15.7M - 18.2.0-1249-ga6f82997 96d837748436 e3dafb8bd166 2023-12-01T01:17:24.220 INFO:teuthology.orchestra.run.smithi077.stdout:mds.i smithi203 running (71s) 57s ago 70s 14.3M - 18.2.0-1249-ga6f82997 96d837748436 6d5196850fb8 2023-12-01T01:17:24.220 INFO:teuthology.orchestra.run.smithi077.stdout:mds.j smithi077 running (84s) 57s ago 83s 14.4M - 18.2.0-1249-ga6f82997 96d837748436 a78b678a6a67 2023-12-01T01:17:24.220 INFO:teuthology.orchestra.run.smithi077.stdout:mds.k smithi137 running (86s) 57s ago 85s 14.4M - 18.2.0-1249-ga6f82997 96d837748436 b5339df18575 2023-12-01T01:17:24.221 INFO:teuthology.orchestra.run.smithi077.stdout:mds.l smithi203 running (88s) 57s ago 88s 14.4M - 18.2.0-1249-ga6f82997 96d837748436 c554faf3c27a 2023-12-01T01:17:24.221 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.x smithi077 *:9283,8765 running (7m) 57s ago 7m 514M - 18.2.0-1249-ga6f82997 96d837748436 c7e874ec0c68 2023-12-01T01:17:24.221 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.y smithi137 *:8443,8765 running (6m) 57s ago 5m 448M - 18.2.0-1249-ga6f82997 96d837748436 17f5ee33ffc9 2023-12-01T01:17:24.221 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.z smithi203 *:8443,8765 running (6m) 57s ago 6m 450M - 18.2.0-1249-ga6f82997 96d837748436 f7dd0598a3fb 2023-12-01T01:17:24.221 INFO:teuthology.orchestra.run.smithi077.stdout:mon.a smithi077 running (7m) 57s ago 7m 86.6M 2048M 18.2.0-1249-ga6f82997 96d837748436 b7510927ab44 2023-12-01T01:17:24.221 INFO:teuthology.orchestra.run.smithi077.stdout:mon.b smithi137 running (6m) 57s ago 6m 55.5M 2048M 18.2.0-1249-ga6f82997 96d837748436 2db06a6075f8 2023-12-01T01:17:24.221 INFO:teuthology.orchestra.run.smithi077.stdout:mon.c smithi203 running (6m) 57s ago 6m 59.2M 2048M 18.2.0-1249-ga6f82997 96d837748436 9cc0caf64657 2023-12-01T01:17:24.221 INFO:teuthology.orchestra.run.smithi077.stdout:osd.0 smithi077 running (5m) 57s ago 5m 53.6M 4292M 18.2.0-1249-ga6f82997 96d837748436 a0574199e830 2023-12-01T01:17:24.221 INFO:teuthology.orchestra.run.smithi077.stdout:osd.1 smithi137 running (5m) 57s ago 5m 59.0M 4292M 18.2.0-1249-ga6f82997 96d837748436 d2f8ff14a167 2023-12-01T01:17:24.221 INFO:teuthology.orchestra.run.smithi077.stdout:osd.2 smithi203 running (5m) 57s ago 5m 55.8M 4292M 18.2.0-1249-ga6f82997 96d837748436 620d0c169f0c 2023-12-01T01:17:24.221 INFO:teuthology.orchestra.run.smithi077.stdout:osd.3 smithi077 running (4m) 57s ago 4m 51.7M 4292M 18.2.0-1249-ga6f82997 96d837748436 0354b8681288 2023-12-01T01:17:24.221 INFO:teuthology.orchestra.run.smithi077.stdout:osd.4 smithi137 running (4m) 57s ago 4m 48.8M 4292M 18.2.0-1249-ga6f82997 96d837748436 3da52659735e 2023-12-01T01:17:24.221 INFO:teuthology.orchestra.run.smithi077.stdout:osd.5 smithi203 running (3m) 57s ago 3m 55.5M 4292M 18.2.0-1249-ga6f82997 96d837748436 dbb87fbdff8c 2023-12-01T01:17:24.221 INFO:teuthology.orchestra.run.smithi077.stdout:osd.6 smithi077 running (3m) 57s ago 3m 47.2M 4292M 18.2.0-1249-ga6f82997 96d837748436 7662d5a2f59b 2023-12-01T01:17:24.221 INFO:teuthology.orchestra.run.smithi077.stdout:osd.7 smithi137 running (3m) 57s ago 3m 53.0M 4292M 18.2.0-1249-ga6f82997 96d837748436 0004a2222869 2023-12-01T01:17:24.222 INFO:teuthology.orchestra.run.smithi077.stdout:osd.8 smithi203 running (2m) 57s ago 2m 52.8M 4292M 18.2.0-1249-ga6f82997 96d837748436 80b16a08e86d 2023-12-01T01:17:24.222 INFO:teuthology.orchestra.run.smithi077.stdout:osd.9 smithi077 running (2m) 57s ago 2m 48.0M 4292M 18.2.0-1249-ga6f82997 96d837748436 7afe374a6ead 2023-12-01T01:17:24.222 INFO:teuthology.orchestra.run.smithi077.stdout:osd.10 smithi137 running (2m) 57s ago 2m 51.9M 4292M 18.2.0-1249-ga6f82997 96d837748436 02f767b551ba 2023-12-01T01:17:24.222 INFO:teuthology.orchestra.run.smithi077.stdout:osd.11 smithi203 running (106s) 57s ago 106s 45.1M 4292M 18.2.0-1249-ga6f82997 96d837748436 caafc6e3e32d 2023-12-01T01:17:24.222 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.220+0000 7f9014ba9700 1 -- 172.21.15.77:0/203364352 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8ff80a05b0 msgr2=0x7f8ff80a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:24.223 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.220+0000 7f9014ba9700 1 --2- 172.21.15.77:0/203364352 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8ff80a05b0 0x7f8ff80a2a70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9010100d80 tx=0x7f900400b410 comp rx=0 tx=0).stop 2023-12-01T01:17:24.223 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.220+0000 7f9014ba9700 1 -- 172.21.15.77:0/203364352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90101018c0 msgr2=0x7f9010102840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:24.223 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.220+0000 7f9014ba9700 1 --2- 172.21.15.77:0/203364352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90101018c0 0x7f9010102840 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f900800b970 tx=0x7f900800bc80 comp rx=0 tx=0).stop 2023-12-01T01:17:24.223 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.220+0000 7f9014ba9700 1 -- 172.21.15.77:0/203364352 shutdown_connections 2023-12-01T01:17:24.223 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.220+0000 7f9014ba9700 1 --2- 172.21.15.77:0/203364352 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8ff80a05b0 0x7f8ff80a2a70 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:24.223 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.220+0000 7f9014ba9700 1 --2- 172.21.15.77:0/203364352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90101018c0 0x7f9010102840 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:24.223 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.220+0000 7f9014ba9700 1 --2- 172.21.15.77:0/203364352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90100ffd20 0x7f9010103cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:24.223 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.220+0000 7f9014ba9700 1 --2- 172.21.15.77:0/203364352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9010100f20 0x7f9010102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:24.224 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.220+0000 7f9014ba9700 1 -- 172.21.15.77:0/203364352 >> 172.21.15.77:0/203364352 conn(0x7f90100fb2a0 msgr2=0x7f9010069160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:24.224 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.220+0000 7f9014ba9700 1 -- 172.21.15.77:0/203364352 shutdown_connections 2023-12-01T01:17:24.224 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:24.220+0000 7f9014ba9700 1 -- 172.21.15.77:0/203364352 wait complete. 2023-12-01T01:17:24.563 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- bash -c 'ceph orch ls' 2023-12-01T01:17:24.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:24 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2023-12-01T01:17:24.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:24 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2023-12-01T01:17:24.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:24 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2023-12-01T01:17:24.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:24 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]: dispatch 2023-12-01T01:17:24.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:24 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2023-12-01T01:17:24.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:24 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2023-12-01T01:17:25.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:24 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2023-12-01T01:17:25.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:24 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2023-12-01T01:17:25.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:24 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2023-12-01T01:17:25.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:24 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]: dispatch 2023-12-01T01:17:25.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:24 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2023-12-01T01:17:25.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:24 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2023-12-01T01:17:25.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:24 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2023-12-01T01:17:25.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:24 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2023-12-01T01:17:25.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:24 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2023-12-01T01:17:25.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:24 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]: dispatch 2023-12-01T01:17:25.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:24 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2023-12-01T01:17:25.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:24 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2023-12-01T01:17:25.119 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:17:25.617 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:25 smithi077 ceph-mon[98867]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:17:25.617 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:25 smithi077 ceph-mon[98867]: pgmap v282: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-12-01T01:17:25.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:25 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2023-12-01T01:17:25.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:25 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2023-12-01T01:17:25.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:25 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2023-12-01T01:17:25.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:25 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]': finished 2023-12-01T01:17:25.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:25 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2023-12-01T01:17:25.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:25 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2023-12-01T01:17:25.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:25 smithi077 ceph-mon[98867]: osdmap e83: 12 total, 12 up, 12 in 2023-12-01T01:17:26.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:25 smithi203 ceph-mon[103581]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:17:26.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:25 smithi203 ceph-mon[103581]: pgmap v282: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-12-01T01:17:26.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:25 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2023-12-01T01:17:26.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:25 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2023-12-01T01:17:26.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:25 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2023-12-01T01:17:26.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:25 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]': finished 2023-12-01T01:17:26.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:25 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2023-12-01T01:17:26.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:25 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2023-12-01T01:17:26.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:25 smithi203 ceph-mon[103581]: osdmap e83: 12 total, 12 up, 12 in 2023-12-01T01:17:26.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:25 smithi137 ceph-mon[103312]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:17:26.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:25 smithi137 ceph-mon[103312]: pgmap v282: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-12-01T01:17:26.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:25 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2023-12-01T01:17:26.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:25 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2023-12-01T01:17:26.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:25 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2023-12-01T01:17:26.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:25 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 8]}]': finished 2023-12-01T01:17:26.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:25 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2023-12-01T01:17:26.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:25 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2023-12-01T01:17:26.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:25 smithi137 ceph-mon[103312]: osdmap e83: 12 total, 12 up, 12 in 2023-12-01T01:17:26.270 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.267+0000 7f4a885b1700 1 -- 172.21.15.77:0/1696835119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a800ffd30 msgr2=0x7f4a80100150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:26.271 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.267+0000 7f4a885b1700 1 --2- 172.21.15.77:0/1696835119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a800ffd30 0x7f4a80100150 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4a700099c0 tx=0x7f4a70009cd0 comp rx=0 tx=0).stop 2023-12-01T01:17:26.271 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.267+0000 7f4a885b1700 1 -- 172.21.15.77:0/1696835119 shutdown_connections 2023-12-01T01:17:26.271 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.267+0000 7f4a885b1700 1 --2- 172.21.15.77:0/1696835119 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a801018d0 0x7f4a80108180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:26.271 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.267+0000 7f4a885b1700 1 --2- 172.21.15.77:0/1696835119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a800ffd30 0x7f4a80100150 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:26.271 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.267+0000 7f4a885b1700 1 --2- 172.21.15.77:0/1696835119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a80100f30 0x7f4a80101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:26.271 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.267+0000 7f4a885b1700 1 -- 172.21.15.77:0/1696835119 >> 172.21.15.77:0/1696835119 conn(0x7f4a800fb2d0 msgr2=0x7f4a800fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:26.271 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.267+0000 7f4a885b1700 1 -- 172.21.15.77:0/1696835119 shutdown_connections 2023-12-01T01:17:26.271 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.267+0000 7f4a885b1700 1 -- 172.21.15.77:0/1696835119 wait complete. 2023-12-01T01:17:26.271 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.268+0000 7f4a885b1700 1 Processor -- start 2023-12-01T01:17:26.271 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.268+0000 7f4a885b1700 1 -- start start 2023-12-01T01:17:26.271 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.268+0000 7f4a885b1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a800ffd30 0x7f4a80196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:26.271 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.268+0000 7f4a885b1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a80100f30 0x7f4a80197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:26.271 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.268+0000 7f4a885b1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a801018d0 0x7f4a8019e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:26.272 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.268+0000 7f4a885b1700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8019e680 con 0x7f4a80100f30 2023-12-01T01:17:26.272 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.268+0000 7f4a885b1700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8019e7c0 con 0x7f4a800ffd30 2023-12-01T01:17:26.272 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.268+0000 7f4a885b1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8019e950 con 0x7f4a801018d0 2023-12-01T01:17:26.272 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.268+0000 7f4a85b4c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a80100f30 0x7f4a80197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:26.272 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.268+0000 7f4a85b4c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a80100f30 0x7f4a80197030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58550/0 (socket says 172.21.15.77:58550) 2023-12-01T01:17:26.272 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.268+0000 7f4a85b4c700 1 -- 172.21.15.77:0/3991157767 learned_addr learned my addr 172.21.15.77:0/3991157767 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:26.272 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.268+0000 7f4a8634d700 1 --2- 172.21.15.77:0/3991157767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a800ffd30 0x7f4a80196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:26.272 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.268+0000 7f4a86b4e700 1 --2- 172.21.15.77:0/3991157767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a801018d0 0x7f4a8019e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:26.272 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.268+0000 7f4a85b4c700 1 -- 172.21.15.77:0/3991157767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a801018d0 msgr2=0x7f4a8019e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:26.272 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.268+0000 7f4a85b4c700 1 --2- 172.21.15.77:0/3991157767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a801018d0 0x7f4a8019e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:26.272 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.268+0000 7f4a85b4c700 1 -- 172.21.15.77:0/3991157767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a800ffd30 msgr2=0x7f4a80196af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:26.272 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.268+0000 7f4a85b4c700 1 --2- 172.21.15.77:0/3991157767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a800ffd30 0x7f4a80196af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:26.272 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.268+0000 7f4a85b4c700 1 -- 172.21.15.77:0/3991157767 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a700095f0 con 0x7f4a80100f30 2023-12-01T01:17:26.272 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.268+0000 7f4a86b4e700 1 --2- 172.21.15.77:0/3991157767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a801018d0 0x7f4a8019e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:17:26.272 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.268+0000 7f4a85b4c700 1 --2- 172.21.15.77:0/3991157767 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a80100f30 0x7f4a80197030 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c00c950 tx=0x7f4a7c00cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:26.273 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.269+0000 7f4a777fe700 1 -- 172.21.15.77:0/3991157767 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c007ac0 con 0x7f4a80100f30 2023-12-01T01:17:26.273 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.269+0000 7f4a777fe700 1 -- 172.21.15.77:0/3991157767 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a7c004cf0 con 0x7f4a80100f30 2023-12-01T01:17:26.273 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.269+0000 7f4a777fe700 1 -- 172.21.15.77:0/3991157767 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c005550 con 0x7f4a80100f30 2023-12-01T01:17:26.273 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.269+0000 7f4a885b1700 1 -- 172.21.15.77:0/3991157767 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a8019ec30 con 0x7f4a80100f30 2023-12-01T01:17:26.273 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.269+0000 7f4a885b1700 1 -- 172.21.15.77:0/3991157767 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a8019f130 con 0x7f4a80100f30 2023-12-01T01:17:26.274 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.270+0000 7f4a885b1700 1 -- 172.21.15.77:0/3991157767 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a8010b530 con 0x7f4a80100f30 2023-12-01T01:17:26.274 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.271+0000 7f4a777fe700 1 -- 172.21.15.77:0/3991157767 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f4a7c004750 con 0x7f4a80100f30 2023-12-01T01:17:26.274 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.271+0000 7f4a777fe700 1 --2- 172.21.15.77:0/3991157767 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f4a6c0a06b0 0x7f4a6c0a2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:26.274 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.272+0000 7f4a8634d700 1 --2- 172.21.15.77:0/3991157767 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f4a6c0a06b0 0x7f4a6c0a2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:26.274 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.272+0000 7f4a777fe700 1 -- 172.21.15.77:0/3991157767 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f4a7c0cb6c0 con 0x7f4a80100f30 2023-12-01T01:17:26.274 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.272+0000 7f4a8634d700 1 --2- 172.21.15.77:0/3991157767 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f4a6c0a06b0 0x7f4a6c0a2b70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4a7000b580 tx=0x7f4a70005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:26.277 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.276+0000 7f4a777fe700 1 -- 172.21.15.77:0/3991157767 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4a7c095100 con 0x7f4a80100f30 2023-12-01T01:17:26.475 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.474+0000 7f4a885b1700 1 -- 172.21.15.77:0/3991157767 --> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a80069940 con 0x7f4a6c0a06b0 2023-12-01T01:17:26.480 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.479+0000 7f4a777fe700 1 -- 172.21.15.77:0/3991157767 <== mgr.14152 v2:172.21.15.77:6800/3838941679 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f4a80069940 con 0x7f4a6c0a06b0 2023-12-01T01:17:26.481 INFO:teuthology.orchestra.run.smithi077.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-12-01T01:17:26.481 INFO:teuthology.orchestra.run.smithi077.stdout:mds.all 12/12 60s ago 92s smithi077=a;smithi077=d;smithi077=g;smithi077=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l;count:12 2023-12-01T01:17:26.481 INFO:teuthology.orchestra.run.smithi077.stdout:mgr 3/3 60s ago 6m smithi077=x;smithi137=y;smithi203=z;count:3 2023-12-01T01:17:26.481 INFO:teuthology.orchestra.run.smithi077.stdout:mon 3/3 60s ago 6m smithi077:172.21.15.77=a;smithi137:172.21.15.137=b;smithi203:172.21.15.203=c;count:3 2023-12-01T01:17:26.481 INFO:teuthology.orchestra.run.smithi077.stdout:osd 12 60s ago - 2023-12-01T01:17:26.483 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.481+0000 7f4a885b1700 1 -- 172.21.15.77:0/3991157767 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f4a6c0a06b0 msgr2=0x7f4a6c0a2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:26.483 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.481+0000 7f4a885b1700 1 --2- 172.21.15.77:0/3991157767 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f4a6c0a06b0 0x7f4a6c0a2b70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4a7000b580 tx=0x7f4a70005fb0 comp rx=0 tx=0).stop 2023-12-01T01:17:26.483 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.481+0000 7f4a885b1700 1 -- 172.21.15.77:0/3991157767 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a80100f30 msgr2=0x7f4a80197030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:26.483 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.481+0000 7f4a885b1700 1 --2- 172.21.15.77:0/3991157767 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a80100f30 0x7f4a80197030 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c00c950 tx=0x7f4a7c00cd10 comp rx=0 tx=0).stop 2023-12-01T01:17:26.483 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.481+0000 7f4a885b1700 1 -- 172.21.15.77:0/3991157767 shutdown_connections 2023-12-01T01:17:26.483 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.481+0000 7f4a885b1700 1 --2- 172.21.15.77:0/3991157767 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f4a6c0a06b0 0x7f4a6c0a2b70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:26.483 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.482+0000 7f4a885b1700 1 --2- 172.21.15.77:0/3991157767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a801018d0 0x7f4a8019e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:26.483 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.482+0000 7f4a885b1700 1 --2- 172.21.15.77:0/3991157767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a800ffd30 0x7f4a80196af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:26.483 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.482+0000 7f4a885b1700 1 --2- 172.21.15.77:0/3991157767 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a80100f30 0x7f4a80197030 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:26.483 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.482+0000 7f4a885b1700 1 -- 172.21.15.77:0/3991157767 >> 172.21.15.77:0/3991157767 conn(0x7f4a800fb2d0 msgr2=0x7f4a80069230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:26.483 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.482+0000 7f4a885b1700 1 -- 172.21.15.77:0/3991157767 shutdown_connections 2023-12-01T01:17:26.483 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:26.482+0000 7f4a885b1700 1 -- 172.21.15.77:0/3991157767 wait complete. 2023-12-01T01:17:26.807 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:26 smithi077 ceph-mon[98867]: osdmap e84: 12 total, 12 up, 12 in 2023-12-01T01:17:26.807 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:26 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:17:26.876 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- bash -c 'ceph orch host ls' 2023-12-01T01:17:27.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:26 smithi203 ceph-mon[103581]: osdmap e84: 12 total, 12 up, 12 in 2023-12-01T01:17:27.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:26 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:17:27.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:26 smithi137 ceph-mon[103312]: osdmap e84: 12 total, 12 up, 12 in 2023-12-01T01:17:27.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:26 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:17:27.432 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:17:27.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:27 smithi077 ceph-mon[98867]: pgmap v285: 97 pgs: 2 peering, 95 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 8 op/s 2023-12-01T01:17:27.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:27 smithi077 ceph-mon[98867]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:17:28.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:27 smithi203 ceph-mon[103581]: pgmap v285: 97 pgs: 2 peering, 95 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 8 op/s 2023-12-01T01:17:28.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:27 smithi203 ceph-mon[103581]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:17:28.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:27 smithi137 ceph-mon[103312]: pgmap v285: 97 pgs: 2 peering, 95 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 8 op/s 2023-12-01T01:17:28.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:27 smithi137 ceph-mon[103312]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:17:28.708 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.705+0000 7f985cc04700 1 -- 172.21.15.77:0/3516801741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98580ffd20 msgr2=0x7f9858100140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:28.708 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.705+0000 7f985cc04700 1 --2- 172.21.15.77:0/3516801741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98580ffd20 0x7f9858100140 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f98480099c0 tx=0x7f9848009cd0 comp rx=0 tx=0).stop 2023-12-01T01:17:28.708 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.705+0000 7f985cc04700 1 -- 172.21.15.77:0/3516801741 shutdown_connections 2023-12-01T01:17:28.709 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.705+0000 7f985cc04700 1 --2- 172.21.15.77:0/3516801741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98581018c0 0x7f9858108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:28.709 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.705+0000 7f985cc04700 1 --2- 172.21.15.77:0/3516801741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98580ffd20 0x7f9858100140 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:28.709 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.705+0000 7f985cc04700 1 --2- 172.21.15.77:0/3516801741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9858100f20 0x7f9858101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:28.709 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.705+0000 7f985cc04700 1 -- 172.21.15.77:0/3516801741 >> 172.21.15.77:0/3516801741 conn(0x7f98580fb2a0 msgr2=0x7f98580fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:28.709 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.705+0000 7f985cc04700 1 -- 172.21.15.77:0/3516801741 shutdown_connections 2023-12-01T01:17:28.709 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.705+0000 7f985cc04700 1 -- 172.21.15.77:0/3516801741 wait complete. 2023-12-01T01:17:28.709 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.706+0000 7f985cc04700 1 Processor -- start 2023-12-01T01:17:28.709 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.706+0000 7f985cc04700 1 -- start start 2023-12-01T01:17:28.709 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.706+0000 7f985cc04700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98580ffd20 0x7f98580693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:28.709 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.706+0000 7f985cc04700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9858100f20 0x7f985819dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:28.709 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.706+0000 7f985cc04700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98581018c0 0x7f98581a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:28.709 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.706+0000 7f985cc04700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98581a0890 con 0x7f98580ffd20 2023-12-01T01:17:28.709 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.706+0000 7f985cc04700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98581a0a00 con 0x7f9858100f20 2023-12-01T01:17:28.710 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.706+0000 7f985cc04700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98581a0bc0 con 0x7f98581018c0 2023-12-01T01:17:28.710 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.706+0000 7f985659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98580ffd20 0x7f98580693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:28.710 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.706+0000 7f985659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98580ffd20 0x7f98580693b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58558/0 (socket says 172.21.15.77:58558) 2023-12-01T01:17:28.710 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.706+0000 7f9855d9b700 1 --2- 172.21.15.77:0/2616574476 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9858100f20 0x7f985819dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:28.710 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.706+0000 7f985659c700 1 -- 172.21.15.77:0/2616574476 learned_addr learned my addr 172.21.15.77:0/2616574476 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:28.710 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.706+0000 7f9856d9d700 1 --2- 172.21.15.77:0/2616574476 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98581018c0 0x7f98581a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:28.710 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.707+0000 7f985659c700 1 -- 172.21.15.77:0/2616574476 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98581018c0 msgr2=0x7f98581a01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:28.710 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.707+0000 7f985659c700 1 --2- 172.21.15.77:0/2616574476 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98581018c0 0x7f98581a01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:28.710 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.707+0000 7f985659c700 1 -- 172.21.15.77:0/2616574476 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9858100f20 msgr2=0x7f985819dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:28.710 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.707+0000 7f985659c700 1 --2- 172.21.15.77:0/2616574476 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9858100f20 0x7f985819dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:28.710 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.707+0000 7f985659c700 1 -- 172.21.15.77:0/2616574476 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98480095f0 con 0x7f98580ffd20 2023-12-01T01:17:28.710 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.707+0000 7f9855d9b700 1 --2- 172.21.15.77:0/2616574476 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9858100f20 0x7f985819dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:17:28.710 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.707+0000 7f9856d9d700 1 --2- 172.21.15.77:0/2616574476 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98581018c0 0x7f98581a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:17:28.710 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.707+0000 7f985659c700 1 --2- 172.21.15.77:0/2616574476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98580ffd20 0x7f98580693b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9848009990 tx=0x7f984800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:28.711 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.707+0000 7f98477fe700 1 -- 172.21.15.77:0/2616574476 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984801d070 con 0x7f98580ffd20 2023-12-01T01:17:28.711 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.707+0000 7f985cc04700 1 -- 172.21.15.77:0/2616574476 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98581a0e40 con 0x7f98580ffd20 2023-12-01T01:17:28.711 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.707+0000 7f98477fe700 1 -- 172.21.15.77:0/2616574476 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9848005030 con 0x7f98580ffd20 2023-12-01T01:17:28.711 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.707+0000 7f98477fe700 1 -- 172.21.15.77:0/2616574476 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9848004a70 con 0x7f98580ffd20 2023-12-01T01:17:28.712 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.707+0000 7f985cc04700 1 -- 172.21.15.77:0/2616574476 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98581a11f0 con 0x7f98580ffd20 2023-12-01T01:17:28.712 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.708+0000 7f985cc04700 1 -- 172.21.15.77:0/2616574476 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9858107310 con 0x7f98580ffd20 2023-12-01T01:17:28.712 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.709+0000 7f98477fe700 1 -- 172.21.15.77:0/2616574476 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f9848005410 con 0x7f98580ffd20 2023-12-01T01:17:28.712 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.709+0000 7f98477fe700 1 --2- 172.21.15.77:0/2616574476 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f98400a05b0 0x7f98400a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:28.712 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.710+0000 7f9855d9b700 1 --2- 172.21.15.77:0/2616574476 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f98400a05b0 0x7f98400a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:28.712 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.710+0000 7f98477fe700 1 -- 172.21.15.77:0/2616574476 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f98480cc750 con 0x7f98580ffd20 2023-12-01T01:17:28.712 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.710+0000 7f9855d9b700 1 --2- 172.21.15.77:0/2616574476 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f98400a05b0 0x7f98400a2a70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9858069930 tx=0x7f984c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:28.715 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.714+0000 7f98477fe700 1 -- 172.21.15.77:0/2616574476 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9848096320 con 0x7f98580ffd20 2023-12-01T01:17:28.918 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.917+0000 7f985cc04700 1 -- 172.21.15.77:0/2616574476 --> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f98580611d0 con 0x7f98400a05b0 2023-12-01T01:17:28.920 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.919+0000 7f98477fe700 1 -- 172.21.15.77:0/2616574476 <== mgr.14152 v2:172.21.15.77:6800/3838941679 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f98580611d0 con 0x7f98400a05b0 2023-12-01T01:17:28.920 INFO:teuthology.orchestra.run.smithi077.stdout:HOST ADDR LABELS STATUS 2023-12-01T01:17:28.920 INFO:teuthology.orchestra.run.smithi077.stdout:smithi077 172.21.15.77 2023-12-01T01:17:28.920 INFO:teuthology.orchestra.run.smithi077.stdout:smithi137 172.21.15.137 2023-12-01T01:17:28.920 INFO:teuthology.orchestra.run.smithi077.stdout:smithi203 172.21.15.203 2023-12-01T01:17:28.920 INFO:teuthology.orchestra.run.smithi077.stdout:3 hosts in cluster 2023-12-01T01:17:28.923 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.922+0000 7f985cc04700 1 -- 172.21.15.77:0/2616574476 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f98400a05b0 msgr2=0x7f98400a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:28.923 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.922+0000 7f985cc04700 1 --2- 172.21.15.77:0/2616574476 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f98400a05b0 0x7f98400a2a70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9858069930 tx=0x7f984c009260 comp rx=0 tx=0).stop 2023-12-01T01:17:28.923 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.922+0000 7f985cc04700 1 -- 172.21.15.77:0/2616574476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98580ffd20 msgr2=0x7f98580693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:28.923 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.922+0000 7f985cc04700 1 --2- 172.21.15.77:0/2616574476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98580ffd20 0x7f98580693b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9848009990 tx=0x7f984800f690 comp rx=0 tx=0).stop 2023-12-01T01:17:28.923 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.922+0000 7f985cc04700 1 -- 172.21.15.77:0/2616574476 shutdown_connections 2023-12-01T01:17:28.923 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.922+0000 7f985cc04700 1 --2- 172.21.15.77:0/2616574476 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f98400a05b0 0x7f98400a2a70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:28.923 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.922+0000 7f985cc04700 1 --2- 172.21.15.77:0/2616574476 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98581018c0 0x7f98581a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:28.923 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.922+0000 7f985cc04700 1 --2- 172.21.15.77:0/2616574476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98580ffd20 0x7f98580693b0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:28.923 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.922+0000 7f985cc04700 1 --2- 172.21.15.77:0/2616574476 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9858100f20 0x7f985819dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:28.924 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.922+0000 7f985cc04700 1 -- 172.21.15.77:0/2616574476 >> 172.21.15.77:0/2616574476 conn(0x7f98580fb2a0 msgr2=0x7f9858104450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:28.924 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.922+0000 7f985cc04700 1 -- 172.21.15.77:0/2616574476 shutdown_connections 2023-12-01T01:17:28.924 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:28.922+0000 7f985cc04700 1 -- 172.21.15.77:0/2616574476 wait complete. 2023-12-01T01:17:29.275 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- bash -c 'ceph orch device ls' 2023-12-01T01:17:29.837 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:17:29.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:29 smithi077 ceph-mon[98867]: pgmap v286: 97 pgs: 2 peering, 95 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 9 op/s 2023-12-01T01:17:29.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:29 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:17:29.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:29 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:17:29.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:29 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:17:29.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:29 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:17:30.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:29 smithi203 ceph-mon[103581]: pgmap v286: 97 pgs: 2 peering, 95 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 9 op/s 2023-12-01T01:17:30.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:29 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:17:30.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:29 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:17:30.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:29 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:17:30.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:29 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:17:30.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:29 smithi137 ceph-mon[103312]: pgmap v286: 97 pgs: 2 peering, 95 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 9 op/s 2023-12-01T01:17:30.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:29 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:17:30.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:29 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:17:30.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:29 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:17:30.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:29 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:17:30.838 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:30 smithi077 ceph-mon[98867]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:17:30.838 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:30 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:17:30.838 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:30 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:17:30.838 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:30 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:17:30.838 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:30 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:17:30.838 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:30 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:17:31.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:30 smithi203 ceph-mon[103581]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:17:31.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:30 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:17:31.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:30 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:17:31.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:30 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:17:31.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:30 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:17:31.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:30 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:17:31.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:30 smithi137 ceph-mon[103312]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:17:31.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:30 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:17:31.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:30 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:17:31.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:30 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:17:31.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:30 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:17:31.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:30 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:17:31.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.029+0000 7f03f0da6700 1 -- 172.21.15.77:0/3100534810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03ec100f20 msgr2=0x7f03ec101380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:31.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.029+0000 7f03f0da6700 1 --2- 172.21.15.77:0/3100534810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03ec100f20 0x7f03ec101380 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f03dc0099c0 tx=0x7f03dc009cd0 comp rx=0 tx=0).stop 2023-12-01T01:17:31.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.029+0000 7f03f0da6700 1 -- 172.21.15.77:0/3100534810 shutdown_connections 2023-12-01T01:17:31.031 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.029+0000 7f03f0da6700 1 --2- 172.21.15.77:0/3100534810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03ec1018c0 0x7f03ec108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:31.031 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.029+0000 7f03f0da6700 1 --2- 172.21.15.77:0/3100534810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03ec0ffd20 0x7f03ec100140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:31.031 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.029+0000 7f03f0da6700 1 --2- 172.21.15.77:0/3100534810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03ec100f20 0x7f03ec101380 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:31.031 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.029+0000 7f03f0da6700 1 -- 172.21.15.77:0/3100534810 >> 172.21.15.77:0/3100534810 conn(0x7f03ec0fb2a0 msgr2=0x7f03ec0fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:31.031 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.029+0000 7f03f0da6700 1 -- 172.21.15.77:0/3100534810 shutdown_connections 2023-12-01T01:17:31.031 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.029+0000 7f03f0da6700 1 -- 172.21.15.77:0/3100534810 wait complete. 2023-12-01T01:17:31.033 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.030+0000 7f03f0da6700 1 Processor -- start 2023-12-01T01:17:31.033 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.030+0000 7f03f0da6700 1 -- start start 2023-12-01T01:17:31.033 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.030+0000 7f03f0da6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03ec0ffd20 0x7f03ec0693d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:31.033 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.030+0000 7f03f0da6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03ec100f20 0x7f03ec19b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:31.033 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.030+0000 7f03f0da6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03ec1018c0 0x7f03ec19df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:31.033 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.030+0000 7f03f0da6700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec19e6b0 con 0x7f03ec100f20 2023-12-01T01:17:31.033 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.030+0000 7f03f0da6700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec19e820 con 0x7f03ec0ffd20 2023-12-01T01:17:31.033 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.030+0000 7f03f0da6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec19e9e0 con 0x7f03ec1018c0 2023-12-01T01:17:31.033 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.030+0000 7f03e9d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03ec100f20 0x7f03ec19b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:31.033 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.030+0000 7f03e9d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03ec100f20 0x7f03ec19b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58572/0 (socket says 172.21.15.77:58572) 2023-12-01T01:17:31.034 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.030+0000 7f03e9d9b700 1 -- 172.21.15.77:0/1664960364 learned_addr learned my addr 172.21.15.77:0/1664960364 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:31.034 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.031+0000 7f03ead9d700 1 --2- 172.21.15.77:0/1664960364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03ec1018c0 0x7f03ec19df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:31.034 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.031+0000 7f03ea59c700 1 --2- 172.21.15.77:0/1664960364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03ec0ffd20 0x7f03ec0693d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:31.034 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.031+0000 7f03e9d9b700 1 -- 172.21.15.77:0/1664960364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03ec1018c0 msgr2=0x7f03ec19df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:31.034 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.031+0000 7f03e9d9b700 1 --2- 172.21.15.77:0/1664960364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03ec1018c0 0x7f03ec19df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:31.034 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.031+0000 7f03e9d9b700 1 -- 172.21.15.77:0/1664960364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03ec0ffd20 msgr2=0x7f03ec0693d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:31.034 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.031+0000 7f03e9d9b700 1 --2- 172.21.15.77:0/1664960364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03ec0ffd20 0x7f03ec0693d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:31.034 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.031+0000 7f03e9d9b700 1 -- 172.21.15.77:0/1664960364 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03dc0095f0 con 0x7f03ec100f20 2023-12-01T01:17:31.034 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.031+0000 7f03ea59c700 1 --2- 172.21.15.77:0/1664960364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03ec0ffd20 0x7f03ec0693d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:17:31.034 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.031+0000 7f03e9d9b700 1 --2- 172.21.15.77:0/1664960364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03ec100f20 0x7f03ec19b8c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f03dc009990 tx=0x7f03dc003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:31.034 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.031+0000 7f03e37fe700 1 -- 172.21.15.77:0/1664960364 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03dc01d070 con 0x7f03ec100f20 2023-12-01T01:17:31.034 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.031+0000 7f03e37fe700 1 -- 172.21.15.77:0/1664960364 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03dc004640 con 0x7f03ec100f20 2023-12-01T01:17:31.034 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.031+0000 7f03f0da6700 1 -- 172.21.15.77:0/1664960364 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03ec19ec60 con 0x7f03ec100f20 2023-12-01T01:17:31.034 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.031+0000 7f03e37fe700 1 -- 172.21.15.77:0/1664960364 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03dc00f460 con 0x7f03ec100f20 2023-12-01T01:17:31.035 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.031+0000 7f03f0da6700 1 -- 172.21.15.77:0/1664960364 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03ec19f100 con 0x7f03ec100f20 2023-12-01T01:17:31.035 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.032+0000 7f03f0da6700 1 -- 172.21.15.77:0/1664960364 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03ec10b5c0 con 0x7f03ec100f20 2023-12-01T01:17:31.035 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.033+0000 7f03e37fe700 1 -- 172.21.15.77:0/1664960364 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f03dc022470 con 0x7f03ec100f20 2023-12-01T01:17:31.035 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.034+0000 7f03e37fe700 1 --2- 172.21.15.77:0/1664960364 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f03d80a05b0 0x7f03d80a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:31.035 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.034+0000 7f03ea59c700 1 --2- 172.21.15.77:0/1664960364 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f03d80a05b0 0x7f03d80a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:31.035 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.034+0000 7f03e37fe700 1 -- 172.21.15.77:0/1664960364 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f03dc0cd770 con 0x7f03ec100f20 2023-12-01T01:17:31.036 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.034+0000 7f03ea59c700 1 --2- 172.21.15.77:0/1664960364 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f03d80a05b0 0x7f03d80a2a70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f03ec100d80 tx=0x7f03d4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:31.038 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.037+0000 7f03e37fe700 1 -- 172.21.15.77:0/1664960364 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f03dc097340 con 0x7f03ec100f20 2023-12-01T01:17:31.236 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.235+0000 7f03f0da6700 1 -- 172.21.15.77:0/1664960364 --> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f03ec0611d0 con 0x7f03d80a05b0 2023-12-01T01:17:31.237 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.236+0000 7f03e37fe700 1 -- 172.21.15.77:0/1664960364 <== mgr.14152 v2:172.21.15.77:6800/3838941679 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f03ec0611d0 con 0x7f03d80a05b0 2023-12-01T01:17:31.238 INFO:teuthology.orchestra.run.smithi077.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-12-01T01:17:31.238 INFO:teuthology.orchestra.run.smithi077.stdout:smithi077 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400ZJ400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-12-01T01:17:31.238 INFO:teuthology.orchestra.run.smithi077.stdout:smithi137 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233004A400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-12-01T01:17:31.238 INFO:teuthology.orchestra.run.smithi077.stdout:smithi203 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GK400BGN 372G No 107s ago Has a FileSystem, LVM detected 2023-12-01T01:17:31.240 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.238+0000 7f03f0da6700 1 -- 172.21.15.77:0/1664960364 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f03d80a05b0 msgr2=0x7f03d80a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:31.240 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.238+0000 7f03f0da6700 1 --2- 172.21.15.77:0/1664960364 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f03d80a05b0 0x7f03d80a2a70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f03ec100d80 tx=0x7f03d4008040 comp rx=0 tx=0).stop 2023-12-01T01:17:31.240 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.238+0000 7f03f0da6700 1 -- 172.21.15.77:0/1664960364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03ec100f20 msgr2=0x7f03ec19b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:31.240 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.238+0000 7f03f0da6700 1 --2- 172.21.15.77:0/1664960364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03ec100f20 0x7f03ec19b8c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f03dc009990 tx=0x7f03dc003ba0 comp rx=0 tx=0).stop 2023-12-01T01:17:31.240 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.238+0000 7f03f0da6700 1 -- 172.21.15.77:0/1664960364 shutdown_connections 2023-12-01T01:17:31.240 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.238+0000 7f03f0da6700 1 --2- 172.21.15.77:0/1664960364 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f03d80a05b0 0x7f03d80a2a70 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:31.240 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.238+0000 7f03f0da6700 1 --2- 172.21.15.77:0/1664960364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03ec1018c0 0x7f03ec19df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:31.240 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.238+0000 7f03f0da6700 1 --2- 172.21.15.77:0/1664960364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03ec0ffd20 0x7f03ec0693d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:31.240 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.238+0000 7f03f0da6700 1 --2- 172.21.15.77:0/1664960364 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03ec100f20 0x7f03ec19b8c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:31.240 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.238+0000 7f03f0da6700 1 -- 172.21.15.77:0/1664960364 >> 172.21.15.77:0/1664960364 conn(0x7f03ec0fb2a0 msgr2=0x7f03ec0fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:31.240 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.238+0000 7f03f0da6700 1 -- 172.21.15.77:0/1664960364 shutdown_connections 2023-12-01T01:17:31.240 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:31.238+0000 7f03f0da6700 1 -- 172.21.15.77:0/1664960364 wait complete. 2023-12-01T01:17:31.586 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-12-01T01:17:31.596 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi077.front.sepia.ceph.com 2023-12-01T01:17:31.596 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- bash -c 'ceph fs dump' 2023-12-01T01:17:31.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:31 smithi077 ceph-mon[98867]: pgmap v287: 97 pgs: 2 peering, 95 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2023-12-01T01:17:32.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:31 smithi203 ceph-mon[103581]: pgmap v287: 97 pgs: 2 peering, 95 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2023-12-01T01:17:32.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:31 smithi137 ceph-mon[103312]: pgmap v287: 97 pgs: 2 peering, 95 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2023-12-01T01:17:32.154 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:17:32.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:32 smithi077 ceph-mon[98867]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:17:33.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:32 smithi203 ceph-mon[103581]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:17:33.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:32 smithi137 ceph-mon[103312]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-01T01:17:33.342 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.341+0000 7f826f4c8700 1 -- 172.21.15.77:0/2563984033 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8268100e50 msgr2=0x7f8268101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:33.343 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.341+0000 7f826f4c8700 1 --2- 172.21.15.77:0/2563984033 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8268100e50 0x7f8268101270 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8264009990 tx=0x7f8264009ca0 comp rx=0 tx=0).stop 2023-12-01T01:17:33.343 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.341+0000 7f826f4c8700 1 -- 172.21.15.77:0/2563984033 shutdown_connections 2023-12-01T01:17:33.343 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.341+0000 7f826f4c8700 1 --2- 172.21.15.77:0/2563984033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8268102970 0x7f8268109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:33.343 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.341+0000 7f826f4c8700 1 --2- 172.21.15.77:0/2563984033 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8268100e50 0x7f8268101270 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:33.343 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.341+0000 7f826f4c8700 1 --2- 172.21.15.77:0/2563984033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8268101fb0 0x7f8268102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:33.343 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.341+0000 7f826f4c8700 1 -- 172.21.15.77:0/2563984033 >> 172.21.15.77:0/2563984033 conn(0x7f82680fc3b0 msgr2=0x7f82680fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:33.343 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.341+0000 7f826f4c8700 1 -- 172.21.15.77:0/2563984033 shutdown_connections 2023-12-01T01:17:33.343 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.341+0000 7f826f4c8700 1 -- 172.21.15.77:0/2563984033 wait complete. 2023-12-01T01:17:33.345 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.341+0000 7f826f4c8700 1 Processor -- start 2023-12-01T01:17:33.346 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.341+0000 7f826f4c8700 1 -- start start 2023-12-01T01:17:33.346 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.341+0000 7f826f4c8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8268100e50 0x7f8268196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:33.346 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.341+0000 7f826f4c8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8268101fb0 0x7f8268196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:33.346 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.341+0000 7f826f4c8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8268102970 0x7f826819e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:33.346 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.341+0000 7f826f4c8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826819e630 con 0x7f8268102970 2023-12-01T01:17:33.346 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.341+0000 7f826f4c8700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826819e770 con 0x7f8268100e50 2023-12-01T01:17:33.346 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.341+0000 7f826f4c8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826819e900 con 0x7f8268101fb0 2023-12-01T01:17:33.346 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.342+0000 7f826da65700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8268102970 0x7f826819e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:33.346 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.342+0000 7f826da65700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8268102970 0x7f826819e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58586/0 (socket says 172.21.15.77:58586) 2023-12-01T01:17:33.346 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.342+0000 7f826da65700 1 -- 172.21.15.77:0/2198606382 learned_addr learned my addr 172.21.15.77:0/2198606382 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:33.346 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.342+0000 7f826ca63700 1 --2- 172.21.15.77:0/2198606382 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8268101fb0 0x7f8268196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:33.346 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.342+0000 7f826d264700 1 --2- 172.21.15.77:0/2198606382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8268100e50 0x7f8268196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:33.347 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.342+0000 7f826da65700 1 -- 172.21.15.77:0/2198606382 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8268101fb0 msgr2=0x7f8268196fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:33.347 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.342+0000 7f826da65700 1 --2- 172.21.15.77:0/2198606382 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8268101fb0 0x7f8268196fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:33.347 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.342+0000 7f826da65700 1 -- 172.21.15.77:0/2198606382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8268100e50 msgr2=0x7f8268196aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:33.347 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.342+0000 7f826da65700 1 --2- 172.21.15.77:0/2198606382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8268100e50 0x7f8268196aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:33.347 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.342+0000 7f826da65700 1 -- 172.21.15.77:0/2198606382 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82640095f0 con 0x7f8268102970 2023-12-01T01:17:33.347 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.342+0000 7f826d264700 1 --2- 172.21.15.77:0/2198606382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8268100e50 0x7f8268196aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:17:33.347 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.342+0000 7f826da65700 1 --2- 172.21.15.77:0/2198606382 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8268102970 0x7f826819e060 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8260007da0 tx=0x7f826000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:33.347 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.342+0000 7f825e7fc700 1 -- 172.21.15.77:0/2198606382 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826000ad00 con 0x7f8268102970 2023-12-01T01:17:33.347 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.342+0000 7f825e7fc700 1 -- 172.21.15.77:0/2198606382 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f826000ae60 con 0x7f8268102970 2023-12-01T01:17:33.347 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.342+0000 7f826f4c8700 1 -- 172.21.15.77:0/2198606382 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f826819ebe0 con 0x7f8268102970 2023-12-01T01:17:33.347 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.342+0000 7f825e7fc700 1 -- 172.21.15.77:0/2198606382 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8260011660 con 0x7f8268102970 2023-12-01T01:17:33.348 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.342+0000 7f826f4c8700 1 -- 172.21.15.77:0/2198606382 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f826819f0e0 con 0x7f8268102970 2023-12-01T01:17:33.348 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.343+0000 7f826f4c8700 1 -- 172.21.15.77:0/2198606382 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f826810c680 con 0x7f8268102970 2023-12-01T01:17:33.348 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.344+0000 7f825e7fc700 1 -- 172.21.15.77:0/2198606382 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f8260004ad0 con 0x7f8268102970 2023-12-01T01:17:33.348 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.345+0000 7f825e7fc700 1 --2- 172.21.15.77:0/2198606382 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f82540a05b0 0x7f82540a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:33.349 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.345+0000 7f826d264700 1 --2- 172.21.15.77:0/2198606382 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f82540a05b0 0x7f82540a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:33.349 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.345+0000 7f825e7fc700 1 -- 172.21.15.77:0/2198606382 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f82600cbf30 con 0x7f8268102970 2023-12-01T01:17:33.349 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.345+0000 7f826d264700 1 --2- 172.21.15.77:0/2198606382 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f82540a05b0 0x7f82540a2a70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8264006010 tx=0x7f826400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:33.350 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.349+0000 7f825e7fc700 1 -- 172.21.15.77:0/2198606382 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8260095b80 con 0x7f8268102970 2023-12-01T01:17:33.579 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.578+0000 7f826f4c8700 1 -- 172.21.15.77:0/2198606382 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8268107d50 con 0x7f8268102970 2023-12-01T01:17:33.580 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.578+0000 7f825e7fc700 1 -- 172.21.15.77:0/2198606382 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) v1 ==== 76+0+2910 (secure 0 0 0) 0x7f8260016070 con 0x7f8268102970 2023-12-01T01:17:33.580 INFO:teuthology.orchestra.run.smithi077.stdout:e27 2023-12-01T01:17:33.580 INFO:teuthology.orchestra.run.smithi077.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-12-01T01:17:33.580 INFO:teuthology.orchestra.run.smithi077.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-12-01T01:17:33.580 INFO:teuthology.orchestra.run.smithi077.stdout:legacy client fscid: 1 2023-12-01T01:17:33.580 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:17:33.580 INFO:teuthology.orchestra.run.smithi077.stdout:Filesystem 'cephfs' (1) 2023-12-01T01:17:33.580 INFO:teuthology.orchestra.run.smithi077.stdout:fs_name cephfs 2023-12-01T01:17:33.580 INFO:teuthology.orchestra.run.smithi077.stdout:epoch 27 2023-12-01T01:17:33.580 INFO:teuthology.orchestra.run.smithi077.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-12-01T01:17:33.580 INFO:teuthology.orchestra.run.smithi077.stdout:created 2023-12-01T01:16:25.345276+0000 2023-12-01T01:17:33.581 INFO:teuthology.orchestra.run.smithi077.stdout:modified 2023-12-01T01:16:32.596617+0000 2023-12-01T01:17:33.581 INFO:teuthology.orchestra.run.smithi077.stdout:tableserver 0 2023-12-01T01:17:33.581 INFO:teuthology.orchestra.run.smithi077.stdout:root 0 2023-12-01T01:17:33.581 INFO:teuthology.orchestra.run.smithi077.stdout:session_timeout 300 2023-12-01T01:17:33.581 INFO:teuthology.orchestra.run.smithi077.stdout:session_autoclose 300 2023-12-01T01:17:33.581 INFO:teuthology.orchestra.run.smithi077.stdout:max_file_size 1099511627776 2023-12-01T01:17:33.581 INFO:teuthology.orchestra.run.smithi077.stdout:required_client_features {} 2023-12-01T01:17:33.581 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure 0 2023-12-01T01:17:33.581 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure_osd_epoch 0 2023-12-01T01:17:33.581 INFO:teuthology.orchestra.run.smithi077.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-12-01T01:17:33.581 INFO:teuthology.orchestra.run.smithi077.stdout:max_mds 5 2023-12-01T01:17:33.581 INFO:teuthology.orchestra.run.smithi077.stdout:in 0,1,2,3,4 2023-12-01T01:17:33.581 INFO:teuthology.orchestra.run.smithi077.stdout:up {0=24500,1=24496,2=24475,3=24473,4=14649} 2023-12-01T01:17:33.582 INFO:teuthology.orchestra.run.smithi077.stdout:failed 2023-12-01T01:17:33.582 INFO:teuthology.orchestra.run.smithi077.stdout:damaged 2023-12-01T01:17:33.582 INFO:teuthology.orchestra.run.smithi077.stdout:stopped 2023-12-01T01:17:33.582 INFO:teuthology.orchestra.run.smithi077.stdout:data_pools [3] 2023-12-01T01:17:33.582 INFO:teuthology.orchestra.run.smithi077.stdout:metadata_pool 2 2023-12-01T01:17:33.582 INFO:teuthology.orchestra.run.smithi077.stdout:inline_data disabled 2023-12-01T01:17:33.582 INFO:teuthology.orchestra.run.smithi077.stdout:balancer 2023-12-01T01:17:33.582 INFO:teuthology.orchestra.run.smithi077.stdout:bal_rank_mask -1 2023-12-01T01:17:33.582 INFO:teuthology.orchestra.run.smithi077.stdout:standby_count_wanted 0 2023-12-01T01:17:33.582 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.b{0:24500} state up:active seq 3 addr [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] compat {c=[1],r=[1],i=[7ff]}] 2023-12-01T01:17:33.582 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.f{0:24463} state up:standby-replay seq 1 addr [v2:172.21.15.203:6834/3237538176,v1:172.21.15.203:6835/3237538176] compat {c=[1],r=[1],i=[7ff]}] 2023-12-01T01:17:33.582 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.i{1:24496} state up:active seq 5 addr [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] compat {c=[1],r=[1],i=[7ff]}] 2023-12-01T01:17:33.582 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.e{1:24449} state up:standby-replay seq 1 addr [v2:172.21.15.137:6834/3935354878,v1:172.21.15.137:6835/3935354878] compat {c=[1],r=[1],i=[7ff]}] 2023-12-01T01:17:33.582 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.c{2:24475} state up:active seq 7 addr [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] compat {c=[1],r=[1],i=[7ff]}] 2023-12-01T01:17:33.582 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.l{2:24430} state up:standby-replay seq 1 addr [v2:172.21.15.203:6832/1201599774,v1:172.21.15.203:6833/1201599774] compat {c=[1],r=[1],i=[7ff]}] 2023-12-01T01:17:33.583 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.h{3:24473} state up:active seq 6 addr [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] compat {c=[1],r=[1],i=[7ff]}] 2023-12-01T01:17:33.583 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.k{3:24428} state up:standby-replay seq 1 addr [v2:172.21.15.137:6832/2249638445,v1:172.21.15.137:6833/2249638445] compat {c=[1],r=[1],i=[7ff]}] 2023-12-01T01:17:33.583 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.a{4:14649} state up:active seq 5 addr [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] compat {c=[1],r=[1],i=[7ff]}] 2023-12-01T01:17:33.583 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.g{4:14637} state up:standby-replay seq 1 addr [v2:172.21.15.77:6838/3726913446,v1:172.21.15.77:6839/3726913446] compat {c=[1],r=[1],i=[7ff]}] 2023-12-01T01:17:33.583 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:17:33.583 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:17:33.583 INFO:teuthology.orchestra.run.smithi077.stdout:Standby daemons: 2023-12-01T01:17:33.583 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:17:33.583 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.j{-1:14574} state up:standby seq 1 addr [v2:172.21.15.77:6834/1435145620,v1:172.21.15.77:6835/1435145620] compat {c=[1],r=[1],i=[7ff]}] 2023-12-01T01:17:33.583 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.d{-1:14580} state up:standby seq 1 addr [v2:172.21.15.77:6836/841838908,v1:172.21.15.77:6837/841838908] compat {c=[1],r=[1],i=[7ff]}] 2023-12-01T01:17:33.584 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.580+0000 7f826f4c8700 1 -- 172.21.15.77:0/2198606382 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f82540a05b0 msgr2=0x7f82540a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:33.584 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.580+0000 7f826f4c8700 1 --2- 172.21.15.77:0/2198606382 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f82540a05b0 0x7f82540a2a70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8264006010 tx=0x7f826400b540 comp rx=0 tx=0).stop 2023-12-01T01:17:33.584 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.580+0000 7f826f4c8700 1 -- 172.21.15.77:0/2198606382 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8268102970 msgr2=0x7f826819e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:33.584 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.580+0000 7f826f4c8700 1 --2- 172.21.15.77:0/2198606382 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8268102970 0x7f826819e060 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8260007da0 tx=0x7f826000a340 comp rx=0 tx=0).stop 2023-12-01T01:17:33.584 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.580+0000 7f826f4c8700 1 -- 172.21.15.77:0/2198606382 shutdown_connections 2023-12-01T01:17:33.584 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.580+0000 7f826f4c8700 1 --2- 172.21.15.77:0/2198606382 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f82540a05b0 0x7f82540a2a70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:33.584 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.580+0000 7f826f4c8700 1 --2- 172.21.15.77:0/2198606382 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8268102970 0x7f826819e060 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:33.584 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.580+0000 7f826f4c8700 1 --2- 172.21.15.77:0/2198606382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8268100e50 0x7f8268196aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:33.585 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.580+0000 7f826f4c8700 1 --2- 172.21.15.77:0/2198606382 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8268101fb0 0x7f8268196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:33.585 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.580+0000 7f826f4c8700 1 -- 172.21.15.77:0/2198606382 >> 172.21.15.77:0/2198606382 conn(0x7f82680fc3b0 msgr2=0x7f82680fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:33.585 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.580+0000 7f826f4c8700 1 -- 172.21.15.77:0/2198606382 shutdown_connections 2023-12-01T01:17:33.585 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:33.580+0000 7f826f4c8700 1 -- 172.21.15.77:0/2198606382 wait complete. 2023-12-01T01:17:33.585 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 27 2023-12-01T01:17:33.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:33 smithi077 ceph-mon[98867]: pgmap v288: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 22 B/s, 1 keys/s, 1 objects/s recovering 2023-12-01T01:17:33.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:33 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/2198606382' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-12-01T01:17:34.010 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- bash -c 'ceph osd dump' 2023-12-01T01:17:34.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:33 smithi203 ceph-mon[103581]: pgmap v288: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 22 B/s, 1 keys/s, 1 objects/s recovering 2023-12-01T01:17:34.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:33 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/2198606382' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-12-01T01:17:34.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:33 smithi137 ceph-mon[103312]: pgmap v288: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 22 B/s, 1 keys/s, 1 objects/s recovering 2023-12-01T01:17:34.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:33 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/2198606382' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-12-01T01:17:34.562 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:17:35.756 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.752+0000 7fa56789a700 1 -- 172.21.15.77:0/1201953946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5601013a0 msgr2=0x7fa5601017c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:35.757 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.752+0000 7fa56789a700 1 --2- 172.21.15.77:0/1201953946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5601013a0 0x7fa5601017c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa550009990 tx=0x7fa550009ca0 comp rx=0 tx=0).stop 2023-12-01T01:17:35.757 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.752+0000 7fa56789a700 1 -- 172.21.15.77:0/1201953946 shutdown_connections 2023-12-01T01:17:35.757 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.752+0000 7fa56789a700 1 --2- 172.21.15.77:0/1201953946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa560100930 0x7fa5601081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:35.757 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.752+0000 7fa56789a700 1 --2- 172.21.15.77:0/1201953946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5601013a0 0x7fa5601017c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:35.757 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.752+0000 7fa56789a700 1 --2- 172.21.15.77:0/1201953946 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5600fff70 0x7fa5601003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:35.757 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.752+0000 7fa56789a700 1 -- 172.21.15.77:0/1201953946 >> 172.21.15.77:0/1201953946 conn(0x7fa5600faa40 msgr2=0x7fa5600fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:35.757 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.753+0000 7fa56789a700 1 -- 172.21.15.77:0/1201953946 shutdown_connections 2023-12-01T01:17:35.757 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.753+0000 7fa56789a700 1 -- 172.21.15.77:0/1201953946 wait complete. 2023-12-01T01:17:35.757 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.753+0000 7fa56789a700 1 Processor -- start 2023-12-01T01:17:35.758 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.753+0000 7fa56789a700 1 -- start start 2023-12-01T01:17:35.758 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.754+0000 7fa56789a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5600fff70 0x7fa5601959e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:35.758 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.754+0000 7fa56789a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa560100930 0x7fa560195f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:35.758 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.754+0000 7fa56789a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5601013a0 0x7fa56019cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:35.758 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.754+0000 7fa56789a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56019d570 con 0x7fa560100930 2023-12-01T01:17:35.758 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.754+0000 7fa56789a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56019d6b0 con 0x7fa5600fff70 2023-12-01T01:17:35.758 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.754+0000 7fa56789a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56019d840 con 0x7fa5601013a0 2023-12-01T01:17:35.758 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.754+0000 7fa564e35700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa560100930 0x7fa560195f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:35.758 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.754+0000 7fa564e35700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa560100930 0x7fa560195f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58598/0 (socket says 172.21.15.77:58598) 2023-12-01T01:17:35.758 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.754+0000 7fa564e35700 1 -- 172.21.15.77:0/685612126 learned_addr learned my addr 172.21.15.77:0/685612126 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:35.759 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.754+0000 7fa565636700 1 --2- 172.21.15.77:0/685612126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5600fff70 0x7fa5601959e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:35.759 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.754+0000 7fa565e37700 1 --2- 172.21.15.77:0/685612126 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5601013a0 0x7fa56019cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:35.759 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.754+0000 7fa564e35700 1 -- 172.21.15.77:0/685612126 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5601013a0 msgr2=0x7fa56019cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:35.759 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.754+0000 7fa564e35700 1 --2- 172.21.15.77:0/685612126 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5601013a0 0x7fa56019cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:35.759 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.754+0000 7fa564e35700 1 -- 172.21.15.77:0/685612126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5600fff70 msgr2=0x7fa5601959e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:35.759 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.754+0000 7fa564e35700 1 --2- 172.21.15.77:0/685612126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5600fff70 0x7fa5601959e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:35.760 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.754+0000 7fa564e35700 1 -- 172.21.15.77:0/685612126 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5500095f0 con 0x7fa560100930 2023-12-01T01:17:35.760 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.754+0000 7fa565e37700 1 --2- 172.21.15.77:0/685612126 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5601013a0 0x7fa56019cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:17:35.760 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.754+0000 7fa564e35700 1 --2- 172.21.15.77:0/685612126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa560100930 0x7fa560195f20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa55c00d980 tx=0x7fa55c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:35.760 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.754+0000 7fa5567fc700 1 -- 172.21.15.77:0/685612126 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c009b40 con 0x7fa560100930 2023-12-01T01:17:35.760 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.754+0000 7fa5567fc700 1 -- 172.21.15.77:0/685612126 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa55c00b6e0 con 0x7fa560100930 2023-12-01T01:17:35.760 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.754+0000 7fa5567fc700 1 -- 172.21.15.77:0/685612126 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c00f620 con 0x7fa560100930 2023-12-01T01:17:35.760 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.754+0000 7fa56789a700 1 -- 172.21.15.77:0/685612126 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa56019db20 con 0x7fa560100930 2023-12-01T01:17:35.760 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.755+0000 7fa56789a700 1 -- 172.21.15.77:0/685612126 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa56019dfd0 con 0x7fa560100930 2023-12-01T01:17:35.760 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.755+0000 7fa56789a700 1 -- 172.21.15.77:0/685612126 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa56010b520 con 0x7fa560100930 2023-12-01T01:17:35.761 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.757+0000 7fa5567fc700 1 -- 172.21.15.77:0/685612126 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fa55c010460 con 0x7fa560100930 2023-12-01T01:17:35.761 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.757+0000 7fa5567fc700 1 --2- 172.21.15.77:0/685612126 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa54c0a05b0 0x7fa54c0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:35.762 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.757+0000 7fa565636700 1 --2- 172.21.15.77:0/685612126 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa54c0a05b0 0x7fa54c0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:35.762 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.757+0000 7fa5567fc700 1 -- 172.21.15.77:0/685612126 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fa55c0cbc80 con 0x7fa560100930 2023-12-01T01:17:35.762 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.758+0000 7fa565636700 1 --2- 172.21.15.77:0/685612126 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa54c0a05b0 0x7fa54c0a2a70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa55000b5c0 tx=0x7fa550005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:35.762 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.761+0000 7fa5567fc700 1 -- 172.21.15.77:0/685612126 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa55c0957c0 con 0x7fa560100930 2023-12-01T01:17:35.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:35 smithi077 ceph-mon[98867]: pgmap v289: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s; 18 B/s, 1 keys/s, 0 objects/s recovering 2023-12-01T01:17:35.949 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.948+0000 7fa56789a700 1 -- 172.21.15.77:0/685612126 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7fa560106d00 con 0x7fa560100930 2023-12-01T01:17:35.950 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.949+0000 7fa5567fc700 1 -- 172.21.15.77:0/685612126 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v84) v1 ==== 56+0+5236 (secure 0 0 0) 0x7fa55c098de0 con 0x7fa560100930 2023-12-01T01:17:35.950 INFO:teuthology.orchestra.run.smithi077.stdout:epoch 84 2023-12-01T01:17:35.951 INFO:teuthology.orchestra.run.smithi077.stdout:fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 2023-12-01T01:17:35.951 INFO:teuthology.orchestra.run.smithi077.stdout:created 2023-12-01T01:09:30.210398+0000 2023-12-01T01:17:35.951 INFO:teuthology.orchestra.run.smithi077.stdout:modified 2023-12-01T01:17:25.609766+0000 2023-12-01T01:17:35.951 INFO:teuthology.orchestra.run.smithi077.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-12-01T01:17:35.951 INFO:teuthology.orchestra.run.smithi077.stdout:crush_version 26 2023-12-01T01:17:35.951 INFO:teuthology.orchestra.run.smithi077.stdout:full_ratio 0.95 2023-12-01T01:17:35.951 INFO:teuthology.orchestra.run.smithi077.stdout:backfillfull_ratio 0.9 2023-12-01T01:17:35.951 INFO:teuthology.orchestra.run.smithi077.stdout:nearfull_ratio 0.85 2023-12-01T01:17:35.951 INFO:teuthology.orchestra.run.smithi077.stdout:require_min_compat_client luminous 2023-12-01T01:17:35.951 INFO:teuthology.orchestra.run.smithi077.stdout:min_compat_client luminous 2023-12-01T01:17:35.951 INFO:teuthology.orchestra.run.smithi077.stdout:require_osd_release reef 2023-12-01T01:17:35.951 INFO:teuthology.orchestra.run.smithi077.stdout:stretch_mode_enabled false 2023-12-01T01:17:35.951 INFO:teuthology.orchestra.run.smithi077.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-12-01T01:17:35.952 INFO:teuthology.orchestra.run.smithi077.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2023-12-01T01:17:35.952 INFO:teuthology.orchestra.run.smithi077.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2023-12-01T01:17:35.952 INFO:teuthology.orchestra.run.smithi077.stdout:max_osd 12 2023-12-01T01:17:35.952 INFO:teuthology.orchestra.run.smithi077.stdout:osd.0 up in weight 1 up_from 8 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6802/2244799094,v1:172.21.15.77:6803/2244799094] [v2:172.21.15.77:6804/2244799094,v1:172.21.15.77:6805/2244799094] exists,up fc651e62-a103-4ab8-a76d-d3708ea6aeec 2023-12-01T01:17:35.952 INFO:teuthology.orchestra.run.smithi077.stdout:osd.1 up in weight 1 up_from 13 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6800/224194890,v1:172.21.15.137:6801/224194890] [v2:172.21.15.137:6802/224194890,v1:172.21.15.137:6803/224194890] exists,up 0adf8c54-49ec-49a0-8bf1-af1f20e60729 2023-12-01T01:17:35.952 INFO:teuthology.orchestra.run.smithi077.stdout:osd.2 up in weight 1 up_from 18 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.203:6800/4078459110,v1:172.21.15.203:6801/4078459110] [v2:172.21.15.203:6802/4078459110,v1:172.21.15.203:6803/4078459110] exists,up 5a69802c-dede-4544-9a12-436f5c1c0038 2023-12-01T01:17:35.952 INFO:teuthology.orchestra.run.smithi077.stdout:osd.3 up in weight 1 up_from 25 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6810/1454440477,v1:172.21.15.77:6811/1454440477] [v2:172.21.15.77:6812/1454440477,v1:172.21.15.77:6813/1454440477] exists,up ad325a97-580b-4c99-bf12-cbb0d92a6d0f 2023-12-01T01:17:35.952 INFO:teuthology.orchestra.run.smithi077.stdout:osd.4 up in weight 1 up_from 32 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6808/1103241451,v1:172.21.15.137:6809/1103241451] [v2:172.21.15.137:6810/1103241451,v1:172.21.15.137:6811/1103241451] exists,up 1e3ac1ba-98bb-4b8b-8e26-50064c66bf2e 2023-12-01T01:17:35.952 INFO:teuthology.orchestra.run.smithi077.stdout:osd.5 up in weight 1 up_from 37 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.203:6808/2648569482,v1:172.21.15.203:6809/2648569482] [v2:172.21.15.203:6810/2648569482,v1:172.21.15.203:6811/2648569482] exists,up eb32c016-abeb-44ab-abf0-19dd09235736 2023-12-01T01:17:35.952 INFO:teuthology.orchestra.run.smithi077.stdout:osd.6 up in weight 1 up_from 45 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6818/2463414651,v1:172.21.15.77:6819/2463414651] [v2:172.21.15.77:6820/2463414651,v1:172.21.15.77:6821/2463414651] exists,up 0e3fa97e-2ca6-418c-9e8d-a6fe3c56adb3 2023-12-01T01:17:35.952 INFO:teuthology.orchestra.run.smithi077.stdout:osd.7 up in weight 1 up_from 50 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6816/3108330766,v1:172.21.15.137:6817/3108330766] [v2:172.21.15.137:6818/3108330766,v1:172.21.15.137:6819/3108330766] exists,up d7398c01-0f98-4138-8a49-c3ff0aae7e88 2023-12-01T01:17:35.952 INFO:teuthology.orchestra.run.smithi077.stdout:osd.8 up in weight 1 up_from 57 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.203:6816/1161158776,v1:172.21.15.203:6817/1161158776] [v2:172.21.15.203:6818/1161158776,v1:172.21.15.203:6819/1161158776] exists,up 0c09050d-3e43-410c-8cd7-bcb891af56b2 2023-12-01T01:17:35.952 INFO:teuthology.orchestra.run.smithi077.stdout:osd.9 up in weight 1 up_from 64 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6826/2409880685,v1:172.21.15.77:6827/2409880685] [v2:172.21.15.77:6828/2409880685,v1:172.21.15.77:6829/2409880685] exists,up 527ecb49-95ec-48bd-9872-3b07fae5c5ba 2023-12-01T01:17:35.952 INFO:teuthology.orchestra.run.smithi077.stdout:osd.10 up in weight 1 up_from 69 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6824/374920146,v1:172.21.15.137:6825/374920146] [v2:172.21.15.137:6826/374920146,v1:172.21.15.137:6827/374920146] exists,up 1680bb3e-a3c8-427b-af68-e2214f32bed3 2023-12-01T01:17:35.952 INFO:teuthology.orchestra.run.smithi077.stdout:osd.11 up in weight 1 up_from 77 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.203:6824/2161588962,v1:172.21.15.203:6825/2161588962] [v2:172.21.15.203:6826/2161588962,v1:172.21.15.203:6827/2161588962] exists,up 8a533466-023d-4803-ae36-9a9f1715e49f 2023-12-01T01:17:35.953 INFO:teuthology.orchestra.run.smithi077.stdout:pg_upmap_items 2.5 [8,11] 2023-12-01T01:17:35.953 INFO:teuthology.orchestra.run.smithi077.stdout:pg_upmap_items 2.13 [2,11] 2023-12-01T01:17:35.953 INFO:teuthology.orchestra.run.smithi077.stdout:pg_upmap_items 3.13 [4,9] 2023-12-01T01:17:35.953 INFO:teuthology.orchestra.run.smithi077.stdout:pg_upmap_items 3.28 [5,8] 2023-12-01T01:17:35.953 INFO:teuthology.orchestra.run.smithi077.stdout:pg_upmap_items 3.36 [4,9] 2023-12-01T01:17:35.953 INFO:teuthology.orchestra.run.smithi077.stdout:pg_upmap_items 3.3c [4,9] 2023-12-01T01:17:35.953 INFO:teuthology.orchestra.run.smithi077.stdout:blocklist 172.21.15.77:0/2260035254 expires 2023-12-02T01:10:24.191835+0000 2023-12-01T01:17:35.953 INFO:teuthology.orchestra.run.smithi077.stdout:blocklist 172.21.15.77:6800/1456621747 expires 2023-12-02T01:10:24.191835+0000 2023-12-01T01:17:35.953 INFO:teuthology.orchestra.run.smithi077.stdout:blocklist 172.21.15.77:0/219838828 expires 2023-12-02T01:10:24.191835+0000 2023-12-01T01:17:35.953 INFO:teuthology.orchestra.run.smithi077.stdout:blocklist 172.21.15.77:6801/1456621747 expires 2023-12-02T01:10:24.191835+0000 2023-12-01T01:17:35.953 INFO:teuthology.orchestra.run.smithi077.stdout:blocklist 172.21.15.77:0/2937149266 expires 2023-12-02T01:10:02.232948+0000 2023-12-01T01:17:35.953 INFO:teuthology.orchestra.run.smithi077.stdout:blocklist 172.21.15.77:0/807520018 expires 2023-12-02T01:10:02.232948+0000 2023-12-01T01:17:35.953 INFO:teuthology.orchestra.run.smithi077.stdout:blocklist 172.21.15.77:6801/1661788026 expires 2023-12-02T01:10:02.232948+0000 2023-12-01T01:17:35.953 INFO:teuthology.orchestra.run.smithi077.stdout:blocklist 172.21.15.77:6800/1661788026 expires 2023-12-02T01:10:02.232948+0000 2023-12-01T01:17:35.953 INFO:teuthology.orchestra.run.smithi077.stdout:blocklist 172.21.15.77:0/3551282473 expires 2023-12-02T01:10:24.191835+0000 2023-12-01T01:17:35.954 INFO:teuthology.orchestra.run.smithi077.stdout:blocklist 172.21.15.77:0/1202222558 expires 2023-12-02T01:10:02.232948+0000 2023-12-01T01:17:35.954 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.951+0000 7fa56789a700 1 -- 172.21.15.77:0/685612126 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa54c0a05b0 msgr2=0x7fa54c0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.951+0000 7fa56789a700 1 --2- 172.21.15.77:0/685612126 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa54c0a05b0 0x7fa54c0a2a70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa55000b5c0 tx=0x7fa550005fd0 comp rx=0 tx=0).stop 2023-12-01T01:17:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.951+0000 7fa56789a700 1 -- 172.21.15.77:0/685612126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa560100930 msgr2=0x7fa560195f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.951+0000 7fa56789a700 1 --2- 172.21.15.77:0/685612126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa560100930 0x7fa560195f20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa55c00d980 tx=0x7fa55c00dc90 comp rx=0 tx=0).stop 2023-12-01T01:17:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.951+0000 7fa56789a700 1 -- 172.21.15.77:0/685612126 shutdown_connections 2023-12-01T01:17:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.951+0000 7fa56789a700 1 --2- 172.21.15.77:0/685612126 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa54c0a05b0 0x7fa54c0a2a70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.951+0000 7fa56789a700 1 --2- 172.21.15.77:0/685612126 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5601013a0 0x7fa56019cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.951+0000 7fa56789a700 1 --2- 172.21.15.77:0/685612126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5600fff70 0x7fa5601959e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.951+0000 7fa56789a700 1 --2- 172.21.15.77:0/685612126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa560100930 0x7fa560195f20 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.951+0000 7fa56789a700 1 -- 172.21.15.77:0/685612126 >> 172.21.15.77:0/685612126 conn(0x7fa5600faa40 msgr2=0x7fa5600fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.951+0000 7fa56789a700 1 -- 172.21.15.77:0/685612126 shutdown_connections 2023-12-01T01:17:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:35.952+0000 7fa56789a700 1 -- 172.21.15.77:0/685612126 wait complete. 2023-12-01T01:17:36.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:35 smithi203 ceph-mon[103581]: pgmap v289: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s; 18 B/s, 1 keys/s, 0 objects/s recovering 2023-12-01T01:17:36.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:35 smithi137 ceph-mon[103312]: pgmap v289: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s; 18 B/s, 1 keys/s, 0 objects/s recovering 2023-12-01T01:17:36.302 INFO:teuthology.run_tasks:Running task fs.ready... 2023-12-01T01:17:36.315 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:17:36.446 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.440+0000 7f6bbfd3a700 1 -- 172.21.15.77:0/136989400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bb81001a0 msgr2=0x7f6bb81005c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:36.446 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.440+0000 7f6bbfd3a700 1 --2- 172.21.15.77:0/136989400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bb81001a0 0x7f6bb81005c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8009990 tx=0x7f6ba8009ca0 comp rx=0 tx=0).stop 2023-12-01T01:17:36.446 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.440+0000 7f6bbfd3a700 1 -- 172.21.15.77:0/136989400 shutdown_connections 2023-12-01T01:17:36.447 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.440+0000 7f6bbfd3a700 1 --2- 172.21.15.77:0/136989400 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bb8101870 0x7f6bb8108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:36.447 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.440+0000 7f6bbfd3a700 1 --2- 172.21.15.77:0/136989400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bb81001a0 0x7f6bb81005c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:36.447 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.440+0000 7f6bbfd3a700 1 --2- 172.21.15.77:0/136989400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bb8100eb0 0x7f6bb8101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:36.447 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.440+0000 7f6bbfd3a700 1 -- 172.21.15.77:0/136989400 >> 172.21.15.77:0/136989400 conn(0x7f6bb80fa950 msgr2=0x7f6bb80fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:36.447 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.440+0000 7f6bbfd3a700 1 -- 172.21.15.77:0/136989400 shutdown_connections 2023-12-01T01:17:36.447 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.440+0000 7f6bbfd3a700 1 -- 172.21.15.77:0/136989400 wait complete. 2023-12-01T01:17:36.447 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.441+0000 7f6bbfd3a700 1 Processor -- start 2023-12-01T01:17:36.448 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.441+0000 7f6bbfd3a700 1 -- start start 2023-12-01T01:17:36.448 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.441+0000 7f6bbfd3a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bb81001a0 0x7f6bb8103f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:36.448 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.441+0000 7f6bbfd3a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bb8100eb0 0x7f6bb81025a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:36.448 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.441+0000 7f6bbfd3a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bb8101870 0x7f6bb8102ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:36.448 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.441+0000 7f6bbfd3a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb8103280 con 0x7f6bb81001a0 2023-12-01T01:17:36.448 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.441+0000 7f6bbfd3a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb81033f0 con 0x7f6bb8101870 2023-12-01T01:17:36.448 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.441+0000 7f6bbfd3a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb81a0b80 con 0x7f6bb8100eb0 2023-12-01T01:17:36.448 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.441+0000 7f6bbdad6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bb81001a0 0x7f6bb8103f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:36.448 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.441+0000 7f6bbdad6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bb81001a0 0x7f6bb8103f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58608/0 (socket says 172.21.15.77:58608) 2023-12-01T01:17:36.449 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.441+0000 7f6bbdad6700 1 -- 172.21.15.77:0/4227748840 learned_addr learned my addr 172.21.15.77:0/4227748840 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:36.449 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.441+0000 7f6bbe2d7700 1 --2- 172.21.15.77:0/4227748840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bb8101870 0x7f6bb8102ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:36.449 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.441+0000 7f6bbd2d5700 1 --2- 172.21.15.77:0/4227748840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bb8100eb0 0x7f6bb81025a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:36.449 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.441+0000 7f6bbdad6700 1 -- 172.21.15.77:0/4227748840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bb8100eb0 msgr2=0x7f6bb81025a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:36.449 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.441+0000 7f6bbdad6700 1 --2- 172.21.15.77:0/4227748840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bb8100eb0 0x7f6bb81025a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:36.449 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.441+0000 7f6bbdad6700 1 -- 172.21.15.77:0/4227748840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bb8101870 msgr2=0x7f6bb8102ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:36.449 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.441+0000 7f6bbdad6700 1 --2- 172.21.15.77:0/4227748840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bb8101870 0x7f6bb8102ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:36.449 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.441+0000 7f6bbdad6700 1 -- 172.21.15.77:0/4227748840 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba80095f0 con 0x7f6bb81001a0 2023-12-01T01:17:36.449 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.441+0000 7f6bbdad6700 1 --2- 172.21.15.77:0/4227748840 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bb81001a0 0x7f6bb8103f60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8004730 tx=0x7f6ba8009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:36.449 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.441+0000 7f6baeffd700 1 -- 172.21.15.77:0/4227748840 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba801d070 con 0x7f6bb81001a0 2023-12-01T01:17:36.449 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.441+0000 7f6baeffd700 1 -- 172.21.15.77:0/4227748840 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ba8004cd0 con 0x7f6bb81001a0 2023-12-01T01:17:36.449 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.441+0000 7f6bbfd3a700 1 -- 172.21.15.77:0/4227748840 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bb81a0e00 con 0x7f6bb81001a0 2023-12-01T01:17:36.449 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.441+0000 7f6baeffd700 1 -- 172.21.15.77:0/4227748840 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8005290 con 0x7f6bb81001a0 2023-12-01T01:17:36.449 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.442+0000 7f6bbfd3a700 1 -- 172.21.15.77:0/4227748840 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bb81a11d0 con 0x7f6bb81001a0 2023-12-01T01:17:36.450 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.442+0000 7f6bbfd3a700 1 -- 172.21.15.77:0/4227748840 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bb8107da0 con 0x7f6bb81001a0 2023-12-01T01:17:36.450 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.443+0000 7f6baeffd700 1 -- 172.21.15.77:0/4227748840 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f6ba8003b90 con 0x7f6bb81001a0 2023-12-01T01:17:36.450 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.444+0000 7f6baeffd700 1 --2- 172.21.15.77:0/4227748840 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f6ba40a05f0 0x7f6ba40a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:36.450 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.444+0000 7f6bbd2d5700 1 --2- 172.21.15.77:0/4227748840 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f6ba40a05f0 0x7f6ba40a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:36.450 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.444+0000 7f6baeffd700 1 -- 172.21.15.77:0/4227748840 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f6ba80cce10 con 0x7f6bb81001a0 2023-12-01T01:17:36.450 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.444+0000 7f6bbd2d5700 1 --2- 172.21.15.77:0/4227748840 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f6ba40a05f0 0x7f6ba40a2ab0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4005950 tx=0x7f6bb40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:36.450 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.447+0000 7f6baeffd700 1 -- 172.21.15.77:0/4227748840 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6ba80969e0 con 0x7f6bb81001a0 2023-12-01T01:17:36.642 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.641+0000 7f6bbfd3a700 1 -- 172.21.15.77:0/4227748840 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6bb810b580 con 0x7f6bb81001a0 2023-12-01T01:17:36.644 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.643+0000 7f6baeffd700 1 -- 172.21.15.77:0/4227748840 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22472 (secure 0 0 0) 0x7f6ba8026090 con 0x7f6bb81001a0 2023-12-01T01:17:36.644 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:17:36.644 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":84,"fsid":"2a06e29a-8fe6-11ee-95a2-87774f69a715","created":"2023-12-01T01:09:30.210398+0000","modified":"2023-12-01T01:17:25.609766+0000","last_up_change":"2023-12-01T01:15:44.586092+0000","last_in_change":"2023-12-01T01:15:26.305439+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:12:30.514229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:16:23.568860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-01T01:16:24.531137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fc651e62-a103-4ab8-a76d-d3708ea6aeec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6803","nonce":2244799094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6805","nonce":2244799094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6809","nonce":2244799094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6807","nonce":2244799094}]},"public_addr":"172.21.15.77:6803/2244799094","cluster_addr":"172.21.15.77:6805/2244799094","heartbeat_back_addr":"172.21.15.77:6809/2244799094","heartbeat_front_addr":"172.21.15.77:6807/2244799094","state":["exists","up"]},{"osd":1,"uuid":"0adf8c54-49ec-49a0-8bf1-af1f20e60729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6801","nonce":224194890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6803","nonce":224194890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6807","nonce":224194890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6805","nonce":224194890}]},"public_addr":"172.21.15.137:6801/224194890","cluster_addr":"172.21.15.137:6803/224194890","heartbeat_back_addr":"172.21.15.137:6807/224194890","heartbeat_front_addr":"172.21.15.137:6805/224194890","state":["exists","up"]},{"osd":2,"uuid":"5a69802c-dede-4544-9a12-436f5c1c0038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6801","nonce":4078459110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6803","nonce":4078459110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6807","nonce":4078459110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6805","nonce":4078459110}]},"public_addr":"172.21.15.203:6801/4078459110","cluster_addr":"172.21.15.203:6803/4078459110","heartbeat_back_addr":"172.21.15.203:6807/4078459110","heartbeat_front_addr":"172.21.15.203:6805/4078459110","state":["exists","up"]},{"osd":3,"uuid":"ad325a97-580b-4c99-bf12-cbb0d92a6d0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6811","nonce":1454440477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6813","nonce":1454440477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6817","nonce":1454440477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6815","nonce":1454440477}]},"public_addr":"172.21.15.77:6811/1454440477","cluster_addr":"172.21.15.77:6813/1454440477","heartbeat_back_addr":"172.21.15.77:6817/1454440477","heartbeat_front_addr":"172.21.15.77:6815/1454440477","state":["exists","up"]},{"osd":4,"uuid":"1e3ac1ba-98bb-4b8b-8e26-50064c66bf2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6809","nonce":1103241451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6811","nonce":1103241451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6815","nonce":1103241451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6813","nonce":1103241451}]},"public_addr":"172.21.15.137:6809/1103241451","cluster_addr":"172.21.15.137:6811/1103241451","heartbeat_back_addr":"172.21.15.137:6815/1103241451","heartbeat_front_addr":"172.21.15.137:6813/1103241451","state":["exists","up"]},{"osd":5,"uuid":"eb32c016-abeb-44ab-abf0-19dd09235736","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6809","nonce":2648569482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6811","nonce":2648569482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6815","nonce":2648569482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6813","nonce":2648569482}]},"public_addr":"172.21.15.203:6809/2648569482","cluster_addr":"172.21.15.203:6811/2648569482","heartbeat_back_addr":"172.21.15.203:6815/2648569482","heartbeat_front_addr":"172.21.15.203:6813/2648569482","state":["exists","up"]},{"osd":6,"uuid":"0e3fa97e-2ca6-418c-9e8d-a6fe3c56adb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6819","nonce":2463414651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6821","nonce":2463414651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6825","nonce":2463414651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6823","nonce":2463414651}]},"public_addr":"172.21.15.77:6819/2463414651","cluster_addr":"172.21.15.77:6821/2463414651","heartbeat_back_addr":"172.21.15.77:6825/2463414651","heartbeat_front_addr":"172.21.15.77:6823/2463414651","state":["exists","up"]},{"osd":7,"uuid":"d7398c01-0f98-4138-8a49-c3ff0aae7e88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6817","nonce":3108330766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6819","nonce":3108330766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6823","nonce":3108330766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6821","nonce":3108330766}]},"public_addr":"172.21.15.137:6817/3108330766","cluster_addr":"172.21.15.137:6819/3108330766","heartbeat_back_addr":"172.21.15.137:6823/3108330766","heartbeat_front_addr":"172.21.15.137:6821/3108330766","state":["exists","up"]},{"osd":8,"uuid":"0c09050d-3e43-410c-8cd7-bcb891af56b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6817","nonce":1161158776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6819","nonce":1161158776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6823","nonce":1161158776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6821","nonce":1161158776}]},"public_addr":"172.21.15.203:6817/1161158776","cluster_addr":"172.21.15.203:6819/1161158776","heartbeat_back_addr":"172.21.15.203:6823/1161158776","heartbeat_front_addr":"172.21.15.203:6821/1161158776","state":["exists","up"]},{"osd":9,"uuid":"527ecb49-95ec-48bd-9872-3b07fae5c5ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6827","nonce":2409880685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6829","nonce":2409880685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6833","nonce":2409880685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6831","nonce":2409880685}]},"public_addr":"172.21.15.77:6827/2409880685","cluster_addr":"172.21.15.77:6829/2409880685","heartbeat_back_addr":"172.21.15.77:6833/2409880685","heartbeat_front_addr":"172.21.15.77:6831/2409880685","state":["exists","up"]},{"osd":10,"uuid":"1680bb3e-a3c8-427b-af68-e2214f32bed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6825","nonce":374920146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6827","nonce":374920146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6831","nonce":374920146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6829","nonce":374920146}]},"public_addr":"172.21.15.137:6825/374920146","cluster_addr":"172.21.15.137:6827/374920146","heartbeat_back_addr":"172.21.15.137:6831/374920146","heartbeat_front_addr":"172.21.15.137:6829/374920146","state":["exists","up"]},{"osd":11,"uuid":"8a533466-023d-4803-ae36-9a9f1715e49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6825","nonce":2161588962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6827","nonce":2161588962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6831","nonce":2161588962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6829","nonce":2161588962}]},"public_addr":"172.21.15.203:6825/2161588962","cluster_addr":"172.21.15.203:6827/2161588962","heartbeat_back_addr":"172.21.15.203:6831/2161588962","heartbeat_front_addr":"172.21.15.203:6829/2161588962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:11:45.420509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:06.210115+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:28.191307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:49.594116+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:11.261689+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:33.376541+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:54.968336+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:16.775310+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:38.733792+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:59.456699+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:15:20.924059+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:15:43.094773+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/2260035254":"2023-12-02T01:10:24.191835+0000","172.21.15.77:6800/1456621747":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/219838828":"2023-12-02T01:10:24.191835+0000","172.21.15.77:6801/1456621747":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/2937149266":"2023-12-02T01:10:02.232948+0000","172.21.15.77:0/807520018":"2023-12-02T01:10:02.232948+0000","172.21.15.77:6801/1661788026":"2023-12-02T01:10:02.232948+0000","172.21.15.77:6800/1661788026":"2023-12-02T01:10:02.232948+0000","172.21.15.77:0/3551282473":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/1202222558":"2023-12-02T01:10:02.232948+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:17:36.647 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.645+0000 7f6bbfd3a700 1 -- 172.21.15.77:0/4227748840 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f6ba40a05f0 msgr2=0x7f6ba40a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:36.647 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.645+0000 7f6bbfd3a700 1 --2- 172.21.15.77:0/4227748840 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f6ba40a05f0 0x7f6ba40a2ab0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4005950 tx=0x7f6bb40058e0 comp rx=0 tx=0).stop 2023-12-01T01:17:36.647 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.645+0000 7f6bbfd3a700 1 -- 172.21.15.77:0/4227748840 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bb81001a0 msgr2=0x7f6bb8103f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:36.647 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.645+0000 7f6bbfd3a700 1 --2- 172.21.15.77:0/4227748840 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bb81001a0 0x7f6bb8103f60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8004730 tx=0x7f6ba8009580 comp rx=0 tx=0).stop 2023-12-01T01:17:36.647 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.645+0000 7f6bbfd3a700 1 -- 172.21.15.77:0/4227748840 shutdown_connections 2023-12-01T01:17:36.647 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.645+0000 7f6bbfd3a700 1 --2- 172.21.15.77:0/4227748840 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f6ba40a05f0 0x7f6ba40a2ab0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:36.647 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.645+0000 7f6bbfd3a700 1 --2- 172.21.15.77:0/4227748840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bb8101870 0x7f6bb8102ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:36.647 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.645+0000 7f6bbfd3a700 1 --2- 172.21.15.77:0/4227748840 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6bb81001a0 0x7f6bb8103f60 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:36.647 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.645+0000 7f6bbfd3a700 1 --2- 172.21.15.77:0/4227748840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bb8100eb0 0x7f6bb81025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:36.647 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.645+0000 7f6bbfd3a700 1 -- 172.21.15.77:0/4227748840 >> 172.21.15.77:0/4227748840 conn(0x7f6bb80fa950 msgr2=0x7f6bb80fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:36.647 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.645+0000 7f6bbfd3a700 1 -- 172.21.15.77:0/4227748840 shutdown_connections 2023-12-01T01:17:36.647 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.645+0000 7f6bbfd3a700 1 -- 172.21.15.77:0/4227748840 wait complete. 2023-12-01T01:17:36.659 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:12:30.514229+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T01:16:23.568860+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T01:16:24.531137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:17:36.660 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:17:36.787 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.783+0000 7fcc20772700 1 -- 172.21.15.77:0/3445412483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc18101280 msgr2=0x7fcc181016a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:36.787 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.783+0000 7fcc20772700 1 --2- 172.21.15.77:0/3445412483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc18101280 0x7fcc181016a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fcc08009990 tx=0x7fcc08009ca0 comp rx=0 tx=0).stop 2023-12-01T01:17:36.787 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.783+0000 7fcc20772700 1 -- 172.21.15.77:0/3445412483 shutdown_connections 2023-12-01T01:17:36.787 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.783+0000 7fcc20772700 1 --2- 172.21.15.77:0/3445412483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc18100890 0x7fcc18108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:36.787 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.783+0000 7fcc20772700 1 --2- 172.21.15.77:0/3445412483 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc18101280 0x7fcc181016a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:36.787 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.783+0000 7fcc20772700 1 --2- 172.21.15.77:0/3445412483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc180ffed0 0x7fcc18100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:36.787 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.783+0000 7fcc20772700 1 -- 172.21.15.77:0/3445412483 >> 172.21.15.77:0/3445412483 conn(0x7fcc180fa960 msgr2=0x7fcc180fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:36.787 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.783+0000 7fcc20772700 1 -- 172.21.15.77:0/3445412483 shutdown_connections 2023-12-01T01:17:36.788 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.783+0000 7fcc20772700 1 -- 172.21.15.77:0/3445412483 wait complete. 2023-12-01T01:17:36.788 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.784+0000 7fcc20772700 1 Processor -- start 2023-12-01T01:17:36.788 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.784+0000 7fcc20772700 1 -- start start 2023-12-01T01:17:36.788 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.784+0000 7fcc20772700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc180ffed0 0x7fcc180ff990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:36.788 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.784+0000 7fcc20772700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc18100890 0x7fcc1819b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:36.788 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.784+0000 7fcc20772700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc18101280 0x7fcc1819de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:36.788 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.784+0000 7fcc20772700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1819e5b0 con 0x7fcc18101280 2023-12-01T01:17:36.788 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.784+0000 7fcc20772700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1819e720 con 0x7fcc18100890 2023-12-01T01:17:36.788 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.784+0000 7fcc20772700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1819e8e0 con 0x7fcc180ffed0 2023-12-01T01:17:36.788 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.784+0000 7fcc1ed0f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc18101280 0x7fcc1819de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:36.788 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.784+0000 7fcc1ed0f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc18101280 0x7fcc1819de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58626/0 (socket says 172.21.15.77:58626) 2023-12-01T01:17:36.788 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.784+0000 7fcc1ed0f700 1 -- 172.21.15.77:0/4247560768 learned_addr learned my addr 172.21.15.77:0/4247560768 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:36.788 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.784+0000 7fcc1e50e700 1 --2- 172.21.15.77:0/4247560768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc180ffed0 0x7fcc180ff990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:36.788 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.784+0000 7fcc1dd0d700 1 --2- 172.21.15.77:0/4247560768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc18100890 0x7fcc1819b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:36.789 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.785+0000 7fcc1ed0f700 1 -- 172.21.15.77:0/4247560768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc180ffed0 msgr2=0x7fcc180ff990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:36.789 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.785+0000 7fcc1ed0f700 1 --2- 172.21.15.77:0/4247560768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc180ffed0 0x7fcc180ff990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:36.789 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.785+0000 7fcc1ed0f700 1 -- 172.21.15.77:0/4247560768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc18100890 msgr2=0x7fcc1819b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:36.789 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.785+0000 7fcc1ed0f700 1 --2- 172.21.15.77:0/4247560768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc18100890 0x7fcc1819b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:36.789 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.785+0000 7fcc1ed0f700 1 -- 172.21.15.77:0/4247560768 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc080095f0 con 0x7fcc18101280 2023-12-01T01:17:36.789 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.785+0000 7fcc1ed0f700 1 --2- 172.21.15.77:0/4247560768 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc18101280 0x7fcc1819de10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fcc10007ee0 tx=0x7fcc1000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:36.789 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.785+0000 7fcc0f7fe700 1 -- 172.21.15.77:0/4247560768 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc10017070 con 0x7fcc18101280 2023-12-01T01:17:36.789 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.785+0000 7fcc0f7fe700 1 -- 172.21.15.77:0/4247560768 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc1000f040 con 0x7fcc18101280 2023-12-01T01:17:36.789 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.785+0000 7fcc20772700 1 -- 172.21.15.77:0/4247560768 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc1819ebc0 con 0x7fcc18101280 2023-12-01T01:17:36.789 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.785+0000 7fcc0f7fe700 1 -- 172.21.15.77:0/4247560768 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc10005580 con 0x7fcc18101280 2023-12-01T01:17:36.789 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.785+0000 7fcc20772700 1 -- 172.21.15.77:0/4247560768 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc1819f080 con 0x7fcc18101280 2023-12-01T01:17:36.789 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.786+0000 7fcc20772700 1 -- 172.21.15.77:0/4247560768 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc18106ca0 con 0x7fcc18101280 2023-12-01T01:17:36.790 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.787+0000 7fcc0f7fe700 1 -- 172.21.15.77:0/4247560768 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fcc1000db20 con 0x7fcc18101280 2023-12-01T01:17:36.790 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.788+0000 7fcc0f7fe700 1 --2- 172.21.15.77:0/4247560768 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fcc040a05b0 0x7fcc040a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:36.790 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.788+0000 7fcc1e50e700 1 --2- 172.21.15.77:0/4247560768 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fcc040a05b0 0x7fcc040a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:36.790 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.788+0000 7fcc0f7fe700 1 -- 172.21.15.77:0/4247560768 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fcc100cb700 con 0x7fcc18101280 2023-12-01T01:17:36.790 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.788+0000 7fcc1e50e700 1 --2- 172.21.15.77:0/4247560768 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fcc040a05b0 0x7fcc040a2a70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcc0800b5c0 tx=0x7fcc08005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:36.793 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.792+0000 7fcc0f7fe700 1 -- 172.21.15.77:0/4247560768 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fcc10095240 con 0x7fcc18101280 2023-12-01T01:17:36.903 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:36 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/685612126' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-12-01T01:17:36.988 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.987+0000 7fcc20772700 1 -- 172.21.15.77:0/4247560768 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcc1802d100 con 0x7fcc18101280 2023-12-01T01:17:36.989 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.987+0000 7fcc0f7fe700 1 -- 172.21.15.77:0/4247560768 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcc10098860 con 0x7fcc18101280 2023-12-01T01:17:36.989 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 1 2023-12-01T01:17:36.991 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.989+0000 7fcc20772700 1 -- 172.21.15.77:0/4247560768 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fcc040a05b0 msgr2=0x7fcc040a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:36.991 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.990+0000 7fcc20772700 1 --2- 172.21.15.77:0/4247560768 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fcc040a05b0 0x7fcc040a2a70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcc0800b5c0 tx=0x7fcc08005fb0 comp rx=0 tx=0).stop 2023-12-01T01:17:36.991 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.990+0000 7fcc20772700 1 -- 172.21.15.77:0/4247560768 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc18101280 msgr2=0x7fcc1819de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:36.991 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.990+0000 7fcc20772700 1 --2- 172.21.15.77:0/4247560768 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc18101280 0x7fcc1819de10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fcc10007ee0 tx=0x7fcc1000d3b0 comp rx=0 tx=0).stop 2023-12-01T01:17:36.991 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.990+0000 7fcc20772700 1 -- 172.21.15.77:0/4247560768 shutdown_connections 2023-12-01T01:17:36.991 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.990+0000 7fcc20772700 1 --2- 172.21.15.77:0/4247560768 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fcc040a05b0 0x7fcc040a2a70 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:36.991 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.990+0000 7fcc20772700 1 --2- 172.21.15.77:0/4247560768 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc18101280 0x7fcc1819de10 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:36.991 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.990+0000 7fcc20772700 1 --2- 172.21.15.77:0/4247560768 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc180ffed0 0x7fcc180ff990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:36.991 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.990+0000 7fcc20772700 1 --2- 172.21.15.77:0/4247560768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc18100890 0x7fcc1819b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:36.991 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.990+0000 7fcc20772700 1 -- 172.21.15.77:0/4247560768 >> 172.21.15.77:0/4247560768 conn(0x7fcc180fa960 msgr2=0x7fcc180fcc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:36.992 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.990+0000 7fcc20772700 1 -- 172.21.15.77:0/4247560768 shutdown_connections 2023-12-01T01:17:36.992 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:36.990+0000 7fcc20772700 1 -- 172.21.15.77:0/4247560768 wait complete. 2023-12-01T01:17:37.003 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T01:17:37.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:36 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/685612126' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-12-01T01:17:37.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:36 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/685612126' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-12-01T01:17:37.130 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.126+0000 7fafecb1b700 1 -- 172.21.15.77:0/336070869 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafe8100e70 msgr2=0x7fafe81012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:37.130 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.126+0000 7fafecb1b700 1 --2- 172.21.15.77:0/336070869 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafe8100e70 0x7fafe81012d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fafd00099c0 tx=0x7fafd0009cd0 comp rx=0 tx=0).stop 2023-12-01T01:17:37.130 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.126+0000 7fafecb1b700 1 -- 172.21.15.77:0/336070869 shutdown_connections 2023-12-01T01:17:37.130 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.126+0000 7fafecb1b700 1 --2- 172.21.15.77:0/336070869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafe8101810 0x7fafe81080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:37.131 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.126+0000 7fafecb1b700 1 --2- 172.21.15.77:0/336070869 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafe80ffc80 0x7fafe81000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:37.131 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.126+0000 7fafecb1b700 1 --2- 172.21.15.77:0/336070869 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafe8100e70 0x7fafe81012d0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:37.131 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.126+0000 7fafecb1b700 1 -- 172.21.15.77:0/336070869 >> 172.21.15.77:0/336070869 conn(0x7fafe80fb1f0 msgr2=0x7fafe80fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:37.131 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.126+0000 7fafecb1b700 1 -- 172.21.15.77:0/336070869 shutdown_connections 2023-12-01T01:17:37.131 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.126+0000 7fafecb1b700 1 -- 172.21.15.77:0/336070869 wait complete. 2023-12-01T01:17:37.131 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.127+0000 7fafecb1b700 1 Processor -- start 2023-12-01T01:17:37.131 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.127+0000 7fafecb1b700 1 -- start start 2023-12-01T01:17:37.131 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.127+0000 7fafecb1b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafe80ffc80 0x7fafe8196ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:37.131 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.127+0000 7fafecb1b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafe8100e70 0x7fafe8197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:37.131 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.127+0000 7fafecb1b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafe8101810 0x7fafe819e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:37.131 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.127+0000 7fafecb1b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe819e660 con 0x7fafe8101810 2023-12-01T01:17:37.131 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.127+0000 7fafecb1b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe819e7a0 con 0x7fafe8100e70 2023-12-01T01:17:37.131 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.127+0000 7fafecb1b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe819e930 con 0x7fafe80ffc80 2023-12-01T01:17:37.132 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.127+0000 7fafe6d9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafe8101810 0x7fafe819e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:37.132 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.127+0000 7fafe6d9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafe8101810 0x7fafe819e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58640/0 (socket says 172.21.15.77:58640) 2023-12-01T01:17:37.132 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.127+0000 7fafe6d9d700 1 -- 172.21.15.77:0/3067123376 learned_addr learned my addr 172.21.15.77:0/3067123376 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:37.132 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.127+0000 7fafe5d9b700 1 --2- 172.21.15.77:0/3067123376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafe8100e70 0x7fafe8197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:37.132 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.127+0000 7fafe659c700 1 --2- 172.21.15.77:0/3067123376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafe80ffc80 0x7fafe8196ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:37.132 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.127+0000 7fafe6d9d700 1 -- 172.21.15.77:0/3067123376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafe80ffc80 msgr2=0x7fafe8196ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:37.132 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.127+0000 7fafe6d9d700 1 --2- 172.21.15.77:0/3067123376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafe80ffc80 0x7fafe8196ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:37.132 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.127+0000 7fafe6d9d700 1 -- 172.21.15.77:0/3067123376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafe8100e70 msgr2=0x7fafe8197020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:37.132 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.127+0000 7fafe6d9d700 1 --2- 172.21.15.77:0/3067123376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafe8100e70 0x7fafe8197020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:37.132 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.127+0000 7fafe6d9d700 1 -- 172.21.15.77:0/3067123376 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd00095f0 con 0x7fafe8101810 2023-12-01T01:17:37.132 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.127+0000 7fafe6d9d700 1 --2- 172.21.15.77:0/3067123376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafe8101810 0x7fafe819e090 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fafe0007ee0 tx=0x7fafe000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:37.132 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.128+0000 7fafdf7fe700 1 -- 172.21.15.77:0/3067123376 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe0017070 con 0x7fafe8101810 2023-12-01T01:17:37.132 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.128+0000 7fafdf7fe700 1 -- 172.21.15.77:0/3067123376 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafe000f040 con 0x7fafe8101810 2023-12-01T01:17:37.132 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.128+0000 7fafecb1b700 1 -- 172.21.15.77:0/3067123376 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafe819ec10 con 0x7fafe8101810 2023-12-01T01:17:37.133 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.128+0000 7fafdf7fe700 1 -- 172.21.15.77:0/3067123376 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe0005580 con 0x7fafe8101810 2023-12-01T01:17:37.133 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.128+0000 7fafecb1b700 1 -- 172.21.15.77:0/3067123376 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafe819f120 con 0x7fafe8101810 2023-12-01T01:17:37.133 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.129+0000 7fafecb1b700 1 -- 172.21.15.77:0/3067123376 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafe810b4e0 con 0x7fafe8101810 2023-12-01T01:17:37.134 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.130+0000 7fafdf7fe700 1 -- 172.21.15.77:0/3067123376 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fafe000db20 con 0x7fafe8101810 2023-12-01T01:17:37.134 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.130+0000 7fafdf7fe700 1 --2- 172.21.15.77:0/3067123376 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fafd40a0630 0x7fafd40a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:37.134 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.130+0000 7fafe659c700 1 --2- 172.21.15.77:0/3067123376 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fafd40a0630 0x7fafd40a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:37.134 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.130+0000 7fafdf7fe700 1 -- 172.21.15.77:0/3067123376 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fafe00cb100 con 0x7fafe8101810 2023-12-01T01:17:37.134 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.131+0000 7fafe659c700 1 --2- 172.21.15.77:0/3067123376 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fafd40a0630 0x7fafd40a2af0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fafe8100cd0 tx=0x7fafd800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:37.135 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.134+0000 7fafdf7fe700 1 -- 172.21.15.77:0/3067123376 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fafe0094c40 con 0x7fafe8101810 2023-12-01T01:17:37.328 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.327+0000 7fafecb1b700 1 -- 172.21.15.77:0/3067123376 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fafe819f6d0 con 0x7fafe8101810 2023-12-01T01:17:37.328 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.327+0000 7fafdf7fe700 1 -- 172.21.15.77:0/3067123376 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7fafe0098260 con 0x7fafe8101810 2023-12-01T01:17:37.329 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 32 2023-12-01T01:17:37.330 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.329+0000 7fafecb1b700 1 -- 172.21.15.77:0/3067123376 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fafd40a0630 msgr2=0x7fafd40a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:37.330 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.329+0000 7fafecb1b700 1 --2- 172.21.15.77:0/3067123376 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fafd40a0630 0x7fafd40a2af0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fafe8100cd0 tx=0x7fafd800b410 comp rx=0 tx=0).stop 2023-12-01T01:17:37.330 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.329+0000 7fafecb1b700 1 -- 172.21.15.77:0/3067123376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafe8101810 msgr2=0x7fafe819e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:37.331 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.329+0000 7fafecb1b700 1 --2- 172.21.15.77:0/3067123376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafe8101810 0x7fafe819e090 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fafe0007ee0 tx=0x7fafe000d3b0 comp rx=0 tx=0).stop 2023-12-01T01:17:37.331 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.329+0000 7fafecb1b700 1 -- 172.21.15.77:0/3067123376 shutdown_connections 2023-12-01T01:17:37.331 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.329+0000 7fafecb1b700 1 --2- 172.21.15.77:0/3067123376 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fafd40a0630 0x7fafd40a2af0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:37.331 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.329+0000 7fafecb1b700 1 --2- 172.21.15.77:0/3067123376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafe8101810 0x7fafe819e090 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:37.331 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.329+0000 7fafecb1b700 1 --2- 172.21.15.77:0/3067123376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafe80ffc80 0x7fafe8196ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:37.331 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.329+0000 7fafecb1b700 1 --2- 172.21.15.77:0/3067123376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafe8100e70 0x7fafe8197020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:37.331 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.329+0000 7fafecb1b700 1 -- 172.21.15.77:0/3067123376 >> 172.21.15.77:0/3067123376 conn(0x7fafe80fb1f0 msgr2=0x7fafe8067f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:37.331 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.329+0000 7fafecb1b700 1 -- 172.21.15.77:0/3067123376 shutdown_connections 2023-12-01T01:17:37.331 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.329+0000 7fafecb1b700 1 -- 172.21.15.77:0/3067123376 wait complete. 2023-12-01T01:17:37.343 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T01:17:37.470 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.468+0000 7fed55348700 1 -- 172.21.15.77:0/1927374915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed500ff070 msgr2=0x7fed500ff490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:37.470 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.468+0000 7fed55348700 1 --2- 172.21.15.77:0/1927374915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed500ff070 0x7fed500ff490 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fed40009990 tx=0x7fed40009ca0 comp rx=0 tx=0).stop 2023-12-01T01:17:37.470 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.468+0000 7fed55348700 1 -- 172.21.15.77:0/1927374915 shutdown_connections 2023-12-01T01:17:37.470 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.468+0000 7fed55348700 1 --2- 172.21.15.77:0/1927374915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed501029e0 0x7fed50109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:37.470 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.468+0000 7fed55348700 1 --2- 172.21.15.77:0/1927374915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed500ff070 0x7fed500ff490 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:37.470 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.468+0000 7fed55348700 1 --2- 172.21.15.77:0/1927374915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed50102040 0x7fed501024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:37.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.468+0000 7fed55348700 1 -- 172.21.15.77:0/1927374915 >> 172.21.15.77:0/1927374915 conn(0x7fed500fa980 msgr2=0x7fed500fcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:37.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.468+0000 7fed55348700 1 -- 172.21.15.77:0/1927374915 shutdown_connections 2023-12-01T01:17:37.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.468+0000 7fed55348700 1 -- 172.21.15.77:0/1927374915 wait complete. 2023-12-01T01:17:37.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.469+0000 7fed55348700 1 Processor -- start 2023-12-01T01:17:37.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.469+0000 7fed55348700 1 -- start start 2023-12-01T01:17:37.473 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.469+0000 7fed55348700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed500ff070 0x7fed50196ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:37.473 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.469+0000 7fed55348700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed50102040 0x7fed50197010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:37.473 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.469+0000 7fed55348700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed501029e0 0x7fed5019e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:37.473 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.469+0000 7fed55348700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5019e650 con 0x7fed50102040 2023-12-01T01:17:37.473 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.469+0000 7fed55348700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5019e790 con 0x7fed500ff070 2023-12-01T01:17:37.473 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.469+0000 7fed55348700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5019e920 con 0x7fed501029e0 2023-12-01T01:17:37.473 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.470+0000 7fed4e7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed50102040 0x7fed50197010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:37.473 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.470+0000 7fed4e7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed50102040 0x7fed50197010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58652/0 (socket says 172.21.15.77:58652) 2023-12-01T01:17:37.473 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.470+0000 7fed4e7fc700 1 -- 172.21.15.77:0/2899022456 learned_addr learned my addr 172.21.15.77:0/2899022456 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:37.473 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.470+0000 7fed4f7fe700 1 --2- 172.21.15.77:0/2899022456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed501029e0 0x7fed5019e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:37.473 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.470+0000 7fed4effd700 1 --2- 172.21.15.77:0/2899022456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed500ff070 0x7fed50196ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:37.473 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.470+0000 7fed4e7fc700 1 -- 172.21.15.77:0/2899022456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed501029e0 msgr2=0x7fed5019e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:37.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.470+0000 7fed4e7fc700 1 --2- 172.21.15.77:0/2899022456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed501029e0 0x7fed5019e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:37.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.470+0000 7fed4e7fc700 1 -- 172.21.15.77:0/2899022456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed500ff070 msgr2=0x7fed50196ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:37.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.470+0000 7fed4e7fc700 1 --2- 172.21.15.77:0/2899022456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed500ff070 0x7fed50196ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:37.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.470+0000 7fed4e7fc700 1 -- 172.21.15.77:0/2899022456 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed400095f0 con 0x7fed50102040 2023-12-01T01:17:37.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.470+0000 7fed4effd700 1 --2- 172.21.15.77:0/2899022456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed500ff070 0x7fed50196ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:17:37.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.470+0000 7fed4e7fc700 1 --2- 172.21.15.77:0/2899022456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed50102040 0x7fed50197010 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fed4400d980 tx=0x7fed4400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:37.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.470+0000 7fed37fff700 1 -- 172.21.15.77:0/2899022456 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4400bb60 con 0x7fed50102040 2023-12-01T01:17:37.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.470+0000 7fed37fff700 1 -- 172.21.15.77:0/2899022456 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed44007640 con 0x7fed50102040 2023-12-01T01:17:37.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.470+0000 7fed55348700 1 -- 172.21.15.77:0/2899022456 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed5019ec00 con 0x7fed50102040 2023-12-01T01:17:37.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.470+0000 7fed37fff700 1 -- 172.21.15.77:0/2899022456 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed44004e90 con 0x7fed50102040 2023-12-01T01:17:37.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.470+0000 7fed55348700 1 -- 172.21.15.77:0/2899022456 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed5019f160 con 0x7fed50102040 2023-12-01T01:17:37.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.471+0000 7fed55348700 1 -- 172.21.15.77:0/2899022456 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed5010c630 con 0x7fed50102040 2023-12-01T01:17:37.475 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.472+0000 7fed37fff700 1 -- 172.21.15.77:0/2899022456 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fed4400bcc0 con 0x7fed50102040 2023-12-01T01:17:37.475 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.473+0000 7fed37fff700 1 --2- 172.21.15.77:0/2899022456 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fed380a0630 0x7fed380a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:37.475 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.473+0000 7fed4effd700 1 --2- 172.21.15.77:0/2899022456 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fed380a0630 0x7fed380a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:37.475 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.473+0000 7fed37fff700 1 -- 172.21.15.77:0/2899022456 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fed440cbea0 con 0x7fed50102040 2023-12-01T01:17:37.475 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.473+0000 7fed4effd700 1 --2- 172.21.15.77:0/2899022456 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fed380a0630 0x7fed380a2af0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fed4000b5c0 tx=0x7fed40005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:37.477 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.476+0000 7fed37fff700 1 -- 172.21.15.77:0/2899022456 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fed440958e0 con 0x7fed50102040 2023-12-01T01:17:37.671 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.669+0000 7fed55348700 1 -- 172.21.15.77:0/2899022456 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fed50107d90 con 0x7fed50102040 2023-12-01T01:17:37.671 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.670+0000 7fed37fff700 1 -- 172.21.15.77:0/2899022456 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7fed44016070 con 0x7fed50102040 2023-12-01T01:17:37.671 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 64 2023-12-01T01:17:37.673 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.672+0000 7fed55348700 1 -- 172.21.15.77:0/2899022456 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fed380a0630 msgr2=0x7fed380a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:37.674 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.672+0000 7fed55348700 1 --2- 172.21.15.77:0/2899022456 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fed380a0630 0x7fed380a2af0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fed4000b5c0 tx=0x7fed40005fb0 comp rx=0 tx=0).stop 2023-12-01T01:17:37.674 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.672+0000 7fed55348700 1 -- 172.21.15.77:0/2899022456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed50102040 msgr2=0x7fed50197010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:37.674 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.672+0000 7fed55348700 1 --2- 172.21.15.77:0/2899022456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed50102040 0x7fed50197010 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fed4400d980 tx=0x7fed4400dd40 comp rx=0 tx=0).stop 2023-12-01T01:17:37.674 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.672+0000 7fed55348700 1 -- 172.21.15.77:0/2899022456 shutdown_connections 2023-12-01T01:17:37.674 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.672+0000 7fed55348700 1 --2- 172.21.15.77:0/2899022456 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fed380a0630 0x7fed380a2af0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:37.674 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.672+0000 7fed55348700 1 --2- 172.21.15.77:0/2899022456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed501029e0 0x7fed5019e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:37.674 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.672+0000 7fed55348700 1 --2- 172.21.15.77:0/2899022456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed500ff070 0x7fed50196ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:37.674 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.672+0000 7fed55348700 1 --2- 172.21.15.77:0/2899022456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed50102040 0x7fed50197010 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:37.674 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.672+0000 7fed55348700 1 -- 172.21.15.77:0/2899022456 >> 172.21.15.77:0/2899022456 conn(0x7fed500fa980 msgr2=0x7fed500fcc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:37.674 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.672+0000 7fed55348700 1 -- 172.21.15.77:0/2899022456 shutdown_connections 2023-12-01T01:17:37.674 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.672+0000 7fed55348700 1 -- 172.21.15.77:0/2899022456 wait complete. 2023-12-01T01:17:37.686 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:17:37.809 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.806+0000 7f0603f27700 1 -- 172.21.15.77:0/1788694020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05fc100e70 msgr2=0x7f05fc1012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:37.809 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.806+0000 7f0603f27700 1 --2- 172.21.15.77:0/1788694020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05fc100e70 0x7f05fc1012d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f05f80099c0 tx=0x7f05f8009cd0 comp rx=0 tx=0).stop 2023-12-01T01:17:37.810 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.806+0000 7f0603f27700 1 -- 172.21.15.77:0/1788694020 shutdown_connections 2023-12-01T01:17:37.810 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.806+0000 7f0603f27700 1 --2- 172.21.15.77:0/1788694020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05fc101810 0x7f05fc1080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:37.810 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.806+0000 7f0603f27700 1 --2- 172.21.15.77:0/1788694020 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05fc0ffc80 0x7f05fc1000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:37.810 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.806+0000 7f0603f27700 1 --2- 172.21.15.77:0/1788694020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05fc100e70 0x7f05fc1012d0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:37.810 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.806+0000 7f0603f27700 1 -- 172.21.15.77:0/1788694020 >> 172.21.15.77:0/1788694020 conn(0x7f05fc0fb1b0 msgr2=0x7f05fc0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:37.810 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.806+0000 7f0603f27700 1 -- 172.21.15.77:0/1788694020 shutdown_connections 2023-12-01T01:17:37.810 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.806+0000 7f0603f27700 1 -- 172.21.15.77:0/1788694020 wait complete. 2023-12-01T01:17:37.810 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.806+0000 7f0603f27700 1 Processor -- start 2023-12-01T01:17:37.810 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.807+0000 7f0603f27700 1 -- start start 2023-12-01T01:17:37.810 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.807+0000 7f0603f27700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05fc0ffc80 0x7f05fc198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:37.810 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.807+0000 7f0603f27700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05fc100e70 0x7f05fc199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:37.810 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.807+0000 7f0603f27700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05fc101810 0x7f05fc1a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:37.810 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.807+0000 7f0603f27700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc1a0770 con 0x7f05fc101810 2023-12-01T01:17:37.811 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.807+0000 7f0603f27700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc1a08b0 con 0x7f05fc100e70 2023-12-01T01:17:37.811 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.807+0000 7f0603f27700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc1a0a40 con 0x7f05fc0ffc80 2023-12-01T01:17:37.811 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.807+0000 7f06024c4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05fc101810 0x7f05fc1a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:37.811 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.807+0000 7f06024c4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05fc101810 0x7f05fc1a01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58668/0 (socket says 172.21.15.77:58668) 2023-12-01T01:17:37.811 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.807+0000 7f06024c4700 1 -- 172.21.15.77:0/1528911082 learned_addr learned my addr 172.21.15.77:0/1528911082 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:37.811 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.807+0000 7f06014c2700 1 --2- 172.21.15.77:0/1528911082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05fc100e70 0x7f05fc199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:37.811 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.807+0000 7f0601cc3700 1 --2- 172.21.15.77:0/1528911082 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05fc0ffc80 0x7f05fc198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:37.811 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.807+0000 7f06024c4700 1 -- 172.21.15.77:0/1528911082 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05fc0ffc80 msgr2=0x7f05fc198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:37.811 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.807+0000 7f06024c4700 1 --2- 172.21.15.77:0/1528911082 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05fc0ffc80 0x7f05fc198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:37.811 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.807+0000 7f06024c4700 1 -- 172.21.15.77:0/1528911082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05fc100e70 msgr2=0x7f05fc199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:37.811 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.807+0000 7f06024c4700 1 --2- 172.21.15.77:0/1528911082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05fc100e70 0x7f05fc199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:37.811 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.807+0000 7f06024c4700 1 -- 172.21.15.77:0/1528911082 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f80095f0 con 0x7f05fc101810 2023-12-01T01:17:37.811 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.807+0000 7f0601cc3700 1 --2- 172.21.15.77:0/1528911082 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05fc0ffc80 0x7f05fc198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:17:37.811 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.807+0000 7f06024c4700 1 --2- 172.21.15.77:0/1528911082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05fc101810 0x7f05fc1a01a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f05f400ef30 tx=0x7f05f400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:37.812 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.807+0000 7f05eeffd700 1 -- 172.21.15.77:0/1528911082 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f4018070 con 0x7f05fc101810 2023-12-01T01:17:37.812 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.807+0000 7f05eeffd700 1 -- 172.21.15.77:0/1528911082 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05f400f040 con 0x7f05fc101810 2023-12-01T01:17:37.812 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.807+0000 7f0603f27700 1 -- 172.21.15.77:0/1528911082 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05fc1a0cc0 con 0x7f05fc101810 2023-12-01T01:17:37.812 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.807+0000 7f05eeffd700 1 -- 172.21.15.77:0/1528911082 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f400cdb0 con 0x7f05fc101810 2023-12-01T01:17:37.812 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.807+0000 7f0603f27700 1 -- 172.21.15.77:0/1528911082 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05fc1a11d0 con 0x7f05fc101810 2023-12-01T01:17:37.812 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.808+0000 7f0603f27700 1 -- 172.21.15.77:0/1528911082 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05fc10b490 con 0x7f05fc101810 2023-12-01T01:17:37.813 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.810+0000 7f05eeffd700 1 -- 172.21.15.77:0/1528911082 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f05f4003a00 con 0x7f05fc101810 2023-12-01T01:17:37.813 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.810+0000 7f05eeffd700 1 --2- 172.21.15.77:0/1528911082 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f05e80a0670 0x7f05e80a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:37.813 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.810+0000 7f0601cc3700 1 --2- 172.21.15.77:0/1528911082 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f05e80a0670 0x7f05e80a2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:37.813 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.811+0000 7f05eeffd700 1 -- 172.21.15.77:0/1528911082 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f05f40cbf70 con 0x7f05fc101810 2023-12-01T01:17:37.813 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.811+0000 7f0601cc3700 1 --2- 172.21.15.77:0/1528911082 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f05e80a0670 0x7f05e80a2b30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f05f0005950 tx=0x7f05f00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:37.815 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:37.814+0000 7f05eeffd700 1 -- 172.21.15.77:0/1528911082 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f05f4095b40 con 0x7f05fc101810 2023-12-01T01:17:37.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:37 smithi077 ceph-mon[98867]: pgmap v290: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s; 16 B/s, 0 keys/s, 0 objects/s recovering 2023-12-01T01:17:37.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:37 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/4227748840' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:17:37.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:37 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/4247560768' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-01T01:17:37.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:37 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/3067123376' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-01T01:17:38.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:37 smithi203 ceph-mon[103581]: pgmap v290: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s; 16 B/s, 0 keys/s, 0 objects/s recovering 2023-12-01T01:17:38.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:37 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/4227748840' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:17:38.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:37 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/4247560768' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-01T01:17:38.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:37 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/3067123376' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-01T01:17:38.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:37 smithi137 ceph-mon[103312]: pgmap v290: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s; 16 B/s, 0 keys/s, 0 objects/s recovering 2023-12-01T01:17:38.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:37 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/4227748840' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:17:38.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:37 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/4247560768' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-01T01:17:38.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:37 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/3067123376' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-01T01:17:38.051 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.050+0000 7f0603f27700 1 -- 172.21.15.77:0/1528911082 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f05fc10b770 con 0x7f05fc101810 2023-12-01T01:17:38.051 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.050+0000 7f05eeffd700 1 -- 172.21.15.77:0/1528911082 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10838 (secure 0 0 0) 0x7f05fc10b770 con 0x7f05fc101810 2023-12-01T01:17:38.052 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:17:38.052 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1435145620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1435145620},{"type":"v1","addr":"172.21.15.77:6835","nonce":1435145620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/841838908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":841838908},{"type":"v1","addr":"172.21.15.77:6837","nonce":841838908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-01T01:16:25.345276+0000","modified":"2023-12-01T01:16:32.596617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24496,"mds_2":24475,"mds_3":24473,"mds_4":14649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/3726913446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3726913446},{"type":"v1","addr":"172.21.15.77:6839","nonce":3726913446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14649":{"gid":14649,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":5,"addr":"172.21.15.77:6841/3023184965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3023184965},{"type":"v1","addr":"172.21.15.77:6841","nonce":3023184965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/2249638445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2249638445},{"type":"v1","addr":"172.21.15.137:6833","nonce":2249638445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6833/1201599774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1201599774},{"type":"v1","addr":"172.21.15.203:6833","nonce":1201599774}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3935354878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3935354878},{"type":"v1","addr":"172.21.15.137:6835","nonce":3935354878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3237538176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3237538176},{"type":"v1","addr":"172.21.15.203:6835","nonce":3237538176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3518881867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3518881867},{"type":"v1","addr":"172.21.15.137:6837","nonce":3518881867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.203:6837/2887080977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2887080977},{"type":"v1","addr":"172.21.15.203:6837","nonce":2887080977}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.203:6839/2888221613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2888221613},{"type":"v1","addr":"172.21.15.203:6839","nonce":2888221613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/1099966660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1099966660},{"type":"v1","addr":"172.21.15.137:6839","nonce":1099966660}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-01T01:17:38.054 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.053+0000 7f0603f27700 1 -- 172.21.15.77:0/1528911082 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f05e80a0670 msgr2=0x7f05e80a2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:38.054 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.053+0000 7f0603f27700 1 --2- 172.21.15.77:0/1528911082 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f05e80a0670 0x7f05e80a2b30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f05f0005950 tx=0x7f05f00058e0 comp rx=0 tx=0).stop 2023-12-01T01:17:38.054 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.053+0000 7f0603f27700 1 -- 172.21.15.77:0/1528911082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05fc101810 msgr2=0x7f05fc1a01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:38.054 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.053+0000 7f0603f27700 1 --2- 172.21.15.77:0/1528911082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05fc101810 0x7f05fc1a01a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f05f400ef30 tx=0x7f05f400c3a0 comp rx=0 tx=0).stop 2023-12-01T01:17:38.054 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.053+0000 7f0603f27700 1 -- 172.21.15.77:0/1528911082 shutdown_connections 2023-12-01T01:17:38.054 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.053+0000 7f0603f27700 1 --2- 172.21.15.77:0/1528911082 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f05e80a0670 0x7f05e80a2b30 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:38.054 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.053+0000 7f0603f27700 1 --2- 172.21.15.77:0/1528911082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05fc101810 0x7f05fc1a01a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:38.054 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.053+0000 7f0603f27700 1 --2- 172.21.15.77:0/1528911082 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05fc0ffc80 0x7f05fc198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:38.054 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.053+0000 7f0603f27700 1 --2- 172.21.15.77:0/1528911082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05fc100e70 0x7f05fc199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:38.055 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.053+0000 7f0603f27700 1 -- 172.21.15.77:0/1528911082 >> 172.21.15.77:0/1528911082 conn(0x7f05fc0fb1b0 msgr2=0x7f05fc068f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:38.055 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.053+0000 7f0603f27700 1 -- 172.21.15.77:0/1528911082 shutdown_connections 2023-12-01T01:17:38.055 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.053+0000 7f0603f27700 1 -- 172.21.15.77:0/1528911082 wait complete. 2023-12-01T01:17:38.055 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 27 2023-12-01T01:17:38.066 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:17:38.189 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.186+0000 7ff3ac941700 1 -- 172.21.15.77:0/2689278611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3a8100e80 msgr2=0x7ff3a81012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:38.189 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.186+0000 7ff3ac941700 1 --2- 172.21.15.77:0/2689278611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3a8100e80 0x7ff3a81012e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff3980099c0 tx=0x7ff398009cd0 comp rx=0 tx=0).stop 2023-12-01T01:17:38.189 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.186+0000 7ff3ac941700 1 -- 172.21.15.77:0/2689278611 shutdown_connections 2023-12-01T01:17:38.189 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.186+0000 7ff3ac941700 1 --2- 172.21.15.77:0/2689278611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3a8101820 0x7ff3a81080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:38.189 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.186+0000 7ff3ac941700 1 --2- 172.21.15.77:0/2689278611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3a80ffc90 0x7ff3a81000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:38.189 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.186+0000 7ff3ac941700 1 --2- 172.21.15.77:0/2689278611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3a8100e80 0x7ff3a81012e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:38.189 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.186+0000 7ff3ac941700 1 -- 172.21.15.77:0/2689278611 >> 172.21.15.77:0/2689278611 conn(0x7ff3a80fb1c0 msgr2=0x7ff3a80fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:38.189 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.186+0000 7ff3ac941700 1 -- 172.21.15.77:0/2689278611 shutdown_connections 2023-12-01T01:17:38.189 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.186+0000 7ff3ac941700 1 -- 172.21.15.77:0/2689278611 wait complete. 2023-12-01T01:17:38.190 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.187+0000 7ff3ac941700 1 Processor -- start 2023-12-01T01:17:38.190 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.187+0000 7ff3ac941700 1 -- start start 2023-12-01T01:17:38.190 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.187+0000 7ff3ac941700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3a80ffc90 0x7ff3a8069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:38.190 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.187+0000 7ff3ac941700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3a8100e80 0x7ff3a819b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:38.190 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.187+0000 7ff3ac941700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3a8101820 0x7ff3a819dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:38.190 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.187+0000 7ff3ac941700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a819e4e0 con 0x7ff3a8100e80 2023-12-01T01:17:38.190 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.187+0000 7ff3ac941700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a819e650 con 0x7ff3a80ffc90 2023-12-01T01:17:38.190 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.187+0000 7ff3ac941700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a819e810 con 0x7ff3a8101820 2023-12-01T01:17:38.190 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.187+0000 7ff3a5d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3a8100e80 0x7ff3a819b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:38.190 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.187+0000 7ff3a5d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3a8100e80 0x7ff3a819b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58680/0 (socket says 172.21.15.77:58680) 2023-12-01T01:17:38.190 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.187+0000 7ff3a5d9b700 1 -- 172.21.15.77:0/2249766524 learned_addr learned my addr 172.21.15.77:0/2249766524 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:38.190 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.187+0000 7ff3a659c700 1 --2- 172.21.15.77:0/2249766524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3a80ffc90 0x7ff3a8069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:38.190 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.187+0000 7ff3a6d9d700 1 --2- 172.21.15.77:0/2249766524 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3a8101820 0x7ff3a819dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:38.190 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.187+0000 7ff3a5d9b700 1 -- 172.21.15.77:0/2249766524 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3a8101820 msgr2=0x7ff3a819dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:38.191 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.187+0000 7ff3a5d9b700 1 --2- 172.21.15.77:0/2249766524 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3a8101820 0x7ff3a819dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:38.191 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.187+0000 7ff3a5d9b700 1 -- 172.21.15.77:0/2249766524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3a80ffc90 msgr2=0x7ff3a8069ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:38.191 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.187+0000 7ff3a5d9b700 1 --2- 172.21.15.77:0/2249766524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3a80ffc90 0x7ff3a8069ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:38.191 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.187+0000 7ff3a5d9b700 1 -- 172.21.15.77:0/2249766524 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3980095f0 con 0x7ff3a8100e80 2023-12-01T01:17:38.191 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.187+0000 7ff3a6d9d700 1 --2- 172.21.15.77:0/2249766524 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3a8101820 0x7ff3a819dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:17:38.191 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.187+0000 7ff3a659c700 1 --2- 172.21.15.77:0/2249766524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3a80ffc90 0x7ff3a8069ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:17:38.191 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.187+0000 7ff3a5d9b700 1 --2- 172.21.15.77:0/2249766524 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3a8100e80 0x7ff3a819b6b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff398009990 tx=0x7ff398005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:38.191 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.187+0000 7ff39f7fe700 1 -- 172.21.15.77:0/2249766524 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39801d070 con 0x7ff3a8100e80 2023-12-01T01:17:38.191 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.188+0000 7ff39f7fe700 1 -- 172.21.15.77:0/2249766524 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff398022470 con 0x7ff3a8100e80 2023-12-01T01:17:38.191 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.188+0000 7ff3ac941700 1 -- 172.21.15.77:0/2249766524 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3a819ea90 con 0x7ff3a8100e80 2023-12-01T01:17:38.191 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.188+0000 7ff39f7fe700 1 -- 172.21.15.77:0/2249766524 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff398004230 con 0x7ff3a8100e80 2023-12-01T01:17:38.191 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.188+0000 7ff3ac941700 1 -- 172.21.15.77:0/2249766524 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3a819ef40 con 0x7ff3a8100e80 2023-12-01T01:17:38.192 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.188+0000 7ff3ac941700 1 -- 172.21.15.77:0/2249766524 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3a81071e0 con 0x7ff3a8100e80 2023-12-01T01:17:38.192 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.189+0000 7ff39f7fe700 1 -- 172.21.15.77:0/2249766524 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7ff3980225e0 con 0x7ff3a8100e80 2023-12-01T01:17:38.192 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.190+0000 7ff39f7fe700 1 --2- 172.21.15.77:0/2249766524 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff3940a05f0 0x7ff3940a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:38.192 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.190+0000 7ff3a659c700 1 --2- 172.21.15.77:0/2249766524 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff3940a05f0 0x7ff3940a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:38.192 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.190+0000 7ff39f7fe700 1 -- 172.21.15.77:0/2249766524 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7ff3980ccc80 con 0x7ff3a8100e80 2023-12-01T01:17:38.192 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.190+0000 7ff3a659c700 1 --2- 172.21.15.77:0/2249766524 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff3940a05f0 0x7ff3940a2ab0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff3a8100ce0 tx=0x7ff39000b670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:38.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.194+0000 7ff39f7fe700 1 -- 172.21.15.77:0/2249766524 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff398096970 con 0x7ff3a8100e80 2023-12-01T01:17:38.390 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.389+0000 7ff3ac941700 1 -- 172.21.15.77:0/2249766524 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3a802cec0 con 0x7ff3a8100e80 2023-12-01T01:17:38.392 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.391+0000 7ff39f7fe700 1 -- 172.21.15.77:0/2249766524 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22472 (secure 0 0 0) 0x7ff398027090 con 0x7ff3a8100e80 2023-12-01T01:17:38.392 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:17:38.392 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":84,"fsid":"2a06e29a-8fe6-11ee-95a2-87774f69a715","created":"2023-12-01T01:09:30.210398+0000","modified":"2023-12-01T01:17:25.609766+0000","last_up_change":"2023-12-01T01:15:44.586092+0000","last_in_change":"2023-12-01T01:15:26.305439+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:12:30.514229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:16:23.568860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-01T01:16:24.531137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fc651e62-a103-4ab8-a76d-d3708ea6aeec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6803","nonce":2244799094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6805","nonce":2244799094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6809","nonce":2244799094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6807","nonce":2244799094}]},"public_addr":"172.21.15.77:6803/2244799094","cluster_addr":"172.21.15.77:6805/2244799094","heartbeat_back_addr":"172.21.15.77:6809/2244799094","heartbeat_front_addr":"172.21.15.77:6807/2244799094","state":["exists","up"]},{"osd":1,"uuid":"0adf8c54-49ec-49a0-8bf1-af1f20e60729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6801","nonce":224194890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6803","nonce":224194890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6807","nonce":224194890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6805","nonce":224194890}]},"public_addr":"172.21.15.137:6801/224194890","cluster_addr":"172.21.15.137:6803/224194890","heartbeat_back_addr":"172.21.15.137:6807/224194890","heartbeat_front_addr":"172.21.15.137:6805/224194890","state":["exists","up"]},{"osd":2,"uuid":"5a69802c-dede-4544-9a12-436f5c1c0038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6801","nonce":4078459110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6803","nonce":4078459110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6807","nonce":4078459110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6805","nonce":4078459110}]},"public_addr":"172.21.15.203:6801/4078459110","cluster_addr":"172.21.15.203:6803/4078459110","heartbeat_back_addr":"172.21.15.203:6807/4078459110","heartbeat_front_addr":"172.21.15.203:6805/4078459110","state":["exists","up"]},{"osd":3,"uuid":"ad325a97-580b-4c99-bf12-cbb0d92a6d0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6811","nonce":1454440477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6813","nonce":1454440477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6817","nonce":1454440477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6815","nonce":1454440477}]},"public_addr":"172.21.15.77:6811/1454440477","cluster_addr":"172.21.15.77:6813/1454440477","heartbeat_back_addr":"172.21.15.77:6817/1454440477","heartbeat_front_addr":"172.21.15.77:6815/1454440477","state":["exists","up"]},{"osd":4,"uuid":"1e3ac1ba-98bb-4b8b-8e26-50064c66bf2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6809","nonce":1103241451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6811","nonce":1103241451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6815","nonce":1103241451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6813","nonce":1103241451}]},"public_addr":"172.21.15.137:6809/1103241451","cluster_addr":"172.21.15.137:6811/1103241451","heartbeat_back_addr":"172.21.15.137:6815/1103241451","heartbeat_front_addr":"172.21.15.137:6813/1103241451","state":["exists","up"]},{"osd":5,"uuid":"eb32c016-abeb-44ab-abf0-19dd09235736","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6809","nonce":2648569482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6811","nonce":2648569482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6815","nonce":2648569482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6813","nonce":2648569482}]},"public_addr":"172.21.15.203:6809/2648569482","cluster_addr":"172.21.15.203:6811/2648569482","heartbeat_back_addr":"172.21.15.203:6815/2648569482","heartbeat_front_addr":"172.21.15.203:6813/2648569482","state":["exists","up"]},{"osd":6,"uuid":"0e3fa97e-2ca6-418c-9e8d-a6fe3c56adb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6819","nonce":2463414651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6821","nonce":2463414651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6825","nonce":2463414651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6823","nonce":2463414651}]},"public_addr":"172.21.15.77:6819/2463414651","cluster_addr":"172.21.15.77:6821/2463414651","heartbeat_back_addr":"172.21.15.77:6825/2463414651","heartbeat_front_addr":"172.21.15.77:6823/2463414651","state":["exists","up"]},{"osd":7,"uuid":"d7398c01-0f98-4138-8a49-c3ff0aae7e88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6817","nonce":3108330766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6819","nonce":3108330766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6823","nonce":3108330766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6821","nonce":3108330766}]},"public_addr":"172.21.15.137:6817/3108330766","cluster_addr":"172.21.15.137:6819/3108330766","heartbeat_back_addr":"172.21.15.137:6823/3108330766","heartbeat_front_addr":"172.21.15.137:6821/3108330766","state":["exists","up"]},{"osd":8,"uuid":"0c09050d-3e43-410c-8cd7-bcb891af56b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6817","nonce":1161158776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6819","nonce":1161158776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6823","nonce":1161158776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6821","nonce":1161158776}]},"public_addr":"172.21.15.203:6817/1161158776","cluster_addr":"172.21.15.203:6819/1161158776","heartbeat_back_addr":"172.21.15.203:6823/1161158776","heartbeat_front_addr":"172.21.15.203:6821/1161158776","state":["exists","up"]},{"osd":9,"uuid":"527ecb49-95ec-48bd-9872-3b07fae5c5ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6827","nonce":2409880685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6829","nonce":2409880685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6833","nonce":2409880685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6831","nonce":2409880685}]},"public_addr":"172.21.15.77:6827/2409880685","cluster_addr":"172.21.15.77:6829/2409880685","heartbeat_back_addr":"172.21.15.77:6833/2409880685","heartbeat_front_addr":"172.21.15.77:6831/2409880685","state":["exists","up"]},{"osd":10,"uuid":"1680bb3e-a3c8-427b-af68-e2214f32bed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6825","nonce":374920146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6827","nonce":374920146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6831","nonce":374920146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6829","nonce":374920146}]},"public_addr":"172.21.15.137:6825/374920146","cluster_addr":"172.21.15.137:6827/374920146","heartbeat_back_addr":"172.21.15.137:6831/374920146","heartbeat_front_addr":"172.21.15.137:6829/374920146","state":["exists","up"]},{"osd":11,"uuid":"8a533466-023d-4803-ae36-9a9f1715e49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6825","nonce":2161588962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6827","nonce":2161588962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6831","nonce":2161588962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6829","nonce":2161588962}]},"public_addr":"172.21.15.203:6825/2161588962","cluster_addr":"172.21.15.203:6827/2161588962","heartbeat_back_addr":"172.21.15.203:6831/2161588962","heartbeat_front_addr":"172.21.15.203:6829/2161588962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:11:45.420509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:06.210115+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:28.191307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:49.594116+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:11.261689+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:33.376541+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:54.968336+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:16.775310+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:38.733792+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:59.456699+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:15:20.924059+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:15:43.094773+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/2260035254":"2023-12-02T01:10:24.191835+0000","172.21.15.77:6800/1456621747":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/219838828":"2023-12-02T01:10:24.191835+0000","172.21.15.77:6801/1456621747":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/2937149266":"2023-12-02T01:10:02.232948+0000","172.21.15.77:0/807520018":"2023-12-02T01:10:02.232948+0000","172.21.15.77:6801/1661788026":"2023-12-02T01:10:02.232948+0000","172.21.15.77:6800/1661788026":"2023-12-02T01:10:02.232948+0000","172.21.15.77:0/3551282473":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/1202222558":"2023-12-02T01:10:02.232948+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:17:38.394 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.393+0000 7ff3ac941700 1 -- 172.21.15.77:0/2249766524 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff3940a05f0 msgr2=0x7ff3940a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:38.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.393+0000 7ff3ac941700 1 --2- 172.21.15.77:0/2249766524 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff3940a05f0 0x7ff3940a2ab0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff3a8100ce0 tx=0x7ff39000b670 comp rx=0 tx=0).stop 2023-12-01T01:17:38.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.393+0000 7ff3ac941700 1 -- 172.21.15.77:0/2249766524 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3a8100e80 msgr2=0x7ff3a819b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:38.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.393+0000 7ff3ac941700 1 --2- 172.21.15.77:0/2249766524 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3a8100e80 0x7ff3a819b6b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff398009990 tx=0x7ff398005e80 comp rx=0 tx=0).stop 2023-12-01T01:17:38.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.393+0000 7ff3ac941700 1 -- 172.21.15.77:0/2249766524 shutdown_connections 2023-12-01T01:17:38.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.393+0000 7ff3ac941700 1 --2- 172.21.15.77:0/2249766524 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff3940a05f0 0x7ff3940a2ab0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:38.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.393+0000 7ff3ac941700 1 --2- 172.21.15.77:0/2249766524 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3a8101820 0x7ff3a819dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:38.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.393+0000 7ff3ac941700 1 --2- 172.21.15.77:0/2249766524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3a80ffc90 0x7ff3a8069ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:38.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.393+0000 7ff3ac941700 1 --2- 172.21.15.77:0/2249766524 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3a8100e80 0x7ff3a819b6b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:38.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.393+0000 7ff3ac941700 1 -- 172.21.15.77:0/2249766524 >> 172.21.15.77:0/2249766524 conn(0x7ff3a80fb1c0 msgr2=0x7ff3a8104320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:38.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.393+0000 7ff3ac941700 1 -- 172.21.15.77:0/2249766524 shutdown_connections 2023-12-01T01:17:38.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.393+0000 7ff3ac941700 1 -- 172.21.15.77:0/2249766524 wait complete. 2023-12-01T01:17:38.407 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:12:30.514229+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T01:16:23.568860+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T01:16:24.531137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:17:38.407 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:17:38.533 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.530+0000 7fc1c6374700 1 -- 172.21.15.77:0/679150361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1c00ff030 msgr2=0x7fc1c00ff450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:38.533 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.530+0000 7fc1c6374700 1 --2- 172.21.15.77:0/679150361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1c00ff030 0x7fc1c00ff450 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4009990 tx=0x7fc1b4009ca0 comp rx=0 tx=0).stop 2023-12-01T01:17:38.533 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.530+0000 7fc1c6374700 1 -- 172.21.15.77:0/679150361 shutdown_connections 2023-12-01T01:17:38.533 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.530+0000 7fc1c6374700 1 --2- 172.21.15.77:0/679150361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1c01029a0 0x7fc1c0109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:38.533 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.530+0000 7fc1c6374700 1 --2- 172.21.15.77:0/679150361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1c00ff030 0x7fc1c00ff450 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:38.533 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.530+0000 7fc1c6374700 1 --2- 172.21.15.77:0/679150361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1c0102000 0x7fc1c0102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:38.533 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.530+0000 7fc1c6374700 1 -- 172.21.15.77:0/679150361 >> 172.21.15.77:0/679150361 conn(0x7fc1c00fa960 msgr2=0x7fc1c00fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:38.534 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.530+0000 7fc1c6374700 1 -- 172.21.15.77:0/679150361 shutdown_connections 2023-12-01T01:17:38.534 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.530+0000 7fc1c6374700 1 -- 172.21.15.77:0/679150361 wait complete. 2023-12-01T01:17:38.534 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.530+0000 7fc1c6374700 1 Processor -- start 2023-12-01T01:17:38.534 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.530+0000 7fc1c6374700 1 -- start start 2023-12-01T01:17:38.534 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.531+0000 7fc1c6374700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1c00ff030 0x7fc1c0101970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:38.534 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.531+0000 7fc1c6374700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1c0102000 0x7fc1c00fffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:38.534 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.531+0000 7fc1c6374700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1c01029a0 0x7fc1c01004f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:38.534 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.531+0000 7fc1c6374700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c0100c90 con 0x7fc1c00ff030 2023-12-01T01:17:38.534 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.531+0000 7fc1c6374700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c0100e00 con 0x7fc1c0102000 2023-12-01T01:17:38.534 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.531+0000 7fc1c6374700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c01a0b00 con 0x7fc1c01029a0 2023-12-01T01:17:38.534 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.531+0000 7fc1bffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1c00ff030 0x7fc1c0101970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:38.534 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.531+0000 7fc1bffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1c00ff030 0x7fc1c0101970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58692/0 (socket says 172.21.15.77:58692) 2023-12-01T01:17:38.534 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.531+0000 7fc1bffff700 1 -- 172.21.15.77:0/17605715 learned_addr learned my addr 172.21.15.77:0/17605715 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:38.535 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.531+0000 7fc1c4911700 1 --2- 172.21.15.77:0/17605715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1c01029a0 0x7fc1c01004f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:38.535 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.531+0000 7fc1bf7fe700 1 --2- 172.21.15.77:0/17605715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1c0102000 0x7fc1c00fffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:38.535 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.531+0000 7fc1bffff700 1 -- 172.21.15.77:0/17605715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1c01029a0 msgr2=0x7fc1c01004f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:38.535 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.531+0000 7fc1bffff700 1 --2- 172.21.15.77:0/17605715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1c01029a0 0x7fc1c01004f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:38.535 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.531+0000 7fc1bffff700 1 -- 172.21.15.77:0/17605715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1c0102000 msgr2=0x7fc1c00fffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:38.535 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.531+0000 7fc1bffff700 1 --2- 172.21.15.77:0/17605715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1c0102000 0x7fc1c00fffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:38.535 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.531+0000 7fc1bffff700 1 -- 172.21.15.77:0/17605715 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b40095f0 con 0x7fc1c00ff030 2023-12-01T01:17:38.535 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.531+0000 7fc1bffff700 1 --2- 172.21.15.77:0/17605715 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1c00ff030 0x7fc1c0101970 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4000c00 tx=0x7fc1b400f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:38.535 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.531+0000 7fc1bd7fa700 1 -- 172.21.15.77:0/17605715 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b401d070 con 0x7fc1c00ff030 2023-12-01T01:17:38.535 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.531+0000 7fc1bd7fa700 1 -- 172.21.15.77:0/17605715 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1b4005050 con 0x7fc1c00ff030 2023-12-01T01:17:38.535 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.531+0000 7fc1c6374700 1 -- 172.21.15.77:0/17605715 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1c01a0d80 con 0x7fc1c00ff030 2023-12-01T01:17:38.535 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.531+0000 7fc1bd7fa700 1 -- 172.21.15.77:0/17605715 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b4004ac0 con 0x7fc1c00ff030 2023-12-01T01:17:38.535 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.531+0000 7fc1c6374700 1 -- 172.21.15.77:0/17605715 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c01a11a0 con 0x7fc1c00ff030 2023-12-01T01:17:38.536 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.532+0000 7fc1c6374700 1 -- 172.21.15.77:0/17605715 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1c0107d90 con 0x7fc1c00ff030 2023-12-01T01:17:38.537 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.533+0000 7fc1bd7fa700 1 -- 172.21.15.77:0/17605715 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fc1b40051c0 con 0x7fc1c00ff030 2023-12-01T01:17:38.537 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.534+0000 7fc1bd7fa700 1 --2- 172.21.15.77:0/17605715 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc1a80a05b0 0x7fc1a80a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:38.537 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.534+0000 7fc1bf7fe700 1 --2- 172.21.15.77:0/17605715 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc1a80a05b0 0x7fc1a80a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:38.537 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.534+0000 7fc1bd7fa700 1 -- 172.21.15.77:0/17605715 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fc1b40ccdd0 con 0x7fc1c00ff030 2023-12-01T01:17:38.537 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.534+0000 7fc1bf7fe700 1 --2- 172.21.15.77:0/17605715 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc1a80a05b0 0x7fc1a80a2a70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc1b000ac50 tx=0x7fc1b000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:38.538 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.537+0000 7fc1bd7fa700 1 -- 172.21.15.77:0/17605715 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc1b4096ac0 con 0x7fc1c00ff030 2023-12-01T01:17:38.732 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.730+0000 7fc1c6374700 1 -- 172.21.15.77:0/17605715 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc1c010c630 con 0x7fc1c00ff030 2023-12-01T01:17:38.732 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.731+0000 7fc1bd7fa700 1 -- 172.21.15.77:0/17605715 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc1b40260a0 con 0x7fc1c00ff030 2023-12-01T01:17:38.733 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 1 2023-12-01T01:17:38.734 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.733+0000 7fc1c6374700 1 -- 172.21.15.77:0/17605715 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc1a80a05b0 msgr2=0x7fc1a80a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:38.735 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.733+0000 7fc1c6374700 1 --2- 172.21.15.77:0/17605715 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc1a80a05b0 0x7fc1a80a2a70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc1b000ac50 tx=0x7fc1b000a380 comp rx=0 tx=0).stop 2023-12-01T01:17:38.735 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.733+0000 7fc1c6374700 1 -- 172.21.15.77:0/17605715 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1c00ff030 msgr2=0x7fc1c0101970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:38.735 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.733+0000 7fc1c6374700 1 --2- 172.21.15.77:0/17605715 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1c00ff030 0x7fc1c0101970 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4000c00 tx=0x7fc1b400f670 comp rx=0 tx=0).stop 2023-12-01T01:17:38.735 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.733+0000 7fc1c6374700 1 -- 172.21.15.77:0/17605715 shutdown_connections 2023-12-01T01:17:38.735 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.733+0000 7fc1c6374700 1 --2- 172.21.15.77:0/17605715 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc1a80a05b0 0x7fc1a80a2a70 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:38.736 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.733+0000 7fc1c6374700 1 --2- 172.21.15.77:0/17605715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1c01029a0 0x7fc1c01004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:38.736 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.733+0000 7fc1c6374700 1 --2- 172.21.15.77:0/17605715 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1c00ff030 0x7fc1c0101970 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:38.737 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.733+0000 7fc1c6374700 1 --2- 172.21.15.77:0/17605715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1c0102000 0x7fc1c00fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:38.737 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.733+0000 7fc1c6374700 1 -- 172.21.15.77:0/17605715 >> 172.21.15.77:0/17605715 conn(0x7fc1c00fa960 msgr2=0x7fc1c00fcc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:38.737 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.733+0000 7fc1c6374700 1 -- 172.21.15.77:0/17605715 shutdown_connections 2023-12-01T01:17:38.737 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.733+0000 7fc1c6374700 1 -- 172.21.15.77:0/17605715 wait complete. 2023-12-01T01:17:38.745 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:38 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/2899022456' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-01T01:17:38.745 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:38 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/1528911082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:17:38.745 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:38 smithi077 ceph-mon[98867]: pgmap v291: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s; 14 B/s, 0 keys/s, 0 objects/s recovering 2023-12-01T01:17:38.745 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:38 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/2249766524' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:17:38.747 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T01:17:38.871 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.869+0000 7fae34a5a700 1 -- 172.21.15.77:0/3869128143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae30102920 msgr2=0x7fae301091d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:38.871 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.869+0000 7fae34a5a700 1 --2- 172.21.15.77:0/3869128143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae30102920 0x7fae301091d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fae2800b0d0 tx=0x7fae2800b3e0 comp rx=0 tx=0).stop 2023-12-01T01:17:38.871 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.869+0000 7fae34a5a700 1 -- 172.21.15.77:0/3869128143 shutdown_connections 2023-12-01T01:17:38.871 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.869+0000 7fae34a5a700 1 --2- 172.21.15.77:0/3869128143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae30102920 0x7fae301091d0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:38.871 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.869+0000 7fae34a5a700 1 --2- 172.21.15.77:0/3869128143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae30100d90 0x7fae301011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:38.871 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.869+0000 7fae34a5a700 1 --2- 172.21.15.77:0/3869128143 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae30101f80 0x7fae301023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:38.872 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.869+0000 7fae34a5a700 1 -- 172.21.15.77:0/3869128143 >> 172.21.15.77:0/3869128143 conn(0x7fae300fc2e0 msgr2=0x7fae300fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:38.872 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.869+0000 7fae34a5a700 1 -- 172.21.15.77:0/3869128143 shutdown_connections 2023-12-01T01:17:38.872 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.869+0000 7fae34a5a700 1 -- 172.21.15.77:0/3869128143 wait complete. 2023-12-01T01:17:38.872 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.870+0000 7fae34a5a700 1 Processor -- start 2023-12-01T01:17:38.872 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.870+0000 7fae34a5a700 1 -- start start 2023-12-01T01:17:38.875 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.870+0000 7fae34a5a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae30100d90 0x7fae30196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:38.875 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.870+0000 7fae34a5a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae30101f80 0x7fae30197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:38.875 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.870+0000 7fae34a5a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae30102920 0x7fae3019e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:38.875 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.870+0000 7fae34a5a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3019e6b0 con 0x7fae30102920 2023-12-01T01:17:38.875 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.870+0000 7fae34a5a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3019e7f0 con 0x7fae30101f80 2023-12-01T01:17:38.875 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.870+0000 7fae34a5a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3019e980 con 0x7fae30100d90 2023-12-01T01:17:38.875 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.870+0000 7fae2ed9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae30102920 0x7fae3019e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:38.875 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.870+0000 7fae2ed9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae30102920 0x7fae3019e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58710/0 (socket says 172.21.15.77:58710) 2023-12-01T01:17:38.875 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.870+0000 7fae2ed9d700 1 -- 172.21.15.77:0/1800814565 learned_addr learned my addr 172.21.15.77:0/1800814565 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:38.875 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.870+0000 7fae2e59c700 1 --2- 172.21.15.77:0/1800814565 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae30100d90 0x7fae30196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:38.876 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.870+0000 7fae2dd9b700 1 --2- 172.21.15.77:0/1800814565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae30101f80 0x7fae30197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:38.876 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.870+0000 7fae2ed9d700 1 -- 172.21.15.77:0/1800814565 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae30100d90 msgr2=0x7fae30196b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:38.876 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.870+0000 7fae2ed9d700 1 --2- 172.21.15.77:0/1800814565 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae30100d90 0x7fae30196b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:38.876 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.870+0000 7fae2ed9d700 1 -- 172.21.15.77:0/1800814565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae30101f80 msgr2=0x7fae30197070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:38.876 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.870+0000 7fae2ed9d700 1 --2- 172.21.15.77:0/1800814565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae30101f80 0x7fae30197070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:38.876 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.870+0000 7fae2ed9d700 1 -- 172.21.15.77:0/1800814565 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae28009d00 con 0x7fae30102920 2023-12-01T01:17:38.876 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.871+0000 7fae2ed9d700 1 --2- 172.21.15.77:0/1800814565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae30102920 0x7fae3019e0e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fae28007a00 tx=0x7fae280094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:38.876 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.871+0000 7fae1f7fe700 1 -- 172.21.15.77:0/1800814565 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2800e040 con 0x7fae30102920 2023-12-01T01:17:38.876 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.871+0000 7fae1f7fe700 1 -- 172.21.15.77:0/1800814565 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae28003e10 con 0x7fae30102920 2023-12-01T01:17:38.876 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.871+0000 7fae34a5a700 1 -- 172.21.15.77:0/1800814565 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae3019ec00 con 0x7fae30102920 2023-12-01T01:17:38.876 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.871+0000 7fae1f7fe700 1 -- 172.21.15.77:0/1800814565 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2801b9e0 con 0x7fae30102920 2023-12-01T01:17:38.876 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.871+0000 7fae34a5a700 1 -- 172.21.15.77:0/1800814565 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae3019f020 con 0x7fae30102920 2023-12-01T01:17:38.876 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.872+0000 7fae34a5a700 1 -- 172.21.15.77:0/1800814565 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae3010c620 con 0x7fae30102920 2023-12-01T01:17:38.876 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.873+0000 7fae1f7fe700 1 -- 172.21.15.77:0/1800814565 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fae28005630 con 0x7fae30102920 2023-12-01T01:17:38.876 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.873+0000 7fae1f7fe700 1 --2- 172.21.15.77:0/1800814565 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fae180a05b0 0x7fae180a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:38.877 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.873+0000 7fae2e59c700 1 --2- 172.21.15.77:0/1800814565 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fae180a05b0 0x7fae180a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:38.877 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.873+0000 7fae1f7fe700 1 -- 172.21.15.77:0/1800814565 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fae280cd810 con 0x7fae30102920 2023-12-01T01:17:38.877 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.874+0000 7fae2e59c700 1 --2- 172.21.15.77:0/1800814565 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fae180a05b0 0x7fae180a2a70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fae30101de0 tx=0x7fae24007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:38.877 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:38.877+0000 7fae1f7fe700 1 -- 172.21.15.77:0/1800814565 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fae280973e0 con 0x7fae30102920 2023-12-01T01:17:39.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:38 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/2899022456' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-01T01:17:39.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:38 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/1528911082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:17:39.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:38 smithi203 ceph-mon[103581]: pgmap v291: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s; 14 B/s, 0 keys/s, 0 objects/s recovering 2023-12-01T01:17:39.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:38 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/2249766524' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:17:39.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:38 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/2899022456' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-01T01:17:39.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:38 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/1528911082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:17:39.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:38 smithi137 ceph-mon[103312]: pgmap v291: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s; 14 B/s, 0 keys/s, 0 objects/s recovering 2023-12-01T01:17:39.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:38 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/2249766524' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:17:39.068 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.067+0000 7fae34a5a700 1 -- 172.21.15.77:0/1800814565 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fae30107c40 con 0x7fae30102920 2023-12-01T01:17:39.068 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.067+0000 7fae1f7fe700 1 -- 172.21.15.77:0/1800814565 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7fae2809aa00 con 0x7fae30102920 2023-12-01T01:17:39.069 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 32 2023-12-01T01:17:39.071 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.069+0000 7fae34a5a700 1 -- 172.21.15.77:0/1800814565 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fae180a05b0 msgr2=0x7fae180a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:39.071 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.069+0000 7fae34a5a700 1 --2- 172.21.15.77:0/1800814565 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fae180a05b0 0x7fae180a2a70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fae30101de0 tx=0x7fae24007400 comp rx=0 tx=0).stop 2023-12-01T01:17:39.071 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.069+0000 7fae34a5a700 1 -- 172.21.15.77:0/1800814565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae30102920 msgr2=0x7fae3019e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:39.071 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.069+0000 7fae34a5a700 1 --2- 172.21.15.77:0/1800814565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae30102920 0x7fae3019e0e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fae28007a00 tx=0x7fae280094c0 comp rx=0 tx=0).stop 2023-12-01T01:17:39.071 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.069+0000 7fae34a5a700 1 -- 172.21.15.77:0/1800814565 shutdown_connections 2023-12-01T01:17:39.072 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.069+0000 7fae34a5a700 1 --2- 172.21.15.77:0/1800814565 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fae180a05b0 0x7fae180a2a70 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:39.072 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.069+0000 7fae34a5a700 1 --2- 172.21.15.77:0/1800814565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae30102920 0x7fae3019e0e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:39.072 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.069+0000 7fae34a5a700 1 --2- 172.21.15.77:0/1800814565 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae30100d90 0x7fae30196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:39.072 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.069+0000 7fae34a5a700 1 --2- 172.21.15.77:0/1800814565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae30101f80 0x7fae30197070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:39.072 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.069+0000 7fae34a5a700 1 -- 172.21.15.77:0/1800814565 >> 172.21.15.77:0/1800814565 conn(0x7fae300fc2e0 msgr2=0x7fae300fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:39.072 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.069+0000 7fae34a5a700 1 -- 172.21.15.77:0/1800814565 shutdown_connections 2023-12-01T01:17:39.072 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.070+0000 7fae34a5a700 1 -- 172.21.15.77:0/1800814565 wait complete. 2023-12-01T01:17:39.083 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T01:17:39.210 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.206+0000 7ff0bbef8700 1 -- 172.21.15.77:0/4199556821 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0b4101850 msgr2=0x7ff0b4108100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:39.211 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.206+0000 7ff0bbef8700 1 --2- 172.21.15.77:0/4199556821 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0b4101850 0x7ff0b4108100 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac00b0d0 tx=0x7ff0ac00b3e0 comp rx=0 tx=0).stop 2023-12-01T01:17:39.211 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.206+0000 7ff0bbef8700 1 -- 172.21.15.77:0/4199556821 shutdown_connections 2023-12-01T01:17:39.211 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.206+0000 7ff0bbef8700 1 --2- 172.21.15.77:0/4199556821 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0b4101850 0x7ff0b4108100 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:39.211 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.206+0000 7ff0bbef8700 1 --2- 172.21.15.77:0/4199556821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0b40ffcc0 0x7ff0b41000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:39.211 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.206+0000 7ff0bbef8700 1 --2- 172.21.15.77:0/4199556821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0b4100eb0 0x7ff0b4101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:39.211 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.206+0000 7ff0bbef8700 1 -- 172.21.15.77:0/4199556821 >> 172.21.15.77:0/4199556821 conn(0x7ff0b40fb210 msgr2=0x7ff0b40fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:39.212 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.206+0000 7ff0bbef8700 1 -- 172.21.15.77:0/4199556821 shutdown_connections 2023-12-01T01:17:39.212 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.206+0000 7ff0bbef8700 1 -- 172.21.15.77:0/4199556821 wait complete. 2023-12-01T01:17:39.212 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.207+0000 7ff0bbef8700 1 Processor -- start 2023-12-01T01:17:39.212 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.207+0000 7ff0bbef8700 1 -- start start 2023-12-01T01:17:39.212 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.207+0000 7ff0bbef8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0b40ffcc0 0x7ff0b4195ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:39.212 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.207+0000 7ff0bbef8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0b4100eb0 0x7ff0b4196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:39.212 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.207+0000 7ff0bbef8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0b4101850 0x7ff0b419d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:39.213 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.207+0000 7ff0bbef8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b419d660 con 0x7ff0b4100eb0 2023-12-01T01:17:39.213 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.207+0000 7ff0bbef8700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b419d7a0 con 0x7ff0b40ffcc0 2023-12-01T01:17:39.213 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.207+0000 7ff0bbef8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b419d930 con 0x7ff0b4101850 2023-12-01T01:17:39.213 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.207+0000 7ff0b9493700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0b4100eb0 0x7ff0b4196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:39.213 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.207+0000 7ff0b9493700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0b4100eb0 0x7ff0b4196020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58722/0 (socket says 172.21.15.77:58722) 2023-12-01T01:17:39.213 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.207+0000 7ff0b9493700 1 -- 172.21.15.77:0/1590835743 learned_addr learned my addr 172.21.15.77:0/1590835743 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:39.213 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.207+0000 7ff0b9c94700 1 --2- 172.21.15.77:0/1590835743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0b40ffcc0 0x7ff0b4195ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:39.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.207+0000 7ff0b9493700 1 -- 172.21.15.77:0/1590835743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0b4101850 msgr2=0x7ff0b419d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:39.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.207+0000 7ff0ba495700 1 --2- 172.21.15.77:0/1590835743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0b4101850 0x7ff0b419d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:39.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.207+0000 7ff0b9493700 1 --2- 172.21.15.77:0/1590835743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0b4101850 0x7ff0b419d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:39.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.207+0000 7ff0b9493700 1 -- 172.21.15.77:0/1590835743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0b40ffcc0 msgr2=0x7ff0b4195ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:39.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.207+0000 7ff0b9493700 1 --2- 172.21.15.77:0/1590835743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0b40ffcc0 0x7ff0b4195ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:39.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.207+0000 7ff0b9493700 1 -- 172.21.15.77:0/1590835743 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ac009d00 con 0x7ff0b4100eb0 2023-12-01T01:17:39.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.207+0000 7ff0b9493700 1 --2- 172.21.15.77:0/1590835743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0b4100eb0 0x7ff0b4196020 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff0b000e9a0 tx=0x7ff0b000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:39.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.207+0000 7ff0a6ffd700 1 -- 172.21.15.77:0/1590835743 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b0009790 con 0x7ff0b4100eb0 2023-12-01T01:17:39.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.208+0000 7ff0a6ffd700 1 -- 172.21.15.77:0/1590835743 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0b000cbd0 con 0x7ff0b4100eb0 2023-12-01T01:17:39.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.208+0000 7ff0a6ffd700 1 -- 172.21.15.77:0/1590835743 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b0018680 con 0x7ff0b4100eb0 2023-12-01T01:17:39.215 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.208+0000 7ff0bbef8700 1 -- 172.21.15.77:0/1590835743 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b419dc10 con 0x7ff0b4100eb0 2023-12-01T01:17:39.215 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.208+0000 7ff0bbef8700 1 -- 172.21.15.77:0/1590835743 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b419e120 con 0x7ff0b4100eb0 2023-12-01T01:17:39.215 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.209+0000 7ff0bbef8700 1 -- 172.21.15.77:0/1590835743 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0b410b530 con 0x7ff0b4100eb0 2023-12-01T01:17:39.216 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.210+0000 7ff0a6ffd700 1 -- 172.21.15.77:0/1590835743 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7ff0b000cd40 con 0x7ff0b4100eb0 2023-12-01T01:17:39.216 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.210+0000 7ff0a6ffd700 1 --2- 172.21.15.77:0/1590835743 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff0a00a0630 0x7ff0a00a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:39.216 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.211+0000 7ff0b9c94700 1 --2- 172.21.15.77:0/1590835743 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff0a00a0630 0x7ff0a00a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:39.216 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.211+0000 7ff0a6ffd700 1 -- 172.21.15.77:0/1590835743 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7ff0b0014070 con 0x7ff0b4100eb0 2023-12-01T01:17:39.216 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.211+0000 7ff0b9c94700 1 --2- 172.21.15.77:0/1590835743 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff0a00a0630 0x7ff0a00a2af0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4100d10 tx=0x7ff0a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:39.216 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.214+0000 7ff0a6ffd700 1 -- 172.21.15.77:0/1590835743 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff0b0095a70 con 0x7ff0b4100eb0 2023-12-01T01:17:39.407 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.405+0000 7ff0bbef8700 1 -- 172.21.15.77:0/1590835743 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff0b4106ba0 con 0x7ff0b4100eb0 2023-12-01T01:17:39.407 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.406+0000 7ff0a6ffd700 1 -- 172.21.15.77:0/1590835743 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff0b0026070 con 0x7ff0b4100eb0 2023-12-01T01:17:39.407 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 64 2023-12-01T01:17:39.409 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.408+0000 7ff0bbef8700 1 -- 172.21.15.77:0/1590835743 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff0a00a0630 msgr2=0x7ff0a00a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:39.410 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.408+0000 7ff0bbef8700 1 --2- 172.21.15.77:0/1590835743 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff0a00a0630 0x7ff0a00a2af0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4100d10 tx=0x7ff0a8009260 comp rx=0 tx=0).stop 2023-12-01T01:17:39.410 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.408+0000 7ff0bbef8700 1 -- 172.21.15.77:0/1590835743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0b4100eb0 msgr2=0x7ff0b4196020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:39.410 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.408+0000 7ff0bbef8700 1 --2- 172.21.15.77:0/1590835743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0b4100eb0 0x7ff0b4196020 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff0b000e9a0 tx=0x7ff0b000ecb0 comp rx=0 tx=0).stop 2023-12-01T01:17:39.410 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.408+0000 7ff0bbef8700 1 -- 172.21.15.77:0/1590835743 shutdown_connections 2023-12-01T01:17:39.410 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.408+0000 7ff0bbef8700 1 --2- 172.21.15.77:0/1590835743 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff0a00a0630 0x7ff0a00a2af0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:39.410 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.408+0000 7ff0bbef8700 1 --2- 172.21.15.77:0/1590835743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0b4101850 0x7ff0b419d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:39.410 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.408+0000 7ff0bbef8700 1 --2- 172.21.15.77:0/1590835743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0b40ffcc0 0x7ff0b4195ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:39.410 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.408+0000 7ff0bbef8700 1 --2- 172.21.15.77:0/1590835743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0b4100eb0 0x7ff0b4196020 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:39.410 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.408+0000 7ff0bbef8700 1 -- 172.21.15.77:0/1590835743 >> 172.21.15.77:0/1590835743 conn(0x7ff0b40fb210 msgr2=0x7ff0b40fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:39.410 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.408+0000 7ff0bbef8700 1 -- 172.21.15.77:0/1590835743 shutdown_connections 2023-12-01T01:17:39.411 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.408+0000 7ff0bbef8700 1 -- 172.21.15.77:0/1590835743 wait complete. 2023-12-01T01:17:39.422 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:17:39.545 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.542+0000 7fc54047b700 1 -- 172.21.15.77:0/1627738069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc538101f90 msgr2=0x7fc5381023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:39.545 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.542+0000 7fc54047b700 1 --2- 172.21.15.77:0/1627738069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc538101f90 0x7fc5381023f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc5340099c0 tx=0x7fc534009cd0 comp rx=0 tx=0).stop 2023-12-01T01:17:39.545 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.542+0000 7fc54047b700 1 -- 172.21.15.77:0/1627738069 shutdown_connections 2023-12-01T01:17:39.546 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.542+0000 7fc54047b700 1 --2- 172.21.15.77:0/1627738069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc538102930 0x7fc5381091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:39.546 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.542+0000 7fc54047b700 1 --2- 172.21.15.77:0/1627738069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc538100da0 0x7fc5381011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:39.546 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.542+0000 7fc54047b700 1 --2- 172.21.15.77:0/1627738069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc538101f90 0x7fc5381023f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:39.546 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.542+0000 7fc54047b700 1 -- 172.21.15.77:0/1627738069 >> 172.21.15.77:0/1627738069 conn(0x7fc5380fc2d0 msgr2=0x7fc5380fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:39.546 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.542+0000 7fc54047b700 1 -- 172.21.15.77:0/1627738069 shutdown_connections 2023-12-01T01:17:39.546 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.542+0000 7fc54047b700 1 -- 172.21.15.77:0/1627738069 wait complete. 2023-12-01T01:17:39.546 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.542+0000 7fc54047b700 1 Processor -- start 2023-12-01T01:17:39.546 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.542+0000 7fc54047b700 1 -- start start 2023-12-01T01:17:39.546 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.543+0000 7fc54047b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc538100da0 0x7fc538196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:39.546 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.543+0000 7fc54047b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc538101f90 0x7fc538196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:39.546 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.543+0000 7fc54047b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc538102930 0x7fc53819e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:39.546 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.543+0000 7fc54047b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53819e610 con 0x7fc538100da0 2023-12-01T01:17:39.546 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.543+0000 7fc54047b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53819e750 con 0x7fc538102930 2023-12-01T01:17:39.546 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.543+0000 7fc54047b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53819e8e0 con 0x7fc538101f90 2023-12-01T01:17:39.547 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.543+0000 7fc53e217700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc538100da0 0x7fc538196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:39.547 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.543+0000 7fc53e217700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc538100da0 0x7fc538196a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58732/0 (socket says 172.21.15.77:58732) 2023-12-01T01:17:39.547 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.543+0000 7fc53e217700 1 -- 172.21.15.77:0/3263773577 learned_addr learned my addr 172.21.15.77:0/3263773577 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:39.547 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.543+0000 7fc53ea18700 1 --2- 172.21.15.77:0/3263773577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc538102930 0x7fc53819e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:39.547 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.543+0000 7fc53da16700 1 --2- 172.21.15.77:0/3263773577 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc538101f90 0x7fc538196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:39.547 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.543+0000 7fc53e217700 1 -- 172.21.15.77:0/3263773577 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc538101f90 msgr2=0x7fc538196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:39.547 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.543+0000 7fc53e217700 1 --2- 172.21.15.77:0/3263773577 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc538101f90 0x7fc538196fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:39.547 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.543+0000 7fc53e217700 1 -- 172.21.15.77:0/3263773577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc538102930 msgr2=0x7fc53819e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:39.547 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.543+0000 7fc53e217700 1 --2- 172.21.15.77:0/3263773577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc538102930 0x7fc53819e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:39.547 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.543+0000 7fc53e217700 1 -- 172.21.15.77:0/3263773577 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5340095f0 con 0x7fc538100da0 2023-12-01T01:17:39.547 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.543+0000 7fc53da16700 1 --2- 172.21.15.77:0/3263773577 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc538101f90 0x7fc538196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:17:39.547 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.543+0000 7fc53e217700 1 --2- 172.21.15.77:0/3263773577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc538100da0 0x7fc538196a90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc52800e970 tx=0x7fc52800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:39.547 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.543+0000 7fc52f7fe700 1 -- 172.21.15.77:0/3263773577 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc528009790 con 0x7fc538100da0 2023-12-01T01:17:39.547 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.543+0000 7fc52f7fe700 1 -- 172.21.15.77:0/3263773577 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5280044e0 con 0x7fc538100da0 2023-12-01T01:17:39.548 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.543+0000 7fc54047b700 1 -- 172.21.15.77:0/3263773577 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc53819ebc0 con 0x7fc538100da0 2023-12-01T01:17:39.548 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.543+0000 7fc52f7fe700 1 -- 172.21.15.77:0/3263773577 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc528003e30 con 0x7fc538100da0 2023-12-01T01:17:39.548 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.543+0000 7fc54047b700 1 -- 172.21.15.77:0/3263773577 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc53819f0d0 con 0x7fc538100da0 2023-12-01T01:17:39.548 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.544+0000 7fc54047b700 1 -- 172.21.15.77:0/3263773577 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc538108340 con 0x7fc538100da0 2023-12-01T01:17:39.548 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.546+0000 7fc52f7fe700 1 -- 172.21.15.77:0/3263773577 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fc528003f90 con 0x7fc538100da0 2023-12-01T01:17:39.549 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.546+0000 7fc52f7fe700 1 --2- 172.21.15.77:0/3263773577 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc5240a05f0 0x7fc5240a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:39.549 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.547+0000 7fc53da16700 1 --2- 172.21.15.77:0/3263773577 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc5240a05f0 0x7fc5240a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:39.549 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.547+0000 7fc52f7fe700 1 -- 172.21.15.77:0/3263773577 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fc528014070 con 0x7fc538100da0 2023-12-01T01:17:39.549 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.547+0000 7fc53da16700 1 --2- 172.21.15.77:0/3263773577 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc5240a05f0 0x7fc5240a2ab0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc5340098e0 tx=0x7fc534019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:39.551 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.550+0000 7fc52f7fe700 1 -- 172.21.15.77:0/3263773577 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc528095a50 con 0x7fc538100da0 2023-12-01T01:17:39.785 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.784+0000 7fc54047b700 1 -- 172.21.15.77:0/3263773577 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc53802cc70 con 0x7fc538100da0 2023-12-01T01:17:39.786 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.785+0000 7fc52f7fe700 1 -- 172.21.15.77:0/3263773577 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10838 (secure 0 0 0) 0x7fc528099070 con 0x7fc538100da0 2023-12-01T01:17:39.786 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:17:39.787 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1435145620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1435145620},{"type":"v1","addr":"172.21.15.77:6835","nonce":1435145620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/841838908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":841838908},{"type":"v1","addr":"172.21.15.77:6837","nonce":841838908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-01T01:16:25.345276+0000","modified":"2023-12-01T01:16:32.596617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24496,"mds_2":24475,"mds_3":24473,"mds_4":14649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/3726913446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3726913446},{"type":"v1","addr":"172.21.15.77:6839","nonce":3726913446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14649":{"gid":14649,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":5,"addr":"172.21.15.77:6841/3023184965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3023184965},{"type":"v1","addr":"172.21.15.77:6841","nonce":3023184965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/2249638445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2249638445},{"type":"v1","addr":"172.21.15.137:6833","nonce":2249638445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6833/1201599774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1201599774},{"type":"v1","addr":"172.21.15.203:6833","nonce":1201599774}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3935354878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3935354878},{"type":"v1","addr":"172.21.15.137:6835","nonce":3935354878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3237538176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3237538176},{"type":"v1","addr":"172.21.15.203:6835","nonce":3237538176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3518881867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3518881867},{"type":"v1","addr":"172.21.15.137:6837","nonce":3518881867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.203:6837/2887080977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2887080977},{"type":"v1","addr":"172.21.15.203:6837","nonce":2887080977}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.203:6839/2888221613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2888221613},{"type":"v1","addr":"172.21.15.203:6839","nonce":2888221613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/1099966660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1099966660},{"type":"v1","addr":"172.21.15.137:6839","nonce":1099966660}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-01T01:17:39.790 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.787+0000 7fc54047b700 1 -- 172.21.15.77:0/3263773577 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc5240a05f0 msgr2=0x7fc5240a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:39.790 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.787+0000 7fc54047b700 1 --2- 172.21.15.77:0/3263773577 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc5240a05f0 0x7fc5240a2ab0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc5340098e0 tx=0x7fc534019040 comp rx=0 tx=0).stop 2023-12-01T01:17:39.790 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.787+0000 7fc54047b700 1 -- 172.21.15.77:0/3263773577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc538100da0 msgr2=0x7fc538196a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:39.791 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.787+0000 7fc54047b700 1 --2- 172.21.15.77:0/3263773577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc538100da0 0x7fc538196a90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc52800e970 tx=0x7fc52800ed30 comp rx=0 tx=0).stop 2023-12-01T01:17:39.791 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.787+0000 7fc54047b700 1 -- 172.21.15.77:0/3263773577 shutdown_connections 2023-12-01T01:17:39.791 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.787+0000 7fc54047b700 1 --2- 172.21.15.77:0/3263773577 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc5240a05f0 0x7fc5240a2ab0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:39.791 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.787+0000 7fc54047b700 1 --2- 172.21.15.77:0/3263773577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc538102930 0x7fc53819e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:39.791 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.787+0000 7fc54047b700 1 --2- 172.21.15.77:0/3263773577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc538100da0 0x7fc538196a90 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:39.792 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.787+0000 7fc54047b700 1 --2- 172.21.15.77:0/3263773577 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc538101f90 0x7fc538196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:39.792 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.787+0000 7fc54047b700 1 -- 172.21.15.77:0/3263773577 >> 172.21.15.77:0/3263773577 conn(0x7fc5380fc2d0 msgr2=0x7fc538105480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:39.792 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.787+0000 7fc54047b700 1 -- 172.21.15.77:0/3263773577 shutdown_connections 2023-12-01T01:17:39.792 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.787+0000 7fc54047b700 1 -- 172.21.15.77:0/3263773577 wait complete. 2023-12-01T01:17:39.792 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 27 2023-12-01T01:17:39.799 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:39 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/17605715' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-01T01:17:39.800 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:39 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/1800814565' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-01T01:17:39.800 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:39 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/1590835743' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-01T01:17:39.801 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:17:39.924 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.921+0000 7ff58f24e700 1 -- 172.21.15.77:0/2541480685 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff588100eb0 msgr2=0x7ff588101310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:39.925 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.921+0000 7ff58f24e700 1 --2- 172.21.15.77:0/2541480685 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff588100eb0 0x7ff588101310 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff5740099c0 tx=0x7ff574009cd0 comp rx=0 tx=0).stop 2023-12-01T01:17:39.925 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.921+0000 7ff58f24e700 1 -- 172.21.15.77:0/2541480685 shutdown_connections 2023-12-01T01:17:39.925 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.921+0000 7ff58f24e700 1 --2- 172.21.15.77:0/2541480685 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff588101850 0x7ff588108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:39.925 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.921+0000 7ff58f24e700 1 --2- 172.21.15.77:0/2541480685 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5880ffcc0 0x7ff5881000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:39.925 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.921+0000 7ff58f24e700 1 --2- 172.21.15.77:0/2541480685 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff588100eb0 0x7ff588101310 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:39.925 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.921+0000 7ff58f24e700 1 -- 172.21.15.77:0/2541480685 >> 172.21.15.77:0/2541480685 conn(0x7ff5880fb210 msgr2=0x7ff5880fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:39.925 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.921+0000 7ff58f24e700 1 -- 172.21.15.77:0/2541480685 shutdown_connections 2023-12-01T01:17:39.925 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.921+0000 7ff58f24e700 1 -- 172.21.15.77:0/2541480685 wait complete. 2023-12-01T01:17:39.925 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.921+0000 7ff58f24e700 1 Processor -- start 2023-12-01T01:17:39.925 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.922+0000 7ff58f24e700 1 -- start start 2023-12-01T01:17:39.925 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.922+0000 7ff58f24e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5880ffcc0 0x7ff588103c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:39.925 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.922+0000 7ff58f24e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff588100eb0 0x7ff588102290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:39.925 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.922+0000 7ff58f24e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff588101850 0x7ff5881027d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:39.926 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.922+0000 7ff58f24e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff588102f70 con 0x7ff5880ffcc0 2023-12-01T01:17:39.926 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.922+0000 7ff58f24e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5881030e0 con 0x7ff588100eb0 2023-12-01T01:17:39.926 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.922+0000 7ff58f24e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5881a0a70 con 0x7ff588101850 2023-12-01T01:17:39.926 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.922+0000 7ff58cfea700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5880ffcc0 0x7ff588103c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:39.926 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.922+0000 7ff58cfea700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5880ffcc0 0x7ff588103c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58746/0 (socket says 172.21.15.77:58746) 2023-12-01T01:17:39.926 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.922+0000 7ff58cfea700 1 -- 172.21.15.77:0/3237973519 learned_addr learned my addr 172.21.15.77:0/3237973519 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:39.926 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.922+0000 7ff587fff700 1 --2- 172.21.15.77:0/3237973519 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff588100eb0 0x7ff588102290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:39.926 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.922+0000 7ff58d7eb700 1 --2- 172.21.15.77:0/3237973519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff588101850 0x7ff5881027d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:39.926 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.922+0000 7ff58cfea700 1 -- 172.21.15.77:0/3237973519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff588101850 msgr2=0x7ff5881027d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:39.926 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.922+0000 7ff58cfea700 1 --2- 172.21.15.77:0/3237973519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff588101850 0x7ff5881027d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:39.926 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.922+0000 7ff58cfea700 1 -- 172.21.15.77:0/3237973519 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff588100eb0 msgr2=0x7ff588102290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:39.926 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.922+0000 7ff58cfea700 1 --2- 172.21.15.77:0/3237973519 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff588100eb0 0x7ff588102290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:39.926 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.922+0000 7ff58cfea700 1 -- 172.21.15.77:0/3237973519 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5740095f0 con 0x7ff5880ffcc0 2023-12-01T01:17:39.926 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.922+0000 7ff58d7eb700 1 --2- 172.21.15.77:0/3237973519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff588101850 0x7ff5881027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:17:39.927 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.922+0000 7ff58cfea700 1 --2- 172.21.15.77:0/3237973519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5880ffcc0 0x7ff588103c50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff57c00e970 tx=0x7ff57c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:39.927 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.922+0000 7ff585ffb700 1 -- 172.21.15.77:0/3237973519 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57c009790 con 0x7ff5880ffcc0 2023-12-01T01:17:39.927 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.922+0000 7ff585ffb700 1 -- 172.21.15.77:0/3237973519 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff57c00eec0 con 0x7ff5880ffcc0 2023-12-01T01:17:39.927 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.922+0000 7ff58f24e700 1 -- 172.21.15.77:0/3237973519 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5881a0d50 con 0x7ff5880ffcc0 2023-12-01T01:17:39.927 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.922+0000 7ff585ffb700 1 -- 172.21.15.77:0/3237973519 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57c003cf0 con 0x7ff5880ffcc0 2023-12-01T01:17:39.927 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.923+0000 7ff58f24e700 1 -- 172.21.15.77:0/3237973519 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5881a11f0 con 0x7ff5880ffcc0 2023-12-01T01:17:39.927 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.923+0000 7ff58f24e700 1 -- 172.21.15.77:0/3237973519 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff58810b520 con 0x7ff5880ffcc0 2023-12-01T01:17:39.928 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.924+0000 7ff585ffb700 1 -- 172.21.15.77:0/3237973519 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7ff57c00ccd0 con 0x7ff5880ffcc0 2023-12-01T01:17:39.928 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.925+0000 7ff585ffb700 1 --2- 172.21.15.77:0/3237973519 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff5780a06f0 0x7ff5780a2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:39.928 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.925+0000 7ff587fff700 1 --2- 172.21.15.77:0/3237973519 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff5780a06f0 0x7ff5780a2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:39.928 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.925+0000 7ff585ffb700 1 -- 172.21.15.77:0/3237973519 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7ff57c014070 con 0x7ff5880ffcc0 2023-12-01T01:17:39.928 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.925+0000 7ff587fff700 1 --2- 172.21.15.77:0/3237973519 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff5780a06f0 0x7ff5780a2bb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff57400b580 tx=0x7ff574005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:39.930 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:39.929+0000 7ff585ffb700 1 -- 172.21.15.77:0/3237973519 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff57c095760 con 0x7ff5880ffcc0 2023-12-01T01:17:40.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:39 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/17605715' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-01T01:17:40.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:39 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/1800814565' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-01T01:17:40.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:39 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/1590835743' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-01T01:17:40.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:39 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/17605715' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-01T01:17:40.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:39 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/1800814565' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-01T01:17:40.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:39 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/1590835743' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-01T01:17:40.125 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.124+0000 7ff58f24e700 1 -- 172.21.15.77:0/3237973519 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff5881a1860 con 0x7ff5880ffcc0 2023-12-01T01:17:40.127 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.126+0000 7ff585ffb700 1 -- 172.21.15.77:0/3237973519 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22472 (secure 0 0 0) 0x7ff57c098d80 con 0x7ff5880ffcc0 2023-12-01T01:17:40.127 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:17:40.127 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":84,"fsid":"2a06e29a-8fe6-11ee-95a2-87774f69a715","created":"2023-12-01T01:09:30.210398+0000","modified":"2023-12-01T01:17:25.609766+0000","last_up_change":"2023-12-01T01:15:44.586092+0000","last_in_change":"2023-12-01T01:15:26.305439+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:12:30.514229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:16:23.568860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-01T01:16:24.531137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fc651e62-a103-4ab8-a76d-d3708ea6aeec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6803","nonce":2244799094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6805","nonce":2244799094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6809","nonce":2244799094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6807","nonce":2244799094}]},"public_addr":"172.21.15.77:6803/2244799094","cluster_addr":"172.21.15.77:6805/2244799094","heartbeat_back_addr":"172.21.15.77:6809/2244799094","heartbeat_front_addr":"172.21.15.77:6807/2244799094","state":["exists","up"]},{"osd":1,"uuid":"0adf8c54-49ec-49a0-8bf1-af1f20e60729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6801","nonce":224194890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6803","nonce":224194890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6807","nonce":224194890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6805","nonce":224194890}]},"public_addr":"172.21.15.137:6801/224194890","cluster_addr":"172.21.15.137:6803/224194890","heartbeat_back_addr":"172.21.15.137:6807/224194890","heartbeat_front_addr":"172.21.15.137:6805/224194890","state":["exists","up"]},{"osd":2,"uuid":"5a69802c-dede-4544-9a12-436f5c1c0038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6801","nonce":4078459110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6803","nonce":4078459110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6807","nonce":4078459110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6805","nonce":4078459110}]},"public_addr":"172.21.15.203:6801/4078459110","cluster_addr":"172.21.15.203:6803/4078459110","heartbeat_back_addr":"172.21.15.203:6807/4078459110","heartbeat_front_addr":"172.21.15.203:6805/4078459110","state":["exists","up"]},{"osd":3,"uuid":"ad325a97-580b-4c99-bf12-cbb0d92a6d0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6811","nonce":1454440477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6813","nonce":1454440477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6817","nonce":1454440477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6815","nonce":1454440477}]},"public_addr":"172.21.15.77:6811/1454440477","cluster_addr":"172.21.15.77:6813/1454440477","heartbeat_back_addr":"172.21.15.77:6817/1454440477","heartbeat_front_addr":"172.21.15.77:6815/1454440477","state":["exists","up"]},{"osd":4,"uuid":"1e3ac1ba-98bb-4b8b-8e26-50064c66bf2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6809","nonce":1103241451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6811","nonce":1103241451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6815","nonce":1103241451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6813","nonce":1103241451}]},"public_addr":"172.21.15.137:6809/1103241451","cluster_addr":"172.21.15.137:6811/1103241451","heartbeat_back_addr":"172.21.15.137:6815/1103241451","heartbeat_front_addr":"172.21.15.137:6813/1103241451","state":["exists","up"]},{"osd":5,"uuid":"eb32c016-abeb-44ab-abf0-19dd09235736","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6809","nonce":2648569482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6811","nonce":2648569482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6815","nonce":2648569482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6813","nonce":2648569482}]},"public_addr":"172.21.15.203:6809/2648569482","cluster_addr":"172.21.15.203:6811/2648569482","heartbeat_back_addr":"172.21.15.203:6815/2648569482","heartbeat_front_addr":"172.21.15.203:6813/2648569482","state":["exists","up"]},{"osd":6,"uuid":"0e3fa97e-2ca6-418c-9e8d-a6fe3c56adb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6819","nonce":2463414651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6821","nonce":2463414651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6825","nonce":2463414651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6823","nonce":2463414651}]},"public_addr":"172.21.15.77:6819/2463414651","cluster_addr":"172.21.15.77:6821/2463414651","heartbeat_back_addr":"172.21.15.77:6825/2463414651","heartbeat_front_addr":"172.21.15.77:6823/2463414651","state":["exists","up"]},{"osd":7,"uuid":"d7398c01-0f98-4138-8a49-c3ff0aae7e88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6817","nonce":3108330766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6819","nonce":3108330766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6823","nonce":3108330766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6821","nonce":3108330766}]},"public_addr":"172.21.15.137:6817/3108330766","cluster_addr":"172.21.15.137:6819/3108330766","heartbeat_back_addr":"172.21.15.137:6823/3108330766","heartbeat_front_addr":"172.21.15.137:6821/3108330766","state":["exists","up"]},{"osd":8,"uuid":"0c09050d-3e43-410c-8cd7-bcb891af56b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6817","nonce":1161158776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6819","nonce":1161158776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6823","nonce":1161158776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6821","nonce":1161158776}]},"public_addr":"172.21.15.203:6817/1161158776","cluster_addr":"172.21.15.203:6819/1161158776","heartbeat_back_addr":"172.21.15.203:6823/1161158776","heartbeat_front_addr":"172.21.15.203:6821/1161158776","state":["exists","up"]},{"osd":9,"uuid":"527ecb49-95ec-48bd-9872-3b07fae5c5ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6827","nonce":2409880685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6829","nonce":2409880685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6833","nonce":2409880685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6831","nonce":2409880685}]},"public_addr":"172.21.15.77:6827/2409880685","cluster_addr":"172.21.15.77:6829/2409880685","heartbeat_back_addr":"172.21.15.77:6833/2409880685","heartbeat_front_addr":"172.21.15.77:6831/2409880685","state":["exists","up"]},{"osd":10,"uuid":"1680bb3e-a3c8-427b-af68-e2214f32bed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6825","nonce":374920146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6827","nonce":374920146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6831","nonce":374920146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6829","nonce":374920146}]},"public_addr":"172.21.15.137:6825/374920146","cluster_addr":"172.21.15.137:6827/374920146","heartbeat_back_addr":"172.21.15.137:6831/374920146","heartbeat_front_addr":"172.21.15.137:6829/374920146","state":["exists","up"]},{"osd":11,"uuid":"8a533466-023d-4803-ae36-9a9f1715e49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6825","nonce":2161588962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6827","nonce":2161588962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6831","nonce":2161588962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6829","nonce":2161588962}]},"public_addr":"172.21.15.203:6825/2161588962","cluster_addr":"172.21.15.203:6827/2161588962","heartbeat_back_addr":"172.21.15.203:6831/2161588962","heartbeat_front_addr":"172.21.15.203:6829/2161588962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:11:45.420509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:06.210115+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:28.191307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:49.594116+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:11.261689+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:33.376541+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:54.968336+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:16.775310+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:38.733792+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:59.456699+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:15:20.924059+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:15:43.094773+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/2260035254":"2023-12-02T01:10:24.191835+0000","172.21.15.77:6800/1456621747":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/219838828":"2023-12-02T01:10:24.191835+0000","172.21.15.77:6801/1456621747":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/2937149266":"2023-12-02T01:10:02.232948+0000","172.21.15.77:0/807520018":"2023-12-02T01:10:02.232948+0000","172.21.15.77:6801/1661788026":"2023-12-02T01:10:02.232948+0000","172.21.15.77:6800/1661788026":"2023-12-02T01:10:02.232948+0000","172.21.15.77:0/3551282473":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/1202222558":"2023-12-02T01:10:02.232948+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:17:40.129 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.128+0000 7ff58f24e700 1 -- 172.21.15.77:0/3237973519 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff5780a06f0 msgr2=0x7ff5780a2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.129 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.128+0000 7ff58f24e700 1 --2- 172.21.15.77:0/3237973519 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff5780a06f0 0x7ff5780a2bb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff57400b580 tx=0x7ff574005fb0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.129 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.128+0000 7ff58f24e700 1 -- 172.21.15.77:0/3237973519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5880ffcc0 msgr2=0x7ff588103c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.130 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.128+0000 7ff58f24e700 1 --2- 172.21.15.77:0/3237973519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5880ffcc0 0x7ff588103c50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff57c00e970 tx=0x7ff57c00ec80 comp rx=0 tx=0).stop 2023-12-01T01:17:40.130 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.128+0000 7ff58f24e700 1 -- 172.21.15.77:0/3237973519 shutdown_connections 2023-12-01T01:17:40.130 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.128+0000 7ff58f24e700 1 --2- 172.21.15.77:0/3237973519 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff5780a06f0 0x7ff5780a2bb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.130 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.128+0000 7ff58f24e700 1 --2- 172.21.15.77:0/3237973519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff588101850 0x7ff5881027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.130 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.128+0000 7ff58f24e700 1 --2- 172.21.15.77:0/3237973519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff5880ffcc0 0x7ff588103c50 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.130 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.128+0000 7ff58f24e700 1 --2- 172.21.15.77:0/3237973519 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff588100eb0 0x7ff588102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.130 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.128+0000 7ff58f24e700 1 -- 172.21.15.77:0/3237973519 >> 172.21.15.77:0/3237973519 conn(0x7ff5880fb210 msgr2=0x7ff5880691d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:40.130 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.128+0000 7ff58f24e700 1 -- 172.21.15.77:0/3237973519 shutdown_connections 2023-12-01T01:17:40.130 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.128+0000 7ff58f24e700 1 -- 172.21.15.77:0/3237973519 wait complete. 2023-12-01T01:17:40.142 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-12-01T01:16:25.345276+0000', 'modified': '2023-12-01T01:16:32.596617+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24500, 'mds_1': 24496, 'mds_2': 24475, 'mds_3': 24473, 'mds_4': 14649}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14637': {'gid': 14637, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.77:6839/3726913446', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6838', 'nonce': 3726913446}, {'type': 'v1', 'addr': '172.21.15.77:6839', 'nonce': 3726913446}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_14649': {'gid': 14649, 'name': 'a', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.77:6841/3023184965', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6840', 'nonce': 3023184965}, {'type': 'v1', 'addr': '172.21.15.77:6841', 'nonce': 3023184965}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24428': {'gid': 24428, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.137:6833/2249638445', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6832', 'nonce': 2249638445}, {'type': 'v1', 'addr': '172.21.15.137:6833', 'nonce': 2249638445}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24430': {'gid': 24430, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.203:6833/1201599774', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6832', 'nonce': 1201599774}, {'type': 'v1', 'addr': '172.21.15.203:6833', 'nonce': 1201599774}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24449': {'gid': 24449, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.137:6835/3935354878', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6834', 'nonce': 3935354878}, {'type': 'v1', 'addr': '172.21.15.137:6835', 'nonce': 3935354878}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24463': {'gid': 24463, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.203:6835/3237538176', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6834', 'nonce': 3237538176}, {'type': 'v1', 'addr': '172.21.15.203:6835', 'nonce': 3237538176}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24473': {'gid': 24473, 'name': 'h', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.137:6837/3518881867', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6836', 'nonce': 3518881867}, {'type': 'v1', 'addr': '172.21.15.137:6837', 'nonce': 3518881867}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24475': {'gid': 24475, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.203:6837/2887080977', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6836', 'nonce': 2887080977}, {'type': 'v1', 'addr': '172.21.15.203:6837', 'nonce': 2887080977}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24496': {'gid': 24496, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.203:6839/2888221613', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6838', 'nonce': 2888221613}, {'type': 'v1', 'addr': '172.21.15.203:6839', 'nonce': 2888221613}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24500': {'gid': 24500, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.137:6839/1099966660', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6838', 'nonce': 1099966660}, {'type': 'v1', 'addr': '172.21.15.137:6839', 'nonce': 1099966660}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-01T01:17:40.142 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-12-01T01:17:40.142 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-12-01T01:17:40.268 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.265+0000 7f803ccf4700 1 -- 172.21.15.77:0/2531396601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80380fff30 msgr2=0x7f8038100390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.268 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.265+0000 7f803ccf4700 1 --2- 172.21.15.77:0/2531396601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80380fff30 0x7f8038100390 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8028009990 tx=0x7f8028009ca0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.268 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.265+0000 7f803ccf4700 1 -- 172.21.15.77:0/2531396601 shutdown_connections 2023-12-01T01:17:40.269 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.265+0000 7f803ccf4700 1 --2- 172.21.15.77:0/2531396601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80381008d0 0x7f8038108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.269 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.265+0000 7f803ccf4700 1 --2- 172.21.15.77:0/2531396601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8038101200 0x7f8038101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.269 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.265+0000 7f803ccf4700 1 --2- 172.21.15.77:0/2531396601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80380fff30 0x7f8038100390 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.269 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.265+0000 7f803ccf4700 1 -- 172.21.15.77:0/2531396601 >> 172.21.15.77:0/2531396601 conn(0x7f80380fa960 msgr2=0x7f80380fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:40.269 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.265+0000 7f803ccf4700 1 -- 172.21.15.77:0/2531396601 shutdown_connections 2023-12-01T01:17:40.269 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.265+0000 7f803ccf4700 1 -- 172.21.15.77:0/2531396601 wait complete. 2023-12-01T01:17:40.269 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.266+0000 7f803ccf4700 1 Processor -- start 2023-12-01T01:17:40.269 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.266+0000 7f803ccf4700 1 -- start start 2023-12-01T01:17:40.269 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.266+0000 7f803ccf4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80380fff30 0x7f8038196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.269 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.266+0000 7f803ccf4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80381008d0 0x7f8038196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.269 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.266+0000 7f803ccf4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8038101200 0x7f803819dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.269 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.266+0000 7f803ccf4700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803819e5b0 con 0x7f80381008d0 2023-12-01T01:17:40.269 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.266+0000 7f803ccf4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803819e6f0 con 0x7f8038101200 2023-12-01T01:17:40.270 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.266+0000 7f803ccf4700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803819e880 con 0x7f80380fff30 2023-12-01T01:17:40.270 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.266+0000 7f8035d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80381008d0 0x7f8038196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.270 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.266+0000 7f8035d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80381008d0 0x7f8038196f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58762/0 (socket says 172.21.15.77:58762) 2023-12-01T01:17:40.270 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.266+0000 7f8035d9b700 1 -- 172.21.15.77:0/3472203035 learned_addr learned my addr 172.21.15.77:0/3472203035 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:40.270 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.266+0000 7f803659c700 1 --2- 172.21.15.77:0/3472203035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80380fff30 0x7f8038196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.270 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.266+0000 7f8035d9b700 1 -- 172.21.15.77:0/3472203035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80380fff30 msgr2=0x7f8038196a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.270 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.266+0000 7f8035d9b700 1 --2- 172.21.15.77:0/3472203035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80380fff30 0x7f8038196a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.270 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.266+0000 7f8035d9b700 1 -- 172.21.15.77:0/3472203035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8038101200 msgr2=0x7f803819dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.270 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.266+0000 7f8036d9d700 1 --2- 172.21.15.77:0/3472203035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8038101200 0x7f803819dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.270 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.266+0000 7f8035d9b700 1 --2- 172.21.15.77:0/3472203035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8038101200 0x7f803819dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.270 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.266+0000 7f8035d9b700 1 -- 172.21.15.77:0/3472203035 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80280095f0 con 0x7f80381008d0 2023-12-01T01:17:40.270 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.266+0000 7f803659c700 1 --2- 172.21.15.77:0/3472203035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80380fff30 0x7f8038196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:17:40.270 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.266+0000 7f8035d9b700 1 --2- 172.21.15.77:0/3472203035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80381008d0 0x7f8038196f70 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8028000c00 tx=0x7f802800f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:40.270 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.266+0000 7f80277fe700 1 -- 172.21.15.77:0/3472203035 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802801d070 con 0x7f80381008d0 2023-12-01T01:17:40.271 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.266+0000 7f803ccf4700 1 -- 172.21.15.77:0/3472203035 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f803819eb00 con 0x7f80381008d0 2023-12-01T01:17:40.271 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.266+0000 7f80277fe700 1 -- 172.21.15.77:0/3472203035 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f802800fc00 con 0x7f80381008d0 2023-12-01T01:17:40.271 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.266+0000 7f80277fe700 1 -- 172.21.15.77:0/3472203035 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8028017650 con 0x7f80381008d0 2023-12-01T01:17:40.271 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.267+0000 7f803ccf4700 1 -- 172.21.15.77:0/3472203035 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f803819efb0 con 0x7f80381008d0 2023-12-01T01:17:40.277 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.269+0000 7f80277fe700 1 -- 172.21.15.77:0/3472203035 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f8028022470 con 0x7f80381008d0 2023-12-01T01:17:40.277 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.270+0000 7f80277fe700 1 --2- 172.21.15.77:0/3472203035 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f80200a05b0 0x7f80200a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.277 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.270+0000 7f803659c700 1 --2- 172.21.15.77:0/3472203035 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f80200a05b0 0x7f80200a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.277 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.270+0000 7f80277fe700 1 -- 172.21.15.77:0/3472203035 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f80280cccb0 con 0x7f80381008d0 2023-12-01T01:17:40.277 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.270+0000 7f803659c700 1 --2- 172.21.15.77:0/3472203035 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f80200a05b0 0x7f80200a2a70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f802c005d90 tx=0x7f802c005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:40.277 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.270+0000 7f8048e84b80 1 Processor -- start 2023-12-01T01:17:40.277 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.271+0000 7f8048e84b80 1 -- start start 2023-12-01T01:17:40.277 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.271+0000 7f8048e84b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f2770f0a0 0x558f2770f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.277 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.271+0000 7f8048e84b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f2770fa20 0x558f2770de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.278 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.271+0000 7f8048e84b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f2770e3a0 0x558f2770e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.278 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.271+0000 7f8048e84b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f276659c0 con 0x558f2770fa20 2023-12-01T01:17:40.278 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.271+0000 7f8048e84b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f27663380 con 0x558f2770e3a0 2023-12-01T01:17:40.278 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.271+0000 7f8048e84b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f2765ec70 con 0x558f2770f0a0 2023-12-01T01:17:40.278 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.271+0000 7f8035d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f2770fa20 0x558f2770de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.278 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.271+0000 7f8035d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f2770fa20 0x558f2770de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58770/0 (socket says 172.21.15.77:58770) 2023-12-01T01:17:40.278 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.271+0000 7f8035d9b700 1 -- 172.21.15.77:0/2607324015 learned_addr learned my addr 172.21.15.77:0/2607324015 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:40.278 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.271+0000 7f8035d9b700 1 -- 172.21.15.77:0/2607324015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f2770f0a0 msgr2=0x558f2770f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:40.278 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.271+0000 7f8036d9d700 1 --2- 172.21.15.77:0/2607324015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f2770e3a0 0x558f2770e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.278 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.271+0000 7f803659c700 1 --2- 172.21.15.77:0/2607324015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f2770f0a0 0x558f2770f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.278 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.271+0000 7f8035d9b700 1 --2- 172.21.15.77:0/2607324015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f2770f0a0 0x558f2770f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.278 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.271+0000 7f8035d9b700 1 -- 172.21.15.77:0/2607324015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f2770e3a0 msgr2=0x558f2770e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:40.278 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.271+0000 7f8035d9b700 1 --2- 172.21.15.77:0/2607324015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f2770e3a0 0x558f2770e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.279 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.271+0000 7f8035d9b700 1 -- 172.21.15.77:0/2607324015 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f2748cd90 con 0x558f2770fa20 2023-12-01T01:17:40.279 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.271+0000 7f8035d9b700 1 --2- 172.21.15.77:0/2607324015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f2770fa20 0x558f2770de10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f80280a1ff0 tx=0x7f8028022bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3658f9b6bd818ef2 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:40.279 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.271+0000 7f8024ff9700 1 -- 172.21.15.77:0/2607324015 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802809a540 con 0x558f2770fa20 2023-12-01T01:17:40.279 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.272+0000 7f8024ff9700 1 -- 172.21.15.77:0/2607324015 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f802809e370 con 0x558f2770fa20 2023-12-01T01:17:40.279 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.272+0000 7f8024ff9700 1 -- 172.21.15.77:0/2607324015 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8028099c90 con 0x558f2770fa20 2023-12-01T01:17:40.279 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.272+0000 7f8048e84b80 1 -- 172.21.15.77:0/2607324015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f2770fa20 msgr2=0x558f2770de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.279 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.272+0000 7f8048e84b80 1 --2- 172.21.15.77:0/2607324015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f2770fa20 0x558f2770de10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f80280a1ff0 tx=0x7f8028022bf0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.279 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.272+0000 7f8048e84b80 1 -- 172.21.15.77:0/2607324015 shutdown_connections 2023-12-01T01:17:40.279 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.272+0000 7f8048e84b80 1 --2- 172.21.15.77:0/2607324015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f2770e3a0 0x558f2770e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.279 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.272+0000 7f8048e84b80 1 --2- 172.21.15.77:0/2607324015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f2770f0a0 0x558f2770f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.279 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.272+0000 7f8048e84b80 1 --2- 172.21.15.77:0/2607324015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f2770fa20 0x558f2770de10 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.280 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.272+0000 7f8048e84b80 1 -- 172.21.15.77:0/2607324015 >> 172.21.15.77:0/2607324015 conn(0x558f2766d080 msgr2=0x558f277088f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:40.280 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.272+0000 7f8048e84b80 1 -- 172.21.15.77:0/2607324015 shutdown_connections 2023-12-01T01:17:40.280 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.272+0000 7f8048e84b80 1 -- 172.21.15.77:0/2607324015 wait complete. 2023-12-01T01:17:40.280 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.273+0000 7f8048e84b80 1 -- start start 2023-12-01T01:17:40.280 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.273+0000 7f8048e84b80 1 Processor -- start 2023-12-01T01:17:40.280 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.273+0000 7f8048e84b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f2770e3a0 0x558f27717650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.280 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.273+0000 7f8048e84b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f2770f0a0 0x558f27717b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.280 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.273+0000 7f8048e84b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f2770fa20 0x558f2771ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.280 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.273+0000 7f8048e84b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f275fb4c0 con 0x558f2770f0a0 2023-12-01T01:17:40.280 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.273+0000 7f8048e84b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f275fb160 con 0x558f2770e3a0 2023-12-01T01:17:40.280 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.273+0000 7f8048e84b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f274e28d0 con 0x558f2770fa20 2023-12-01T01:17:40.280 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.273+0000 7f8048e84b80 10 client.0 _ll_register_callbacks cb 0x558f27548a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:17:40.280 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.273+0000 7f8035d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f2770f0a0 0x558f27717b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.280 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.273+0000 7f8035d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f2770f0a0 0x558f27717b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58776/0 (socket says 172.21.15.77:58776) 2023-12-01T01:17:40.281 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.273+0000 7f8035d9b700 1 -- 172.21.15.77:0/145927262 learned_addr learned my addr 172.21.15.77:0/145927262 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:40.281 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.273+0000 7f8035d9b700 1 -- 172.21.15.77:0/145927262 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f2770fa20 msgr2=0x558f2771ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:40.281 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.273+0000 7f8035d9b700 1 --2- 172.21.15.77:0/145927262 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f2770fa20 0x558f2771ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.281 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.273+0000 7f8035d9b700 1 -- 172.21.15.77:0/145927262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f2770e3a0 msgr2=0x558f27717650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:40.281 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.273+0000 7f803659c700 1 --2- 172.21.15.77:0/145927262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f2770e3a0 0x558f27717650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.281 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.273+0000 7f8035d9b700 1 --2- 172.21.15.77:0/145927262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f2770e3a0 0x558f27717650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.281 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.273+0000 7f8035d9b700 1 -- 172.21.15.77:0/145927262 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f803819efb0 con 0x558f2770f0a0 2023-12-01T01:17:40.281 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.273+0000 7f8035d9b700 1 --2- 172.21.15.77:0/145927262 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f2770f0a0 0x558f27717b90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8028096fb0 tx=0x7f8028005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e38c43bd8d3948d2 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:40.281 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.273+0000 7f801dffb700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:17:40.281 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.273+0000 7f801dffb700 1 -- 172.21.15.77:0/145927262 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80280a2290 con 0x558f2770f0a0 2023-12-01T01:17:40.281 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.274+0000 7f801dffb700 1 -- 172.21.15.77:0/145927262 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f802809a540 con 0x558f2770f0a0 2023-12-01T01:17:40.281 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.274+0000 7f8048e84b80 1 -- 172.21.15.77:0/145927262 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558f27689cc0 con 0x558f2770f0a0 2023-12-01T01:17:40.281 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.274+0000 7f801dffb700 1 -- 172.21.15.77:0/145927262 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f802809e170 con 0x558f2770f0a0 2023-12-01T01:17:40.281 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.274+0000 7f8048e84b80 10 client.15258 fetch_fsmap learned FSMap version 27 2023-12-01T01:17:40.282 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.274+0000 7f8048e84b80 1 -- 172.21.15.77:0/145927262 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558f27475c10 con 0x558f2770f0a0 2023-12-01T01:17:40.282 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.274+0000 7f801dffb700 1 -- 172.21.15.77:0/145927262 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558f27689cc0 con 0x558f2770f0a0 2023-12-01T01:17:40.282 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.274+0000 7f801dffb700 1 -- 172.21.15.77:0/145927262 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f80280a0130 con 0x558f2770f0a0 2023-12-01T01:17:40.282 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.274+0000 7f8048e84b80 10 client.15258 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:17:40.282 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.274+0000 7f8048e84b80 10 client.15258 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-01T01:17:40.282 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.274+0000 7f8048e84b80 20 client.15258 populate_metadata read hostname 'smithi077' 2023-12-01T01:17:40.282 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.274+0000 7f8048e84b80 1 --2- 172.21.15.77:0/145927262 >> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] conn(0x558f27722340 0x558f27721b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.282 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.274+0000 7f8048e84b80 4 client.15258 mds_command: new command op to 14649 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:17:40.282 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.274+0000 7f8048e84b80 1 -- 172.21.15.77:0/145927262 --> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558f276dcdc0 con 0x558f27722340 2023-12-01T01:17:40.282 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.274+0000 7f803659c700 1 --2- 172.21.15.77:0/145927262 >> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] conn(0x558f27722340 0x558f27721b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.282 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.274+0000 7f801dffb700 10 client.15258 ms_handle_connect on v2:172.21.15.77:6840/3023184965 2023-12-01T01:17:40.282 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.274+0000 7f803659c700 1 --2- 172.21.15.77:0/145927262 >> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] conn(0x558f27722340 0x558f27721b00 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f76474586a96bafb server_cookie=f9b973c615dd9b86 in_seq=0 out_seq=0 2023-12-01T01:17:40.282 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.275+0000 7f801dffb700 1 -- 172.21.15.77:0/145927262 <== mds.4 v2:172.21.15.77:6840/3023184965 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x558f276dcdc0 con 0x558f27722340 2023-12-01T01:17:40.282 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.275+0000 7f801dffb700 10 client.15258 handle_command_reply: tid=0 2023-12-01T01:17:40.283 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.275+0000 7f8048e84b80 1 client.15258 shutdown 2023-12-01T01:17:40.283 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.275+0000 7f8048e84b80 1 -- 172.21.15.77:0/145927262 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f2770f0a0 msgr2=0x558f27717b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.283 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.275+0000 7f8048e84b80 1 --2- 172.21.15.77:0/145927262 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f2770f0a0 0x558f27717b90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8028096fb0 tx=0x7f8028005fd0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.283 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.275+0000 7f8048e84b80 1 -- 172.21.15.77:0/145927262 shutdown_connections 2023-12-01T01:17:40.283 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.275+0000 7f8048e84b80 1 --2- 172.21.15.77:0/145927262 >> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] conn(0x558f27722340 0x558f27721b00 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.283 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.275+0000 7f8048e84b80 1 --2- 172.21.15.77:0/145927262 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f2770fa20 0x558f2771ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.283 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.275+0000 7f801dffb700 0 client.15258 ms_handle_reset on v2:172.21.15.77:6840/3023184965 2023-12-01T01:17:40.283 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.275+0000 7f8048e84b80 1 --2- 172.21.15.77:0/145927262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f2770e3a0 0x558f27717650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.283 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.275+0000 7f8048e84b80 1 --2- 172.21.15.77:0/145927262 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f2770f0a0 0x558f27717b90 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.283 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.275+0000 7f8048e84b80 1 -- 172.21.15.77:0/145927262 >> 172.21.15.77:0/145927262 conn(0x558f2766d080 msgr2=0x558f27708060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:40.283 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.275+0000 7f8048e84b80 1 -- 172.21.15.77:0/145927262 shutdown_connections 2023-12-01T01:17:40.283 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.275+0000 7f8048e84b80 1 -- 172.21.15.77:0/145927262 wait complete. 2023-12-01T01:17:40.283 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.275+0000 7f8048e84b80 20 client.15258 trim_cache size 0 max 16384 2023-12-01T01:17:40.301 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.292+0000 7f8048e84b80 1 Processor -- start 2023-12-01T01:17:40.301 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.292+0000 7f8048e84b80 1 -- start start 2023-12-01T01:17:40.301 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.292+0000 7f8048e84b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f2770e3a0 0x558f2770de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.301 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.292+0000 7f8048e84b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f2770f0a0 0x558f2772b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.301 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.292+0000 7f8048e84b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f2770fa20 0x558f27712360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.301 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.292+0000 7f8048e84b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f275fb4c0 con 0x558f2770e3a0 2023-12-01T01:17:40.301 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.292+0000 7f8048e84b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f276659c0 con 0x558f2770fa20 2023-12-01T01:17:40.301 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.292+0000 7f8048e84b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f2765ec70 con 0x558f2770f0a0 2023-12-01T01:17:40.302 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.293+0000 7f803659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f2770e3a0 0x558f2770de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.302 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.293+0000 7f803659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f2770e3a0 0x558f2770de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58784/0 (socket says 172.21.15.77:58784) 2023-12-01T01:17:40.302 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.293+0000 7f803659c700 1 -- 172.21.15.77:0/43166860 learned_addr learned my addr 172.21.15.77:0/43166860 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:40.302 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.293+0000 7f8035d9b700 1 --2- 172.21.15.77:0/43166860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f2770f0a0 0x558f2772b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.302 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.293+0000 7f8036d9d700 1 --2- 172.21.15.77:0/43166860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f2770fa20 0x558f27712360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.302 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.293+0000 7f803659c700 1 -- 172.21.15.77:0/43166860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f2770f0a0 msgr2=0x558f2772b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:40.302 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.293+0000 7f803659c700 1 --2- 172.21.15.77:0/43166860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f2770f0a0 0x558f2772b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.302 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.293+0000 7f803659c700 1 -- 172.21.15.77:0/43166860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f2770fa20 msgr2=0x558f27712360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:40.302 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.293+0000 7f803659c700 1 --2- 172.21.15.77:0/43166860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f2770fa20 0x558f27712360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.302 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.293+0000 7f803659c700 1 -- 172.21.15.77:0/43166860 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f274b3200 con 0x558f2770e3a0 2023-12-01T01:17:40.302 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.293+0000 7f803659c700 1 --2- 172.21.15.77:0/43166860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f2770e3a0 0x558f2770de10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x558f275f5da0 tx=0x7f802c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9f22e6dcdc6d3c9 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:40.302 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.293+0000 7f80257fa700 1 -- 172.21.15.77:0/43166860 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c024040 con 0x558f2770e3a0 2023-12-01T01:17:40.302 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.293+0000 7f80257fa700 1 -- 172.21.15.77:0/43166860 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f802c00eed0 con 0x558f2770e3a0 2023-12-01T01:17:40.302 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.293+0000 7f80257fa700 1 -- 172.21.15.77:0/43166860 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c0053a0 con 0x558f2770e3a0 2023-12-01T01:17:40.303 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.294+0000 7f8048e84b80 1 -- 172.21.15.77:0/43166860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f2770e3a0 msgr2=0x558f2770de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.303 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.294+0000 7f8048e84b80 1 --2- 172.21.15.77:0/43166860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f2770e3a0 0x558f2770de10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x558f275f5da0 tx=0x7f802c013000 comp rx=0 tx=0).stop 2023-12-01T01:17:40.303 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.294+0000 7f8048e84b80 1 -- 172.21.15.77:0/43166860 shutdown_connections 2023-12-01T01:17:40.303 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.294+0000 7f8048e84b80 1 --2- 172.21.15.77:0/43166860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f2770fa20 0x558f27712360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.303 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.294+0000 7f8048e84b80 1 --2- 172.21.15.77:0/43166860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f2770e3a0 0x558f2770de10 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.303 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.294+0000 7f8048e84b80 1 --2- 172.21.15.77:0/43166860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f2770f0a0 0x558f2772b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.303 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.294+0000 7f8048e84b80 1 -- 172.21.15.77:0/43166860 >> 172.21.15.77:0/43166860 conn(0x558f2766d080 msgr2=0x558f27716380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:40.303 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.294+0000 7f8048e84b80 1 -- 172.21.15.77:0/43166860 shutdown_connections 2023-12-01T01:17:40.303 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.294+0000 7f8048e84b80 1 -- 172.21.15.77:0/43166860 wait complete. 2023-12-01T01:17:40.303 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.294+0000 7f8048e84b80 1 -- start start 2023-12-01T01:17:40.303 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.294+0000 7f8048e84b80 1 Processor -- start 2023-12-01T01:17:40.303 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.294+0000 7f8048e84b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f2770e3a0 0x558f27727c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.303 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.294+0000 7f8048e84b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f2770f0a0 0x558f277281c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.303 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.294+0000 7f8048e84b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f2770fa20 0x558f27773e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.303 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.294+0000 7f8048e84b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f277091f0 con 0x558f2770fa20 2023-12-01T01:17:40.304 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.294+0000 7f8048e84b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f2772b1e0 con 0x558f2770f0a0 2023-12-01T01:17:40.304 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.294+0000 7f8048e84b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f2772b370 con 0x558f2770e3a0 2023-12-01T01:17:40.304 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.294+0000 7f8048e84b80 10 client.0 _ll_register_callbacks cb 0x7f802809a3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:17:40.304 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.294+0000 7f8036d9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f2770fa20 0x558f27773e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.304 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.294+0000 7f8036d9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f2770fa20 0x558f27773e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58792/0 (socket says 172.21.15.77:58792) 2023-12-01T01:17:40.304 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.294+0000 7f8036d9d700 1 -- 172.21.15.77:0/3392967002 learned_addr learned my addr 172.21.15.77:0/3392967002 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:40.304 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.294+0000 7f8035d9b700 1 --2- 172.21.15.77:0/3392967002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f2770f0a0 0x558f277281c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.304 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.294+0000 7f803659c700 1 --2- 172.21.15.77:0/3392967002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f2770e3a0 0x558f27727c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.304 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.295+0000 7f8036d9d700 1 -- 172.21.15.77:0/3392967002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f2770e3a0 msgr2=0x558f27727c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:40.304 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.295+0000 7f8036d9d700 1 --2- 172.21.15.77:0/3392967002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f2770e3a0 0x558f27727c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.305 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.295+0000 7f8036d9d700 1 -- 172.21.15.77:0/3392967002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f2770f0a0 msgr2=0x558f277281c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:40.305 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.295+0000 7f8036d9d700 1 --2- 172.21.15.77:0/3392967002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f2770f0a0 0x558f277281c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.305 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.295+0000 7f8036d9d700 1 -- 172.21.15.77:0/3392967002 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f802c00ea10 con 0x558f2770fa20 2023-12-01T01:17:40.305 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.295+0000 7f8036d9d700 1 --2- 172.21.15.77:0/3392967002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f2770fa20 0x558f27773e50 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8030004a60 tx=0x7f8030007940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=340ef2e4329ddc40 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:40.305 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.295+0000 7f801dffb700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:17:40.305 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.295+0000 7f801dffb700 1 -- 172.21.15.77:0/3392967002 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8030015600 con 0x558f2770fa20 2023-12-01T01:17:40.305 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.295+0000 7f801dffb700 1 -- 172.21.15.77:0/3392967002 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f8030015760 con 0x558f2770fa20 2023-12-01T01:17:40.305 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.295+0000 7f8048e84b80 1 -- 172.21.15.77:0/3392967002 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f802809e370 con 0x558f2770fa20 2023-12-01T01:17:40.305 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.295+0000 7f801dffb700 1 -- 172.21.15.77:0/3392967002 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8030015920 con 0x558f2770fa20 2023-12-01T01:17:40.305 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.295+0000 7f8048e84b80 10 client.15264 fetch_fsmap learned FSMap version 27 2023-12-01T01:17:40.305 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.295+0000 7f8048e84b80 1 -- 172.21.15.77:0/3392967002 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558f2748b260 con 0x558f2770fa20 2023-12-01T01:17:40.305 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.295+0000 7f801dffb700 1 -- 172.21.15.77:0/3392967002 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f802809e370 con 0x558f2770fa20 2023-12-01T01:17:40.305 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.295+0000 7f801dffb700 1 -- 172.21.15.77:0/3392967002 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f8030014070 con 0x558f2770fa20 2023-12-01T01:17:40.305 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.295+0000 7f8048e84b80 10 client.15264 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:17:40.306 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.295+0000 7f8048e84b80 10 client.15264 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-01T01:17:40.306 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.295+0000 7f8048e84b80 20 client.15264 populate_metadata read hostname 'smithi077' 2023-12-01T01:17:40.306 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.296+0000 7f8048e84b80 1 --2- 172.21.15.77:0/3392967002 >> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] conn(0x558f27778fc0 0x558f27775910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.306 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.296+0000 7f8048e84b80 4 client.15264 mds_command: new command op to 14649 tid=0[{"prefix": "status"}] 2023-12-01T01:17:40.306 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.296+0000 7f8048e84b80 1 -- 172.21.15.77:0/3392967002 --> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558f27657cf0 con 0x558f27778fc0 2023-12-01T01:17:40.306 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.296+0000 7f803659c700 1 --2- 172.21.15.77:0/3392967002 >> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] conn(0x558f27778fc0 0x558f27775910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.306 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.296+0000 7f803659c700 1 --2- 172.21.15.77:0/3392967002 >> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] conn(0x558f27778fc0 0x558f27775910 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=89a8951f0f0c50b5 server_cookie=1b29d0b4bfe350d6 in_seq=0 out_seq=0 2023-12-01T01:17:40.306 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.296+0000 7f801dffb700 10 client.15264 ms_handle_connect on v2:172.21.15.77:6840/3023184965 2023-12-01T01:17:40.306 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.296+0000 7f801dffb700 1 -- 172.21.15.77:0/3392967002 <== mds.4 v2:172.21.15.77:6840/3023184965 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x558f27657cf0 con 0x558f27778fc0 2023-12-01T01:17:40.306 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.296+0000 7f801dffb700 10 client.15264 handle_command_reply: tid=0 2023-12-01T01:17:40.306 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.296+0000 7f8048e84b80 1 client.15264 shutdown 2023-12-01T01:17:40.306 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.296+0000 7f8048e84b80 1 -- 172.21.15.77:0/3392967002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f2770fa20 msgr2=0x558f27773e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.306 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.296+0000 7f8048e84b80 1 --2- 172.21.15.77:0/3392967002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f2770fa20 0x558f27773e50 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8030004a60 tx=0x7f8030007940 comp rx=0 tx=0).stop 2023-12-01T01:17:40.306 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.296+0000 7f8048e84b80 1 -- 172.21.15.77:0/3392967002 shutdown_connections 2023-12-01T01:17:40.307 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.296+0000 7f8048e84b80 1 --2- 172.21.15.77:0/3392967002 >> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] conn(0x558f27778fc0 0x558f27775910 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.307 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.296+0000 7f8048e84b80 1 --2- 172.21.15.77:0/3392967002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558f2770fa20 0x558f27773e50 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.307 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.296+0000 7f801dffb700 0 client.15264 ms_handle_reset on v2:172.21.15.77:6840/3023184965 2023-12-01T01:17:40.307 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.296+0000 7f8048e84b80 1 --2- 172.21.15.77:0/3392967002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558f2770e3a0 0x558f27727c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.307 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.296+0000 7f8048e84b80 1 --2- 172.21.15.77:0/3392967002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558f2770f0a0 0x558f277281c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.307 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.296+0000 7f8048e84b80 1 -- 172.21.15.77:0/3392967002 >> 172.21.15.77:0/3392967002 conn(0x558f2766d080 msgr2=0x558f27712360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:40.307 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.296+0000 7f8048e84b80 1 -- 172.21.15.77:0/3392967002 shutdown_connections 2023-12-01T01:17:40.307 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.296+0000 7f8048e84b80 1 -- 172.21.15.77:0/3392967002 wait complete. 2023-12-01T01:17:40.307 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.296+0000 7f8048e84b80 20 client.15264 trim_cache size 0 max 16384 2023-12-01T01:17:40.307 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2023-12-01T01:17:40.307 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "2a06e29a-8fe6-11ee-95a2-87774f69a715", 2023-12-01T01:17:40.308 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 4, 2023-12-01T01:17:40.308 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 14649, 2023-12-01T01:17:40.308 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2023-12-01T01:17:40.308 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2023-12-01T01:17:40.308 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2023-12-01T01:17:40.308 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 68.703941725999996, 2023-12-01T01:17:40.308 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 27, 2023-12-01T01:17:40.308 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 84, 2023-12-01T01:17:40.308 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 82, 2023-12-01T01:17:40.308 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 82.690674607999995 2023-12-01T01:17:40.308 INFO:teuthology.orchestra.run.smithi077.stdout:} 2023-12-01T01:17:40.309 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.297+0000 7f803ccf4700 1 -- 172.21.15.77:0/3472203035 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f80200a05b0 msgr2=0x7f80200a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.309 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.297+0000 7f803ccf4700 1 --2- 172.21.15.77:0/3472203035 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f80200a05b0 0x7f80200a2a70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f802c005d90 tx=0x7f802c005d00 comp rx=0 tx=0).stop 2023-12-01T01:17:40.309 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.297+0000 7f803ccf4700 1 -- 172.21.15.77:0/3472203035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80381008d0 msgr2=0x7f8038196f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.309 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.297+0000 7f803ccf4700 1 --2- 172.21.15.77:0/3472203035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80381008d0 0x7f8038196f70 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8028000c00 tx=0x7f802800f710 comp rx=0 tx=0).stop 2023-12-01T01:17:40.309 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.297+0000 7f803ccf4700 1 -- 172.21.15.77:0/3472203035 shutdown_connections 2023-12-01T01:17:40.309 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.297+0000 7f803ccf4700 1 --2- 172.21.15.77:0/3472203035 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f80200a05b0 0x7f80200a2a70 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.309 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.297+0000 7f803ccf4700 1 --2- 172.21.15.77:0/3472203035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8038101200 0x7f803819dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.310 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.297+0000 7f803ccf4700 1 --2- 172.21.15.77:0/3472203035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80380fff30 0x7f8038196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.310 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.297+0000 7f803ccf4700 1 --2- 172.21.15.77:0/3472203035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80381008d0 0x7f8038196f70 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.310 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.297+0000 7f803ccf4700 1 -- 172.21.15.77:0/3472203035 >> 172.21.15.77:0/3472203035 conn(0x7f80380fa960 msgr2=0x7f80380fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:40.310 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.297+0000 7f803ccf4700 1 -- 172.21.15.77:0/3472203035 shutdown_connections 2023-12-01T01:17:40.310 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.297+0000 7f803ccf4700 1 -- 172.21.15.77:0/3472203035 wait complete. 2023-12-01T01:17:40.311 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-12-01T01:17:40.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.431+0000 7f978c27e700 1 -- 172.21.15.77:0/398845546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97840ffcb0 msgr2=0x7f97841000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.431+0000 7f978c27e700 1 --2- 172.21.15.77:0/398845546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97840ffcb0 0x7f97841000d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f97800099c0 tx=0x7f9780009cd0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.431+0000 7f978c27e700 1 -- 172.21.15.77:0/398845546 shutdown_connections 2023-12-01T01:17:40.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.431+0000 7f978c27e700 1 --2- 172.21.15.77:0/398845546 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9784101840 0x7f97841080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.431+0000 7f978c27e700 1 --2- 172.21.15.77:0/398845546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97840ffcb0 0x7f97841000d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.431+0000 7f978c27e700 1 --2- 172.21.15.77:0/398845546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9784100ea0 0x7f9784101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.431+0000 7f978c27e700 1 -- 172.21.15.77:0/398845546 >> 172.21.15.77:0/398845546 conn(0x7f97840fb220 msgr2=0x7f97840fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:40.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.431+0000 7f978c27e700 1 -- 172.21.15.77:0/398845546 shutdown_connections 2023-12-01T01:17:40.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.431+0000 7f978c27e700 1 -- 172.21.15.77:0/398845546 wait complete. 2023-12-01T01:17:40.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.432+0000 7f978c27e700 1 Processor -- start 2023-12-01T01:17:40.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.432+0000 7f978c27e700 1 -- start start 2023-12-01T01:17:40.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.432+0000 7f978c27e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97840ffcb0 0x7f9784196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.432+0000 7f978c27e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9784100ea0 0x7f97841970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.432+0000 7f978c27e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9784101840 0x7f978419e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.432+0000 7f978c27e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978419e6e0 con 0x7f97840ffcb0 2023-12-01T01:17:40.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.432+0000 7f978c27e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978419e820 con 0x7f9784101840 2023-12-01T01:17:40.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.432+0000 7f978c27e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978419e9b0 con 0x7f9784100ea0 2023-12-01T01:17:40.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.432+0000 7f978a01a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97840ffcb0 0x7f9784196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.432+0000 7f978a01a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97840ffcb0 0x7f9784196b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58802/0 (socket says 172.21.15.77:58802) 2023-12-01T01:17:40.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.432+0000 7f978a01a700 1 -- 172.21.15.77:0/2055794943 learned_addr learned my addr 172.21.15.77:0/2055794943 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:40.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.432+0000 7f9789819700 1 --2- 172.21.15.77:0/2055794943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9784100ea0 0x7f97841970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.432+0000 7f978a81b700 1 --2- 172.21.15.77:0/2055794943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9784101840 0x7f978419e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.432+0000 7f978a01a700 1 -- 172.21.15.77:0/2055794943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9784100ea0 msgr2=0x7f97841970a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.432+0000 7f978a01a700 1 --2- 172.21.15.77:0/2055794943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9784100ea0 0x7f97841970a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.432+0000 7f978a01a700 1 -- 172.21.15.77:0/2055794943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9784101840 msgr2=0x7f978419e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.432+0000 7f978a01a700 1 --2- 172.21.15.77:0/2055794943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9784101840 0x7f978419e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.433+0000 7f978a01a700 1 -- 172.21.15.77:0/2055794943 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97800095f0 con 0x7f97840ffcb0 2023-12-01T01:17:40.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.433+0000 7f978a81b700 1 --2- 172.21.15.77:0/2055794943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9784101840 0x7f978419e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:17:40.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.433+0000 7f978a01a700 1 --2- 172.21.15.77:0/2055794943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97840ffcb0 0x7f9784196b60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9780009990 tx=0x7f9780003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:40.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.433+0000 7f97777fe700 1 -- 172.21.15.77:0/2055794943 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978001d070 con 0x7f97840ffcb0 2023-12-01T01:17:40.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.433+0000 7f97777fe700 1 -- 172.21.15.77:0/2055794943 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9780004640 con 0x7f97840ffcb0 2023-12-01T01:17:40.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.433+0000 7f978c27e700 1 -- 172.21.15.77:0/2055794943 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f978419ec30 con 0x7f97840ffcb0 2023-12-01T01:17:40.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.433+0000 7f97777fe700 1 -- 172.21.15.77:0/2055794943 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978000f460 con 0x7f97840ffcb0 2023-12-01T01:17:40.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.433+0000 7f978c27e700 1 -- 172.21.15.77:0/2055794943 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f978419f0e0 con 0x7f97840ffcb0 2023-12-01T01:17:40.443 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.435+0000 7f97777fe700 1 -- 172.21.15.77:0/2055794943 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f9780004050 con 0x7f97840ffcb0 2023-12-01T01:17:40.443 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.435+0000 7f97777fe700 1 --2- 172.21.15.77:0/2055794943 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f97700a05b0 0x7f97700a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.443 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.436+0000 7f9789819700 1 --2- 172.21.15.77:0/2055794943 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f97700a05b0 0x7f97700a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.443 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.436+0000 7f97777fe700 1 -- 172.21.15.77:0/2055794943 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f97800cd060 con 0x7f97840ffcb0 2023-12-01T01:17:40.444 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.436+0000 7f9789819700 1 --2- 172.21.15.77:0/2055794943 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f97700a05b0 0x7f97700a2a70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9778006fd0 tx=0x7f9778008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:40.444 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.436+0000 7f979840eb80 1 Processor -- start 2023-12-01T01:17:40.444 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.437+0000 7f979840eb80 1 -- start start 2023-12-01T01:17:40.444 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.437+0000 7f979840eb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b5dee30a0 0x561b5dee34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.444 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.437+0000 7f979840eb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561b5dee3a20 0x561b5dee1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.444 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.437+0000 7f979840eb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561b5dee23a0 0x561b5dee2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.444 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.437+0000 7f979840eb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b5de399c0 con 0x561b5dee30a0 2023-12-01T01:17:40.444 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.437+0000 7f979840eb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b5de37380 con 0x561b5dee23a0 2023-12-01T01:17:40.444 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.437+0000 7f979840eb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b5de32c70 con 0x561b5dee3a20 2023-12-01T01:17:40.444 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.437+0000 7f978a01a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b5dee30a0 0x561b5dee34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.444 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.437+0000 7f978a01a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b5dee30a0 0x561b5dee34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58810/0 (socket says 172.21.15.77:58810) 2023-12-01T01:17:40.444 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.437+0000 7f978a01a700 1 -- 172.21.15.77:0/4212119901 learned_addr learned my addr 172.21.15.77:0/4212119901 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:40.444 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.437+0000 7f978a81b700 1 --2- 172.21.15.77:0/4212119901 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561b5dee23a0 0x561b5dee2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.445 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.437+0000 7f978a01a700 1 -- 172.21.15.77:0/4212119901 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561b5dee3a20 msgr2=0x561b5dee1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:40.445 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.437+0000 7f978a01a700 1 --2- 172.21.15.77:0/4212119901 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561b5dee3a20 0x561b5dee1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.445 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.437+0000 7f978a01a700 1 -- 172.21.15.77:0/4212119901 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561b5dee23a0 msgr2=0x561b5dee2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:40.445 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.437+0000 7f978a01a700 1 --2- 172.21.15.77:0/4212119901 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561b5dee23a0 0x561b5dee2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.445 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.437+0000 7f978a01a700 1 -- 172.21.15.77:0/4212119901 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b5dc60d90 con 0x561b5dee30a0 2023-12-01T01:17:40.445 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.437+0000 7f978a81b700 1 --2- 172.21.15.77:0/4212119901 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561b5dee23a0 0x561b5dee2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:17:40.445 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.437+0000 7f978a01a700 1 --2- 172.21.15.77:0/4212119901 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b5dee30a0 0x561b5dee34e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f978009a780 tx=0x7f97800059a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64877775e14b57d3 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:40.445 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.437+0000 7f9774da9700 1 -- 172.21.15.77:0/4212119901 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978009a7f0 con 0x561b5dee30a0 2023-12-01T01:17:40.445 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.438+0000 7f9774da9700 1 -- 172.21.15.77:0/4212119901 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f978009e620 con 0x561b5dee30a0 2023-12-01T01:17:40.445 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.438+0000 7f9774da9700 1 -- 172.21.15.77:0/4212119901 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97800a2340 con 0x561b5dee30a0 2023-12-01T01:17:40.445 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.438+0000 7f979840eb80 1 -- 172.21.15.77:0/4212119901 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b5dee30a0 msgr2=0x561b5dee34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.445 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.438+0000 7f979840eb80 1 --2- 172.21.15.77:0/4212119901 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b5dee30a0 0x561b5dee34e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f978009a780 tx=0x7f97800059a0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.445 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.438+0000 7f979840eb80 1 -- 172.21.15.77:0/4212119901 shutdown_connections 2023-12-01T01:17:40.446 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.438+0000 7f979840eb80 1 --2- 172.21.15.77:0/4212119901 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561b5dee23a0 0x561b5dee2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.446 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.438+0000 7f979840eb80 1 --2- 172.21.15.77:0/4212119901 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b5dee30a0 0x561b5dee34e0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.446 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.438+0000 7f979840eb80 1 --2- 172.21.15.77:0/4212119901 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561b5dee3a20 0x561b5dee1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.446 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.438+0000 7f979840eb80 1 -- 172.21.15.77:0/4212119901 >> 172.21.15.77:0/4212119901 conn(0x561b5de41080 msgr2=0x561b5dedc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:40.446 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.438+0000 7f979840eb80 1 -- 172.21.15.77:0/4212119901 shutdown_connections 2023-12-01T01:17:40.446 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.438+0000 7f979840eb80 1 -- 172.21.15.77:0/4212119901 wait complete. 2023-12-01T01:17:40.446 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.439+0000 7f979840eb80 1 -- start start 2023-12-01T01:17:40.446 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.439+0000 7f979840eb80 1 Processor -- start 2023-12-01T01:17:40.446 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.439+0000 7f979840eb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b5dee23a0 0x561b5deeb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.446 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.439+0000 7f979840eb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561b5dee30a0 0x561b5deebba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.446 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.439+0000 7f979840eb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561b5dee3a20 0x561b5def2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.446 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.439+0000 7f979840eb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b5ddcf4c0 con 0x561b5dee23a0 2023-12-01T01:17:40.446 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.439+0000 7f979840eb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b5ddcf160 con 0x561b5dee3a20 2023-12-01T01:17:40.446 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.439+0000 7f979840eb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b5dcb68d0 con 0x561b5dee30a0 2023-12-01T01:17:40.447 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.439+0000 7f978a01a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b5dee23a0 0x561b5deeb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.447 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.439+0000 7f979840eb80 10 client.0 _ll_register_callbacks cb 0x561b5dd1ca30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:17:40.447 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.439+0000 7f978a01a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b5dee23a0 0x561b5deeb660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58816/0 (socket says 172.21.15.77:58816) 2023-12-01T01:17:40.447 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.439+0000 7f978a01a700 1 -- 172.21.15.77:0/3548503044 learned_addr learned my addr 172.21.15.77:0/3548503044 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:40.447 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.439+0000 7f978a01a700 1 -- 172.21.15.77:0/3548503044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561b5dee30a0 msgr2=0x561b5deebba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:40.447 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.439+0000 7f978a01a700 1 --2- 172.21.15.77:0/3548503044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561b5dee30a0 0x561b5deebba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.447 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.439+0000 7f978a01a700 1 -- 172.21.15.77:0/3548503044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561b5dee3a20 msgr2=0x561b5def2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:40.447 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.439+0000 7f978a01a700 1 --2- 172.21.15.77:0/3548503044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561b5dee3a20 0x561b5def2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.447 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.439+0000 7f978a01a700 1 -- 172.21.15.77:0/3548503044 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f978419f0e0 con 0x561b5dee23a0 2023-12-01T01:17:40.447 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.439+0000 7f978a01a700 1 --2- 172.21.15.77:0/3548503044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b5dee23a0 0x561b5deeb660 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9780009f50 tx=0x7f9780000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68648a0c9fb60070 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:40.447 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.439+0000 7f976dffb700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:17:40.447 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.439+0000 7f976dffb700 1 -- 172.21.15.77:0/3548503044 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978009a7f0 con 0x561b5dee23a0 2023-12-01T01:17:40.447 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.440+0000 7f976dffb700 1 -- 172.21.15.77:0/3548503044 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f9780096260 con 0x561b5dee23a0 2023-12-01T01:17:40.447 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.440+0000 7f979840eb80 1 -- 172.21.15.77:0/3548503044 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561b5de5dcc0 con 0x561b5dee23a0 2023-12-01T01:17:40.448 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.440+0000 7f976dffb700 1 -- 172.21.15.77:0/3548503044 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f978009e120 con 0x561b5dee23a0 2023-12-01T01:17:40.448 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.440+0000 7f979840eb80 10 client.15276 fetch_fsmap learned FSMap version 27 2023-12-01T01:17:40.448 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.440+0000 7f979840eb80 1 -- 172.21.15.77:0/3548503044 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561b5dc49c10 con 0x561b5dee23a0 2023-12-01T01:17:40.448 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.440+0000 7f976dffb700 1 -- 172.21.15.77:0/3548503044 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f978007d020 con 0x561b5dee23a0 2023-12-01T01:17:40.448 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.440+0000 7f976dffb700 1 -- 172.21.15.77:0/3548503044 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f97800965e0 con 0x561b5dee23a0 2023-12-01T01:17:40.448 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.440+0000 7f979840eb80 10 client.15276 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:17:40.448 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.440+0000 7f979840eb80 10 client.15276 resolve_mds: resolved name 'h' to daemon mds.h 2023-12-01T01:17:40.448 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.440+0000 7f979840eb80 20 client.15276 populate_metadata read hostname 'smithi077' 2023-12-01T01:17:40.448 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.440+0000 7f979840eb80 1 --2- 172.21.15.77:0/3548503044 >> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] conn(0x561b5def6440 0x561b5def5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.448 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.440+0000 7f979840eb80 4 client.15276 mds_command: new command op to 24473 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:17:40.448 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.440+0000 7f979840eb80 1 -- 172.21.15.77:0/3548503044 --> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561b5deb0dc0 con 0x561b5def6440 2023-12-01T01:17:40.448 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.440+0000 7f9789819700 1 --2- 172.21.15.77:0/3548503044 >> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] conn(0x561b5def6440 0x561b5def5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.448 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.441+0000 7f976dffb700 10 client.15276 ms_handle_connect on v2:172.21.15.137:6836/3518881867 2023-12-01T01:17:40.449 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.441+0000 7f9789819700 1 --2- 172.21.15.77:0/3548503044 >> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] conn(0x561b5def6440 0x561b5def5a80 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1244571fc056e3a4 server_cookie=9f13f83a33ce1f21 in_seq=0 out_seq=0 2023-12-01T01:17:40.449 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.442+0000 7f976dffb700 1 -- 172.21.15.77:0/3548503044 <== mds.3 v2:172.21.15.137:6836/3518881867 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x561b5deb0dc0 con 0x561b5def6440 2023-12-01T01:17:40.449 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.442+0000 7f976dffb700 10 client.15276 handle_command_reply: tid=0 2023-12-01T01:17:40.449 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.442+0000 7f979840eb80 1 client.15276 shutdown 2023-12-01T01:17:40.449 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.442+0000 7f979840eb80 1 -- 172.21.15.77:0/3548503044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b5dee23a0 msgr2=0x561b5deeb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.449 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.442+0000 7f979840eb80 1 --2- 172.21.15.77:0/3548503044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b5dee23a0 0x561b5deeb660 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9780009f50 tx=0x7f9780000bc0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.449 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.442+0000 7f979840eb80 1 -- 172.21.15.77:0/3548503044 shutdown_connections 2023-12-01T01:17:40.449 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.442+0000 7f979840eb80 1 --2- 172.21.15.77:0/3548503044 >> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] conn(0x561b5def6440 0x561b5def5a80 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.449 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.442+0000 7f979840eb80 1 --2- 172.21.15.77:0/3548503044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561b5dee3a20 0x561b5def2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.449 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.442+0000 7f979840eb80 1 --2- 172.21.15.77:0/3548503044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b5dee23a0 0x561b5deeb660 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.449 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.442+0000 7f976dffb700 0 client.15276 ms_handle_reset on v2:172.21.15.137:6836/3518881867 2023-12-01T01:17:40.450 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.442+0000 7f979840eb80 1 --2- 172.21.15.77:0/3548503044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561b5dee30a0 0x561b5deebba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.450 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.442+0000 7f979840eb80 1 -- 172.21.15.77:0/3548503044 >> 172.21.15.77:0/3548503044 conn(0x561b5de41080 msgr2=0x561b5dedc060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:40.450 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.442+0000 7f979840eb80 1 -- 172.21.15.77:0/3548503044 shutdown_connections 2023-12-01T01:17:40.450 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.442+0000 7f979840eb80 1 -- 172.21.15.77:0/3548503044 wait complete. 2023-12-01T01:17:40.450 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.442+0000 7f979840eb80 20 client.15276 trim_cache size 0 max 16384 2023-12-01T01:17:40.465 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.459+0000 7f979840eb80 1 Processor -- start 2023-12-01T01:17:40.465 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.459+0000 7f979840eb80 1 -- start start 2023-12-01T01:17:40.466 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.459+0000 7f979840eb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b5dee23a0 0x561b5dee1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.466 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.459+0000 7f979840eb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561b5dee30a0 0x561b5deff1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.466 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.459+0000 7f979840eb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561b5dee3a20 0x561b5dee6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.466 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.459+0000 7f979840eb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b5de32c70 con 0x561b5dee23a0 2023-12-01T01:17:40.466 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.459+0000 7f979840eb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b5de399c0 con 0x561b5dee3a20 2023-12-01T01:17:40.466 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.459+0000 7f979840eb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b5dedd810 con 0x561b5dee30a0 2023-12-01T01:17:40.466 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.459+0000 7f978a01a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b5dee23a0 0x561b5dee1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.466 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.459+0000 7f978a01a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b5dee23a0 0x561b5dee1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58824/0 (socket says 172.21.15.77:58824) 2023-12-01T01:17:40.466 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.459+0000 7f978a01a700 1 -- 172.21.15.77:0/4233082195 learned_addr learned my addr 172.21.15.77:0/4233082195 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:40.466 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.459+0000 7f978a81b700 1 --2- 172.21.15.77:0/4233082195 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561b5dee3a20 0x561b5dee6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.466 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.459+0000 7f978a01a700 1 -- 172.21.15.77:0/4233082195 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561b5dee30a0 msgr2=0x561b5deff1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:40.466 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.459+0000 7f9789819700 1 --2- 172.21.15.77:0/4233082195 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561b5dee30a0 0x561b5deff1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.467 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.459+0000 7f978a01a700 1 --2- 172.21.15.77:0/4233082195 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561b5dee30a0 0x561b5deff1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.467 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.459+0000 7f978a01a700 1 -- 172.21.15.77:0/4233082195 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561b5dee3a20 msgr2=0x561b5dee6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:40.467 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.459+0000 7f978a01a700 1 --2- 172.21.15.77:0/4233082195 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561b5dee3a20 0x561b5dee6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.467 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.459+0000 7f978a01a700 1 -- 172.21.15.77:0/4233082195 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b5dc87200 con 0x561b5dee23a0 2023-12-01T01:17:40.467 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.459+0000 7f978a81b700 1 --2- 172.21.15.77:0/4233082195 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561b5dee3a20 0x561b5dee6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:17:40.467 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.459+0000 7f9789819700 1 --2- 172.21.15.77:0/4233082195 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561b5dee30a0 0x561b5deff1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:17:40.467 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.460+0000 7f978a01a700 1 --2- 172.21.15.77:0/4233082195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b5dee23a0 0x561b5dee1e10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9780012040 tx=0x7f9780009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece4daa2960c1f9a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:40.467 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.460+0000 7f97755aa700 1 -- 172.21.15.77:0/4233082195 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9780096260 con 0x561b5dee23a0 2023-12-01T01:17:40.467 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.460+0000 7f97755aa700 1 -- 172.21.15.77:0/4233082195 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f978007d030 con 0x561b5dee23a0 2023-12-01T01:17:40.467 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.460+0000 7f97755aa700 1 -- 172.21.15.77:0/4233082195 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97800a0630 con 0x561b5dee23a0 2023-12-01T01:17:40.467 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.460+0000 7f979840eb80 1 -- 172.21.15.77:0/4233082195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b5dee23a0 msgr2=0x561b5dee1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.467 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.460+0000 7f979840eb80 1 --2- 172.21.15.77:0/4233082195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b5dee23a0 0x561b5dee1e10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9780012040 tx=0x7f9780009310 comp rx=0 tx=0).stop 2023-12-01T01:17:40.467 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.460+0000 7f979840eb80 1 -- 172.21.15.77:0/4233082195 shutdown_connections 2023-12-01T01:17:40.467 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.460+0000 7f979840eb80 1 --2- 172.21.15.77:0/4233082195 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561b5dee3a20 0x561b5dee6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.467 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.460+0000 7f979840eb80 1 --2- 172.21.15.77:0/4233082195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b5dee23a0 0x561b5dee1e10 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.468 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.460+0000 7f979840eb80 1 --2- 172.21.15.77:0/4233082195 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561b5dee30a0 0x561b5deff1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.468 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.460+0000 7f979840eb80 1 -- 172.21.15.77:0/4233082195 >> 172.21.15.77:0/4233082195 conn(0x561b5de41080 msgr2=0x561b5def5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:40.468 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.460+0000 7f979840eb80 1 -- 172.21.15.77:0/4233082195 shutdown_connections 2023-12-01T01:17:40.468 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.460+0000 7f979840eb80 1 -- 172.21.15.77:0/4233082195 wait complete. 2023-12-01T01:17:40.468 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.460+0000 7f979840eb80 1 -- start start 2023-12-01T01:17:40.468 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.461+0000 7f979840eb80 1 Processor -- start 2023-12-01T01:17:40.468 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.461+0000 7f979840eb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561b5dee23a0 0x561b5defeff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.468 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.461+0000 7f979840eb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561b5dee30a0 0x561b5defba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.468 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.461+0000 7f979840eb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b5dee3a20 0x561b5defbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.468 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.461+0000 7f979840eb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b5defc620 con 0x561b5dee3a20 2023-12-01T01:17:40.468 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.461+0000 7f979840eb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b5df47f50 con 0x561b5dee30a0 2023-12-01T01:17:40.468 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.461+0000 7f979840eb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b5df48090 con 0x561b5dee23a0 2023-12-01T01:17:40.468 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.461+0000 7f979840eb80 10 client.0 _ll_register_callbacks cb 0x7f978009a330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:17:40.468 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.461+0000 7f978a81b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b5dee3a20 0x561b5defbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.469 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.461+0000 7f978a81b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b5dee3a20 0x561b5defbfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58834/0 (socket says 172.21.15.77:58834) 2023-12-01T01:17:40.469 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.461+0000 7f978a81b700 1 -- 172.21.15.77:0/1115349173 learned_addr learned my addr 172.21.15.77:0/1115349173 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:40.469 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.461+0000 7f978a01a700 1 --2- 172.21.15.77:0/1115349173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561b5dee23a0 0x561b5defeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.469 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.461+0000 7f978a81b700 1 -- 172.21.15.77:0/1115349173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561b5dee23a0 msgr2=0x561b5defeff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:40.469 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.461+0000 7f978a81b700 1 --2- 172.21.15.77:0/1115349173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561b5dee23a0 0x561b5defeff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.469 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.461+0000 7f978a81b700 1 -- 172.21.15.77:0/1115349173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561b5dee30a0 msgr2=0x561b5defba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:40.469 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.461+0000 7f978a81b700 1 --2- 172.21.15.77:0/1115349173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561b5dee30a0 0x561b5defba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.469 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.461+0000 7f978a81b700 1 -- 172.21.15.77:0/1115349173 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561b5dc49c10 con 0x561b5dee3a20 2023-12-01T01:17:40.469 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.461+0000 7f978a01a700 1 --2- 172.21.15.77:0/1115349173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561b5dee23a0 0x561b5defeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:17:40.469 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.461+0000 7f978a81b700 1 --2- 172.21.15.77:0/1115349173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b5dee3a20 0x561b5defbfd0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f977c00e9f0 tx=0x7f977c00edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67cd7b3c27989f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:40.469 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.461+0000 7f976dffb700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:17:40.469 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.461+0000 7f976dffb700 1 -- 172.21.15.77:0/1115349173 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977c00cc20 con 0x561b5dee3a20 2023-12-01T01:17:40.469 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.461+0000 7f976dffb700 1 -- 172.21.15.77:0/1115349173 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f977c006e40 con 0x561b5dee3a20 2023-12-01T01:17:40.470 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.461+0000 7f979840eb80 1 -- 172.21.15.77:0/1115349173 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f978009e620 con 0x561b5dee3a20 2023-12-01T01:17:40.470 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.462+0000 7f976dffb700 1 -- 172.21.15.77:0/1115349173 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f977c00cd80 con 0x561b5dee3a20 2023-12-01T01:17:40.470 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.462+0000 7f979840eb80 10 client.15282 fetch_fsmap learned FSMap version 27 2023-12-01T01:17:40.470 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.462+0000 7f979840eb80 1 -- 172.21.15.77:0/1115349173 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561b5dc5f260 con 0x561b5dee3a20 2023-12-01T01:17:40.470 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.462+0000 7f976dffb700 1 -- 172.21.15.77:0/1115349173 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f978009e620 con 0x561b5dee3a20 2023-12-01T01:17:40.470 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.462+0000 7f976dffb700 1 -- 172.21.15.77:0/1115349173 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f977c022470 con 0x561b5dee3a20 2023-12-01T01:17:40.470 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.462+0000 7f979840eb80 10 client.15282 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:17:40.470 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.462+0000 7f979840eb80 10 client.15282 resolve_mds: resolved name 'h' to daemon mds.h 2023-12-01T01:17:40.470 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.462+0000 7f979840eb80 20 client.15282 populate_metadata read hostname 'smithi077' 2023-12-01T01:17:40.470 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.462+0000 7f979840eb80 1 --2- 172.21.15.77:0/1115349173 >> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] conn(0x561b5df4ce40 0x561b5df497e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.470 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.462+0000 7f979840eb80 4 client.15282 mds_command: new command op to 24473 tid=0[{"prefix": "status"}] 2023-12-01T01:17:40.470 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.462+0000 7f979840eb80 1 -- 172.21.15.77:0/1115349173 --> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561b5de2bcf0 con 0x561b5df4ce40 2023-12-01T01:17:40.470 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.462+0000 7f978a01a700 1 --2- 172.21.15.77:0/1115349173 >> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] conn(0x561b5df4ce40 0x561b5df497e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.470 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.463+0000 7f978a01a700 1 --2- 172.21.15.77:0/1115349173 >> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] conn(0x561b5df4ce40 0x561b5df497e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=11fdcb7c927ddb98 server_cookie=2f3c5419867a0f9f in_seq=0 out_seq=0 2023-12-01T01:17:40.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.463+0000 7f976dffb700 10 client.15282 ms_handle_connect on v2:172.21.15.137:6836/3518881867 2023-12-01T01:17:40.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.463+0000 7f976dffb700 1 -- 172.21.15.77:0/1115349173 <== mds.3 v2:172.21.15.137:6836/3518881867 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x561b5de2bcf0 con 0x561b5df4ce40 2023-12-01T01:17:40.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.463+0000 7f976dffb700 10 client.15282 handle_command_reply: tid=0 2023-12-01T01:17:40.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.463+0000 7f979840eb80 1 client.15282 shutdown 2023-12-01T01:17:40.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.463+0000 7f979840eb80 1 -- 172.21.15.77:0/1115349173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b5dee3a20 msgr2=0x561b5defbfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.463+0000 7f979840eb80 1 --2- 172.21.15.77:0/1115349173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b5dee3a20 0x561b5defbfd0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f977c00e9f0 tx=0x7f977c00edb0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.463+0000 7f979840eb80 1 -- 172.21.15.77:0/1115349173 shutdown_connections 2023-12-01T01:17:40.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.463+0000 7f979840eb80 1 --2- 172.21.15.77:0/1115349173 >> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] conn(0x561b5df4ce40 0x561b5df497e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.463+0000 7f979840eb80 1 --2- 172.21.15.77:0/1115349173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b5dee3a20 0x561b5defbfd0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.463+0000 7f976dffb700 0 client.15282 ms_handle_reset on v2:172.21.15.137:6836/3518881867 2023-12-01T01:17:40.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.463+0000 7f979840eb80 1 --2- 172.21.15.77:0/1115349173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561b5dee23a0 0x561b5defeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.463+0000 7f979840eb80 1 --2- 172.21.15.77:0/1115349173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561b5dee30a0 0x561b5defba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.472 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.463+0000 7f979840eb80 1 -- 172.21.15.77:0/1115349173 >> 172.21.15.77:0/1115349173 conn(0x561b5de41080 msgr2=0x561b5dee6360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:40.472 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.463+0000 7f979840eb80 1 -- 172.21.15.77:0/1115349173 shutdown_connections 2023-12-01T01:17:40.472 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.463+0000 7f979840eb80 1 -- 172.21.15.77:0/1115349173 wait complete. 2023-12-01T01:17:40.472 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.463+0000 7f979840eb80 20 client.15282 trim_cache size 0 max 16384 2023-12-01T01:17:40.472 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2023-12-01T01:17:40.472 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "2a06e29a-8fe6-11ee-95a2-87774f69a715", 2023-12-01T01:17:40.472 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 3, 2023-12-01T01:17:40.472 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24473, 2023-12-01T01:17:40.472 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2023-12-01T01:17:40.472 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2023-12-01T01:17:40.472 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2023-12-01T01:17:40.473 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 69.819798685999999, 2023-12-01T01:17:40.473 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 27, 2023-12-01T01:17:40.473 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 84, 2023-12-01T01:17:40.473 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 82, 2023-12-01T01:17:40.473 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 89.262788369000006 2023-12-01T01:17:40.473 INFO:teuthology.orchestra.run.smithi077.stdout:} 2023-12-01T01:17:40.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.464+0000 7f978c27e700 1 -- 172.21.15.77:0/2055794943 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f97700a05b0 msgr2=0x7f97700a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.464+0000 7f978c27e700 1 --2- 172.21.15.77:0/2055794943 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f97700a05b0 0x7f97700a2a70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9778006fd0 tx=0x7f9778008040 comp rx=0 tx=0).stop 2023-12-01T01:17:40.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.464+0000 7f978c27e700 1 -- 172.21.15.77:0/2055794943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97840ffcb0 msgr2=0x7f9784196b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.464+0000 7f978c27e700 1 --2- 172.21.15.77:0/2055794943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97840ffcb0 0x7f9784196b60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9780009990 tx=0x7f9780003ba0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.464+0000 7f978c27e700 1 -- 172.21.15.77:0/2055794943 shutdown_connections 2023-12-01T01:17:40.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.464+0000 7f978c27e700 1 --2- 172.21.15.77:0/2055794943 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f97700a05b0 0x7f97700a2a70 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.464+0000 7f978c27e700 1 --2- 172.21.15.77:0/2055794943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9784101840 0x7f978419e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.464+0000 7f978c27e700 1 --2- 172.21.15.77:0/2055794943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97840ffcb0 0x7f9784196b60 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.464+0000 7f978c27e700 1 --2- 172.21.15.77:0/2055794943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9784100ea0 0x7f97841970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.464+0000 7f978c27e700 1 -- 172.21.15.77:0/2055794943 >> 172.21.15.77:0/2055794943 conn(0x7f97840fb220 msgr2=0x7f97840fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:40.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.464+0000 7f978c27e700 1 -- 172.21.15.77:0/2055794943 shutdown_connections 2023-12-01T01:17:40.475 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.464+0000 7f978c27e700 1 -- 172.21.15.77:0/2055794943 wait complete. 2023-12-01T01:17:40.477 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-12-01T01:17:40.601 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.597+0000 7fe451b26700 1 -- 172.21.15.77:0/3141937318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe44c100e70 msgr2=0x7fe44c1012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.601 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.597+0000 7fe451b26700 1 --2- 172.21.15.77:0/3141937318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe44c100e70 0x7fe44c1012d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe43c0099c0 tx=0x7fe43c009cd0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.601 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.597+0000 7fe451b26700 1 -- 172.21.15.77:0/3141937318 shutdown_connections 2023-12-01T01:17:40.601 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.597+0000 7fe451b26700 1 --2- 172.21.15.77:0/3141937318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe44c101810 0x7fe44c1080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.601 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.597+0000 7fe451b26700 1 --2- 172.21.15.77:0/3141937318 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe44c0ffc80 0x7fe44c1000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.601 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.597+0000 7fe451b26700 1 --2- 172.21.15.77:0/3141937318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe44c100e70 0x7fe44c1012d0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.601 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.597+0000 7fe451b26700 1 -- 172.21.15.77:0/3141937318 >> 172.21.15.77:0/3141937318 conn(0x7fe44c0fb1f0 msgr2=0x7fe44c0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:40.601 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.598+0000 7fe451b26700 1 -- 172.21.15.77:0/3141937318 shutdown_connections 2023-12-01T01:17:40.601 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.598+0000 7fe451b26700 1 -- 172.21.15.77:0/3141937318 wait complete. 2023-12-01T01:17:40.601 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.598+0000 7fe451b26700 1 Processor -- start 2023-12-01T01:17:40.601 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.598+0000 7fe451b26700 1 -- start start 2023-12-01T01:17:40.601 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.598+0000 7fe451b26700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe44c0ffc80 0x7fe44c068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.602 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.598+0000 7fe451b26700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe44c100e70 0x7fe44c19b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.602 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.598+0000 7fe451b26700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe44c101810 0x7fe44c19df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.602 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.598+0000 7fe451b26700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe44c19e730 con 0x7fe44c100e70 2023-12-01T01:17:40.602 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.598+0000 7fe451b26700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe44c19e8a0 con 0x7fe44c0ffc80 2023-12-01T01:17:40.602 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.598+0000 7fe451b26700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe44c19ea60 con 0x7fe44c101810 2023-12-01T01:17:40.602 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.599+0000 7fe44affd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe44c100e70 0x7fe44c19b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.602 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.599+0000 7fe44affd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe44c100e70 0x7fe44c19b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58846/0 (socket says 172.21.15.77:58846) 2023-12-01T01:17:40.602 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.599+0000 7fe44affd700 1 -- 172.21.15.77:0/89231516 learned_addr learned my addr 172.21.15.77:0/89231516 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:40.602 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.599+0000 7fe44bfff700 1 --2- 172.21.15.77:0/89231516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe44c101810 0x7fe44c19df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.602 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.599+0000 7fe44b7fe700 1 --2- 172.21.15.77:0/89231516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe44c0ffc80 0x7fe44c068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.602 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.599+0000 7fe44affd700 1 -- 172.21.15.77:0/89231516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe44c101810 msgr2=0x7fe44c19df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.602 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.599+0000 7fe44affd700 1 --2- 172.21.15.77:0/89231516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe44c101810 0x7fe44c19df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.602 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.599+0000 7fe44affd700 1 -- 172.21.15.77:0/89231516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe44c0ffc80 msgr2=0x7fe44c068a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.602 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.599+0000 7fe44affd700 1 --2- 172.21.15.77:0/89231516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe44c0ffc80 0x7fe44c068a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.603 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.599+0000 7fe44affd700 1 -- 172.21.15.77:0/89231516 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe43c0095f0 con 0x7fe44c100e70 2023-12-01T01:17:40.603 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.599+0000 7fe44affd700 1 --2- 172.21.15.77:0/89231516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe44c100e70 0x7fe44c19b910 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009990 tx=0x7fe43c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:40.603 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.599+0000 7fe448ff9700 1 -- 172.21.15.77:0/89231516 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43c01d070 con 0x7fe44c100e70 2023-12-01T01:17:40.603 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.599+0000 7fe451b26700 1 -- 172.21.15.77:0/89231516 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe44c19ece0 con 0x7fe44c100e70 2023-12-01T01:17:40.603 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.599+0000 7fe448ff9700 1 -- 172.21.15.77:0/89231516 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe43c004640 con 0x7fe44c100e70 2023-12-01T01:17:40.603 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.599+0000 7fe448ff9700 1 -- 172.21.15.77:0/89231516 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43c00f460 con 0x7fe44c100e70 2023-12-01T01:17:40.603 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.599+0000 7fe451b26700 1 -- 172.21.15.77:0/89231516 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe44c19f190 con 0x7fe44c100e70 2023-12-01T01:17:40.608 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.601+0000 7fe448ff9700 1 -- 172.21.15.77:0/89231516 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fe43c022470 con 0x7fe44c100e70 2023-12-01T01:17:40.608 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.602+0000 7fe448ff9700 1 --2- 172.21.15.77:0/89231516 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fe4340a05b0 0x7fe4340a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.608 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.602+0000 7fe44b7fe700 1 --2- 172.21.15.77:0/89231516 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fe4340a05b0 0x7fe4340a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.608 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.602+0000 7fe448ff9700 1 -- 172.21.15.77:0/89231516 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fe43c0cd120 con 0x7fe44c100e70 2023-12-01T01:17:40.608 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.602+0000 7fe44b7fe700 1 --2- 172.21.15.77:0/89231516 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fe4340a05b0 0x7fe4340a2a70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe440006fd0 tx=0x7fe440008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:40.609 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.603+0000 7fe45dcb6b80 1 Processor -- start 2023-12-01T01:17:40.609 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.603+0000 7fe45dcb6b80 1 -- start start 2023-12-01T01:17:40.609 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.603+0000 7fe45dcb6b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56424449a0a0 0x56424449a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.609 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.603+0000 7fe45dcb6b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56424449aa20 0x564244498e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.609 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.603+0000 7fe45dcb6b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642444993a0 0x564244499820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.609 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.603+0000 7fe45dcb6b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642443f09c0 con 0x56424449aa20 2023-12-01T01:17:40.609 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.603+0000 7fe45dcb6b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642443ee380 con 0x5642444993a0 2023-12-01T01:17:40.609 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.603+0000 7fe45dcb6b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642443e9c70 con 0x56424449a0a0 2023-12-01T01:17:40.609 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.603+0000 7fe44affd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56424449aa20 0x564244498e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.609 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.603+0000 7fe44affd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56424449aa20 0x564244498e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58854/0 (socket says 172.21.15.77:58854) 2023-12-01T01:17:40.609 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.603+0000 7fe44affd700 1 -- 172.21.15.77:0/1920403526 learned_addr learned my addr 172.21.15.77:0/1920403526 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:40.609 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.603+0000 7fe44bfff700 1 --2- 172.21.15.77:0/1920403526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642444993a0 0x564244499820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.609 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.603+0000 7fe44b7fe700 1 --2- 172.21.15.77:0/1920403526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56424449a0a0 0x56424449a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.609 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.603+0000 7fe44affd700 1 -- 172.21.15.77:0/1920403526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56424449a0a0 msgr2=0x56424449a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:40.609 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.603+0000 7fe44affd700 1 --2- 172.21.15.77:0/1920403526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56424449a0a0 0x56424449a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.610 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.603+0000 7fe44affd700 1 -- 172.21.15.77:0/1920403526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642444993a0 msgr2=0x564244499820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:40.610 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.603+0000 7fe44affd700 1 --2- 172.21.15.77:0/1920403526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642444993a0 0x564244499820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.610 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.603+0000 7fe44affd700 1 -- 172.21.15.77:0/1920403526 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564244217d90 con 0x56424449aa20 2023-12-01T01:17:40.610 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.603+0000 7fe44affd700 1 --2- 172.21.15.77:0/1920403526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56424449aa20 0x564244498e10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe43c0a2360 tx=0x7fe43c009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd71a7760b8f2d55 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:40.610 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.604+0000 7fe431ffb700 1 -- 172.21.15.77:0/1920403526 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43c09a8b0 con 0x56424449aa20 2023-12-01T01:17:40.610 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.604+0000 7fe431ffb700 1 -- 172.21.15.77:0/1920403526 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe43c09e6e0 con 0x56424449aa20 2023-12-01T01:17:40.610 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.604+0000 7fe431ffb700 1 -- 172.21.15.77:0/1920403526 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43c09ad90 con 0x56424449aa20 2023-12-01T01:17:40.610 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.604+0000 7fe45dcb6b80 1 -- 172.21.15.77:0/1920403526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56424449aa20 msgr2=0x564244498e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.610 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.604+0000 7fe45dcb6b80 1 --2- 172.21.15.77:0/1920403526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56424449aa20 0x564244498e10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe43c0a2360 tx=0x7fe43c009cd0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.610 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.604+0000 7fe45dcb6b80 1 -- 172.21.15.77:0/1920403526 shutdown_connections 2023-12-01T01:17:40.610 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.604+0000 7fe45dcb6b80 1 --2- 172.21.15.77:0/1920403526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642444993a0 0x564244499820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.610 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.604+0000 7fe45dcb6b80 1 --2- 172.21.15.77:0/1920403526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56424449a0a0 0x56424449a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.610 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.604+0000 7fe45dcb6b80 1 --2- 172.21.15.77:0/1920403526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56424449aa20 0x564244498e10 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.610 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.604+0000 7fe45dcb6b80 1 -- 172.21.15.77:0/1920403526 >> 172.21.15.77:0/1920403526 conn(0x5642443f8080 msgr2=0x5642444938f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:40.611 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.604+0000 7fe45dcb6b80 1 -- 172.21.15.77:0/1920403526 shutdown_connections 2023-12-01T01:17:40.611 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.604+0000 7fe45dcb6b80 1 -- 172.21.15.77:0/1920403526 wait complete. 2023-12-01T01:17:40.611 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.605+0000 7fe45dcb6b80 1 -- start start 2023-12-01T01:17:40.611 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.605+0000 7fe45dcb6b80 1 Processor -- start 2023-12-01T01:17:40.611 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.605+0000 7fe45dcb6b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642444993a0 0x5642444a2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.611 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.605+0000 7fe45dcb6b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56424449a0a0 0x5642444a2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.611 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.605+0000 7fe45dcb6b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56424449aa20 0x5642444a9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.611 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.605+0000 7fe45dcb6b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642443864c0 con 0x56424449aa20 2023-12-01T01:17:40.611 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.605+0000 7fe45dcb6b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564244386160 con 0x56424449a0a0 2023-12-01T01:17:40.611 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.605+0000 7fe45dcb6b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56424426d8d0 con 0x5642444993a0 2023-12-01T01:17:40.611 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.605+0000 7fe45dcb6b80 10 client.0 _ll_register_callbacks cb 0x5642442d3a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:17:40.611 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.605+0000 7fe44bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56424449aa20 0x5642444a9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.611 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.605+0000 7fe44bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56424449aa20 0x5642444a9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58862/0 (socket says 172.21.15.77:58862) 2023-12-01T01:17:40.611 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.605+0000 7fe44bfff700 1 -- 172.21.15.77:0/1066219673 learned_addr learned my addr 172.21.15.77:0/1066219673 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:40.611 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.605+0000 7fe44b7fe700 1 --2- 172.21.15.77:0/1066219673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642444993a0 0x5642444a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.612 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.605+0000 7fe44affd700 1 --2- 172.21.15.77:0/1066219673 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56424449a0a0 0x5642444a2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.612 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.605+0000 7fe44bfff700 1 -- 172.21.15.77:0/1066219673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642444993a0 msgr2=0x5642444a2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:40.612 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.606+0000 7fe44bfff700 1 --2- 172.21.15.77:0/1066219673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642444993a0 0x5642444a2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.612 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.606+0000 7fe44bfff700 1 -- 172.21.15.77:0/1066219673 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56424449a0a0 msgr2=0x5642444a2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:40.612 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.606+0000 7fe44bfff700 1 --2- 172.21.15.77:0/1066219673 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56424449a0a0 0x5642444a2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.612 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.606+0000 7fe44bfff700 1 -- 172.21.15.77:0/1066219673 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe44c19f190 con 0x56424449aa20 2023-12-01T01:17:40.612 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.606+0000 7fe44bfff700 1 --2- 172.21.15.77:0/1066219673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56424449aa20 0x5642444a9b00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe44400be50 tx=0x7fe44400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c48192211cdef0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:40.612 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.606+0000 7fe42affd700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:17:40.612 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.606+0000 7fe42affd700 1 -- 172.21.15.77:0/1066219673 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe444013070 con 0x56424449aa20 2023-12-01T01:17:40.612 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.606+0000 7fe42affd700 1 -- 172.21.15.77:0/1066219673 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fe444006e40 con 0x56424449aa20 2023-12-01T01:17:40.612 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.606+0000 7fe45dcb6b80 1 -- 172.21.15.77:0/1066219673 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564244414cc0 con 0x56424449aa20 2023-12-01T01:17:40.612 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.606+0000 7fe42affd700 1 -- 172.21.15.77:0/1066219673 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe444007640 con 0x56424449aa20 2023-12-01T01:17:40.612 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.606+0000 7fe45dcb6b80 10 client.15294 fetch_fsmap learned FSMap version 27 2023-12-01T01:17:40.612 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.606+0000 7fe45dcb6b80 1 -- 172.21.15.77:0/1066219673 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564244200c10 con 0x56424449aa20 2023-12-01T01:17:40.612 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.606+0000 7fe42affd700 1 -- 172.21.15.77:0/1066219673 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564244414cc0 con 0x56424449aa20 2023-12-01T01:17:40.612 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.606+0000 7fe42affd700 1 -- 172.21.15.77:0/1066219673 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fe444024840 con 0x56424449aa20 2023-12-01T01:17:40.613 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.606+0000 7fe45dcb6b80 10 client.15294 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:17:40.613 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.606+0000 7fe45dcb6b80 10 client.15294 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-01T01:17:40.613 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.606+0000 7fe45dcb6b80 20 client.15294 populate_metadata read hostname 'smithi077' 2023-12-01T01:17:40.613 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.606+0000 7fe45dcb6b80 1 --2- 172.21.15.77:0/1066219673 >> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] conn(0x5642444ad860 0x5642444acc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.613 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.606+0000 7fe45dcb6b80 4 client.15294 mds_command: new command op to 24475 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:17:40.613 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.606+0000 7fe45dcb6b80 1 -- 172.21.15.77:0/1066219673 --> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564244467dc0 con 0x5642444ad860 2023-12-01T01:17:40.613 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.607+0000 7fe44b7fe700 1 --2- 172.21.15.77:0/1066219673 >> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] conn(0x5642444ad860 0x5642444acc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.613 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.607+0000 7fe42affd700 10 client.15294 ms_handle_connect on v2:172.21.15.203:6836/2887080977 2023-12-01T01:17:40.613 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.607+0000 7fe44b7fe700 1 --2- 172.21.15.77:0/1066219673 >> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] conn(0x5642444ad860 0x5642444acc10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=47c4bc9deb64afe3 server_cookie=6f7001ac073d3a45 in_seq=0 out_seq=0 2023-12-01T01:17:40.614 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.608+0000 7fe42affd700 1 -- 172.21.15.77:0/1066219673 <== mds.2 v2:172.21.15.203:6836/2887080977 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x564244467dc0 con 0x5642444ad860 2023-12-01T01:17:40.614 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.608+0000 7fe42affd700 10 client.15294 handle_command_reply: tid=0 2023-12-01T01:17:40.614 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.608+0000 7fe45dcb6b80 1 client.15294 shutdown 2023-12-01T01:17:40.614 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.608+0000 7fe45dcb6b80 1 -- 172.21.15.77:0/1066219673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56424449aa20 msgr2=0x5642444a9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.614 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.608+0000 7fe45dcb6b80 1 --2- 172.21.15.77:0/1066219673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56424449aa20 0x5642444a9b00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe44400be50 tx=0x7fe44400be80 comp rx=0 tx=0).stop 2023-12-01T01:17:40.614 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.608+0000 7fe45dcb6b80 1 -- 172.21.15.77:0/1066219673 shutdown_connections 2023-12-01T01:17:40.615 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.608+0000 7fe45dcb6b80 1 --2- 172.21.15.77:0/1066219673 >> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] conn(0x5642444ad860 0x5642444acc10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.615 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.608+0000 7fe45dcb6b80 1 --2- 172.21.15.77:0/1066219673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56424449aa20 0x5642444a9b00 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.615 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.608+0000 7fe45dcb6b80 1 --2- 172.21.15.77:0/1066219673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642444993a0 0x5642444a2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.615 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.608+0000 7fe42affd700 0 client.15294 ms_handle_reset on v2:172.21.15.203:6836/2887080977 2023-12-01T01:17:40.615 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.608+0000 7fe45dcb6b80 1 --2- 172.21.15.77:0/1066219673 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56424449a0a0 0x5642444a2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.615 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.608+0000 7fe45dcb6b80 1 -- 172.21.15.77:0/1066219673 >> 172.21.15.77:0/1066219673 conn(0x5642443f8080 msgr2=0x564244493060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:40.615 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.608+0000 7fe45dcb6b80 1 -- 172.21.15.77:0/1066219673 shutdown_connections 2023-12-01T01:17:40.615 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.608+0000 7fe45dcb6b80 1 -- 172.21.15.77:0/1066219673 wait complete. 2023-12-01T01:17:40.615 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.608+0000 7fe45dcb6b80 20 client.15294 trim_cache size 0 max 16384 2023-12-01T01:17:40.628 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.625+0000 7fe45dcb6b80 1 Processor -- start 2023-12-01T01:17:40.628 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.625+0000 7fe45dcb6b80 1 -- start start 2023-12-01T01:17:40.628 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.625+0000 7fe45dcb6b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642444993a0 0x564244498e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.628 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.625+0000 7fe45dcb6b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56424449a0a0 0x5642444b65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.628 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.625+0000 7fe45dcb6b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56424449aa20 0x5642444a2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.629 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.625+0000 7fe45dcb6b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642443e9c70 con 0x56424449a0a0 2023-12-01T01:17:40.629 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.626+0000 7fe45dcb6b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642443f09c0 con 0x56424449aa20 2023-12-01T01:17:40.629 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.626+0000 7fe45dcb6b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642444aed40 con 0x5642444993a0 2023-12-01T01:17:40.629 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.626+0000 7fe44affd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56424449a0a0 0x5642444b65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.629 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.626+0000 7fe44affd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56424449a0a0 0x5642444b65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58868/0 (socket says 172.21.15.77:58868) 2023-12-01T01:17:40.629 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.626+0000 7fe44affd700 1 -- 172.21.15.77:0/78678070 learned_addr learned my addr 172.21.15.77:0/78678070 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:40.629 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.626+0000 7fe44b7fe700 1 --2- 172.21.15.77:0/78678070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642444993a0 0x564244498e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.629 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.626+0000 7fe44bfff700 1 --2- 172.21.15.77:0/78678070 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56424449aa20 0x5642444a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.629 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.626+0000 7fe44affd700 1 -- 172.21.15.77:0/78678070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642444993a0 msgr2=0x564244498e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:40.629 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.626+0000 7fe44affd700 1 --2- 172.21.15.77:0/78678070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642444993a0 0x564244498e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.629 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.626+0000 7fe44affd700 1 -- 172.21.15.77:0/78678070 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56424449aa20 msgr2=0x5642444a2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:40.629 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.626+0000 7fe44affd700 1 --2- 172.21.15.77:0/78678070 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56424449aa20 0x5642444a2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.629 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.626+0000 7fe44affd700 1 -- 172.21.15.77:0/78678070 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56424423e200 con 0x56424449a0a0 2023-12-01T01:17:40.629 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.626+0000 7fe44affd700 1 --2- 172.21.15.77:0/78678070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56424449a0a0 0x5642444b65d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe43c029000 tx=0x7fe43c022aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=442678eee3024cf1 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:40.630 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.626+0000 7fe4327fc700 1 -- 172.21.15.77:0/78678070 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43c0a2890 con 0x56424449a0a0 2023-12-01T01:17:40.630 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.626+0000 7fe4327fc700 1 -- 172.21.15.77:0/78678070 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe43c0cae70 con 0x56424449a0a0 2023-12-01T01:17:40.630 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.626+0000 7fe4327fc700 1 -- 172.21.15.77:0/78678070 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43c09ab10 con 0x56424449a0a0 2023-12-01T01:17:40.630 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.627+0000 7fe45dcb6b80 1 -- 172.21.15.77:0/78678070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56424449a0a0 msgr2=0x5642444b65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.630 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.627+0000 7fe45dcb6b80 1 --2- 172.21.15.77:0/78678070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56424449a0a0 0x5642444b65d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe43c029000 tx=0x7fe43c022aa0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.630 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.627+0000 7fe45dcb6b80 1 -- 172.21.15.77:0/78678070 shutdown_connections 2023-12-01T01:17:40.630 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.627+0000 7fe45dcb6b80 1 --2- 172.21.15.77:0/78678070 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56424449aa20 0x5642444a2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.630 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.627+0000 7fe45dcb6b80 1 --2- 172.21.15.77:0/78678070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642444993a0 0x564244498e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.630 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.627+0000 7fe45dcb6b80 1 --2- 172.21.15.77:0/78678070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56424449a0a0 0x5642444b65d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.630 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.627+0000 7fe45dcb6b80 1 -- 172.21.15.77:0/78678070 >> 172.21.15.77:0/78678070 conn(0x5642443f8080 msgr2=0x56424449d280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:40.630 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.627+0000 7fe45dcb6b80 1 -- 172.21.15.77:0/78678070 shutdown_connections 2023-12-01T01:17:40.630 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.627+0000 7fe45dcb6b80 1 -- 172.21.15.77:0/78678070 wait complete. 2023-12-01T01:17:40.630 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.627+0000 7fe45dcb6b80 1 -- start start 2023-12-01T01:17:40.630 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.627+0000 7fe45dcb6b80 1 Processor -- start 2023-12-01T01:17:40.634 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.628+0000 7fe45dcb6b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642444993a0 0x5642444b1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.635 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.628+0000 7fe45dcb6b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56424449a0a0 0x5642444b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.635 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.628+0000 7fe45dcb6b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56424449aa20 0x5642444b4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.635 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.628+0000 7fe45dcb6b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642444b50a0 con 0x56424449a0a0 2023-12-01T01:17:40.635 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.628+0000 7fe45dcb6b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642444b5210 con 0x5642444993a0 2023-12-01T01:17:40.635 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.628+0000 7fe45dcb6b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642444b53d0 con 0x56424449aa20 2023-12-01T01:17:40.635 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.628+0000 7fe45dcb6b80 10 client.0 _ll_register_callbacks cb 0x7fe44400b7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:17:40.635 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.628+0000 7fe44affd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56424449a0a0 0x5642444b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.635 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.628+0000 7fe44affd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56424449a0a0 0x5642444b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58874/0 (socket says 172.21.15.77:58874) 2023-12-01T01:17:40.635 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.628+0000 7fe44affd700 1 -- 172.21.15.77:0/3455653796 learned_addr learned my addr 172.21.15.77:0/3455653796 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:40.635 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.628+0000 7fe44b7fe700 1 --2- 172.21.15.77:0/3455653796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642444993a0 0x5642444b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.635 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.628+0000 7fe44affd700 1 -- 172.21.15.77:0/3455653796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56424449aa20 msgr2=0x5642444b4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:40.635 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.628+0000 7fe44affd700 1 --2- 172.21.15.77:0/3455653796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56424449aa20 0x5642444b4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.635 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.628+0000 7fe44affd700 1 -- 172.21.15.77:0/3455653796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642444993a0 msgr2=0x5642444b1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:40.636 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.628+0000 7fe44affd700 1 --2- 172.21.15.77:0/3455653796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642444993a0 0x5642444b1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.636 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.628+0000 7fe44affd700 1 -- 172.21.15.77:0/3455653796 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564244217d90 con 0x56424449a0a0 2023-12-01T01:17:40.636 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.628+0000 7fe44affd700 1 --2- 172.21.15.77:0/3455653796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56424449a0a0 0x5642444b1b40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009fd0 tx=0x7fe43c004080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=339a59a0e67c03c5 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:40.636 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.628+0000 7fe42affd700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:17:40.636 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.628+0000 7fe42affd700 1 -- 172.21.15.77:0/3455653796 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43c09ad10 con 0x56424449a0a0 2023-12-01T01:17:40.636 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.628+0000 7fe42affd700 1 -- 172.21.15.77:0/3455653796 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fe43c09a8b0 con 0x56424449a0a0 2023-12-01T01:17:40.636 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.628+0000 7fe45dcb6b80 1 -- 172.21.15.77:0/3455653796 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe43c09e6e0 con 0x56424449a0a0 2023-12-01T01:17:40.636 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.628+0000 7fe42affd700 1 -- 172.21.15.77:0/3455653796 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe43c09a020 con 0x56424449a0a0 2023-12-01T01:17:40.636 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.628+0000 7fe45dcb6b80 10 client.15300 fetch_fsmap learned FSMap version 27 2023-12-01T01:17:40.636 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.628+0000 7fe45dcb6b80 1 -- 172.21.15.77:0/3455653796 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564244216260 con 0x56424449a0a0 2023-12-01T01:17:40.636 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.629+0000 7fe42affd700 1 -- 172.21.15.77:0/3455653796 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe43c09e6e0 con 0x56424449a0a0 2023-12-01T01:17:40.636 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.629+0000 7fe42affd700 1 -- 172.21.15.77:0/3455653796 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fe43c0960d0 con 0x56424449a0a0 2023-12-01T01:17:40.636 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.629+0000 7fe45dcb6b80 10 client.15300 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:17:40.636 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.629+0000 7fe45dcb6b80 10 client.15300 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-01T01:17:40.637 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.629+0000 7fe45dcb6b80 20 client.15300 populate_metadata read hostname 'smithi077' 2023-12-01T01:17:40.637 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.629+0000 7fe45dcb6b80 1 --2- 172.21.15.77:0/3455653796 >> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] conn(0x5642445043f0 0x5642444b65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.637 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.629+0000 7fe45dcb6b80 4 client.15300 mds_command: new command op to 24475 tid=0[{"prefix": "status"}] 2023-12-01T01:17:40.637 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.629+0000 7fe45dcb6b80 1 -- 172.21.15.77:0/3455653796 --> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5642443e2cf0 con 0x5642445043f0 2023-12-01T01:17:40.637 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.629+0000 7fe44b7fe700 1 --2- 172.21.15.77:0/3455653796 >> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] conn(0x5642445043f0 0x5642444b65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.637 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.629+0000 7fe42affd700 10 client.15300 ms_handle_connect on v2:172.21.15.203:6836/2887080977 2023-12-01T01:17:40.637 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.629+0000 7fe44b7fe700 1 --2- 172.21.15.77:0/3455653796 >> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] conn(0x5642445043f0 0x5642444b65c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=55ca7e246b1682a6 server_cookie=ef0414c4add2559b in_seq=0 out_seq=0 2023-12-01T01:17:40.637 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.630+0000 7fe42affd700 1 -- 172.21.15.77:0/3455653796 <== mds.2 v2:172.21.15.203:6836/2887080977 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5642443e2cf0 con 0x5642445043f0 2023-12-01T01:17:40.637 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.630+0000 7fe42affd700 10 client.15300 handle_command_reply: tid=0 2023-12-01T01:17:40.637 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.630+0000 7fe45dcb6b80 1 client.15300 shutdown 2023-12-01T01:17:40.637 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.630+0000 7fe45dcb6b80 1 -- 172.21.15.77:0/3455653796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56424449a0a0 msgr2=0x5642444b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.637 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.630+0000 7fe45dcb6b80 1 --2- 172.21.15.77:0/3455653796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56424449a0a0 0x5642444b1b40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009fd0 tx=0x7fe43c004080 comp rx=0 tx=0).stop 2023-12-01T01:17:40.637 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.630+0000 7fe45dcb6b80 1 -- 172.21.15.77:0/3455653796 shutdown_connections 2023-12-01T01:17:40.637 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.630+0000 7fe45dcb6b80 1 --2- 172.21.15.77:0/3455653796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56424449aa20 0x5642444b4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.638 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.630+0000 7fe45dcb6b80 1 --2- 172.21.15.77:0/3455653796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642444993a0 0x5642444b1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.638 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.630+0000 7fe45dcb6b80 1 --2- 172.21.15.77:0/3455653796 >> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] conn(0x5642445043f0 0x5642444b65c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.638 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.630+0000 7fe45dcb6b80 1 --2- 172.21.15.77:0/3455653796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56424449a0a0 0x5642444b1b40 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.638 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.630+0000 7fe42affd700 0 client.15300 ms_handle_reset on v2:172.21.15.203:6836/2887080977 2023-12-01T01:17:40.638 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.630+0000 7fe45dcb6b80 1 -- 172.21.15.77:0/3455653796 >> 172.21.15.77:0/3455653796 conn(0x5642443f8080 msgr2=0x5642444a2330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:40.638 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.630+0000 7fe45dcb6b80 1 -- 172.21.15.77:0/3455653796 shutdown_connections 2023-12-01T01:17:40.638 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.630+0000 7fe45dcb6b80 1 -- 172.21.15.77:0/3455653796 wait complete. 2023-12-01T01:17:40.638 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.630+0000 7fe45dcb6b80 20 client.15300 trim_cache size 0 max 16384 2023-12-01T01:17:40.638 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2023-12-01T01:17:40.638 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "2a06e29a-8fe6-11ee-95a2-87774f69a715", 2023-12-01T01:17:40.639 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 2, 2023-12-01T01:17:40.639 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24475, 2023-12-01T01:17:40.639 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2023-12-01T01:17:40.639 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2023-12-01T01:17:40.639 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2023-12-01T01:17:40.639 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 71.002336196000002, 2023-12-01T01:17:40.639 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 27, 2023-12-01T01:17:40.639 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 84, 2023-12-01T01:17:40.639 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 82, 2023-12-01T01:17:40.639 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 91.741158964999997 2023-12-01T01:17:40.639 INFO:teuthology.orchestra.run.smithi077.stdout:} 2023-12-01T01:17:40.640 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.630+0000 7fe451b26700 1 -- 172.21.15.77:0/89231516 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fe4340a05b0 msgr2=0x7fe4340a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.640 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.630+0000 7fe451b26700 1 --2- 172.21.15.77:0/89231516 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fe4340a05b0 0x7fe4340a2a70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe440006fd0 tx=0x7fe440008040 comp rx=0 tx=0).stop 2023-12-01T01:17:40.640 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.630+0000 7fe451b26700 1 -- 172.21.15.77:0/89231516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe44c100e70 msgr2=0x7fe44c19b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.640 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.630+0000 7fe451b26700 1 --2- 172.21.15.77:0/89231516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe44c100e70 0x7fe44c19b910 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009990 tx=0x7fe43c003ba0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.640 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.631+0000 7fe451b26700 1 -- 172.21.15.77:0/89231516 shutdown_connections 2023-12-01T01:17:40.640 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.631+0000 7fe451b26700 1 --2- 172.21.15.77:0/89231516 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fe4340a05b0 0x7fe4340a2a70 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.640 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.631+0000 7fe451b26700 1 --2- 172.21.15.77:0/89231516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe44c101810 0x7fe44c19df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.641 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.631+0000 7fe451b26700 1 --2- 172.21.15.77:0/89231516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe44c0ffc80 0x7fe44c068a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.641 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.631+0000 7fe451b26700 1 --2- 172.21.15.77:0/89231516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe44c100e70 0x7fe44c19b910 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.641 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.631+0000 7fe451b26700 1 -- 172.21.15.77:0/89231516 >> 172.21.15.77:0/89231516 conn(0x7fe44c0fb1f0 msgr2=0x7fe44c1043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:40.641 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.631+0000 7fe451b26700 1 -- 172.21.15.77:0/89231516 shutdown_connections 2023-12-01T01:17:40.641 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.631+0000 7fe451b26700 1 -- 172.21.15.77:0/89231516 wait complete. 2023-12-01T01:17:40.644 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-12-01T01:17:40.768 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.765+0000 7f59cde9b700 1 -- 172.21.15.77:0/3795388852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59c8101810 msgr2=0x7f59c81080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.768 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.765+0000 7f59cde9b700 1 --2- 172.21.15.77:0/3795388852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59c8101810 0x7f59c81080c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f59c000b0d0 tx=0x7f59c000b3e0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.768 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.765+0000 7f59cde9b700 1 -- 172.21.15.77:0/3795388852 shutdown_connections 2023-12-01T01:17:40.768 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.765+0000 7f59cde9b700 1 --2- 172.21.15.77:0/3795388852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59c8101810 0x7f59c81080c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.768 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.765+0000 7f59cde9b700 1 --2- 172.21.15.77:0/3795388852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59c80ffc80 0x7f59c81000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.765+0000 7f59cde9b700 1 --2- 172.21.15.77:0/3795388852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59c8100e70 0x7f59c81012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.765+0000 7f59cde9b700 1 -- 172.21.15.77:0/3795388852 >> 172.21.15.77:0/3795388852 conn(0x7f59c80fb1f0 msgr2=0x7f59c80fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:40.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.765+0000 7f59cde9b700 1 -- 172.21.15.77:0/3795388852 shutdown_connections 2023-12-01T01:17:40.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.765+0000 7f59cde9b700 1 -- 172.21.15.77:0/3795388852 wait complete. 2023-12-01T01:17:40.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.765+0000 7f59cde9b700 1 Processor -- start 2023-12-01T01:17:40.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.765+0000 7f59cde9b700 1 -- start start 2023-12-01T01:17:40.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.766+0000 7f59cde9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59c80ffc80 0x7f59c8068a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.766+0000 7f59cde9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59c8100e70 0x7f59c819b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.766+0000 7f59cde9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59c8101810 0x7f59c819df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.766+0000 7f59cde9b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c819e6e0 con 0x7f59c80ffc80 2023-12-01T01:17:40.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.766+0000 7f59cde9b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c819e850 con 0x7f59c8100e70 2023-12-01T01:17:40.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.766+0000 7f59cde9b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c819ea10 con 0x7f59c8101810 2023-12-01T01:17:40.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.766+0000 7f59c77fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59c80ffc80 0x7f59c8068a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.766+0000 7f59c77fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59c80ffc80 0x7f59c8068a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58886/0 (socket says 172.21.15.77:58886) 2023-12-01T01:17:40.770 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.766+0000 7f59c77fe700 1 -- 172.21.15.77:0/2274697879 learned_addr learned my addr 172.21.15.77:0/2274697879 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:40.770 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.766+0000 7f59c7fff700 1 --2- 172.21.15.77:0/2274697879 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59c8101810 0x7f59c819df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.770 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.766+0000 7f59c6ffd700 1 --2- 172.21.15.77:0/2274697879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59c8100e70 0x7f59c819b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.770 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.766+0000 7f59c77fe700 1 -- 172.21.15.77:0/2274697879 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59c8101810 msgr2=0x7f59c819df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.770 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.766+0000 7f59c77fe700 1 --2- 172.21.15.77:0/2274697879 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59c8101810 0x7f59c819df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.770 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.766+0000 7f59c77fe700 1 -- 172.21.15.77:0/2274697879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59c8100e70 msgr2=0x7f59c819b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.770 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.766+0000 7f59c77fe700 1 --2- 172.21.15.77:0/2274697879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59c8100e70 0x7f59c819b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.770 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.766+0000 7f59c77fe700 1 -- 172.21.15.77:0/2274697879 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c0009d00 con 0x7f59c80ffc80 2023-12-01T01:17:40.770 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.766+0000 7f59c77fe700 1 --2- 172.21.15.77:0/2274697879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59c80ffc80 0x7f59c8068a60 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f59b800b500 tx=0x7f59b800b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:40.770 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.766+0000 7f59c4ff9700 1 -- 172.21.15.77:0/2274697879 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b8011830 con 0x7f59c80ffc80 2023-12-01T01:17:40.770 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.766+0000 7f59cde9b700 1 -- 172.21.15.77:0/2274697879 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59c819ecf0 con 0x7f59c80ffc80 2023-12-01T01:17:40.770 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.766+0000 7f59c4ff9700 1 -- 172.21.15.77:0/2274697879 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59b8011e50 con 0x7f59c80ffc80 2023-12-01T01:17:40.770 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.766+0000 7f59c4ff9700 1 -- 172.21.15.77:0/2274697879 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b800f2e0 con 0x7f59c80ffc80 2023-12-01T01:17:40.770 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.766+0000 7f59cde9b700 1 -- 172.21.15.77:0/2274697879 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59c819f220 con 0x7f59c80ffc80 2023-12-01T01:17:40.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.768+0000 7f59c4ff9700 1 -- 172.21.15.77:0/2274697879 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f59b8010580 con 0x7f59c80ffc80 2023-12-01T01:17:40.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.769+0000 7f59c4ff9700 1 --2- 172.21.15.77:0/2274697879 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f59b00a05b0 0x7f59b00a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.769+0000 7f59c4ff9700 1 -- 172.21.15.77:0/2274697879 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f59b80cb410 con 0x7f59c80ffc80 2023-12-01T01:17:40.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.769+0000 7f59c6ffd700 1 --2- 172.21.15.77:0/2274697879 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f59b00a05b0 0x7f59b00a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.769+0000 7f59c6ffd700 1 --2- 172.21.15.77:0/2274697879 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f59b00a05b0 0x7f59b00a2a70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f59bc005950 tx=0x7f59bc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:40.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.770+0000 7f59da02bb80 1 Processor -- start 2023-12-01T01:17:40.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.770+0000 7f59da02bb80 1 -- start start 2023-12-01T01:17:40.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.770+0000 7f59da02bb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e63d37b0a0 0x55e63d37b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.770+0000 7f59da02bb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e63d37ba20 0x55e63d379e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.770+0000 7f59da02bb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e63d37a3a0 0x55e63d37a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.770+0000 7f59da02bb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e63d2d19c0 con 0x55e63d37b0a0 2023-12-01T01:17:40.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.770+0000 7f59da02bb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e63d2cf380 con 0x55e63d37a3a0 2023-12-01T01:17:40.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.770+0000 7f59da02bb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e63d2cac70 con 0x55e63d37ba20 2023-12-01T01:17:40.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.770+0000 7f59c77fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e63d37b0a0 0x55e63d37b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.770+0000 7f59c77fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e63d37b0a0 0x55e63d37b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58894/0 (socket says 172.21.15.77:58894) 2023-12-01T01:17:40.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.770+0000 7f59c77fe700 1 -- 172.21.15.77:0/1911399274 learned_addr learned my addr 172.21.15.77:0/1911399274 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:40.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.770+0000 7f59c6ffd700 1 --2- 172.21.15.77:0/1911399274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e63d37ba20 0x55e63d379e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.770+0000 7f59c77fe700 1 -- 172.21.15.77:0/1911399274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e63d37ba20 msgr2=0x55e63d379e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:40.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.770+0000 7f59c77fe700 1 --2- 172.21.15.77:0/1911399274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e63d37ba20 0x55e63d379e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.770+0000 7f59c77fe700 1 -- 172.21.15.77:0/1911399274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e63d37a3a0 msgr2=0x55e63d37a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:40.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.770+0000 7f59c7fff700 1 --2- 172.21.15.77:0/1911399274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e63d37a3a0 0x55e63d37a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.770+0000 7f59c77fe700 1 --2- 172.21.15.77:0/1911399274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e63d37a3a0 0x55e63d37a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.771+0000 7f59c77fe700 1 -- 172.21.15.77:0/1911399274 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e63d0f8d90 con 0x55e63d37b0a0 2023-12-01T01:17:40.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.771+0000 7f59c6ffd700 1 --2- 172.21.15.77:0/1911399274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e63d37ba20 0x55e63d379e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:17:40.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.771+0000 7f59c77fe700 1 --2- 172.21.15.77:0/1911399274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e63d37b0a0 0x55e63d37b4e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f59b8095790 tx=0x7f59b8098930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ac20ee449cecde server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:40.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.771+0000 7f59adffb700 1 -- 172.21.15.77:0/1911399274 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b80cb250 con 0x55e63d37b0a0 2023-12-01T01:17:40.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.771+0000 7f59adffb700 1 -- 172.21.15.77:0/1911399274 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59b8099420 con 0x55e63d37b0a0 2023-12-01T01:17:40.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.771+0000 7f59adffb700 1 -- 172.21.15.77:0/1911399274 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b8098210 con 0x55e63d37b0a0 2023-12-01T01:17:40.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.771+0000 7f59da02bb80 1 -- 172.21.15.77:0/1911399274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e63d37b0a0 msgr2=0x55e63d37b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.771+0000 7f59da02bb80 1 --2- 172.21.15.77:0/1911399274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e63d37b0a0 0x55e63d37b4e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f59b8095790 tx=0x7f59b8098930 comp rx=0 tx=0).stop 2023-12-01T01:17:40.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.771+0000 7f59da02bb80 1 -- 172.21.15.77:0/1911399274 shutdown_connections 2023-12-01T01:17:40.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.771+0000 7f59da02bb80 1 --2- 172.21.15.77:0/1911399274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e63d37a3a0 0x55e63d37a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.771+0000 7f59da02bb80 1 --2- 172.21.15.77:0/1911399274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e63d37b0a0 0x55e63d37b4e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.771+0000 7f59da02bb80 1 --2- 172.21.15.77:0/1911399274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e63d37ba20 0x55e63d379e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.771+0000 7f59da02bb80 1 -- 172.21.15.77:0/1911399274 >> 172.21.15.77:0/1911399274 conn(0x55e63d2d9080 msgr2=0x55e63d3748f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:40.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.772+0000 7f59da02bb80 1 -- 172.21.15.77:0/1911399274 shutdown_connections 2023-12-01T01:17:40.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.772+0000 7f59da02bb80 1 -- 172.21.15.77:0/1911399274 wait complete. 2023-12-01T01:17:40.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.772+0000 7f59da02bb80 1 -- start start 2023-12-01T01:17:40.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.772+0000 7f59da02bb80 1 Processor -- start 2023-12-01T01:17:40.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.772+0000 7f59da02bb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e63d37a3a0 0x55e63d383a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.772+0000 7f59da02bb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e63d37b0a0 0x55e63d383f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.772+0000 7f59da02bb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e63d37ba20 0x55e63d38af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.772+0000 7f59da02bb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e63d2674c0 con 0x55e63d37a3a0 2023-12-01T01:17:40.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.772+0000 7f59da02bb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e63d267160 con 0x55e63d37ba20 2023-12-01T01:17:40.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.772+0000 7f59da02bb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e63d14e8d0 con 0x55e63d37b0a0 2023-12-01T01:17:40.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.772+0000 7f59da02bb80 10 client.0 _ll_register_callbacks cb 0x55e63d1b4a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:17:40.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.772+0000 7f59c77fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e63d37a3a0 0x55e63d383a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.772+0000 7f59c77fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e63d37a3a0 0x55e63d383a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58900/0 (socket says 172.21.15.77:58900) 2023-12-01T01:17:40.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.772+0000 7f59c77fe700 1 -- 172.21.15.77:0/4166210125 learned_addr learned my addr 172.21.15.77:0/4166210125 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:40.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.772+0000 7f59c7fff700 1 --2- 172.21.15.77:0/4166210125 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e63d37ba20 0x55e63d38af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.773+0000 7f59c6ffd700 1 --2- 172.21.15.77:0/4166210125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e63d37b0a0 0x55e63d383f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.773+0000 7f59c77fe700 1 -- 172.21.15.77:0/4166210125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e63d37b0a0 msgr2=0x55e63d383f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:40.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.773+0000 7f59c77fe700 1 --2- 172.21.15.77:0/4166210125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e63d37b0a0 0x55e63d383f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.773+0000 7f59c77fe700 1 -- 172.21.15.77:0/4166210125 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e63d37ba20 msgr2=0x55e63d38af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:40.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.773+0000 7f59c77fe700 1 --2- 172.21.15.77:0/4166210125 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e63d37ba20 0x55e63d38af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.773+0000 7f59c77fe700 1 -- 172.21.15.77:0/4166210125 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59c819f220 con 0x55e63d37a3a0 2023-12-01T01:17:40.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.773+0000 7f59c6ffd700 1 --2- 172.21.15.77:0/4166210125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e63d37b0a0 0x55e63d383f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:17:40.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.773+0000 7f59c7fff700 1 --2- 172.21.15.77:0/4166210125 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e63d37ba20 0x55e63d38af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:17:40.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.773+0000 7f59c77fe700 1 --2- 172.21.15.77:0/4166210125 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e63d37a3a0 0x55e63d383a10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f59b8098a20 tx=0x7f59b8004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=987bab295c581e73 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:40.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.773+0000 7f59a6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:17:40.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.773+0000 7f59a6ffd700 1 -- 172.21.15.77:0/4166210125 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b809cb50 con 0x55e63d37a3a0 2023-12-01T01:17:40.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.773+0000 7f59a6ffd700 1 -- 172.21.15.77:0/4166210125 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f59b8098210 con 0x55e63d37a3a0 2023-12-01T01:17:40.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.773+0000 7f59da02bb80 1 -- 172.21.15.77:0/4166210125 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e63d2f5cc0 con 0x55e63d37a3a0 2023-12-01T01:17:40.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.773+0000 7f59a6ffd700 1 -- 172.21.15.77:0/4166210125 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59b80983d0 con 0x55e63d37a3a0 2023-12-01T01:17:40.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.773+0000 7f59da02bb80 10 client.15312 fetch_fsmap learned FSMap version 27 2023-12-01T01:17:40.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.773+0000 7f59da02bb80 1 -- 172.21.15.77:0/4166210125 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e63d0e1c10 con 0x55e63d37a3a0 2023-12-01T01:17:40.782 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.773+0000 7f59a6ffd700 1 -- 172.21.15.77:0/4166210125 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e63d2f5cc0 con 0x55e63d37a3a0 2023-12-01T01:17:40.782 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.773+0000 7f59a6ffd700 1 -- 172.21.15.77:0/4166210125 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f59b8094b10 con 0x55e63d37a3a0 2023-12-01T01:17:40.782 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.773+0000 7f59da02bb80 10 client.15312 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:17:40.782 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.773+0000 7f59da02bb80 10 client.15312 resolve_mds: resolved name 'i' to daemon mds.i 2023-12-01T01:17:40.782 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.773+0000 7f59da02bb80 20 client.15312 populate_metadata read hostname 'smithi077' 2023-12-01T01:17:40.782 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.773+0000 7f59da02bb80 1 --2- 172.21.15.77:0/4166210125 >> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] conn(0x55e63d38ed20 0x55e63d38de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.782 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.773+0000 7f59da02bb80 4 client.15312 mds_command: new command op to 24496 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:17:40.782 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.773+0000 7f59da02bb80 1 -- 172.21.15.77:0/4166210125 --> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e63d348dc0 con 0x55e63d38ed20 2023-12-01T01:17:40.782 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.774+0000 7f59c6ffd700 1 --2- 172.21.15.77:0/4166210125 >> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] conn(0x55e63d38ed20 0x55e63d38de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.782 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.774+0000 7f59a6ffd700 10 client.15312 ms_handle_connect on v2:172.21.15.203:6838/2888221613 2023-12-01T01:17:40.782 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.774+0000 7f59c6ffd700 1 --2- 172.21.15.77:0/4166210125 >> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] conn(0x55e63d38ed20 0x55e63d38de30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=69222bfce064e238 server_cookie=5a2e3061341d43c9 in_seq=0 out_seq=0 2023-12-01T01:17:40.782 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.775+0000 7f59a6ffd700 1 -- 172.21.15.77:0/4166210125 <== mds.1 v2:172.21.15.203:6838/2888221613 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55e63d348dc0 con 0x55e63d38ed20 2023-12-01T01:17:40.783 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.775+0000 7f59a6ffd700 10 client.15312 handle_command_reply: tid=0 2023-12-01T01:17:40.783 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.775+0000 7f59da02bb80 1 client.15312 shutdown 2023-12-01T01:17:40.783 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.775+0000 7f59da02bb80 1 -- 172.21.15.77:0/4166210125 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e63d37a3a0 msgr2=0x55e63d383a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.783 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.775+0000 7f59da02bb80 1 --2- 172.21.15.77:0/4166210125 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e63d37a3a0 0x55e63d383a10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f59b8098a20 tx=0x7f59b8004860 comp rx=0 tx=0).stop 2023-12-01T01:17:40.783 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.775+0000 7f59da02bb80 1 -- 172.21.15.77:0/4166210125 shutdown_connections 2023-12-01T01:17:40.783 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.775+0000 7f59da02bb80 1 --2- 172.21.15.77:0/4166210125 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e63d37ba20 0x55e63d38af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.783 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.775+0000 7f59da02bb80 1 --2- 172.21.15.77:0/4166210125 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e63d37a3a0 0x55e63d383a10 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.783 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.775+0000 7f59da02bb80 1 --2- 172.21.15.77:0/4166210125 >> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] conn(0x55e63d38ed20 0x55e63d38de30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.783 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.775+0000 7f59da02bb80 1 --2- 172.21.15.77:0/4166210125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e63d37b0a0 0x55e63d383f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.783 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.775+0000 7f59a6ffd700 0 client.15312 ms_handle_reset on v2:172.21.15.203:6838/2888221613 2023-12-01T01:17:40.783 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.775+0000 7f59da02bb80 1 -- 172.21.15.77:0/4166210125 >> 172.21.15.77:0/4166210125 conn(0x55e63d2d9080 msgr2=0x55e63d374060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:40.783 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.775+0000 7f59da02bb80 1 -- 172.21.15.77:0/4166210125 shutdown_connections 2023-12-01T01:17:40.783 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.775+0000 7f59da02bb80 1 -- 172.21.15.77:0/4166210125 wait complete. 2023-12-01T01:17:40.783 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.775+0000 7f59da02bb80 20 client.15312 trim_cache size 0 max 16384 2023-12-01T01:17:40.799 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.792+0000 7f59da02bb80 1 Processor -- start 2023-12-01T01:17:40.799 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.792+0000 7f59da02bb80 1 -- start start 2023-12-01T01:17:40.799 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.792+0000 7f59da02bb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e63d37a3a0 0x55e63d382660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.799 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.792+0000 7f59da02bb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e63d37b0a0 0x55e63d397480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.799 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.792+0000 7f59da02bb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e63d37ba20 0x55e63d37e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.799 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.792+0000 7f59da02bb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e63d3835e0 con 0x55e63d37ba20 2023-12-01T01:17:40.799 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.792+0000 7f59da02bb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e63d267160 con 0x55e63d37b0a0 2023-12-01T01:17:40.799 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.792+0000 7f59da02bb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e63d3820c0 con 0x55e63d37a3a0 2023-12-01T01:17:40.800 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.792+0000 7f59c7fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e63d37ba20 0x55e63d37e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.800 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.792+0000 7f59c7fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e63d37ba20 0x55e63d37e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58912/0 (socket says 172.21.15.77:58912) 2023-12-01T01:17:40.800 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.792+0000 7f59c7fff700 1 -- 172.21.15.77:0/1490057361 learned_addr learned my addr 172.21.15.77:0/1490057361 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:40.800 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.793+0000 7f59c77fe700 1 --2- 172.21.15.77:0/1490057361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e63d37a3a0 0x55e63d382660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.800 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.793+0000 7f59c7fff700 1 -- 172.21.15.77:0/1490057361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e63d37a3a0 msgr2=0x55e63d382660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:40.800 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.793+0000 7f59c6ffd700 1 --2- 172.21.15.77:0/1490057361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e63d37b0a0 0x55e63d397480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.800 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.793+0000 7f59c7fff700 1 --2- 172.21.15.77:0/1490057361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e63d37a3a0 0x55e63d382660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.800 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.793+0000 7f59c7fff700 1 -- 172.21.15.77:0/1490057361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e63d37b0a0 msgr2=0x55e63d397480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:40.800 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.793+0000 7f59c7fff700 1 --2- 172.21.15.77:0/1490057361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e63d37b0a0 0x55e63d397480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.800 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.793+0000 7f59c7fff700 1 -- 172.21.15.77:0/1490057361 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e63d11f200 con 0x55e63d37ba20 2023-12-01T01:17:40.800 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.793+0000 7f59c7fff700 1 --2- 172.21.15.77:0/1490057361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e63d37ba20 0x55e63d37e360 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f59c0015000 tx=0x7f59c00074e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be87c8068e9e37a7 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:40.800 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.793+0000 7f59ae7fc700 1 -- 172.21.15.77:0/1490057361 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c000e040 con 0x55e63d37ba20 2023-12-01T01:17:40.800 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.793+0000 7f59ae7fc700 1 -- 172.21.15.77:0/1490057361 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59c0004a50 con 0x55e63d37ba20 2023-12-01T01:17:40.800 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.793+0000 7f59ae7fc700 1 -- 172.21.15.77:0/1490057361 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c0026ab0 con 0x55e63d37ba20 2023-12-01T01:17:40.801 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.793+0000 7f59da02bb80 1 -- 172.21.15.77:0/1490057361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e63d37ba20 msgr2=0x55e63d37e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.801 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.794+0000 7f59da02bb80 1 --2- 172.21.15.77:0/1490057361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e63d37ba20 0x55e63d37e360 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f59c0015000 tx=0x7f59c00074e0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.801 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.794+0000 7f59da02bb80 1 -- 172.21.15.77:0/1490057361 shutdown_connections 2023-12-01T01:17:40.801 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.794+0000 7f59da02bb80 1 --2- 172.21.15.77:0/1490057361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e63d37ba20 0x55e63d37e360 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.801 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.794+0000 7f59da02bb80 1 --2- 172.21.15.77:0/1490057361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e63d37a3a0 0x55e63d382660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.801 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.794+0000 7f59da02bb80 1 --2- 172.21.15.77:0/1490057361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e63d37b0a0 0x55e63d397480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.801 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.794+0000 7f59da02bb80 1 -- 172.21.15.77:0/1490057361 >> 172.21.15.77:0/1490057361 conn(0x55e63d2d9080 msgr2=0x55e63d38de30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:40.801 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.794+0000 7f59da02bb80 1 -- 172.21.15.77:0/1490057361 shutdown_connections 2023-12-01T01:17:40.801 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.794+0000 7f59da02bb80 1 -- 172.21.15.77:0/1490057361 wait complete. 2023-12-01T01:17:40.801 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.794+0000 7f59da02bb80 1 -- start start 2023-12-01T01:17:40.801 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.794+0000 7f59da02bb80 1 Processor -- start 2023-12-01T01:17:40.801 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.795+0000 7f59da02bb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e63d37a3a0 0x55e63d394080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.801 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.795+0000 7f59da02bb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e63d37b0a0 0x55e63d3945c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.801 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.795+0000 7f59da02bb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e63d37ba20 0x55e63d3e0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.802 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.795+0000 7f59da02bb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e63d2674c0 con 0x55e63d37a3a0 2023-12-01T01:17:40.802 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.795+0000 7f59da02bb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e63d3975e0 con 0x55e63d37b0a0 2023-12-01T01:17:40.802 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.795+0000 7f59da02bb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e63d397770 con 0x55e63d37ba20 2023-12-01T01:17:40.802 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.795+0000 7f59da02bb80 10 client.0 _ll_register_callbacks cb 0x7f59b8098bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:17:40.802 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.795+0000 7f59c77fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e63d37a3a0 0x55e63d394080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.802 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.795+0000 7f59c77fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e63d37a3a0 0x55e63d394080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58914/0 (socket says 172.21.15.77:58914) 2023-12-01T01:17:40.802 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.795+0000 7f59c77fe700 1 -- 172.21.15.77:0/1747738172 learned_addr learned my addr 172.21.15.77:0/1747738172 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:40.802 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.795+0000 7f59c77fe700 1 -- 172.21.15.77:0/1747738172 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e63d37ba20 msgr2=0x55e63d3e0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:40.802 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.795+0000 7f59c77fe700 1 --2- 172.21.15.77:0/1747738172 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e63d37ba20 0x55e63d3e0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.802 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.795+0000 7f59c77fe700 1 -- 172.21.15.77:0/1747738172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e63d37b0a0 msgr2=0x55e63d3945c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:40.802 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.795+0000 7f59c77fe700 1 --2- 172.21.15.77:0/1747738172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e63d37b0a0 0x55e63d3945c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.802 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.795+0000 7f59c77fe700 1 -- 172.21.15.77:0/1747738172 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59c8108a10 con 0x55e63d37a3a0 2023-12-01T01:17:40.802 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.795+0000 7f59c77fe700 1 --2- 172.21.15.77:0/1747738172 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e63d37a3a0 0x55e63d394080 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f59b800bcc0 tx=0x7f59b80c8fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d34502d96489e8 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:40.803 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.795+0000 7f59a6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:17:40.803 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.795+0000 7f59a6ffd700 1 -- 172.21.15.77:0/1747738172 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b8098f80 con 0x55e63d37a3a0 2023-12-01T01:17:40.803 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.795+0000 7f59a6ffd700 1 -- 172.21.15.77:0/1747738172 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f59b809eef0 con 0x55e63d37a3a0 2023-12-01T01:17:40.803 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.795+0000 7f59da02bb80 1 -- 172.21.15.77:0/1747738172 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f59b8099420 con 0x55e63d37a3a0 2023-12-01T01:17:40.803 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.795+0000 7f59a6ffd700 1 -- 172.21.15.77:0/1747738172 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59b800d040 con 0x55e63d37a3a0 2023-12-01T01:17:40.803 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.795+0000 7f59da02bb80 10 client.15318 fetch_fsmap learned FSMap version 27 2023-12-01T01:17:40.803 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.795+0000 7f59da02bb80 1 -- 172.21.15.77:0/1747738172 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e63d0f7260 con 0x55e63d37a3a0 2023-12-01T01:17:40.803 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.795+0000 7f59a6ffd700 1 -- 172.21.15.77:0/1747738172 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59b8099420 con 0x55e63d37a3a0 2023-12-01T01:17:40.803 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.796+0000 7f59a6ffd700 1 -- 172.21.15.77:0/1747738172 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f59b8098210 con 0x55e63d37a3a0 2023-12-01T01:17:40.803 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.796+0000 7f59da02bb80 10 client.15318 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:17:40.803 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.796+0000 7f59da02bb80 10 client.15318 resolve_mds: resolved name 'i' to daemon mds.i 2023-12-01T01:17:40.803 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.796+0000 7f59da02bb80 20 client.15318 populate_metadata read hostname 'smithi077' 2023-12-01T01:17:40.803 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.796+0000 7f59da02bb80 1 --2- 172.21.15.77:0/1747738172 >> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] conn(0x55e63d3e4f60 0x55e63d3e1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.803 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.796+0000 7f59da02bb80 4 client.15318 mds_command: new command op to 24496 tid=0[{"prefix": "status"}] 2023-12-01T01:17:40.804 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.796+0000 7f59da02bb80 1 -- 172.21.15.77:0/1747738172 --> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e63d2c3cf0 con 0x55e63d3e4f60 2023-12-01T01:17:40.804 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.796+0000 7f59c6ffd700 1 --2- 172.21.15.77:0/1747738172 >> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] conn(0x55e63d3e4f60 0x55e63d3e1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.804 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.796+0000 7f59c6ffd700 1 --2- 172.21.15.77:0/1747738172 >> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] conn(0x55e63d3e4f60 0x55e63d3e1890 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=269bf36576a1f6be server_cookie=64ee6a6fad638281 in_seq=0 out_seq=0 2023-12-01T01:17:40.804 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.796+0000 7f59a6ffd700 10 client.15318 ms_handle_connect on v2:172.21.15.203:6838/2888221613 2023-12-01T01:17:40.804 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.797+0000 7f59a6ffd700 1 -- 172.21.15.77:0/1747738172 <== mds.1 v2:172.21.15.203:6838/2888221613 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55e63d2c3cf0 con 0x55e63d3e4f60 2023-12-01T01:17:40.804 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.797+0000 7f59a6ffd700 10 client.15318 handle_command_reply: tid=0 2023-12-01T01:17:40.804 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.797+0000 7f59da02bb80 1 client.15318 shutdown 2023-12-01T01:17:40.804 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.797+0000 7f59da02bb80 1 -- 172.21.15.77:0/1747738172 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e63d37a3a0 msgr2=0x55e63d394080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.804 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.797+0000 7f59da02bb80 1 --2- 172.21.15.77:0/1747738172 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e63d37a3a0 0x55e63d394080 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f59b800bcc0 tx=0x7f59b80c8fc0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.804 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.797+0000 7f59da02bb80 1 -- 172.21.15.77:0/1747738172 shutdown_connections 2023-12-01T01:17:40.804 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.797+0000 7f59da02bb80 1 --2- 172.21.15.77:0/1747738172 >> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] conn(0x55e63d3e4f60 0x55e63d3e1890 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.804 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.797+0000 7f59da02bb80 1 --2- 172.21.15.77:0/1747738172 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e63d37ba20 0x55e63d3e0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.804 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.797+0000 7f59da02bb80 1 --2- 172.21.15.77:0/1747738172 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e63d37a3a0 0x55e63d394080 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.804 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.797+0000 7f59da02bb80 1 --2- 172.21.15.77:0/1747738172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e63d37b0a0 0x55e63d3945c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.804 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.797+0000 7f59a6ffd700 0 client.15318 ms_handle_reset on v2:172.21.15.203:6838/2888221613 2023-12-01T01:17:40.804 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.797+0000 7f59da02bb80 1 -- 172.21.15.77:0/1747738172 >> 172.21.15.77:0/1747738172 conn(0x55e63d2d9080 msgr2=0x55e63d382660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:40.805 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.797+0000 7f59da02bb80 1 -- 172.21.15.77:0/1747738172 shutdown_connections 2023-12-01T01:17:40.805 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.797+0000 7f59da02bb80 1 -- 172.21.15.77:0/1747738172 wait complete. 2023-12-01T01:17:40.805 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.797+0000 7f59da02bb80 20 client.15318 trim_cache size 0 max 16384 2023-12-01T01:17:40.805 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2023-12-01T01:17:40.805 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "2a06e29a-8fe6-11ee-95a2-87774f69a715", 2023-12-01T01:17:40.805 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 1, 2023-12-01T01:17:40.805 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24496, 2023-12-01T01:17:40.805 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2023-12-01T01:17:40.805 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2023-12-01T01:17:40.805 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2023-12-01T01:17:40.805 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 72.174240329, 2023-12-01T01:17:40.805 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 27, 2023-12-01T01:17:40.806 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 84, 2023-12-01T01:17:40.806 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 82, 2023-12-01T01:17:40.806 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 87.446049039000002 2023-12-01T01:17:40.806 INFO:teuthology.orchestra.run.smithi077.stdout:} 2023-12-01T01:17:40.807 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.797+0000 7f59cde9b700 1 -- 172.21.15.77:0/2274697879 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f59b00a05b0 msgr2=0x7f59b00a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.807 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.797+0000 7f59cde9b700 1 --2- 172.21.15.77:0/2274697879 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f59b00a05b0 0x7f59b00a2a70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f59bc005950 tx=0x7f59bc0058e0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.807 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.797+0000 7f59cde9b700 1 -- 172.21.15.77:0/2274697879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59c80ffc80 msgr2=0x7f59c8068a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.807 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.797+0000 7f59cde9b700 1 --2- 172.21.15.77:0/2274697879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59c80ffc80 0x7f59c8068a60 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f59b800b500 tx=0x7f59b800b8c0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.807 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.797+0000 7f59cde9b700 1 -- 172.21.15.77:0/2274697879 shutdown_connections 2023-12-01T01:17:40.807 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.797+0000 7f59cde9b700 1 --2- 172.21.15.77:0/2274697879 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f59b00a05b0 0x7f59b00a2a70 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.807 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.797+0000 7f59cde9b700 1 --2- 172.21.15.77:0/2274697879 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59c8101810 0x7f59c819df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.807 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.797+0000 7f59cde9b700 1 --2- 172.21.15.77:0/2274697879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59c80ffc80 0x7f59c8068a60 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.807 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.797+0000 7f59cde9b700 1 --2- 172.21.15.77:0/2274697879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59c8100e70 0x7f59c819b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.807 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.797+0000 7f59cde9b700 1 -- 172.21.15.77:0/2274697879 >> 172.21.15.77:0/2274697879 conn(0x7f59c80fb1f0 msgr2=0x7f59c80fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:40.807 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.798+0000 7f59cde9b700 1 -- 172.21.15.77:0/2274697879 shutdown_connections 2023-12-01T01:17:40.807 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.798+0000 7f59cde9b700 1 -- 172.21.15.77:0/2274697879 wait complete. 2023-12-01T01:17:40.810 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:40 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/3263773577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:17:40.810 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:40 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/3237973519' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:17:40.810 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:40 smithi077 ceph-mon[98867]: pgmap v292: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 14 B/s, 0 keys/s, 0 objects/s recovering 2023-12-01T01:17:40.811 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-12-01T01:17:40.934 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.930+0000 7fa5c4139700 1 -- 172.21.15.77:0/1519908354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5bc102980 msgr2=0x7fa5bc109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.934 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.930+0000 7fa5c4139700 1 --2- 172.21.15.77:0/1519908354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5bc102980 0x7fa5bc109230 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa5b400b0d0 tx=0x7fa5b400b3e0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.934 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.930+0000 7fa5c4139700 1 -- 172.21.15.77:0/1519908354 shutdown_connections 2023-12-01T01:17:40.934 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.930+0000 7fa5c4139700 1 --2- 172.21.15.77:0/1519908354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5bc102980 0x7fa5bc109230 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.934 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.930+0000 7fa5c4139700 1 --2- 172.21.15.77:0/1519908354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5bc100df0 0x7fa5bc101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.934 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.930+0000 7fa5c4139700 1 --2- 172.21.15.77:0/1519908354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5bc101fe0 0x7fa5bc102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.934 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.930+0000 7fa5c4139700 1 -- 172.21.15.77:0/1519908354 >> 172.21.15.77:0/1519908354 conn(0x7fa5bc0fc360 msgr2=0x7fa5bc0fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:40.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.931+0000 7fa5c4139700 1 -- 172.21.15.77:0/1519908354 shutdown_connections 2023-12-01T01:17:40.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.931+0000 7fa5c4139700 1 -- 172.21.15.77:0/1519908354 wait complete. 2023-12-01T01:17:40.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.931+0000 7fa5c4139700 1 Processor -- start 2023-12-01T01:17:40.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.931+0000 7fa5c4139700 1 -- start start 2023-12-01T01:17:40.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.931+0000 7fa5c4139700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5bc100df0 0x7fa5bc196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.931+0000 7fa5c4139700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5bc101fe0 0x7fa5bc1970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.931+0000 7fa5c4139700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5bc102980 0x7fa5bc19e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.931+0000 7fa5c4139700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5bc19e6e0 con 0x7fa5bc101fe0 2023-12-01T01:17:40.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.931+0000 7fa5c4139700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5bc19e820 con 0x7fa5bc102980 2023-12-01T01:17:40.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.931+0000 7fa5c4139700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5bc19e9b0 con 0x7fa5bc100df0 2023-12-01T01:17:40.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.931+0000 7fa5c16d4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5bc101fe0 0x7fa5bc1970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.931+0000 7fa5c16d4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5bc101fe0 0x7fa5bc1970a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58930/0 (socket says 172.21.15.77:58930) 2023-12-01T01:17:40.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.931+0000 7fa5c16d4700 1 -- 172.21.15.77:0/423081746 learned_addr learned my addr 172.21.15.77:0/423081746 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:40.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.931+0000 7fa5c26d6700 1 --2- 172.21.15.77:0/423081746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5bc102980 0x7fa5bc19e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.932+0000 7fa5c1ed5700 1 --2- 172.21.15.77:0/423081746 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5bc100df0 0x7fa5bc196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.936 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.932+0000 7fa5c16d4700 1 -- 172.21.15.77:0/423081746 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5bc100df0 msgr2=0x7fa5bc196b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.936 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.932+0000 7fa5c16d4700 1 --2- 172.21.15.77:0/423081746 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5bc100df0 0x7fa5bc196b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.936 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.932+0000 7fa5c16d4700 1 -- 172.21.15.77:0/423081746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5bc102980 msgr2=0x7fa5bc19e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.936 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.932+0000 7fa5c16d4700 1 --2- 172.21.15.77:0/423081746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5bc102980 0x7fa5bc19e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.936 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.932+0000 7fa5c16d4700 1 -- 172.21.15.77:0/423081746 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b4009d00 con 0x7fa5bc101fe0 2023-12-01T01:17:40.936 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.932+0000 7fa5c1ed5700 1 --2- 172.21.15.77:0/423081746 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5bc100df0 0x7fa5bc196b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:17:40.936 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.932+0000 7fa5c16d4700 1 --2- 172.21.15.77:0/423081746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5bc101fe0 0x7fa5bc1970a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac00e9a0 tx=0x7fa5ac00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:40.936 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.932+0000 7fa5b2ffd700 1 -- 172.21.15.77:0/423081746 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac009790 con 0x7fa5bc101fe0 2023-12-01T01:17:40.936 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.932+0000 7fa5b2ffd700 1 -- 172.21.15.77:0/423081746 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5ac00cbd0 con 0x7fa5bc101fe0 2023-12-01T01:17:40.936 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.932+0000 7fa5c4139700 1 -- 172.21.15.77:0/423081746 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5bc19ec90 con 0x7fa5bc101fe0 2023-12-01T01:17:40.936 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.932+0000 7fa5b2ffd700 1 -- 172.21.15.77:0/423081746 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac018680 con 0x7fa5bc101fe0 2023-12-01T01:17:40.936 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.932+0000 7fa5c4139700 1 -- 172.21.15.77:0/423081746 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5bc19f1a0 con 0x7fa5bc101fe0 2023-12-01T01:17:40.942 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.934+0000 7fa5b2ffd700 1 -- 172.21.15.77:0/423081746 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fa5ac00cd40 con 0x7fa5bc101fe0 2023-12-01T01:17:40.942 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.935+0000 7fa5b2ffd700 1 --2- 172.21.15.77:0/423081746 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa5a80a0630 0x7fa5a80a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.942 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.935+0000 7fa5c1ed5700 1 --2- 172.21.15.77:0/423081746 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa5a80a0630 0x7fa5a80a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.943 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.935+0000 7fa5b2ffd700 1 -- 172.21.15.77:0/423081746 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fa5ac014070 con 0x7fa5bc101fe0 2023-12-01T01:17:40.943 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.935+0000 7fa5c1ed5700 1 --2- 172.21.15.77:0/423081746 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa5a80a0630 0x7fa5a80a2af0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8005950 tx=0x7fa5b80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:40.943 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.935+0000 7fa5d02c9b80 1 Processor -- start 2023-12-01T01:17:40.943 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.936+0000 7fa5d02c9b80 1 -- start start 2023-12-01T01:17:40.943 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.936+0000 7fa5d02c9b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebceeb4b0 0x55cebceeb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.943 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.936+0000 7fa5d02c9b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cebceebe30 0x55cebceea220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.943 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.936+0000 7fa5d02c9b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cebceea7b0 0x55cebceeac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.943 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.936+0000 7fa5d02c9b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cebce3d9c0 con 0x55cebceeb4b0 2023-12-01T01:17:40.943 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.936+0000 7fa5d02c9b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cebce3b380 con 0x55cebceea7b0 2023-12-01T01:17:40.943 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.936+0000 7fa5d02c9b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cebce36c70 con 0x55cebceebe30 2023-12-01T01:17:40.943 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.936+0000 7fa5c1ed5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebceeb4b0 0x55cebceeb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.943 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.936+0000 7fa5c1ed5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebceeb4b0 0x55cebceeb8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58936/0 (socket says 172.21.15.77:58936) 2023-12-01T01:17:40.943 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.936+0000 7fa5c1ed5700 1 -- 172.21.15.77:0/4041896503 learned_addr learned my addr 172.21.15.77:0/4041896503 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:40.943 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.936+0000 7fa5c26d6700 1 --2- 172.21.15.77:0/4041896503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cebceea7b0 0x55cebceeac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.944 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.936+0000 7fa5c1ed5700 1 -- 172.21.15.77:0/4041896503 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cebceebe30 msgr2=0x55cebceea220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:40.944 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.936+0000 7fa5c16d4700 1 --2- 172.21.15.77:0/4041896503 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cebceebe30 0x55cebceea220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.944 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.936+0000 7fa5c1ed5700 1 --2- 172.21.15.77:0/4041896503 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cebceebe30 0x55cebceea220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.944 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.936+0000 7fa5c1ed5700 1 -- 172.21.15.77:0/4041896503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cebceea7b0 msgr2=0x55cebceeac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:40.944 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.936+0000 7fa5c1ed5700 1 --2- 172.21.15.77:0/4041896503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cebceea7b0 0x55cebceeac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.944 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.936+0000 7fa5c1ed5700 1 -- 172.21.15.77:0/4041896503 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cebcc64d90 con 0x55cebceeb4b0 2023-12-01T01:17:40.944 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.936+0000 7fa5c26d6700 1 --2- 172.21.15.77:0/4041896503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cebceea7b0 0x55cebceeac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:17:40.944 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.936+0000 7fa5c1ed5700 1 --2- 172.21.15.77:0/4041896503 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebceeb4b0 0x55cebceeb8f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa5b800ff00 tx=0x7fa5b8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4235ea2eba6cc774 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:40.944 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.936+0000 7fa5a7fff700 1 -- 172.21.15.77:0/4041896503 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8011c60 con 0x55cebceeb4b0 2023-12-01T01:17:40.944 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.937+0000 7fa5a7fff700 1 -- 172.21.15.77:0/4041896503 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5b8011dc0 con 0x55cebceeb4b0 2023-12-01T01:17:40.944 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.937+0000 7fa5a7fff700 1 -- 172.21.15.77:0/4041896503 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b801d720 con 0x55cebceeb4b0 2023-12-01T01:17:40.944 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.937+0000 7fa5d02c9b80 1 -- 172.21.15.77:0/4041896503 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebceeb4b0 msgr2=0x55cebceeb8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.944 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.937+0000 7fa5d02c9b80 1 --2- 172.21.15.77:0/4041896503 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebceeb4b0 0x55cebceeb8f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa5b800ff00 tx=0x7fa5b8012000 comp rx=0 tx=0).stop 2023-12-01T01:17:40.944 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.937+0000 7fa5d02c9b80 1 -- 172.21.15.77:0/4041896503 shutdown_connections 2023-12-01T01:17:40.945 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.937+0000 7fa5d02c9b80 1 --2- 172.21.15.77:0/4041896503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cebceea7b0 0x55cebceeac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.945 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.937+0000 7fa5d02c9b80 1 --2- 172.21.15.77:0/4041896503 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebceeb4b0 0x55cebceeb8f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.945 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.937+0000 7fa5d02c9b80 1 --2- 172.21.15.77:0/4041896503 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cebceebe30 0x55cebceea220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.945 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.937+0000 7fa5d02c9b80 1 -- 172.21.15.77:0/4041896503 >> 172.21.15.77:0/4041896503 conn(0x55cebce45080 msgr2=0x55cebcee08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:40.945 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.937+0000 7fa5d02c9b80 1 -- 172.21.15.77:0/4041896503 shutdown_connections 2023-12-01T01:17:40.945 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.937+0000 7fa5d02c9b80 1 -- 172.21.15.77:0/4041896503 wait complete. 2023-12-01T01:17:40.945 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.938+0000 7fa5d02c9b80 1 -- start start 2023-12-01T01:17:40.945 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.938+0000 7fa5d02c9b80 1 Processor -- start 2023-12-01T01:17:40.945 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.938+0000 7fa5d02c9b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cebceea7b0 0x55cebcef3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.945 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.938+0000 7fa5d02c9b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cebceeb4b0 0x55cebcef42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.945 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.938+0000 7fa5d02c9b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebceebe30 0x55cebcefb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.945 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.938+0000 7fa5d02c9b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cebcdd34c0 con 0x55cebceebe30 2023-12-01T01:17:40.945 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.938+0000 7fa5d02c9b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cebcdd3160 con 0x55cebceeb4b0 2023-12-01T01:17:40.945 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.938+0000 7fa5d02c9b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cebccba8d0 con 0x55cebceea7b0 2023-12-01T01:17:40.945 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.938+0000 7fa5d02c9b80 10 client.0 _ll_register_callbacks cb 0x55cebcd20a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:17:40.945 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.938+0000 7fa5c26d6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebceebe30 0x55cebcefb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.946 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.938+0000 7fa5c26d6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebceebe30 0x55cebcefb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58946/0 (socket says 172.21.15.77:58946) 2023-12-01T01:17:40.946 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.938+0000 7fa5c26d6700 1 -- 172.21.15.77:0/2600421950 learned_addr learned my addr 172.21.15.77:0/2600421950 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:40.946 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.938+0000 7fa5c16d4700 1 --2- 172.21.15.77:0/2600421950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cebceeb4b0 0x55cebcef42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.946 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.938+0000 7fa5c1ed5700 1 --2- 172.21.15.77:0/2600421950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cebceea7b0 0x55cebcef3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.946 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.939+0000 7fa5c26d6700 1 -- 172.21.15.77:0/2600421950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cebceea7b0 msgr2=0x55cebcef3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:40.946 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.939+0000 7fa5c26d6700 1 --2- 172.21.15.77:0/2600421950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cebceea7b0 0x55cebcef3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.946 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.939+0000 7fa5c26d6700 1 -- 172.21.15.77:0/2600421950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cebceeb4b0 msgr2=0x55cebcef42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:40.946 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.939+0000 7fa5c26d6700 1 --2- 172.21.15.77:0/2600421950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cebceeb4b0 0x55cebcef42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.946 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.939+0000 7fa5c26d6700 1 -- 172.21.15.77:0/2600421950 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5b800fbb0 con 0x55cebceebe30 2023-12-01T01:17:40.946 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.939+0000 7fa5c26d6700 1 --2- 172.21.15.77:0/2600421950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebceebe30 0x55cebcefb320 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4007710 tx=0x7fa5b4007740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11058acc41a117ba server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:40.946 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.939+0000 7fa5a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:17:40.946 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.939+0000 7fa5a57fa700 1 -- 172.21.15.77:0/2600421950 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b400e040 con 0x55cebceebe30 2023-12-01T01:17:40.946 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.939+0000 7fa5a57fa700 1 -- 172.21.15.77:0/2600421950 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fa5b4003a00 con 0x55cebceebe30 2023-12-01T01:17:40.946 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.939+0000 7fa5d02c9b80 1 -- 172.21.15.77:0/2600421950 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cebce61cc0 con 0x55cebceebe30 2023-12-01T01:17:40.946 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.939+0000 7fa5a57fa700 1 -- 172.21.15.77:0/2600421950 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5b4008f10 con 0x55cebceebe30 2023-12-01T01:17:40.947 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.939+0000 7fa5d02c9b80 10 client.15330 fetch_fsmap learned FSMap version 27 2023-12-01T01:17:40.947 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.939+0000 7fa5d02c9b80 1 -- 172.21.15.77:0/2600421950 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55cebcc4dc10 con 0x55cebceebe30 2023-12-01T01:17:40.947 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.939+0000 7fa5a57fa700 1 -- 172.21.15.77:0/2600421950 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cebce61cc0 con 0x55cebceebe30 2023-12-01T01:17:40.947 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.939+0000 7fa5a57fa700 1 -- 172.21.15.77:0/2600421950 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fa5b400b100 con 0x55cebceebe30 2023-12-01T01:17:40.947 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.939+0000 7fa5d02c9b80 10 client.15330 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:17:40.947 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.939+0000 7fa5d02c9b80 10 client.15330 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-01T01:17:40.947 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.939+0000 7fa5d02c9b80 20 client.15330 populate_metadata read hostname 'smithi077' 2023-12-01T01:17:40.947 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.939+0000 7fa5d02c9b80 1 --2- 172.21.15.77:0/2600421950 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55cebceff070 0x55cebcefe470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.947 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.939+0000 7fa5d02c9b80 4 client.15330 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:17:40.947 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.939+0000 7fa5d02c9b80 1 -- 172.21.15.77:0/2600421950 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cebceb4dc0 con 0x55cebceff070 2023-12-01T01:17:40.947 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.940+0000 7fa5c1ed5700 1 --2- 172.21.15.77:0/2600421950 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55cebceff070 0x55cebcefe470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.947 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.940+0000 7fa5a57fa700 10 client.15330 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:17:40.947 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.940+0000 7fa5c1ed5700 1 --2- 172.21.15.77:0/2600421950 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55cebceff070 0x55cebcefe470 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3bf0b3997fa6046 server_cookie=801beda852b1f983 in_seq=0 out_seq=0 2023-12-01T01:17:40.947 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.941+0000 7fa5a57fa700 1 -- 172.21.15.77:0/2600421950 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55cebceb4dc0 con 0x55cebceff070 2023-12-01T01:17:40.947 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.941+0000 7fa5a57fa700 10 client.15330 handle_command_reply: tid=0 2023-12-01T01:17:40.948 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.941+0000 7fa5d02c9b80 1 client.15330 shutdown 2023-12-01T01:17:40.948 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.941+0000 7fa5d02c9b80 1 -- 172.21.15.77:0/2600421950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebceebe30 msgr2=0x55cebcefb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.948 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.941+0000 7fa5d02c9b80 1 --2- 172.21.15.77:0/2600421950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebceebe30 0x55cebcefb320 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4007710 tx=0x7fa5b4007740 comp rx=0 tx=0).stop 2023-12-01T01:17:40.948 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.941+0000 7fa5d02c9b80 1 -- 172.21.15.77:0/2600421950 shutdown_connections 2023-12-01T01:17:40.948 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.941+0000 7fa5d02c9b80 1 --2- 172.21.15.77:0/2600421950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebceebe30 0x55cebcefb320 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.948 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.941+0000 7fa5d02c9b80 1 --2- 172.21.15.77:0/2600421950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cebceea7b0 0x55cebcef3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.948 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.941+0000 7fa5d02c9b80 1 --2- 172.21.15.77:0/2600421950 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55cebceff070 0x55cebcefe470 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.948 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.941+0000 7fa5d02c9b80 1 --2- 172.21.15.77:0/2600421950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cebceeb4b0 0x55cebcef42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.948 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.941+0000 7fa5a57fa700 0 client.15330 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:17:40.948 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.941+0000 7fa5d02c9b80 1 -- 172.21.15.77:0/2600421950 >> 172.21.15.77:0/2600421950 conn(0x55cebce45080 msgr2=0x55cebcee0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:40.948 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.941+0000 7fa5d02c9b80 1 -- 172.21.15.77:0/2600421950 shutdown_connections 2023-12-01T01:17:40.948 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.941+0000 7fa5d02c9b80 1 -- 172.21.15.77:0/2600421950 wait complete. 2023-12-01T01:17:40.948 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.941+0000 7fa5d02c9b80 20 client.15330 trim_cache size 0 max 16384 2023-12-01T01:17:40.964 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.958+0000 7fa5d02c9b80 1 Processor -- start 2023-12-01T01:17:40.964 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.958+0000 7fa5d02c9b80 1 -- start start 2023-12-01T01:17:40.965 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.958+0000 7fa5d02c9b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cebce45080 0x55cebceee710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.965 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.958+0000 7fa5d02c9b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebceea7b0 0x55cebcf07da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.965 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.958+0000 7fa5d02c9b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cebceeb4b0 0x55cebceeec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.965 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.958+0000 7fa5d02c9b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cebcef2460 con 0x55cebceea7b0 2023-12-01T01:17:40.965 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.958+0000 7fa5d02c9b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cebceb4dc0 con 0x55cebceeb4b0 2023-12-01T01:17:40.965 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.958+0000 7fa5d02c9b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cebce61cc0 con 0x55cebce45080 2023-12-01T01:17:40.965 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.958+0000 7fa5c16d4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebceea7b0 0x55cebcf07da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.965 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.958+0000 7fa5c16d4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebceea7b0 0x55cebcf07da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58952/0 (socket says 172.21.15.77:58952) 2023-12-01T01:17:40.965 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.958+0000 7fa5c16d4700 1 -- 172.21.15.77:0/2326904472 learned_addr learned my addr 172.21.15.77:0/2326904472 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:40.965 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.958+0000 7fa5c1ed5700 1 --2- 172.21.15.77:0/2326904472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cebce45080 0x55cebceee710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.965 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.959+0000 7fa5c16d4700 1 -- 172.21.15.77:0/2326904472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cebce45080 msgr2=0x55cebceee710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:40.965 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.959+0000 7fa5c16d4700 1 --2- 172.21.15.77:0/2326904472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cebce45080 0x55cebceee710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.965 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.959+0000 7fa5c16d4700 1 -- 172.21.15.77:0/2326904472 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cebceeb4b0 msgr2=0x55cebceeec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:40.965 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.959+0000 7fa5c26d6700 1 --2- 172.21.15.77:0/2326904472 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cebceeb4b0 0x55cebceeec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.966 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.959+0000 7fa5c16d4700 1 --2- 172.21.15.77:0/2326904472 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cebceeb4b0 0x55cebceeec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.966 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.959+0000 7fa5c16d4700 1 -- 172.21.15.77:0/2326904472 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cebcc8b200 con 0x55cebceea7b0 2023-12-01T01:17:40.966 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.959+0000 7fa5c16d4700 1 --2- 172.21.15.77:0/2326904472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebceea7b0 0x55cebcf07da0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac0993b0 tx=0x7fa5ac010b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd86dd7fcc585d31 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:40.966 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.959+0000 7fa5b0ff9700 1 -- 172.21.15.77:0/2326904472 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac09d250 con 0x55cebceea7b0 2023-12-01T01:17:40.966 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.959+0000 7fa5b0ff9700 1 -- 172.21.15.77:0/2326904472 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5ac0a1490 con 0x55cebceea7b0 2023-12-01T01:17:40.966 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.959+0000 7fa5b0ff9700 1 -- 172.21.15.77:0/2326904472 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac098b70 con 0x55cebceea7b0 2023-12-01T01:17:40.966 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.959+0000 7fa5d02c9b80 1 -- 172.21.15.77:0/2326904472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebceea7b0 msgr2=0x55cebcf07da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.966 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.959+0000 7fa5d02c9b80 1 --2- 172.21.15.77:0/2326904472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebceea7b0 0x55cebcf07da0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac0993b0 tx=0x7fa5ac010b80 comp rx=0 tx=0).stop 2023-12-01T01:17:40.966 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.959+0000 7fa5d02c9b80 1 -- 172.21.15.77:0/2326904472 shutdown_connections 2023-12-01T01:17:40.966 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.959+0000 7fa5d02c9b80 1 --2- 172.21.15.77:0/2326904472 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cebceeb4b0 0x55cebceeec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.966 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.959+0000 7fa5d02c9b80 1 --2- 172.21.15.77:0/2326904472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cebce45080 0x55cebceee710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.966 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.959+0000 7fa5d02c9b80 1 --2- 172.21.15.77:0/2326904472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebceea7b0 0x55cebcf07da0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.966 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.959+0000 7fa5d02c9b80 1 -- 172.21.15.77:0/2326904472 >> 172.21.15.77:0/2326904472 conn(0x55cebcee0c20 msgr2=0x55cebcefe470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:40.966 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.960+0000 7fa5d02c9b80 1 -- 172.21.15.77:0/2326904472 shutdown_connections 2023-12-01T01:17:40.967 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.960+0000 7fa5d02c9b80 1 -- 172.21.15.77:0/2326904472 wait complete. 2023-12-01T01:17:40.967 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.960+0000 7fa5d02c9b80 1 -- start start 2023-12-01T01:17:40.967 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.960+0000 7fa5d02c9b80 1 Processor -- start 2023-12-01T01:17:40.967 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.960+0000 7fa5d02c9b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebce45080 0x55cebcf034d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.967 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.960+0000 7fa5d02c9b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cebceea7b0 0x55cebcf03a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.967 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.960+0000 7fa5d02c9b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cebceeb4b0 0x55cebcf06a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.967 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.960+0000 7fa5d02c9b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cebcf06f70 con 0x55cebce45080 2023-12-01T01:17:40.967 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.960+0000 7fa5d02c9b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cebcf070e0 con 0x55cebceea7b0 2023-12-01T01:17:40.967 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.960+0000 7fa5d02c9b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cebcf072a0 con 0x55cebceeb4b0 2023-12-01T01:17:40.967 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.960+0000 7fa5d02c9b80 10 client.0 _ll_register_callbacks cb 0x7fa5b4008c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:17:40.967 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.960+0000 7fa5c1ed5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebce45080 0x55cebcf034d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.967 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.960+0000 7fa5c1ed5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebce45080 0x55cebcf034d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58956/0 (socket says 172.21.15.77:58956) 2023-12-01T01:17:40.967 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.960+0000 7fa5c1ed5700 1 -- 172.21.15.77:0/1608447526 learned_addr learned my addr 172.21.15.77:0/1608447526 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:40.967 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.960+0000 7fa5c16d4700 1 --2- 172.21.15.77:0/1608447526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cebceea7b0 0x55cebcf03a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.967 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.960+0000 7fa5c26d6700 1 --2- 172.21.15.77:0/1608447526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cebceeb4b0 0x55cebcf06a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.968 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.961+0000 7fa5c1ed5700 1 -- 172.21.15.77:0/1608447526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cebceeb4b0 msgr2=0x55cebcf06a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:40.968 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.961+0000 7fa5c1ed5700 1 --2- 172.21.15.77:0/1608447526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cebceeb4b0 0x55cebcf06a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.968 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.961+0000 7fa5c1ed5700 1 -- 172.21.15.77:0/1608447526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cebceea7b0 msgr2=0x55cebcf03a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:40.968 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.961+0000 7fa5c1ed5700 1 --2- 172.21.15.77:0/1608447526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cebceea7b0 0x55cebcf03a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.968 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.961+0000 7fa5c1ed5700 1 -- 172.21.15.77:0/1608447526 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5bc19f1a0 con 0x55cebce45080 2023-12-01T01:17:40.968 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.961+0000 7fa5c16d4700 1 --2- 172.21.15.77:0/1608447526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cebceea7b0 0x55cebcf03a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:17:40.968 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.961+0000 7fa5c1ed5700 1 --2- 172.21.15.77:0/1608447526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebce45080 0x55cebcf034d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa5b800fea0 tx=0x7fa5b8005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e1eb655ac2f79e1 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:40.968 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.961+0000 7fa5a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:17:40.968 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.961+0000 7fa5a57fa700 1 -- 172.21.15.77:0/1608447526 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8035450 con 0x55cebce45080 2023-12-01T01:17:40.968 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.961+0000 7fa5a57fa700 1 -- 172.21.15.77:0/1608447526 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fa5b8004020 con 0x55cebce45080 2023-12-01T01:17:40.968 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.961+0000 7fa5d02c9b80 1 -- 172.21.15.77:0/1608447526 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa5b8011dc0 con 0x55cebce45080 2023-12-01T01:17:40.968 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.961+0000 7fa5a57fa700 1 -- 172.21.15.77:0/1608447526 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5b8004220 con 0x55cebce45080 2023-12-01T01:17:40.968 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.961+0000 7fa5d02c9b80 10 client.15336 fetch_fsmap learned FSMap version 27 2023-12-01T01:17:40.968 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.961+0000 7fa5d02c9b80 1 -- 172.21.15.77:0/1608447526 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55cebcc63260 con 0x55cebce45080 2023-12-01T01:17:40.968 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.961+0000 7fa5a57fa700 1 -- 172.21.15.77:0/1608447526 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5b8011dc0 con 0x55cebce45080 2023-12-01T01:17:40.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.961+0000 7fa5a57fa700 1 -- 172.21.15.77:0/1608447526 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fa5b80036f0 con 0x55cebce45080 2023-12-01T01:17:40.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.961+0000 7fa5d02c9b80 10 client.15336 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:17:40.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.961+0000 7fa5d02c9b80 10 client.15336 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-01T01:17:40.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.961+0000 7fa5d02c9b80 20 client.15336 populate_metadata read hostname 'smithi077' 2023-12-01T01:17:40.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.961+0000 7fa5d02c9b80 1 --2- 172.21.15.77:0/1608447526 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55cebceebe30 0x55cebcf531c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:40.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.961+0000 7fa5d02c9b80 4 client.15336 mds_command: new command op to 24500 tid=0[{"prefix": "status"}] 2023-12-01T01:17:40.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.961+0000 7fa5d02c9b80 1 -- 172.21.15.77:0/1608447526 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55cebce2fcf0 con 0x55cebceebe30 2023-12-01T01:17:40.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.962+0000 7fa5c16d4700 1 --2- 172.21.15.77:0/1608447526 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55cebceebe30 0x55cebcf531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:40.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.962+0000 7fa5c16d4700 1 --2- 172.21.15.77:0/1608447526 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55cebceebe30 0x55cebcf531c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae26eb75c1bff04a server_cookie=b1c0f7882074e8d5 in_seq=0 out_seq=0 2023-12-01T01:17:40.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.962+0000 7fa5a57fa700 10 client.15336 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:17:40.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.962+0000 7fa5a57fa700 1 -- 172.21.15.77:0/1608447526 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55cebce2fcf0 con 0x55cebceebe30 2023-12-01T01:17:40.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.962+0000 7fa5a57fa700 10 client.15336 handle_command_reply: tid=0 2023-12-01T01:17:40.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.962+0000 7fa5d02c9b80 1 client.15336 shutdown 2023-12-01T01:17:40.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.962+0000 7fa5d02c9b80 1 -- 172.21.15.77:0/1608447526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebce45080 msgr2=0x55cebcf034d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.962+0000 7fa5d02c9b80 1 --2- 172.21.15.77:0/1608447526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebce45080 0x55cebcf034d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa5b800fea0 tx=0x7fa5b8005ee0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.970 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.962+0000 7fa5d02c9b80 1 -- 172.21.15.77:0/1608447526 shutdown_connections 2023-12-01T01:17:40.970 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.962+0000 7fa5d02c9b80 1 --2- 172.21.15.77:0/1608447526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cebceeb4b0 0x55cebcf06a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.970 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.962+0000 7fa5d02c9b80 1 --2- 172.21.15.77:0/1608447526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cebce45080 0x55cebcf034d0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.970 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.962+0000 7fa5d02c9b80 1 --2- 172.21.15.77:0/1608447526 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55cebceebe30 0x55cebcf531c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.970 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.962+0000 7fa5d02c9b80 1 --2- 172.21.15.77:0/1608447526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cebceea7b0 0x55cebcf03a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.970 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.962+0000 7fa5a57fa700 0 client.15336 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:17:40.970 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.962+0000 7fa5d02c9b80 1 -- 172.21.15.77:0/1608447526 >> 172.21.15.77:0/1608447526 conn(0x55cebcee0c20 msgr2=0x55cebceee710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:40.970 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.962+0000 7fa5d02c9b80 1 -- 172.21.15.77:0/1608447526 shutdown_connections 2023-12-01T01:17:40.970 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.962+0000 7fa5d02c9b80 1 -- 172.21.15.77:0/1608447526 wait complete. 2023-12-01T01:17:40.970 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.963+0000 7fa5d02c9b80 20 client.15336 trim_cache size 0 max 16384 2023-12-01T01:17:40.970 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2023-12-01T01:17:40.971 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "2a06e29a-8fe6-11ee-95a2-87774f69a715", 2023-12-01T01:17:40.971 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 0, 2023-12-01T01:17:40.971 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24500, 2023-12-01T01:17:40.971 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2023-12-01T01:17:40.971 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2023-12-01T01:17:40.971 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2023-12-01T01:17:40.971 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 75.615598692000006, 2023-12-01T01:17:40.971 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 27, 2023-12-01T01:17:40.971 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 84, 2023-12-01T01:17:40.971 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 82, 2023-12-01T01:17:40.971 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 81.093571373000003 2023-12-01T01:17:40.971 INFO:teuthology.orchestra.run.smithi077.stdout:} 2023-12-01T01:17:40.972 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.963+0000 7fa5c4139700 1 -- 172.21.15.77:0/423081746 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa5a80a0630 msgr2=0x7fa5a80a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.972 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.963+0000 7fa5c4139700 1 --2- 172.21.15.77:0/423081746 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa5a80a0630 0x7fa5a80a2af0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8005950 tx=0x7fa5b80058e0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.972 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.963+0000 7fa5c4139700 1 -- 172.21.15.77:0/423081746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5bc101fe0 msgr2=0x7fa5bc1970a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:40.972 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.963+0000 7fa5c4139700 1 --2- 172.21.15.77:0/423081746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5bc101fe0 0x7fa5bc1970a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac00e9a0 tx=0x7fa5ac00ecb0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.972 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.963+0000 7fa5c4139700 1 -- 172.21.15.77:0/423081746 shutdown_connections 2023-12-01T01:17:40.972 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.963+0000 7fa5c4139700 1 --2- 172.21.15.77:0/423081746 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa5a80a0630 0x7fa5a80a2af0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.973 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.963+0000 7fa5c4139700 1 --2- 172.21.15.77:0/423081746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5bc102980 0x7fa5bc19e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.973 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.963+0000 7fa5c4139700 1 --2- 172.21.15.77:0/423081746 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5bc100df0 0x7fa5bc196b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.973 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.963+0000 7fa5c4139700 1 --2- 172.21.15.77:0/423081746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5bc101fe0 0x7fa5bc1970a0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:40.973 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.963+0000 7fa5c4139700 1 -- 172.21.15.77:0/423081746 >> 172.21.15.77:0/423081746 conn(0x7fa5bc0fc360 msgr2=0x7fa5bc1054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:40.973 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.963+0000 7fa5c4139700 1 -- 172.21.15.77:0/423081746 shutdown_connections 2023-12-01T01:17:40.973 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:40.963+0000 7fa5c4139700 1 -- 172.21.15.77:0/423081746 wait complete. 2023-12-01T01:17:40.976 INFO:teuthology.run_tasks:Running task kclient... 2023-12-01T01:17:40.998 INFO:tasks.kclient:Mounting kernel clients... 2023-12-01T01:17:40.999 INFO:tasks.kclient:config is {'client.0': {}} 2023-12-01T01:17:40.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:17:40.999 DEBUG:teuthology.orchestra.run.smithi077:> ip netns list 2023-12-01T01:17:41.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:40 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/3263773577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:17:41.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:40 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/3237973519' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:17:41.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:40 smithi203 ceph-mon[103581]: pgmap v292: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 14 B/s, 0 keys/s, 0 objects/s recovering 2023-12-01T01:17:41.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:17:41.016 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link delete ceph-brx 2023-12-01T01:17:41.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:40 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/3263773577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:17:41.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:40 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/3237973519' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:17:41.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:40 smithi137 ceph-mon[103312]: pgmap v292: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 14 B/s, 0 keys/s, 0 objects/s recovering 2023-12-01T01:17:41.084 INFO:teuthology.orchestra.run.smithi077.stderr:Cannot find device "ceph-brx" 2023-12-01T01:17:41.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-01T01:17:41.086 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2023-12-01T01:17:41.086 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-01T01:17:41.086 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-01T01:17:41.086 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph fs ls 2023-12-01T01:17:41.675 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:17:42.895 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:42.892+0000 7fd6d5cd9700 1 -- 172.21.15.77:0/1195877574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6d01005f0 msgr2=0x7fd6d01009d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:42.895 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:42.892+0000 7fd6d5cd9700 1 --2- 172.21.15.77:0/1195877574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6d01005f0 0x7fd6d01009d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4009990 tx=0x7fd6c4009ca0 comp rx=0 tx=0).stop 2023-12-01T01:17:42.895 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:42.892+0000 7fd6d5cd9700 1 -- 172.21.15.77:0/1195877574 shutdown_connections 2023-12-01T01:17:42.895 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:42.892+0000 7fd6d5cd9700 1 --2- 172.21.15.77:0/1195877574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6d0104fe0 0x7fd6d010cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:42.895 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:42.892+0000 7fd6d5cd9700 1 --2- 172.21.15.77:0/1195877574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6d01005f0 0x7fd6d01009d0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:42.896 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:42.892+0000 7fd6d5cd9700 1 --2- 172.21.15.77:0/1195877574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6d0100fa0 0x7fd6d0104900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:42.896 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:42.892+0000 7fd6d5cd9700 1 -- 172.21.15.77:0/1195877574 >> 172.21.15.77:0/1195877574 conn(0x7fd6d0076200 msgr2=0x7fd6d00fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:42.896 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:42.892+0000 7fd6d5cd9700 1 -- 172.21.15.77:0/1195877574 shutdown_connections 2023-12-01T01:17:42.896 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:42.892+0000 7fd6d5cd9700 1 -- 172.21.15.77:0/1195877574 wait complete. 2023-12-01T01:17:42.896 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:42.892+0000 7fd6d5cd9700 1 Processor -- start 2023-12-01T01:17:42.896 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:42.892+0000 7fd6d5cd9700 1 -- start start 2023-12-01T01:17:42.896 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:42.893+0000 7fd6d5cd9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6d01005f0 0x7fd6d0197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:42.896 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:42.893+0000 7fd6d5cd9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6d0100fa0 0x7fd6d0198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:42.896 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:42.893+0000 7fd6d5cd9700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6d0104fe0 0x7fd6d01a2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:42.896 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:42.893+0000 7fd6d5cd9700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d0198b40 con 0x7fd6d0100fa0 2023-12-01T01:17:42.896 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:42.893+0000 7fd6d5cd9700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d01a3280 con 0x7fd6d0104fe0 2023-12-01T01:17:42.896 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:42.893+0000 7fd6d5cd9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d01a3440 con 0x7fd6d01005f0 2023-12-01T01:17:42.896 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:42.893+0000 7fd6ceffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6d0100fa0 0x7fd6d0198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:42.897 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:42.893+0000 7fd6ceffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6d0100fa0 0x7fd6d0198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58970/0 (socket says 172.21.15.77:58970) 2023-12-01T01:17:42.897 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:42.893+0000 7fd6ceffd700 1 -- 172.21.15.77:0/1122505504 learned_addr learned my addr 172.21.15.77:0/1122505504 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:42.897 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:42.893+0000 7fd6cffff700 1 --2- 172.21.15.77:0/1122505504 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6d0104fe0 0x7fd6d01a2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:42.897 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:42.893+0000 7fd6cf7fe700 1 --2- 172.21.15.77:0/1122505504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6d01005f0 0x7fd6d0197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:42.897 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:42.893+0000 7fd6ceffd700 1 -- 172.21.15.77:0/1122505504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6d01005f0 msgr2=0x7fd6d0197cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:42.897 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:42.893+0000 7fd6ceffd700 1 --2- 172.21.15.77:0/1122505504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6d01005f0 0x7fd6d0197cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:42.897 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:42.893+0000 7fd6ceffd700 1 -- 172.21.15.77:0/1122505504 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6d0104fe0 msgr2=0x7fd6d01a2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:42.897 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:42.893+0000 7fd6ceffd700 1 --2- 172.21.15.77:0/1122505504 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6d0104fe0 0x7fd6d01a2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:42.897 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:42.893+0000 7fd6ceffd700 1 -- 172.21.15.77:0/1122505504 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c40095f0 con 0x7fd6d0100fa0 2023-12-01T01:17:42.897 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:42.893+0000 7fd6cf7fe700 1 --2- 172.21.15.77:0/1122505504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6d01005f0 0x7fd6d0197cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:17:42.897 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:42.893+0000 7fd6ceffd700 1 --2- 172.21.15.77:0/1122505504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6d0100fa0 0x7fd6d0198200 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd6c000c8f0 tx=0x7fd6c000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:42.897 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:42.893+0000 7fd6ccff9700 1 -- 172.21.15.77:0/1122505504 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0007940 con 0x7fd6d0100fa0 2023-12-01T01:17:42.897 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:42.893+0000 7fd6d5cd9700 1 -- 172.21.15.77:0/1122505504 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6d01a3720 con 0x7fd6d0100fa0 2023-12-01T01:17:42.897 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:42.893+0000 7fd6ccff9700 1 -- 172.21.15.77:0/1122505504 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6c000f450 con 0x7fd6d0100fa0 2023-12-01T01:17:42.898 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:42.893+0000 7fd6ccff9700 1 -- 172.21.15.77:0/1122505504 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0018660 con 0x7fd6d0100fa0 2023-12-01T01:17:42.898 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:42.893+0000 7fd6d5cd9700 1 -- 172.21.15.77:0/1122505504 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6d01a3c70 con 0x7fd6d0100fa0 2023-12-01T01:17:42.898 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:42.894+0000 7fd6d5cd9700 1 -- 172.21.15.77:0/1122505504 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6bc005320 con 0x7fd6d0100fa0 2023-12-01T01:17:42.898 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:42.895+0000 7fd6ccff9700 1 -- 172.21.15.77:0/1122505504 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fd6c0007aa0 con 0x7fd6d0100fa0 2023-12-01T01:17:42.899 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:42.896+0000 7fd6ccff9700 1 --2- 172.21.15.77:0/1122505504 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd6b80a05b0 0x7fd6b80a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:42.899 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:42.896+0000 7fd6ccff9700 1 -- 172.21.15.77:0/1122505504 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fd6c00cb900 con 0x7fd6d0100fa0 2023-12-01T01:17:42.899 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:42.896+0000 7fd6cf7fe700 1 --2- 172.21.15.77:0/1122505504 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd6b80a05b0 0x7fd6b80a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:42.899 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:42.896+0000 7fd6cf7fe700 1 --2- 172.21.15.77:0/1122505504 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd6b80a05b0 0x7fd6b80a2a70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd6c400b2e0 tx=0x7fd6c4005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:42.900 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:42.899+0000 7fd6ccff9700 1 -- 172.21.15.77:0/1122505504 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd6c00954d0 con 0x7fd6d0100fa0 2023-12-01T01:17:43.131 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:43.129+0000 7fd6d5cd9700 1 -- 172.21.15.77:0/1122505504 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd6bc005cc0 con 0x7fd6d0100fa0 2023-12-01T01:17:43.131 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:43.130+0000 7fd6ccff9700 1 -- 172.21.15.77:0/1122505504 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd6c0098af0 con 0x7fd6d0100fa0 2023-12-01T01:17:43.131 INFO:teuthology.orchestra.run.smithi077.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-01T01:17:43.133 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:43.132+0000 7fd6d5cd9700 1 -- 172.21.15.77:0/1122505504 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd6b80a05b0 msgr2=0x7fd6b80a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:43.133 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:43.132+0000 7fd6d5cd9700 1 --2- 172.21.15.77:0/1122505504 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd6b80a05b0 0x7fd6b80a2a70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd6c400b2e0 tx=0x7fd6c4005e80 comp rx=0 tx=0).stop 2023-12-01T01:17:43.134 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:43.132+0000 7fd6d5cd9700 1 -- 172.21.15.77:0/1122505504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6d0100fa0 msgr2=0x7fd6d0198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:43.134 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:43.132+0000 7fd6d5cd9700 1 --2- 172.21.15.77:0/1122505504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6d0100fa0 0x7fd6d0198200 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd6c000c8f0 tx=0x7fd6c000cc00 comp rx=0 tx=0).stop 2023-12-01T01:17:43.134 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:43.132+0000 7fd6d5cd9700 1 -- 172.21.15.77:0/1122505504 shutdown_connections 2023-12-01T01:17:43.134 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:43.132+0000 7fd6d5cd9700 1 --2- 172.21.15.77:0/1122505504 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd6b80a05b0 0x7fd6b80a2a70 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:43.134 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:43.132+0000 7fd6d5cd9700 1 --2- 172.21.15.77:0/1122505504 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6d0104fe0 0x7fd6d01a2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:43.134 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:43.132+0000 7fd6d5cd9700 1 --2- 172.21.15.77:0/1122505504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6d01005f0 0x7fd6d0197cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:43.134 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:43.132+0000 7fd6d5cd9700 1 --2- 172.21.15.77:0/1122505504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6d0100fa0 0x7fd6d0198200 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:43.134 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:43.132+0000 7fd6d5cd9700 1 -- 172.21.15.77:0/1122505504 >> 172.21.15.77:0/1122505504 conn(0x7fd6d0076200 msgr2=0x7fd6d0103890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:43.134 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:43.132+0000 7fd6d5cd9700 1 -- 172.21.15.77:0/1122505504 shutdown_connections 2023-12-01T01:17:43.134 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:43.132+0000 7fd6d5cd9700 1 -- 172.21.15.77:0/1122505504 wait complete. 2023-12-01T01:17:43.530 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-01T01:17:43.530 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-01T01:17:43.530 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi077.front.sepia.ceph.com 2023-12-01T01:17:43.530 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-01T01:17:43.530 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-01T01:17:43.531 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-01T01:17:43.531 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-01T01:17:43.531 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-01T01:17:43.531 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-01T01:17:43.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:17:43.531 DEBUG:teuthology.orchestra.run.smithi077:> ip addr 2023-12-01T01:17:43.549 INFO:teuthology.orchestra.run.smithi077.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-01T01:17:43.550 INFO:teuthology.orchestra.run.smithi077.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-01T01:17:43.550 INFO:teuthology.orchestra.run.smithi077.stdout: inet 127.0.0.1/8 scope host lo 2023-12-01T01:17:43.550 INFO:teuthology.orchestra.run.smithi077.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:17:43.550 INFO:teuthology.orchestra.run.smithi077.stdout: inet6 ::1/128 scope host 2023-12-01T01:17:43.550 INFO:teuthology.orchestra.run.smithi077.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:17:43.550 INFO:teuthology.orchestra.run.smithi077.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T01:17:43.550 INFO:teuthology.orchestra.run.smithi077.stdout: link/ether 0c:c4:7a:d8:d9:d8 brd ff:ff:ff:ff:ff:ff 2023-12-01T01:17:43.550 INFO:teuthology.orchestra.run.smithi077.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T01:17:43.550 INFO:teuthology.orchestra.run.smithi077.stdout: link/ether 0c:c4:7a:d8:d9:d9 brd ff:ff:ff:ff:ff:ff 2023-12-01T01:17:43.550 INFO:teuthology.orchestra.run.smithi077.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-01T01:17:43.550 INFO:teuthology.orchestra.run.smithi077.stdout: link/ether 0c:c4:7a:8f:d3:d2 brd ff:ff:ff:ff:ff:ff 2023-12-01T01:17:43.551 INFO:teuthology.orchestra.run.smithi077.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-01T01:17:43.551 INFO:teuthology.orchestra.run.smithi077.stdout: link/ether 0c:c4:7a:8f:d3:d3 brd ff:ff:ff:ff:ff:ff 2023-12-01T01:17:43.551 INFO:teuthology.orchestra.run.smithi077.stdout: inet 172.21.15.77/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-01T01:17:43.551 INFO:teuthology.orchestra.run.smithi077.stdout: valid_lft 41861sec preferred_lft 41861sec 2023-12-01T01:17:43.551 INFO:teuthology.orchestra.run.smithi077.stdout: inet6 fe80::579d:dec9:4881:625c/64 scope link noprefixroute 2023-12-01T01:17:43.551 INFO:teuthology.orchestra.run.smithi077.stdout: valid_lft forever preferred_lft forever 2023-12-01T01:17:43.551 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-12-01T01:17:43.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:17:43.552 DEBUG:teuthology.orchestra.run.smithi077:> (cd / && exec bash -c ' 2023-12-01T01:17:43.552 DEBUG:teuthology.orchestra.run.smithi077:> set -e 2023-12-01T01:17:43.552 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link add name ceph-brx type bridge 2023-12-01T01:17:43.552 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip addr flush dev ceph-brx 2023-12-01T01:17:43.552 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link set ceph-brx up 2023-12-01T01:17:43.552 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-12-01T01:17:43.552 DEBUG:teuthology.orchestra.run.smithi077:> ') 2023-12-01T01:17:43.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:17:43.716 DEBUG:teuthology.orchestra.run.smithi077:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-01T01:17:43.720 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:43 smithi077 ceph-mon[98867]: pgmap v293: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s; 14 B/s, 0 keys/s, 0 objects/s recovering 2023-12-01T01:17:43.720 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:43 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/1122505504' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-12-01T01:17:43.744 INFO:teuthology.orchestra.run.smithi077.stdout:1 2023-12-01T01:17:43.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:17:43.745 DEBUG:teuthology.orchestra.run.smithi077:> route 2023-12-01T01:17:43.815 INFO:teuthology.orchestra.run.smithi077.stdout:Kernel IP routing table 2023-12-01T01:17:43.815 INFO:teuthology.orchestra.run.smithi077.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-01T01:17:43.816 INFO:teuthology.orchestra.run.smithi077.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-01T01:17:43.816 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-01T01:17:43.816 INFO:teuthology.orchestra.run.smithi077.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-01T01:17:43.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:17:43.816 DEBUG:teuthology.orchestra.run.smithi077:> (cd / && exec bash -c ' 2023-12-01T01:17:43.816 DEBUG:teuthology.orchestra.run.smithi077:> set -e 2023-12-01T01:17:43.816 DEBUG:teuthology.orchestra.run.smithi077:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-01T01:17:43.816 DEBUG:teuthology.orchestra.run.smithi077:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-01T01:17:43.817 DEBUG:teuthology.orchestra.run.smithi077:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-01T01:17:43.817 DEBUG:teuthology.orchestra.run.smithi077:> ') 2023-12-01T01:17:44.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:43 smithi203 ceph-mon[103581]: pgmap v293: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s; 14 B/s, 0 keys/s, 0 objects/s recovering 2023-12-01T01:17:44.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:43 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/1122505504' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-12-01T01:17:44.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:43 smithi137 ceph-mon[103312]: pgmap v293: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s; 14 B/s, 0 keys/s, 0 objects/s recovering 2023-12-01T01:17:44.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:43 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/1122505504' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-12-01T01:17:44.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:17:44.067 DEBUG:teuthology.orchestra.run.smithi077:> ip netns list 2023-12-01T01:17:44.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:17:44.085 DEBUG:teuthology.orchestra.run.smithi077:> ip netns list-id 2023-12-01T01:17:44.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:17:44.142 DEBUG:teuthology.orchestra.run.smithi077:> (cd / && exec bash -c ' 2023-12-01T01:17:44.142 DEBUG:teuthology.orchestra.run.smithi077:> set -e 2023-12-01T01:17:44.142 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-01T01:17:44.142 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-12-01T01:17:44.142 DEBUG:teuthology.orchestra.run.smithi077:> ') 2023-12-01T01:17:44.237 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-12-01T01:17:44.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:17:44.237 DEBUG:teuthology.orchestra.run.smithi077:> (cd / && exec bash -c ' 2023-12-01T01:17:44.238 DEBUG:teuthology.orchestra.run.smithi077:> set -e 2023-12-01T01:17:44.238 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-12-01T01:17:44.238 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-12-01T01:17:44.238 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-12-01T01:17:44.238 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-12-01T01:17:44.238 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-12-01T01:17:44.238 DEBUG:teuthology.orchestra.run.smithi077:> ') 2023-12-01T01:17:44.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:17:44.453 DEBUG:teuthology.orchestra.run.smithi077:> (cd / && exec bash -c ' 2023-12-01T01:17:44.453 DEBUG:teuthology.orchestra.run.smithi077:> set -e 2023-12-01T01:17:44.453 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link set brx.0 up 2023-12-01T01:17:44.453 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link set dev brx.0 master ceph-brx 2023-12-01T01:17:44.453 DEBUG:teuthology.orchestra.run.smithi077:> ') 2023-12-01T01:17:44.551 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-01T01:17:44.552 DEBUG:teuthology.orchestra.run.smithi077:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-01T01:17:44.569 INFO:teuthology.orchestra.run.smithi077.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-01T01:17:44.569 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-01T01:17:44.569 DEBUG:teuthology.orchestra.run.smithi077:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-01T01:17:44.626 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2023-12-01T01:17:44.626 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2023-12-01T01:17:44.626 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 2023-12-01T01:17:44.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:17:44.626 DEBUG:teuthology.orchestra.run.smithi077:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2023-12-01T01:17:44.724 INFO:teuthology.orchestra.run.smithi077.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2023-12-01T01:17:44.724 INFO:teuthology.orchestra.run.smithi077.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2023-12-01T01:17:44.874 INFO:teuthology.orchestra.run.smithi077.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2023-12-01T01:17:44.874 INFO:teuthology.orchestra.run.smithi077.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2023-12-01T01:17:44.874 INFO:teuthology.orchestra.run.smithi077.stdout:mount.ceph: resolved to: "172.21.15.77:3300,172.21.15.137:3300,172.21.15.203:3300" 2023-12-01T01:17:44.874 INFO:teuthology.orchestra.run.smithi077.stdout:mount.ceph: trying mount with new device syntax: 0@2a06e29a-8fe6-11ee-95a2-87774f69a715.cephfs=/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 2023-12-01T01:17:44.874 INFO:teuthology.orchestra.run.smithi077.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync,name=0,key=0,mon_addr=172.21.15.77:3300/172.21.15.137:3300/172.21.15.203:3300" will pass to kernel 2023-12-01T01:17:44.874 INFO:teuthology.orchestra.run.smithi077.stdout:mount.ceph: v2 mount support check returned 2 2023-12-01T01:17:44.874 INFO:teuthology.orchestra.run.smithi077.stdout:mount.ceph: kernel does not support v2 syntax 2023-12-01T01:17:44.875 INFO:teuthology.orchestra.run.smithi077.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.77:3300,172.21.15.137:3300,172.21.15.203:3300:/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 2023-12-01T01:17:44.875 INFO:teuthology.orchestra.run.smithi077.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync,name=0,key=0,mds_namespace=cephfs,fsid=2a06e29a-8fe6-11ee-95a2-87774f69a715" will pass to kernel 2023-12-01T01:17:44.878 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-01T01:17:44.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:17:44.879 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-01T01:17:44.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:17:44.909 DEBUG:teuthology.orchestra.run.smithi077:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-01T01:17:45.003 INFO:teuthology.orchestra.run.smithi077.stdout:client24992 2023-12-01T01:17:45.004 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-12-01T01:17:45.004 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:17:45.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:17:45.022 DEBUG:teuthology.orchestra.run.smithi077:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-01T01:17:45.112 INFO:teuthology.orchestra.run.smithi077.stdout:client24992 2023-12-01T01:17:45.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:17:45.113 DEBUG:teuthology.orchestra.run.smithi077:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-01T01:17:45.165 INFO:teuthology.orchestra.run.smithi077.stdout:client24992 2023-12-01T01:17:45.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:17:45.165 DEBUG:teuthology.orchestra.run.smithi077:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715.client24992/status') 2023-12-01T01:17:45.222 INFO:teuthology.orchestra.run.smithi077.stdout:instance: client.24992 (3)172.21.15.77:0/1858281526 2023-12-01T01:17:45.223 INFO:teuthology.orchestra.run.smithi077.stdout:blocklisted: false 2023-12-01T01:17:45.223 INFO:teuthology.orchestra.run.smithi077.stderr:0+1 records in 2023-12-01T01:17:45.223 INFO:teuthology.orchestra.run.smithi077.stderr:0+1 records out 2023-12-01T01:17:45.223 INFO:teuthology.orchestra.run.smithi077.stderr:71 bytes copied, 2.3569e-05 s, 3.0 MB/s 2023-12-01T01:17:45.224 INFO:teuthology.run_tasks:Running task check-counter... 2023-12-01T01:17:45.236 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-12-01T01:17:45.249 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:17:45.375 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.373+0000 7fc06ad71700 1 -- 172.21.15.77:0/979510859 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0640ff5e0 msgr2=0x7fc0640ffa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:45.375 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.373+0000 7fc06ad71700 1 --2- 172.21.15.77:0/979510859 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0640ff5e0 0x7fc0640ffa00 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc058009990 tx=0x7fc058009ca0 comp rx=0 tx=0).stop 2023-12-01T01:17:45.375 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.373+0000 7fc06ad71700 1 -- 172.21.15.77:0/979510859 shutdown_connections 2023-12-01T01:17:45.375 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.373+0000 7fc06ad71700 1 --2- 172.21.15.77:0/979510859 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc064100900 0x7fc064100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:45.375 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.373+0000 7fc06ad71700 1 --2- 172.21.15.77:0/979510859 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0640ff5e0 0x7fc0640ffa00 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:45.375 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.373+0000 7fc06ad71700 1 --2- 172.21.15.77:0/979510859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0640fff40 0x7fc0641003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:45.375 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.373+0000 7fc06ad71700 1 -- 172.21.15.77:0/979510859 >> 172.21.15.77:0/979510859 conn(0x7fc0640fb1c0 msgr2=0x7fc0640fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:45.375 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.373+0000 7fc06ad71700 1 -- 172.21.15.77:0/979510859 shutdown_connections 2023-12-01T01:17:45.375 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.373+0000 7fc06ad71700 1 -- 172.21.15.77:0/979510859 wait complete. 2023-12-01T01:17:45.377 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.374+0000 7fc06ad71700 1 Processor -- start 2023-12-01T01:17:45.377 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.374+0000 7fc06ad71700 1 -- start start 2023-12-01T01:17:45.377 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.374+0000 7fc06ad71700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0640ff5e0 0x7fc064198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:45.377 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.374+0000 7fc06ad71700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0640fff40 0x7fc064199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:45.377 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.374+0000 7fc06ad71700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc064100900 0x7fc0641a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:45.377 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.374+0000 7fc06ad71700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0641a0760 con 0x7fc0640fff40 2023-12-01T01:17:45.377 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.374+0000 7fc06ad71700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0641a08a0 con 0x7fc0640ff5e0 2023-12-01T01:17:45.378 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.374+0000 7fc06ad71700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0641a0a30 con 0x7fc064100900 2023-12-01T01:17:45.378 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.374+0000 7fc063fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0640fff40 0x7fc064199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:45.378 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.374+0000 7fc063fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0640fff40 0x7fc064199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59010/0 (socket says 172.21.15.77:59010) 2023-12-01T01:17:45.378 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.374+0000 7fc068b0d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0640ff5e0 0x7fc064198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:45.378 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.374+0000 7fc063fff700 1 -- 172.21.15.77:0/2226894236 learned_addr learned my addr 172.21.15.77:0/2226894236 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:45.378 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.374+0000 7fc06930e700 1 --2- 172.21.15.77:0/2226894236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc064100900 0x7fc0641a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:45.378 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.374+0000 7fc063fff700 1 -- 172.21.15.77:0/2226894236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc064100900 msgr2=0x7fc0641a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:45.378 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.374+0000 7fc063fff700 1 --2- 172.21.15.77:0/2226894236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc064100900 0x7fc0641a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:45.378 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.374+0000 7fc063fff700 1 -- 172.21.15.77:0/2226894236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0640ff5e0 msgr2=0x7fc064198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:45.378 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.374+0000 7fc063fff700 1 --2- 172.21.15.77:0/2226894236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0640ff5e0 0x7fc064198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:45.378 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.374+0000 7fc063fff700 1 -- 172.21.15.77:0/2226894236 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0580095f0 con 0x7fc0640fff40 2023-12-01T01:17:45.378 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.374+0000 7fc068b0d700 1 --2- 172.21.15.77:0/2226894236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0640ff5e0 0x7fc064198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:17:45.378 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.375+0000 7fc063fff700 1 --2- 172.21.15.77:0/2226894236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0640fff40 0x7fc064199120 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc05000d980 tx=0x7fc05000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:45.378 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.375+0000 7fc061ffb700 1 -- 172.21.15.77:0/2226894236 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc050004220 con 0x7fc0640fff40 2023-12-01T01:17:45.379 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.375+0000 7fc061ffb700 1 -- 172.21.15.77:0/2226894236 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc050007640 con 0x7fc0640fff40 2023-12-01T01:17:45.379 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.375+0000 7fc06ad71700 1 -- 172.21.15.77:0/2226894236 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0641a0d10 con 0x7fc0640fff40 2023-12-01T01:17:45.379 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.375+0000 7fc061ffb700 1 -- 172.21.15.77:0/2226894236 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc050005690 con 0x7fc0640fff40 2023-12-01T01:17:45.379 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.375+0000 7fc06ad71700 1 -- 172.21.15.77:0/2226894236 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0641a11d0 con 0x7fc0640fff40 2023-12-01T01:17:45.379 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.376+0000 7fc06ad71700 1 -- 172.21.15.77:0/2226894236 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc06410a3c0 con 0x7fc0640fff40 2023-12-01T01:17:45.380 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.377+0000 7fc061ffb700 1 -- 172.21.15.77:0/2226894236 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fc050004750 con 0x7fc0640fff40 2023-12-01T01:17:45.380 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.377+0000 7fc061ffb700 1 --2- 172.21.15.77:0/2226894236 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc0540a05f0 0x7fc0540a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:45.380 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.378+0000 7fc061ffb700 1 -- 172.21.15.77:0/2226894236 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fc0500cb950 con 0x7fc0640fff40 2023-12-01T01:17:45.380 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.378+0000 7fc068b0d700 1 --2- 172.21.15.77:0/2226894236 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc0540a05f0 0x7fc0540a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:45.380 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.378+0000 7fc068b0d700 1 --2- 172.21.15.77:0/2226894236 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc0540a05f0 0x7fc0540a2ab0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc05800b2e0 tx=0x7fc058005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:45.383 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.382+0000 7fc061ffb700 1 -- 172.21.15.77:0/2226894236 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc050095390 con 0x7fc0640fff40 2023-12-01T01:17:45.578 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.576+0000 7fc06ad71700 1 -- 172.21.15.77:0/2226894236 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc06402cec0 con 0x7fc0640fff40 2023-12-01T01:17:45.579 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.578+0000 7fc061ffb700 1 -- 172.21.15.77:0/2226894236 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22472 (secure 0 0 0) 0x7fc050016020 con 0x7fc0640fff40 2023-12-01T01:17:45.579 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:17:45.580 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":84,"fsid":"2a06e29a-8fe6-11ee-95a2-87774f69a715","created":"2023-12-01T01:09:30.210398+0000","modified":"2023-12-01T01:17:25.609766+0000","last_up_change":"2023-12-01T01:15:44.586092+0000","last_in_change":"2023-12-01T01:15:26.305439+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:12:30.514229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:16:23.568860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-01T01:16:24.531137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fc651e62-a103-4ab8-a76d-d3708ea6aeec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6803","nonce":2244799094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6805","nonce":2244799094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6809","nonce":2244799094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6807","nonce":2244799094}]},"public_addr":"172.21.15.77:6803/2244799094","cluster_addr":"172.21.15.77:6805/2244799094","heartbeat_back_addr":"172.21.15.77:6809/2244799094","heartbeat_front_addr":"172.21.15.77:6807/2244799094","state":["exists","up"]},{"osd":1,"uuid":"0adf8c54-49ec-49a0-8bf1-af1f20e60729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6801","nonce":224194890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6803","nonce":224194890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6807","nonce":224194890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6805","nonce":224194890}]},"public_addr":"172.21.15.137:6801/224194890","cluster_addr":"172.21.15.137:6803/224194890","heartbeat_back_addr":"172.21.15.137:6807/224194890","heartbeat_front_addr":"172.21.15.137:6805/224194890","state":["exists","up"]},{"osd":2,"uuid":"5a69802c-dede-4544-9a12-436f5c1c0038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6801","nonce":4078459110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6803","nonce":4078459110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6807","nonce":4078459110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6805","nonce":4078459110}]},"public_addr":"172.21.15.203:6801/4078459110","cluster_addr":"172.21.15.203:6803/4078459110","heartbeat_back_addr":"172.21.15.203:6807/4078459110","heartbeat_front_addr":"172.21.15.203:6805/4078459110","state":["exists","up"]},{"osd":3,"uuid":"ad325a97-580b-4c99-bf12-cbb0d92a6d0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6811","nonce":1454440477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6813","nonce":1454440477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6817","nonce":1454440477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6815","nonce":1454440477}]},"public_addr":"172.21.15.77:6811/1454440477","cluster_addr":"172.21.15.77:6813/1454440477","heartbeat_back_addr":"172.21.15.77:6817/1454440477","heartbeat_front_addr":"172.21.15.77:6815/1454440477","state":["exists","up"]},{"osd":4,"uuid":"1e3ac1ba-98bb-4b8b-8e26-50064c66bf2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6809","nonce":1103241451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6811","nonce":1103241451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6815","nonce":1103241451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6813","nonce":1103241451}]},"public_addr":"172.21.15.137:6809/1103241451","cluster_addr":"172.21.15.137:6811/1103241451","heartbeat_back_addr":"172.21.15.137:6815/1103241451","heartbeat_front_addr":"172.21.15.137:6813/1103241451","state":["exists","up"]},{"osd":5,"uuid":"eb32c016-abeb-44ab-abf0-19dd09235736","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6809","nonce":2648569482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6811","nonce":2648569482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6815","nonce":2648569482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6813","nonce":2648569482}]},"public_addr":"172.21.15.203:6809/2648569482","cluster_addr":"172.21.15.203:6811/2648569482","heartbeat_back_addr":"172.21.15.203:6815/2648569482","heartbeat_front_addr":"172.21.15.203:6813/2648569482","state":["exists","up"]},{"osd":6,"uuid":"0e3fa97e-2ca6-418c-9e8d-a6fe3c56adb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6819","nonce":2463414651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6821","nonce":2463414651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6825","nonce":2463414651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6823","nonce":2463414651}]},"public_addr":"172.21.15.77:6819/2463414651","cluster_addr":"172.21.15.77:6821/2463414651","heartbeat_back_addr":"172.21.15.77:6825/2463414651","heartbeat_front_addr":"172.21.15.77:6823/2463414651","state":["exists","up"]},{"osd":7,"uuid":"d7398c01-0f98-4138-8a49-c3ff0aae7e88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6817","nonce":3108330766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6819","nonce":3108330766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6823","nonce":3108330766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6821","nonce":3108330766}]},"public_addr":"172.21.15.137:6817/3108330766","cluster_addr":"172.21.15.137:6819/3108330766","heartbeat_back_addr":"172.21.15.137:6823/3108330766","heartbeat_front_addr":"172.21.15.137:6821/3108330766","state":["exists","up"]},{"osd":8,"uuid":"0c09050d-3e43-410c-8cd7-bcb891af56b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6817","nonce":1161158776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6819","nonce":1161158776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6823","nonce":1161158776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6821","nonce":1161158776}]},"public_addr":"172.21.15.203:6817/1161158776","cluster_addr":"172.21.15.203:6819/1161158776","heartbeat_back_addr":"172.21.15.203:6823/1161158776","heartbeat_front_addr":"172.21.15.203:6821/1161158776","state":["exists","up"]},{"osd":9,"uuid":"527ecb49-95ec-48bd-9872-3b07fae5c5ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6827","nonce":2409880685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6829","nonce":2409880685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6833","nonce":2409880685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6831","nonce":2409880685}]},"public_addr":"172.21.15.77:6827/2409880685","cluster_addr":"172.21.15.77:6829/2409880685","heartbeat_back_addr":"172.21.15.77:6833/2409880685","heartbeat_front_addr":"172.21.15.77:6831/2409880685","state":["exists","up"]},{"osd":10,"uuid":"1680bb3e-a3c8-427b-af68-e2214f32bed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6825","nonce":374920146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6827","nonce":374920146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6831","nonce":374920146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6829","nonce":374920146}]},"public_addr":"172.21.15.137:6825/374920146","cluster_addr":"172.21.15.137:6827/374920146","heartbeat_back_addr":"172.21.15.137:6831/374920146","heartbeat_front_addr":"172.21.15.137:6829/374920146","state":["exists","up"]},{"osd":11,"uuid":"8a533466-023d-4803-ae36-9a9f1715e49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6825","nonce":2161588962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6827","nonce":2161588962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6831","nonce":2161588962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6829","nonce":2161588962}]},"public_addr":"172.21.15.203:6825/2161588962","cluster_addr":"172.21.15.203:6827/2161588962","heartbeat_back_addr":"172.21.15.203:6831/2161588962","heartbeat_front_addr":"172.21.15.203:6829/2161588962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:11:45.420509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:06.210115+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:28.191307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:49.594116+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:11.261689+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:33.376541+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:54.968336+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:16.775310+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:38.733792+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:59.456699+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:15:20.924059+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:15:43.094773+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/2260035254":"2023-12-02T01:10:24.191835+0000","172.21.15.77:6800/1456621747":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/219838828":"2023-12-02T01:10:24.191835+0000","172.21.15.77:6801/1456621747":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/2937149266":"2023-12-02T01:10:02.232948+0000","172.21.15.77:0/807520018":"2023-12-02T01:10:02.232948+0000","172.21.15.77:6801/1661788026":"2023-12-02T01:10:02.232948+0000","172.21.15.77:6800/1661788026":"2023-12-02T01:10:02.232948+0000","172.21.15.77:0/3551282473":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/1202222558":"2023-12-02T01:10:02.232948+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:17:45.582 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.580+0000 7fc06ad71700 1 -- 172.21.15.77:0/2226894236 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc0540a05f0 msgr2=0x7fc0540a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:45.582 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.580+0000 7fc06ad71700 1 --2- 172.21.15.77:0/2226894236 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc0540a05f0 0x7fc0540a2ab0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc05800b2e0 tx=0x7fc058005fb0 comp rx=0 tx=0).stop 2023-12-01T01:17:45.582 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.580+0000 7fc06ad71700 1 -- 172.21.15.77:0/2226894236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0640fff40 msgr2=0x7fc064199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:45.582 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.580+0000 7fc06ad71700 1 --2- 172.21.15.77:0/2226894236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0640fff40 0x7fc064199120 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc05000d980 tx=0x7fc05000dc90 comp rx=0 tx=0).stop 2023-12-01T01:17:45.582 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.580+0000 7fc06ad71700 1 -- 172.21.15.77:0/2226894236 shutdown_connections 2023-12-01T01:17:45.582 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.580+0000 7fc06ad71700 1 --2- 172.21.15.77:0/2226894236 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc0540a05f0 0x7fc0540a2ab0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:45.582 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.580+0000 7fc06ad71700 1 --2- 172.21.15.77:0/2226894236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc064100900 0x7fc0641a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:45.582 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.580+0000 7fc06ad71700 1 --2- 172.21.15.77:0/2226894236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0640ff5e0 0x7fc064198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:45.582 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.580+0000 7fc06ad71700 1 --2- 172.21.15.77:0/2226894236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0640fff40 0x7fc064199120 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:45.582 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.580+0000 7fc06ad71700 1 -- 172.21.15.77:0/2226894236 >> 172.21.15.77:0/2226894236 conn(0x7fc0640fb1c0 msgr2=0x7fc0640fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:45.582 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.580+0000 7fc06ad71700 1 -- 172.21.15.77:0/2226894236 shutdown_connections 2023-12-01T01:17:45.582 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.580+0000 7fc06ad71700 1 -- 172.21.15.77:0/2226894236 wait complete. 2023-12-01T01:17:45.595 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:12:30.514229+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T01:16:23.568860+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T01:16:24.531137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:17:45.595 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:17:45.723 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.719+0000 7f1733821700 1 -- 172.21.15.77:0/2299035333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f172c0ffc80 msgr2=0x7f172c1000a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:45.723 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.719+0000 7f1733821700 1 --2- 172.21.15.77:0/2299035333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f172c0ffc80 0x7f172c1000a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f17280099c0 tx=0x7f1728009cd0 comp rx=0 tx=0).stop 2023-12-01T01:17:45.723 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.720+0000 7f1733821700 1 -- 172.21.15.77:0/2299035333 shutdown_connections 2023-12-01T01:17:45.723 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.720+0000 7f1733821700 1 --2- 172.21.15.77:0/2299035333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f172c101810 0x7f172c1080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:45.724 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.720+0000 7f1733821700 1 --2- 172.21.15.77:0/2299035333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f172c0ffc80 0x7f172c1000a0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:45.724 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.720+0000 7f1733821700 1 --2- 172.21.15.77:0/2299035333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f172c100e70 0x7f172c1012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:45.724 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.720+0000 7f1733821700 1 -- 172.21.15.77:0/2299035333 >> 172.21.15.77:0/2299035333 conn(0x7f172c0fb1f0 msgr2=0x7f172c0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:45.724 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.720+0000 7f1733821700 1 -- 172.21.15.77:0/2299035333 shutdown_connections 2023-12-01T01:17:45.724 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.720+0000 7f1733821700 1 -- 172.21.15.77:0/2299035333 wait complete. 2023-12-01T01:17:45.724 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.720+0000 7f1733821700 1 Processor -- start 2023-12-01T01:17:45.724 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.720+0000 7f1733821700 1 -- start start 2023-12-01T01:17:45.724 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.720+0000 7f1733821700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f172c0ffc80 0x7f172c068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:45.724 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.720+0000 7f1733821700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f172c100e70 0x7f172c19b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:45.724 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.720+0000 7f1733821700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f172c101810 0x7f172c19de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:45.724 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.720+0000 7f1733821700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172c19e630 con 0x7f172c0ffc80 2023-12-01T01:17:45.724 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.720+0000 7f1733821700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172c19e7a0 con 0x7f172c100e70 2023-12-01T01:17:45.725 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.720+0000 7f1733821700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172c19e960 con 0x7f172c101810 2023-12-01T01:17:45.725 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.721+0000 7f17315bd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f172c0ffc80 0x7f172c068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:45.725 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.721+0000 7f17315bd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f172c0ffc80 0x7f172c068a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59022/0 (socket says 172.21.15.77:59022) 2023-12-01T01:17:45.725 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.721+0000 7f17315bd700 1 -- 172.21.15.77:0/467232245 learned_addr learned my addr 172.21.15.77:0/467232245 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:45.725 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.721+0000 7f1730dbc700 1 --2- 172.21.15.77:0/467232245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f172c100e70 0x7f172c19b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:45.725 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.721+0000 7f1731dbe700 1 --2- 172.21.15.77:0/467232245 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f172c101810 0x7f172c19de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:45.725 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.721+0000 7f17315bd700 1 -- 172.21.15.77:0/467232245 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f172c101810 msgr2=0x7f172c19de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:45.725 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.721+0000 7f17315bd700 1 --2- 172.21.15.77:0/467232245 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f172c101810 0x7f172c19de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:45.725 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.721+0000 7f17315bd700 1 -- 172.21.15.77:0/467232245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f172c100e70 msgr2=0x7f172c19b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:45.725 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.721+0000 7f17315bd700 1 --2- 172.21.15.77:0/467232245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f172c100e70 0x7f172c19b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:45.725 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.721+0000 7f17315bd700 1 -- 172.21.15.77:0/467232245 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17280095f0 con 0x7f172c0ffc80 2023-12-01T01:17:45.726 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.721+0000 7f17315bd700 1 --2- 172.21.15.77:0/467232245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f172c0ffc80 0x7f172c068a80 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f17280036f0 tx=0x7f1728009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:45.726 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.721+0000 7f17227fc700 1 -- 172.21.15.77:0/467232245 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172801d070 con 0x7f172c0ffc80 2023-12-01T01:17:45.726 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.721+0000 7f17227fc700 1 -- 172.21.15.77:0/467232245 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1728004010 con 0x7f172c0ffc80 2023-12-01T01:17:45.726 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.721+0000 7f1733821700 1 -- 172.21.15.77:0/467232245 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f172c19ebe0 con 0x7f172c0ffc80 2023-12-01T01:17:45.726 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.721+0000 7f17227fc700 1 -- 172.21.15.77:0/467232245 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172800fe70 con 0x7f172c0ffc80 2023-12-01T01:17:45.726 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.721+0000 7f1733821700 1 -- 172.21.15.77:0/467232245 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f172c19f000 con 0x7f172c0ffc80 2023-12-01T01:17:45.726 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.722+0000 7f1733821700 1 -- 172.21.15.77:0/467232245 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f172c10b550 con 0x7f172c0ffc80 2023-12-01T01:17:45.727 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.723+0000 7f17227fc700 1 -- 172.21.15.77:0/467232245 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f172800f560 con 0x7f172c0ffc80 2023-12-01T01:17:45.727 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.724+0000 7f17227fc700 1 --2- 172.21.15.77:0/467232245 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f17180a05b0 0x7f17180a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:45.727 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.724+0000 7f1730dbc700 1 --2- 172.21.15.77:0/467232245 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f17180a05b0 0x7f17180a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:45.727 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.724+0000 7f17227fc700 1 -- 172.21.15.77:0/467232245 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f17280cce90 con 0x7f172c0ffc80 2023-12-01T01:17:45.727 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.724+0000 7f1730dbc700 1 --2- 172.21.15.77:0/467232245 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f17180a05b0 0x7f17180a2a70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f172c0687f0 tx=0x7f171c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:45.728 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.727+0000 7f17227fc700 1 -- 172.21.15.77:0/467232245 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1728096960 con 0x7f172c0ffc80 2023-12-01T01:17:45.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:45 smithi077 ceph-mon[98867]: pgmap v294: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-12-01T01:17:45.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:45 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/2226894236' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:17:45.921 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.920+0000 7f1733821700 1 -- 172.21.15.77:0/467232245 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f172c106c20 con 0x7f172c0ffc80 2023-12-01T01:17:45.921 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.920+0000 7f17227fc700 1 -- 172.21.15.77:0/467232245 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1728099f80 con 0x7f172c0ffc80 2023-12-01T01:17:45.921 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 1 2023-12-01T01:17:45.923 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.922+0000 7f1733821700 1 -- 172.21.15.77:0/467232245 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f17180a05b0 msgr2=0x7f17180a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:45.924 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.922+0000 7f1733821700 1 --2- 172.21.15.77:0/467232245 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f17180a05b0 0x7f17180a2a70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f172c0687f0 tx=0x7f171c00b410 comp rx=0 tx=0).stop 2023-12-01T01:17:45.924 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.922+0000 7f1733821700 1 -- 172.21.15.77:0/467232245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f172c0ffc80 msgr2=0x7f172c068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:45.924 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.922+0000 7f1733821700 1 --2- 172.21.15.77:0/467232245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f172c0ffc80 0x7f172c068a80 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f17280036f0 tx=0x7f1728009580 comp rx=0 tx=0).stop 2023-12-01T01:17:45.924 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.922+0000 7f1733821700 1 -- 172.21.15.77:0/467232245 shutdown_connections 2023-12-01T01:17:45.924 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.922+0000 7f1733821700 1 --2- 172.21.15.77:0/467232245 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f17180a05b0 0x7f17180a2a70 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:45.924 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.922+0000 7f1733821700 1 --2- 172.21.15.77:0/467232245 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f172c101810 0x7f172c19de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:45.924 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.922+0000 7f1733821700 1 --2- 172.21.15.77:0/467232245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f172c0ffc80 0x7f172c068a80 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:45.924 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.922+0000 7f1733821700 1 --2- 172.21.15.77:0/467232245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f172c100e70 0x7f172c19b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:45.924 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.922+0000 7f1733821700 1 -- 172.21.15.77:0/467232245 >> 172.21.15.77:0/467232245 conn(0x7f172c0fb1f0 msgr2=0x7f172c0fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:45.924 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.922+0000 7f1733821700 1 -- 172.21.15.77:0/467232245 shutdown_connections 2023-12-01T01:17:45.924 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:45.922+0000 7f1733821700 1 -- 172.21.15.77:0/467232245 wait complete. 2023-12-01T01:17:45.936 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T01:17:46.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:45 smithi203 ceph-mon[103581]: pgmap v294: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-12-01T01:17:46.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:45 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/2226894236' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:17:46.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:45 smithi137 ceph-mon[103312]: pgmap v294: 97 pgs: 97 active+clean; 640 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-12-01T01:17:46.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:45 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/2226894236' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:17:46.062 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.059+0000 7f341ffc0700 1 -- 172.21.15.77:0/3653438520 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3418101880 msgr2=0x7f3418108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:46.062 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.059+0000 7f341ffc0700 1 --2- 172.21.15.77:0/3653438520 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3418101880 0x7f3418108130 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f341000b0d0 tx=0x7f341000b3e0 comp rx=0 tx=0).stop 2023-12-01T01:17:46.062 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.059+0000 7f341ffc0700 1 -- 172.21.15.77:0/3653438520 shutdown_connections 2023-12-01T01:17:46.062 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.059+0000 7f341ffc0700 1 --2- 172.21.15.77:0/3653438520 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3418101880 0x7f3418108130 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:46.062 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.059+0000 7f341ffc0700 1 --2- 172.21.15.77:0/3653438520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34180ffcf0 0x7f3418100110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:46.062 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.059+0000 7f341ffc0700 1 --2- 172.21.15.77:0/3653438520 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3418100ee0 0x7f3418101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:46.062 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.059+0000 7f341ffc0700 1 -- 172.21.15.77:0/3653438520 >> 172.21.15.77:0/3653438520 conn(0x7f34180fb260 msgr2=0x7f34180fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:46.063 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.059+0000 7f341ffc0700 1 -- 172.21.15.77:0/3653438520 shutdown_connections 2023-12-01T01:17:46.063 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.059+0000 7f341ffc0700 1 -- 172.21.15.77:0/3653438520 wait complete. 2023-12-01T01:17:46.063 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.059+0000 7f341ffc0700 1 Processor -- start 2023-12-01T01:17:46.063 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.059+0000 7f341ffc0700 1 -- start start 2023-12-01T01:17:46.063 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.059+0000 7f341ffc0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34180ffcf0 0x7f3418069c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:46.063 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.059+0000 7f341ffc0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3418100ee0 0x7f341819b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:46.063 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.059+0000 7f341ffc0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3418101880 0x7f341819def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:46.063 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.059+0000 7f341ffc0700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341819e6c0 con 0x7f34180ffcf0 2023-12-01T01:17:46.063 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.059+0000 7f341ffc0700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341819e830 con 0x7f3418100ee0 2023-12-01T01:17:46.063 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.059+0000 7f341ffc0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341819e9f0 con 0x7f3418101880 2023-12-01T01:17:46.063 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.060+0000 7f341dd5c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34180ffcf0 0x7f3418069c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:46.063 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.060+0000 7f341dd5c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34180ffcf0 0x7f3418069c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59036/0 (socket says 172.21.15.77:59036) 2023-12-01T01:17:46.063 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.060+0000 7f341dd5c700 1 -- 172.21.15.77:0/4214216122 learned_addr learned my addr 172.21.15.77:0/4214216122 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:46.064 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.060+0000 7f341d55b700 1 --2- 172.21.15.77:0/4214216122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3418100ee0 0x7f341819b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:46.064 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.060+0000 7f341e55d700 1 --2- 172.21.15.77:0/4214216122 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3418101880 0x7f341819def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:46.064 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.060+0000 7f341dd5c700 1 -- 172.21.15.77:0/4214216122 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3418101880 msgr2=0x7f341819def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:46.064 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.060+0000 7f341dd5c700 1 --2- 172.21.15.77:0/4214216122 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3418101880 0x7f341819def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:46.064 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.060+0000 7f341dd5c700 1 -- 172.21.15.77:0/4214216122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3418100ee0 msgr2=0x7f341819b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:46.064 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.060+0000 7f341dd5c700 1 --2- 172.21.15.77:0/4214216122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3418100ee0 0x7f341819b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:46.064 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.060+0000 7f341dd5c700 1 -- 172.21.15.77:0/4214216122 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3410009d00 con 0x7f34180ffcf0 2023-12-01T01:17:46.064 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.060+0000 7f341e55d700 1 --2- 172.21.15.77:0/4214216122 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3418101880 0x7f341819def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:17:46.064 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.060+0000 7f341dd5c700 1 --2- 172.21.15.77:0/4214216122 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34180ffcf0 0x7f3418069c30 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f340800d980 tx=0x7f340800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:46.064 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.060+0000 7f340effd700 1 -- 172.21.15.77:0/4214216122 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340800de60 con 0x7f34180ffcf0 2023-12-01T01:17:46.064 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.060+0000 7f340effd700 1 -- 172.21.15.77:0/4214216122 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34080044e0 con 0x7f34180ffcf0 2023-12-01T01:17:46.064 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.060+0000 7f340effd700 1 -- 172.21.15.77:0/4214216122 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3408003be0 con 0x7f34180ffcf0 2023-12-01T01:17:46.064 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.060+0000 7f341ffc0700 1 -- 172.21.15.77:0/4214216122 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f341819ecd0 con 0x7f34180ffcf0 2023-12-01T01:17:46.064 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.060+0000 7f341ffc0700 1 -- 172.21.15.77:0/4214216122 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f341819f190 con 0x7f34180ffcf0 2023-12-01T01:17:46.065 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.061+0000 7f341ffc0700 1 -- 172.21.15.77:0/4214216122 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f341810b4c0 con 0x7f34180ffcf0 2023-12-01T01:17:46.065 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.062+0000 7f340effd700 1 -- 172.21.15.77:0/4214216122 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f3408009bb0 con 0x7f34180ffcf0 2023-12-01T01:17:46.065 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.063+0000 7f340effd700 1 --2- 172.21.15.77:0/4214216122 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f34040a0630 0x7f34040a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:46.066 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.063+0000 7f341d55b700 1 --2- 172.21.15.77:0/4214216122 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f34040a0630 0x7f34040a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:46.066 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.063+0000 7f340effd700 1 -- 172.21.15.77:0/4214216122 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f34080cb850 con 0x7f34180ffcf0 2023-12-01T01:17:46.066 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.063+0000 7f341d55b700 1 --2- 172.21.15.77:0/4214216122 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f34040a0630 0x7f34040a2af0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f34180699a0 tx=0x7f3414009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:46.068 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.067+0000 7f340effd700 1 -- 172.21.15.77:0/4214216122 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3408095290 con 0x7f34180ffcf0 2023-12-01T01:17:46.261 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.260+0000 7f341ffc0700 1 -- 172.21.15.77:0/4214216122 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3418106b80 con 0x7f34180ffcf0 2023-12-01T01:17:46.261 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.260+0000 7f340effd700 1 -- 172.21.15.77:0/4214216122 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3408009940 con 0x7f34180ffcf0 2023-12-01T01:17:46.262 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 32 2023-12-01T01:17:46.263 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.262+0000 7f341ffc0700 1 -- 172.21.15.77:0/4214216122 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f34040a0630 msgr2=0x7f34040a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:46.264 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.262+0000 7f341ffc0700 1 --2- 172.21.15.77:0/4214216122 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f34040a0630 0x7f34040a2af0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f34180699a0 tx=0x7f3414009260 comp rx=0 tx=0).stop 2023-12-01T01:17:46.264 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.262+0000 7f341ffc0700 1 -- 172.21.15.77:0/4214216122 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34180ffcf0 msgr2=0x7f3418069c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:46.264 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.262+0000 7f341ffc0700 1 --2- 172.21.15.77:0/4214216122 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34180ffcf0 0x7f3418069c30 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f340800d980 tx=0x7f340800dd40 comp rx=0 tx=0).stop 2023-12-01T01:17:46.264 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.262+0000 7f341ffc0700 1 -- 172.21.15.77:0/4214216122 shutdown_connections 2023-12-01T01:17:46.264 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.262+0000 7f341ffc0700 1 --2- 172.21.15.77:0/4214216122 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f34040a0630 0x7f34040a2af0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:46.264 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.262+0000 7f341ffc0700 1 --2- 172.21.15.77:0/4214216122 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3418101880 0x7f341819def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:46.264 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.262+0000 7f341ffc0700 1 --2- 172.21.15.77:0/4214216122 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34180ffcf0 0x7f3418069c30 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:46.264 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.262+0000 7f341ffc0700 1 --2- 172.21.15.77:0/4214216122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3418100ee0 0x7f341819b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:46.264 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.262+0000 7f341ffc0700 1 -- 172.21.15.77:0/4214216122 >> 172.21.15.77:0/4214216122 conn(0x7f34180fb260 msgr2=0x7f34180fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:46.264 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.262+0000 7f341ffc0700 1 -- 172.21.15.77:0/4214216122 shutdown_connections 2023-12-01T01:17:46.264 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.262+0000 7f341ffc0700 1 -- 172.21.15.77:0/4214216122 wait complete. 2023-12-01T01:17:46.276 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T01:17:46.399 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.398+0000 7f0740b03700 1 -- 172.21.15.77:0/2190845520 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f073c102910 msgr2=0x7f073c1091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:46.400 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.398+0000 7f0740b03700 1 --2- 172.21.15.77:0/2190845520 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f073c102910 0x7f073c1091c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f073400b0d0 tx=0x7f073400b3e0 comp rx=0 tx=0).stop 2023-12-01T01:17:46.400 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.398+0000 7f0740b03700 1 -- 172.21.15.77:0/2190845520 shutdown_connections 2023-12-01T01:17:46.400 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.398+0000 7f0740b03700 1 --2- 172.21.15.77:0/2190845520 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f073c102910 0x7f073c1091c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:46.400 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.398+0000 7f0740b03700 1 --2- 172.21.15.77:0/2190845520 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f073c100d80 0x7f073c1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:46.400 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.398+0000 7f0740b03700 1 --2- 172.21.15.77:0/2190845520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f073c101f70 0x7f073c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:46.400 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.398+0000 7f0740b03700 1 -- 172.21.15.77:0/2190845520 >> 172.21.15.77:0/2190845520 conn(0x7f073c0fc310 msgr2=0x7f073c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:46.400 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.398+0000 7f0740b03700 1 -- 172.21.15.77:0/2190845520 shutdown_connections 2023-12-01T01:17:46.400 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.398+0000 7f0740b03700 1 -- 172.21.15.77:0/2190845520 wait complete. 2023-12-01T01:17:46.402 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.399+0000 7f0740b03700 1 Processor -- start 2023-12-01T01:17:46.403 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.399+0000 7f0740b03700 1 -- start start 2023-12-01T01:17:46.403 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.399+0000 7f0740b03700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f073c100d80 0x7f073c198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:46.403 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.399+0000 7f0740b03700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f073c101f70 0x7f073c1992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:46.403 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.399+0000 7f0740b03700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f073c102910 0x7f073c1a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:46.403 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.399+0000 7f0740b03700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073c1a08f0 con 0x7f073c100d80 2023-12-01T01:17:46.403 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.399+0000 7f0740b03700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073c1a0a30 con 0x7f073c101f70 2023-12-01T01:17:46.403 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.399+0000 7f0740b03700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073c1a0bc0 con 0x7f073c102910 2023-12-01T01:17:46.403 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.399+0000 7f073a59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f073c100d80 0x7f073c198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:46.403 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.399+0000 7f073a59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f073c100d80 0x7f073c198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59050/0 (socket says 172.21.15.77:59050) 2023-12-01T01:17:46.403 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.399+0000 7f073a59c700 1 -- 172.21.15.77:0/994482276 learned_addr learned my addr 172.21.15.77:0/994482276 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:46.403 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.399+0000 7f073ad9d700 1 --2- 172.21.15.77:0/994482276 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f073c102910 0x7f073c1a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:46.403 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.399+0000 7f0739d9b700 1 --2- 172.21.15.77:0/994482276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f073c101f70 0x7f073c1992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:46.403 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.399+0000 7f073a59c700 1 -- 172.21.15.77:0/994482276 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f073c102910 msgr2=0x7f073c1a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:46.403 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.399+0000 7f073a59c700 1 --2- 172.21.15.77:0/994482276 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f073c102910 0x7f073c1a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:46.404 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.399+0000 7f073a59c700 1 -- 172.21.15.77:0/994482276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f073c101f70 msgr2=0x7f073c1992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:46.404 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.399+0000 7f073a59c700 1 --2- 172.21.15.77:0/994482276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f073c101f70 0x7f073c1992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:46.404 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.399+0000 7f073a59c700 1 -- 172.21.15.77:0/994482276 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0734009d00 con 0x7f073c100d80 2023-12-01T01:17:46.404 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.400+0000 7f073a59c700 1 --2- 172.21.15.77:0/994482276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f073c100d80 0x7f073c198d70 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f073000e950 tx=0x7f073000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:46.404 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.400+0000 7f072b7fe700 1 -- 172.21.15.77:0/994482276 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0730009770 con 0x7f073c100d80 2023-12-01T01:17:46.404 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.400+0000 7f072b7fe700 1 -- 172.21.15.77:0/994482276 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f073000eea0 con 0x7f073c100d80 2023-12-01T01:17:46.404 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.400+0000 7f0740b03700 1 -- 172.21.15.77:0/994482276 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f073c1a0ea0 con 0x7f073c100d80 2023-12-01T01:17:46.404 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.400+0000 7f072b7fe700 1 -- 172.21.15.77:0/994482276 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0730003cf0 con 0x7f073c100d80 2023-12-01T01:17:46.404 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.400+0000 7f0740b03700 1 -- 172.21.15.77:0/994482276 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f073c1a1270 con 0x7f073c100d80 2023-12-01T01:17:46.404 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.401+0000 7f0740b03700 1 -- 172.21.15.77:0/994482276 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f073c1933b0 con 0x7f073c100d80 2023-12-01T01:17:46.404 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.402+0000 7f072b7fe700 1 -- 172.21.15.77:0/994482276 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f073000ccd0 con 0x7f073c100d80 2023-12-01T01:17:46.405 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.402+0000 7f072b7fe700 1 --2- 172.21.15.77:0/994482276 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f07240a05b0 0x7f07240a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:46.405 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.402+0000 7f0739d9b700 1 --2- 172.21.15.77:0/994482276 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f07240a05b0 0x7f07240a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:46.405 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.402+0000 7f072b7fe700 1 -- 172.21.15.77:0/994482276 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f0730014070 con 0x7f073c100d80 2023-12-01T01:17:46.405 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.403+0000 7f0739d9b700 1 --2- 172.21.15.77:0/994482276 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f07240a05b0 0x7f07240a2a70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f072c009af0 tx=0x7f072c0094f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:46.407 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.406+0000 7f072b7fe700 1 -- 172.21.15.77:0/994482276 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0730095880 con 0x7f073c100d80 2023-12-01T01:17:46.599 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.598+0000 7f0740b03700 1 -- 172.21.15.77:0/994482276 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f073c02cee0 con 0x7f073c100d80 2023-12-01T01:17:46.599 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.598+0000 7f072b7fe700 1 -- 172.21.15.77:0/994482276 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0730098ea0 con 0x7f073c100d80 2023-12-01T01:17:46.600 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 64 2023-12-01T01:17:46.602 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.600+0000 7f0740b03700 1 -- 172.21.15.77:0/994482276 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f07240a05b0 msgr2=0x7f07240a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:46.602 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.600+0000 7f0740b03700 1 --2- 172.21.15.77:0/994482276 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f07240a05b0 0x7f07240a2a70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f072c009af0 tx=0x7f072c0094f0 comp rx=0 tx=0).stop 2023-12-01T01:17:46.602 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.600+0000 7f0740b03700 1 -- 172.21.15.77:0/994482276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f073c100d80 msgr2=0x7f073c198d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:46.602 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.600+0000 7f0740b03700 1 --2- 172.21.15.77:0/994482276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f073c100d80 0x7f073c198d70 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f073000e950 tx=0x7f073000ec60 comp rx=0 tx=0).stop 2023-12-01T01:17:46.602 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.600+0000 7f0740b03700 1 -- 172.21.15.77:0/994482276 shutdown_connections 2023-12-01T01:17:46.602 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.600+0000 7f0740b03700 1 --2- 172.21.15.77:0/994482276 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f07240a05b0 0x7f07240a2a70 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:46.602 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.600+0000 7f0740b03700 1 --2- 172.21.15.77:0/994482276 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f073c102910 0x7f073c1a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:46.602 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.600+0000 7f0740b03700 1 --2- 172.21.15.77:0/994482276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f073c100d80 0x7f073c198d70 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:46.602 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.601+0000 7f0740b03700 1 --2- 172.21.15.77:0/994482276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f073c101f70 0x7f073c1992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:46.602 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.601+0000 7f0740b03700 1 -- 172.21.15.77:0/994482276 >> 172.21.15.77:0/994482276 conn(0x7f073c0fc310 msgr2=0x7f073c107850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:46.603 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.601+0000 7f0740b03700 1 -- 172.21.15.77:0/994482276 shutdown_connections 2023-12-01T01:17:46.603 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.601+0000 7f0740b03700 1 -- 172.21.15.77:0/994482276 wait complete. 2023-12-01T01:17:46.615 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:17:46.739 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.735+0000 7fcd62b45700 1 -- 172.21.15.77:0/3731105301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd5c100e60 msgr2=0x7fcd5c1012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:46.739 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.735+0000 7fcd62b45700 1 --2- 172.21.15.77:0/3731105301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd5c100e60 0x7fcd5c1012c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fcd480099c0 tx=0x7fcd48009cd0 comp rx=0 tx=0).stop 2023-12-01T01:17:46.739 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.735+0000 7fcd62b45700 1 -- 172.21.15.77:0/3731105301 shutdown_connections 2023-12-01T01:17:46.739 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.735+0000 7fcd62b45700 1 --2- 172.21.15.77:0/3731105301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd5c101800 0x7fcd5c1080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:46.739 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.735+0000 7fcd62b45700 1 --2- 172.21.15.77:0/3731105301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd5c0ffc70 0x7fcd5c100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:46.739 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.735+0000 7fcd62b45700 1 --2- 172.21.15.77:0/3731105301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd5c100e60 0x7fcd5c1012c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:46.739 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.735+0000 7fcd62b45700 1 -- 172.21.15.77:0/3731105301 >> 172.21.15.77:0/3731105301 conn(0x7fcd5c0fb1c0 msgr2=0x7fcd5c0fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:46.739 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.735+0000 7fcd62b45700 1 -- 172.21.15.77:0/3731105301 shutdown_connections 2023-12-01T01:17:46.739 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.735+0000 7fcd62b45700 1 -- 172.21.15.77:0/3731105301 wait complete. 2023-12-01T01:17:46.740 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.736+0000 7fcd62b45700 1 Processor -- start 2023-12-01T01:17:46.740 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.736+0000 7fcd62b45700 1 -- start start 2023-12-01T01:17:46.740 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.736+0000 7fcd62b45700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd5c0ffc70 0x7fcd5c195940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:46.740 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.736+0000 7fcd62b45700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd5c100e60 0x7fcd5c195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:46.740 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.736+0000 7fcd62b45700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd5c101800 0x7fcd5c19cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:46.740 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.736+0000 7fcd62b45700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5c19d4c0 con 0x7fcd5c100e60 2023-12-01T01:17:46.740 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.736+0000 7fcd62b45700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5c19d600 con 0x7fcd5c0ffc70 2023-12-01T01:17:46.740 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.736+0000 7fcd62b45700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5c19d790 con 0x7fcd5c101800 2023-12-01T01:17:46.740 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.736+0000 7fcd5bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd5c100e60 0x7fcd5c195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:46.740 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.736+0000 7fcd5bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd5c100e60 0x7fcd5c195e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59064/0 (socket says 172.21.15.77:59064) 2023-12-01T01:17:46.740 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.736+0000 7fcd5bfff700 1 -- 172.21.15.77:0/2230027688 learned_addr learned my addr 172.21.15.77:0/2230027688 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:46.740 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.736+0000 7fcd610e2700 1 --2- 172.21.15.77:0/2230027688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd5c101800 0x7fcd5c19cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:46.740 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.736+0000 7fcd608e1700 1 --2- 172.21.15.77:0/2230027688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd5c0ffc70 0x7fcd5c195940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:46.740 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.736+0000 7fcd5bfff700 1 -- 172.21.15.77:0/2230027688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd5c101800 msgr2=0x7fcd5c19cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:46.740 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.736+0000 7fcd5bfff700 1 --2- 172.21.15.77:0/2230027688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd5c101800 0x7fcd5c19cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:46.741 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.736+0000 7fcd5bfff700 1 -- 172.21.15.77:0/2230027688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd5c0ffc70 msgr2=0x7fcd5c195940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:46.741 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.736+0000 7fcd5bfff700 1 --2- 172.21.15.77:0/2230027688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd5c0ffc70 0x7fcd5c195940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:46.741 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.736+0000 7fcd5bfff700 1 -- 172.21.15.77:0/2230027688 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd480095f0 con 0x7fcd5c100e60 2023-12-01T01:17:46.741 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.736+0000 7fcd5bfff700 1 --2- 172.21.15.77:0/2230027688 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd5c100e60 0x7fcd5c195e80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fcd48009990 tx=0x7fcd4800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:46.741 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.737+0000 7fcd59ffb700 1 -- 172.21.15.77:0/2230027688 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4801d070 con 0x7fcd5c100e60 2023-12-01T01:17:46.741 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.737+0000 7fcd59ffb700 1 -- 172.21.15.77:0/2230027688 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd480054f0 con 0x7fcd5c100e60 2023-12-01T01:17:46.741 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.737+0000 7fcd59ffb700 1 -- 172.21.15.77:0/2230027688 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd48004bd0 con 0x7fcd5c100e60 2023-12-01T01:17:46.741 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.737+0000 7fcd62b45700 1 -- 172.21.15.77:0/2230027688 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd5c19da10 con 0x7fcd5c100e60 2023-12-01T01:17:46.741 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.737+0000 7fcd62b45700 1 -- 172.21.15.77:0/2230027688 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd5c19dde0 con 0x7fcd5c100e60 2023-12-01T01:17:46.741 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.738+0000 7fcd62b45700 1 -- 172.21.15.77:0/2230027688 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd5c106ba0 con 0x7fcd5c100e60 2023-12-01T01:17:46.742 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.739+0000 7fcd59ffb700 1 -- 172.21.15.77:0/2230027688 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fcd4800fb80 con 0x7fcd5c100e60 2023-12-01T01:17:46.742 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.739+0000 7fcd59ffb700 1 --2- 172.21.15.77:0/2230027688 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fcd4c0a0670 0x7fcd4c0a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:46.742 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.739+0000 7fcd608e1700 1 --2- 172.21.15.77:0/2230027688 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fcd4c0a0670 0x7fcd4c0a2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:46.742 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.739+0000 7fcd59ffb700 1 -- 172.21.15.77:0/2230027688 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fcd480ccc60 con 0x7fcd5c100e60 2023-12-01T01:17:46.742 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.740+0000 7fcd608e1700 1 --2- 172.21.15.77:0/2230027688 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fcd4c0a0670 0x7fcd4c0a2b30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcd50005fd0 tx=0x7fcd50005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:46.744 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.743+0000 7fcd59ffb700 1 -- 172.21.15.77:0/2230027688 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fcd48096830 con 0x7fcd5c100e60 2023-12-01T01:17:46.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:46 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/467232245' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-01T01:17:46.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:46 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/4214216122' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-01T01:17:46.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:46 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/994482276' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-01T01:17:46.937 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.936+0000 7fcd62b45700 1 -- 172.21.15.77:0/2230027688 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcd5c10b540 con 0x7fcd5c100e60 2023-12-01T01:17:46.938 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.937+0000 7fcd59ffb700 1 -- 172.21.15.77:0/2230027688 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22472 (secure 0 0 0) 0x7fcd48099e50 con 0x7fcd5c100e60 2023-12-01T01:17:46.939 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:17:46.939 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":84,"fsid":"2a06e29a-8fe6-11ee-95a2-87774f69a715","created":"2023-12-01T01:09:30.210398+0000","modified":"2023-12-01T01:17:25.609766+0000","last_up_change":"2023-12-01T01:15:44.586092+0000","last_in_change":"2023-12-01T01:15:26.305439+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:12:30.514229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:16:23.568860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-01T01:16:24.531137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fc651e62-a103-4ab8-a76d-d3708ea6aeec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6803","nonce":2244799094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6805","nonce":2244799094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6809","nonce":2244799094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6807","nonce":2244799094}]},"public_addr":"172.21.15.77:6803/2244799094","cluster_addr":"172.21.15.77:6805/2244799094","heartbeat_back_addr":"172.21.15.77:6809/2244799094","heartbeat_front_addr":"172.21.15.77:6807/2244799094","state":["exists","up"]},{"osd":1,"uuid":"0adf8c54-49ec-49a0-8bf1-af1f20e60729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6801","nonce":224194890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6803","nonce":224194890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6807","nonce":224194890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6805","nonce":224194890}]},"public_addr":"172.21.15.137:6801/224194890","cluster_addr":"172.21.15.137:6803/224194890","heartbeat_back_addr":"172.21.15.137:6807/224194890","heartbeat_front_addr":"172.21.15.137:6805/224194890","state":["exists","up"]},{"osd":2,"uuid":"5a69802c-dede-4544-9a12-436f5c1c0038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6801","nonce":4078459110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6803","nonce":4078459110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6807","nonce":4078459110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6805","nonce":4078459110}]},"public_addr":"172.21.15.203:6801/4078459110","cluster_addr":"172.21.15.203:6803/4078459110","heartbeat_back_addr":"172.21.15.203:6807/4078459110","heartbeat_front_addr":"172.21.15.203:6805/4078459110","state":["exists","up"]},{"osd":3,"uuid":"ad325a97-580b-4c99-bf12-cbb0d92a6d0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6811","nonce":1454440477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6813","nonce":1454440477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6817","nonce":1454440477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6815","nonce":1454440477}]},"public_addr":"172.21.15.77:6811/1454440477","cluster_addr":"172.21.15.77:6813/1454440477","heartbeat_back_addr":"172.21.15.77:6817/1454440477","heartbeat_front_addr":"172.21.15.77:6815/1454440477","state":["exists","up"]},{"osd":4,"uuid":"1e3ac1ba-98bb-4b8b-8e26-50064c66bf2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6809","nonce":1103241451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6811","nonce":1103241451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6815","nonce":1103241451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6813","nonce":1103241451}]},"public_addr":"172.21.15.137:6809/1103241451","cluster_addr":"172.21.15.137:6811/1103241451","heartbeat_back_addr":"172.21.15.137:6815/1103241451","heartbeat_front_addr":"172.21.15.137:6813/1103241451","state":["exists","up"]},{"osd":5,"uuid":"eb32c016-abeb-44ab-abf0-19dd09235736","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6809","nonce":2648569482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6811","nonce":2648569482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6815","nonce":2648569482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6813","nonce":2648569482}]},"public_addr":"172.21.15.203:6809/2648569482","cluster_addr":"172.21.15.203:6811/2648569482","heartbeat_back_addr":"172.21.15.203:6815/2648569482","heartbeat_front_addr":"172.21.15.203:6813/2648569482","state":["exists","up"]},{"osd":6,"uuid":"0e3fa97e-2ca6-418c-9e8d-a6fe3c56adb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6819","nonce":2463414651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6821","nonce":2463414651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6825","nonce":2463414651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6823","nonce":2463414651}]},"public_addr":"172.21.15.77:6819/2463414651","cluster_addr":"172.21.15.77:6821/2463414651","heartbeat_back_addr":"172.21.15.77:6825/2463414651","heartbeat_front_addr":"172.21.15.77:6823/2463414651","state":["exists","up"]},{"osd":7,"uuid":"d7398c01-0f98-4138-8a49-c3ff0aae7e88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6817","nonce":3108330766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6819","nonce":3108330766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6823","nonce":3108330766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6821","nonce":3108330766}]},"public_addr":"172.21.15.137:6817/3108330766","cluster_addr":"172.21.15.137:6819/3108330766","heartbeat_back_addr":"172.21.15.137:6823/3108330766","heartbeat_front_addr":"172.21.15.137:6821/3108330766","state":["exists","up"]},{"osd":8,"uuid":"0c09050d-3e43-410c-8cd7-bcb891af56b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6817","nonce":1161158776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6819","nonce":1161158776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6823","nonce":1161158776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6821","nonce":1161158776}]},"public_addr":"172.21.15.203:6817/1161158776","cluster_addr":"172.21.15.203:6819/1161158776","heartbeat_back_addr":"172.21.15.203:6823/1161158776","heartbeat_front_addr":"172.21.15.203:6821/1161158776","state":["exists","up"]},{"osd":9,"uuid":"527ecb49-95ec-48bd-9872-3b07fae5c5ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6827","nonce":2409880685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6829","nonce":2409880685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6833","nonce":2409880685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6831","nonce":2409880685}]},"public_addr":"172.21.15.77:6827/2409880685","cluster_addr":"172.21.15.77:6829/2409880685","heartbeat_back_addr":"172.21.15.77:6833/2409880685","heartbeat_front_addr":"172.21.15.77:6831/2409880685","state":["exists","up"]},{"osd":10,"uuid":"1680bb3e-a3c8-427b-af68-e2214f32bed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6825","nonce":374920146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6827","nonce":374920146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6831","nonce":374920146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6829","nonce":374920146}]},"public_addr":"172.21.15.137:6825/374920146","cluster_addr":"172.21.15.137:6827/374920146","heartbeat_back_addr":"172.21.15.137:6831/374920146","heartbeat_front_addr":"172.21.15.137:6829/374920146","state":["exists","up"]},{"osd":11,"uuid":"8a533466-023d-4803-ae36-9a9f1715e49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6825","nonce":2161588962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6827","nonce":2161588962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6831","nonce":2161588962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6829","nonce":2161588962}]},"public_addr":"172.21.15.203:6825/2161588962","cluster_addr":"172.21.15.203:6827/2161588962","heartbeat_back_addr":"172.21.15.203:6831/2161588962","heartbeat_front_addr":"172.21.15.203:6829/2161588962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:11:45.420509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:06.210115+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:28.191307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:49.594116+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:11.261689+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:33.376541+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:54.968336+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:16.775310+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:38.733792+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:59.456699+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:15:20.924059+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:15:43.094773+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/2260035254":"2023-12-02T01:10:24.191835+0000","172.21.15.77:6800/1456621747":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/219838828":"2023-12-02T01:10:24.191835+0000","172.21.15.77:6801/1456621747":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/2937149266":"2023-12-02T01:10:02.232948+0000","172.21.15.77:0/807520018":"2023-12-02T01:10:02.232948+0000","172.21.15.77:6801/1661788026":"2023-12-02T01:10:02.232948+0000","172.21.15.77:6800/1661788026":"2023-12-02T01:10:02.232948+0000","172.21.15.77:0/3551282473":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/1202222558":"2023-12-02T01:10:02.232948+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:17:46.941 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.940+0000 7fcd62b45700 1 -- 172.21.15.77:0/2230027688 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fcd4c0a0670 msgr2=0x7fcd4c0a2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:46.941 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.940+0000 7fcd62b45700 1 --2- 172.21.15.77:0/2230027688 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fcd4c0a0670 0x7fcd4c0a2b30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcd50005fd0 tx=0x7fcd50005dd0 comp rx=0 tx=0).stop 2023-12-01T01:17:46.941 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.940+0000 7fcd62b45700 1 -- 172.21.15.77:0/2230027688 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd5c100e60 msgr2=0x7fcd5c195e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:46.941 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.940+0000 7fcd62b45700 1 --2- 172.21.15.77:0/2230027688 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd5c100e60 0x7fcd5c195e80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fcd48009990 tx=0x7fcd4800f690 comp rx=0 tx=0).stop 2023-12-01T01:17:46.941 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.940+0000 7fcd62b45700 1 -- 172.21.15.77:0/2230027688 shutdown_connections 2023-12-01T01:17:46.941 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.940+0000 7fcd62b45700 1 --2- 172.21.15.77:0/2230027688 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fcd4c0a0670 0x7fcd4c0a2b30 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:46.941 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.940+0000 7fcd62b45700 1 --2- 172.21.15.77:0/2230027688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd5c101800 0x7fcd5c19cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:46.941 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.940+0000 7fcd62b45700 1 --2- 172.21.15.77:0/2230027688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd5c0ffc70 0x7fcd5c195940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:46.942 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.940+0000 7fcd62b45700 1 --2- 172.21.15.77:0/2230027688 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd5c100e60 0x7fcd5c195e80 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:46.942 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.940+0000 7fcd62b45700 1 -- 172.21.15.77:0/2230027688 >> 172.21.15.77:0/2230027688 conn(0x7fcd5c0fb1c0 msgr2=0x7fcd5c0fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:46.942 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.940+0000 7fcd62b45700 1 -- 172.21.15.77:0/2230027688 shutdown_connections 2023-12-01T01:17:46.942 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:46.940+0000 7fcd62b45700 1 -- 172.21.15.77:0/2230027688 wait complete. 2023-12-01T01:17:46.953 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:12:30.514229+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T01:16:23.568860+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T01:16:24.531137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:17:46.954 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:17:47.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:46 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/467232245' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-01T01:17:47.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:46 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/4214216122' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-01T01:17:47.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:46 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/994482276' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-01T01:17:47.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:46 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/467232245' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-01T01:17:47.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:46 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/4214216122' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-01T01:17:47.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:46 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/994482276' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-01T01:17:47.079 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.077+0000 7f4f8e61d700 1 -- 172.21.15.77:0/3275778634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f880ffc80 msgr2=0x7f4f881000a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:47.079 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.077+0000 7f4f8e61d700 1 --2- 172.21.15.77:0/3275778634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f880ffc80 0x7f4f881000a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4f780099c0 tx=0x7f4f78009cd0 comp rx=0 tx=0).stop 2023-12-01T01:17:47.079 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.077+0000 7f4f8e61d700 1 -- 172.21.15.77:0/3275778634 shutdown_connections 2023-12-01T01:17:47.079 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.077+0000 7f4f8e61d700 1 --2- 172.21.15.77:0/3275778634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f88101810 0x7f4f881080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:47.079 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.077+0000 7f4f8e61d700 1 --2- 172.21.15.77:0/3275778634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f880ffc80 0x7f4f881000a0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:47.079 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.077+0000 7f4f8e61d700 1 --2- 172.21.15.77:0/3275778634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f88100e70 0x7f4f881012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:47.079 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.077+0000 7f4f8e61d700 1 -- 172.21.15.77:0/3275778634 >> 172.21.15.77:0/3275778634 conn(0x7f4f880fb1f0 msgr2=0x7f4f880fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:47.079 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.077+0000 7f4f8e61d700 1 -- 172.21.15.77:0/3275778634 shutdown_connections 2023-12-01T01:17:47.079 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.077+0000 7f4f8e61d700 1 -- 172.21.15.77:0/3275778634 wait complete. 2023-12-01T01:17:47.082 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.078+0000 7f4f8e61d700 1 Processor -- start 2023-12-01T01:17:47.082 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.078+0000 7f4f8e61d700 1 -- start start 2023-12-01T01:17:47.082 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.078+0000 7f4f8e61d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f880ffc80 0x7f4f88195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:47.082 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.078+0000 7f4f8e61d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f88100e70 0x7f4f88195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:47.082 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.078+0000 7f4f8e61d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f88101810 0x7f4f8819ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:47.082 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.078+0000 7f4f8e61d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8819d480 con 0x7f4f880ffc80 2023-12-01T01:17:47.082 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.078+0000 7f4f8e61d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8819d5c0 con 0x7f4f88100e70 2023-12-01T01:17:47.082 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.078+0000 7f4f8e61d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8819d750 con 0x7f4f88101810 2023-12-01T01:17:47.082 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.078+0000 7f4f87fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f880ffc80 0x7f4f88195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:47.083 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.078+0000 7f4f87fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f880ffc80 0x7f4f88195900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59076/0 (socket says 172.21.15.77:59076) 2023-12-01T01:17:47.083 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.079+0000 7f4f87fff700 1 -- 172.21.15.77:0/200769697 learned_addr learned my addr 172.21.15.77:0/200769697 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:47.083 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.079+0000 7f4f8cbba700 1 --2- 172.21.15.77:0/200769697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f88101810 0x7f4f8819ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:47.083 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.079+0000 7f4f877fe700 1 --2- 172.21.15.77:0/200769697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f88100e70 0x7f4f88195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:47.083 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.079+0000 7f4f87fff700 1 -- 172.21.15.77:0/200769697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f88101810 msgr2=0x7f4f8819ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:47.083 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.079+0000 7f4f87fff700 1 --2- 172.21.15.77:0/200769697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f88101810 0x7f4f8819ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:47.083 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.079+0000 7f4f87fff700 1 -- 172.21.15.77:0/200769697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f88100e70 msgr2=0x7f4f88195e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:47.083 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.079+0000 7f4f87fff700 1 --2- 172.21.15.77:0/200769697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f88100e70 0x7f4f88195e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:47.083 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.079+0000 7f4f87fff700 1 -- 172.21.15.77:0/200769697 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f780095f0 con 0x7f4f880ffc80 2023-12-01T01:17:47.083 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.079+0000 7f4f877fe700 1 --2- 172.21.15.77:0/200769697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f88100e70 0x7f4f88195e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:17:47.083 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.079+0000 7f4f8cbba700 1 --2- 172.21.15.77:0/200769697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f88101810 0x7f4f8819ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:17:47.083 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.079+0000 7f4f87fff700 1 --2- 172.21.15.77:0/200769697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f880ffc80 0x7f4f88195900 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4f78009f50 tx=0x7f4f780057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:47.083 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.079+0000 7f4f857fa700 1 -- 172.21.15.77:0/200769697 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7801d070 con 0x7f4f880ffc80 2023-12-01T01:17:47.083 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.079+0000 7f4f8e61d700 1 -- 172.21.15.77:0/200769697 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f8819d9d0 con 0x7f4f880ffc80 2023-12-01T01:17:47.083 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.079+0000 7f4f857fa700 1 -- 172.21.15.77:0/200769697 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f78004110 con 0x7f4f880ffc80 2023-12-01T01:17:47.084 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.079+0000 7f4f857fa700 1 -- 172.21.15.77:0/200769697 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f78022dd0 con 0x7f4f880ffc80 2023-12-01T01:17:47.084 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.079+0000 7f4f8e61d700 1 -- 172.21.15.77:0/200769697 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f8819ded0 con 0x7f4f880ffc80 2023-12-01T01:17:47.084 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.080+0000 7f4f8e61d700 1 -- 172.21.15.77:0/200769697 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f88106bb0 con 0x7f4f880ffc80 2023-12-01T01:17:47.084 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.082+0000 7f4f857fa700 1 -- 172.21.15.77:0/200769697 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f4f78022500 con 0x7f4f880ffc80 2023-12-01T01:17:47.085 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.082+0000 7f4f857fa700 1 --2- 172.21.15.77:0/200769697 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f4f700a05b0 0x7f4f700a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:47.085 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.082+0000 7f4f857fa700 1 -- 172.21.15.77:0/200769697 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f4f780cd5c0 con 0x7f4f880ffc80 2023-12-01T01:17:47.085 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.082+0000 7f4f877fe700 1 --2- 172.21.15.77:0/200769697 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f4f700a05b0 0x7f4f700a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:47.085 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.083+0000 7f4f877fe700 1 --2- 172.21.15.77:0/200769697 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f4f700a05b0 0x7f4f700a2a70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c005fd0 tx=0x7f4f7c005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:47.087 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.086+0000 7f4f857fa700 1 -- 172.21.15.77:0/200769697 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4f78097190 con 0x7f4f880ffc80 2023-12-01T01:17:47.279 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.278+0000 7f4f8e61d700 1 -- 172.21.15.77:0/200769697 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4f8802cc70 con 0x7f4f880ffc80 2023-12-01T01:17:47.279 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.278+0000 7f4f857fa700 1 -- 172.21.15.77:0/200769697 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4f7809a7b0 con 0x7f4f880ffc80 2023-12-01T01:17:47.279 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 1 2023-12-01T01:17:47.282 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.281+0000 7f4f8e61d700 1 -- 172.21.15.77:0/200769697 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f4f700a05b0 msgr2=0x7f4f700a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:47.282 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.281+0000 7f4f8e61d700 1 --2- 172.21.15.77:0/200769697 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f4f700a05b0 0x7f4f700a2a70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c005fd0 tx=0x7f4f7c005ee0 comp rx=0 tx=0).stop 2023-12-01T01:17:47.282 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.281+0000 7f4f8e61d700 1 -- 172.21.15.77:0/200769697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f880ffc80 msgr2=0x7f4f88195900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:47.282 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.281+0000 7f4f8e61d700 1 --2- 172.21.15.77:0/200769697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f880ffc80 0x7f4f88195900 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4f78009f50 tx=0x7f4f780057d0 comp rx=0 tx=0).stop 2023-12-01T01:17:47.282 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.281+0000 7f4f8e61d700 1 -- 172.21.15.77:0/200769697 shutdown_connections 2023-12-01T01:17:47.282 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.281+0000 7f4f8e61d700 1 --2- 172.21.15.77:0/200769697 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f4f700a05b0 0x7f4f700a2a70 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:47.282 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.281+0000 7f4f8e61d700 1 --2- 172.21.15.77:0/200769697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f88101810 0x7f4f8819ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:47.282 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.281+0000 7f4f8e61d700 1 --2- 172.21.15.77:0/200769697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f880ffc80 0x7f4f88195900 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:47.283 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.281+0000 7f4f8e61d700 1 --2- 172.21.15.77:0/200769697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f88100e70 0x7f4f88195e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:47.283 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.281+0000 7f4f8e61d700 1 -- 172.21.15.77:0/200769697 >> 172.21.15.77:0/200769697 conn(0x7f4f880fb1f0 msgr2=0x7f4f880fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:47.283 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.281+0000 7f4f8e61d700 1 -- 172.21.15.77:0/200769697 shutdown_connections 2023-12-01T01:17:47.283 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.281+0000 7f4f8e61d700 1 -- 172.21.15.77:0/200769697 wait complete. 2023-12-01T01:17:47.294 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T01:17:47.421 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.418+0000 7fb4ea46a700 1 -- 172.21.15.77:0/2772821904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4e40ffcb0 msgr2=0x7fb4e41000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:47.421 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.418+0000 7fb4ea46a700 1 --2- 172.21.15.77:0/2772821904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4e40ffcb0 0x7fb4e41000d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb4d80099c0 tx=0x7fb4d8009cd0 comp rx=0 tx=0).stop 2023-12-01T01:17:47.421 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.418+0000 7fb4ea46a700 1 -- 172.21.15.77:0/2772821904 shutdown_connections 2023-12-01T01:17:47.421 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.418+0000 7fb4ea46a700 1 --2- 172.21.15.77:0/2772821904 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4e4101840 0x7fb4e41080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:47.421 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.418+0000 7fb4ea46a700 1 --2- 172.21.15.77:0/2772821904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4e40ffcb0 0x7fb4e41000d0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:47.421 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.418+0000 7fb4ea46a700 1 --2- 172.21.15.77:0/2772821904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4e4100ea0 0x7fb4e4101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:47.421 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.418+0000 7fb4ea46a700 1 -- 172.21.15.77:0/2772821904 >> 172.21.15.77:0/2772821904 conn(0x7fb4e40fb220 msgr2=0x7fb4e40fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:47.422 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.418+0000 7fb4ea46a700 1 -- 172.21.15.77:0/2772821904 shutdown_connections 2023-12-01T01:17:47.422 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.418+0000 7fb4ea46a700 1 -- 172.21.15.77:0/2772821904 wait complete. 2023-12-01T01:17:47.422 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.418+0000 7fb4ea46a700 1 Processor -- start 2023-12-01T01:17:47.422 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.418+0000 7fb4ea46a700 1 -- start start 2023-12-01T01:17:47.422 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.419+0000 7fb4ea46a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4e40ffcb0 0x7fb4e41959b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:47.422 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.419+0000 7fb4ea46a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4e4100ea0 0x7fb4e4195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:47.422 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.419+0000 7fb4ea46a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4e4101840 0x7fb4e419cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:47.422 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.419+0000 7fb4ea46a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e419d530 con 0x7fb4e40ffcb0 2023-12-01T01:17:47.422 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.419+0000 7fb4ea46a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e419d670 con 0x7fb4e4101840 2023-12-01T01:17:47.422 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.419+0000 7fb4ea46a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e419d800 con 0x7fb4e4100ea0 2023-12-01T01:17:47.422 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.419+0000 7fb4e3fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4e40ffcb0 0x7fb4e41959b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:47.422 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.419+0000 7fb4e3fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4e40ffcb0 0x7fb4e41959b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59090/0 (socket says 172.21.15.77:59090) 2023-12-01T01:17:47.422 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.419+0000 7fb4e3fff700 1 -- 172.21.15.77:0/4068514129 learned_addr learned my addr 172.21.15.77:0/4068514129 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:47.423 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.419+0000 7fb4e37fe700 1 --2- 172.21.15.77:0/4068514129 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4e4100ea0 0x7fb4e4195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:47.423 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.419+0000 7fb4e8a07700 1 --2- 172.21.15.77:0/4068514129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4e4101840 0x7fb4e419cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:47.423 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.419+0000 7fb4e3fff700 1 -- 172.21.15.77:0/4068514129 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4e4100ea0 msgr2=0x7fb4e4195ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:47.423 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.419+0000 7fb4e3fff700 1 --2- 172.21.15.77:0/4068514129 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4e4100ea0 0x7fb4e4195ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:47.423 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.419+0000 7fb4e3fff700 1 -- 172.21.15.77:0/4068514129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4e4101840 msgr2=0x7fb4e419cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:47.423 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.419+0000 7fb4e3fff700 1 --2- 172.21.15.77:0/4068514129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4e4101840 0x7fb4e419cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:47.423 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.419+0000 7fb4e3fff700 1 -- 172.21.15.77:0/4068514129 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d80095f0 con 0x7fb4e40ffcb0 2023-12-01T01:17:47.423 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.419+0000 7fb4e8a07700 1 --2- 172.21.15.77:0/4068514129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4e4101840 0x7fb4e419cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:17:47.423 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.419+0000 7fb4e3fff700 1 --2- 172.21.15.77:0/4068514129 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4e40ffcb0 0x7fb4e41959b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8009fd0 tx=0x7fb4d800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:47.423 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.419+0000 7fb4e17fa700 1 -- 172.21.15.77:0/4068514129 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d801d070 con 0x7fb4e40ffcb0 2023-12-01T01:17:47.423 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.420+0000 7fb4ea46a700 1 -- 172.21.15.77:0/4068514129 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4e419da80 con 0x7fb4e40ffcb0 2023-12-01T01:17:47.423 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.420+0000 7fb4e17fa700 1 -- 172.21.15.77:0/4068514129 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4d80054f0 con 0x7fb4e40ffcb0 2023-12-01T01:17:47.423 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.420+0000 7fb4e17fa700 1 -- 172.21.15.77:0/4068514129 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8004c40 con 0x7fb4e40ffcb0 2023-12-01T01:17:47.424 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.420+0000 7fb4ea46a700 1 -- 172.21.15.77:0/4068514129 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4e419dea0 con 0x7fb4e40ffcb0 2023-12-01T01:17:47.424 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.420+0000 7fb4ea46a700 1 -- 172.21.15.77:0/4068514129 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4e410b4c0 con 0x7fb4e40ffcb0 2023-12-01T01:17:47.424 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.422+0000 7fb4e17fa700 1 -- 172.21.15.77:0/4068514129 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fb4d800fc30 con 0x7fb4e40ffcb0 2023-12-01T01:17:47.425 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.422+0000 7fb4e17fa700 1 --2- 172.21.15.77:0/4068514129 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fb4cc0a05b0 0x7fb4cc0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:47.425 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.423+0000 7fb4e37fe700 1 --2- 172.21.15.77:0/4068514129 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fb4cc0a05b0 0x7fb4cc0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:47.425 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.423+0000 7fb4e17fa700 1 -- 172.21.15.77:0/4068514129 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fb4d80ccd60 con 0x7fb4e40ffcb0 2023-12-01T01:17:47.425 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.423+0000 7fb4e37fe700 1 --2- 172.21.15.77:0/4068514129 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fb4cc0a05b0 0x7fb4cc0a2a70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4009b30 tx=0x7fb4d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:47.427 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.426+0000 7fb4e17fa700 1 -- 172.21.15.77:0/4068514129 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb4d8096930 con 0x7fb4e40ffcb0 2023-12-01T01:17:47.619 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.617+0000 7fb4ea46a700 1 -- 172.21.15.77:0/4068514129 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb4e4106c00 con 0x7fb4e40ffcb0 2023-12-01T01:17:47.619 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.618+0000 7fb4e17fa700 1 -- 172.21.15.77:0/4068514129 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb4d8099f50 con 0x7fb4e40ffcb0 2023-12-01T01:17:47.619 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 32 2023-12-01T01:17:47.621 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.620+0000 7fb4ea46a700 1 -- 172.21.15.77:0/4068514129 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fb4cc0a05b0 msgr2=0x7fb4cc0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:47.621 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.620+0000 7fb4ea46a700 1 --2- 172.21.15.77:0/4068514129 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fb4cc0a05b0 0x7fb4cc0a2a70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4009b30 tx=0x7fb4d4009260 comp rx=0 tx=0).stop 2023-12-01T01:17:47.622 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.620+0000 7fb4ea46a700 1 -- 172.21.15.77:0/4068514129 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4e40ffcb0 msgr2=0x7fb4e41959b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:47.622 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.620+0000 7fb4ea46a700 1 --2- 172.21.15.77:0/4068514129 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4e40ffcb0 0x7fb4e41959b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8009fd0 tx=0x7fb4d800f740 comp rx=0 tx=0).stop 2023-12-01T01:17:47.622 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.620+0000 7fb4ea46a700 1 -- 172.21.15.77:0/4068514129 shutdown_connections 2023-12-01T01:17:47.622 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.620+0000 7fb4ea46a700 1 --2- 172.21.15.77:0/4068514129 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fb4cc0a05b0 0x7fb4cc0a2a70 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:47.622 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.620+0000 7fb4ea46a700 1 --2- 172.21.15.77:0/4068514129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4e4101840 0x7fb4e419cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:47.622 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.620+0000 7fb4ea46a700 1 --2- 172.21.15.77:0/4068514129 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4e40ffcb0 0x7fb4e41959b0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:47.622 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.620+0000 7fb4ea46a700 1 --2- 172.21.15.77:0/4068514129 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4e4100ea0 0x7fb4e4195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:47.622 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.620+0000 7fb4ea46a700 1 -- 172.21.15.77:0/4068514129 >> 172.21.15.77:0/4068514129 conn(0x7fb4e40fb220 msgr2=0x7fb4e40fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:47.622 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.620+0000 7fb4ea46a700 1 -- 172.21.15.77:0/4068514129 shutdown_connections 2023-12-01T01:17:47.622 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.620+0000 7fb4ea46a700 1 -- 172.21.15.77:0/4068514129 wait complete. 2023-12-01T01:17:47.634 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T01:17:47.760 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.756+0000 7f00e843d700 1 -- 172.21.15.77:0/158352031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00e0101810 msgr2=0x7f00e01080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:47.760 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.756+0000 7f00e843d700 1 --2- 172.21.15.77:0/158352031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00e0101810 0x7f00e01080c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f00d800b0d0 tx=0x7f00d800b3e0 comp rx=0 tx=0).stop 2023-12-01T01:17:47.761 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.757+0000 7f00e843d700 1 -- 172.21.15.77:0/158352031 shutdown_connections 2023-12-01T01:17:47.761 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.757+0000 7f00e843d700 1 --2- 172.21.15.77:0/158352031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00e0101810 0x7f00e01080c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:47.761 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.757+0000 7f00e843d700 1 --2- 172.21.15.77:0/158352031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00e00ffc80 0x7f00e01000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:47.761 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.757+0000 7f00e843d700 1 --2- 172.21.15.77:0/158352031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00e0100e70 0x7f00e01012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:47.761 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.757+0000 7f00e843d700 1 -- 172.21.15.77:0/158352031 >> 172.21.15.77:0/158352031 conn(0x7f00e00fb1f0 msgr2=0x7f00e00fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:47.761 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.757+0000 7f00e843d700 1 -- 172.21.15.77:0/158352031 shutdown_connections 2023-12-01T01:17:47.761 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.757+0000 7f00e843d700 1 -- 172.21.15.77:0/158352031 wait complete. 2023-12-01T01:17:47.761 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.757+0000 7f00e843d700 1 Processor -- start 2023-12-01T01:17:47.761 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.757+0000 7f00e843d700 1 -- start start 2023-12-01T01:17:47.761 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.757+0000 7f00e843d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00e00ffc80 0x7f00e0195a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:47.761 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.757+0000 7f00e843d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00e0100e70 0x7f00e0195fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:47.762 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.758+0000 7f00e843d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00e0101810 0x7f00e019d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:47.762 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.758+0000 7f00e843d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e019d5f0 con 0x7f00e0100e70 2023-12-01T01:17:47.762 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.758+0000 7f00e843d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e019d730 con 0x7f00e0101810 2023-12-01T01:17:47.762 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.758+0000 7f00e843d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e019d8c0 con 0x7f00e00ffc80 2023-12-01T01:17:47.762 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.758+0000 7f00e59d8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00e0100e70 0x7f00e0195fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:47.762 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.758+0000 7f00e59d8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00e0100e70 0x7f00e0195fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59104/0 (socket says 172.21.15.77:59104) 2023-12-01T01:17:47.762 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.758+0000 7f00e59d8700 1 -- 172.21.15.77:0/1820936573 learned_addr learned my addr 172.21.15.77:0/1820936573 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:47.762 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.758+0000 7f00e69da700 1 --2- 172.21.15.77:0/1820936573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00e0101810 0x7f00e019d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:47.762 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.758+0000 7f00e61d9700 1 --2- 172.21.15.77:0/1820936573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00e00ffc80 0x7f00e0195a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:47.762 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.758+0000 7f00e59d8700 1 -- 172.21.15.77:0/1820936573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00e00ffc80 msgr2=0x7f00e0195a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:47.762 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.758+0000 7f00e59d8700 1 --2- 172.21.15.77:0/1820936573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00e00ffc80 0x7f00e0195a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:47.762 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.758+0000 7f00e59d8700 1 -- 172.21.15.77:0/1820936573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00e0101810 msgr2=0x7f00e019d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:47.763 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.758+0000 7f00e59d8700 1 --2- 172.21.15.77:0/1820936573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00e0101810 0x7f00e019d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:47.763 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.758+0000 7f00e59d8700 1 -- 172.21.15.77:0/1820936573 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d8009d00 con 0x7f00e0100e70 2023-12-01T01:17:47.763 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.758+0000 7f00e59d8700 1 --2- 172.21.15.77:0/1820936573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00e0100e70 0x7f00e0195fb0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f00d000d950 tx=0x7f00d000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:47.763 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.758+0000 7f00d77fe700 1 -- 172.21.15.77:0/1820936573 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d000f890 con 0x7f00e0100e70 2023-12-01T01:17:47.763 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.758+0000 7f00d77fe700 1 -- 172.21.15.77:0/1820936573 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00d0010460 con 0x7f00e0100e70 2023-12-01T01:17:47.763 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.758+0000 7f00e843d700 1 -- 172.21.15.77:0/1820936573 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00e019dba0 con 0x7f00e0100e70 2023-12-01T01:17:47.763 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.758+0000 7f00d77fe700 1 -- 172.21.15.77:0/1820936573 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d000e610 con 0x7f00e0100e70 2023-12-01T01:17:47.763 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.758+0000 7f00e843d700 1 -- 172.21.15.77:0/1820936573 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00e019e0b0 con 0x7f00e0100e70 2023-12-01T01:17:47.763 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.759+0000 7f00e843d700 1 -- 172.21.15.77:0/1820936573 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00e010b4a0 con 0x7f00e0100e70 2023-12-01T01:17:47.764 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.760+0000 7f00d77fe700 1 -- 172.21.15.77:0/1820936573 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f00d00108d0 con 0x7f00e0100e70 2023-12-01T01:17:47.764 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.761+0000 7f00d77fe700 1 --2- 172.21.15.77:0/1820936573 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f00cc0a05b0 0x7f00cc0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:47.764 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.761+0000 7f00d77fe700 1 -- 172.21.15.77:0/1820936573 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f00d00cb7e0 con 0x7f00e0100e70 2023-12-01T01:17:47.764 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.761+0000 7f00e61d9700 1 --2- 172.21.15.77:0/1820936573 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f00cc0a05b0 0x7f00cc0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:47.764 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.761+0000 7f00e61d9700 1 --2- 172.21.15.77:0/1820936573 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f00cc0a05b0 0x7f00cc0a2a70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f00e0100cd0 tx=0x7f00dc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:47.766 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.765+0000 7f00d77fe700 1 -- 172.21.15.77:0/1820936573 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f00d0095320 con 0x7f00e0100e70 2023-12-01T01:17:47.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:47 smithi077 ceph-mon[98867]: pgmap v295: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-12-01T01:17:47.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:47 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/2230027688' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:17:47.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:47 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/200769697' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-01T01:17:47.956 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.955+0000 7f00e843d700 1 -- 172.21.15.77:0/1820936573 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f00e0106b30 con 0x7f00e0100e70 2023-12-01T01:17:47.957 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.956+0000 7f00d77fe700 1 -- 172.21.15.77:0/1820936573 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7f00d0020020 con 0x7f00e0100e70 2023-12-01T01:17:47.957 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 64 2023-12-01T01:17:47.960 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.958+0000 7f00e843d700 1 -- 172.21.15.77:0/1820936573 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f00cc0a05b0 msgr2=0x7f00cc0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:47.960 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.958+0000 7f00e843d700 1 --2- 172.21.15.77:0/1820936573 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f00cc0a05b0 0x7f00cc0a2a70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f00e0100cd0 tx=0x7f00dc00b410 comp rx=0 tx=0).stop 2023-12-01T01:17:47.960 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.958+0000 7f00e843d700 1 -- 172.21.15.77:0/1820936573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00e0100e70 msgr2=0x7f00e0195fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:47.960 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.958+0000 7f00e843d700 1 --2- 172.21.15.77:0/1820936573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00e0100e70 0x7f00e0195fb0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f00d000d950 tx=0x7f00d000dc60 comp rx=0 tx=0).stop 2023-12-01T01:17:47.960 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.958+0000 7f00e843d700 1 -- 172.21.15.77:0/1820936573 shutdown_connections 2023-12-01T01:17:47.960 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.958+0000 7f00e843d700 1 --2- 172.21.15.77:0/1820936573 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f00cc0a05b0 0x7f00cc0a2a70 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:47.960 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.958+0000 7f00e843d700 1 --2- 172.21.15.77:0/1820936573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00e0101810 0x7f00e019d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:47.960 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.958+0000 7f00e843d700 1 --2- 172.21.15.77:0/1820936573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00e00ffc80 0x7f00e0195a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:47.960 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.958+0000 7f00e843d700 1 --2- 172.21.15.77:0/1820936573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00e0100e70 0x7f00e0195fb0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:47.960 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.958+0000 7f00e843d700 1 -- 172.21.15.77:0/1820936573 >> 172.21.15.77:0/1820936573 conn(0x7f00e00fb1f0 msgr2=0x7f00e00fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:47.960 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.958+0000 7f00e843d700 1 -- 172.21.15.77:0/1820936573 shutdown_connections 2023-12-01T01:17:47.961 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:47.958+0000 7f00e843d700 1 -- 172.21.15.77:0/1820936573 wait complete. 2023-12-01T01:17:47.972 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-12-01T01:17:47.972 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:17:48.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:47 smithi203 ceph-mon[103581]: pgmap v295: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-12-01T01:17:48.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:47 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/2230027688' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:17:48.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:47 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/200769697' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-01T01:17:48.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:47 smithi137 ceph-mon[103312]: pgmap v295: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-12-01T01:17:48.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:47 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/2230027688' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:17:48.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:47 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/200769697' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-01T01:17:48.094 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.091+0000 7f7351f5e700 1 -- 172.21.15.77:0/1704854970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f734c100d90 msgr2=0x7f734c1011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:48.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.091+0000 7f7351f5e700 1 --2- 172.21.15.77:0/1704854970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f734c100d90 0x7f734c1011b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f733c0099c0 tx=0x7f733c009cd0 comp rx=0 tx=0).stop 2023-12-01T01:17:48.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.091+0000 7f7351f5e700 1 -- 172.21.15.77:0/1704854970 shutdown_connections 2023-12-01T01:17:48.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.091+0000 7f7351f5e700 1 --2- 172.21.15.77:0/1704854970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f734c102920 0x7f734c1091d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:48.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.091+0000 7f7351f5e700 1 --2- 172.21.15.77:0/1704854970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f734c100d90 0x7f734c1011b0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:48.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.091+0000 7f7351f5e700 1 --2- 172.21.15.77:0/1704854970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f734c101f80 0x7f734c1023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:48.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.091+0000 7f7351f5e700 1 -- 172.21.15.77:0/1704854970 >> 172.21.15.77:0/1704854970 conn(0x7f734c0fc2e0 msgr2=0x7f734c0fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:48.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.091+0000 7f7351f5e700 1 -- 172.21.15.77:0/1704854970 shutdown_connections 2023-12-01T01:17:48.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.091+0000 7f7351f5e700 1 -- 172.21.15.77:0/1704854970 wait complete. 2023-12-01T01:17:48.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.092+0000 7f7351f5e700 1 Processor -- start 2023-12-01T01:17:48.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.092+0000 7f7351f5e700 1 -- start start 2023-12-01T01:17:48.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.092+0000 7f7351f5e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f734c100d90 0x7f734c196a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:48.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.092+0000 7f7351f5e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f734c101f80 0x7f734c196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:48.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.092+0000 7f7351f5e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f734c102920 0x7f734c19e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:48.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.092+0000 7f7351f5e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734c19e600 con 0x7f734c100d90 2023-12-01T01:17:48.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.092+0000 7f7351f5e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734c19e740 con 0x7f734c102920 2023-12-01T01:17:48.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.092+0000 7f7351f5e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734c19e8d0 con 0x7f734c101f80 2023-12-01T01:17:48.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.092+0000 7f734b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f734c100d90 0x7f734c196a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:48.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.092+0000 7f734b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f734c100d90 0x7f734c196a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59116/0 (socket says 172.21.15.77:59116) 2023-12-01T01:17:48.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.092+0000 7f734b7fe700 1 -- 172.21.15.77:0/3512934800 learned_addr learned my addr 172.21.15.77:0/3512934800 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:48.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.092+0000 7f7343fff700 1 --2- 172.21.15.77:0/3512934800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f734c101f80 0x7f734c196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:48.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.092+0000 7f734b7fe700 1 -- 172.21.15.77:0/3512934800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f734c101f80 msgr2=0x7f734c196fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:48.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.092+0000 7f734b7fe700 1 --2- 172.21.15.77:0/3512934800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f734c101f80 0x7f734c196fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:48.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.092+0000 7f734b7fe700 1 -- 172.21.15.77:0/3512934800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f734c102920 msgr2=0x7f734c19e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:48.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.092+0000 7f734b7fe700 1 --2- 172.21.15.77:0/3512934800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f734c102920 0x7f734c19e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:48.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.092+0000 7f734b7fe700 1 -- 172.21.15.77:0/3512934800 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f733c0095f0 con 0x7f734c100d90 2023-12-01T01:17:48.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.093+0000 7f734b7fe700 1 --2- 172.21.15.77:0/3512934800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f734c100d90 0x7f734c196a80 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f733c0036f0 tx=0x7f733c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:48.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.093+0000 7f73497fa700 1 -- 172.21.15.77:0/3512934800 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733c01d070 con 0x7f734c100d90 2023-12-01T01:17:48.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.093+0000 7f73497fa700 1 -- 172.21.15.77:0/3512934800 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f733c004110 con 0x7f734c100d90 2023-12-01T01:17:48.097 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.093+0000 7f7351f5e700 1 -- 172.21.15.77:0/3512934800 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f734c19eb50 con 0x7f734c100d90 2023-12-01T01:17:48.097 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.093+0000 7f73497fa700 1 -- 172.21.15.77:0/3512934800 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733c021410 con 0x7f734c100d90 2023-12-01T01:17:48.097 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.093+0000 7f7351f5e700 1 -- 172.21.15.77:0/3512934800 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f734c19efd0 con 0x7f734c100d90 2023-12-01T01:17:48.098 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.094+0000 7f7351f5e700 1 -- 172.21.15.77:0/3512934800 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f734c10c5f0 con 0x7f734c100d90 2023-12-01T01:17:48.098 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.095+0000 7f73497fa700 1 -- 172.21.15.77:0/3512934800 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f733c00f660 con 0x7f734c100d90 2023-12-01T01:17:48.098 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.095+0000 7f73497fa700 1 --2- 172.21.15.77:0/3512934800 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f73380a0670 0x7f73380a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:48.098 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.096+0000 7f7343fff700 1 --2- 172.21.15.77:0/3512934800 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f73380a0670 0x7f73380a2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:48.098 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.096+0000 7f73497fa700 1 -- 172.21.15.77:0/3512934800 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f733c0ccf00 con 0x7f734c100d90 2023-12-01T01:17:48.098 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.096+0000 7f7343fff700 1 --2- 172.21.15.77:0/3512934800 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f73380a0670 0x7f73380a2b30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7334005fd0 tx=0x7f7334005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:48.100 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.099+0000 7f73497fa700 1 -- 172.21.15.77:0/3512934800 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f733c0969d0 con 0x7f734c100d90 2023-12-01T01:17:48.334 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.333+0000 7f7351f5e700 1 -- 172.21.15.77:0/3512934800 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f734c107cb0 con 0x7f734c100d90 2023-12-01T01:17:48.334 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.333+0000 7f73497fa700 1 -- 172.21.15.77:0/3512934800 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10838 (secure 0 0 0) 0x7f733c099ff0 con 0x7f734c100d90 2023-12-01T01:17:48.334 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:17:48.335 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1435145620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1435145620},{"type":"v1","addr":"172.21.15.77:6835","nonce":1435145620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/841838908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":841838908},{"type":"v1","addr":"172.21.15.77:6837","nonce":841838908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-01T01:16:25.345276+0000","modified":"2023-12-01T01:16:32.596617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24496,"mds_2":24475,"mds_3":24473,"mds_4":14649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/3726913446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3726913446},{"type":"v1","addr":"172.21.15.77:6839","nonce":3726913446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14649":{"gid":14649,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":5,"addr":"172.21.15.77:6841/3023184965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3023184965},{"type":"v1","addr":"172.21.15.77:6841","nonce":3023184965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/2249638445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2249638445},{"type":"v1","addr":"172.21.15.137:6833","nonce":2249638445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6833/1201599774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1201599774},{"type":"v1","addr":"172.21.15.203:6833","nonce":1201599774}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3935354878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3935354878},{"type":"v1","addr":"172.21.15.137:6835","nonce":3935354878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3237538176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3237538176},{"type":"v1","addr":"172.21.15.203:6835","nonce":3237538176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3518881867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3518881867},{"type":"v1","addr":"172.21.15.137:6837","nonce":3518881867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.203:6837/2887080977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2887080977},{"type":"v1","addr":"172.21.15.203:6837","nonce":2887080977}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.203:6839/2888221613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2888221613},{"type":"v1","addr":"172.21.15.203:6839","nonce":2888221613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/1099966660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1099966660},{"type":"v1","addr":"172.21.15.137:6839","nonce":1099966660}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-01T01:17:48.337 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.335+0000 7f7351f5e700 1 -- 172.21.15.77:0/3512934800 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f73380a0670 msgr2=0x7f73380a2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:48.337 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.335+0000 7f7351f5e700 1 --2- 172.21.15.77:0/3512934800 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f73380a0670 0x7f73380a2b30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7334005fd0 tx=0x7f7334005f60 comp rx=0 tx=0).stop 2023-12-01T01:17:48.337 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.335+0000 7f7351f5e700 1 -- 172.21.15.77:0/3512934800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f734c100d90 msgr2=0x7f734c196a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:48.337 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.335+0000 7f7351f5e700 1 --2- 172.21.15.77:0/3512934800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f734c100d90 0x7f734c196a80 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f733c0036f0 tx=0x7f733c009580 comp rx=0 tx=0).stop 2023-12-01T01:17:48.337 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.335+0000 7f7351f5e700 1 -- 172.21.15.77:0/3512934800 shutdown_connections 2023-12-01T01:17:48.337 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.335+0000 7f7351f5e700 1 --2- 172.21.15.77:0/3512934800 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f73380a0670 0x7f73380a2b30 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:48.337 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.335+0000 7f7351f5e700 1 --2- 172.21.15.77:0/3512934800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f734c102920 0x7f734c19e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:48.337 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.335+0000 7f7351f5e700 1 --2- 172.21.15.77:0/3512934800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f734c100d90 0x7f734c196a80 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:48.337 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.335+0000 7f7351f5e700 1 --2- 172.21.15.77:0/3512934800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f734c101f80 0x7f734c196fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:48.337 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.335+0000 7f7351f5e700 1 -- 172.21.15.77:0/3512934800 >> 172.21.15.77:0/3512934800 conn(0x7f734c0fc2e0 msgr2=0x7f734c0fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:48.337 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.336+0000 7f7351f5e700 1 -- 172.21.15.77:0/3512934800 shutdown_connections 2023-12-01T01:17:48.337 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.336+0000 7f7351f5e700 1 -- 172.21.15.77:0/3512934800 wait complete. 2023-12-01T01:17:48.338 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 27 2023-12-01T01:17:48.349 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-12-01T01:17:48.349 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-12-01T01:17:48.349 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T01:17:48.473 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.469+0000 7fcf00437700 1 -- 172.21.15.77:0/2811930340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcef8100e90 msgr2=0x7fcef81012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:48.473 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.469+0000 7fcf00437700 1 --2- 172.21.15.77:0/2811930340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcef8100e90 0x7fcef81012f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fcee80099c0 tx=0x7fcee8009cd0 comp rx=0 tx=0).stop 2023-12-01T01:17:48.473 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.470+0000 7fcf00437700 1 -- 172.21.15.77:0/2811930340 shutdown_connections 2023-12-01T01:17:48.473 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.470+0000 7fcf00437700 1 --2- 172.21.15.77:0/2811930340 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcef8101830 0x7fcef81080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:48.473 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.470+0000 7fcf00437700 1 --2- 172.21.15.77:0/2811930340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcef80ffca0 0x7fcef81000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:48.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.470+0000 7fcf00437700 1 --2- 172.21.15.77:0/2811930340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcef8100e90 0x7fcef81012f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:48.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.470+0000 7fcf00437700 1 -- 172.21.15.77:0/2811930340 >> 172.21.15.77:0/2811930340 conn(0x7fcef80fb1f0 msgr2=0x7fcef80fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:48.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.470+0000 7fcf00437700 1 -- 172.21.15.77:0/2811930340 shutdown_connections 2023-12-01T01:17:48.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.470+0000 7fcf00437700 1 -- 172.21.15.77:0/2811930340 wait complete. 2023-12-01T01:17:48.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.470+0000 7fcf00437700 1 Processor -- start 2023-12-01T01:17:48.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.470+0000 7fcf00437700 1 -- start start 2023-12-01T01:17:48.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.470+0000 7fcf00437700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcef80ffca0 0x7fcef8198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:48.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.470+0000 7fcf00437700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcef8100e90 0x7fcef8199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:48.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.470+0000 7fcf00437700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcef8101830 0x7fcef81a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:48.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.470+0000 7fcf00437700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef81a0760 con 0x7fcef8101830 2023-12-01T01:17:48.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.470+0000 7fcf00437700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef81a08d0 con 0x7fcef80ffca0 2023-12-01T01:17:48.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.470+0000 7fcf00437700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef81a0a90 con 0x7fcef8100e90 2023-12-01T01:17:48.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.470+0000 7fcefe9d4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcef8101830 0x7fcef81a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:48.475 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.470+0000 7fcefe9d4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcef8101830 0x7fcef81a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59134/0 (socket says 172.21.15.77:59134) 2023-12-01T01:17:48.475 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.470+0000 7fcefe9d4700 1 -- 172.21.15.77:0/979261516 learned_addr learned my addr 172.21.15.77:0/979261516 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:48.475 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.471+0000 7fcefe1d3700 1 --2- 172.21.15.77:0/979261516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcef80ffca0 0x7fcef8198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:48.475 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.471+0000 7fcefd9d2700 1 --2- 172.21.15.77:0/979261516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcef8100e90 0x7fcef8199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:48.475 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.471+0000 7fcefe9d4700 1 -- 172.21.15.77:0/979261516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcef8100e90 msgr2=0x7fcef8199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:48.475 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.471+0000 7fcefe9d4700 1 --2- 172.21.15.77:0/979261516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcef8100e90 0x7fcef8199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:48.475 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.471+0000 7fcefe9d4700 1 -- 172.21.15.77:0/979261516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcef80ffca0 msgr2=0x7fcef8198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:48.475 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.471+0000 7fcefe9d4700 1 --2- 172.21.15.77:0/979261516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcef80ffca0 0x7fcef8198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:48.475 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.471+0000 7fcefe9d4700 1 -- 172.21.15.77:0/979261516 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee80095f0 con 0x7fcef8101830 2023-12-01T01:17:48.475 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.471+0000 7fcefe9d4700 1 --2- 172.21.15.77:0/979261516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcef8101830 0x7fcef81a0190 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fcef0007ee0 tx=0x7fcef000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:48.475 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.471+0000 7fceef7fe700 1 -- 172.21.15.77:0/979261516 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef000dd20 con 0x7fcef8101830 2023-12-01T01:17:48.475 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.471+0000 7fceef7fe700 1 -- 172.21.15.77:0/979261516 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcef000f040 con 0x7fcef8101830 2023-12-01T01:17:48.475 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.471+0000 7fcf00437700 1 -- 172.21.15.77:0/979261516 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcef81a0d70 con 0x7fcef8101830 2023-12-01T01:17:48.476 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.471+0000 7fceef7fe700 1 -- 172.21.15.77:0/979261516 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef00048a0 con 0x7fcef8101830 2023-12-01T01:17:48.476 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.471+0000 7fcf00437700 1 -- 172.21.15.77:0/979261516 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef81a11d0 con 0x7fcef8101830 2023-12-01T01:17:48.476 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.472+0000 7fcf00437700 1 -- 172.21.15.77:0/979261516 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcef810b4b0 con 0x7fcef8101830 2023-12-01T01:17:48.476 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.473+0000 7fceef7fe700 1 -- 172.21.15.77:0/979261516 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fcef0003a00 con 0x7fcef8101830 2023-12-01T01:17:48.477 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.474+0000 7fceef7fe700 1 --2- 172.21.15.77:0/979261516 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fcee40a0670 0x7fcee40a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:48.477 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.474+0000 7fcefe1d3700 1 --2- 172.21.15.77:0/979261516 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fcee40a0670 0x7fcee40a2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:48.477 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.474+0000 7fceef7fe700 1 -- 172.21.15.77:0/979261516 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fcef00cbb20 con 0x7fcef8101830 2023-12-01T01:17:48.477 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.474+0000 7fcefe1d3700 1 --2- 172.21.15.77:0/979261516 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fcee40a0670 0x7fcee40a2b30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcef819a0f0 tx=0x7fcef400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:48.478 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.477+0000 7fceef7fe700 1 -- 172.21.15.77:0/979261516 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fcef0095560 con 0x7fcef8101830 2023-12-01T01:17:48.672 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.670+0000 7fcf00437700 1 -- 172.21.15.77:0/979261516 --> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcef8069940 con 0x7fcee40a0670 2023-12-01T01:17:48.681 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.680+0000 7fceef7fe700 1 -- 172.21.15.77:0/979261516 <== mgr.14152 v2:172.21.15.77:6800/3838941679 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344189 (secure 0 0 0) 0x7fcef8069940 con 0x7fcee40a0670 2023-12-01T01:17:48.686 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:17:48.688 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.684+0000 7fcf00437700 1 -- 172.21.15.77:0/979261516 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fcee40a0670 msgr2=0x7fcee40a2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:48.689 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.684+0000 7fcf00437700 1 --2- 172.21.15.77:0/979261516 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fcee40a0670 0x7fcee40a2b30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcef819a0f0 tx=0x7fcef400b410 comp rx=0 tx=0).stop 2023-12-01T01:17:48.689 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.684+0000 7fcf00437700 1 -- 172.21.15.77:0/979261516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcef8101830 msgr2=0x7fcef81a0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:48.689 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.684+0000 7fcf00437700 1 --2- 172.21.15.77:0/979261516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcef8101830 0x7fcef81a0190 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fcef0007ee0 tx=0x7fcef000d3b0 comp rx=0 tx=0).stop 2023-12-01T01:17:48.689 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.684+0000 7fcf00437700 1 -- 172.21.15.77:0/979261516 shutdown_connections 2023-12-01T01:17:48.689 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.684+0000 7fcf00437700 1 --2- 172.21.15.77:0/979261516 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fcee40a0670 0x7fcee40a2b30 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:48.689 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.684+0000 7fcf00437700 1 --2- 172.21.15.77:0/979261516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcef8101830 0x7fcef81a0190 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:48.689 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.684+0000 7fcf00437700 1 --2- 172.21.15.77:0/979261516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcef80ffca0 0x7fcef8198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:48.689 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.684+0000 7fcf00437700 1 --2- 172.21.15.77:0/979261516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcef8100e90 0x7fcef8199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:48.689 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.684+0000 7fcf00437700 1 -- 172.21.15.77:0/979261516 >> 172.21.15.77:0/979261516 conn(0x7fcef80fb1f0 msgr2=0x7fcef8069230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:48.689 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.684+0000 7fcf00437700 1 -- 172.21.15.77:0/979261516 shutdown_connections 2023-12-01T01:17:48.689 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.684+0000 7fcf00437700 1 -- 172.21.15.77:0/979261516 wait complete. 2023-12-01T01:17:48.689 INFO:teuthology.orchestra.run.smithi077.stderr:dumped all 2023-12-01T01:17:48.698 INFO:teuthology.orchestra.run.smithi077.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2023-12-01T01:17:48.286446+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656403,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":907,"num_read_kb":577,"num_write":416,"num_write_kb":3070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1180916,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336340,"kb_used_data":12916,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124605484,"statfs":{"total":1151940427776,"available":1151596015616,"internally_reserved":0,"allocated":13225984,"data_stored":7244401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":636,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":131,"num_read_kb":76,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003165"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.612614+0000","last_change":"2023-12-01T01:16:25.373817+0000","last_active":"2023-12-01T01:17:24.612614+0000","last_peered":"2023-12-01T01:17:24.612614+0000","last_clean":"2023-12-01T01:17:24.612614+0000","last_became_active":"2023-12-01T01:16:25.373154+0000","last_became_peered":"2023-12-01T01:16:25.373154+0000","last_unstale":"2023-12-01T01:17:24.612614+0000","last_undegraded":"2023-12-01T01:17:24.612614+0000","last_fullsized":"2023-12-01T01:17:24.612614+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:33:43.284106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.615673+0000","last_change":"2023-12-01T01:16:25.364855+0000","last_active":"2023-12-01T01:17:24.615673+0000","last_peered":"2023-12-01T01:17:24.615673+0000","last_clean":"2023-12-01T01:17:24.615673+0000","last_became_active":"2023-12-01T01:16:25.364536+0000","last_became_peered":"2023-12-01T01:16:25.364536+0000","last_unstale":"2023-12-01T01:17:24.615673+0000","last_undegraded":"2023-12-01T01:17:24.615673+0000","last_fullsized":"2023-12-01T01:17:24.615673+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:45:23.471557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.615287+0000","last_change":"2023-12-01T01:16:25.372443+0000","last_active":"2023-12-01T01:17:24.615287+0000","last_peered":"2023-12-01T01:17:24.615287+0000","last_clean":"2023-12-01T01:17:24.615287+0000","last_became_active":"2023-12-01T01:16:25.372133+0000","last_became_peered":"2023-12-01T01:16:25.372133+0000","last_unstale":"2023-12-01T01:17:24.615287+0000","last_undegraded":"2023-12-01T01:17:24.615287+0000","last_fullsized":"2023-12-01T01:17:24.615287+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:54:19.439408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-01T01:17:25.621327+0000","last_change":"2023-12-01T01:17:25.619573+0000","last_active":"2023-12-01T01:17:25.621327+0000","last_peered":"2023-12-01T01:17:25.621327+0000","last_clean":"2023-12-01T01:17:25.621327+0000","last_became_active":"2023-12-01T01:17:25.619265+0000","last_became_peered":"2023-12-01T01:17:25.619265+0000","last_unstale":"2023-12-01T01:17:25.621327+0000","last_undegraded":"2023-12-01T01:17:25.621327+0000","last_fullsized":"2023-12-01T01:17:25.621327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:03:13.174659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.615064+0000","last_change":"2023-12-01T01:16:25.364081+0000","last_active":"2023-12-01T01:17:24.615064+0000","last_peered":"2023-12-01T01:17:24.615064+0000","last_clean":"2023-12-01T01:17:24.615064+0000","last_became_active":"2023-12-01T01:16:25.363187+0000","last_became_peered":"2023-12-01T01:16:25.363187+0000","last_unstale":"2023-12-01T01:17:24.615064+0000","last_undegraded":"2023-12-01T01:17:24.615064+0000","last_fullsized":"2023-12-01T01:17:24.615064+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:05:42.852128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.619241+0000","last_change":"2023-12-01T01:16:25.366104+0000","last_active":"2023-12-01T01:17:24.619241+0000","last_peered":"2023-12-01T01:17:24.619241+0000","last_clean":"2023-12-01T01:17:24.619241+0000","last_became_active":"2023-12-01T01:16:25.365778+0000","last_became_peered":"2023-12-01T01:16:25.365778+0000","last_unstale":"2023-12-01T01:17:24.619241+0000","last_undegraded":"2023-12-01T01:17:24.619241+0000","last_fullsized":"2023-12-01T01:17:24.619241+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:30:09.485609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.613507+0000","last_change":"2023-12-01T01:16:25.371354+0000","last_active":"2023-12-01T01:17:24.613507+0000","last_peered":"2023-12-01T01:17:24.613507+0000","last_clean":"2023-12-01T01:17:24.613507+0000","last_became_active":"2023-12-01T01:16:25.370955+0000","last_became_peered":"2023-12-01T01:16:25.370955+0000","last_unstale":"2023-12-01T01:17:24.613507+0000","last_undegraded":"2023-12-01T01:17:24.613507+0000","last_fullsized":"2023-12-01T01:17:24.613507+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:38:53.508285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617357+0000","last_change":"2023-12-01T01:16:25.368187+0000","last_active":"2023-12-01T01:17:24.617357+0000","last_peered":"2023-12-01T01:17:24.617357+0000","last_clean":"2023-12-01T01:17:24.617357+0000","last_became_active":"2023-12-01T01:16:25.367854+0000","last_became_peered":"2023-12-01T01:16:25.367854+0000","last_unstale":"2023-12-01T01:17:24.617357+0000","last_undegraded":"2023-12-01T01:17:24.617357+0000","last_fullsized":"2023-12-01T01:17:24.617357+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:22:35.159185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.802923+0000","last_change":"2023-12-01T01:16:25.372829+0000","last_active":"2023-12-01T01:17:24.802923+0000","last_peered":"2023-12-01T01:17:24.802923+0000","last_clean":"2023-12-01T01:17:24.802923+0000","last_became_active":"2023-12-01T01:16:25.372469+0000","last_became_peered":"2023-12-01T01:16:25.372469+0000","last_unstale":"2023-12-01T01:17:24.802923+0000","last_undegraded":"2023-12-01T01:17:24.802923+0000","last_fullsized":"2023-12-01T01:17:24.802923+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:25:10.300138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-01T01:17:25.618881+0000","last_change":"2023-12-01T01:17:25.618881+0000","last_active":"2023-12-01T01:17:25.618881+0000","last_peered":"2023-12-01T01:17:25.618881+0000","last_clean":"2023-12-01T01:17:25.618881+0000","last_became_active":"2023-12-01T01:17:25.618457+0000","last_became_peered":"2023-12-01T01:17:25.618457+0000","last_unstale":"2023-12-01T01:17:25.618881+0000","last_undegraded":"2023-12-01T01:17:25.618881+0000","last_fullsized":"2023-12-01T01:17:25.618881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:44:13.172660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.613960+0000","last_change":"2023-12-01T01:16:25.367602+0000","last_active":"2023-12-01T01:17:24.613960+0000","last_peered":"2023-12-01T01:17:24.613960+0000","last_clean":"2023-12-01T01:17:24.613960+0000","last_became_active":"2023-12-01T01:16:25.366565+0000","last_became_peered":"2023-12-01T01:16:25.366565+0000","last_unstale":"2023-12-01T01:17:24.613960+0000","last_undegraded":"2023-12-01T01:17:24.613960+0000","last_fullsized":"2023-12-01T01:17:24.613960+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:28:52.576838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.616474+0000","last_change":"2023-12-01T01:16:25.375004+0000","last_active":"2023-12-01T01:17:24.616474+0000","last_peered":"2023-12-01T01:17:24.616474+0000","last_clean":"2023-12-01T01:17:24.616474+0000","last_became_active":"2023-12-01T01:16:25.374320+0000","last_became_peered":"2023-12-01T01:16:25.374320+0000","last_unstale":"2023-12-01T01:17:24.616474+0000","last_undegraded":"2023-12-01T01:17:24.616474+0000","last_fullsized":"2023-12-01T01:17:24.616474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:40:49.482061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617481+0000","last_change":"2023-12-01T01:16:25.367858+0000","last_active":"2023-12-01T01:17:24.617481+0000","last_peered":"2023-12-01T01:17:24.617481+0000","last_clean":"2023-12-01T01:17:24.617481+0000","last_became_active":"2023-12-01T01:16:25.367560+0000","last_became_peered":"2023-12-01T01:16:25.367560+0000","last_unstale":"2023-12-01T01:17:24.617481+0000","last_undegraded":"2023-12-01T01:17:24.617481+0000","last_fullsized":"2023-12-01T01:17:24.617481+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:33:21.048672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.615018+0000","last_change":"2023-12-01T01:16:25.375654+0000","last_active":"2023-12-01T01:17:24.615018+0000","last_peered":"2023-12-01T01:17:24.615018+0000","last_clean":"2023-12-01T01:17:24.615018+0000","last_became_active":"2023-12-01T01:16:25.375379+0000","last_became_peered":"2023-12-01T01:16:25.375379+0000","last_unstale":"2023-12-01T01:17:24.615018+0000","last_undegraded":"2023-12-01T01:17:24.615018+0000","last_fullsized":"2023-12-01T01:17:24.615018+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:33:42.335365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.614888+0000","last_change":"2023-12-01T01:16:25.372546+0000","last_active":"2023-12-01T01:17:24.614888+0000","last_peered":"2023-12-01T01:17:24.614888+0000","last_clean":"2023-12-01T01:17:24.614888+0000","last_became_active":"2023-12-01T01:16:25.372327+0000","last_became_peered":"2023-12-01T01:16:25.372327+0000","last_unstale":"2023-12-01T01:17:24.614888+0000","last_undegraded":"2023-12-01T01:17:24.614888+0000","last_fullsized":"2023-12-01T01:17:24.614888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:12:34.171834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.802497+0000","last_change":"2023-12-01T01:16:25.366449+0000","last_active":"2023-12-01T01:17:24.802497+0000","last_peered":"2023-12-01T01:17:24.802497+0000","last_clean":"2023-12-01T01:17:24.802497+0000","last_became_active":"2023-12-01T01:16:25.366147+0000","last_became_peered":"2023-12-01T01:16:25.366147+0000","last_unstale":"2023-12-01T01:17:24.802497+0000","last_undegraded":"2023-12-01T01:17:24.802497+0000","last_fullsized":"2023-12-01T01:17:24.802497+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:01:29.670329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.612762+0000","last_change":"2023-12-01T01:16:25.374016+0000","last_active":"2023-12-01T01:17:24.612762+0000","last_peered":"2023-12-01T01:17:24.612762+0000","last_clean":"2023-12-01T01:17:24.612762+0000","last_became_active":"2023-12-01T01:16:25.373543+0000","last_became_peered":"2023-12-01T01:16:25.373543+0000","last_unstale":"2023-12-01T01:17:24.612762+0000","last_undegraded":"2023-12-01T01:17:24.612762+0000","last_fullsized":"2023-12-01T01:17:24.612762+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:36:22.858446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.612411+0000","last_change":"2023-12-01T01:16:25.363598+0000","last_active":"2023-12-01T01:17:24.612411+0000","last_peered":"2023-12-01T01:17:24.612411+0000","last_clean":"2023-12-01T01:17:24.612411+0000","last_became_active":"2023-12-01T01:16:25.363111+0000","last_became_peered":"2023-12-01T01:16:25.363111+0000","last_unstale":"2023-12-01T01:17:24.612411+0000","last_undegraded":"2023-12-01T01:17:24.612411+0000","last_fullsized":"2023-12-01T01:17:24.612411+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:42:08.171789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.619102+0000","last_change":"2023-12-01T01:16:25.368446+0000","last_active":"2023-12-01T01:17:24.619102+0000","last_peered":"2023-12-01T01:17:24.619102+0000","last_clean":"2023-12-01T01:17:24.619102+0000","last_became_active":"2023-12-01T01:16:25.361893+0000","last_became_peered":"2023-12-01T01:16:25.361893+0000","last_unstale":"2023-12-01T01:17:24.619102+0000","last_undegraded":"2023-12-01T01:17:24.619102+0000","last_fullsized":"2023-12-01T01:17:24.619102+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:05:45.697338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'13","reported_seq":102,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-01T01:17:44.717304+0000","last_change":"2023-12-01T01:16:24.934939+0000","last_active":"2023-12-01T01:17:44.717304+0000","last_peered":"2023-12-01T01:17:44.717304+0000","last_clean":"2023-12-01T01:17:44.717304+0000","last_became_active":"2023-12-01T01:16:24.934481+0000","last_became_peered":"2023-12-01T01:16:24.934481+0000","last_unstale":"2023-12-01T01:17:44.717304+0000","last_undegraded":"2023-12-01T01:17:44.717304+0000","last_fullsized":"2023-12-01T01:17:44.717304+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:39:49.804567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":75,"num_read_kb":75,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617562+0000","last_change":"2023-12-01T01:16:25.364061+0000","last_active":"2023-12-01T01:17:24.617562+0000","last_peered":"2023-12-01T01:17:24.617562+0000","last_clean":"2023-12-01T01:17:24.617562+0000","last_became_active":"2023-12-01T01:16:25.363678+0000","last_became_peered":"2023-12-01T01:16:25.363678+0000","last_unstale":"2023-12-01T01:17:24.617562+0000","last_undegraded":"2023-12-01T01:17:24.617562+0000","last_fullsized":"2023-12-01T01:17:24.617562+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:24:07.107604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617794+0000","last_change":"2023-12-01T01:16:24.939338+0000","last_active":"2023-12-01T01:17:24.617794+0000","last_peered":"2023-12-01T01:17:24.617794+0000","last_clean":"2023-12-01T01:17:24.617794+0000","last_became_active":"2023-12-01T01:16:24.939007+0000","last_became_peered":"2023-12-01T01:16:24.939007+0000","last_unstale":"2023-12-01T01:17:24.617794+0000","last_undegraded":"2023-12-01T01:17:24.617794+0000","last_fullsized":"2023-12-01T01:17:24.617794+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:56:50.732188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.615519+0000","last_change":"2023-12-01T01:16:25.362231+0000","last_active":"2023-12-01T01:17:24.615519+0000","last_peered":"2023-12-01T01:17:24.615519+0000","last_clean":"2023-12-01T01:17:24.615519+0000","last_became_active":"2023-12-01T01:16:25.361920+0000","last_became_peered":"2023-12-01T01:16:25.361920+0000","last_unstale":"2023-12-01T01:17:24.615519+0000","last_undegraded":"2023-12-01T01:17:24.615519+0000","last_fullsized":"2023-12-01T01:17:24.615519+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:22:22.104480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'12","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.616167+0000","last_change":"2023-12-01T01:16:24.931801+0000","last_active":"2023-12-01T01:17:24.616167+0000","last_peered":"2023-12-01T01:17:24.616167+0000","last_clean":"2023-12-01T01:17:24.616167+0000","last_became_active":"2023-12-01T01:16:24.931451+0000","last_became_peered":"2023-12-01T01:16:24.931451+0000","last_unstale":"2023-12-01T01:17:24.616167+0000","last_undegraded":"2023-12-01T01:17:24.616167+0000","last_fullsized":"2023-12-01T01:17:24.616167+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:22:41.666502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.802631+0000","last_change":"2023-12-01T01:16:25.361789+0000","last_active":"2023-12-01T01:17:24.802631+0000","last_peered":"2023-12-01T01:17:24.802631+0000","last_clean":"2023-12-01T01:17:24.802631+0000","last_became_active":"2023-12-01T01:16:25.361480+0000","last_became_peered":"2023-12-01T01:16:25.361480+0000","last_unstale":"2023-12-01T01:17:24.802631+0000","last_undegraded":"2023-12-01T01:17:24.802631+0000","last_fullsized":"2023-12-01T01:17:24.802631+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:07:00.742167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'5","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.615819+0000","last_change":"2023-12-01T01:16:24.933362+0000","last_active":"2023-12-01T01:17:24.615819+0000","last_peered":"2023-12-01T01:17:24.615819+0000","last_clean":"2023-12-01T01:17:24.615819+0000","last_became_active":"2023-12-01T01:16:24.933049+0000","last_became_peered":"2023-12-01T01:16:24.933049+0000","last_unstale":"2023-12-01T01:17:24.615819+0000","last_undegraded":"2023-12-01T01:17:24.615819+0000","last_fullsized":"2023-12-01T01:17:24.615819+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:47:37.153344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617087+0000","last_change":"2023-12-01T01:16:25.364419+0000","last_active":"2023-12-01T01:17:24.617087+0000","last_peered":"2023-12-01T01:17:24.617087+0000","last_clean":"2023-12-01T01:17:24.617087+0000","last_became_active":"2023-12-01T01:16:25.363022+0000","last_became_peered":"2023-12-01T01:16:25.363022+0000","last_unstale":"2023-12-01T01:17:24.617087+0000","last_undegraded":"2023-12-01T01:17:24.617087+0000","last_fullsized":"2023-12-01T01:17:24.617087+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:58:45.660625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'6","reported_seq":95,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-01T01:17:42.703276+0000","last_change":"2023-12-01T01:16:24.935947+0000","last_active":"2023-12-01T01:17:42.703276+0000","last_peered":"2023-12-01T01:17:42.703276+0000","last_clean":"2023-12-01T01:17:42.703276+0000","last_became_active":"2023-12-01T01:16:24.935061+0000","last_became_peered":"2023-12-01T01:16:24.935061+0000","last_unstale":"2023-12-01T01:17:42.703276+0000","last_undegraded":"2023-12-01T01:17:42.703276+0000","last_fullsized":"2023-12-01T01:17:42.703276+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:00:31.551362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":967,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.615303+0000","last_change":"2023-12-01T01:16:25.363229+0000","last_active":"2023-12-01T01:17:24.615303+0000","last_peered":"2023-12-01T01:17:24.615303+0000","last_clean":"2023-12-01T01:17:24.615303+0000","last_became_active":"2023-12-01T01:16:25.362770+0000","last_became_peered":"2023-12-01T01:16:25.362770+0000","last_unstale":"2023-12-01T01:17:24.615303+0000","last_undegraded":"2023-12-01T01:17:24.615303+0000","last_fullsized":"2023-12-01T01:17:24.615303+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:45:02.782201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.619079+0000","last_change":"2023-12-01T01:16:24.930509+0000","last_active":"2023-12-01T01:17:24.619079+0000","last_peered":"2023-12-01T01:17:24.619079+0000","last_clean":"2023-12-01T01:17:24.619079+0000","last_became_active":"2023-12-01T01:16:24.930188+0000","last_became_peered":"2023-12-01T01:16:24.930188+0000","last_unstale":"2023-12-01T01:17:24.619079+0000","last_undegraded":"2023-12-01T01:17:24.619079+0000","last_fullsized":"2023-12-01T01:17:24.619079+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:04:17.748363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617184+0000","last_change":"2023-12-01T01:16:25.363383+0000","last_active":"2023-12-01T01:17:24.617184+0000","last_peered":"2023-12-01T01:17:24.617184+0000","last_clean":"2023-12-01T01:17:24.617184+0000","last_became_active":"2023-12-01T01:16:25.363085+0000","last_became_peered":"2023-12-01T01:16:25.363085+0000","last_unstale":"2023-12-01T01:17:24.617184+0000","last_undegraded":"2023-12-01T01:17:24.617184+0000","last_fullsized":"2023-12-01T01:17:24.617184+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:34:36.921706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"82'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617436+0000","last_change":"2023-12-01T01:16:24.929081+0000","last_active":"2023-12-01T01:17:24.617436+0000","last_peered":"2023-12-01T01:17:24.617436+0000","last_clean":"2023-12-01T01:17:24.617436+0000","last_became_active":"2023-12-01T01:16:24.928747+0000","last_became_peered":"2023-12-01T01:16:24.928747+0000","last_unstale":"2023-12-01T01:17:24.617436+0000","last_undegraded":"2023-12-01T01:17:24.617436+0000","last_fullsized":"2023-12-01T01:17:24.617436+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:02:24.727835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617440+0000","last_change":"2023-12-01T01:16:25.360959+0000","last_active":"2023-12-01T01:17:24.617440+0000","last_peered":"2023-12-01T01:17:24.617440+0000","last_clean":"2023-12-01T01:17:24.617440+0000","last_became_active":"2023-12-01T01:16:25.360647+0000","last_became_peered":"2023-12-01T01:16:25.360647+0000","last_unstale":"2023-12-01T01:17:24.617440+0000","last_undegraded":"2023-12-01T01:17:24.617440+0000","last_fullsized":"2023-12-01T01:17:24.617440+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:50:24.151273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.616895+0000","last_change":"2023-12-01T01:16:24.938322+0000","last_active":"2023-12-01T01:17:24.616895+0000","last_peered":"2023-12-01T01:17:24.616895+0000","last_clean":"2023-12-01T01:17:24.616895+0000","last_became_active":"2023-12-01T01:16:24.937747+0000","last_became_peered":"2023-12-01T01:16:24.937747+0000","last_unstale":"2023-12-01T01:17:24.616895+0000","last_undegraded":"2023-12-01T01:17:24.616895+0000","last_fullsized":"2023-12-01T01:17:24.616895+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:37:59.999748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.611759+0000","last_change":"2023-12-01T01:16:25.362779+0000","last_active":"2023-12-01T01:17:24.611759+0000","last_peered":"2023-12-01T01:17:24.611759+0000","last_clean":"2023-12-01T01:17:24.611759+0000","last_became_active":"2023-12-01T01:16:25.362487+0000","last_became_peered":"2023-12-01T01:16:25.362487+0000","last_unstale":"2023-12-01T01:17:24.611759+0000","last_undegraded":"2023-12-01T01:17:24.611759+0000","last_fullsized":"2023-12-01T01:17:24.611759+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:04:42.409498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'5","reported_seq":171,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-01T01:17:46.708105+0000","last_change":"2023-12-01T01:16:24.933187+0000","last_active":"2023-12-01T01:17:46.708105+0000","last_peered":"2023-12-01T01:17:46.708105+0000","last_clean":"2023-12-01T01:17:46.708105+0000","last_became_active":"2023-12-01T01:16:24.932819+0000","last_became_peered":"2023-12-01T01:16:24.932819+0000","last_unstale":"2023-12-01T01:17:46.708105+0000","last_undegraded":"2023-12-01T01:17:46.708105+0000","last_fullsized":"2023-12-01T01:17:46.708105+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:54:34.739815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":152,"num_read_kb":152,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.802822+0000","last_change":"2023-12-01T01:16:25.361183+0000","last_active":"2023-12-01T01:17:24.802822+0000","last_peered":"2023-12-01T01:17:24.802822+0000","last_clean":"2023-12-01T01:17:24.802822+0000","last_became_active":"2023-12-01T01:16:25.360871+0000","last_became_peered":"2023-12-01T01:16:25.360871+0000","last_unstale":"2023-12-01T01:17:24.802822+0000","last_undegraded":"2023-12-01T01:17:24.802822+0000","last_fullsized":"2023-12-01T01:17:24.802822+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:06:04.748159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.613828+0000","last_change":"2023-12-01T01:16:24.939355+0000","last_active":"2023-12-01T01:17:24.613828+0000","last_peered":"2023-12-01T01:17:24.613828+0000","last_clean":"2023-12-01T01:17:24.613828+0000","last_became_active":"2023-12-01T01:16:24.938967+0000","last_became_peered":"2023-12-01T01:16:24.938967+0000","last_unstale":"2023-12-01T01:17:24.613828+0000","last_undegraded":"2023-12-01T01:17:24.613828+0000","last_fullsized":"2023-12-01T01:17:24.613828+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:20:51.172753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.615582+0000","last_change":"2023-12-01T01:16:25.366752+0000","last_active":"2023-12-01T01:17:24.615582+0000","last_peered":"2023-12-01T01:17:24.615582+0000","last_clean":"2023-12-01T01:17:24.615582+0000","last_became_active":"2023-12-01T01:16:25.366466+0000","last_became_peered":"2023-12-01T01:16:25.366466+0000","last_unstale":"2023-12-01T01:17:24.615582+0000","last_undegraded":"2023-12-01T01:17:24.615582+0000","last_fullsized":"2023-12-01T01:17:24.615582+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:09:53.716127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617178+0000","last_change":"2023-12-01T01:16:24.940827+0000","last_active":"2023-12-01T01:17:24.617178+0000","last_peered":"2023-12-01T01:17:24.617178+0000","last_clean":"2023-12-01T01:17:24.617178+0000","last_became_active":"2023-12-01T01:16:24.940515+0000","last_became_peered":"2023-12-01T01:16:24.940515+0000","last_unstale":"2023-12-01T01:17:24.617178+0000","last_undegraded":"2023-12-01T01:17:24.617178+0000","last_fullsized":"2023-12-01T01:17:24.617178+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:08:26.409014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.802749+0000","last_change":"2023-12-01T01:16:25.369662+0000","last_active":"2023-12-01T01:17:24.802749+0000","last_peered":"2023-12-01T01:17:24.802749+0000","last_clean":"2023-12-01T01:17:24.802749+0000","last_became_active":"2023-12-01T01:16:25.369178+0000","last_became_peered":"2023-12-01T01:16:25.369178+0000","last_unstale":"2023-12-01T01:17:24.802749+0000","last_undegraded":"2023-12-01T01:17:24.802749+0000","last_fullsized":"2023-12-01T01:17:24.802749+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:40:39.919414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'31","reported_seq":205,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-01T01:17:46.708885+0000","last_change":"2023-12-01T01:16:24.936733+0000","last_active":"2023-12-01T01:17:46.708885+0000","last_peered":"2023-12-01T01:17:46.708885+0000","last_clean":"2023-12-01T01:17:46.708885+0000","last_became_active":"2023-12-01T01:16:24.936459+0000","last_became_peered":"2023-12-01T01:16:24.936459+0000","last_unstale":"2023-12-01T01:17:46.708885+0000","last_undegraded":"2023-12-01T01:17:46.708885+0000","last_fullsized":"2023-12-01T01:17:46.708885+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:19:58.387650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58848,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":160,"num_read_kb":137,"num_write":33,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.616046+0000","last_change":"2023-12-01T01:16:25.364292+0000","last_active":"2023-12-01T01:17:24.616046+0000","last_peered":"2023-12-01T01:17:24.616046+0000","last_clean":"2023-12-01T01:17:24.616046+0000","last_became_active":"2023-12-01T01:16:25.362615+0000","last_became_peered":"2023-12-01T01:16:25.362615+0000","last_unstale":"2023-12-01T01:17:24.616046+0000","last_undegraded":"2023-12-01T01:17:24.616046+0000","last_fullsized":"2023-12-01T01:17:24.616046+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:06:07.350607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'9","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-01T01:17:25.632990+0000","last_change":"2023-12-01T01:17:25.632990+0000","last_active":"2023-12-01T01:17:25.632990+0000","last_peered":"2023-12-01T01:17:25.632990+0000","last_clean":"2023-12-01T01:17:25.632990+0000","last_became_active":"2023-12-01T01:17:25.621584+0000","last_became_peered":"2023-12-01T01:17:25.621584+0000","last_unstale":"2023-12-01T01:17:25.632990+0000","last_undegraded":"2023-12-01T01:17:25.632990+0000","last_fullsized":"2023-12-01T01:17:25.632990+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:33:26.502407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617099+0000","last_change":"2023-12-01T01:16:25.360104+0000","last_active":"2023-12-01T01:17:24.617099+0000","last_peered":"2023-12-01T01:17:24.617099+0000","last_clean":"2023-12-01T01:17:24.617099+0000","last_became_active":"2023-12-01T01:16:25.359807+0000","last_became_peered":"2023-12-01T01:16:25.359807+0000","last_unstale":"2023-12-01T01:17:24.617099+0000","last_undegraded":"2023-12-01T01:17:24.617099+0000","last_fullsized":"2023-12-01T01:17:24.617099+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:04:05.100258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'5","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617006+0000","last_change":"2023-12-01T01:16:24.945074+0000","last_active":"2023-12-01T01:17:24.617006+0000","last_peered":"2023-12-01T01:17:24.617006+0000","last_clean":"2023-12-01T01:17:24.617006+0000","last_became_active":"2023-12-01T01:16:24.944737+0000","last_became_peered":"2023-12-01T01:16:24.944737+0000","last_unstale":"2023-12-01T01:17:24.617006+0000","last_undegraded":"2023-12-01T01:17:24.617006+0000","last_fullsized":"2023-12-01T01:17:24.617006+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:44:53.931454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-01T01:17:25.617181+0000","last_change":"2023-12-01T01:17:25.617181+0000","last_active":"2023-12-01T01:17:25.617181+0000","last_peered":"2023-12-01T01:17:25.617181+0000","last_clean":"2023-12-01T01:17:25.617181+0000","last_became_active":"2023-12-01T01:17:25.616647+0000","last_became_peered":"2023-12-01T01:17:25.616647+0000","last_unstale":"2023-12-01T01:17:25.617181+0000","last_undegraded":"2023-12-01T01:17:25.617181+0000","last_fullsized":"2023-12-01T01:17:25.617181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:59.017264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":93,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-01T01:17:41.726984+0000","last_change":"2023-12-01T01:16:24.929103+0000","last_active":"2023-12-01T01:17:41.726984+0000","last_peered":"2023-12-01T01:17:41.726984+0000","last_clean":"2023-12-01T01:17:41.726984+0000","last_became_active":"2023-12-01T01:16:24.928769+0000","last_became_peered":"2023-12-01T01:16:24.928769+0000","last_unstale":"2023-12-01T01:17:41.726984+0000","last_undegraded":"2023-12-01T01:17:41.726984+0000","last_fullsized":"2023-12-01T01:17:41.726984+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:45:24.542610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.613692+0000","last_change":"2023-12-01T01:16:25.370816+0000","last_active":"2023-12-01T01:17:24.613692+0000","last_peered":"2023-12-01T01:17:24.613692+0000","last_clean":"2023-12-01T01:17:24.613692+0000","last_became_active":"2023-12-01T01:16:25.370465+0000","last_became_peered":"2023-12-01T01:16:25.370465+0000","last_unstale":"2023-12-01T01:17:24.613692+0000","last_undegraded":"2023-12-01T01:17:24.613692+0000","last_fullsized":"2023-12-01T01:17:24.613692+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:52:28.990416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'17","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.614954+0000","last_change":"2023-12-01T01:16:24.933022+0000","last_active":"2023-12-01T01:17:24.614954+0000","last_peered":"2023-12-01T01:17:24.614954+0000","last_clean":"2023-12-01T01:17:24.614954+0000","last_became_active":"2023-12-01T01:16:24.932705+0000","last_became_peered":"2023-12-01T01:16:24.932705+0000","last_unstale":"2023-12-01T01:17:24.614954+0000","last_undegraded":"2023-12-01T01:17:24.614954+0000","last_fullsized":"2023-12-01T01:17:24.614954+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:36:44.747984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.616969+0000","last_change":"2023-12-01T01:16:25.370881+0000","last_active":"2023-12-01T01:17:24.616969+0000","last_peered":"2023-12-01T01:17:24.616969+0000","last_clean":"2023-12-01T01:17:24.616969+0000","last_became_active":"2023-12-01T01:16:25.368242+0000","last_became_peered":"2023-12-01T01:16:25.368242+0000","last_unstale":"2023-12-01T01:17:24.616969+0000","last_undegraded":"2023-12-01T01:17:24.616969+0000","last_fullsized":"2023-12-01T01:17:24.616969+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:35:41.128841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.616226+0000","last_change":"2023-12-01T01:16:24.933579+0000","last_active":"2023-12-01T01:17:24.616226+0000","last_peered":"2023-12-01T01:17:24.616226+0000","last_clean":"2023-12-01T01:17:24.616226+0000","last_became_active":"2023-12-01T01:16:24.933207+0000","last_became_peered":"2023-12-01T01:16:24.933207+0000","last_unstale":"2023-12-01T01:17:24.616226+0000","last_undegraded":"2023-12-01T01:17:24.616226+0000","last_fullsized":"2023-12-01T01:17:24.616226+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:28.309551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.612414+0000","last_change":"2023-12-01T01:16:25.368611+0000","last_active":"2023-12-01T01:17:24.612414+0000","last_peered":"2023-12-01T01:17:24.612414+0000","last_clean":"2023-12-01T01:17:24.612414+0000","last_became_active":"2023-12-01T01:16:25.368291+0000","last_became_peered":"2023-12-01T01:16:25.368291+0000","last_unstale":"2023-12-01T01:17:24.612414+0000","last_undegraded":"2023-12-01T01:17:24.612414+0000","last_fullsized":"2023-12-01T01:17:24.612414+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:17:21.824737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.611963+0000","last_change":"2023-12-01T01:16:24.941410+0000","last_active":"2023-12-01T01:17:24.611963+0000","last_peered":"2023-12-01T01:17:24.611963+0000","last_clean":"2023-12-01T01:17:24.611963+0000","last_became_active":"2023-12-01T01:16:24.941127+0000","last_became_peered":"2023-12-01T01:16:24.941127+0000","last_unstale":"2023-12-01T01:17:24.611963+0000","last_undegraded":"2023-12-01T01:17:24.611963+0000","last_fullsized":"2023-12-01T01:17:24.611963+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:25:20.911892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.619230+0000","last_change":"2023-12-01T01:16:25.368235+0000","last_active":"2023-12-01T01:17:24.619230+0000","last_peered":"2023-12-01T01:17:24.619230+0000","last_clean":"2023-12-01T01:17:24.619230+0000","last_became_active":"2023-12-01T01:16:25.367918+0000","last_became_peered":"2023-12-01T01:16:25.367918+0000","last_unstale":"2023-12-01T01:17:24.619230+0000","last_undegraded":"2023-12-01T01:17:24.619230+0000","last_fullsized":"2023-12-01T01:17:24.619230+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:12:26.665404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'5","reported_seq":91,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-01T01:17:41.714223+0000","last_change":"2023-12-01T01:16:24.932057+0000","last_active":"2023-12-01T01:17:41.714223+0000","last_peered":"2023-12-01T01:17:41.714223+0000","last_clean":"2023-12-01T01:17:41.714223+0000","last_became_active":"2023-12-01T01:16:24.931817+0000","last_became_peered":"2023-12-01T01:16:24.931817+0000","last_unstale":"2023-12-01T01:17:41.714223+0000","last_undegraded":"2023-12-01T01:17:41.714223+0000","last_fullsized":"2023-12-01T01:17:41.714223+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:19:02.039334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.619475+0000","last_change":"2023-12-01T01:16:25.367431+0000","last_active":"2023-12-01T01:17:24.619475+0000","last_peered":"2023-12-01T01:17:24.619475+0000","last_clean":"2023-12-01T01:17:24.619475+0000","last_became_active":"2023-12-01T01:16:25.367184+0000","last_became_peered":"2023-12-01T01:16:25.367184+0000","last_unstale":"2023-12-01T01:17:24.619475+0000","last_undegraded":"2023-12-01T01:17:24.619475+0000","last_fullsized":"2023-12-01T01:17:24.619475+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:21:19.132725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617731+0000","last_change":"2023-12-01T01:16:25.370593+0000","last_active":"2023-12-01T01:17:24.617731+0000","last_peered":"2023-12-01T01:17:24.617731+0000","last_clean":"2023-12-01T01:17:24.617731+0000","last_became_active":"2023-12-01T01:16:25.367614+0000","last_became_peered":"2023-12-01T01:16:25.367614+0000","last_unstale":"2023-12-01T01:17:24.617731+0000","last_undegraded":"2023-12-01T01:17:24.617731+0000","last_fullsized":"2023-12-01T01:17:24.617731+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:56:34.388180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":96,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-01T01:17:45.733343+0000","last_change":"2023-12-01T01:16:24.931188+0000","last_active":"2023-12-01T01:17:45.733343+0000","last_peered":"2023-12-01T01:17:45.733343+0000","last_clean":"2023-12-01T01:17:45.733343+0000","last_became_active":"2023-12-01T01:16:24.930875+0000","last_became_peered":"2023-12-01T01:16:24.930875+0000","last_unstale":"2023-12-01T01:17:45.733343+0000","last_undegraded":"2023-12-01T01:17:45.733343+0000","last_fullsized":"2023-12-01T01:17:45.733343+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:17.729286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":77,"num_read_kb":77,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.616545+0000","last_change":"2023-12-01T01:16:25.361359+0000","last_active":"2023-12-01T01:17:24.616545+0000","last_peered":"2023-12-01T01:17:24.616545+0000","last_clean":"2023-12-01T01:17:24.616545+0000","last_became_active":"2023-12-01T01:16:25.361083+0000","last_became_peered":"2023-12-01T01:16:25.361083+0000","last_unstale":"2023-12-01T01:17:24.616545+0000","last_undegraded":"2023-12-01T01:17:24.616545+0000","last_fullsized":"2023-12-01T01:17:24.616545+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:42:09.417021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617032+0000","last_change":"2023-12-01T01:16:24.927614+0000","last_active":"2023-12-01T01:17:24.617032+0000","last_peered":"2023-12-01T01:17:24.617032+0000","last_clean":"2023-12-01T01:17:24.617032+0000","last_became_active":"2023-12-01T01:16:24.926459+0000","last_became_peered":"2023-12-01T01:16:24.926459+0000","last_unstale":"2023-12-01T01:17:24.617032+0000","last_undegraded":"2023-12-01T01:17:24.617032+0000","last_fullsized":"2023-12-01T01:17:24.617032+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:41:25.435261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":325,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.613554+0000","last_change":"2023-12-01T01:15:25.599551+0000","last_active":"2023-12-01T01:17:24.613554+0000","last_peered":"2023-12-01T01:17:24.613554+0000","last_clean":"2023-12-01T01:17:24.613554+0000","last_became_active":"2023-12-01T01:15:25.598951+0000","last_became_peered":"2023-12-01T01:15:25.598951+0000","last_unstale":"2023-12-01T01:17:24.613554+0000","last_undegraded":"2023-12-01T01:17:24.613554+0000","last_fullsized":"2023-12-01T01:17:24.613554+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:12:31.173756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:12:31.173756+0000","last_clean_scrub_stamp":"2023-12-01T01:12:31.173756+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:00:11.343823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617315+0000","last_change":"2023-12-01T01:16:25.358617+0000","last_active":"2023-12-01T01:17:24.617315+0000","last_peered":"2023-12-01T01:17:24.617315+0000","last_clean":"2023-12-01T01:17:24.617315+0000","last_became_active":"2023-12-01T01:16:25.358325+0000","last_became_peered":"2023-12-01T01:16:25.358325+0000","last_unstale":"2023-12-01T01:17:24.617315+0000","last_undegraded":"2023-12-01T01:17:24.617315+0000","last_fullsized":"2023-12-01T01:17:24.617315+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:40:01.111777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.613450+0000","last_change":"2023-12-01T01:16:24.935103+0000","last_active":"2023-12-01T01:17:24.613450+0000","last_peered":"2023-12-01T01:17:24.613450+0000","last_clean":"2023-12-01T01:17:24.613450+0000","last_became_active":"2023-12-01T01:16:24.934900+0000","last_became_peered":"2023-12-01T01:16:24.934900+0000","last_unstale":"2023-12-01T01:17:24.613450+0000","last_undegraded":"2023-12-01T01:17:24.613450+0000","last_fullsized":"2023-12-01T01:17:24.613450+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:59:37.138090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"82'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-01T01:17:25.638223+0000","last_change":"2023-12-01T01:17:25.638223+0000","last_active":"2023-12-01T01:17:25.638223+0000","last_peered":"2023-12-01T01:17:25.638223+0000","last_clean":"2023-12-01T01:17:25.638223+0000","last_became_active":"2023-12-01T01:17:25.621853+0000","last_became_peered":"2023-12-01T01:17:25.621853+0000","last_unstale":"2023-12-01T01:17:25.638223+0000","last_undegraded":"2023-12-01T01:17:25.638223+0000","last_fullsized":"2023-12-01T01:17:25.638223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:02:03.387431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617685+0000","last_change":"2023-12-01T01:16:25.362137+0000","last_active":"2023-12-01T01:17:24.617685+0000","last_peered":"2023-12-01T01:17:24.617685+0000","last_clean":"2023-12-01T01:17:24.617685+0000","last_became_active":"2023-12-01T01:16:25.361563+0000","last_became_peered":"2023-12-01T01:16:25.361563+0000","last_unstale":"2023-12-01T01:17:24.617685+0000","last_undegraded":"2023-12-01T01:17:24.617685+0000","last_fullsized":"2023-12-01T01:17:24.617685+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:37:56.473920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'9","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.612460+0000","last_change":"2023-12-01T01:16:24.935692+0000","last_active":"2023-12-01T01:17:24.612460+0000","last_peered":"2023-12-01T01:17:24.612460+0000","last_clean":"2023-12-01T01:17:24.612460+0000","last_became_active":"2023-12-01T01:16:24.934799+0000","last_became_peered":"2023-12-01T01:16:24.934799+0000","last_unstale":"2023-12-01T01:17:24.612460+0000","last_undegraded":"2023-12-01T01:17:24.612460+0000","last_fullsized":"2023-12-01T01:17:24.612460+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:22:51.586089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617743+0000","last_change":"2023-12-01T01:16:25.368672+0000","last_active":"2023-12-01T01:17:24.617743+0000","last_peered":"2023-12-01T01:17:24.617743+0000","last_clean":"2023-12-01T01:17:24.617743+0000","last_became_active":"2023-12-01T01:16:25.368309+0000","last_became_peered":"2023-12-01T01:16:25.368309+0000","last_unstale":"2023-12-01T01:17:24.617743+0000","last_undegraded":"2023-12-01T01:17:24.617743+0000","last_fullsized":"2023-12-01T01:17:24.617743+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:30:59.029250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.616635+0000","last_change":"2023-12-01T01:16:24.931322+0000","last_active":"2023-12-01T01:17:24.616635+0000","last_peered":"2023-12-01T01:17:24.616635+0000","last_clean":"2023-12-01T01:17:24.616635+0000","last_became_active":"2023-12-01T01:16:24.930961+0000","last_became_peered":"2023-12-01T01:16:24.930961+0000","last_unstale":"2023-12-01T01:17:24.616635+0000","last_undegraded":"2023-12-01T01:17:24.616635+0000","last_fullsized":"2023-12-01T01:17:24.616635+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:47:16.317035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.615754+0000","last_change":"2023-12-01T01:16:25.364065+0000","last_active":"2023-12-01T01:17:24.615754+0000","last_peered":"2023-12-01T01:17:24.615754+0000","last_clean":"2023-12-01T01:17:24.615754+0000","last_became_active":"2023-12-01T01:16:25.363694+0000","last_became_peered":"2023-12-01T01:16:25.363694+0000","last_unstale":"2023-12-01T01:17:24.615754+0000","last_undegraded":"2023-12-01T01:17:24.615754+0000","last_fullsized":"2023-12-01T01:17:24.615754+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:49:39.091587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.802537+0000","last_change":"2023-12-01T01:16:24.932730+0000","last_active":"2023-12-01T01:17:24.802537+0000","last_peered":"2023-12-01T01:17:24.802537+0000","last_clean":"2023-12-01T01:17:24.802537+0000","last_became_active":"2023-12-01T01:16:24.932404+0000","last_became_peered":"2023-12-01T01:16:24.932404+0000","last_unstale":"2023-12-01T01:17:24.802537+0000","last_undegraded":"2023-12-01T01:17:24.802537+0000","last_fullsized":"2023-12-01T01:17:24.802537+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:41:33.104418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.614957+0000","last_change":"2023-12-01T01:16:25.375795+0000","last_active":"2023-12-01T01:17:24.614957+0000","last_peered":"2023-12-01T01:17:24.614957+0000","last_clean":"2023-12-01T01:17:24.614957+0000","last_became_active":"2023-12-01T01:16:25.375485+0000","last_became_peered":"2023-12-01T01:16:25.375485+0000","last_unstale":"2023-12-01T01:17:24.614957+0000","last_undegraded":"2023-12-01T01:17:24.614957+0000","last_fullsized":"2023-12-01T01:17:24.614957+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:10:04.009511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.616284+0000","last_change":"2023-12-01T01:16:24.936770+0000","last_active":"2023-12-01T01:17:24.616284+0000","last_peered":"2023-12-01T01:17:24.616284+0000","last_clean":"2023-12-01T01:17:24.616284+0000","last_became_active":"2023-12-01T01:16:24.936058+0000","last_became_peered":"2023-12-01T01:16:24.936058+0000","last_unstale":"2023-12-01T01:17:24.616284+0000","last_undegraded":"2023-12-01T01:17:24.616284+0000","last_fullsized":"2023-12-01T01:17:24.616284+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:22:21.414131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.612201+0000","last_change":"2023-12-01T01:16:25.372183+0000","last_active":"2023-12-01T01:17:24.612201+0000","last_peered":"2023-12-01T01:17:24.612201+0000","last_clean":"2023-12-01T01:17:24.612201+0000","last_became_active":"2023-12-01T01:16:25.371865+0000","last_became_peered":"2023-12-01T01:16:25.371865+0000","last_unstale":"2023-12-01T01:17:24.612201+0000","last_undegraded":"2023-12-01T01:17:24.612201+0000","last_fullsized":"2023-12-01T01:17:24.612201+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:19:54.675425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.619142+0000","last_change":"2023-12-01T01:16:24.939397+0000","last_active":"2023-12-01T01:17:24.619142+0000","last_peered":"2023-12-01T01:17:24.619142+0000","last_clean":"2023-12-01T01:17:24.619142+0000","last_became_active":"2023-12-01T01:16:24.939081+0000","last_became_peered":"2023-12-01T01:16:24.939081+0000","last_unstale":"2023-12-01T01:17:24.619142+0000","last_undegraded":"2023-12-01T01:17:24.619142+0000","last_fullsized":"2023-12-01T01:17:24.619142+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:21:34.967255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.616722+0000","last_change":"2023-12-01T01:16:25.368484+0000","last_active":"2023-12-01T01:17:24.616722+0000","last_peered":"2023-12-01T01:17:24.616722+0000","last_clean":"2023-12-01T01:17:24.616722+0000","last_became_active":"2023-12-01T01:16:25.367638+0000","last_became_peered":"2023-12-01T01:16:25.367638+0000","last_unstale":"2023-12-01T01:17:24.616722+0000","last_undegraded":"2023-12-01T01:17:24.616722+0000","last_fullsized":"2023-12-01T01:17:24.616722+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:06:05.354443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.615886+0000","last_change":"2023-12-01T01:16:24.933304+0000","last_active":"2023-12-01T01:17:24.615886+0000","last_peered":"2023-12-01T01:17:24.615886+0000","last_clean":"2023-12-01T01:17:24.615886+0000","last_became_active":"2023-12-01T01:16:24.933028+0000","last_became_peered":"2023-12-01T01:16:24.933028+0000","last_unstale":"2023-12-01T01:17:24.615886+0000","last_undegraded":"2023-12-01T01:17:24.615886+0000","last_fullsized":"2023-12-01T01:17:24.615886+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:26:51.810995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.613747+0000","last_change":"2023-12-01T01:16:25.364888+0000","last_active":"2023-12-01T01:17:24.613747+0000","last_peered":"2023-12-01T01:17:24.613747+0000","last_clean":"2023-12-01T01:17:24.613747+0000","last_became_active":"2023-12-01T01:16:25.364595+0000","last_became_peered":"2023-12-01T01:16:25.364595+0000","last_unstale":"2023-12-01T01:17:24.613747+0000","last_undegraded":"2023-12-01T01:17:24.613747+0000","last_fullsized":"2023-12-01T01:17:24.613747+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:54:23.178824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'6","reported_seq":96,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-01T01:17:44.705967+0000","last_change":"2023-12-01T01:16:24.945771+0000","last_active":"2023-12-01T01:17:44.705967+0000","last_peered":"2023-12-01T01:17:44.705967+0000","last_clean":"2023-12-01T01:17:44.705967+0000","last_became_active":"2023-12-01T01:16:24.945441+0000","last_became_peered":"2023-12-01T01:16:24.945441+0000","last_unstale":"2023-12-01T01:17:44.705967+0000","last_undegraded":"2023-12-01T01:17:44.705967+0000","last_fullsized":"2023-12-01T01:17:44.705967+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:49:41.234472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1440,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":76,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.613993+0000","last_change":"2023-12-01T01:16:25.365246+0000","last_active":"2023-12-01T01:17:24.613993+0000","last_peered":"2023-12-01T01:17:24.613993+0000","last_clean":"2023-12-01T01:17:24.613993+0000","last_became_active":"2023-12-01T01:16:25.364364+0000","last_became_peered":"2023-12-01T01:16:25.364364+0000","last_unstale":"2023-12-01T01:17:24.613993+0000","last_undegraded":"2023-12-01T01:17:24.613993+0000","last_fullsized":"2023-12-01T01:17:24.613993+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:56:44.815172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.616942+0000","last_change":"2023-12-01T01:16:24.944229+0000","last_active":"2023-12-01T01:17:24.616942+0000","last_peered":"2023-12-01T01:17:24.616942+0000","last_clean":"2023-12-01T01:17:24.616942+0000","last_became_active":"2023-12-01T01:16:24.943880+0000","last_became_peered":"2023-12-01T01:16:24.943880+0000","last_unstale":"2023-12-01T01:17:24.616942+0000","last_undegraded":"2023-12-01T01:17:24.616942+0000","last_fullsized":"2023-12-01T01:17:24.616942+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:44:26.769358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.613765+0000","last_change":"2023-12-01T01:16:25.361028+0000","last_active":"2023-12-01T01:17:24.613765+0000","last_peered":"2023-12-01T01:17:24.613765+0000","last_clean":"2023-12-01T01:17:24.613765+0000","last_became_active":"2023-12-01T01:16:25.360702+0000","last_became_peered":"2023-12-01T01:16:25.360702+0000","last_unstale":"2023-12-01T01:17:24.613765+0000","last_undegraded":"2023-12-01T01:17:24.613765+0000","last_fullsized":"2023-12-01T01:17:24.613765+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:45:09.472497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'5","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617330+0000","last_change":"2023-12-01T01:16:24.945567+0000","last_active":"2023-12-01T01:17:24.617330+0000","last_peered":"2023-12-01T01:17:24.617330+0000","last_clean":"2023-12-01T01:17:24.617330+0000","last_became_active":"2023-12-01T01:16:24.945256+0000","last_became_peered":"2023-12-01T01:16:24.945256+0000","last_unstale":"2023-12-01T01:17:24.617330+0000","last_undegraded":"2023-12-01T01:17:24.617330+0000","last_fullsized":"2023-12-01T01:17:24.617330+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:19:30.944573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.613989+0000","last_change":"2023-12-01T01:16:25.367247+0000","last_active":"2023-12-01T01:17:24.613989+0000","last_peered":"2023-12-01T01:17:24.613989+0000","last_clean":"2023-12-01T01:17:24.613989+0000","last_became_active":"2023-12-01T01:16:25.366872+0000","last_became_peered":"2023-12-01T01:16:25.366872+0000","last_unstale":"2023-12-01T01:17:24.613989+0000","last_undegraded":"2023-12-01T01:17:24.613989+0000","last_fullsized":"2023-12-01T01:17:24.613989+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:03:07.697233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.612290+0000","last_change":"2023-12-01T01:16:25.376005+0000","last_active":"2023-12-01T01:17:24.612290+0000","last_peered":"2023-12-01T01:17:24.612290+0000","last_clean":"2023-12-01T01:17:24.612290+0000","last_became_active":"2023-12-01T01:16:25.375702+0000","last_became_peered":"2023-12-01T01:16:25.375702+0000","last_unstale":"2023-12-01T01:17:24.612290+0000","last_undegraded":"2023-12-01T01:17:24.612290+0000","last_fullsized":"2023-12-01T01:17:24.612290+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:14:00.955730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.619071+0000","last_change":"2023-12-01T01:16:25.371023+0000","last_active":"2023-12-01T01:17:24.619071+0000","last_peered":"2023-12-01T01:17:24.619071+0000","last_clean":"2023-12-01T01:17:24.619071+0000","last_became_active":"2023-12-01T01:16:25.370694+0000","last_became_peered":"2023-12-01T01:16:25.370694+0000","last_unstale":"2023-12-01T01:17:24.619071+0000","last_undegraded":"2023-12-01T01:17:24.619071+0000","last_fullsized":"2023-12-01T01:17:24.619071+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:03:00.715413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.618988+0000","last_change":"2023-12-01T01:16:25.361795+0000","last_active":"2023-12-01T01:17:24.618988+0000","last_peered":"2023-12-01T01:17:24.618988+0000","last_clean":"2023-12-01T01:17:24.618988+0000","last_became_active":"2023-12-01T01:16:25.361459+0000","last_became_peered":"2023-12-01T01:16:25.361459+0000","last_unstale":"2023-12-01T01:17:24.618988+0000","last_undegraded":"2023-12-01T01:17:24.618988+0000","last_fullsized":"2023-12-01T01:17:24.618988+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:18:57.531240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.615185+0000","last_change":"2023-12-01T01:16:25.363773+0000","last_active":"2023-12-01T01:17:24.615185+0000","last_peered":"2023-12-01T01:17:24.615185+0000","last_clean":"2023-12-01T01:17:24.615185+0000","last_became_active":"2023-12-01T01:16:25.363422+0000","last_became_peered":"2023-12-01T01:16:25.363422+0000","last_unstale":"2023-12-01T01:17:24.615185+0000","last_undegraded":"2023-12-01T01:17:24.615185+0000","last_fullsized":"2023-12-01T01:17:24.615185+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:48:33.395566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617624+0000","last_change":"2023-12-01T01:16:25.359431+0000","last_active":"2023-12-01T01:17:24.617624+0000","last_peered":"2023-12-01T01:17:24.617624+0000","last_clean":"2023-12-01T01:17:24.617624+0000","last_became_active":"2023-12-01T01:16:25.359116+0000","last_became_peered":"2023-12-01T01:16:25.359116+0000","last_unstale":"2023-12-01T01:17:24.617624+0000","last_undegraded":"2023-12-01T01:17:24.617624+0000","last_fullsized":"2023-12-01T01:17:24.617624+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:32:26.545269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.612682+0000","last_change":"2023-12-01T01:16:25.368008+0000","last_active":"2023-12-01T01:17:24.612682+0000","last_peered":"2023-12-01T01:17:24.612682+0000","last_clean":"2023-12-01T01:17:24.612682+0000","last_became_active":"2023-12-01T01:16:25.367726+0000","last_became_peered":"2023-12-01T01:16:25.367726+0000","last_unstale":"2023-12-01T01:17:24.612682+0000","last_undegraded":"2023-12-01T01:17:24.612682+0000","last_fullsized":"2023-12-01T01:17:24.612682+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:38:33.837846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.619425+0000","last_change":"2023-12-01T01:16:25.372781+0000","last_active":"2023-12-01T01:17:24.619425+0000","last_peered":"2023-12-01T01:17:24.619425+0000","last_clean":"2023-12-01T01:17:24.619425+0000","last_became_active":"2023-12-01T01:16:25.371443+0000","last_became_peered":"2023-12-01T01:16:25.371443+0000","last_unstale":"2023-12-01T01:17:24.619425+0000","last_undegraded":"2023-12-01T01:17:24.619425+0000","last_fullsized":"2023-12-01T01:17:24.619425+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:49:06.422055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617645+0000","last_change":"2023-12-01T01:16:25.363048+0000","last_active":"2023-12-01T01:17:24.617645+0000","last_peered":"2023-12-01T01:17:24.617645+0000","last_clean":"2023-12-01T01:17:24.617645+0000","last_became_active":"2023-12-01T01:16:25.362752+0000","last_became_peered":"2023-12-01T01:16:25.362752+0000","last_unstale":"2023-12-01T01:17:24.617645+0000","last_undegraded":"2023-12-01T01:17:24.617645+0000","last_fullsized":"2023-12-01T01:17:24.617645+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:54:16.492562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-01T01:17:25.622863+0000","last_change":"2023-12-01T01:17:25.622338+0000","last_active":"2023-12-01T01:17:25.622863+0000","last_peered":"2023-12-01T01:17:25.622863+0000","last_clean":"2023-12-01T01:17:25.622863+0000","last_became_active":"2023-12-01T01:17:25.621964+0000","last_became_peered":"2023-12-01T01:17:25.621964+0000","last_unstale":"2023-12-01T01:17:25.622863+0000","last_undegraded":"2023-12-01T01:17:25.622863+0000","last_fullsized":"2023-12-01T01:17:25.622863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:05:58.699885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.616247+0000","last_change":"2023-12-01T01:16:25.373615+0000","last_active":"2023-12-01T01:17:24.616247+0000","last_peered":"2023-12-01T01:17:24.616247+0000","last_clean":"2023-12-01T01:17:24.616247+0000","last_became_active":"2023-12-01T01:16:25.373247+0000","last_became_peered":"2023-12-01T01:16:25.373247+0000","last_unstale":"2023-12-01T01:17:24.616247+0000","last_undegraded":"2023-12-01T01:17:24.616247+0000","last_fullsized":"2023-12-01T01:17:24.616247+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:28:10.815906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.802585+0000","last_change":"2023-12-01T01:16:25.367404+0000","last_active":"2023-12-01T01:17:24.802585+0000","last_peered":"2023-12-01T01:17:24.802585+0000","last_clean":"2023-12-01T01:17:24.802585+0000","last_became_active":"2023-12-01T01:16:25.366404+0000","last_became_peered":"2023-12-01T01:16:25.366404+0000","last_unstale":"2023-12-01T01:17:24.802585+0000","last_undegraded":"2023-12-01T01:17:24.802585+0000","last_fullsized":"2023-12-01T01:17:24.802585+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:54:49.668036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617543+0000","last_change":"2023-12-01T01:16:25.360315+0000","last_active":"2023-12-01T01:17:24.617543+0000","last_peered":"2023-12-01T01:17:24.617543+0000","last_clean":"2023-12-01T01:17:24.617543+0000","last_became_active":"2023-12-01T01:16:25.360031+0000","last_became_peered":"2023-12-01T01:16:25.360031+0000","last_unstale":"2023-12-01T01:17:24.617543+0000","last_undegraded":"2023-12-01T01:17:24.617543+0000","last_fullsized":"2023-12-01T01:17:24.617543+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:13:24.437697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.612327+0000","last_change":"2023-12-01T01:16:25.367820+0000","last_active":"2023-12-01T01:17:24.612327+0000","last_peered":"2023-12-01T01:17:24.612327+0000","last_clean":"2023-12-01T01:17:24.612327+0000","last_became_active":"2023-12-01T01:16:25.367525+0000","last_became_peered":"2023-12-01T01:16:25.367525+0000","last_unstale":"2023-12-01T01:17:24.612327+0000","last_undegraded":"2023-12-01T01:17:24.612327+0000","last_fullsized":"2023-12-01T01:17:24.612327+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:28:01.552342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65789,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":781,"num_read_kb":468,"num_write":233,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":180,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":197049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481820,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":933888,"data_stored":441376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.70399999999999996}]},{"osd":1,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.66000000000000003}]},{"osd":2,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.499},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.53200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":4,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.77600000000000002}]},{"osd":5,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.80800000000000005}]},{"osd":6,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.66200000000000003}]},{"osd":7,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.83699999999999997}]},{"osd":8,"last update":"Fri Dec 1 01:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.82299999999999995}]},{"osd":9,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.92400000000000004}]},{"osd":10,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.878},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.649,"5min":1.649,"15min":1.649},"last":0.84899999999999998}]}]},{"osd":10,"up_from":69,"seq":296352743456,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28528,"kb_used_data":1576,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716624,"statfs":{"total":95995035648,"available":95965822976,"internally_reserved":0,"allocated":1613824,"data_stored":1090430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.499}]},{"osd":1,"last update":"Fri Dec 1 01:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.46000000000000002,"15min":0.46800000000000003},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.56200000000000006,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.48199999999999998,"15min":0.48399999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70899999999999996,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.52400000000000002}]},{"osd":2,"last update":"Fri Dec 1 01:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57499999999999996,"15min":0.56699999999999995},"min":{"1min":0.45900000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.82799999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81200000000000006,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.74099999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.59199999999999997,"15min":0.60199999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.82599999999999996,"15min":0.874},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.70899999999999996,"5min":5.7240000000000002,"15min":5.7240000000000002},"last":0.66100000000000003}]},{"osd":4,"last update":"Fri Dec 1 01:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.58599999999999997,"15min":0.59599999999999997},"min":{"1min":0.35899999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.70999999999999996,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.57799999999999996,"15min":0.58499999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.68999999999999995,"5min":1.016,"15min":1.016},"last":0.70899999999999996}]},{"osd":5,"last update":"Fri Dec 1 01:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.67700000000000005,"15min":0.67900000000000005},"min":{"1min":0.50900000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.78500000000000003,"5min":1.109,"15min":1.109},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.629,"5min":0.67300000000000004,"15min":0.68100000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.87,"5min":0.998,"15min":0.998},"last":0.80200000000000005}]},{"osd":6,"last update":"Fri Dec 1 01:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.69799999999999995,"15min":0.71699999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81999999999999995,"5min":1.054,"15min":1.054},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.63,"5min":0.90600000000000003,"15min":0.95199999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.82699999999999996,"5min":5.7510000000000003,"15min":5.7510000000000003},"last":0.78300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":8,"last update":"Fri Dec 1 01:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.73799999999999999,"15min":0.74399999999999999},"min":{"1min":0.56699999999999995,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.871,"5min":1.23,"15min":1.23},"last":0.878},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.76200000000000001,"15min":0.76900000000000002},"min":{"1min":0.53700000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.91700000000000004,"5min":1.216,"15min":1.216},"last":0.84199999999999997}]},{"osd":9,"last update":"Fri Dec 1 01:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.78900000000000003,"15min":0.78800000000000003},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.8839999999999999,"5min":2.8839999999999999,"15min":2.8839999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.78300000000000003,"15min":0.79500000000000004},"min":{"1min":0.52300000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88700000000000001,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.77100000000000002}]},{"osd":11,"last update":"Fri Dec 1 01:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.91400000000000003}]}]},{"osd":9,"up_from":64,"seq":274877906981,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":942080,"data_stored":440856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.46700000000000003,"15min":0.44900000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":3.6589999999999998,"5min":3.6589999999999998,"15min":3.6589999999999998},"last":0.371},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.45600000000000002,"15min":0.46100000000000002},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.47799999999999998}]},{"osd":1,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.58199999999999996,"15min":0.59199999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.65100000000000002,"15min":0.63200000000000001},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":3.4969999999999999,"5min":3.4969999999999999,"15min":3.4969999999999999},"last":0.496}]},{"osd":2,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57999999999999996,"15min":0.58499999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.72299999999999998,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.64200000000000002,"15min":0.625},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":3.5470000000000002,"5min":3.5470000000000002,"15min":3.5470000000000002},"last":0.45900000000000002}]},{"osd":3,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.54500000000000004,"15min":0.55700000000000005},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.53300000000000003,"15min":0.54500000000000004},"min":{"1min":0.33800000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.40600000000000003}]},{"osd":4,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.71299999999999997,"15min":0.69899999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":3.6019999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.69499999999999995,"15min":0.71299999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.872,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.52700000000000002}]},{"osd":5,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.67400000000000004,"15min":0.68999999999999995},"min":{"1min":0.46500000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.79100000000000004,"5min":1.099,"15min":1.099},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.66600000000000004,"15min":0.68200000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.70999999999999996,"5min":1.052,"15min":1.052},"last":0.64700000000000002}]},{"osd":6,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.626,"15min":0.64400000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.69799999999999995,"15min":0.69199999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":3.8410000000000002,"5min":3.8410000000000002,"15min":3.8410000000000002},"last":0.54000000000000004}]},{"osd":7,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.76600000000000001,"15min":0.79000000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.83199999999999996,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.749,"15min":0.77200000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81299999999999994,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.68300000000000005}]},{"osd":8,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.77400000000000002,"15min":0.76600000000000001},"min":{"1min":0.50900000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":3.5760000000000001,"5min":3.5760000000000001,"15min":3.5760000000000001},"last":0.628},{"interface":"front","average":{"1min":0.629,"5min":0.751,"15min":0.77100000000000002},"min":{"1min":0.5,"5min":0.495,"15min":0.495},"max":{"1min":0.76900000000000002,"5min":1.157,"15min":1.157},"last":0.65600000000000003}]},{"osd":10,"last update":"Fri Dec 1 01:17:25 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.74199999999999999,"15min":0.72599999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":3.629,"5min":3.629,"15min":3.629},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.70599999999999996,"15min":0.71699999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.85399999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.64000000000000001}]}]},{"osd":8,"up_from":57,"seq":244813135912,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":929792,"data_stored":441122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.57899999999999996,"15min":0.55700000000000005},"min":{"1min":0.39500000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":3.4089999999999998,"5min":3.4089999999999998,"15min":3.4089999999999998},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54000000000000004,"15min":0.53700000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.497}]},{"osd":1,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.626,"15min":0.58399999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":6.0960000000000001,"5min":6.0960000000000001,"15min":6.0960000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.61099999999999999,"15min":0.58899999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":3.556,"5min":3.556,"15min":3.556},"last":0.63}]},{"osd":2,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.50900000000000001,"15min":0.47099999999999997},"min":{"1min":0.28399999999999997,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":5.9199999999999999,"5min":5.9199999999999999,"15min":5.9199999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.48499999999999999,"15min":0.45400000000000001},"min":{"1min":0.311,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":3.6099999999999999,"5min":3.6099999999999999,"15min":3.6099999999999999},"last":0.51500000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.627,"15min":0.58199999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":6.0209999999999999,"5min":6.0209999999999999,"15min":6.0209999999999999},"last":0.47399999999999998}]},{"osd":4,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.60999999999999999,"15min":0.60399999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.66400000000000003,"15min":0.61899999999999999},"min":{"1min":0.39100000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":6.0709999999999997,"5min":6.0709999999999997,"15min":6.0709999999999997},"last":0.69199999999999995}]},{"osd":5,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.58399999999999996,"15min":0.55700000000000005},"min":{"1min":0.35799999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":3.4580000000000002,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.55500000000000005,"15min":0.54000000000000004},"min":{"1min":0.38200000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.61299999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.73399999999999999,"15min":0.69499999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":5.9969999999999999,"5min":5.9969999999999999,"15min":5.9969999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.71499999999999997,"15min":0.68899999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":3.4980000000000002,"5min":3.4980000000000002,"15min":3.4980000000000002},"last":0.58399999999999996}]},{"osd":7,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.72299999999999998,"15min":0.69999999999999996},"min":{"1min":0.498,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":3.5800000000000001,"5min":3.5800000000000001,"15min":3.5800000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.69399999999999995,"15min":0.68600000000000005},"min":{"1min":0.51800000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0229999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.72399999999999998}]},{"osd":9,"last update":"Fri Dec 1 01:17:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":1.0409999999999999,"15min":1.085},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.647,"5min":5.9690000000000003,"15min":5.9690000000000003},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.91300000000000003,"15min":0.93500000000000005},"min":{"1min":0.51700000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.532,"5min":3.5259999999999998,"15min":3.5259999999999998},"last":0.68000000000000005}]},{"osd":10,"last update":"Fri Dec 1 01:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.71499999999999997,"15min":0.69899999999999995},"min":{"1min":0.60399999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.97999999999999998,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.96999999999999997,"15min":0.99099999999999999},"min":{"1min":0.60599999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.95499999999999996,"5min":6.0469999999999997,"15min":6.0469999999999997},"last":0.83399999999999996}]},{"osd":11,"last update":"Fri Dec 1 01:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.80800000000000005}]}]},{"osd":7,"up_from":50,"seq":214748364844,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28452,"kb_used_data":1500,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716700,"statfs":{"total":95995035648,"available":95965900800,"internally_reserved":0,"allocated":1536000,"data_stored":1033616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.45000000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.40799999999999997,"15min":0.39500000000000002},"min":{"1min":0.27000000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.66100000000000003,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.42699999999999999,"15min":0.40699999999999997},"min":{"1min":0.314,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.67700000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.56999999999999995}]},{"osd":2,"last update":"Fri Dec 1 01:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.57899999999999996,"15min":0.56399999999999995},"min":{"1min":0.432,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.876,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58899999999999997,"15min":0.58299999999999996},"min":{"1min":0.39300000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.71499999999999997,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.71599999999999997}]},{"osd":3,"last update":"Fri Dec 1 01:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56399999999999995,"15min":0.55200000000000005},"min":{"1min":0.42099999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.72799999999999998,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.45400000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.71499999999999997,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.47799999999999998}]},{"osd":4,"last update":"Fri Dec 1 01:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.52000000000000002,"15min":0.51400000000000001},"min":{"1min":0.41499999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.71499999999999997,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51200000000000001,"15min":0.51000000000000001},"min":{"1min":0.28899999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.66900000000000004,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.54400000000000004}]},{"osd":5,"last update":"Fri Dec 1 01:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.629,"15min":0.61699999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.84299999999999997,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.62},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.628,"15min":0.61199999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79700000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.66800000000000004}]},{"osd":6,"last update":"Fri Dec 1 01:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65000000000000002,"15min":0.64400000000000002},"min":{"1min":0.46100000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.81599999999999995,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.66000000000000003,"15min":0.65600000000000003},"min":{"1min":0.46800000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.873,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.73299999999999998}]},{"osd":8,"last update":"Fri Dec 1 01:16:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.71699999999999997,"15min":0.72099999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.89400000000000002,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.748,"5min":0.74199999999999999,"15min":0.74099999999999999},"min":{"1min":0.49199999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.86699999999999999}]},{"osd":9,"last update":"Fri Dec 1 01:17:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.72099999999999997,"15min":0.72399999999999998},"min":{"1min":0.51200000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.93600000000000005,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.71899999999999997,"15min":0.72099999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.874,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.79600000000000004}]},{"osd":10,"last update":"Fri Dec 1 01:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.71499999999999997,"15min":0.71999999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.91000000000000003,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.71699999999999997,"15min":0.71799999999999997},"min":{"1min":0.52500000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.88500000000000001,"5min":1.032,"15min":1.032},"last":0.77200000000000002}]},{"osd":11,"last update":"Fri Dec 1 01:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.81799999999999995}]}]},{"osd":6,"up_from":45,"seq":193273528370,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":440531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.27600000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.502,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.42299999999999999,"15min":0.41299999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":0.40300000000000002}]},{"osd":1,"last update":"Fri Dec 1 01:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61899999999999999,"15min":0.625},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.57499999999999996,"15min":0.59399999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.68100000000000005,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.47999999999999998}]},{"osd":2,"last update":"Fri Dec 1 01:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.59599999999999997,"15min":0.57699999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":2.9870000000000001,"5min":2.9870000000000001,"15min":2.9870000000000001},"last":0.502},{"interface":"front","average":{"1min":0.623,"5min":0.629,"15min":0.623},"min":{"1min":0.42199999999999999,"5min":0.38,"15min":0.38},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.42099999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48399999999999999,"15min":0.46999999999999997},"min":{"1min":0.318,"5min":0.308,"15min":0.308},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.45100000000000001,"15min":0.45900000000000002},"min":{"1min":0.29999999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.501,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.311}]},{"osd":4,"last update":"Fri Dec 1 01:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.61899999999999999,"15min":0.61099999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":2.919,"5min":2.919,"15min":2.919},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.65600000000000003,"15min":0.65200000000000002},"min":{"1min":0.45400000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.49199999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.61699999999999999,"15min":0.63300000000000001},"min":{"1min":0.40300000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.67000000000000004,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.439},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.67100000000000004,"15min":0.65700000000000003},"min":{"1min":0.437,"5min":0.373,"15min":0.373},"max":{"1min":3.0179999999999998,"5min":3.0179999999999998,"15min":3.0179999999999998},"last":0.53600000000000003}]},{"osd":7,"last update":"Fri Dec 1 01:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.70399999999999996,"15min":0.70599999999999996},"min":{"1min":0.45700000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":2.956,"5min":2.956,"15min":2.956},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.76100000000000001,"15min":0.747},"min":{"1min":0.504,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":3.4660000000000002,"5min":3.4660000000000002,"15min":3.4660000000000002},"last":0.59399999999999997}]},{"osd":8,"last update":"Fri Dec 1 01:17:42 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.74099999999999999,"15min":0.76400000000000001},"min":{"1min":0.47099999999999997,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.81200000000000006,"5min":1.635,"15min":1.635},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.65500000000000003,"15min":0.68300000000000005},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.69699999999999995,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.51100000000000001}]},{"osd":9,"last update":"Fri Dec 1 01:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.69699999999999995,"15min":0.70399999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":2.1320000000000001,"5min":2.1320000000000001,"15min":2.1320000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.71999999999999997,"15min":0.73099999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":0.56200000000000006}]},{"osd":10,"last update":"Fri Dec 1 01:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.70799999999999996,"15min":0.69299999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":3.9830000000000001,"5min":3.9830000000000001,"15min":3.9830000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.78400000000000003,"15min":0.79100000000000004},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":0.624}]}]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":987136,"data_stored":499968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:17:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.51400000000000001,"15min":0.51700000000000002},"min":{"1min":0.378,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.64600000000000002,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.496},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.54300000000000004,"15min":0.52300000000000002},"min":{"1min":0.38100000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":0.623}]},{"osd":2,"last update":"Fri Dec 1 01:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.57999999999999996,"15min":0.50800000000000001},"min":{"1min":0.47199999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80800000000000005,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.61599999999999999,"15min":0.54600000000000004},"min":{"1min":0.38,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85699999999999998,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":0.73499999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58399999999999996,"15min":0.55900000000000005},"min":{"1min":0.44700000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72099999999999997,"5min":1.198,"15min":1.198},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59999999999999998,"15min":0.57399999999999995},"min":{"1min":0.46700000000000003,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.77400000000000002,"5min":1.282,"15min":1.282},"last":0.55200000000000005}]},{"osd":4,"last update":"Fri Dec 1 01:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.46100000000000002,"15min":0.42499999999999999},"min":{"1min":0.39800000000000002,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.748,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.52300000000000002,"15min":0.48999999999999999},"min":{"1min":0.372,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.76800000000000002,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.67300000000000004}]},{"osd":5,"last update":"Fri Dec 1 01:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.60199999999999998,"15min":0.53300000000000003},"min":{"1min":0.42799999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85399999999999998,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.61399999999999999,"15min":0.53900000000000003},"min":{"1min":0.56200000000000006,"5min":0.316,"15min":0.316},"max":{"1min":0.85899999999999999,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.66000000000000003}]},{"osd":6,"last update":"Fri Dec 1 01:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60599999999999998,"15min":0.59599999999999997},"min":{"1min":0.47699999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79900000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.628,"15min":0.61599999999999999},"min":{"1min":0.46400000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.89600000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.69899999999999995}]},{"osd":7,"last update":"Fri Dec 1 01:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.57799999999999996,"15min":0.54600000000000004},"min":{"1min":0.44500000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.84999999999999998,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.59199999999999997,"15min":0.55600000000000005},"min":{"1min":0.44500000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82999999999999996,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.56499999999999995}]},{"osd":8,"last update":"Fri Dec 1 01:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.72399999999999998,"15min":0.71899999999999997},"min":{"1min":0.58999999999999997,"5min":0.439,"15min":0.439},"max":{"1min":0.92800000000000005,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.74199999999999999,"15min":0.74299999999999999},"min":{"1min":0.55000000000000004,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92200000000000004,"5min":1.252,"15min":1.252},"last":0.72199999999999998}]},{"osd":9,"last update":"Fri Dec 1 01:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.73099999999999998,"15min":0.73499999999999999},"min":{"1min":0.503,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88500000000000001,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.76200000000000001,"15min":0.76600000000000001},"min":{"1min":0.58699999999999997,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.93999999999999995,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.76400000000000001}]},{"osd":10,"last update":"Fri Dec 1 01:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.73099999999999998,"15min":0.72699999999999998},"min":{"1min":0.54600000000000004,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.91900000000000004,"5min":1,"15min":1},"last":0.751},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.77300000000000002,"15min":0.77100000000000002},"min":{"1min":0.54900000000000004,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93999999999999995,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.58599999999999997}]},{"osd":11,"last update":"Fri Dec 1 01:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.78500000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738444,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":946176,"data_stored":442182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 01:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.53100000000000003,"15min":0.47899999999999998},"min":{"1min":0.39100000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":3.6160000000000001,"5min":3.6160000000000001,"15min":3.6160000000000001},"last":0.505},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55000000000000004,"15min":0.498},"min":{"1min":0.375,"5min":0.316,"15min":0.316},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.59599999999999997}]},{"osd":2,"last update":"Fri Dec 1 01:17:39 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.621,"15min":0.53000000000000003},"min":{"1min":0.41199999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":4.5010000000000003,"5min":4.5010000000000003,"15min":4.5010000000000003},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56699999999999995,"15min":0.51900000000000002},"min":{"1min":0.39300000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.909,"5min":3.1429999999999998,"15min":3.1429999999999998},"last":0.49199999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:17:25 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":3.6640000000000001,"5min":3.6640000000000001,"15min":3.6640000000000001},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":3.734,"5min":3.734,"15min":3.734},"last":0.42099999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.57299999999999995,"15min":0.54300000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":4.5129999999999999,"5min":4.5129999999999999,"15min":4.5129999999999999},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.58899999999999997,"15min":0.57499999999999996},"min":{"1min":0.40500000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.52000000000000002}]},{"osd":5,"last update":"Fri Dec 1 01:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57399999999999995,"15min":0.55900000000000005},"min":{"1min":0.39000000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.948,"5min":1.948,"15min":1.948},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.64300000000000002,"15min":0.61099999999999999},"min":{"1min":0.45600000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":3.573,"5min":3.573,"15min":3.573},"last":0.628}]},{"osd":6,"last update":"Fri Dec 1 01:17:00 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.59599999999999997,"15min":0.55500000000000005},"min":{"1min":0.27900000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":3.7189999999999999,"5min":3.7189999999999999,"15min":3.7189999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.60199999999999998,"15min":0.56699999999999995},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":3.9340000000000002,"5min":3.9340000000000002,"15min":3.9340000000000002},"last":0.54400000000000004}]},{"osd":7,"last update":"Fri Dec 1 01:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.623,"15min":0.621},"min":{"1min":0.43099999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67800000000000005,"15min":0.68200000000000005},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":1.6220000000000001}]},{"osd":8,"last update":"Fri Dec 1 01:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.66400000000000003,"15min":0.66300000000000003},"min":{"1min":0.42599999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":3.996,"5min":3.996,"15min":3.996},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.69699999999999995,"15min":0.68300000000000005},"min":{"1min":0.47899999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":4.5529999999999999,"5min":4.5529999999999999,"15min":4.5529999999999999},"last":0.58299999999999996}]},{"osd":9,"last update":"Fri Dec 1 01:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.67600000000000005,"15min":0.67200000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":3.694,"5min":3.694,"15min":3.694},"last":0.65900000000000003},{"interface":"front","average":{"1min":1.1910000000000001,"5min":0.751,"15min":0.67700000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":14.702,"5min":14.702,"15min":14.702},"last":0.60999999999999999}]},{"osd":10,"last update":"Fri Dec 1 01:17:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.66200000000000003,"15min":0.67300000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.73999999999999999,"15min":0.75800000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.78100000000000003,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.68000000000000005}]},{"osd":11,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":3.9689999999999999,"5min":3.9689999999999999,"15min":3.9689999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":0.61699999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411394,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":999424,"data_stored":500465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:17:39 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.503,"15min":0.48199999999999998},"min":{"1min":0.38700000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.84999999999999998,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.497,"15min":0.47899999999999998},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70199999999999996,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.51000000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:17:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.54200000000000004,"15min":0.51500000000000001},"min":{"1min":0.34799999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.755,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61299999999999999,"15min":0.55000000000000004},"min":{"1min":0.37,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.755,"5min":5.4169999999999998,"15min":5.4169999999999998},"last":0.72599999999999998}]},{"osd":3,"last update":"Fri Dec 1 01:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.59799999999999998,"15min":0.53000000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.91400000000000003,"5min":5.4710000000000001,"15min":5.4710000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56999999999999995,"15min":0.53000000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.78500000000000003,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.626}]},{"osd":4,"last update":"Fri Dec 1 01:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.745},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.85799999999999998}]},{"osd":5,"last update":"Fri Dec 1 01:17:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.45800000000000002,"15min":0.42899999999999999},"min":{"1min":0.39900000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.70799999999999996,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.52000000000000002,"15min":0.47999999999999998},"min":{"1min":0.39300000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.80100000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.80100000000000005}]},{"osd":6,"last update":"Fri Dec 1 01:17:00 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.64500000000000002,"15min":0.60499999999999998},"min":{"1min":0.43099999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.89300000000000002,"5min":5.5019999999999998,"15min":5.5019999999999998},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.63800000000000001,"15min":0.63400000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77100000000000002,"5min":1.429,"15min":1.429},"last":0.67300000000000004}]},{"osd":7,"last update":"Fri Dec 1 01:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.54000000000000004}]},{"osd":8,"last update":"Fri Dec 1 01:17:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.56299999999999994,"15min":0.53500000000000003},"min":{"1min":0.38300000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59199999999999997,"15min":0.58399999999999996},"min":{"1min":0.41299999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.79500000000000004,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.68799999999999994}]},{"osd":9,"last update":"Fri Dec 1 01:17:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.69799999999999995,"15min":0.70099999999999996},"min":{"1min":0.52000000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.004,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.68700000000000006,"15min":0.68200000000000005},"min":{"1min":0.51700000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94299999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.81599999999999995}]},{"osd":10,"last update":"Fri Dec 1 01:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.66200000000000003,"15min":0.65800000000000003},"min":{"1min":0.55900000000000005,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.88200000000000001,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.86799999999999999,"15min":0.88500000000000001},"min":{"1min":0.65800000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.97899999999999998,"5min":5.4450000000000003,"15min":5.4450000000000003},"last":0.89600000000000002}]},{"osd":11,"last update":"Fri Dec 1 01:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.93600000000000005}]}]},{"osd":3,"up_from":25,"seq":107374182462,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":942080,"data_stored":440410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:17:02 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.34300000000000003,"15min":0.31900000000000001},"min":{"1min":0.26800000000000002,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.51300000000000001,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.35699999999999998,"15min":0.33800000000000002},"min":{"1min":0.26600000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.48299999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.32500000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:17:02 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.53500000000000003,"15min":0.505},"min":{"1min":0.38200000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73099999999999998,"5min":0.996,"15min":0.996},"last":0.443},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.375,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.69599999999999995,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.47199999999999998}]},{"osd":2,"last update":"Fri Dec 1 01:17:02 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.51700000000000002},"min":{"1min":0.39700000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.749,"5min":0.88,"15min":0.88},"last":0.495},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55100000000000005,"15min":0.52800000000000002},"min":{"1min":0.41599999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.78600000000000003,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.53200000000000003}]},{"osd":4,"last update":"Fri Dec 1 01:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58499999999999996,"15min":0.55900000000000005},"min":{"1min":0.40100000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78000000000000003,"5min":1.329,"15min":1.329},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61099999999999999,"15min":0.58899999999999997},"min":{"1min":0.42299999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.74099999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.50700000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:17:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.55200000000000005,"15min":0.55300000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.69699999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81100000000000005,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.42599999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:17:02 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.46500000000000002,"15min":0.47299999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.503,"5min":0.55100000000000005,"15min":0.54500000000000004},"min":{"1min":0.34999999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.69899999999999995,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.40200000000000002}]},{"osd":7,"last update":"Fri Dec 1 01:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.63200000000000001,"15min":0.63},"min":{"1min":0.46100000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.84699999999999998,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.65100000000000002,"15min":0.64900000000000002},"min":{"1min":0.439,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83999999999999997,"5min":1.446,"15min":1.446},"last":0.45200000000000001}]},{"osd":8,"last update":"Fri Dec 1 01:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.51200000000000001}]},{"osd":9,"last update":"Fri Dec 1 01:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.71199999999999997,"15min":0.73399999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77300000000000002,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.63600000000000001,"15min":0.65100000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.75,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.54000000000000004}]},{"osd":10,"last update":"Fri Dec 1 01:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.70799999999999996,"15min":0.71399999999999997},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.77800000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.73299999999999998,"15min":0.745},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.90700000000000003,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.52200000000000002}]},{"osd":11,"last update":"Fri Dec 1 01:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.56200000000000006}]}]},{"osd":4,"up_from":32,"seq":137438953530,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":440386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.54300000000000004,"15min":0.50900000000000001},"min":{"1min":0.48799999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.373},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.53400000000000003,"15min":0.499},"min":{"1min":0.47199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.497}]},{"osd":1,"last update":"Fri Dec 1 01:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.41599999999999998,"15min":0.38200000000000001},"min":{"1min":0.28999999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.58199999999999996,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.41099999999999998,"15min":0.375},"min":{"1min":0.32600000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.54300000000000004,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.56299999999999994}]},{"osd":2,"last update":"Fri Dec 1 01:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.59099999999999997,"15min":0.54100000000000004},"min":{"1min":0.40999999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.60999999999999999,"15min":0.57299999999999995},"min":{"1min":0.40600000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.81000000000000005,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.63400000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.56100000000000005,"15min":0.50900000000000001},"min":{"1min":0.42999999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.52400000000000002,"15min":0.505},"min":{"1min":0.36099999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.751,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.44500000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.59499999999999997,"15min":0.56499999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.67400000000000004,"15min":0.61599999999999999},"min":{"1min":0.46000000000000002,"5min":0.377,"15min":0.377},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.69399999999999995}]},{"osd":6,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.629,"15min":0.61199999999999999},"min":{"1min":0.42999999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.94699999999999995,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.66000000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.51600000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57899999999999996,"15min":0.55700000000000005},"min":{"1min":0.38,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.73999999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.58199999999999996,"15min":0.55500000000000005},"min":{"1min":0.438,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.872,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.70699999999999996}]},{"osd":8,"last update":"Fri Dec 1 01:16:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.63100000000000001,"15min":0.63600000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.498},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76000000000000001,"15min":0.75900000000000001},"min":{"1min":0.47699999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.59199999999999997}]},{"osd":9,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.72999999999999998,"15min":0.73499999999999999},"min":{"1min":0.46600000000000003,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.79000000000000004,"15min":0.80000000000000004},"min":{"1min":0.51600000000000001,"5min":0.503,"15min":0.503},"max":{"1min":1.0900000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.66300000000000003}]},{"osd":10,"last update":"Fri Dec 1 01:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.78700000000000003,"15min":0.79900000000000004},"min":{"1min":0.496,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.93700000000000006,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.74199999999999999,"15min":0.74399999999999999},"min":{"1min":0.46999999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90900000000000003,"5min":0.997,"15min":0.997},"last":0.75600000000000001}]},{"osd":11,"last update":"Fri Dec 1 01:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.61299999999999999}]}]},{"osd":5,"up_from":37,"seq":158913790005,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28460,"kb_used_data":1508,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716692,"statfs":{"total":95995035648,"available":95965892608,"internally_reserved":0,"allocated":1544192,"data_stored":1033059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.51200000000000001},"min":{"1min":0.36199999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.54000000000000004,"15min":0.52800000000000002},"min":{"1min":0.34999999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":2.5099999999999998,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":0.61099999999999999}]},{"osd":1,"last update":"Fri Dec 1 01:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.53600000000000003,"15min":0.495},"min":{"1min":0.41999999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.55500000000000005,"15min":0.50900000000000001},"min":{"1min":0.442,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80700000000000005,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.51700000000000002}]},{"osd":2,"last update":"Fri Dec 1 01:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.40000000000000002,"15min":0.38800000000000001},"min":{"1min":0.27100000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.39900000000000002,"15min":0.379},"min":{"1min":0.28499999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.40000000000000002}]},{"osd":3,"last update":"Fri Dec 1 01:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.56999999999999995,"15min":0.54100000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":2.6379999999999999,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.56399999999999995,"15min":0.53700000000000003},"min":{"1min":0.40899999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.825,"5min":1.825,"15min":1.825},"last":0.53200000000000003}]},{"osd":4,"last update":"Fri Dec 1 01:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.57499999999999996,"15min":0.53800000000000003},"min":{"1min":0.35099999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56899999999999995,"15min":0.53500000000000003},"min":{"1min":0.41299999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.56999999999999995}]},{"osd":6,"last update":"Fri Dec 1 01:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.64200000000000002,"15min":0.626},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":2.391,"5min":2.391,"15min":2.391},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.66000000000000003,"15min":0.63},"min":{"1min":0.42899999999999999,"5min":0.37,"15min":0.37},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":0.75600000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.66100000000000003,"15min":0.65500000000000003},"min":{"1min":0.55300000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.62,"15min":0.61399999999999999},"min":{"1min":0.48499999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.84599999999999997,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.58099999999999996}]},{"osd":8,"last update":"Fri Dec 1 01:16:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58099999999999996,"15min":0.57999999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.754,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.48999999999999999}]},{"osd":9,"last update":"Fri Dec 1 01:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.70999999999999996,"15min":0.70199999999999996},"min":{"1min":0.47299999999999998,"5min":0.443,"15min":0.443},"max":{"1min":2.8069999999999999,"5min":2.8069999999999999,"15min":2.8069999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.71799999999999997,"15min":0.70199999999999996},"min":{"1min":0.48199999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":3.9049999999999998,"5min":3.9049999999999998,"15min":3.9049999999999998},"last":0.70399999999999996}]},{"osd":10,"last update":"Fri Dec 1 01:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.68400000000000005,"15min":0.67100000000000004},"min":{"1min":0.54300000000000004,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.68600000000000005,"15min":0.68000000000000005},"min":{"1min":0.57699999999999996,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.85499999999999998,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.74099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":61367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T01:17:48.704 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T01:17:48.862 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.858+0000 7fc7982cb700 1 -- 172.21.15.77:0/756789721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc790108ef0 msgr2=0x7fc79010f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:48.862 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.858+0000 7fc7982cb700 1 --2- 172.21.15.77:0/756789721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc790108ef0 0x7fc79010f7a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc7880099c0 tx=0x7fc788009cd0 comp rx=0 tx=0).stop 2023-12-01T01:17:48.862 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.858+0000 7fc7982cb700 1 -- 172.21.15.77:0/756789721 shutdown_connections 2023-12-01T01:17:48.862 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.858+0000 7fc7982cb700 1 --2- 172.21.15.77:0/756789721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc790108ef0 0x7fc79010f7a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:48.862 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.858+0000 7fc7982cb700 1 --2- 172.21.15.77:0/756789721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc790107360 0x7fc790107780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:48.862 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.858+0000 7fc7982cb700 1 --2- 172.21.15.77:0/756789721 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc790108550 0x7fc7901089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:48.862 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.858+0000 7fc7982cb700 1 -- 172.21.15.77:0/756789721 >> 172.21.15.77:0/756789721 conn(0x7fc7900762e0 msgr2=0x7fc790078740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:48.862 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.858+0000 7fc7982cb700 1 -- 172.21.15.77:0/756789721 shutdown_connections 2023-12-01T01:17:48.862 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.858+0000 7fc7982cb700 1 -- 172.21.15.77:0/756789721 wait complete. 2023-12-01T01:17:48.862 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.858+0000 7fc7982cb700 1 Processor -- start 2023-12-01T01:17:48.863 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.858+0000 7fc7982cb700 1 -- start start 2023-12-01T01:17:48.863 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.858+0000 7fc7982cb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc790107360 0x7fc79019cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:48.863 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.858+0000 7fc7982cb700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc790108550 0x7fc79019d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:48.863 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.858+0000 7fc7982cb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc790108ef0 0x7fc7901a4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:48.863 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.858+0000 7fc7982cb700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7901a4ad0 con 0x7fc790108ef0 2023-12-01T01:17:48.863 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.858+0000 7fc7982cb700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7901a4c10 con 0x7fc790107360 2023-12-01T01:17:48.863 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.858+0000 7fc7982cb700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7901a4da0 con 0x7fc790108550 2023-12-01T01:17:48.863 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.859+0000 7fc796868700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc790108ef0 0x7fc7901a4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:48.863 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.859+0000 7fc796868700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc790108ef0 0x7fc7901a4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59148/0 (socket says 172.21.15.77:59148) 2023-12-01T01:17:48.863 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.859+0000 7fc796868700 1 -- 172.21.15.77:0/3084779465 learned_addr learned my addr 172.21.15.77:0/3084779465 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:48.863 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.859+0000 7fc796067700 1 --2- 172.21.15.77:0/3084779465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc790107360 0x7fc79019cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:48.863 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.859+0000 7fc795866700 1 --2- 172.21.15.77:0/3084779465 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc790108550 0x7fc79019d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:48.863 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.859+0000 7fc796868700 1 -- 172.21.15.77:0/3084779465 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc790108550 msgr2=0x7fc79019d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:48.864 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.859+0000 7fc796868700 1 --2- 172.21.15.77:0/3084779465 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc790108550 0x7fc79019d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:48.864 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.859+0000 7fc796868700 1 -- 172.21.15.77:0/3084779465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc790107360 msgr2=0x7fc79019cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:48.864 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.859+0000 7fc796868700 1 --2- 172.21.15.77:0/3084779465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc790107360 0x7fc79019cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:48.864 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.859+0000 7fc796868700 1 -- 172.21.15.77:0/3084779465 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7880095f0 con 0x7fc790108ef0 2023-12-01T01:17:48.864 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.859+0000 7fc796868700 1 --2- 172.21.15.77:0/3084779465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc790108ef0 0x7fc7901a4500 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc7880098e0 tx=0x7fc78800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:48.864 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.859+0000 7fc7877fe700 1 -- 172.21.15.77:0/3084779465 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78801d070 con 0x7fc790108ef0 2023-12-01T01:17:48.864 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.859+0000 7fc7877fe700 1 -- 172.21.15.77:0/3084779465 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc788005030 con 0x7fc790108ef0 2023-12-01T01:17:48.865 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.859+0000 7fc7982cb700 1 -- 172.21.15.77:0/3084779465 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7901a5020 con 0x7fc790108ef0 2023-12-01T01:17:48.865 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.859+0000 7fc7877fe700 1 -- 172.21.15.77:0/3084779465 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc788004aa0 con 0x7fc790108ef0 2023-12-01T01:17:48.865 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.859+0000 7fc7982cb700 1 -- 172.21.15.77:0/3084779465 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7901a5440 con 0x7fc790108ef0 2023-12-01T01:17:48.865 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.860+0000 7fc7982cb700 1 -- 172.21.15.77:0/3084779465 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc790112bc0 con 0x7fc790108ef0 2023-12-01T01:17:48.865 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.861+0000 7fc7877fe700 1 -- 172.21.15.77:0/3084779465 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fc788005480 con 0x7fc790108ef0 2023-12-01T01:17:48.866 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.862+0000 7fc7877fe700 1 --2- 172.21.15.77:0/3084779465 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc77c0a05f0 0x7fc77c0a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:48.866 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.862+0000 7fc796067700 1 --2- 172.21.15.77:0/3084779465 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc77c0a05f0 0x7fc77c0a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:48.866 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.862+0000 7fc7877fe700 1 -- 172.21.15.77:0/3084779465 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fc7880ccc20 con 0x7fc790108ef0 2023-12-01T01:17:48.866 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.862+0000 7fc796067700 1 --2- 172.21.15.77:0/3084779465 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc77c0a05f0 0x7fc77c0a2ab0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc7901083b0 tx=0x7fc78c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:48.867 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:48.865+0000 7fc7877fe700 1 -- 172.21.15.77:0/3084779465 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc7880967f0 con 0x7fc790108ef0 2023-12-01T01:17:48.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:48 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/4068514129' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-01T01:17:48.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:48 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/1820936573' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-01T01:17:48.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:48 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/3512934800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:17:49.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:48 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/4068514129' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-01T01:17:49.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:48 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/1820936573' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-01T01:17:49.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:48 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/3512934800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:17:49.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:48 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/4068514129' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-01T01:17:49.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:48 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/1820936573' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-01T01:17:49.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:48 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/3512934800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:17:49.056 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.055+0000 7fc7982cb700 1 -- 172.21.15.77:0/3084779465 --> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc7900611d0 con 0x7fc77c0a05f0 2023-12-01T01:17:49.066 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.065+0000 7fc7877fe700 1 -- 172.21.15.77:0/3084779465 <== mgr.14152 v2:172.21.15.77:6800/3838941679 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344189 (secure 0 0 0) 0x7fc7900611d0 con 0x7fc77c0a05f0 2023-12-01T01:17:49.070 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:17:49.073 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.069+0000 7fc7982cb700 1 -- 172.21.15.77:0/3084779465 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc77c0a05f0 msgr2=0x7fc77c0a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:49.073 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.069+0000 7fc7982cb700 1 --2- 172.21.15.77:0/3084779465 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc77c0a05f0 0x7fc77c0a2ab0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc7901083b0 tx=0x7fc78c00b040 comp rx=0 tx=0).stop 2023-12-01T01:17:49.073 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.069+0000 7fc7982cb700 1 -- 172.21.15.77:0/3084779465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc790108ef0 msgr2=0x7fc7901a4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:49.073 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.069+0000 7fc7982cb700 1 --2- 172.21.15.77:0/3084779465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc790108ef0 0x7fc7901a4500 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc7880098e0 tx=0x7fc78800f690 comp rx=0 tx=0).stop 2023-12-01T01:17:49.073 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.069+0000 7fc7982cb700 1 -- 172.21.15.77:0/3084779465 shutdown_connections 2023-12-01T01:17:49.073 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.069+0000 7fc7982cb700 1 --2- 172.21.15.77:0/3084779465 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc77c0a05f0 0x7fc77c0a2ab0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:49.073 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.069+0000 7fc7982cb700 1 --2- 172.21.15.77:0/3084779465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc790108ef0 0x7fc7901a4500 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:49.074 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.069+0000 7fc7982cb700 1 --2- 172.21.15.77:0/3084779465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc790107360 0x7fc79019cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:49.074 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.069+0000 7fc7982cb700 1 --2- 172.21.15.77:0/3084779465 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc790108550 0x7fc79019d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:49.074 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.069+0000 7fc7982cb700 1 -- 172.21.15.77:0/3084779465 >> 172.21.15.77:0/3084779465 conn(0x7fc7900762e0 msgr2=0x7fc790078490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:49.074 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.069+0000 7fc7982cb700 1 -- 172.21.15.77:0/3084779465 shutdown_connections 2023-12-01T01:17:49.074 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.069+0000 7fc7982cb700 1 -- 172.21.15.77:0/3084779465 wait complete. 2023-12-01T01:17:49.074 INFO:teuthology.orchestra.run.smithi077.stderr:dumped all 2023-12-01T01:17:49.083 INFO:teuthology.orchestra.run.smithi077.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2023-12-01T01:17:48.286446+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656403,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":907,"num_read_kb":577,"num_write":416,"num_write_kb":3070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1180916,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336340,"kb_used_data":12916,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124605484,"statfs":{"total":1151940427776,"available":1151596015616,"internally_reserved":0,"allocated":13225984,"data_stored":7244401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":636,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":131,"num_read_kb":76,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003165"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.612614+0000","last_change":"2023-12-01T01:16:25.373817+0000","last_active":"2023-12-01T01:17:24.612614+0000","last_peered":"2023-12-01T01:17:24.612614+0000","last_clean":"2023-12-01T01:17:24.612614+0000","last_became_active":"2023-12-01T01:16:25.373154+0000","last_became_peered":"2023-12-01T01:16:25.373154+0000","last_unstale":"2023-12-01T01:17:24.612614+0000","last_undegraded":"2023-12-01T01:17:24.612614+0000","last_fullsized":"2023-12-01T01:17:24.612614+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:33:43.284106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.615673+0000","last_change":"2023-12-01T01:16:25.364855+0000","last_active":"2023-12-01T01:17:24.615673+0000","last_peered":"2023-12-01T01:17:24.615673+0000","last_clean":"2023-12-01T01:17:24.615673+0000","last_became_active":"2023-12-01T01:16:25.364536+0000","last_became_peered":"2023-12-01T01:16:25.364536+0000","last_unstale":"2023-12-01T01:17:24.615673+0000","last_undegraded":"2023-12-01T01:17:24.615673+0000","last_fullsized":"2023-12-01T01:17:24.615673+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:45:23.471557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.615287+0000","last_change":"2023-12-01T01:16:25.372443+0000","last_active":"2023-12-01T01:17:24.615287+0000","last_peered":"2023-12-01T01:17:24.615287+0000","last_clean":"2023-12-01T01:17:24.615287+0000","last_became_active":"2023-12-01T01:16:25.372133+0000","last_became_peered":"2023-12-01T01:16:25.372133+0000","last_unstale":"2023-12-01T01:17:24.615287+0000","last_undegraded":"2023-12-01T01:17:24.615287+0000","last_fullsized":"2023-12-01T01:17:24.615287+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:54:19.439408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-01T01:17:25.621327+0000","last_change":"2023-12-01T01:17:25.619573+0000","last_active":"2023-12-01T01:17:25.621327+0000","last_peered":"2023-12-01T01:17:25.621327+0000","last_clean":"2023-12-01T01:17:25.621327+0000","last_became_active":"2023-12-01T01:17:25.619265+0000","last_became_peered":"2023-12-01T01:17:25.619265+0000","last_unstale":"2023-12-01T01:17:25.621327+0000","last_undegraded":"2023-12-01T01:17:25.621327+0000","last_fullsized":"2023-12-01T01:17:25.621327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:03:13.174659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.615064+0000","last_change":"2023-12-01T01:16:25.364081+0000","last_active":"2023-12-01T01:17:24.615064+0000","last_peered":"2023-12-01T01:17:24.615064+0000","last_clean":"2023-12-01T01:17:24.615064+0000","last_became_active":"2023-12-01T01:16:25.363187+0000","last_became_peered":"2023-12-01T01:16:25.363187+0000","last_unstale":"2023-12-01T01:17:24.615064+0000","last_undegraded":"2023-12-01T01:17:24.615064+0000","last_fullsized":"2023-12-01T01:17:24.615064+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:05:42.852128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.619241+0000","last_change":"2023-12-01T01:16:25.366104+0000","last_active":"2023-12-01T01:17:24.619241+0000","last_peered":"2023-12-01T01:17:24.619241+0000","last_clean":"2023-12-01T01:17:24.619241+0000","last_became_active":"2023-12-01T01:16:25.365778+0000","last_became_peered":"2023-12-01T01:16:25.365778+0000","last_unstale":"2023-12-01T01:17:24.619241+0000","last_undegraded":"2023-12-01T01:17:24.619241+0000","last_fullsized":"2023-12-01T01:17:24.619241+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:30:09.485609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.613507+0000","last_change":"2023-12-01T01:16:25.371354+0000","last_active":"2023-12-01T01:17:24.613507+0000","last_peered":"2023-12-01T01:17:24.613507+0000","last_clean":"2023-12-01T01:17:24.613507+0000","last_became_active":"2023-12-01T01:16:25.370955+0000","last_became_peered":"2023-12-01T01:16:25.370955+0000","last_unstale":"2023-12-01T01:17:24.613507+0000","last_undegraded":"2023-12-01T01:17:24.613507+0000","last_fullsized":"2023-12-01T01:17:24.613507+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:38:53.508285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617357+0000","last_change":"2023-12-01T01:16:25.368187+0000","last_active":"2023-12-01T01:17:24.617357+0000","last_peered":"2023-12-01T01:17:24.617357+0000","last_clean":"2023-12-01T01:17:24.617357+0000","last_became_active":"2023-12-01T01:16:25.367854+0000","last_became_peered":"2023-12-01T01:16:25.367854+0000","last_unstale":"2023-12-01T01:17:24.617357+0000","last_undegraded":"2023-12-01T01:17:24.617357+0000","last_fullsized":"2023-12-01T01:17:24.617357+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:22:35.159185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.802923+0000","last_change":"2023-12-01T01:16:25.372829+0000","last_active":"2023-12-01T01:17:24.802923+0000","last_peered":"2023-12-01T01:17:24.802923+0000","last_clean":"2023-12-01T01:17:24.802923+0000","last_became_active":"2023-12-01T01:16:25.372469+0000","last_became_peered":"2023-12-01T01:16:25.372469+0000","last_unstale":"2023-12-01T01:17:24.802923+0000","last_undegraded":"2023-12-01T01:17:24.802923+0000","last_fullsized":"2023-12-01T01:17:24.802923+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:25:10.300138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-01T01:17:25.618881+0000","last_change":"2023-12-01T01:17:25.618881+0000","last_active":"2023-12-01T01:17:25.618881+0000","last_peered":"2023-12-01T01:17:25.618881+0000","last_clean":"2023-12-01T01:17:25.618881+0000","last_became_active":"2023-12-01T01:17:25.618457+0000","last_became_peered":"2023-12-01T01:17:25.618457+0000","last_unstale":"2023-12-01T01:17:25.618881+0000","last_undegraded":"2023-12-01T01:17:25.618881+0000","last_fullsized":"2023-12-01T01:17:25.618881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:44:13.172660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.613960+0000","last_change":"2023-12-01T01:16:25.367602+0000","last_active":"2023-12-01T01:17:24.613960+0000","last_peered":"2023-12-01T01:17:24.613960+0000","last_clean":"2023-12-01T01:17:24.613960+0000","last_became_active":"2023-12-01T01:16:25.366565+0000","last_became_peered":"2023-12-01T01:16:25.366565+0000","last_unstale":"2023-12-01T01:17:24.613960+0000","last_undegraded":"2023-12-01T01:17:24.613960+0000","last_fullsized":"2023-12-01T01:17:24.613960+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:28:52.576838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.616474+0000","last_change":"2023-12-01T01:16:25.375004+0000","last_active":"2023-12-01T01:17:24.616474+0000","last_peered":"2023-12-01T01:17:24.616474+0000","last_clean":"2023-12-01T01:17:24.616474+0000","last_became_active":"2023-12-01T01:16:25.374320+0000","last_became_peered":"2023-12-01T01:16:25.374320+0000","last_unstale":"2023-12-01T01:17:24.616474+0000","last_undegraded":"2023-12-01T01:17:24.616474+0000","last_fullsized":"2023-12-01T01:17:24.616474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:40:49.482061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617481+0000","last_change":"2023-12-01T01:16:25.367858+0000","last_active":"2023-12-01T01:17:24.617481+0000","last_peered":"2023-12-01T01:17:24.617481+0000","last_clean":"2023-12-01T01:17:24.617481+0000","last_became_active":"2023-12-01T01:16:25.367560+0000","last_became_peered":"2023-12-01T01:16:25.367560+0000","last_unstale":"2023-12-01T01:17:24.617481+0000","last_undegraded":"2023-12-01T01:17:24.617481+0000","last_fullsized":"2023-12-01T01:17:24.617481+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:33:21.048672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.615018+0000","last_change":"2023-12-01T01:16:25.375654+0000","last_active":"2023-12-01T01:17:24.615018+0000","last_peered":"2023-12-01T01:17:24.615018+0000","last_clean":"2023-12-01T01:17:24.615018+0000","last_became_active":"2023-12-01T01:16:25.375379+0000","last_became_peered":"2023-12-01T01:16:25.375379+0000","last_unstale":"2023-12-01T01:17:24.615018+0000","last_undegraded":"2023-12-01T01:17:24.615018+0000","last_fullsized":"2023-12-01T01:17:24.615018+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:33:42.335365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.614888+0000","last_change":"2023-12-01T01:16:25.372546+0000","last_active":"2023-12-01T01:17:24.614888+0000","last_peered":"2023-12-01T01:17:24.614888+0000","last_clean":"2023-12-01T01:17:24.614888+0000","last_became_active":"2023-12-01T01:16:25.372327+0000","last_became_peered":"2023-12-01T01:16:25.372327+0000","last_unstale":"2023-12-01T01:17:24.614888+0000","last_undegraded":"2023-12-01T01:17:24.614888+0000","last_fullsized":"2023-12-01T01:17:24.614888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:12:34.171834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.802497+0000","last_change":"2023-12-01T01:16:25.366449+0000","last_active":"2023-12-01T01:17:24.802497+0000","last_peered":"2023-12-01T01:17:24.802497+0000","last_clean":"2023-12-01T01:17:24.802497+0000","last_became_active":"2023-12-01T01:16:25.366147+0000","last_became_peered":"2023-12-01T01:16:25.366147+0000","last_unstale":"2023-12-01T01:17:24.802497+0000","last_undegraded":"2023-12-01T01:17:24.802497+0000","last_fullsized":"2023-12-01T01:17:24.802497+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:01:29.670329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.612762+0000","last_change":"2023-12-01T01:16:25.374016+0000","last_active":"2023-12-01T01:17:24.612762+0000","last_peered":"2023-12-01T01:17:24.612762+0000","last_clean":"2023-12-01T01:17:24.612762+0000","last_became_active":"2023-12-01T01:16:25.373543+0000","last_became_peered":"2023-12-01T01:16:25.373543+0000","last_unstale":"2023-12-01T01:17:24.612762+0000","last_undegraded":"2023-12-01T01:17:24.612762+0000","last_fullsized":"2023-12-01T01:17:24.612762+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:36:22.858446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.612411+0000","last_change":"2023-12-01T01:16:25.363598+0000","last_active":"2023-12-01T01:17:24.612411+0000","last_peered":"2023-12-01T01:17:24.612411+0000","last_clean":"2023-12-01T01:17:24.612411+0000","last_became_active":"2023-12-01T01:16:25.363111+0000","last_became_peered":"2023-12-01T01:16:25.363111+0000","last_unstale":"2023-12-01T01:17:24.612411+0000","last_undegraded":"2023-12-01T01:17:24.612411+0000","last_fullsized":"2023-12-01T01:17:24.612411+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:42:08.171789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.619102+0000","last_change":"2023-12-01T01:16:25.368446+0000","last_active":"2023-12-01T01:17:24.619102+0000","last_peered":"2023-12-01T01:17:24.619102+0000","last_clean":"2023-12-01T01:17:24.619102+0000","last_became_active":"2023-12-01T01:16:25.361893+0000","last_became_peered":"2023-12-01T01:16:25.361893+0000","last_unstale":"2023-12-01T01:17:24.619102+0000","last_undegraded":"2023-12-01T01:17:24.619102+0000","last_fullsized":"2023-12-01T01:17:24.619102+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:05:45.697338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'13","reported_seq":102,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-01T01:17:44.717304+0000","last_change":"2023-12-01T01:16:24.934939+0000","last_active":"2023-12-01T01:17:44.717304+0000","last_peered":"2023-12-01T01:17:44.717304+0000","last_clean":"2023-12-01T01:17:44.717304+0000","last_became_active":"2023-12-01T01:16:24.934481+0000","last_became_peered":"2023-12-01T01:16:24.934481+0000","last_unstale":"2023-12-01T01:17:44.717304+0000","last_undegraded":"2023-12-01T01:17:44.717304+0000","last_fullsized":"2023-12-01T01:17:44.717304+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:39:49.804567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":75,"num_read_kb":75,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617562+0000","last_change":"2023-12-01T01:16:25.364061+0000","last_active":"2023-12-01T01:17:24.617562+0000","last_peered":"2023-12-01T01:17:24.617562+0000","last_clean":"2023-12-01T01:17:24.617562+0000","last_became_active":"2023-12-01T01:16:25.363678+0000","last_became_peered":"2023-12-01T01:16:25.363678+0000","last_unstale":"2023-12-01T01:17:24.617562+0000","last_undegraded":"2023-12-01T01:17:24.617562+0000","last_fullsized":"2023-12-01T01:17:24.617562+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:24:07.107604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617794+0000","last_change":"2023-12-01T01:16:24.939338+0000","last_active":"2023-12-01T01:17:24.617794+0000","last_peered":"2023-12-01T01:17:24.617794+0000","last_clean":"2023-12-01T01:17:24.617794+0000","last_became_active":"2023-12-01T01:16:24.939007+0000","last_became_peered":"2023-12-01T01:16:24.939007+0000","last_unstale":"2023-12-01T01:17:24.617794+0000","last_undegraded":"2023-12-01T01:17:24.617794+0000","last_fullsized":"2023-12-01T01:17:24.617794+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:56:50.732188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.615519+0000","last_change":"2023-12-01T01:16:25.362231+0000","last_active":"2023-12-01T01:17:24.615519+0000","last_peered":"2023-12-01T01:17:24.615519+0000","last_clean":"2023-12-01T01:17:24.615519+0000","last_became_active":"2023-12-01T01:16:25.361920+0000","last_became_peered":"2023-12-01T01:16:25.361920+0000","last_unstale":"2023-12-01T01:17:24.615519+0000","last_undegraded":"2023-12-01T01:17:24.615519+0000","last_fullsized":"2023-12-01T01:17:24.615519+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:22:22.104480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'12","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.616167+0000","last_change":"2023-12-01T01:16:24.931801+0000","last_active":"2023-12-01T01:17:24.616167+0000","last_peered":"2023-12-01T01:17:24.616167+0000","last_clean":"2023-12-01T01:17:24.616167+0000","last_became_active":"2023-12-01T01:16:24.931451+0000","last_became_peered":"2023-12-01T01:16:24.931451+0000","last_unstale":"2023-12-01T01:17:24.616167+0000","last_undegraded":"2023-12-01T01:17:24.616167+0000","last_fullsized":"2023-12-01T01:17:24.616167+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:22:41.666502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.802631+0000","last_change":"2023-12-01T01:16:25.361789+0000","last_active":"2023-12-01T01:17:24.802631+0000","last_peered":"2023-12-01T01:17:24.802631+0000","last_clean":"2023-12-01T01:17:24.802631+0000","last_became_active":"2023-12-01T01:16:25.361480+0000","last_became_peered":"2023-12-01T01:16:25.361480+0000","last_unstale":"2023-12-01T01:17:24.802631+0000","last_undegraded":"2023-12-01T01:17:24.802631+0000","last_fullsized":"2023-12-01T01:17:24.802631+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:07:00.742167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'5","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.615819+0000","last_change":"2023-12-01T01:16:24.933362+0000","last_active":"2023-12-01T01:17:24.615819+0000","last_peered":"2023-12-01T01:17:24.615819+0000","last_clean":"2023-12-01T01:17:24.615819+0000","last_became_active":"2023-12-01T01:16:24.933049+0000","last_became_peered":"2023-12-01T01:16:24.933049+0000","last_unstale":"2023-12-01T01:17:24.615819+0000","last_undegraded":"2023-12-01T01:17:24.615819+0000","last_fullsized":"2023-12-01T01:17:24.615819+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:47:37.153344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617087+0000","last_change":"2023-12-01T01:16:25.364419+0000","last_active":"2023-12-01T01:17:24.617087+0000","last_peered":"2023-12-01T01:17:24.617087+0000","last_clean":"2023-12-01T01:17:24.617087+0000","last_became_active":"2023-12-01T01:16:25.363022+0000","last_became_peered":"2023-12-01T01:16:25.363022+0000","last_unstale":"2023-12-01T01:17:24.617087+0000","last_undegraded":"2023-12-01T01:17:24.617087+0000","last_fullsized":"2023-12-01T01:17:24.617087+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:58:45.660625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'6","reported_seq":95,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-01T01:17:42.703276+0000","last_change":"2023-12-01T01:16:24.935947+0000","last_active":"2023-12-01T01:17:42.703276+0000","last_peered":"2023-12-01T01:17:42.703276+0000","last_clean":"2023-12-01T01:17:42.703276+0000","last_became_active":"2023-12-01T01:16:24.935061+0000","last_became_peered":"2023-12-01T01:16:24.935061+0000","last_unstale":"2023-12-01T01:17:42.703276+0000","last_undegraded":"2023-12-01T01:17:42.703276+0000","last_fullsized":"2023-12-01T01:17:42.703276+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:00:31.551362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":967,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.615303+0000","last_change":"2023-12-01T01:16:25.363229+0000","last_active":"2023-12-01T01:17:24.615303+0000","last_peered":"2023-12-01T01:17:24.615303+0000","last_clean":"2023-12-01T01:17:24.615303+0000","last_became_active":"2023-12-01T01:16:25.362770+0000","last_became_peered":"2023-12-01T01:16:25.362770+0000","last_unstale":"2023-12-01T01:17:24.615303+0000","last_undegraded":"2023-12-01T01:17:24.615303+0000","last_fullsized":"2023-12-01T01:17:24.615303+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:45:02.782201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.619079+0000","last_change":"2023-12-01T01:16:24.930509+0000","last_active":"2023-12-01T01:17:24.619079+0000","last_peered":"2023-12-01T01:17:24.619079+0000","last_clean":"2023-12-01T01:17:24.619079+0000","last_became_active":"2023-12-01T01:16:24.930188+0000","last_became_peered":"2023-12-01T01:16:24.930188+0000","last_unstale":"2023-12-01T01:17:24.619079+0000","last_undegraded":"2023-12-01T01:17:24.619079+0000","last_fullsized":"2023-12-01T01:17:24.619079+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:04:17.748363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617184+0000","last_change":"2023-12-01T01:16:25.363383+0000","last_active":"2023-12-01T01:17:24.617184+0000","last_peered":"2023-12-01T01:17:24.617184+0000","last_clean":"2023-12-01T01:17:24.617184+0000","last_became_active":"2023-12-01T01:16:25.363085+0000","last_became_peered":"2023-12-01T01:16:25.363085+0000","last_unstale":"2023-12-01T01:17:24.617184+0000","last_undegraded":"2023-12-01T01:17:24.617184+0000","last_fullsized":"2023-12-01T01:17:24.617184+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:34:36.921706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"82'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617436+0000","last_change":"2023-12-01T01:16:24.929081+0000","last_active":"2023-12-01T01:17:24.617436+0000","last_peered":"2023-12-01T01:17:24.617436+0000","last_clean":"2023-12-01T01:17:24.617436+0000","last_became_active":"2023-12-01T01:16:24.928747+0000","last_became_peered":"2023-12-01T01:16:24.928747+0000","last_unstale":"2023-12-01T01:17:24.617436+0000","last_undegraded":"2023-12-01T01:17:24.617436+0000","last_fullsized":"2023-12-01T01:17:24.617436+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:02:24.727835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617440+0000","last_change":"2023-12-01T01:16:25.360959+0000","last_active":"2023-12-01T01:17:24.617440+0000","last_peered":"2023-12-01T01:17:24.617440+0000","last_clean":"2023-12-01T01:17:24.617440+0000","last_became_active":"2023-12-01T01:16:25.360647+0000","last_became_peered":"2023-12-01T01:16:25.360647+0000","last_unstale":"2023-12-01T01:17:24.617440+0000","last_undegraded":"2023-12-01T01:17:24.617440+0000","last_fullsized":"2023-12-01T01:17:24.617440+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:50:24.151273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.616895+0000","last_change":"2023-12-01T01:16:24.938322+0000","last_active":"2023-12-01T01:17:24.616895+0000","last_peered":"2023-12-01T01:17:24.616895+0000","last_clean":"2023-12-01T01:17:24.616895+0000","last_became_active":"2023-12-01T01:16:24.937747+0000","last_became_peered":"2023-12-01T01:16:24.937747+0000","last_unstale":"2023-12-01T01:17:24.616895+0000","last_undegraded":"2023-12-01T01:17:24.616895+0000","last_fullsized":"2023-12-01T01:17:24.616895+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:37:59.999748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.611759+0000","last_change":"2023-12-01T01:16:25.362779+0000","last_active":"2023-12-01T01:17:24.611759+0000","last_peered":"2023-12-01T01:17:24.611759+0000","last_clean":"2023-12-01T01:17:24.611759+0000","last_became_active":"2023-12-01T01:16:25.362487+0000","last_became_peered":"2023-12-01T01:16:25.362487+0000","last_unstale":"2023-12-01T01:17:24.611759+0000","last_undegraded":"2023-12-01T01:17:24.611759+0000","last_fullsized":"2023-12-01T01:17:24.611759+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:04:42.409498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'5","reported_seq":171,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-01T01:17:46.708105+0000","last_change":"2023-12-01T01:16:24.933187+0000","last_active":"2023-12-01T01:17:46.708105+0000","last_peered":"2023-12-01T01:17:46.708105+0000","last_clean":"2023-12-01T01:17:46.708105+0000","last_became_active":"2023-12-01T01:16:24.932819+0000","last_became_peered":"2023-12-01T01:16:24.932819+0000","last_unstale":"2023-12-01T01:17:46.708105+0000","last_undegraded":"2023-12-01T01:17:46.708105+0000","last_fullsized":"2023-12-01T01:17:46.708105+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:54:34.739815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":152,"num_read_kb":152,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.802822+0000","last_change":"2023-12-01T01:16:25.361183+0000","last_active":"2023-12-01T01:17:24.802822+0000","last_peered":"2023-12-01T01:17:24.802822+0000","last_clean":"2023-12-01T01:17:24.802822+0000","last_became_active":"2023-12-01T01:16:25.360871+0000","last_became_peered":"2023-12-01T01:16:25.360871+0000","last_unstale":"2023-12-01T01:17:24.802822+0000","last_undegraded":"2023-12-01T01:17:24.802822+0000","last_fullsized":"2023-12-01T01:17:24.802822+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:06:04.748159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.613828+0000","last_change":"2023-12-01T01:16:24.939355+0000","last_active":"2023-12-01T01:17:24.613828+0000","last_peered":"2023-12-01T01:17:24.613828+0000","last_clean":"2023-12-01T01:17:24.613828+0000","last_became_active":"2023-12-01T01:16:24.938967+0000","last_became_peered":"2023-12-01T01:16:24.938967+0000","last_unstale":"2023-12-01T01:17:24.613828+0000","last_undegraded":"2023-12-01T01:17:24.613828+0000","last_fullsized":"2023-12-01T01:17:24.613828+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:20:51.172753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.615582+0000","last_change":"2023-12-01T01:16:25.366752+0000","last_active":"2023-12-01T01:17:24.615582+0000","last_peered":"2023-12-01T01:17:24.615582+0000","last_clean":"2023-12-01T01:17:24.615582+0000","last_became_active":"2023-12-01T01:16:25.366466+0000","last_became_peered":"2023-12-01T01:16:25.366466+0000","last_unstale":"2023-12-01T01:17:24.615582+0000","last_undegraded":"2023-12-01T01:17:24.615582+0000","last_fullsized":"2023-12-01T01:17:24.615582+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:09:53.716127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617178+0000","last_change":"2023-12-01T01:16:24.940827+0000","last_active":"2023-12-01T01:17:24.617178+0000","last_peered":"2023-12-01T01:17:24.617178+0000","last_clean":"2023-12-01T01:17:24.617178+0000","last_became_active":"2023-12-01T01:16:24.940515+0000","last_became_peered":"2023-12-01T01:16:24.940515+0000","last_unstale":"2023-12-01T01:17:24.617178+0000","last_undegraded":"2023-12-01T01:17:24.617178+0000","last_fullsized":"2023-12-01T01:17:24.617178+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:08:26.409014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.802749+0000","last_change":"2023-12-01T01:16:25.369662+0000","last_active":"2023-12-01T01:17:24.802749+0000","last_peered":"2023-12-01T01:17:24.802749+0000","last_clean":"2023-12-01T01:17:24.802749+0000","last_became_active":"2023-12-01T01:16:25.369178+0000","last_became_peered":"2023-12-01T01:16:25.369178+0000","last_unstale":"2023-12-01T01:17:24.802749+0000","last_undegraded":"2023-12-01T01:17:24.802749+0000","last_fullsized":"2023-12-01T01:17:24.802749+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:40:39.919414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'31","reported_seq":205,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-01T01:17:46.708885+0000","last_change":"2023-12-01T01:16:24.936733+0000","last_active":"2023-12-01T01:17:46.708885+0000","last_peered":"2023-12-01T01:17:46.708885+0000","last_clean":"2023-12-01T01:17:46.708885+0000","last_became_active":"2023-12-01T01:16:24.936459+0000","last_became_peered":"2023-12-01T01:16:24.936459+0000","last_unstale":"2023-12-01T01:17:46.708885+0000","last_undegraded":"2023-12-01T01:17:46.708885+0000","last_fullsized":"2023-12-01T01:17:46.708885+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:19:58.387650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58848,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":160,"num_read_kb":137,"num_write":33,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.616046+0000","last_change":"2023-12-01T01:16:25.364292+0000","last_active":"2023-12-01T01:17:24.616046+0000","last_peered":"2023-12-01T01:17:24.616046+0000","last_clean":"2023-12-01T01:17:24.616046+0000","last_became_active":"2023-12-01T01:16:25.362615+0000","last_became_peered":"2023-12-01T01:16:25.362615+0000","last_unstale":"2023-12-01T01:17:24.616046+0000","last_undegraded":"2023-12-01T01:17:24.616046+0000","last_fullsized":"2023-12-01T01:17:24.616046+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:06:07.350607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'9","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-01T01:17:25.632990+0000","last_change":"2023-12-01T01:17:25.632990+0000","last_active":"2023-12-01T01:17:25.632990+0000","last_peered":"2023-12-01T01:17:25.632990+0000","last_clean":"2023-12-01T01:17:25.632990+0000","last_became_active":"2023-12-01T01:17:25.621584+0000","last_became_peered":"2023-12-01T01:17:25.621584+0000","last_unstale":"2023-12-01T01:17:25.632990+0000","last_undegraded":"2023-12-01T01:17:25.632990+0000","last_fullsized":"2023-12-01T01:17:25.632990+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:33:26.502407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617099+0000","last_change":"2023-12-01T01:16:25.360104+0000","last_active":"2023-12-01T01:17:24.617099+0000","last_peered":"2023-12-01T01:17:24.617099+0000","last_clean":"2023-12-01T01:17:24.617099+0000","last_became_active":"2023-12-01T01:16:25.359807+0000","last_became_peered":"2023-12-01T01:16:25.359807+0000","last_unstale":"2023-12-01T01:17:24.617099+0000","last_undegraded":"2023-12-01T01:17:24.617099+0000","last_fullsized":"2023-12-01T01:17:24.617099+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:04:05.100258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'5","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617006+0000","last_change":"2023-12-01T01:16:24.945074+0000","last_active":"2023-12-01T01:17:24.617006+0000","last_peered":"2023-12-01T01:17:24.617006+0000","last_clean":"2023-12-01T01:17:24.617006+0000","last_became_active":"2023-12-01T01:16:24.944737+0000","last_became_peered":"2023-12-01T01:16:24.944737+0000","last_unstale":"2023-12-01T01:17:24.617006+0000","last_undegraded":"2023-12-01T01:17:24.617006+0000","last_fullsized":"2023-12-01T01:17:24.617006+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:44:53.931454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-01T01:17:25.617181+0000","last_change":"2023-12-01T01:17:25.617181+0000","last_active":"2023-12-01T01:17:25.617181+0000","last_peered":"2023-12-01T01:17:25.617181+0000","last_clean":"2023-12-01T01:17:25.617181+0000","last_became_active":"2023-12-01T01:17:25.616647+0000","last_became_peered":"2023-12-01T01:17:25.616647+0000","last_unstale":"2023-12-01T01:17:25.617181+0000","last_undegraded":"2023-12-01T01:17:25.617181+0000","last_fullsized":"2023-12-01T01:17:25.617181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:59.017264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":93,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-01T01:17:41.726984+0000","last_change":"2023-12-01T01:16:24.929103+0000","last_active":"2023-12-01T01:17:41.726984+0000","last_peered":"2023-12-01T01:17:41.726984+0000","last_clean":"2023-12-01T01:17:41.726984+0000","last_became_active":"2023-12-01T01:16:24.928769+0000","last_became_peered":"2023-12-01T01:16:24.928769+0000","last_unstale":"2023-12-01T01:17:41.726984+0000","last_undegraded":"2023-12-01T01:17:41.726984+0000","last_fullsized":"2023-12-01T01:17:41.726984+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:45:24.542610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.613692+0000","last_change":"2023-12-01T01:16:25.370816+0000","last_active":"2023-12-01T01:17:24.613692+0000","last_peered":"2023-12-01T01:17:24.613692+0000","last_clean":"2023-12-01T01:17:24.613692+0000","last_became_active":"2023-12-01T01:16:25.370465+0000","last_became_peered":"2023-12-01T01:16:25.370465+0000","last_unstale":"2023-12-01T01:17:24.613692+0000","last_undegraded":"2023-12-01T01:17:24.613692+0000","last_fullsized":"2023-12-01T01:17:24.613692+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:52:28.990416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'17","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.614954+0000","last_change":"2023-12-01T01:16:24.933022+0000","last_active":"2023-12-01T01:17:24.614954+0000","last_peered":"2023-12-01T01:17:24.614954+0000","last_clean":"2023-12-01T01:17:24.614954+0000","last_became_active":"2023-12-01T01:16:24.932705+0000","last_became_peered":"2023-12-01T01:16:24.932705+0000","last_unstale":"2023-12-01T01:17:24.614954+0000","last_undegraded":"2023-12-01T01:17:24.614954+0000","last_fullsized":"2023-12-01T01:17:24.614954+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:36:44.747984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.616969+0000","last_change":"2023-12-01T01:16:25.370881+0000","last_active":"2023-12-01T01:17:24.616969+0000","last_peered":"2023-12-01T01:17:24.616969+0000","last_clean":"2023-12-01T01:17:24.616969+0000","last_became_active":"2023-12-01T01:16:25.368242+0000","last_became_peered":"2023-12-01T01:16:25.368242+0000","last_unstale":"2023-12-01T01:17:24.616969+0000","last_undegraded":"2023-12-01T01:17:24.616969+0000","last_fullsized":"2023-12-01T01:17:24.616969+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:35:41.128841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.616226+0000","last_change":"2023-12-01T01:16:24.933579+0000","last_active":"2023-12-01T01:17:24.616226+0000","last_peered":"2023-12-01T01:17:24.616226+0000","last_clean":"2023-12-01T01:17:24.616226+0000","last_became_active":"2023-12-01T01:16:24.933207+0000","last_became_peered":"2023-12-01T01:16:24.933207+0000","last_unstale":"2023-12-01T01:17:24.616226+0000","last_undegraded":"2023-12-01T01:17:24.616226+0000","last_fullsized":"2023-12-01T01:17:24.616226+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:28.309551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.612414+0000","last_change":"2023-12-01T01:16:25.368611+0000","last_active":"2023-12-01T01:17:24.612414+0000","last_peered":"2023-12-01T01:17:24.612414+0000","last_clean":"2023-12-01T01:17:24.612414+0000","last_became_active":"2023-12-01T01:16:25.368291+0000","last_became_peered":"2023-12-01T01:16:25.368291+0000","last_unstale":"2023-12-01T01:17:24.612414+0000","last_undegraded":"2023-12-01T01:17:24.612414+0000","last_fullsized":"2023-12-01T01:17:24.612414+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:17:21.824737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.611963+0000","last_change":"2023-12-01T01:16:24.941410+0000","last_active":"2023-12-01T01:17:24.611963+0000","last_peered":"2023-12-01T01:17:24.611963+0000","last_clean":"2023-12-01T01:17:24.611963+0000","last_became_active":"2023-12-01T01:16:24.941127+0000","last_became_peered":"2023-12-01T01:16:24.941127+0000","last_unstale":"2023-12-01T01:17:24.611963+0000","last_undegraded":"2023-12-01T01:17:24.611963+0000","last_fullsized":"2023-12-01T01:17:24.611963+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:25:20.911892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.619230+0000","last_change":"2023-12-01T01:16:25.368235+0000","last_active":"2023-12-01T01:17:24.619230+0000","last_peered":"2023-12-01T01:17:24.619230+0000","last_clean":"2023-12-01T01:17:24.619230+0000","last_became_active":"2023-12-01T01:16:25.367918+0000","last_became_peered":"2023-12-01T01:16:25.367918+0000","last_unstale":"2023-12-01T01:17:24.619230+0000","last_undegraded":"2023-12-01T01:17:24.619230+0000","last_fullsized":"2023-12-01T01:17:24.619230+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:12:26.665404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'5","reported_seq":91,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-01T01:17:41.714223+0000","last_change":"2023-12-01T01:16:24.932057+0000","last_active":"2023-12-01T01:17:41.714223+0000","last_peered":"2023-12-01T01:17:41.714223+0000","last_clean":"2023-12-01T01:17:41.714223+0000","last_became_active":"2023-12-01T01:16:24.931817+0000","last_became_peered":"2023-12-01T01:16:24.931817+0000","last_unstale":"2023-12-01T01:17:41.714223+0000","last_undegraded":"2023-12-01T01:17:41.714223+0000","last_fullsized":"2023-12-01T01:17:41.714223+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:19:02.039334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.619475+0000","last_change":"2023-12-01T01:16:25.367431+0000","last_active":"2023-12-01T01:17:24.619475+0000","last_peered":"2023-12-01T01:17:24.619475+0000","last_clean":"2023-12-01T01:17:24.619475+0000","last_became_active":"2023-12-01T01:16:25.367184+0000","last_became_peered":"2023-12-01T01:16:25.367184+0000","last_unstale":"2023-12-01T01:17:24.619475+0000","last_undegraded":"2023-12-01T01:17:24.619475+0000","last_fullsized":"2023-12-01T01:17:24.619475+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:21:19.132725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617731+0000","last_change":"2023-12-01T01:16:25.370593+0000","last_active":"2023-12-01T01:17:24.617731+0000","last_peered":"2023-12-01T01:17:24.617731+0000","last_clean":"2023-12-01T01:17:24.617731+0000","last_became_active":"2023-12-01T01:16:25.367614+0000","last_became_peered":"2023-12-01T01:16:25.367614+0000","last_unstale":"2023-12-01T01:17:24.617731+0000","last_undegraded":"2023-12-01T01:17:24.617731+0000","last_fullsized":"2023-12-01T01:17:24.617731+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:56:34.388180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":96,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-01T01:17:45.733343+0000","last_change":"2023-12-01T01:16:24.931188+0000","last_active":"2023-12-01T01:17:45.733343+0000","last_peered":"2023-12-01T01:17:45.733343+0000","last_clean":"2023-12-01T01:17:45.733343+0000","last_became_active":"2023-12-01T01:16:24.930875+0000","last_became_peered":"2023-12-01T01:16:24.930875+0000","last_unstale":"2023-12-01T01:17:45.733343+0000","last_undegraded":"2023-12-01T01:17:45.733343+0000","last_fullsized":"2023-12-01T01:17:45.733343+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:17.729286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":77,"num_read_kb":77,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.616545+0000","last_change":"2023-12-01T01:16:25.361359+0000","last_active":"2023-12-01T01:17:24.616545+0000","last_peered":"2023-12-01T01:17:24.616545+0000","last_clean":"2023-12-01T01:17:24.616545+0000","last_became_active":"2023-12-01T01:16:25.361083+0000","last_became_peered":"2023-12-01T01:16:25.361083+0000","last_unstale":"2023-12-01T01:17:24.616545+0000","last_undegraded":"2023-12-01T01:17:24.616545+0000","last_fullsized":"2023-12-01T01:17:24.616545+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:42:09.417021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617032+0000","last_change":"2023-12-01T01:16:24.927614+0000","last_active":"2023-12-01T01:17:24.617032+0000","last_peered":"2023-12-01T01:17:24.617032+0000","last_clean":"2023-12-01T01:17:24.617032+0000","last_became_active":"2023-12-01T01:16:24.926459+0000","last_became_peered":"2023-12-01T01:16:24.926459+0000","last_unstale":"2023-12-01T01:17:24.617032+0000","last_undegraded":"2023-12-01T01:17:24.617032+0000","last_fullsized":"2023-12-01T01:17:24.617032+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:41:25.435261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":325,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.613554+0000","last_change":"2023-12-01T01:15:25.599551+0000","last_active":"2023-12-01T01:17:24.613554+0000","last_peered":"2023-12-01T01:17:24.613554+0000","last_clean":"2023-12-01T01:17:24.613554+0000","last_became_active":"2023-12-01T01:15:25.598951+0000","last_became_peered":"2023-12-01T01:15:25.598951+0000","last_unstale":"2023-12-01T01:17:24.613554+0000","last_undegraded":"2023-12-01T01:17:24.613554+0000","last_fullsized":"2023-12-01T01:17:24.613554+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:12:31.173756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:12:31.173756+0000","last_clean_scrub_stamp":"2023-12-01T01:12:31.173756+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:00:11.343823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617315+0000","last_change":"2023-12-01T01:16:25.358617+0000","last_active":"2023-12-01T01:17:24.617315+0000","last_peered":"2023-12-01T01:17:24.617315+0000","last_clean":"2023-12-01T01:17:24.617315+0000","last_became_active":"2023-12-01T01:16:25.358325+0000","last_became_peered":"2023-12-01T01:16:25.358325+0000","last_unstale":"2023-12-01T01:17:24.617315+0000","last_undegraded":"2023-12-01T01:17:24.617315+0000","last_fullsized":"2023-12-01T01:17:24.617315+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:40:01.111777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.613450+0000","last_change":"2023-12-01T01:16:24.935103+0000","last_active":"2023-12-01T01:17:24.613450+0000","last_peered":"2023-12-01T01:17:24.613450+0000","last_clean":"2023-12-01T01:17:24.613450+0000","last_became_active":"2023-12-01T01:16:24.934900+0000","last_became_peered":"2023-12-01T01:16:24.934900+0000","last_unstale":"2023-12-01T01:17:24.613450+0000","last_undegraded":"2023-12-01T01:17:24.613450+0000","last_fullsized":"2023-12-01T01:17:24.613450+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:59:37.138090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"82'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-01T01:17:25.638223+0000","last_change":"2023-12-01T01:17:25.638223+0000","last_active":"2023-12-01T01:17:25.638223+0000","last_peered":"2023-12-01T01:17:25.638223+0000","last_clean":"2023-12-01T01:17:25.638223+0000","last_became_active":"2023-12-01T01:17:25.621853+0000","last_became_peered":"2023-12-01T01:17:25.621853+0000","last_unstale":"2023-12-01T01:17:25.638223+0000","last_undegraded":"2023-12-01T01:17:25.638223+0000","last_fullsized":"2023-12-01T01:17:25.638223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:02:03.387431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617685+0000","last_change":"2023-12-01T01:16:25.362137+0000","last_active":"2023-12-01T01:17:24.617685+0000","last_peered":"2023-12-01T01:17:24.617685+0000","last_clean":"2023-12-01T01:17:24.617685+0000","last_became_active":"2023-12-01T01:16:25.361563+0000","last_became_peered":"2023-12-01T01:16:25.361563+0000","last_unstale":"2023-12-01T01:17:24.617685+0000","last_undegraded":"2023-12-01T01:17:24.617685+0000","last_fullsized":"2023-12-01T01:17:24.617685+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:37:56.473920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'9","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.612460+0000","last_change":"2023-12-01T01:16:24.935692+0000","last_active":"2023-12-01T01:17:24.612460+0000","last_peered":"2023-12-01T01:17:24.612460+0000","last_clean":"2023-12-01T01:17:24.612460+0000","last_became_active":"2023-12-01T01:16:24.934799+0000","last_became_peered":"2023-12-01T01:16:24.934799+0000","last_unstale":"2023-12-01T01:17:24.612460+0000","last_undegraded":"2023-12-01T01:17:24.612460+0000","last_fullsized":"2023-12-01T01:17:24.612460+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:22:51.586089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617743+0000","last_change":"2023-12-01T01:16:25.368672+0000","last_active":"2023-12-01T01:17:24.617743+0000","last_peered":"2023-12-01T01:17:24.617743+0000","last_clean":"2023-12-01T01:17:24.617743+0000","last_became_active":"2023-12-01T01:16:25.368309+0000","last_became_peered":"2023-12-01T01:16:25.368309+0000","last_unstale":"2023-12-01T01:17:24.617743+0000","last_undegraded":"2023-12-01T01:17:24.617743+0000","last_fullsized":"2023-12-01T01:17:24.617743+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:30:59.029250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.616635+0000","last_change":"2023-12-01T01:16:24.931322+0000","last_active":"2023-12-01T01:17:24.616635+0000","last_peered":"2023-12-01T01:17:24.616635+0000","last_clean":"2023-12-01T01:17:24.616635+0000","last_became_active":"2023-12-01T01:16:24.930961+0000","last_became_peered":"2023-12-01T01:16:24.930961+0000","last_unstale":"2023-12-01T01:17:24.616635+0000","last_undegraded":"2023-12-01T01:17:24.616635+0000","last_fullsized":"2023-12-01T01:17:24.616635+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:47:16.317035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.615754+0000","last_change":"2023-12-01T01:16:25.364065+0000","last_active":"2023-12-01T01:17:24.615754+0000","last_peered":"2023-12-01T01:17:24.615754+0000","last_clean":"2023-12-01T01:17:24.615754+0000","last_became_active":"2023-12-01T01:16:25.363694+0000","last_became_peered":"2023-12-01T01:16:25.363694+0000","last_unstale":"2023-12-01T01:17:24.615754+0000","last_undegraded":"2023-12-01T01:17:24.615754+0000","last_fullsized":"2023-12-01T01:17:24.615754+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:49:39.091587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.802537+0000","last_change":"2023-12-01T01:16:24.932730+0000","last_active":"2023-12-01T01:17:24.802537+0000","last_peered":"2023-12-01T01:17:24.802537+0000","last_clean":"2023-12-01T01:17:24.802537+0000","last_became_active":"2023-12-01T01:16:24.932404+0000","last_became_peered":"2023-12-01T01:16:24.932404+0000","last_unstale":"2023-12-01T01:17:24.802537+0000","last_undegraded":"2023-12-01T01:17:24.802537+0000","last_fullsized":"2023-12-01T01:17:24.802537+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:41:33.104418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.614957+0000","last_change":"2023-12-01T01:16:25.375795+0000","last_active":"2023-12-01T01:17:24.614957+0000","last_peered":"2023-12-01T01:17:24.614957+0000","last_clean":"2023-12-01T01:17:24.614957+0000","last_became_active":"2023-12-01T01:16:25.375485+0000","last_became_peered":"2023-12-01T01:16:25.375485+0000","last_unstale":"2023-12-01T01:17:24.614957+0000","last_undegraded":"2023-12-01T01:17:24.614957+0000","last_fullsized":"2023-12-01T01:17:24.614957+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:10:04.009511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.616284+0000","last_change":"2023-12-01T01:16:24.936770+0000","last_active":"2023-12-01T01:17:24.616284+0000","last_peered":"2023-12-01T01:17:24.616284+0000","last_clean":"2023-12-01T01:17:24.616284+0000","last_became_active":"2023-12-01T01:16:24.936058+0000","last_became_peered":"2023-12-01T01:16:24.936058+0000","last_unstale":"2023-12-01T01:17:24.616284+0000","last_undegraded":"2023-12-01T01:17:24.616284+0000","last_fullsized":"2023-12-01T01:17:24.616284+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:22:21.414131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.612201+0000","last_change":"2023-12-01T01:16:25.372183+0000","last_active":"2023-12-01T01:17:24.612201+0000","last_peered":"2023-12-01T01:17:24.612201+0000","last_clean":"2023-12-01T01:17:24.612201+0000","last_became_active":"2023-12-01T01:16:25.371865+0000","last_became_peered":"2023-12-01T01:16:25.371865+0000","last_unstale":"2023-12-01T01:17:24.612201+0000","last_undegraded":"2023-12-01T01:17:24.612201+0000","last_fullsized":"2023-12-01T01:17:24.612201+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:19:54.675425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.619142+0000","last_change":"2023-12-01T01:16:24.939397+0000","last_active":"2023-12-01T01:17:24.619142+0000","last_peered":"2023-12-01T01:17:24.619142+0000","last_clean":"2023-12-01T01:17:24.619142+0000","last_became_active":"2023-12-01T01:16:24.939081+0000","last_became_peered":"2023-12-01T01:16:24.939081+0000","last_unstale":"2023-12-01T01:17:24.619142+0000","last_undegraded":"2023-12-01T01:17:24.619142+0000","last_fullsized":"2023-12-01T01:17:24.619142+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:21:34.967255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.616722+0000","last_change":"2023-12-01T01:16:25.368484+0000","last_active":"2023-12-01T01:17:24.616722+0000","last_peered":"2023-12-01T01:17:24.616722+0000","last_clean":"2023-12-01T01:17:24.616722+0000","last_became_active":"2023-12-01T01:16:25.367638+0000","last_became_peered":"2023-12-01T01:16:25.367638+0000","last_unstale":"2023-12-01T01:17:24.616722+0000","last_undegraded":"2023-12-01T01:17:24.616722+0000","last_fullsized":"2023-12-01T01:17:24.616722+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:06:05.354443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.615886+0000","last_change":"2023-12-01T01:16:24.933304+0000","last_active":"2023-12-01T01:17:24.615886+0000","last_peered":"2023-12-01T01:17:24.615886+0000","last_clean":"2023-12-01T01:17:24.615886+0000","last_became_active":"2023-12-01T01:16:24.933028+0000","last_became_peered":"2023-12-01T01:16:24.933028+0000","last_unstale":"2023-12-01T01:17:24.615886+0000","last_undegraded":"2023-12-01T01:17:24.615886+0000","last_fullsized":"2023-12-01T01:17:24.615886+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:26:51.810995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.613747+0000","last_change":"2023-12-01T01:16:25.364888+0000","last_active":"2023-12-01T01:17:24.613747+0000","last_peered":"2023-12-01T01:17:24.613747+0000","last_clean":"2023-12-01T01:17:24.613747+0000","last_became_active":"2023-12-01T01:16:25.364595+0000","last_became_peered":"2023-12-01T01:16:25.364595+0000","last_unstale":"2023-12-01T01:17:24.613747+0000","last_undegraded":"2023-12-01T01:17:24.613747+0000","last_fullsized":"2023-12-01T01:17:24.613747+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:54:23.178824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'6","reported_seq":96,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-01T01:17:44.705967+0000","last_change":"2023-12-01T01:16:24.945771+0000","last_active":"2023-12-01T01:17:44.705967+0000","last_peered":"2023-12-01T01:17:44.705967+0000","last_clean":"2023-12-01T01:17:44.705967+0000","last_became_active":"2023-12-01T01:16:24.945441+0000","last_became_peered":"2023-12-01T01:16:24.945441+0000","last_unstale":"2023-12-01T01:17:44.705967+0000","last_undegraded":"2023-12-01T01:17:44.705967+0000","last_fullsized":"2023-12-01T01:17:44.705967+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:49:41.234472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1440,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":76,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.613993+0000","last_change":"2023-12-01T01:16:25.365246+0000","last_active":"2023-12-01T01:17:24.613993+0000","last_peered":"2023-12-01T01:17:24.613993+0000","last_clean":"2023-12-01T01:17:24.613993+0000","last_became_active":"2023-12-01T01:16:25.364364+0000","last_became_peered":"2023-12-01T01:16:25.364364+0000","last_unstale":"2023-12-01T01:17:24.613993+0000","last_undegraded":"2023-12-01T01:17:24.613993+0000","last_fullsized":"2023-12-01T01:17:24.613993+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:56:44.815172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.616942+0000","last_change":"2023-12-01T01:16:24.944229+0000","last_active":"2023-12-01T01:17:24.616942+0000","last_peered":"2023-12-01T01:17:24.616942+0000","last_clean":"2023-12-01T01:17:24.616942+0000","last_became_active":"2023-12-01T01:16:24.943880+0000","last_became_peered":"2023-12-01T01:16:24.943880+0000","last_unstale":"2023-12-01T01:17:24.616942+0000","last_undegraded":"2023-12-01T01:17:24.616942+0000","last_fullsized":"2023-12-01T01:17:24.616942+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:44:26.769358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.613765+0000","last_change":"2023-12-01T01:16:25.361028+0000","last_active":"2023-12-01T01:17:24.613765+0000","last_peered":"2023-12-01T01:17:24.613765+0000","last_clean":"2023-12-01T01:17:24.613765+0000","last_became_active":"2023-12-01T01:16:25.360702+0000","last_became_peered":"2023-12-01T01:16:25.360702+0000","last_unstale":"2023-12-01T01:17:24.613765+0000","last_undegraded":"2023-12-01T01:17:24.613765+0000","last_fullsized":"2023-12-01T01:17:24.613765+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:45:09.472497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'5","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617330+0000","last_change":"2023-12-01T01:16:24.945567+0000","last_active":"2023-12-01T01:17:24.617330+0000","last_peered":"2023-12-01T01:17:24.617330+0000","last_clean":"2023-12-01T01:17:24.617330+0000","last_became_active":"2023-12-01T01:16:24.945256+0000","last_became_peered":"2023-12-01T01:16:24.945256+0000","last_unstale":"2023-12-01T01:17:24.617330+0000","last_undegraded":"2023-12-01T01:17:24.617330+0000","last_fullsized":"2023-12-01T01:17:24.617330+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:19:30.944573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.613989+0000","last_change":"2023-12-01T01:16:25.367247+0000","last_active":"2023-12-01T01:17:24.613989+0000","last_peered":"2023-12-01T01:17:24.613989+0000","last_clean":"2023-12-01T01:17:24.613989+0000","last_became_active":"2023-12-01T01:16:25.366872+0000","last_became_peered":"2023-12-01T01:16:25.366872+0000","last_unstale":"2023-12-01T01:17:24.613989+0000","last_undegraded":"2023-12-01T01:17:24.613989+0000","last_fullsized":"2023-12-01T01:17:24.613989+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:03:07.697233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.612290+0000","last_change":"2023-12-01T01:16:25.376005+0000","last_active":"2023-12-01T01:17:24.612290+0000","last_peered":"2023-12-01T01:17:24.612290+0000","last_clean":"2023-12-01T01:17:24.612290+0000","last_became_active":"2023-12-01T01:16:25.375702+0000","last_became_peered":"2023-12-01T01:16:25.375702+0000","last_unstale":"2023-12-01T01:17:24.612290+0000","last_undegraded":"2023-12-01T01:17:24.612290+0000","last_fullsized":"2023-12-01T01:17:24.612290+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:14:00.955730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.619071+0000","last_change":"2023-12-01T01:16:25.371023+0000","last_active":"2023-12-01T01:17:24.619071+0000","last_peered":"2023-12-01T01:17:24.619071+0000","last_clean":"2023-12-01T01:17:24.619071+0000","last_became_active":"2023-12-01T01:16:25.370694+0000","last_became_peered":"2023-12-01T01:16:25.370694+0000","last_unstale":"2023-12-01T01:17:24.619071+0000","last_undegraded":"2023-12-01T01:17:24.619071+0000","last_fullsized":"2023-12-01T01:17:24.619071+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:03:00.715413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.618988+0000","last_change":"2023-12-01T01:16:25.361795+0000","last_active":"2023-12-01T01:17:24.618988+0000","last_peered":"2023-12-01T01:17:24.618988+0000","last_clean":"2023-12-01T01:17:24.618988+0000","last_became_active":"2023-12-01T01:16:25.361459+0000","last_became_peered":"2023-12-01T01:16:25.361459+0000","last_unstale":"2023-12-01T01:17:24.618988+0000","last_undegraded":"2023-12-01T01:17:24.618988+0000","last_fullsized":"2023-12-01T01:17:24.618988+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:18:57.531240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.615185+0000","last_change":"2023-12-01T01:16:25.363773+0000","last_active":"2023-12-01T01:17:24.615185+0000","last_peered":"2023-12-01T01:17:24.615185+0000","last_clean":"2023-12-01T01:17:24.615185+0000","last_became_active":"2023-12-01T01:16:25.363422+0000","last_became_peered":"2023-12-01T01:16:25.363422+0000","last_unstale":"2023-12-01T01:17:24.615185+0000","last_undegraded":"2023-12-01T01:17:24.615185+0000","last_fullsized":"2023-12-01T01:17:24.615185+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:48:33.395566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617624+0000","last_change":"2023-12-01T01:16:25.359431+0000","last_active":"2023-12-01T01:17:24.617624+0000","last_peered":"2023-12-01T01:17:24.617624+0000","last_clean":"2023-12-01T01:17:24.617624+0000","last_became_active":"2023-12-01T01:16:25.359116+0000","last_became_peered":"2023-12-01T01:16:25.359116+0000","last_unstale":"2023-12-01T01:17:24.617624+0000","last_undegraded":"2023-12-01T01:17:24.617624+0000","last_fullsized":"2023-12-01T01:17:24.617624+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:32:26.545269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.612682+0000","last_change":"2023-12-01T01:16:25.368008+0000","last_active":"2023-12-01T01:17:24.612682+0000","last_peered":"2023-12-01T01:17:24.612682+0000","last_clean":"2023-12-01T01:17:24.612682+0000","last_became_active":"2023-12-01T01:16:25.367726+0000","last_became_peered":"2023-12-01T01:16:25.367726+0000","last_unstale":"2023-12-01T01:17:24.612682+0000","last_undegraded":"2023-12-01T01:17:24.612682+0000","last_fullsized":"2023-12-01T01:17:24.612682+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:38:33.837846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.619425+0000","last_change":"2023-12-01T01:16:25.372781+0000","last_active":"2023-12-01T01:17:24.619425+0000","last_peered":"2023-12-01T01:17:24.619425+0000","last_clean":"2023-12-01T01:17:24.619425+0000","last_became_active":"2023-12-01T01:16:25.371443+0000","last_became_peered":"2023-12-01T01:16:25.371443+0000","last_unstale":"2023-12-01T01:17:24.619425+0000","last_undegraded":"2023-12-01T01:17:24.619425+0000","last_fullsized":"2023-12-01T01:17:24.619425+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:49:06.422055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617645+0000","last_change":"2023-12-01T01:16:25.363048+0000","last_active":"2023-12-01T01:17:24.617645+0000","last_peered":"2023-12-01T01:17:24.617645+0000","last_clean":"2023-12-01T01:17:24.617645+0000","last_became_active":"2023-12-01T01:16:25.362752+0000","last_became_peered":"2023-12-01T01:16:25.362752+0000","last_unstale":"2023-12-01T01:17:24.617645+0000","last_undegraded":"2023-12-01T01:17:24.617645+0000","last_fullsized":"2023-12-01T01:17:24.617645+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:54:16.492562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-01T01:17:25.622863+0000","last_change":"2023-12-01T01:17:25.622338+0000","last_active":"2023-12-01T01:17:25.622863+0000","last_peered":"2023-12-01T01:17:25.622863+0000","last_clean":"2023-12-01T01:17:25.622863+0000","last_became_active":"2023-12-01T01:17:25.621964+0000","last_became_peered":"2023-12-01T01:17:25.621964+0000","last_unstale":"2023-12-01T01:17:25.622863+0000","last_undegraded":"2023-12-01T01:17:25.622863+0000","last_fullsized":"2023-12-01T01:17:25.622863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:05:58.699885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.616247+0000","last_change":"2023-12-01T01:16:25.373615+0000","last_active":"2023-12-01T01:17:24.616247+0000","last_peered":"2023-12-01T01:17:24.616247+0000","last_clean":"2023-12-01T01:17:24.616247+0000","last_became_active":"2023-12-01T01:16:25.373247+0000","last_became_peered":"2023-12-01T01:16:25.373247+0000","last_unstale":"2023-12-01T01:17:24.616247+0000","last_undegraded":"2023-12-01T01:17:24.616247+0000","last_fullsized":"2023-12-01T01:17:24.616247+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:28:10.815906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.802585+0000","last_change":"2023-12-01T01:16:25.367404+0000","last_active":"2023-12-01T01:17:24.802585+0000","last_peered":"2023-12-01T01:17:24.802585+0000","last_clean":"2023-12-01T01:17:24.802585+0000","last_became_active":"2023-12-01T01:16:25.366404+0000","last_became_peered":"2023-12-01T01:16:25.366404+0000","last_unstale":"2023-12-01T01:17:24.802585+0000","last_undegraded":"2023-12-01T01:17:24.802585+0000","last_fullsized":"2023-12-01T01:17:24.802585+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:54:49.668036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617543+0000","last_change":"2023-12-01T01:16:25.360315+0000","last_active":"2023-12-01T01:17:24.617543+0000","last_peered":"2023-12-01T01:17:24.617543+0000","last_clean":"2023-12-01T01:17:24.617543+0000","last_became_active":"2023-12-01T01:16:25.360031+0000","last_became_peered":"2023-12-01T01:16:25.360031+0000","last_unstale":"2023-12-01T01:17:24.617543+0000","last_undegraded":"2023-12-01T01:17:24.617543+0000","last_fullsized":"2023-12-01T01:17:24.617543+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:13:24.437697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.612327+0000","last_change":"2023-12-01T01:16:25.367820+0000","last_active":"2023-12-01T01:17:24.612327+0000","last_peered":"2023-12-01T01:17:24.612327+0000","last_clean":"2023-12-01T01:17:24.612327+0000","last_became_active":"2023-12-01T01:16:25.367525+0000","last_became_peered":"2023-12-01T01:16:25.367525+0000","last_unstale":"2023-12-01T01:17:24.612327+0000","last_undegraded":"2023-12-01T01:17:24.612327+0000","last_fullsized":"2023-12-01T01:17:24.612327+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:28:01.552342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65789,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":781,"num_read_kb":468,"num_write":233,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":180,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":197049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481820,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":933888,"data_stored":441376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.70399999999999996}]},{"osd":1,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.66000000000000003}]},{"osd":2,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.499},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.53200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":4,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.77600000000000002}]},{"osd":5,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.80800000000000005}]},{"osd":6,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.66200000000000003}]},{"osd":7,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.83699999999999997}]},{"osd":8,"last update":"Fri Dec 1 01:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.82299999999999995}]},{"osd":9,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.92400000000000004}]},{"osd":10,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.878},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.649,"5min":1.649,"15min":1.649},"last":0.84899999999999998}]}]},{"osd":10,"up_from":69,"seq":296352743456,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28528,"kb_used_data":1576,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716624,"statfs":{"total":95995035648,"available":95965822976,"internally_reserved":0,"allocated":1613824,"data_stored":1090430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.499}]},{"osd":1,"last update":"Fri Dec 1 01:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.46000000000000002,"15min":0.46800000000000003},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.56200000000000006,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.48199999999999998,"15min":0.48399999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70899999999999996,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.52400000000000002}]},{"osd":2,"last update":"Fri Dec 1 01:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57499999999999996,"15min":0.56699999999999995},"min":{"1min":0.45900000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.82799999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81200000000000006,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.74099999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.59199999999999997,"15min":0.60199999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.82599999999999996,"15min":0.874},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.70899999999999996,"5min":5.7240000000000002,"15min":5.7240000000000002},"last":0.66100000000000003}]},{"osd":4,"last update":"Fri Dec 1 01:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.58599999999999997,"15min":0.59599999999999997},"min":{"1min":0.35899999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.70999999999999996,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.57799999999999996,"15min":0.58499999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.68999999999999995,"5min":1.016,"15min":1.016},"last":0.70899999999999996}]},{"osd":5,"last update":"Fri Dec 1 01:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.67700000000000005,"15min":0.67900000000000005},"min":{"1min":0.50900000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.78500000000000003,"5min":1.109,"15min":1.109},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.629,"5min":0.67300000000000004,"15min":0.68100000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.87,"5min":0.998,"15min":0.998},"last":0.80200000000000005}]},{"osd":6,"last update":"Fri Dec 1 01:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.69799999999999995,"15min":0.71699999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81999999999999995,"5min":1.054,"15min":1.054},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.63,"5min":0.90600000000000003,"15min":0.95199999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.82699999999999996,"5min":5.7510000000000003,"15min":5.7510000000000003},"last":0.78300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":8,"last update":"Fri Dec 1 01:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.73799999999999999,"15min":0.74399999999999999},"min":{"1min":0.56699999999999995,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.871,"5min":1.23,"15min":1.23},"last":0.878},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.76200000000000001,"15min":0.76900000000000002},"min":{"1min":0.53700000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.91700000000000004,"5min":1.216,"15min":1.216},"last":0.84199999999999997}]},{"osd":9,"last update":"Fri Dec 1 01:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.78900000000000003,"15min":0.78800000000000003},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.8839999999999999,"5min":2.8839999999999999,"15min":2.8839999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.78300000000000003,"15min":0.79500000000000004},"min":{"1min":0.52300000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88700000000000001,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.77100000000000002}]},{"osd":11,"last update":"Fri Dec 1 01:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.91400000000000003}]}]},{"osd":9,"up_from":64,"seq":274877906981,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":942080,"data_stored":440856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.46700000000000003,"15min":0.44900000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":3.6589999999999998,"5min":3.6589999999999998,"15min":3.6589999999999998},"last":0.371},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.45600000000000002,"15min":0.46100000000000002},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.47799999999999998}]},{"osd":1,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.58199999999999996,"15min":0.59199999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.65100000000000002,"15min":0.63200000000000001},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":3.4969999999999999,"5min":3.4969999999999999,"15min":3.4969999999999999},"last":0.496}]},{"osd":2,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57999999999999996,"15min":0.58499999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.72299999999999998,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.64200000000000002,"15min":0.625},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":3.5470000000000002,"5min":3.5470000000000002,"15min":3.5470000000000002},"last":0.45900000000000002}]},{"osd":3,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.54500000000000004,"15min":0.55700000000000005},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.53300000000000003,"15min":0.54500000000000004},"min":{"1min":0.33800000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.40600000000000003}]},{"osd":4,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.71299999999999997,"15min":0.69899999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":3.6019999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.69499999999999995,"15min":0.71299999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.872,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.52700000000000002}]},{"osd":5,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.67400000000000004,"15min":0.68999999999999995},"min":{"1min":0.46500000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.79100000000000004,"5min":1.099,"15min":1.099},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.66600000000000004,"15min":0.68200000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.70999999999999996,"5min":1.052,"15min":1.052},"last":0.64700000000000002}]},{"osd":6,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.626,"15min":0.64400000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.69799999999999995,"15min":0.69199999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":3.8410000000000002,"5min":3.8410000000000002,"15min":3.8410000000000002},"last":0.54000000000000004}]},{"osd":7,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.76600000000000001,"15min":0.79000000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.83199999999999996,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.749,"15min":0.77200000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81299999999999994,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.68300000000000005}]},{"osd":8,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.77400000000000002,"15min":0.76600000000000001},"min":{"1min":0.50900000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":3.5760000000000001,"5min":3.5760000000000001,"15min":3.5760000000000001},"last":0.628},{"interface":"front","average":{"1min":0.629,"5min":0.751,"15min":0.77100000000000002},"min":{"1min":0.5,"5min":0.495,"15min":0.495},"max":{"1min":0.76900000000000002,"5min":1.157,"15min":1.157},"last":0.65600000000000003}]},{"osd":10,"last update":"Fri Dec 1 01:17:25 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.74199999999999999,"15min":0.72599999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":3.629,"5min":3.629,"15min":3.629},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.70599999999999996,"15min":0.71699999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.85399999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.64000000000000001}]}]},{"osd":8,"up_from":57,"seq":244813135912,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":929792,"data_stored":441122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.57899999999999996,"15min":0.55700000000000005},"min":{"1min":0.39500000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":3.4089999999999998,"5min":3.4089999999999998,"15min":3.4089999999999998},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54000000000000004,"15min":0.53700000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.497}]},{"osd":1,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.626,"15min":0.58399999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":6.0960000000000001,"5min":6.0960000000000001,"15min":6.0960000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.61099999999999999,"15min":0.58899999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":3.556,"5min":3.556,"15min":3.556},"last":0.63}]},{"osd":2,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.50900000000000001,"15min":0.47099999999999997},"min":{"1min":0.28399999999999997,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":5.9199999999999999,"5min":5.9199999999999999,"15min":5.9199999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.48499999999999999,"15min":0.45400000000000001},"min":{"1min":0.311,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":3.6099999999999999,"5min":3.6099999999999999,"15min":3.6099999999999999},"last":0.51500000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.627,"15min":0.58199999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":6.0209999999999999,"5min":6.0209999999999999,"15min":6.0209999999999999},"last":0.47399999999999998}]},{"osd":4,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.60999999999999999,"15min":0.60399999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.66400000000000003,"15min":0.61899999999999999},"min":{"1min":0.39100000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":6.0709999999999997,"5min":6.0709999999999997,"15min":6.0709999999999997},"last":0.69199999999999995}]},{"osd":5,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.58399999999999996,"15min":0.55700000000000005},"min":{"1min":0.35799999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":3.4580000000000002,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.55500000000000005,"15min":0.54000000000000004},"min":{"1min":0.38200000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.61299999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.73399999999999999,"15min":0.69499999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":5.9969999999999999,"5min":5.9969999999999999,"15min":5.9969999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.71499999999999997,"15min":0.68899999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":3.4980000000000002,"5min":3.4980000000000002,"15min":3.4980000000000002},"last":0.58399999999999996}]},{"osd":7,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.72299999999999998,"15min":0.69999999999999996},"min":{"1min":0.498,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":3.5800000000000001,"5min":3.5800000000000001,"15min":3.5800000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.69399999999999995,"15min":0.68600000000000005},"min":{"1min":0.51800000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0229999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.72399999999999998}]},{"osd":9,"last update":"Fri Dec 1 01:17:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":1.0409999999999999,"15min":1.085},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.647,"5min":5.9690000000000003,"15min":5.9690000000000003},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.91300000000000003,"15min":0.93500000000000005},"min":{"1min":0.51700000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.532,"5min":3.5259999999999998,"15min":3.5259999999999998},"last":0.68000000000000005}]},{"osd":10,"last update":"Fri Dec 1 01:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.71499999999999997,"15min":0.69899999999999995},"min":{"1min":0.60399999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.97999999999999998,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.96999999999999997,"15min":0.99099999999999999},"min":{"1min":0.60599999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.95499999999999996,"5min":6.0469999999999997,"15min":6.0469999999999997},"last":0.83399999999999996}]},{"osd":11,"last update":"Fri Dec 1 01:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.80800000000000005}]}]},{"osd":7,"up_from":50,"seq":214748364844,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28452,"kb_used_data":1500,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716700,"statfs":{"total":95995035648,"available":95965900800,"internally_reserved":0,"allocated":1536000,"data_stored":1033616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.45000000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.40799999999999997,"15min":0.39500000000000002},"min":{"1min":0.27000000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.66100000000000003,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.42699999999999999,"15min":0.40699999999999997},"min":{"1min":0.314,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.67700000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.56999999999999995}]},{"osd":2,"last update":"Fri Dec 1 01:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.57899999999999996,"15min":0.56399999999999995},"min":{"1min":0.432,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.876,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58899999999999997,"15min":0.58299999999999996},"min":{"1min":0.39300000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.71499999999999997,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.71599999999999997}]},{"osd":3,"last update":"Fri Dec 1 01:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56399999999999995,"15min":0.55200000000000005},"min":{"1min":0.42099999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.72799999999999998,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.45400000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.71499999999999997,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.47799999999999998}]},{"osd":4,"last update":"Fri Dec 1 01:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.52000000000000002,"15min":0.51400000000000001},"min":{"1min":0.41499999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.71499999999999997,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51200000000000001,"15min":0.51000000000000001},"min":{"1min":0.28899999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.66900000000000004,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.54400000000000004}]},{"osd":5,"last update":"Fri Dec 1 01:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.629,"15min":0.61699999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.84299999999999997,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.62},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.628,"15min":0.61199999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79700000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.66800000000000004}]},{"osd":6,"last update":"Fri Dec 1 01:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65000000000000002,"15min":0.64400000000000002},"min":{"1min":0.46100000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.81599999999999995,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.66000000000000003,"15min":0.65600000000000003},"min":{"1min":0.46800000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.873,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.73299999999999998}]},{"osd":8,"last update":"Fri Dec 1 01:16:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.71699999999999997,"15min":0.72099999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.89400000000000002,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.748,"5min":0.74199999999999999,"15min":0.74099999999999999},"min":{"1min":0.49199999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.86699999999999999}]},{"osd":9,"last update":"Fri Dec 1 01:17:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.72099999999999997,"15min":0.72399999999999998},"min":{"1min":0.51200000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.93600000000000005,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.71899999999999997,"15min":0.72099999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.874,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.79600000000000004}]},{"osd":10,"last update":"Fri Dec 1 01:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.71499999999999997,"15min":0.71999999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.91000000000000003,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.71699999999999997,"15min":0.71799999999999997},"min":{"1min":0.52500000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.88500000000000001,"5min":1.032,"15min":1.032},"last":0.77200000000000002}]},{"osd":11,"last update":"Fri Dec 1 01:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.81799999999999995}]}]},{"osd":6,"up_from":45,"seq":193273528370,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":440531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.27600000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.502,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.42299999999999999,"15min":0.41299999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":0.40300000000000002}]},{"osd":1,"last update":"Fri Dec 1 01:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61899999999999999,"15min":0.625},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.57499999999999996,"15min":0.59399999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.68100000000000005,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.47999999999999998}]},{"osd":2,"last update":"Fri Dec 1 01:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.59599999999999997,"15min":0.57699999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":2.9870000000000001,"5min":2.9870000000000001,"15min":2.9870000000000001},"last":0.502},{"interface":"front","average":{"1min":0.623,"5min":0.629,"15min":0.623},"min":{"1min":0.42199999999999999,"5min":0.38,"15min":0.38},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.42099999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48399999999999999,"15min":0.46999999999999997},"min":{"1min":0.318,"5min":0.308,"15min":0.308},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.45100000000000001,"15min":0.45900000000000002},"min":{"1min":0.29999999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.501,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.311}]},{"osd":4,"last update":"Fri Dec 1 01:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.61899999999999999,"15min":0.61099999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":2.919,"5min":2.919,"15min":2.919},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.65600000000000003,"15min":0.65200000000000002},"min":{"1min":0.45400000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.49199999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.61699999999999999,"15min":0.63300000000000001},"min":{"1min":0.40300000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.67000000000000004,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.439},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.67100000000000004,"15min":0.65700000000000003},"min":{"1min":0.437,"5min":0.373,"15min":0.373},"max":{"1min":3.0179999999999998,"5min":3.0179999999999998,"15min":3.0179999999999998},"last":0.53600000000000003}]},{"osd":7,"last update":"Fri Dec 1 01:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.70399999999999996,"15min":0.70599999999999996},"min":{"1min":0.45700000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":2.956,"5min":2.956,"15min":2.956},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.76100000000000001,"15min":0.747},"min":{"1min":0.504,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":3.4660000000000002,"5min":3.4660000000000002,"15min":3.4660000000000002},"last":0.59399999999999997}]},{"osd":8,"last update":"Fri Dec 1 01:17:42 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.74099999999999999,"15min":0.76400000000000001},"min":{"1min":0.47099999999999997,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.81200000000000006,"5min":1.635,"15min":1.635},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.65500000000000003,"15min":0.68300000000000005},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.69699999999999995,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.51100000000000001}]},{"osd":9,"last update":"Fri Dec 1 01:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.69699999999999995,"15min":0.70399999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":2.1320000000000001,"5min":2.1320000000000001,"15min":2.1320000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.71999999999999997,"15min":0.73099999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":0.56200000000000006}]},{"osd":10,"last update":"Fri Dec 1 01:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.70799999999999996,"15min":0.69299999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":3.9830000000000001,"5min":3.9830000000000001,"15min":3.9830000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.78400000000000003,"15min":0.79100000000000004},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":0.624}]}]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":987136,"data_stored":499968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:17:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.51400000000000001,"15min":0.51700000000000002},"min":{"1min":0.378,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.64600000000000002,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.496},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.54300000000000004,"15min":0.52300000000000002},"min":{"1min":0.38100000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":0.623}]},{"osd":2,"last update":"Fri Dec 1 01:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.57999999999999996,"15min":0.50800000000000001},"min":{"1min":0.47199999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80800000000000005,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.61599999999999999,"15min":0.54600000000000004},"min":{"1min":0.38,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85699999999999998,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":0.73499999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58399999999999996,"15min":0.55900000000000005},"min":{"1min":0.44700000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72099999999999997,"5min":1.198,"15min":1.198},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59999999999999998,"15min":0.57399999999999995},"min":{"1min":0.46700000000000003,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.77400000000000002,"5min":1.282,"15min":1.282},"last":0.55200000000000005}]},{"osd":4,"last update":"Fri Dec 1 01:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.46100000000000002,"15min":0.42499999999999999},"min":{"1min":0.39800000000000002,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.748,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.52300000000000002,"15min":0.48999999999999999},"min":{"1min":0.372,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.76800000000000002,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.67300000000000004}]},{"osd":5,"last update":"Fri Dec 1 01:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.60199999999999998,"15min":0.53300000000000003},"min":{"1min":0.42799999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85399999999999998,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.61399999999999999,"15min":0.53900000000000003},"min":{"1min":0.56200000000000006,"5min":0.316,"15min":0.316},"max":{"1min":0.85899999999999999,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.66000000000000003}]},{"osd":6,"last update":"Fri Dec 1 01:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60599999999999998,"15min":0.59599999999999997},"min":{"1min":0.47699999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79900000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.628,"15min":0.61599999999999999},"min":{"1min":0.46400000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.89600000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.69899999999999995}]},{"osd":7,"last update":"Fri Dec 1 01:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.57799999999999996,"15min":0.54600000000000004},"min":{"1min":0.44500000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.84999999999999998,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.59199999999999997,"15min":0.55600000000000005},"min":{"1min":0.44500000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82999999999999996,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.56499999999999995}]},{"osd":8,"last update":"Fri Dec 1 01:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.72399999999999998,"15min":0.71899999999999997},"min":{"1min":0.58999999999999997,"5min":0.439,"15min":0.439},"max":{"1min":0.92800000000000005,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.74199999999999999,"15min":0.74299999999999999},"min":{"1min":0.55000000000000004,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92200000000000004,"5min":1.252,"15min":1.252},"last":0.72199999999999998}]},{"osd":9,"last update":"Fri Dec 1 01:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.73099999999999998,"15min":0.73499999999999999},"min":{"1min":0.503,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88500000000000001,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.76200000000000001,"15min":0.76600000000000001},"min":{"1min":0.58699999999999997,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.93999999999999995,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.76400000000000001}]},{"osd":10,"last update":"Fri Dec 1 01:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.73099999999999998,"15min":0.72699999999999998},"min":{"1min":0.54600000000000004,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.91900000000000004,"5min":1,"15min":1},"last":0.751},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.77300000000000002,"15min":0.77100000000000002},"min":{"1min":0.54900000000000004,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93999999999999995,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.58599999999999997}]},{"osd":11,"last update":"Fri Dec 1 01:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.78500000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738444,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":946176,"data_stored":442182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 01:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.53100000000000003,"15min":0.47899999999999998},"min":{"1min":0.39100000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":3.6160000000000001,"5min":3.6160000000000001,"15min":3.6160000000000001},"last":0.505},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55000000000000004,"15min":0.498},"min":{"1min":0.375,"5min":0.316,"15min":0.316},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.59599999999999997}]},{"osd":2,"last update":"Fri Dec 1 01:17:39 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.621,"15min":0.53000000000000003},"min":{"1min":0.41199999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":4.5010000000000003,"5min":4.5010000000000003,"15min":4.5010000000000003},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56699999999999995,"15min":0.51900000000000002},"min":{"1min":0.39300000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.909,"5min":3.1429999999999998,"15min":3.1429999999999998},"last":0.49199999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:17:25 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":3.6640000000000001,"5min":3.6640000000000001,"15min":3.6640000000000001},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":3.734,"5min":3.734,"15min":3.734},"last":0.42099999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.57299999999999995,"15min":0.54300000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":4.5129999999999999,"5min":4.5129999999999999,"15min":4.5129999999999999},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.58899999999999997,"15min":0.57499999999999996},"min":{"1min":0.40500000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.52000000000000002}]},{"osd":5,"last update":"Fri Dec 1 01:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57399999999999995,"15min":0.55900000000000005},"min":{"1min":0.39000000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.948,"5min":1.948,"15min":1.948},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.64300000000000002,"15min":0.61099999999999999},"min":{"1min":0.45600000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":3.573,"5min":3.573,"15min":3.573},"last":0.628}]},{"osd":6,"last update":"Fri Dec 1 01:17:00 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.59599999999999997,"15min":0.55500000000000005},"min":{"1min":0.27900000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":3.7189999999999999,"5min":3.7189999999999999,"15min":3.7189999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.60199999999999998,"15min":0.56699999999999995},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":3.9340000000000002,"5min":3.9340000000000002,"15min":3.9340000000000002},"last":0.54400000000000004}]},{"osd":7,"last update":"Fri Dec 1 01:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.623,"15min":0.621},"min":{"1min":0.43099999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67800000000000005,"15min":0.68200000000000005},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":1.6220000000000001}]},{"osd":8,"last update":"Fri Dec 1 01:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.66400000000000003,"15min":0.66300000000000003},"min":{"1min":0.42599999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":3.996,"5min":3.996,"15min":3.996},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.69699999999999995,"15min":0.68300000000000005},"min":{"1min":0.47899999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":4.5529999999999999,"5min":4.5529999999999999,"15min":4.5529999999999999},"last":0.58299999999999996}]},{"osd":9,"last update":"Fri Dec 1 01:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.67600000000000005,"15min":0.67200000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":3.694,"5min":3.694,"15min":3.694},"last":0.65900000000000003},{"interface":"front","average":{"1min":1.1910000000000001,"5min":0.751,"15min":0.67700000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":14.702,"5min":14.702,"15min":14.702},"last":0.60999999999999999}]},{"osd":10,"last update":"Fri Dec 1 01:17:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.66200000000000003,"15min":0.67300000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.73999999999999999,"15min":0.75800000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.78100000000000003,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.68000000000000005}]},{"osd":11,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":3.9689999999999999,"5min":3.9689999999999999,"15min":3.9689999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":0.61699999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411394,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":999424,"data_stored":500465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:17:39 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.503,"15min":0.48199999999999998},"min":{"1min":0.38700000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.84999999999999998,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.497,"15min":0.47899999999999998},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70199999999999996,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.51000000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:17:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.54200000000000004,"15min":0.51500000000000001},"min":{"1min":0.34799999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.755,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61299999999999999,"15min":0.55000000000000004},"min":{"1min":0.37,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.755,"5min":5.4169999999999998,"15min":5.4169999999999998},"last":0.72599999999999998}]},{"osd":3,"last update":"Fri Dec 1 01:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.59799999999999998,"15min":0.53000000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.91400000000000003,"5min":5.4710000000000001,"15min":5.4710000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56999999999999995,"15min":0.53000000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.78500000000000003,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.626}]},{"osd":4,"last update":"Fri Dec 1 01:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.745},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.85799999999999998}]},{"osd":5,"last update":"Fri Dec 1 01:17:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.45800000000000002,"15min":0.42899999999999999},"min":{"1min":0.39900000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.70799999999999996,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.52000000000000002,"15min":0.47999999999999998},"min":{"1min":0.39300000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.80100000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.80100000000000005}]},{"osd":6,"last update":"Fri Dec 1 01:17:00 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.64500000000000002,"15min":0.60499999999999998},"min":{"1min":0.43099999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.89300000000000002,"5min":5.5019999999999998,"15min":5.5019999999999998},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.63800000000000001,"15min":0.63400000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77100000000000002,"5min":1.429,"15min":1.429},"last":0.67300000000000004}]},{"osd":7,"last update":"Fri Dec 1 01:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.54000000000000004}]},{"osd":8,"last update":"Fri Dec 1 01:17:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.56299999999999994,"15min":0.53500000000000003},"min":{"1min":0.38300000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59199999999999997,"15min":0.58399999999999996},"min":{"1min":0.41299999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.79500000000000004,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.68799999999999994}]},{"osd":9,"last update":"Fri Dec 1 01:17:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.69799999999999995,"15min":0.70099999999999996},"min":{"1min":0.52000000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.004,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.68700000000000006,"15min":0.68200000000000005},"min":{"1min":0.51700000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94299999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.81599999999999995}]},{"osd":10,"last update":"Fri Dec 1 01:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.66200000000000003,"15min":0.65800000000000003},"min":{"1min":0.55900000000000005,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.88200000000000001,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.86799999999999999,"15min":0.88500000000000001},"min":{"1min":0.65800000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.97899999999999998,"5min":5.4450000000000003,"15min":5.4450000000000003},"last":0.89600000000000002}]},{"osd":11,"last update":"Fri Dec 1 01:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.93600000000000005}]}]},{"osd":3,"up_from":25,"seq":107374182462,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":942080,"data_stored":440410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:17:02 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.34300000000000003,"15min":0.31900000000000001},"min":{"1min":0.26800000000000002,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.51300000000000001,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.35699999999999998,"15min":0.33800000000000002},"min":{"1min":0.26600000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.48299999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.32500000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:17:02 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.53500000000000003,"15min":0.505},"min":{"1min":0.38200000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73099999999999998,"5min":0.996,"15min":0.996},"last":0.443},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.375,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.69599999999999995,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.47199999999999998}]},{"osd":2,"last update":"Fri Dec 1 01:17:02 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.51700000000000002},"min":{"1min":0.39700000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.749,"5min":0.88,"15min":0.88},"last":0.495},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55100000000000005,"15min":0.52800000000000002},"min":{"1min":0.41599999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.78600000000000003,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.53200000000000003}]},{"osd":4,"last update":"Fri Dec 1 01:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58499999999999996,"15min":0.55900000000000005},"min":{"1min":0.40100000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78000000000000003,"5min":1.329,"15min":1.329},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61099999999999999,"15min":0.58899999999999997},"min":{"1min":0.42299999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.74099999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.50700000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:17:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.55200000000000005,"15min":0.55300000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.69699999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81100000000000005,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.42599999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:17:02 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.46500000000000002,"15min":0.47299999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.503,"5min":0.55100000000000005,"15min":0.54500000000000004},"min":{"1min":0.34999999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.69899999999999995,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.40200000000000002}]},{"osd":7,"last update":"Fri Dec 1 01:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.63200000000000001,"15min":0.63},"min":{"1min":0.46100000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.84699999999999998,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.65100000000000002,"15min":0.64900000000000002},"min":{"1min":0.439,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83999999999999997,"5min":1.446,"15min":1.446},"last":0.45200000000000001}]},{"osd":8,"last update":"Fri Dec 1 01:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.51200000000000001}]},{"osd":9,"last update":"Fri Dec 1 01:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.71199999999999997,"15min":0.73399999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77300000000000002,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.63600000000000001,"15min":0.65100000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.75,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.54000000000000004}]},{"osd":10,"last update":"Fri Dec 1 01:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.70799999999999996,"15min":0.71399999999999997},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.77800000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.73299999999999998,"15min":0.745},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.90700000000000003,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.52200000000000002}]},{"osd":11,"last update":"Fri Dec 1 01:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.56200000000000006}]}]},{"osd":4,"up_from":32,"seq":137438953530,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":440386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.54300000000000004,"15min":0.50900000000000001},"min":{"1min":0.48799999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.373},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.53400000000000003,"15min":0.499},"min":{"1min":0.47199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.497}]},{"osd":1,"last update":"Fri Dec 1 01:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.41599999999999998,"15min":0.38200000000000001},"min":{"1min":0.28999999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.58199999999999996,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.41099999999999998,"15min":0.375},"min":{"1min":0.32600000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.54300000000000004,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.56299999999999994}]},{"osd":2,"last update":"Fri Dec 1 01:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.59099999999999997,"15min":0.54100000000000004},"min":{"1min":0.40999999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.60999999999999999,"15min":0.57299999999999995},"min":{"1min":0.40600000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.81000000000000005,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.63400000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.56100000000000005,"15min":0.50900000000000001},"min":{"1min":0.42999999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.52400000000000002,"15min":0.505},"min":{"1min":0.36099999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.751,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.44500000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.59499999999999997,"15min":0.56499999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.67400000000000004,"15min":0.61599999999999999},"min":{"1min":0.46000000000000002,"5min":0.377,"15min":0.377},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.69399999999999995}]},{"osd":6,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.629,"15min":0.61199999999999999},"min":{"1min":0.42999999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.94699999999999995,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.66000000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.51600000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57899999999999996,"15min":0.55700000000000005},"min":{"1min":0.38,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.73999999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.58199999999999996,"15min":0.55500000000000005},"min":{"1min":0.438,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.872,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.70699999999999996}]},{"osd":8,"last update":"Fri Dec 1 01:16:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.63100000000000001,"15min":0.63600000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.498},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76000000000000001,"15min":0.75900000000000001},"min":{"1min":0.47699999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.59199999999999997}]},{"osd":9,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.72999999999999998,"15min":0.73499999999999999},"min":{"1min":0.46600000000000003,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.79000000000000004,"15min":0.80000000000000004},"min":{"1min":0.51600000000000001,"5min":0.503,"15min":0.503},"max":{"1min":1.0900000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.66300000000000003}]},{"osd":10,"last update":"Fri Dec 1 01:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.78700000000000003,"15min":0.79900000000000004},"min":{"1min":0.496,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.93700000000000006,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.74199999999999999,"15min":0.74399999999999999},"min":{"1min":0.46999999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90900000000000003,"5min":0.997,"15min":0.997},"last":0.75600000000000001}]},{"osd":11,"last update":"Fri Dec 1 01:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.61299999999999999}]}]},{"osd":5,"up_from":37,"seq":158913790005,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28460,"kb_used_data":1508,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716692,"statfs":{"total":95995035648,"available":95965892608,"internally_reserved":0,"allocated":1544192,"data_stored":1033059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.51200000000000001},"min":{"1min":0.36199999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.54000000000000004,"15min":0.52800000000000002},"min":{"1min":0.34999999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":2.5099999999999998,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":0.61099999999999999}]},{"osd":1,"last update":"Fri Dec 1 01:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.53600000000000003,"15min":0.495},"min":{"1min":0.41999999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.55500000000000005,"15min":0.50900000000000001},"min":{"1min":0.442,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80700000000000005,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.51700000000000002}]},{"osd":2,"last update":"Fri Dec 1 01:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.40000000000000002,"15min":0.38800000000000001},"min":{"1min":0.27100000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.39900000000000002,"15min":0.379},"min":{"1min":0.28499999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.40000000000000002}]},{"osd":3,"last update":"Fri Dec 1 01:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.56999999999999995,"15min":0.54100000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":2.6379999999999999,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.56399999999999995,"15min":0.53700000000000003},"min":{"1min":0.40899999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.825,"5min":1.825,"15min":1.825},"last":0.53200000000000003}]},{"osd":4,"last update":"Fri Dec 1 01:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.57499999999999996,"15min":0.53800000000000003},"min":{"1min":0.35099999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56899999999999995,"15min":0.53500000000000003},"min":{"1min":0.41299999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.56999999999999995}]},{"osd":6,"last update":"Fri Dec 1 01:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.64200000000000002,"15min":0.626},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":2.391,"5min":2.391,"15min":2.391},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.66000000000000003,"15min":0.63},"min":{"1min":0.42899999999999999,"5min":0.37,"15min":0.37},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":0.75600000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.66100000000000003,"15min":0.65500000000000003},"min":{"1min":0.55300000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.62,"15min":0.61399999999999999},"min":{"1min":0.48499999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.84599999999999997,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.58099999999999996}]},{"osd":8,"last update":"Fri Dec 1 01:16:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58099999999999996,"15min":0.57999999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.754,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.48999999999999999}]},{"osd":9,"last update":"Fri Dec 1 01:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.70999999999999996,"15min":0.70199999999999996},"min":{"1min":0.47299999999999998,"5min":0.443,"15min":0.443},"max":{"1min":2.8069999999999999,"5min":2.8069999999999999,"15min":2.8069999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.71799999999999997,"15min":0.70199999999999996},"min":{"1min":0.48199999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":3.9049999999999998,"5min":3.9049999999999998,"15min":3.9049999999999998},"last":0.70399999999999996}]},{"osd":10,"last update":"Fri Dec 1 01:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.68400000000000005,"15min":0.67100000000000004},"min":{"1min":0.54300000000000004,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.68600000000000005,"15min":0.68000000000000005},"min":{"1min":0.57699999999999996,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.85499999999999998,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.74099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":61367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T01:17:49.089 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-12-01T01:17:49.089 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-01T01:17:49.246 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.243+0000 7fde6e764700 1 -- 172.21.15.77:0/2905888830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde68100da0 msgr2=0x7fde681011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:49.246 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.243+0000 7fde6e764700 1 --2- 172.21.15.77:0/2905888830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde68100da0 0x7fde681011c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fde580099c0 tx=0x7fde58009cd0 comp rx=0 tx=0).stop 2023-12-01T01:17:49.247 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.243+0000 7fde6e764700 1 -- 172.21.15.77:0/2905888830 shutdown_connections 2023-12-01T01:17:49.247 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.243+0000 7fde6e764700 1 --2- 172.21.15.77:0/2905888830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde68102930 0x7fde681091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:49.247 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.243+0000 7fde6e764700 1 --2- 172.21.15.77:0/2905888830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde68100da0 0x7fde681011c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:49.247 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.243+0000 7fde6e764700 1 --2- 172.21.15.77:0/2905888830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde68101f90 0x7fde681023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:49.247 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.243+0000 7fde6e764700 1 -- 172.21.15.77:0/2905888830 >> 172.21.15.77:0/2905888830 conn(0x7fde680fc2d0 msgr2=0x7fde680fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:49.247 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.243+0000 7fde6e764700 1 -- 172.21.15.77:0/2905888830 shutdown_connections 2023-12-01T01:17:49.247 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.243+0000 7fde6e764700 1 -- 172.21.15.77:0/2905888830 wait complete. 2023-12-01T01:17:49.247 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.243+0000 7fde6e764700 1 Processor -- start 2023-12-01T01:17:49.247 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.243+0000 7fde6e764700 1 -- start start 2023-12-01T01:17:49.247 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.244+0000 7fde6e764700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde68100da0 0x7fde68196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:49.247 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.244+0000 7fde6e764700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde68101f90 0x7fde68196fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:49.247 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.244+0000 7fde6e764700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde68102930 0x7fde6819e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:49.248 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.244+0000 7fde6e764700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6819e5e0 con 0x7fde68102930 2023-12-01T01:17:49.248 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.244+0000 7fde6e764700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6819e720 con 0x7fde68101f90 2023-12-01T01:17:49.248 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.244+0000 7fde6e764700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6819e8b0 con 0x7fde68100da0 2023-12-01T01:17:49.248 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.244+0000 7fde6cd01700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde68102930 0x7fde6819e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:49.248 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.244+0000 7fde6cd01700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde68102930 0x7fde6819e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59162/0 (socket says 172.21.15.77:59162) 2023-12-01T01:17:49.248 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.244+0000 7fde6cd01700 1 -- 172.21.15.77:0/2836865212 learned_addr learned my addr 172.21.15.77:0/2836865212 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:49.248 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.244+0000 7fde67fff700 1 --2- 172.21.15.77:0/2836865212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde68100da0 0x7fde68196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:49.248 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.244+0000 7fde677fe700 1 --2- 172.21.15.77:0/2836865212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde68101f90 0x7fde68196fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:49.248 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.244+0000 7fde6cd01700 1 -- 172.21.15.77:0/2836865212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde68100da0 msgr2=0x7fde68196a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:49.248 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.244+0000 7fde6cd01700 1 --2- 172.21.15.77:0/2836865212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde68100da0 0x7fde68196a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:49.249 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.244+0000 7fde6cd01700 1 -- 172.21.15.77:0/2836865212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde68101f90 msgr2=0x7fde68196fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:49.249 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.244+0000 7fde6cd01700 1 --2- 172.21.15.77:0/2836865212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde68101f90 0x7fde68196fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:49.249 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.244+0000 7fde6cd01700 1 -- 172.21.15.77:0/2836865212 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde580095f0 con 0x7fde68102930 2023-12-01T01:17:49.249 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.244+0000 7fde677fe700 1 --2- 172.21.15.77:0/2836865212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde68101f90 0x7fde68196fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:17:49.249 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.244+0000 7fde6cd01700 1 --2- 172.21.15.77:0/2836865212 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde68102930 0x7fde6819e010 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fde60007d70 tx=0x7fde6000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:49.249 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.244+0000 7fde657fa700 1 -- 172.21.15.77:0/2836865212 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6000ad40 con 0x7fde68102930 2023-12-01T01:17:49.249 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.245+0000 7fde657fa700 1 -- 172.21.15.77:0/2836865212 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde600128e0 con 0x7fde68102930 2023-12-01T01:17:49.249 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.245+0000 7fde6e764700 1 -- 172.21.15.77:0/2836865212 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde6819eb90 con 0x7fde68102930 2023-12-01T01:17:49.249 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.245+0000 7fde657fa700 1 -- 172.21.15.77:0/2836865212 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde600116b0 con 0x7fde68102930 2023-12-01T01:17:49.249 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.245+0000 7fde6e764700 1 -- 172.21.15.77:0/2836865212 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde6819f050 con 0x7fde68102930 2023-12-01T01:17:49.250 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.246+0000 7fde6e764700 1 -- 172.21.15.77:0/2836865212 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde681082f0 con 0x7fde68102930 2023-12-01T01:17:49.250 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.247+0000 7fde657fa700 1 -- 172.21.15.77:0/2836865212 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fde60012420 con 0x7fde68102930 2023-12-01T01:17:49.250 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.247+0000 7fde657fa700 1 --2- 172.21.15.77:0/2836865212 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fde500a05f0 0x7fde500a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:49.250 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.247+0000 7fde67fff700 1 --2- 172.21.15.77:0/2836865212 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fde500a05f0 0x7fde500a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:49.251 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.247+0000 7fde657fa700 1 -- 172.21.15.77:0/2836865212 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fde600cbc20 con 0x7fde68102930 2023-12-01T01:17:49.251 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.247+0000 7fde67fff700 1 --2- 172.21.15.77:0/2836865212 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fde500a05f0 0x7fde500a2ab0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fde5800b580 tx=0x7fde58009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:49.252 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.251+0000 7fde657fa700 1 -- 172.21.15.77:0/2836865212 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fde60095770 con 0x7fde68102930 2023-12-01T01:17:49.441 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.440+0000 7fde6e764700 1 -- 172.21.15.77:0/2836865212 --> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fde680611d0 con 0x7fde500a05f0 2023-12-01T01:17:49.451 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.450+0000 7fde657fa700 1 -- 172.21.15.77:0/2836865212 <== mgr.14152 v2:172.21.15.77:6800/3838941679 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344189 (secure 0 0 0) 0x7fde680611d0 con 0x7fde500a05f0 2023-12-01T01:17:49.455 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:17:49.458 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.453+0000 7fde6e764700 1 -- 172.21.15.77:0/2836865212 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fde500a05f0 msgr2=0x7fde500a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:49.458 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.453+0000 7fde6e764700 1 --2- 172.21.15.77:0/2836865212 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fde500a05f0 0x7fde500a2ab0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fde5800b580 tx=0x7fde58009f10 comp rx=0 tx=0).stop 2023-12-01T01:17:49.458 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.453+0000 7fde6e764700 1 -- 172.21.15.77:0/2836865212 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde68102930 msgr2=0x7fde6819e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:49.458 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.453+0000 7fde6e764700 1 --2- 172.21.15.77:0/2836865212 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde68102930 0x7fde6819e010 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fde60007d70 tx=0x7fde6000a340 comp rx=0 tx=0).stop 2023-12-01T01:17:49.458 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.453+0000 7fde6e764700 1 -- 172.21.15.77:0/2836865212 shutdown_connections 2023-12-01T01:17:49.458 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.453+0000 7fde6e764700 1 --2- 172.21.15.77:0/2836865212 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fde500a05f0 0x7fde500a2ab0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:49.458 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.453+0000 7fde6e764700 1 --2- 172.21.15.77:0/2836865212 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde68102930 0x7fde6819e010 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:49.458 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.453+0000 7fde6e764700 1 --2- 172.21.15.77:0/2836865212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde68100da0 0x7fde68196a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:49.458 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.453+0000 7fde6e764700 1 --2- 172.21.15.77:0/2836865212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde68101f90 0x7fde68196fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:49.458 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.453+0000 7fde6e764700 1 -- 172.21.15.77:0/2836865212 >> 172.21.15.77:0/2836865212 conn(0x7fde680fc2d0 msgr2=0x7fde68105430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:49.458 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.454+0000 7fde6e764700 1 -- 172.21.15.77:0/2836865212 shutdown_connections 2023-12-01T01:17:49.459 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.454+0000 7fde6e764700 1 -- 172.21.15.77:0/2836865212 wait complete. 2023-12-01T01:17:49.459 INFO:teuthology.orchestra.run.smithi077.stderr:dumped all 2023-12-01T01:17:49.467 INFO:teuthology.orchestra.run.smithi077.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2023-12-01T01:17:48.286446+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656403,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":907,"num_read_kb":577,"num_write":416,"num_write_kb":3070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1180916,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336340,"kb_used_data":12916,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124605484,"statfs":{"total":1151940427776,"available":1151596015616,"internally_reserved":0,"allocated":13225984,"data_stored":7244401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":636,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":131,"num_read_kb":76,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003165"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.612614+0000","last_change":"2023-12-01T01:16:25.373817+0000","last_active":"2023-12-01T01:17:24.612614+0000","last_peered":"2023-12-01T01:17:24.612614+0000","last_clean":"2023-12-01T01:17:24.612614+0000","last_became_active":"2023-12-01T01:16:25.373154+0000","last_became_peered":"2023-12-01T01:16:25.373154+0000","last_unstale":"2023-12-01T01:17:24.612614+0000","last_undegraded":"2023-12-01T01:17:24.612614+0000","last_fullsized":"2023-12-01T01:17:24.612614+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:33:43.284106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.615673+0000","last_change":"2023-12-01T01:16:25.364855+0000","last_active":"2023-12-01T01:17:24.615673+0000","last_peered":"2023-12-01T01:17:24.615673+0000","last_clean":"2023-12-01T01:17:24.615673+0000","last_became_active":"2023-12-01T01:16:25.364536+0000","last_became_peered":"2023-12-01T01:16:25.364536+0000","last_unstale":"2023-12-01T01:17:24.615673+0000","last_undegraded":"2023-12-01T01:17:24.615673+0000","last_fullsized":"2023-12-01T01:17:24.615673+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:45:23.471557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.615287+0000","last_change":"2023-12-01T01:16:25.372443+0000","last_active":"2023-12-01T01:17:24.615287+0000","last_peered":"2023-12-01T01:17:24.615287+0000","last_clean":"2023-12-01T01:17:24.615287+0000","last_became_active":"2023-12-01T01:16:25.372133+0000","last_became_peered":"2023-12-01T01:16:25.372133+0000","last_unstale":"2023-12-01T01:17:24.615287+0000","last_undegraded":"2023-12-01T01:17:24.615287+0000","last_fullsized":"2023-12-01T01:17:24.615287+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:54:19.439408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-01T01:17:25.621327+0000","last_change":"2023-12-01T01:17:25.619573+0000","last_active":"2023-12-01T01:17:25.621327+0000","last_peered":"2023-12-01T01:17:25.621327+0000","last_clean":"2023-12-01T01:17:25.621327+0000","last_became_active":"2023-12-01T01:17:25.619265+0000","last_became_peered":"2023-12-01T01:17:25.619265+0000","last_unstale":"2023-12-01T01:17:25.621327+0000","last_undegraded":"2023-12-01T01:17:25.621327+0000","last_fullsized":"2023-12-01T01:17:25.621327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:03:13.174659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.615064+0000","last_change":"2023-12-01T01:16:25.364081+0000","last_active":"2023-12-01T01:17:24.615064+0000","last_peered":"2023-12-01T01:17:24.615064+0000","last_clean":"2023-12-01T01:17:24.615064+0000","last_became_active":"2023-12-01T01:16:25.363187+0000","last_became_peered":"2023-12-01T01:16:25.363187+0000","last_unstale":"2023-12-01T01:17:24.615064+0000","last_undegraded":"2023-12-01T01:17:24.615064+0000","last_fullsized":"2023-12-01T01:17:24.615064+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:05:42.852128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.619241+0000","last_change":"2023-12-01T01:16:25.366104+0000","last_active":"2023-12-01T01:17:24.619241+0000","last_peered":"2023-12-01T01:17:24.619241+0000","last_clean":"2023-12-01T01:17:24.619241+0000","last_became_active":"2023-12-01T01:16:25.365778+0000","last_became_peered":"2023-12-01T01:16:25.365778+0000","last_unstale":"2023-12-01T01:17:24.619241+0000","last_undegraded":"2023-12-01T01:17:24.619241+0000","last_fullsized":"2023-12-01T01:17:24.619241+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:30:09.485609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.613507+0000","last_change":"2023-12-01T01:16:25.371354+0000","last_active":"2023-12-01T01:17:24.613507+0000","last_peered":"2023-12-01T01:17:24.613507+0000","last_clean":"2023-12-01T01:17:24.613507+0000","last_became_active":"2023-12-01T01:16:25.370955+0000","last_became_peered":"2023-12-01T01:16:25.370955+0000","last_unstale":"2023-12-01T01:17:24.613507+0000","last_undegraded":"2023-12-01T01:17:24.613507+0000","last_fullsized":"2023-12-01T01:17:24.613507+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:38:53.508285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617357+0000","last_change":"2023-12-01T01:16:25.368187+0000","last_active":"2023-12-01T01:17:24.617357+0000","last_peered":"2023-12-01T01:17:24.617357+0000","last_clean":"2023-12-01T01:17:24.617357+0000","last_became_active":"2023-12-01T01:16:25.367854+0000","last_became_peered":"2023-12-01T01:16:25.367854+0000","last_unstale":"2023-12-01T01:17:24.617357+0000","last_undegraded":"2023-12-01T01:17:24.617357+0000","last_fullsized":"2023-12-01T01:17:24.617357+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:22:35.159185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.802923+0000","last_change":"2023-12-01T01:16:25.372829+0000","last_active":"2023-12-01T01:17:24.802923+0000","last_peered":"2023-12-01T01:17:24.802923+0000","last_clean":"2023-12-01T01:17:24.802923+0000","last_became_active":"2023-12-01T01:16:25.372469+0000","last_became_peered":"2023-12-01T01:16:25.372469+0000","last_unstale":"2023-12-01T01:17:24.802923+0000","last_undegraded":"2023-12-01T01:17:24.802923+0000","last_fullsized":"2023-12-01T01:17:24.802923+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:25:10.300138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-01T01:17:25.618881+0000","last_change":"2023-12-01T01:17:25.618881+0000","last_active":"2023-12-01T01:17:25.618881+0000","last_peered":"2023-12-01T01:17:25.618881+0000","last_clean":"2023-12-01T01:17:25.618881+0000","last_became_active":"2023-12-01T01:17:25.618457+0000","last_became_peered":"2023-12-01T01:17:25.618457+0000","last_unstale":"2023-12-01T01:17:25.618881+0000","last_undegraded":"2023-12-01T01:17:25.618881+0000","last_fullsized":"2023-12-01T01:17:25.618881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:44:13.172660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.613960+0000","last_change":"2023-12-01T01:16:25.367602+0000","last_active":"2023-12-01T01:17:24.613960+0000","last_peered":"2023-12-01T01:17:24.613960+0000","last_clean":"2023-12-01T01:17:24.613960+0000","last_became_active":"2023-12-01T01:16:25.366565+0000","last_became_peered":"2023-12-01T01:16:25.366565+0000","last_unstale":"2023-12-01T01:17:24.613960+0000","last_undegraded":"2023-12-01T01:17:24.613960+0000","last_fullsized":"2023-12-01T01:17:24.613960+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:28:52.576838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.616474+0000","last_change":"2023-12-01T01:16:25.375004+0000","last_active":"2023-12-01T01:17:24.616474+0000","last_peered":"2023-12-01T01:17:24.616474+0000","last_clean":"2023-12-01T01:17:24.616474+0000","last_became_active":"2023-12-01T01:16:25.374320+0000","last_became_peered":"2023-12-01T01:16:25.374320+0000","last_unstale":"2023-12-01T01:17:24.616474+0000","last_undegraded":"2023-12-01T01:17:24.616474+0000","last_fullsized":"2023-12-01T01:17:24.616474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:40:49.482061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617481+0000","last_change":"2023-12-01T01:16:25.367858+0000","last_active":"2023-12-01T01:17:24.617481+0000","last_peered":"2023-12-01T01:17:24.617481+0000","last_clean":"2023-12-01T01:17:24.617481+0000","last_became_active":"2023-12-01T01:16:25.367560+0000","last_became_peered":"2023-12-01T01:16:25.367560+0000","last_unstale":"2023-12-01T01:17:24.617481+0000","last_undegraded":"2023-12-01T01:17:24.617481+0000","last_fullsized":"2023-12-01T01:17:24.617481+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:33:21.048672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.615018+0000","last_change":"2023-12-01T01:16:25.375654+0000","last_active":"2023-12-01T01:17:24.615018+0000","last_peered":"2023-12-01T01:17:24.615018+0000","last_clean":"2023-12-01T01:17:24.615018+0000","last_became_active":"2023-12-01T01:16:25.375379+0000","last_became_peered":"2023-12-01T01:16:25.375379+0000","last_unstale":"2023-12-01T01:17:24.615018+0000","last_undegraded":"2023-12-01T01:17:24.615018+0000","last_fullsized":"2023-12-01T01:17:24.615018+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:33:42.335365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.614888+0000","last_change":"2023-12-01T01:16:25.372546+0000","last_active":"2023-12-01T01:17:24.614888+0000","last_peered":"2023-12-01T01:17:24.614888+0000","last_clean":"2023-12-01T01:17:24.614888+0000","last_became_active":"2023-12-01T01:16:25.372327+0000","last_became_peered":"2023-12-01T01:16:25.372327+0000","last_unstale":"2023-12-01T01:17:24.614888+0000","last_undegraded":"2023-12-01T01:17:24.614888+0000","last_fullsized":"2023-12-01T01:17:24.614888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:12:34.171834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.802497+0000","last_change":"2023-12-01T01:16:25.366449+0000","last_active":"2023-12-01T01:17:24.802497+0000","last_peered":"2023-12-01T01:17:24.802497+0000","last_clean":"2023-12-01T01:17:24.802497+0000","last_became_active":"2023-12-01T01:16:25.366147+0000","last_became_peered":"2023-12-01T01:16:25.366147+0000","last_unstale":"2023-12-01T01:17:24.802497+0000","last_undegraded":"2023-12-01T01:17:24.802497+0000","last_fullsized":"2023-12-01T01:17:24.802497+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:01:29.670329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.612762+0000","last_change":"2023-12-01T01:16:25.374016+0000","last_active":"2023-12-01T01:17:24.612762+0000","last_peered":"2023-12-01T01:17:24.612762+0000","last_clean":"2023-12-01T01:17:24.612762+0000","last_became_active":"2023-12-01T01:16:25.373543+0000","last_became_peered":"2023-12-01T01:16:25.373543+0000","last_unstale":"2023-12-01T01:17:24.612762+0000","last_undegraded":"2023-12-01T01:17:24.612762+0000","last_fullsized":"2023-12-01T01:17:24.612762+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:36:22.858446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.612411+0000","last_change":"2023-12-01T01:16:25.363598+0000","last_active":"2023-12-01T01:17:24.612411+0000","last_peered":"2023-12-01T01:17:24.612411+0000","last_clean":"2023-12-01T01:17:24.612411+0000","last_became_active":"2023-12-01T01:16:25.363111+0000","last_became_peered":"2023-12-01T01:16:25.363111+0000","last_unstale":"2023-12-01T01:17:24.612411+0000","last_undegraded":"2023-12-01T01:17:24.612411+0000","last_fullsized":"2023-12-01T01:17:24.612411+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:42:08.171789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.619102+0000","last_change":"2023-12-01T01:16:25.368446+0000","last_active":"2023-12-01T01:17:24.619102+0000","last_peered":"2023-12-01T01:17:24.619102+0000","last_clean":"2023-12-01T01:17:24.619102+0000","last_became_active":"2023-12-01T01:16:25.361893+0000","last_became_peered":"2023-12-01T01:16:25.361893+0000","last_unstale":"2023-12-01T01:17:24.619102+0000","last_undegraded":"2023-12-01T01:17:24.619102+0000","last_fullsized":"2023-12-01T01:17:24.619102+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:05:45.697338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'13","reported_seq":102,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-01T01:17:44.717304+0000","last_change":"2023-12-01T01:16:24.934939+0000","last_active":"2023-12-01T01:17:44.717304+0000","last_peered":"2023-12-01T01:17:44.717304+0000","last_clean":"2023-12-01T01:17:44.717304+0000","last_became_active":"2023-12-01T01:16:24.934481+0000","last_became_peered":"2023-12-01T01:16:24.934481+0000","last_unstale":"2023-12-01T01:17:44.717304+0000","last_undegraded":"2023-12-01T01:17:44.717304+0000","last_fullsized":"2023-12-01T01:17:44.717304+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:39:49.804567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":75,"num_read_kb":75,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617562+0000","last_change":"2023-12-01T01:16:25.364061+0000","last_active":"2023-12-01T01:17:24.617562+0000","last_peered":"2023-12-01T01:17:24.617562+0000","last_clean":"2023-12-01T01:17:24.617562+0000","last_became_active":"2023-12-01T01:16:25.363678+0000","last_became_peered":"2023-12-01T01:16:25.363678+0000","last_unstale":"2023-12-01T01:17:24.617562+0000","last_undegraded":"2023-12-01T01:17:24.617562+0000","last_fullsized":"2023-12-01T01:17:24.617562+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:24:07.107604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617794+0000","last_change":"2023-12-01T01:16:24.939338+0000","last_active":"2023-12-01T01:17:24.617794+0000","last_peered":"2023-12-01T01:17:24.617794+0000","last_clean":"2023-12-01T01:17:24.617794+0000","last_became_active":"2023-12-01T01:16:24.939007+0000","last_became_peered":"2023-12-01T01:16:24.939007+0000","last_unstale":"2023-12-01T01:17:24.617794+0000","last_undegraded":"2023-12-01T01:17:24.617794+0000","last_fullsized":"2023-12-01T01:17:24.617794+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:56:50.732188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.615519+0000","last_change":"2023-12-01T01:16:25.362231+0000","last_active":"2023-12-01T01:17:24.615519+0000","last_peered":"2023-12-01T01:17:24.615519+0000","last_clean":"2023-12-01T01:17:24.615519+0000","last_became_active":"2023-12-01T01:16:25.361920+0000","last_became_peered":"2023-12-01T01:16:25.361920+0000","last_unstale":"2023-12-01T01:17:24.615519+0000","last_undegraded":"2023-12-01T01:17:24.615519+0000","last_fullsized":"2023-12-01T01:17:24.615519+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:22:22.104480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'12","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.616167+0000","last_change":"2023-12-01T01:16:24.931801+0000","last_active":"2023-12-01T01:17:24.616167+0000","last_peered":"2023-12-01T01:17:24.616167+0000","last_clean":"2023-12-01T01:17:24.616167+0000","last_became_active":"2023-12-01T01:16:24.931451+0000","last_became_peered":"2023-12-01T01:16:24.931451+0000","last_unstale":"2023-12-01T01:17:24.616167+0000","last_undegraded":"2023-12-01T01:17:24.616167+0000","last_fullsized":"2023-12-01T01:17:24.616167+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:22:41.666502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.802631+0000","last_change":"2023-12-01T01:16:25.361789+0000","last_active":"2023-12-01T01:17:24.802631+0000","last_peered":"2023-12-01T01:17:24.802631+0000","last_clean":"2023-12-01T01:17:24.802631+0000","last_became_active":"2023-12-01T01:16:25.361480+0000","last_became_peered":"2023-12-01T01:16:25.361480+0000","last_unstale":"2023-12-01T01:17:24.802631+0000","last_undegraded":"2023-12-01T01:17:24.802631+0000","last_fullsized":"2023-12-01T01:17:24.802631+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:07:00.742167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'5","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.615819+0000","last_change":"2023-12-01T01:16:24.933362+0000","last_active":"2023-12-01T01:17:24.615819+0000","last_peered":"2023-12-01T01:17:24.615819+0000","last_clean":"2023-12-01T01:17:24.615819+0000","last_became_active":"2023-12-01T01:16:24.933049+0000","last_became_peered":"2023-12-01T01:16:24.933049+0000","last_unstale":"2023-12-01T01:17:24.615819+0000","last_undegraded":"2023-12-01T01:17:24.615819+0000","last_fullsized":"2023-12-01T01:17:24.615819+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:47:37.153344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617087+0000","last_change":"2023-12-01T01:16:25.364419+0000","last_active":"2023-12-01T01:17:24.617087+0000","last_peered":"2023-12-01T01:17:24.617087+0000","last_clean":"2023-12-01T01:17:24.617087+0000","last_became_active":"2023-12-01T01:16:25.363022+0000","last_became_peered":"2023-12-01T01:16:25.363022+0000","last_unstale":"2023-12-01T01:17:24.617087+0000","last_undegraded":"2023-12-01T01:17:24.617087+0000","last_fullsized":"2023-12-01T01:17:24.617087+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:58:45.660625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'6","reported_seq":95,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-01T01:17:42.703276+0000","last_change":"2023-12-01T01:16:24.935947+0000","last_active":"2023-12-01T01:17:42.703276+0000","last_peered":"2023-12-01T01:17:42.703276+0000","last_clean":"2023-12-01T01:17:42.703276+0000","last_became_active":"2023-12-01T01:16:24.935061+0000","last_became_peered":"2023-12-01T01:16:24.935061+0000","last_unstale":"2023-12-01T01:17:42.703276+0000","last_undegraded":"2023-12-01T01:17:42.703276+0000","last_fullsized":"2023-12-01T01:17:42.703276+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:00:31.551362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":967,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.615303+0000","last_change":"2023-12-01T01:16:25.363229+0000","last_active":"2023-12-01T01:17:24.615303+0000","last_peered":"2023-12-01T01:17:24.615303+0000","last_clean":"2023-12-01T01:17:24.615303+0000","last_became_active":"2023-12-01T01:16:25.362770+0000","last_became_peered":"2023-12-01T01:16:25.362770+0000","last_unstale":"2023-12-01T01:17:24.615303+0000","last_undegraded":"2023-12-01T01:17:24.615303+0000","last_fullsized":"2023-12-01T01:17:24.615303+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:45:02.782201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"82'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.619079+0000","last_change":"2023-12-01T01:16:24.930509+0000","last_active":"2023-12-01T01:17:24.619079+0000","last_peered":"2023-12-01T01:17:24.619079+0000","last_clean":"2023-12-01T01:17:24.619079+0000","last_became_active":"2023-12-01T01:16:24.930188+0000","last_became_peered":"2023-12-01T01:16:24.930188+0000","last_unstale":"2023-12-01T01:17:24.619079+0000","last_undegraded":"2023-12-01T01:17:24.619079+0000","last_fullsized":"2023-12-01T01:17:24.619079+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:04:17.748363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617184+0000","last_change":"2023-12-01T01:16:25.363383+0000","last_active":"2023-12-01T01:17:24.617184+0000","last_peered":"2023-12-01T01:17:24.617184+0000","last_clean":"2023-12-01T01:17:24.617184+0000","last_became_active":"2023-12-01T01:16:25.363085+0000","last_became_peered":"2023-12-01T01:16:25.363085+0000","last_unstale":"2023-12-01T01:17:24.617184+0000","last_undegraded":"2023-12-01T01:17:24.617184+0000","last_fullsized":"2023-12-01T01:17:24.617184+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:34:36.921706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"82'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617436+0000","last_change":"2023-12-01T01:16:24.929081+0000","last_active":"2023-12-01T01:17:24.617436+0000","last_peered":"2023-12-01T01:17:24.617436+0000","last_clean":"2023-12-01T01:17:24.617436+0000","last_became_active":"2023-12-01T01:16:24.928747+0000","last_became_peered":"2023-12-01T01:16:24.928747+0000","last_unstale":"2023-12-01T01:17:24.617436+0000","last_undegraded":"2023-12-01T01:17:24.617436+0000","last_fullsized":"2023-12-01T01:17:24.617436+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:02:24.727835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617440+0000","last_change":"2023-12-01T01:16:25.360959+0000","last_active":"2023-12-01T01:17:24.617440+0000","last_peered":"2023-12-01T01:17:24.617440+0000","last_clean":"2023-12-01T01:17:24.617440+0000","last_became_active":"2023-12-01T01:16:25.360647+0000","last_became_peered":"2023-12-01T01:16:25.360647+0000","last_unstale":"2023-12-01T01:17:24.617440+0000","last_undegraded":"2023-12-01T01:17:24.617440+0000","last_fullsized":"2023-12-01T01:17:24.617440+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:50:24.151273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.616895+0000","last_change":"2023-12-01T01:16:24.938322+0000","last_active":"2023-12-01T01:17:24.616895+0000","last_peered":"2023-12-01T01:17:24.616895+0000","last_clean":"2023-12-01T01:17:24.616895+0000","last_became_active":"2023-12-01T01:16:24.937747+0000","last_became_peered":"2023-12-01T01:16:24.937747+0000","last_unstale":"2023-12-01T01:17:24.616895+0000","last_undegraded":"2023-12-01T01:17:24.616895+0000","last_fullsized":"2023-12-01T01:17:24.616895+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:37:59.999748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.611759+0000","last_change":"2023-12-01T01:16:25.362779+0000","last_active":"2023-12-01T01:17:24.611759+0000","last_peered":"2023-12-01T01:17:24.611759+0000","last_clean":"2023-12-01T01:17:24.611759+0000","last_became_active":"2023-12-01T01:16:25.362487+0000","last_became_peered":"2023-12-01T01:16:25.362487+0000","last_unstale":"2023-12-01T01:17:24.611759+0000","last_undegraded":"2023-12-01T01:17:24.611759+0000","last_fullsized":"2023-12-01T01:17:24.611759+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:04:42.409498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'5","reported_seq":171,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-01T01:17:46.708105+0000","last_change":"2023-12-01T01:16:24.933187+0000","last_active":"2023-12-01T01:17:46.708105+0000","last_peered":"2023-12-01T01:17:46.708105+0000","last_clean":"2023-12-01T01:17:46.708105+0000","last_became_active":"2023-12-01T01:16:24.932819+0000","last_became_peered":"2023-12-01T01:16:24.932819+0000","last_unstale":"2023-12-01T01:17:46.708105+0000","last_undegraded":"2023-12-01T01:17:46.708105+0000","last_fullsized":"2023-12-01T01:17:46.708105+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:54:34.739815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":152,"num_read_kb":152,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.802822+0000","last_change":"2023-12-01T01:16:25.361183+0000","last_active":"2023-12-01T01:17:24.802822+0000","last_peered":"2023-12-01T01:17:24.802822+0000","last_clean":"2023-12-01T01:17:24.802822+0000","last_became_active":"2023-12-01T01:16:25.360871+0000","last_became_peered":"2023-12-01T01:16:25.360871+0000","last_unstale":"2023-12-01T01:17:24.802822+0000","last_undegraded":"2023-12-01T01:17:24.802822+0000","last_fullsized":"2023-12-01T01:17:24.802822+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:06:04.748159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.613828+0000","last_change":"2023-12-01T01:16:24.939355+0000","last_active":"2023-12-01T01:17:24.613828+0000","last_peered":"2023-12-01T01:17:24.613828+0000","last_clean":"2023-12-01T01:17:24.613828+0000","last_became_active":"2023-12-01T01:16:24.938967+0000","last_became_peered":"2023-12-01T01:16:24.938967+0000","last_unstale":"2023-12-01T01:17:24.613828+0000","last_undegraded":"2023-12-01T01:17:24.613828+0000","last_fullsized":"2023-12-01T01:17:24.613828+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:20:51.172753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.615582+0000","last_change":"2023-12-01T01:16:25.366752+0000","last_active":"2023-12-01T01:17:24.615582+0000","last_peered":"2023-12-01T01:17:24.615582+0000","last_clean":"2023-12-01T01:17:24.615582+0000","last_became_active":"2023-12-01T01:16:25.366466+0000","last_became_peered":"2023-12-01T01:16:25.366466+0000","last_unstale":"2023-12-01T01:17:24.615582+0000","last_undegraded":"2023-12-01T01:17:24.615582+0000","last_fullsized":"2023-12-01T01:17:24.615582+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:09:53.716127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'5","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617178+0000","last_change":"2023-12-01T01:16:24.940827+0000","last_active":"2023-12-01T01:17:24.617178+0000","last_peered":"2023-12-01T01:17:24.617178+0000","last_clean":"2023-12-01T01:17:24.617178+0000","last_became_active":"2023-12-01T01:16:24.940515+0000","last_became_peered":"2023-12-01T01:16:24.940515+0000","last_unstale":"2023-12-01T01:17:24.617178+0000","last_undegraded":"2023-12-01T01:17:24.617178+0000","last_fullsized":"2023-12-01T01:17:24.617178+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:08:26.409014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.802749+0000","last_change":"2023-12-01T01:16:25.369662+0000","last_active":"2023-12-01T01:17:24.802749+0000","last_peered":"2023-12-01T01:17:24.802749+0000","last_clean":"2023-12-01T01:17:24.802749+0000","last_became_active":"2023-12-01T01:16:25.369178+0000","last_became_peered":"2023-12-01T01:16:25.369178+0000","last_unstale":"2023-12-01T01:17:24.802749+0000","last_undegraded":"2023-12-01T01:17:24.802749+0000","last_fullsized":"2023-12-01T01:17:24.802749+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:40:39.919414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'31","reported_seq":205,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-01T01:17:46.708885+0000","last_change":"2023-12-01T01:16:24.936733+0000","last_active":"2023-12-01T01:17:46.708885+0000","last_peered":"2023-12-01T01:17:46.708885+0000","last_clean":"2023-12-01T01:17:46.708885+0000","last_became_active":"2023-12-01T01:16:24.936459+0000","last_became_peered":"2023-12-01T01:16:24.936459+0000","last_unstale":"2023-12-01T01:17:46.708885+0000","last_undegraded":"2023-12-01T01:17:46.708885+0000","last_fullsized":"2023-12-01T01:17:46.708885+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:19:58.387650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58848,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":160,"num_read_kb":137,"num_write":33,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.616046+0000","last_change":"2023-12-01T01:16:25.364292+0000","last_active":"2023-12-01T01:17:24.616046+0000","last_peered":"2023-12-01T01:17:24.616046+0000","last_clean":"2023-12-01T01:17:24.616046+0000","last_became_active":"2023-12-01T01:16:25.362615+0000","last_became_peered":"2023-12-01T01:16:25.362615+0000","last_unstale":"2023-12-01T01:17:24.616046+0000","last_undegraded":"2023-12-01T01:17:24.616046+0000","last_fullsized":"2023-12-01T01:17:24.616046+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:06:07.350607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'9","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-01T01:17:25.632990+0000","last_change":"2023-12-01T01:17:25.632990+0000","last_active":"2023-12-01T01:17:25.632990+0000","last_peered":"2023-12-01T01:17:25.632990+0000","last_clean":"2023-12-01T01:17:25.632990+0000","last_became_active":"2023-12-01T01:17:25.621584+0000","last_became_peered":"2023-12-01T01:17:25.621584+0000","last_unstale":"2023-12-01T01:17:25.632990+0000","last_undegraded":"2023-12-01T01:17:25.632990+0000","last_fullsized":"2023-12-01T01:17:25.632990+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:33:26.502407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617099+0000","last_change":"2023-12-01T01:16:25.360104+0000","last_active":"2023-12-01T01:17:24.617099+0000","last_peered":"2023-12-01T01:17:24.617099+0000","last_clean":"2023-12-01T01:17:24.617099+0000","last_became_active":"2023-12-01T01:16:25.359807+0000","last_became_peered":"2023-12-01T01:16:25.359807+0000","last_unstale":"2023-12-01T01:17:24.617099+0000","last_undegraded":"2023-12-01T01:17:24.617099+0000","last_fullsized":"2023-12-01T01:17:24.617099+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:04:05.100258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'5","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617006+0000","last_change":"2023-12-01T01:16:24.945074+0000","last_active":"2023-12-01T01:17:24.617006+0000","last_peered":"2023-12-01T01:17:24.617006+0000","last_clean":"2023-12-01T01:17:24.617006+0000","last_became_active":"2023-12-01T01:16:24.944737+0000","last_became_peered":"2023-12-01T01:16:24.944737+0000","last_unstale":"2023-12-01T01:17:24.617006+0000","last_undegraded":"2023-12-01T01:17:24.617006+0000","last_fullsized":"2023-12-01T01:17:24.617006+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:44:53.931454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-01T01:17:25.617181+0000","last_change":"2023-12-01T01:17:25.617181+0000","last_active":"2023-12-01T01:17:25.617181+0000","last_peered":"2023-12-01T01:17:25.617181+0000","last_clean":"2023-12-01T01:17:25.617181+0000","last_became_active":"2023-12-01T01:17:25.616647+0000","last_became_peered":"2023-12-01T01:17:25.616647+0000","last_unstale":"2023-12-01T01:17:25.617181+0000","last_undegraded":"2023-12-01T01:17:25.617181+0000","last_fullsized":"2023-12-01T01:17:25.617181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:59.017264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":93,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-01T01:17:41.726984+0000","last_change":"2023-12-01T01:16:24.929103+0000","last_active":"2023-12-01T01:17:41.726984+0000","last_peered":"2023-12-01T01:17:41.726984+0000","last_clean":"2023-12-01T01:17:41.726984+0000","last_became_active":"2023-12-01T01:16:24.928769+0000","last_became_peered":"2023-12-01T01:16:24.928769+0000","last_unstale":"2023-12-01T01:17:41.726984+0000","last_undegraded":"2023-12-01T01:17:41.726984+0000","last_fullsized":"2023-12-01T01:17:41.726984+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:45:24.542610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.613692+0000","last_change":"2023-12-01T01:16:25.370816+0000","last_active":"2023-12-01T01:17:24.613692+0000","last_peered":"2023-12-01T01:17:24.613692+0000","last_clean":"2023-12-01T01:17:24.613692+0000","last_became_active":"2023-12-01T01:16:25.370465+0000","last_became_peered":"2023-12-01T01:16:25.370465+0000","last_unstale":"2023-12-01T01:17:24.613692+0000","last_undegraded":"2023-12-01T01:17:24.613692+0000","last_fullsized":"2023-12-01T01:17:24.613692+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:52:28.990416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'17","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.614954+0000","last_change":"2023-12-01T01:16:24.933022+0000","last_active":"2023-12-01T01:17:24.614954+0000","last_peered":"2023-12-01T01:17:24.614954+0000","last_clean":"2023-12-01T01:17:24.614954+0000","last_became_active":"2023-12-01T01:16:24.932705+0000","last_became_peered":"2023-12-01T01:16:24.932705+0000","last_unstale":"2023-12-01T01:17:24.614954+0000","last_undegraded":"2023-12-01T01:17:24.614954+0000","last_fullsized":"2023-12-01T01:17:24.614954+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:36:44.747984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.616969+0000","last_change":"2023-12-01T01:16:25.370881+0000","last_active":"2023-12-01T01:17:24.616969+0000","last_peered":"2023-12-01T01:17:24.616969+0000","last_clean":"2023-12-01T01:17:24.616969+0000","last_became_active":"2023-12-01T01:16:25.368242+0000","last_became_peered":"2023-12-01T01:16:25.368242+0000","last_unstale":"2023-12-01T01:17:24.616969+0000","last_undegraded":"2023-12-01T01:17:24.616969+0000","last_fullsized":"2023-12-01T01:17:24.616969+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:35:41.128841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.616226+0000","last_change":"2023-12-01T01:16:24.933579+0000","last_active":"2023-12-01T01:17:24.616226+0000","last_peered":"2023-12-01T01:17:24.616226+0000","last_clean":"2023-12-01T01:17:24.616226+0000","last_became_active":"2023-12-01T01:16:24.933207+0000","last_became_peered":"2023-12-01T01:16:24.933207+0000","last_unstale":"2023-12-01T01:17:24.616226+0000","last_undegraded":"2023-12-01T01:17:24.616226+0000","last_fullsized":"2023-12-01T01:17:24.616226+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:05:28.309551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.612414+0000","last_change":"2023-12-01T01:16:25.368611+0000","last_active":"2023-12-01T01:17:24.612414+0000","last_peered":"2023-12-01T01:17:24.612414+0000","last_clean":"2023-12-01T01:17:24.612414+0000","last_became_active":"2023-12-01T01:16:25.368291+0000","last_became_peered":"2023-12-01T01:16:25.368291+0000","last_unstale":"2023-12-01T01:17:24.612414+0000","last_undegraded":"2023-12-01T01:17:24.612414+0000","last_fullsized":"2023-12-01T01:17:24.612414+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:17:21.824737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"82'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.611963+0000","last_change":"2023-12-01T01:16:24.941410+0000","last_active":"2023-12-01T01:17:24.611963+0000","last_peered":"2023-12-01T01:17:24.611963+0000","last_clean":"2023-12-01T01:17:24.611963+0000","last_became_active":"2023-12-01T01:16:24.941127+0000","last_became_peered":"2023-12-01T01:16:24.941127+0000","last_unstale":"2023-12-01T01:17:24.611963+0000","last_undegraded":"2023-12-01T01:17:24.611963+0000","last_fullsized":"2023-12-01T01:17:24.611963+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:25:20.911892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.619230+0000","last_change":"2023-12-01T01:16:25.368235+0000","last_active":"2023-12-01T01:17:24.619230+0000","last_peered":"2023-12-01T01:17:24.619230+0000","last_clean":"2023-12-01T01:17:24.619230+0000","last_became_active":"2023-12-01T01:16:25.367918+0000","last_became_peered":"2023-12-01T01:16:25.367918+0000","last_unstale":"2023-12-01T01:17:24.619230+0000","last_undegraded":"2023-12-01T01:17:24.619230+0000","last_fullsized":"2023-12-01T01:17:24.619230+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:12:26.665404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'5","reported_seq":91,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-01T01:17:41.714223+0000","last_change":"2023-12-01T01:16:24.932057+0000","last_active":"2023-12-01T01:17:41.714223+0000","last_peered":"2023-12-01T01:17:41.714223+0000","last_clean":"2023-12-01T01:17:41.714223+0000","last_became_active":"2023-12-01T01:16:24.931817+0000","last_became_peered":"2023-12-01T01:16:24.931817+0000","last_unstale":"2023-12-01T01:17:41.714223+0000","last_undegraded":"2023-12-01T01:17:41.714223+0000","last_fullsized":"2023-12-01T01:17:41.714223+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:19:02.039334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.619475+0000","last_change":"2023-12-01T01:16:25.367431+0000","last_active":"2023-12-01T01:17:24.619475+0000","last_peered":"2023-12-01T01:17:24.619475+0000","last_clean":"2023-12-01T01:17:24.619475+0000","last_became_active":"2023-12-01T01:16:25.367184+0000","last_became_peered":"2023-12-01T01:16:25.367184+0000","last_unstale":"2023-12-01T01:17:24.619475+0000","last_undegraded":"2023-12-01T01:17:24.619475+0000","last_fullsized":"2023-12-01T01:17:24.619475+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:21:19.132725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617731+0000","last_change":"2023-12-01T01:16:25.370593+0000","last_active":"2023-12-01T01:17:24.617731+0000","last_peered":"2023-12-01T01:17:24.617731+0000","last_clean":"2023-12-01T01:17:24.617731+0000","last_became_active":"2023-12-01T01:16:25.367614+0000","last_became_peered":"2023-12-01T01:16:25.367614+0000","last_unstale":"2023-12-01T01:17:24.617731+0000","last_undegraded":"2023-12-01T01:17:24.617731+0000","last_fullsized":"2023-12-01T01:17:24.617731+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:56:34.388180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":96,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-01T01:17:45.733343+0000","last_change":"2023-12-01T01:16:24.931188+0000","last_active":"2023-12-01T01:17:45.733343+0000","last_peered":"2023-12-01T01:17:45.733343+0000","last_clean":"2023-12-01T01:17:45.733343+0000","last_became_active":"2023-12-01T01:16:24.930875+0000","last_became_peered":"2023-12-01T01:16:24.930875+0000","last_unstale":"2023-12-01T01:17:45.733343+0000","last_undegraded":"2023-12-01T01:17:45.733343+0000","last_fullsized":"2023-12-01T01:17:45.733343+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:46:17.729286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":77,"num_read_kb":77,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.616545+0000","last_change":"2023-12-01T01:16:25.361359+0000","last_active":"2023-12-01T01:17:24.616545+0000","last_peered":"2023-12-01T01:17:24.616545+0000","last_clean":"2023-12-01T01:17:24.616545+0000","last_became_active":"2023-12-01T01:16:25.361083+0000","last_became_peered":"2023-12-01T01:16:25.361083+0000","last_unstale":"2023-12-01T01:17:24.616545+0000","last_undegraded":"2023-12-01T01:17:24.616545+0000","last_fullsized":"2023-12-01T01:17:24.616545+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:42:09.417021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617032+0000","last_change":"2023-12-01T01:16:24.927614+0000","last_active":"2023-12-01T01:17:24.617032+0000","last_peered":"2023-12-01T01:17:24.617032+0000","last_clean":"2023-12-01T01:17:24.617032+0000","last_became_active":"2023-12-01T01:16:24.926459+0000","last_became_peered":"2023-12-01T01:16:24.926459+0000","last_unstale":"2023-12-01T01:17:24.617032+0000","last_undegraded":"2023-12-01T01:17:24.617032+0000","last_fullsized":"2023-12-01T01:17:24.617032+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:41:25.435261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":325,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.613554+0000","last_change":"2023-12-01T01:15:25.599551+0000","last_active":"2023-12-01T01:17:24.613554+0000","last_peered":"2023-12-01T01:17:24.613554+0000","last_clean":"2023-12-01T01:17:24.613554+0000","last_became_active":"2023-12-01T01:15:25.598951+0000","last_became_peered":"2023-12-01T01:15:25.598951+0000","last_unstale":"2023-12-01T01:17:24.613554+0000","last_undegraded":"2023-12-01T01:17:24.613554+0000","last_fullsized":"2023-12-01T01:17:24.613554+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:12:31.173756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:12:31.173756+0000","last_clean_scrub_stamp":"2023-12-01T01:12:31.173756+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:00:11.343823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617315+0000","last_change":"2023-12-01T01:16:25.358617+0000","last_active":"2023-12-01T01:17:24.617315+0000","last_peered":"2023-12-01T01:17:24.617315+0000","last_clean":"2023-12-01T01:17:24.617315+0000","last_became_active":"2023-12-01T01:16:25.358325+0000","last_became_peered":"2023-12-01T01:16:25.358325+0000","last_unstale":"2023-12-01T01:17:24.617315+0000","last_undegraded":"2023-12-01T01:17:24.617315+0000","last_fullsized":"2023-12-01T01:17:24.617315+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:40:01.111777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.613450+0000","last_change":"2023-12-01T01:16:24.935103+0000","last_active":"2023-12-01T01:17:24.613450+0000","last_peered":"2023-12-01T01:17:24.613450+0000","last_clean":"2023-12-01T01:17:24.613450+0000","last_became_active":"2023-12-01T01:16:24.934900+0000","last_became_peered":"2023-12-01T01:16:24.934900+0000","last_unstale":"2023-12-01T01:17:24.613450+0000","last_undegraded":"2023-12-01T01:17:24.613450+0000","last_fullsized":"2023-12-01T01:17:24.613450+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T10:59:37.138090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"82'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-01T01:17:25.638223+0000","last_change":"2023-12-01T01:17:25.638223+0000","last_active":"2023-12-01T01:17:25.638223+0000","last_peered":"2023-12-01T01:17:25.638223+0000","last_clean":"2023-12-01T01:17:25.638223+0000","last_became_active":"2023-12-01T01:17:25.621853+0000","last_became_peered":"2023-12-01T01:17:25.621853+0000","last_unstale":"2023-12-01T01:17:25.638223+0000","last_undegraded":"2023-12-01T01:17:25.638223+0000","last_fullsized":"2023-12-01T01:17:25.638223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:02:03.387431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617685+0000","last_change":"2023-12-01T01:16:25.362137+0000","last_active":"2023-12-01T01:17:24.617685+0000","last_peered":"2023-12-01T01:17:24.617685+0000","last_clean":"2023-12-01T01:17:24.617685+0000","last_became_active":"2023-12-01T01:16:25.361563+0000","last_became_peered":"2023-12-01T01:16:25.361563+0000","last_unstale":"2023-12-01T01:17:24.617685+0000","last_undegraded":"2023-12-01T01:17:24.617685+0000","last_fullsized":"2023-12-01T01:17:24.617685+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:37:56.473920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'9","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.612460+0000","last_change":"2023-12-01T01:16:24.935692+0000","last_active":"2023-12-01T01:17:24.612460+0000","last_peered":"2023-12-01T01:17:24.612460+0000","last_clean":"2023-12-01T01:17:24.612460+0000","last_became_active":"2023-12-01T01:16:24.934799+0000","last_became_peered":"2023-12-01T01:16:24.934799+0000","last_unstale":"2023-12-01T01:17:24.612460+0000","last_undegraded":"2023-12-01T01:17:24.612460+0000","last_fullsized":"2023-12-01T01:17:24.612460+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:22:51.586089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617743+0000","last_change":"2023-12-01T01:16:25.368672+0000","last_active":"2023-12-01T01:17:24.617743+0000","last_peered":"2023-12-01T01:17:24.617743+0000","last_clean":"2023-12-01T01:17:24.617743+0000","last_became_active":"2023-12-01T01:16:25.368309+0000","last_became_peered":"2023-12-01T01:16:25.368309+0000","last_unstale":"2023-12-01T01:17:24.617743+0000","last_undegraded":"2023-12-01T01:17:24.617743+0000","last_fullsized":"2023-12-01T01:17:24.617743+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:30:59.029250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'7","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.616635+0000","last_change":"2023-12-01T01:16:24.931322+0000","last_active":"2023-12-01T01:17:24.616635+0000","last_peered":"2023-12-01T01:17:24.616635+0000","last_clean":"2023-12-01T01:17:24.616635+0000","last_became_active":"2023-12-01T01:16:24.930961+0000","last_became_peered":"2023-12-01T01:16:24.930961+0000","last_unstale":"2023-12-01T01:17:24.616635+0000","last_undegraded":"2023-12-01T01:17:24.616635+0000","last_fullsized":"2023-12-01T01:17:24.616635+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:47:16.317035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.615754+0000","last_change":"2023-12-01T01:16:25.364065+0000","last_active":"2023-12-01T01:17:24.615754+0000","last_peered":"2023-12-01T01:17:24.615754+0000","last_clean":"2023-12-01T01:17:24.615754+0000","last_became_active":"2023-12-01T01:16:25.363694+0000","last_became_peered":"2023-12-01T01:16:25.363694+0000","last_unstale":"2023-12-01T01:17:24.615754+0000","last_undegraded":"2023-12-01T01:17:24.615754+0000","last_fullsized":"2023-12-01T01:17:24.615754+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:49:39.091587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.802537+0000","last_change":"2023-12-01T01:16:24.932730+0000","last_active":"2023-12-01T01:17:24.802537+0000","last_peered":"2023-12-01T01:17:24.802537+0000","last_clean":"2023-12-01T01:17:24.802537+0000","last_became_active":"2023-12-01T01:16:24.932404+0000","last_became_peered":"2023-12-01T01:16:24.932404+0000","last_unstale":"2023-12-01T01:17:24.802537+0000","last_undegraded":"2023-12-01T01:17:24.802537+0000","last_fullsized":"2023-12-01T01:17:24.802537+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:41:33.104418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.614957+0000","last_change":"2023-12-01T01:16:25.375795+0000","last_active":"2023-12-01T01:17:24.614957+0000","last_peered":"2023-12-01T01:17:24.614957+0000","last_clean":"2023-12-01T01:17:24.614957+0000","last_became_active":"2023-12-01T01:16:25.375485+0000","last_became_peered":"2023-12-01T01:16:25.375485+0000","last_unstale":"2023-12-01T01:17:24.614957+0000","last_undegraded":"2023-12-01T01:17:24.614957+0000","last_fullsized":"2023-12-01T01:17:24.614957+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T13:10:04.009511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.616284+0000","last_change":"2023-12-01T01:16:24.936770+0000","last_active":"2023-12-01T01:17:24.616284+0000","last_peered":"2023-12-01T01:17:24.616284+0000","last_clean":"2023-12-01T01:17:24.616284+0000","last_became_active":"2023-12-01T01:16:24.936058+0000","last_became_peered":"2023-12-01T01:16:24.936058+0000","last_unstale":"2023-12-01T01:17:24.616284+0000","last_undegraded":"2023-12-01T01:17:24.616284+0000","last_fullsized":"2023-12-01T01:17:24.616284+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:22:21.414131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.612201+0000","last_change":"2023-12-01T01:16:25.372183+0000","last_active":"2023-12-01T01:17:24.612201+0000","last_peered":"2023-12-01T01:17:24.612201+0000","last_clean":"2023-12-01T01:17:24.612201+0000","last_became_active":"2023-12-01T01:16:25.371865+0000","last_became_peered":"2023-12-01T01:16:25.371865+0000","last_unstale":"2023-12-01T01:17:24.612201+0000","last_undegraded":"2023-12-01T01:17:24.612201+0000","last_fullsized":"2023-12-01T01:17:24.612201+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T03:19:54.675425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.619142+0000","last_change":"2023-12-01T01:16:24.939397+0000","last_active":"2023-12-01T01:17:24.619142+0000","last_peered":"2023-12-01T01:17:24.619142+0000","last_clean":"2023-12-01T01:17:24.619142+0000","last_became_active":"2023-12-01T01:16:24.939081+0000","last_became_peered":"2023-12-01T01:16:24.939081+0000","last_unstale":"2023-12-01T01:17:24.619142+0000","last_undegraded":"2023-12-01T01:17:24.619142+0000","last_fullsized":"2023-12-01T01:17:24.619142+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:21:34.967255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.616722+0000","last_change":"2023-12-01T01:16:25.368484+0000","last_active":"2023-12-01T01:17:24.616722+0000","last_peered":"2023-12-01T01:17:24.616722+0000","last_clean":"2023-12-01T01:17:24.616722+0000","last_became_active":"2023-12-01T01:16:25.367638+0000","last_became_peered":"2023-12-01T01:16:25.367638+0000","last_unstale":"2023-12-01T01:17:24.616722+0000","last_undegraded":"2023-12-01T01:17:24.616722+0000","last_fullsized":"2023-12-01T01:17:24.616722+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:06:05.354443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.615886+0000","last_change":"2023-12-01T01:16:24.933304+0000","last_active":"2023-12-01T01:17:24.615886+0000","last_peered":"2023-12-01T01:17:24.615886+0000","last_clean":"2023-12-01T01:17:24.615886+0000","last_became_active":"2023-12-01T01:16:24.933028+0000","last_became_peered":"2023-12-01T01:16:24.933028+0000","last_unstale":"2023-12-01T01:17:24.615886+0000","last_undegraded":"2023-12-01T01:17:24.615886+0000","last_fullsized":"2023-12-01T01:17:24.615886+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:26:51.810995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.613747+0000","last_change":"2023-12-01T01:16:25.364888+0000","last_active":"2023-12-01T01:17:24.613747+0000","last_peered":"2023-12-01T01:17:24.613747+0000","last_clean":"2023-12-01T01:17:24.613747+0000","last_became_active":"2023-12-01T01:16:25.364595+0000","last_became_peered":"2023-12-01T01:16:25.364595+0000","last_unstale":"2023-12-01T01:17:24.613747+0000","last_undegraded":"2023-12-01T01:17:24.613747+0000","last_fullsized":"2023-12-01T01:17:24.613747+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:54:23.178824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'6","reported_seq":96,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-01T01:17:44.705967+0000","last_change":"2023-12-01T01:16:24.945771+0000","last_active":"2023-12-01T01:17:44.705967+0000","last_peered":"2023-12-01T01:17:44.705967+0000","last_clean":"2023-12-01T01:17:44.705967+0000","last_became_active":"2023-12-01T01:16:24.945441+0000","last_became_peered":"2023-12-01T01:16:24.945441+0000","last_unstale":"2023-12-01T01:17:44.705967+0000","last_undegraded":"2023-12-01T01:17:44.705967+0000","last_fullsized":"2023-12-01T01:17:44.705967+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:49:41.234472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1440,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":76,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.613993+0000","last_change":"2023-12-01T01:16:25.365246+0000","last_active":"2023-12-01T01:17:24.613993+0000","last_peered":"2023-12-01T01:17:24.613993+0000","last_clean":"2023-12-01T01:17:24.613993+0000","last_became_active":"2023-12-01T01:16:25.364364+0000","last_became_peered":"2023-12-01T01:16:25.364364+0000","last_unstale":"2023-12-01T01:17:24.613993+0000","last_undegraded":"2023-12-01T01:17:24.613993+0000","last_fullsized":"2023-12-01T01:17:24.613993+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:56:44.815172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.616942+0000","last_change":"2023-12-01T01:16:24.944229+0000","last_active":"2023-12-01T01:17:24.616942+0000","last_peered":"2023-12-01T01:17:24.616942+0000","last_clean":"2023-12-01T01:17:24.616942+0000","last_became_active":"2023-12-01T01:16:24.943880+0000","last_became_peered":"2023-12-01T01:16:24.943880+0000","last_unstale":"2023-12-01T01:17:24.616942+0000","last_undegraded":"2023-12-01T01:17:24.616942+0000","last_fullsized":"2023-12-01T01:17:24.616942+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:44:26.769358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.613765+0000","last_change":"2023-12-01T01:16:25.361028+0000","last_active":"2023-12-01T01:17:24.613765+0000","last_peered":"2023-12-01T01:17:24.613765+0000","last_clean":"2023-12-01T01:17:24.613765+0000","last_became_active":"2023-12-01T01:16:25.360702+0000","last_became_peered":"2023-12-01T01:16:25.360702+0000","last_unstale":"2023-12-01T01:17:24.613765+0000","last_undegraded":"2023-12-01T01:17:24.613765+0000","last_fullsized":"2023-12-01T01:17:24.613765+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:45:09.472497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'5","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617330+0000","last_change":"2023-12-01T01:16:24.945567+0000","last_active":"2023-12-01T01:17:24.617330+0000","last_peered":"2023-12-01T01:17:24.617330+0000","last_clean":"2023-12-01T01:17:24.617330+0000","last_became_active":"2023-12-01T01:16:24.945256+0000","last_became_peered":"2023-12-01T01:16:24.945256+0000","last_unstale":"2023-12-01T01:17:24.617330+0000","last_undegraded":"2023-12-01T01:17:24.617330+0000","last_fullsized":"2023-12-01T01:17:24.617330+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:23.910296+0000","last_clean_scrub_stamp":"2023-12-01T01:16:23.910296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:19:30.944573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.613989+0000","last_change":"2023-12-01T01:16:25.367247+0000","last_active":"2023-12-01T01:17:24.613989+0000","last_peered":"2023-12-01T01:17:24.613989+0000","last_clean":"2023-12-01T01:17:24.613989+0000","last_became_active":"2023-12-01T01:16:25.366872+0000","last_became_peered":"2023-12-01T01:16:25.366872+0000","last_unstale":"2023-12-01T01:17:24.613989+0000","last_undegraded":"2023-12-01T01:17:24.613989+0000","last_fullsized":"2023-12-01T01:17:24.613989+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:03:07.697233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.612290+0000","last_change":"2023-12-01T01:16:25.376005+0000","last_active":"2023-12-01T01:17:24.612290+0000","last_peered":"2023-12-01T01:17:24.612290+0000","last_clean":"2023-12-01T01:17:24.612290+0000","last_became_active":"2023-12-01T01:16:25.375702+0000","last_became_peered":"2023-12-01T01:16:25.375702+0000","last_unstale":"2023-12-01T01:17:24.612290+0000","last_undegraded":"2023-12-01T01:17:24.612290+0000","last_fullsized":"2023-12-01T01:17:24.612290+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:14:00.955730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.619071+0000","last_change":"2023-12-01T01:16:25.371023+0000","last_active":"2023-12-01T01:17:24.619071+0000","last_peered":"2023-12-01T01:17:24.619071+0000","last_clean":"2023-12-01T01:17:24.619071+0000","last_became_active":"2023-12-01T01:16:25.370694+0000","last_became_peered":"2023-12-01T01:16:25.370694+0000","last_unstale":"2023-12-01T01:17:24.619071+0000","last_undegraded":"2023-12-01T01:17:24.619071+0000","last_fullsized":"2023-12-01T01:17:24.619071+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T04:03:00.715413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.618988+0000","last_change":"2023-12-01T01:16:25.361795+0000","last_active":"2023-12-01T01:17:24.618988+0000","last_peered":"2023-12-01T01:17:24.618988+0000","last_clean":"2023-12-01T01:17:24.618988+0000","last_became_active":"2023-12-01T01:16:25.361459+0000","last_became_peered":"2023-12-01T01:16:25.361459+0000","last_unstale":"2023-12-01T01:17:24.618988+0000","last_undegraded":"2023-12-01T01:17:24.618988+0000","last_fullsized":"2023-12-01T01:17:24.618988+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T01:18:57.531240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.615185+0000","last_change":"2023-12-01T01:16:25.363773+0000","last_active":"2023-12-01T01:17:24.615185+0000","last_peered":"2023-12-01T01:17:24.615185+0000","last_clean":"2023-12-01T01:17:24.615185+0000","last_became_active":"2023-12-01T01:16:25.363422+0000","last_became_peered":"2023-12-01T01:16:25.363422+0000","last_unstale":"2023-12-01T01:17:24.615185+0000","last_undegraded":"2023-12-01T01:17:24.615185+0000","last_fullsized":"2023-12-01T01:17:24.615185+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T02:48:33.395566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617624+0000","last_change":"2023-12-01T01:16:25.359431+0000","last_active":"2023-12-01T01:17:24.617624+0000","last_peered":"2023-12-01T01:17:24.617624+0000","last_clean":"2023-12-01T01:17:24.617624+0000","last_became_active":"2023-12-01T01:16:25.359116+0000","last_became_peered":"2023-12-01T01:16:25.359116+0000","last_unstale":"2023-12-01T01:17:24.617624+0000","last_undegraded":"2023-12-01T01:17:24.617624+0000","last_fullsized":"2023-12-01T01:17:24.617624+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:32:26.545269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.612682+0000","last_change":"2023-12-01T01:16:25.368008+0000","last_active":"2023-12-01T01:17:24.612682+0000","last_peered":"2023-12-01T01:17:24.612682+0000","last_clean":"2023-12-01T01:17:24.612682+0000","last_became_active":"2023-12-01T01:16:25.367726+0000","last_became_peered":"2023-12-01T01:16:25.367726+0000","last_unstale":"2023-12-01T01:17:24.612682+0000","last_undegraded":"2023-12-01T01:17:24.612682+0000","last_fullsized":"2023-12-01T01:17:24.612682+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T07:38:33.837846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.619425+0000","last_change":"2023-12-01T01:16:25.372781+0000","last_active":"2023-12-01T01:17:24.619425+0000","last_peered":"2023-12-01T01:17:24.619425+0000","last_clean":"2023-12-01T01:17:24.619425+0000","last_became_active":"2023-12-01T01:16:25.371443+0000","last_became_peered":"2023-12-01T01:16:25.371443+0000","last_unstale":"2023-12-01T01:17:24.619425+0000","last_undegraded":"2023-12-01T01:17:24.619425+0000","last_fullsized":"2023-12-01T01:17:24.619425+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T06:49:06.422055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617645+0000","last_change":"2023-12-01T01:16:25.363048+0000","last_active":"2023-12-01T01:17:24.617645+0000","last_peered":"2023-12-01T01:17:24.617645+0000","last_clean":"2023-12-01T01:17:24.617645+0000","last_became_active":"2023-12-01T01:16:25.362752+0000","last_became_peered":"2023-12-01T01:16:25.362752+0000","last_unstale":"2023-12-01T01:17:24.617645+0000","last_undegraded":"2023-12-01T01:17:24.617645+0000","last_fullsized":"2023-12-01T01:17:24.617645+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T12:54:16.492562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-01T01:17:25.622863+0000","last_change":"2023-12-01T01:17:25.622338+0000","last_active":"2023-12-01T01:17:25.622863+0000","last_peered":"2023-12-01T01:17:25.622863+0000","last_clean":"2023-12-01T01:17:25.622863+0000","last_became_active":"2023-12-01T01:17:25.621964+0000","last_became_peered":"2023-12-01T01:17:25.621964+0000","last_unstale":"2023-12-01T01:17:25.622863+0000","last_undegraded":"2023-12-01T01:17:25.622863+0000","last_fullsized":"2023-12-01T01:17:25.622863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:05:58.699885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,3],"acting":[8,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.616247+0000","last_change":"2023-12-01T01:16:25.373615+0000","last_active":"2023-12-01T01:17:24.616247+0000","last_peered":"2023-12-01T01:17:24.616247+0000","last_clean":"2023-12-01T01:17:24.616247+0000","last_became_active":"2023-12-01T01:16:25.373247+0000","last_became_peered":"2023-12-01T01:16:25.373247+0000","last_unstale":"2023-12-01T01:17:24.616247+0000","last_undegraded":"2023-12-01T01:17:24.616247+0000","last_fullsized":"2023-12-01T01:17:24.616247+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T08:28:10.815906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.802585+0000","last_change":"2023-12-01T01:16:25.367404+0000","last_active":"2023-12-01T01:17:24.802585+0000","last_peered":"2023-12-01T01:17:24.802585+0000","last_clean":"2023-12-01T01:17:24.802585+0000","last_became_active":"2023-12-01T01:16:25.366404+0000","last_became_peered":"2023-12-01T01:16:25.366404+0000","last_unstale":"2023-12-01T01:17:24.802585+0000","last_undegraded":"2023-12-01T01:17:24.802585+0000","last_fullsized":"2023-12-01T01:17:24.802585+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T09:54:49.668036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.617543+0000","last_change":"2023-12-01T01:16:25.360315+0000","last_active":"2023-12-01T01:17:24.617543+0000","last_peered":"2023-12-01T01:17:24.617543+0000","last_clean":"2023-12-01T01:17:24.617543+0000","last_became_active":"2023-12-01T01:16:25.360031+0000","last_became_peered":"2023-12-01T01:16:25.360031+0000","last_unstale":"2023-12-01T01:17:24.617543+0000","last_undegraded":"2023-12-01T01:17:24.617543+0000","last_fullsized":"2023-12-01T01:17:24.617543+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T11:13:24.437697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-12-01T01:17:24.612327+0000","last_change":"2023-12-01T01:16:25.367820+0000","last_active":"2023-12-01T01:17:24.612327+0000","last_peered":"2023-12-01T01:17:24.612327+0000","last_clean":"2023-12-01T01:17:24.612327+0000","last_became_active":"2023-12-01T01:16:25.367525+0000","last_became_peered":"2023-12-01T01:16:25.367525+0000","last_unstale":"2023-12-01T01:17:24.612327+0000","last_undegraded":"2023-12-01T01:17:24.612327+0000","last_fullsized":"2023-12-01T01:17:24.612327+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-01T01:16:24.913882+0000","last_clean_scrub_stamp":"2023-12-01T01:16:24.913882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-02T05:28:01.552342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65789,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":781,"num_read_kb":468,"num_write":233,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":180,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":197049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481820,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":933888,"data_stored":441376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.70399999999999996}]},{"osd":1,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.66000000000000003}]},{"osd":2,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.499},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.53200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":4,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.77600000000000002}]},{"osd":5,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.80800000000000005}]},{"osd":6,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.66200000000000003}]},{"osd":7,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.83699999999999997}]},{"osd":8,"last update":"Fri Dec 1 01:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.82299999999999995}]},{"osd":9,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.92400000000000004}]},{"osd":10,"last update":"Fri Dec 1 01:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.878},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.649,"5min":1.649,"15min":1.649},"last":0.84899999999999998}]}]},{"osd":10,"up_from":69,"seq":296352743456,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28528,"kb_used_data":1576,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716624,"statfs":{"total":95995035648,"available":95965822976,"internally_reserved":0,"allocated":1613824,"data_stored":1090430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.499}]},{"osd":1,"last update":"Fri Dec 1 01:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.46000000000000002,"15min":0.46800000000000003},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.56200000000000006,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.48199999999999998,"15min":0.48399999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70899999999999996,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.52400000000000002}]},{"osd":2,"last update":"Fri Dec 1 01:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57499999999999996,"15min":0.56699999999999995},"min":{"1min":0.45900000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.82799999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81200000000000006,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.74099999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.59199999999999997,"15min":0.60199999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.82599999999999996,"15min":0.874},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.70899999999999996,"5min":5.7240000000000002,"15min":5.7240000000000002},"last":0.66100000000000003}]},{"osd":4,"last update":"Fri Dec 1 01:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.58599999999999997,"15min":0.59599999999999997},"min":{"1min":0.35899999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.70999999999999996,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.57799999999999996,"15min":0.58499999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.68999999999999995,"5min":1.016,"15min":1.016},"last":0.70899999999999996}]},{"osd":5,"last update":"Fri Dec 1 01:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.67700000000000005,"15min":0.67900000000000005},"min":{"1min":0.50900000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.78500000000000003,"5min":1.109,"15min":1.109},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.629,"5min":0.67300000000000004,"15min":0.68100000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.87,"5min":0.998,"15min":0.998},"last":0.80200000000000005}]},{"osd":6,"last update":"Fri Dec 1 01:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.69799999999999995,"15min":0.71699999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81999999999999995,"5min":1.054,"15min":1.054},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.63,"5min":0.90600000000000003,"15min":0.95199999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.82699999999999996,"5min":5.7510000000000003,"15min":5.7510000000000003},"last":0.78300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":8,"last update":"Fri Dec 1 01:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.73799999999999999,"15min":0.74399999999999999},"min":{"1min":0.56699999999999995,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.871,"5min":1.23,"15min":1.23},"last":0.878},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.76200000000000001,"15min":0.76900000000000002},"min":{"1min":0.53700000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.91700000000000004,"5min":1.216,"15min":1.216},"last":0.84199999999999997}]},{"osd":9,"last update":"Fri Dec 1 01:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.78900000000000003,"15min":0.78800000000000003},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.8839999999999999,"5min":2.8839999999999999,"15min":2.8839999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.78300000000000003,"15min":0.79500000000000004},"min":{"1min":0.52300000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88700000000000001,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.77100000000000002}]},{"osd":11,"last update":"Fri Dec 1 01:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.91400000000000003}]}]},{"osd":9,"up_from":64,"seq":274877906981,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":942080,"data_stored":440856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.46700000000000003,"15min":0.44900000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":3.6589999999999998,"5min":3.6589999999999998,"15min":3.6589999999999998},"last":0.371},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.45600000000000002,"15min":0.46100000000000002},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.47799999999999998}]},{"osd":1,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.58199999999999996,"15min":0.59199999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.65100000000000002,"15min":0.63200000000000001},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":3.4969999999999999,"5min":3.4969999999999999,"15min":3.4969999999999999},"last":0.496}]},{"osd":2,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57999999999999996,"15min":0.58499999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.72299999999999998,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.64200000000000002,"15min":0.625},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":3.5470000000000002,"5min":3.5470000000000002,"15min":3.5470000000000002},"last":0.45900000000000002}]},{"osd":3,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.54500000000000004,"15min":0.55700000000000005},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.53300000000000003,"15min":0.54500000000000004},"min":{"1min":0.33800000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.40600000000000003}]},{"osd":4,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.71299999999999997,"15min":0.69899999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":3.6019999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.69499999999999995,"15min":0.71299999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.872,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.52700000000000002}]},{"osd":5,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.67400000000000004,"15min":0.68999999999999995},"min":{"1min":0.46500000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.79100000000000004,"5min":1.099,"15min":1.099},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.66600000000000004,"15min":0.68200000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.70999999999999996,"5min":1.052,"15min":1.052},"last":0.64700000000000002}]},{"osd":6,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.626,"15min":0.64400000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.69799999999999995,"15min":0.69199999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":3.8410000000000002,"5min":3.8410000000000002,"15min":3.8410000000000002},"last":0.54000000000000004}]},{"osd":7,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.76600000000000001,"15min":0.79000000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.83199999999999996,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.749,"15min":0.77200000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81299999999999994,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.68300000000000005}]},{"osd":8,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.77400000000000002,"15min":0.76600000000000001},"min":{"1min":0.50900000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":3.5760000000000001,"5min":3.5760000000000001,"15min":3.5760000000000001},"last":0.628},{"interface":"front","average":{"1min":0.629,"5min":0.751,"15min":0.77100000000000002},"min":{"1min":0.5,"5min":0.495,"15min":0.495},"max":{"1min":0.76900000000000002,"5min":1.157,"15min":1.157},"last":0.65600000000000003}]},{"osd":10,"last update":"Fri Dec 1 01:17:25 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.74199999999999999,"15min":0.72599999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":3.629,"5min":3.629,"15min":3.629},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.70599999999999996,"15min":0.71699999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.85399999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.64000000000000001}]}]},{"osd":8,"up_from":57,"seq":244813135912,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":929792,"data_stored":441122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.57899999999999996,"15min":0.55700000000000005},"min":{"1min":0.39500000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":3.4089999999999998,"5min":3.4089999999999998,"15min":3.4089999999999998},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54000000000000004,"15min":0.53700000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.497}]},{"osd":1,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.626,"15min":0.58399999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":6.0960000000000001,"5min":6.0960000000000001,"15min":6.0960000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.61099999999999999,"15min":0.58899999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":3.556,"5min":3.556,"15min":3.556},"last":0.63}]},{"osd":2,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.50900000000000001,"15min":0.47099999999999997},"min":{"1min":0.28399999999999997,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":5.9199999999999999,"5min":5.9199999999999999,"15min":5.9199999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.48499999999999999,"15min":0.45400000000000001},"min":{"1min":0.311,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":3.6099999999999999,"5min":3.6099999999999999,"15min":3.6099999999999999},"last":0.51500000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58299999999999996,"15min":0.57699999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.627,"15min":0.58199999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":6.0209999999999999,"5min":6.0209999999999999,"15min":6.0209999999999999},"last":0.47399999999999998}]},{"osd":4,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.60999999999999999,"15min":0.60399999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.66400000000000003,"15min":0.61899999999999999},"min":{"1min":0.39100000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":6.0709999999999997,"5min":6.0709999999999997,"15min":6.0709999999999997},"last":0.69199999999999995}]},{"osd":5,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.58399999999999996,"15min":0.55700000000000005},"min":{"1min":0.35799999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":3.4580000000000002,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.55500000000000005,"15min":0.54000000000000004},"min":{"1min":0.38200000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.61299999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.73399999999999999,"15min":0.69499999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":5.9969999999999999,"5min":5.9969999999999999,"15min":5.9969999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.71499999999999997,"15min":0.68899999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":3.4980000000000002,"5min":3.4980000000000002,"15min":3.4980000000000002},"last":0.58399999999999996}]},{"osd":7,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.72299999999999998,"15min":0.69999999999999996},"min":{"1min":0.498,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":3.5800000000000001,"5min":3.5800000000000001,"15min":3.5800000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.69399999999999995,"15min":0.68600000000000005},"min":{"1min":0.51800000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0229999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.72399999999999998}]},{"osd":9,"last update":"Fri Dec 1 01:17:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":1.0409999999999999,"15min":1.085},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.647,"5min":5.9690000000000003,"15min":5.9690000000000003},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.91300000000000003,"15min":0.93500000000000005},"min":{"1min":0.51700000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.532,"5min":3.5259999999999998,"15min":3.5259999999999998},"last":0.68000000000000005}]},{"osd":10,"last update":"Fri Dec 1 01:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.71499999999999997,"15min":0.69899999999999995},"min":{"1min":0.60399999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.97999999999999998,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.96999999999999997,"15min":0.99099999999999999},"min":{"1min":0.60599999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.95499999999999996,"5min":6.0469999999999997,"15min":6.0469999999999997},"last":0.83399999999999996}]},{"osd":11,"last update":"Fri Dec 1 01:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.80800000000000005}]}]},{"osd":7,"up_from":50,"seq":214748364844,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28452,"kb_used_data":1500,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716700,"statfs":{"total":95995035648,"available":95965900800,"internally_reserved":0,"allocated":1536000,"data_stored":1033616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.45000000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.40799999999999997,"15min":0.39500000000000002},"min":{"1min":0.27000000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.66100000000000003,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.42699999999999999,"15min":0.40699999999999997},"min":{"1min":0.314,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.67700000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.56999999999999995}]},{"osd":2,"last update":"Fri Dec 1 01:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.57899999999999996,"15min":0.56399999999999995},"min":{"1min":0.432,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.876,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58899999999999997,"15min":0.58299999999999996},"min":{"1min":0.39300000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.71499999999999997,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.71599999999999997}]},{"osd":3,"last update":"Fri Dec 1 01:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56399999999999995,"15min":0.55200000000000005},"min":{"1min":0.42099999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.72799999999999998,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.45400000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.71499999999999997,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.47799999999999998}]},{"osd":4,"last update":"Fri Dec 1 01:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.52000000000000002,"15min":0.51400000000000001},"min":{"1min":0.41499999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.71499999999999997,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51200000000000001,"15min":0.51000000000000001},"min":{"1min":0.28899999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.66900000000000004,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.54400000000000004}]},{"osd":5,"last update":"Fri Dec 1 01:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.629,"15min":0.61699999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.84299999999999997,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.62},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.628,"15min":0.61199999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79700000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.66800000000000004}]},{"osd":6,"last update":"Fri Dec 1 01:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65000000000000002,"15min":0.64400000000000002},"min":{"1min":0.46100000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.81599999999999995,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.66000000000000003,"15min":0.65600000000000003},"min":{"1min":0.46800000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.873,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.73299999999999998}]},{"osd":8,"last update":"Fri Dec 1 01:16:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.71699999999999997,"15min":0.72099999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.89400000000000002,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.748,"5min":0.74199999999999999,"15min":0.74099999999999999},"min":{"1min":0.49199999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.86699999999999999}]},{"osd":9,"last update":"Fri Dec 1 01:17:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.72099999999999997,"15min":0.72399999999999998},"min":{"1min":0.51200000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.93600000000000005,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.71899999999999997,"15min":0.72099999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.874,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.79600000000000004}]},{"osd":10,"last update":"Fri Dec 1 01:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.71499999999999997,"15min":0.71999999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.91000000000000003,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.71699999999999997,"15min":0.71799999999999997},"min":{"1min":0.52500000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.88500000000000001,"5min":1.032,"15min":1.032},"last":0.77200000000000002}]},{"osd":11,"last update":"Fri Dec 1 01:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.81799999999999995}]}]},{"osd":6,"up_from":45,"seq":193273528370,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":440531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.27600000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.502,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.42299999999999999,"15min":0.41299999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":0.40300000000000002}]},{"osd":1,"last update":"Fri Dec 1 01:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61899999999999999,"15min":0.625},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.57499999999999996,"15min":0.59399999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.68100000000000005,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.47999999999999998}]},{"osd":2,"last update":"Fri Dec 1 01:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.59599999999999997,"15min":0.57699999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":2.9870000000000001,"5min":2.9870000000000001,"15min":2.9870000000000001},"last":0.502},{"interface":"front","average":{"1min":0.623,"5min":0.629,"15min":0.623},"min":{"1min":0.42199999999999999,"5min":0.38,"15min":0.38},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.42099999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48399999999999999,"15min":0.46999999999999997},"min":{"1min":0.318,"5min":0.308,"15min":0.308},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.45100000000000001,"15min":0.45900000000000002},"min":{"1min":0.29999999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.501,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.311}]},{"osd":4,"last update":"Fri Dec 1 01:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.61899999999999999,"15min":0.61099999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":2.919,"5min":2.919,"15min":2.919},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.65600000000000003,"15min":0.65200000000000002},"min":{"1min":0.45400000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.49199999999999999}]},{"osd":5,"last update":"Fri Dec 1 01:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.61699999999999999,"15min":0.63300000000000001},"min":{"1min":0.40300000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.67000000000000004,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.439},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.67100000000000004,"15min":0.65700000000000003},"min":{"1min":0.437,"5min":0.373,"15min":0.373},"max":{"1min":3.0179999999999998,"5min":3.0179999999999998,"15min":3.0179999999999998},"last":0.53600000000000003}]},{"osd":7,"last update":"Fri Dec 1 01:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.70399999999999996,"15min":0.70599999999999996},"min":{"1min":0.45700000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":2.956,"5min":2.956,"15min":2.956},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.76100000000000001,"15min":0.747},"min":{"1min":0.504,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":3.4660000000000002,"5min":3.4660000000000002,"15min":3.4660000000000002},"last":0.59399999999999997}]},{"osd":8,"last update":"Fri Dec 1 01:17:42 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.74099999999999999,"15min":0.76400000000000001},"min":{"1min":0.47099999999999997,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.81200000000000006,"5min":1.635,"15min":1.635},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.65500000000000003,"15min":0.68300000000000005},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.69699999999999995,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.51100000000000001}]},{"osd":9,"last update":"Fri Dec 1 01:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.69699999999999995,"15min":0.70399999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":2.1320000000000001,"5min":2.1320000000000001,"15min":2.1320000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.71999999999999997,"15min":0.73099999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":0.56200000000000006}]},{"osd":10,"last update":"Fri Dec 1 01:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.70799999999999996,"15min":0.69299999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":3.9830000000000001,"5min":3.9830000000000001,"15min":3.9830000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.78400000000000003,"15min":0.79100000000000004},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":0.624}]}]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":987136,"data_stored":499968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:17:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.51400000000000001,"15min":0.51700000000000002},"min":{"1min":0.378,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.64600000000000002,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.496},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.54300000000000004,"15min":0.52300000000000002},"min":{"1min":0.38100000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":0.623}]},{"osd":2,"last update":"Fri Dec 1 01:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.57999999999999996,"15min":0.50800000000000001},"min":{"1min":0.47199999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80800000000000005,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.61599999999999999,"15min":0.54600000000000004},"min":{"1min":0.38,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85699999999999998,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":0.73499999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58399999999999996,"15min":0.55900000000000005},"min":{"1min":0.44700000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72099999999999997,"5min":1.198,"15min":1.198},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59999999999999998,"15min":0.57399999999999995},"min":{"1min":0.46700000000000003,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.77400000000000002,"5min":1.282,"15min":1.282},"last":0.55200000000000005}]},{"osd":4,"last update":"Fri Dec 1 01:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.46100000000000002,"15min":0.42499999999999999},"min":{"1min":0.39800000000000002,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.748,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.52300000000000002,"15min":0.48999999999999999},"min":{"1min":0.372,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.76800000000000002,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.67300000000000004}]},{"osd":5,"last update":"Fri Dec 1 01:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.60199999999999998,"15min":0.53300000000000003},"min":{"1min":0.42799999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85399999999999998,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.61399999999999999,"15min":0.53900000000000003},"min":{"1min":0.56200000000000006,"5min":0.316,"15min":0.316},"max":{"1min":0.85899999999999999,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.66000000000000003}]},{"osd":6,"last update":"Fri Dec 1 01:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60599999999999998,"15min":0.59599999999999997},"min":{"1min":0.47699999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79900000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.628,"15min":0.61599999999999999},"min":{"1min":0.46400000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.89600000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.69899999999999995}]},{"osd":7,"last update":"Fri Dec 1 01:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.57799999999999996,"15min":0.54600000000000004},"min":{"1min":0.44500000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.84999999999999998,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.59199999999999997,"15min":0.55600000000000005},"min":{"1min":0.44500000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82999999999999996,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.56499999999999995}]},{"osd":8,"last update":"Fri Dec 1 01:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.72399999999999998,"15min":0.71899999999999997},"min":{"1min":0.58999999999999997,"5min":0.439,"15min":0.439},"max":{"1min":0.92800000000000005,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.74199999999999999,"15min":0.74299999999999999},"min":{"1min":0.55000000000000004,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92200000000000004,"5min":1.252,"15min":1.252},"last":0.72199999999999998}]},{"osd":9,"last update":"Fri Dec 1 01:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.73099999999999998,"15min":0.73499999999999999},"min":{"1min":0.503,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88500000000000001,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.76200000000000001,"15min":0.76600000000000001},"min":{"1min":0.58699999999999997,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.93999999999999995,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.76400000000000001}]},{"osd":10,"last update":"Fri Dec 1 01:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.73099999999999998,"15min":0.72699999999999998},"min":{"1min":0.54600000000000004,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.91900000000000004,"5min":1,"15min":1},"last":0.751},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.77300000000000002,"15min":0.77100000000000002},"min":{"1min":0.54900000000000004,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93999999999999995,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.58599999999999997}]},{"osd":11,"last update":"Fri Dec 1 01:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.78500000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738444,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":946176,"data_stored":442182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 1 01:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.53100000000000003,"15min":0.47899999999999998},"min":{"1min":0.39100000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":3.6160000000000001,"5min":3.6160000000000001,"15min":3.6160000000000001},"last":0.505},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55000000000000004,"15min":0.498},"min":{"1min":0.375,"5min":0.316,"15min":0.316},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.59599999999999997}]},{"osd":2,"last update":"Fri Dec 1 01:17:39 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.621,"15min":0.53000000000000003},"min":{"1min":0.41199999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":4.5010000000000003,"5min":4.5010000000000003,"15min":4.5010000000000003},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56699999999999995,"15min":0.51900000000000002},"min":{"1min":0.39300000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.909,"5min":3.1429999999999998,"15min":3.1429999999999998},"last":0.49199999999999999}]},{"osd":3,"last update":"Fri Dec 1 01:17:25 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":3.6640000000000001,"5min":3.6640000000000001,"15min":3.6640000000000001},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":3.734,"5min":3.734,"15min":3.734},"last":0.42099999999999999}]},{"osd":4,"last update":"Fri Dec 1 01:17:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.57299999999999995,"15min":0.54300000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":4.5129999999999999,"5min":4.5129999999999999,"15min":4.5129999999999999},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.58899999999999997,"15min":0.57499999999999996},"min":{"1min":0.40500000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.52000000000000002}]},{"osd":5,"last update":"Fri Dec 1 01:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57399999999999995,"15min":0.55900000000000005},"min":{"1min":0.39000000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.948,"5min":1.948,"15min":1.948},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.64300000000000002,"15min":0.61099999999999999},"min":{"1min":0.45600000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":3.573,"5min":3.573,"15min":3.573},"last":0.628}]},{"osd":6,"last update":"Fri Dec 1 01:17:00 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.59599999999999997,"15min":0.55500000000000005},"min":{"1min":0.27900000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":3.7189999999999999,"5min":3.7189999999999999,"15min":3.7189999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.60199999999999998,"15min":0.56699999999999995},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":3.9340000000000002,"5min":3.9340000000000002,"15min":3.9340000000000002},"last":0.54400000000000004}]},{"osd":7,"last update":"Fri Dec 1 01:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.623,"15min":0.621},"min":{"1min":0.43099999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67800000000000005,"15min":0.68200000000000005},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":1.6220000000000001}]},{"osd":8,"last update":"Fri Dec 1 01:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.66400000000000003,"15min":0.66300000000000003},"min":{"1min":0.42599999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":3.996,"5min":3.996,"15min":3.996},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.69699999999999995,"15min":0.68300000000000005},"min":{"1min":0.47899999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":4.5529999999999999,"5min":4.5529999999999999,"15min":4.5529999999999999},"last":0.58299999999999996}]},{"osd":9,"last update":"Fri Dec 1 01:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.67600000000000005,"15min":0.67200000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":3.694,"5min":3.694,"15min":3.694},"last":0.65900000000000003},{"interface":"front","average":{"1min":1.1910000000000001,"5min":0.751,"15min":0.67700000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":14.702,"5min":14.702,"15min":14.702},"last":0.60999999999999999}]},{"osd":10,"last update":"Fri Dec 1 01:17:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.66200000000000003,"15min":0.67300000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.73999999999999999,"15min":0.75800000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.78100000000000003,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.68000000000000005}]},{"osd":11,"last update":"Fri Dec 1 01:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":3.9689999999999999,"5min":3.9689999999999999,"15min":3.9689999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":0.61699999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411394,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":999424,"data_stored":500465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:17:39 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.503,"15min":0.48199999999999998},"min":{"1min":0.38700000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.84999999999999998,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.497,"15min":0.47899999999999998},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.70199999999999996,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.51000000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:17:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.54200000000000004,"15min":0.51500000000000001},"min":{"1min":0.34799999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.755,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61299999999999999,"15min":0.55000000000000004},"min":{"1min":0.37,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.755,"5min":5.4169999999999998,"15min":5.4169999999999998},"last":0.72599999999999998}]},{"osd":3,"last update":"Fri Dec 1 01:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.59799999999999998,"15min":0.53000000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.91400000000000003,"5min":5.4710000000000001,"15min":5.4710000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56999999999999995,"15min":0.53000000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.78500000000000003,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.626}]},{"osd":4,"last update":"Fri Dec 1 01:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.745},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.85799999999999998}]},{"osd":5,"last update":"Fri Dec 1 01:17:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.45800000000000002,"15min":0.42899999999999999},"min":{"1min":0.39900000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.70799999999999996,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.52000000000000002,"15min":0.47999999999999998},"min":{"1min":0.39300000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.80100000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.80100000000000005}]},{"osd":6,"last update":"Fri Dec 1 01:17:00 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.64500000000000002,"15min":0.60499999999999998},"min":{"1min":0.43099999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.89300000000000002,"5min":5.5019999999999998,"15min":5.5019999999999998},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.63800000000000001,"15min":0.63400000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77100000000000002,"5min":1.429,"15min":1.429},"last":0.67300000000000004}]},{"osd":7,"last update":"Fri Dec 1 01:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.54000000000000004}]},{"osd":8,"last update":"Fri Dec 1 01:17:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.56299999999999994,"15min":0.53500000000000003},"min":{"1min":0.38300000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59199999999999997,"15min":0.58399999999999996},"min":{"1min":0.41299999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.79500000000000004,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.68799999999999994}]},{"osd":9,"last update":"Fri Dec 1 01:17:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.69799999999999995,"15min":0.70099999999999996},"min":{"1min":0.52000000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.004,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.68700000000000006,"15min":0.68200000000000005},"min":{"1min":0.51700000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94299999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.81599999999999995}]},{"osd":10,"last update":"Fri Dec 1 01:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.66200000000000003,"15min":0.65800000000000003},"min":{"1min":0.55900000000000005,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.88200000000000001,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.86799999999999999,"15min":0.88500000000000001},"min":{"1min":0.65800000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.97899999999999998,"5min":5.4450000000000003,"15min":5.4450000000000003},"last":0.89600000000000002}]},{"osd":11,"last update":"Fri Dec 1 01:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.93600000000000005}]}]},{"osd":3,"up_from":25,"seq":107374182462,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":942080,"data_stored":440410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:17:02 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.34300000000000003,"15min":0.31900000000000001},"min":{"1min":0.26800000000000002,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.51300000000000001,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.35699999999999998,"15min":0.33800000000000002},"min":{"1min":0.26600000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.48299999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.32500000000000001}]},{"osd":1,"last update":"Fri Dec 1 01:17:02 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.53500000000000003,"15min":0.505},"min":{"1min":0.38200000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73099999999999998,"5min":0.996,"15min":0.996},"last":0.443},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.375,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.69599999999999995,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.47199999999999998}]},{"osd":2,"last update":"Fri Dec 1 01:17:02 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.51700000000000002},"min":{"1min":0.39700000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.749,"5min":0.88,"15min":0.88},"last":0.495},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55100000000000005,"15min":0.52800000000000002},"min":{"1min":0.41599999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.78600000000000003,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.53200000000000003}]},{"osd":4,"last update":"Fri Dec 1 01:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58499999999999996,"15min":0.55900000000000005},"min":{"1min":0.40100000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78000000000000003,"5min":1.329,"15min":1.329},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61099999999999999,"15min":0.58899999999999997},"min":{"1min":0.42299999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.74099999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.50700000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:17:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.55200000000000005,"15min":0.55300000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.69699999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81100000000000005,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.42599999999999999}]},{"osd":6,"last update":"Fri Dec 1 01:17:02 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.46500000000000002,"15min":0.47299999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.503,"5min":0.55100000000000005,"15min":0.54500000000000004},"min":{"1min":0.34999999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.69899999999999995,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.40200000000000002}]},{"osd":7,"last update":"Fri Dec 1 01:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.63200000000000001,"15min":0.63},"min":{"1min":0.46100000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.84699999999999998,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.65100000000000002,"15min":0.64900000000000002},"min":{"1min":0.439,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83999999999999997,"5min":1.446,"15min":1.446},"last":0.45200000000000001}]},{"osd":8,"last update":"Fri Dec 1 01:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.51200000000000001}]},{"osd":9,"last update":"Fri Dec 1 01:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.71199999999999997,"15min":0.73399999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77300000000000002,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.63600000000000001,"15min":0.65100000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.75,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.54000000000000004}]},{"osd":10,"last update":"Fri Dec 1 01:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.70799999999999996,"15min":0.71399999999999997},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.77800000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.73299999999999998,"15min":0.745},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.90700000000000003,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.52200000000000002}]},{"osd":11,"last update":"Fri Dec 1 01:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.56200000000000006}]}]},{"osd":4,"up_from":32,"seq":137438953530,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":440386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.54300000000000004,"15min":0.50900000000000001},"min":{"1min":0.48799999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.373},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.53400000000000003,"15min":0.499},"min":{"1min":0.47199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.497}]},{"osd":1,"last update":"Fri Dec 1 01:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.41599999999999998,"15min":0.38200000000000001},"min":{"1min":0.28999999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.58199999999999996,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.41099999999999998,"15min":0.375},"min":{"1min":0.32600000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.54300000000000004,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.56299999999999994}]},{"osd":2,"last update":"Fri Dec 1 01:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.59099999999999997,"15min":0.54100000000000004},"min":{"1min":0.40999999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.60999999999999999,"15min":0.57299999999999995},"min":{"1min":0.40600000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.81000000000000005,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.63400000000000001}]},{"osd":3,"last update":"Fri Dec 1 01:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.56100000000000005,"15min":0.50900000000000001},"min":{"1min":0.42999999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.52400000000000002,"15min":0.505},"min":{"1min":0.36099999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.751,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.44500000000000001}]},{"osd":5,"last update":"Fri Dec 1 01:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.59499999999999997,"15min":0.56499999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.67400000000000004,"15min":0.61599999999999999},"min":{"1min":0.46000000000000002,"5min":0.377,"15min":0.377},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.69399999999999995}]},{"osd":6,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.629,"15min":0.61199999999999999},"min":{"1min":0.42999999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.94699999999999995,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.66000000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.51600000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:17:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57899999999999996,"15min":0.55700000000000005},"min":{"1min":0.38,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.73999999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.58199999999999996,"15min":0.55500000000000005},"min":{"1min":0.438,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.872,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.70699999999999996}]},{"osd":8,"last update":"Fri Dec 1 01:16:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.63100000000000001,"15min":0.63600000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.498},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76000000000000001,"15min":0.75900000000000001},"min":{"1min":0.47699999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.59199999999999997}]},{"osd":9,"last update":"Fri Dec 1 01:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.72999999999999998,"15min":0.73499999999999999},"min":{"1min":0.46600000000000003,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.79000000000000004,"15min":0.80000000000000004},"min":{"1min":0.51600000000000001,"5min":0.503,"15min":0.503},"max":{"1min":1.0900000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.66300000000000003}]},{"osd":10,"last update":"Fri Dec 1 01:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.78700000000000003,"15min":0.79900000000000004},"min":{"1min":0.496,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.93700000000000006,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.74199999999999999,"15min":0.74399999999999999},"min":{"1min":0.46999999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90900000000000003,"5min":0.997,"15min":0.997},"last":0.75600000000000001}]},{"osd":11,"last update":"Fri Dec 1 01:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.61299999999999999}]}]},{"osd":5,"up_from":37,"seq":158913790005,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28460,"kb_used_data":1508,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716692,"statfs":{"total":95995035648,"available":95965892608,"internally_reserved":0,"allocated":1544192,"data_stored":1033059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 1 01:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.51200000000000001},"min":{"1min":0.36199999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.54000000000000004,"15min":0.52800000000000002},"min":{"1min":0.34999999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":2.5099999999999998,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":0.61099999999999999}]},{"osd":1,"last update":"Fri Dec 1 01:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.53600000000000003,"15min":0.495},"min":{"1min":0.41999999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.55500000000000005,"15min":0.50900000000000001},"min":{"1min":0.442,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80700000000000005,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.51700000000000002}]},{"osd":2,"last update":"Fri Dec 1 01:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.40000000000000002,"15min":0.38800000000000001},"min":{"1min":0.27100000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.39900000000000002,"15min":0.379},"min":{"1min":0.28499999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.40000000000000002}]},{"osd":3,"last update":"Fri Dec 1 01:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.56999999999999995,"15min":0.54100000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":2.6379999999999999,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.56399999999999995,"15min":0.53700000000000003},"min":{"1min":0.40899999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.825,"5min":1.825,"15min":1.825},"last":0.53200000000000003}]},{"osd":4,"last update":"Fri Dec 1 01:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.57499999999999996,"15min":0.53800000000000003},"min":{"1min":0.35099999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56899999999999995,"15min":0.53500000000000003},"min":{"1min":0.41299999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.56999999999999995}]},{"osd":6,"last update":"Fri Dec 1 01:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.64200000000000002,"15min":0.626},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":2.391,"5min":2.391,"15min":2.391},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.66000000000000003,"15min":0.63},"min":{"1min":0.42899999999999999,"5min":0.37,"15min":0.37},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":0.75600000000000001}]},{"osd":7,"last update":"Fri Dec 1 01:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.66100000000000003,"15min":0.65500000000000003},"min":{"1min":0.55300000000000005,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.62,"15min":0.61399999999999999},"min":{"1min":0.48499999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.84599999999999997,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.58099999999999996}]},{"osd":8,"last update":"Fri Dec 1 01:16:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58099999999999996,"15min":0.57999999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.754,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.48999999999999999}]},{"osd":9,"last update":"Fri Dec 1 01:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.70999999999999996,"15min":0.70199999999999996},"min":{"1min":0.47299999999999998,"5min":0.443,"15min":0.443},"max":{"1min":2.8069999999999999,"5min":2.8069999999999999,"15min":2.8069999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.71799999999999997,"15min":0.70199999999999996},"min":{"1min":0.48199999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":3.9049999999999998,"5min":3.9049999999999998,"15min":3.9049999999999998},"last":0.70399999999999996}]},{"osd":10,"last update":"Fri Dec 1 01:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.68400000000000005,"15min":0.67100000000000004},"min":{"1min":0.54300000000000004,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.68600000000000005,"15min":0.68000000000000005},"min":{"1min":0.57699999999999996,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.85499999999999998,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.74099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":61367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-01T01:17:49.473 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:17:49.632 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.628+0000 7fd9e36cf700 1 -- 172.21.15.77:0/435430878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9dc100da0 msgr2=0x7fd9dc1011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:49.632 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.628+0000 7fd9e36cf700 1 --2- 172.21.15.77:0/435430878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9dc100da0 0x7fd9dc1011c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd9d80099c0 tx=0x7fd9d8009cd0 comp rx=0 tx=0).stop 2023-12-01T01:17:49.632 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.628+0000 7fd9e36cf700 1 -- 172.21.15.77:0/435430878 shutdown_connections 2023-12-01T01:17:49.632 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.628+0000 7fd9e36cf700 1 --2- 172.21.15.77:0/435430878 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9dc102930 0x7fd9dc1091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:49.632 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.628+0000 7fd9e36cf700 1 --2- 172.21.15.77:0/435430878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9dc100da0 0x7fd9dc1011c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:49.632 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.628+0000 7fd9e36cf700 1 --2- 172.21.15.77:0/435430878 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd9dc101f90 0x7fd9dc1023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:49.632 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.628+0000 7fd9e36cf700 1 -- 172.21.15.77:0/435430878 >> 172.21.15.77:0/435430878 conn(0x7fd9dc0fc2d0 msgr2=0x7fd9dc0fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:49.632 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.629+0000 7fd9e36cf700 1 -- 172.21.15.77:0/435430878 shutdown_connections 2023-12-01T01:17:49.632 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.629+0000 7fd9e36cf700 1 -- 172.21.15.77:0/435430878 wait complete. 2023-12-01T01:17:49.633 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.629+0000 7fd9e36cf700 1 Processor -- start 2023-12-01T01:17:49.633 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.629+0000 7fd9e36cf700 1 -- start start 2023-12-01T01:17:49.633 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.629+0000 7fd9e36cf700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9dc100da0 0x7fd9dc074b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:49.633 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.629+0000 7fd9e36cf700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9dc101f90 0x7fd9dc073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:49.633 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.629+0000 7fd9e36cf700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd9dc102930 0x7fd9dc0736c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:49.633 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.629+0000 7fd9e36cf700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc073e60 con 0x7fd9dc100da0 2023-12-01T01:17:49.633 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.629+0000 7fd9e36cf700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc073fd0 con 0x7fd9dc102930 2023-12-01T01:17:49.633 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.629+0000 7fd9e36cf700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc1a0ac0 con 0x7fd9dc101f90 2023-12-01T01:17:49.633 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.629+0000 7fd9e146b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9dc100da0 0x7fd9dc074b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:49.633 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.629+0000 7fd9e146b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9dc100da0 0x7fd9dc074b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59174/0 (socket says 172.21.15.77:59174) 2023-12-01T01:17:49.633 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.629+0000 7fd9e146b700 1 -- 172.21.15.77:0/2237650081 learned_addr learned my addr 172.21.15.77:0/2237650081 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:49.633 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.629+0000 7fd9e0c6a700 1 --2- 172.21.15.77:0/2237650081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9dc101f90 0x7fd9dc073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:49.633 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.630+0000 7fd9e1c6c700 1 --2- 172.21.15.77:0/2237650081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd9dc102930 0x7fd9dc0736c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:49.634 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.630+0000 7fd9e146b700 1 -- 172.21.15.77:0/2237650081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9dc101f90 msgr2=0x7fd9dc073180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:49.634 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.630+0000 7fd9e146b700 1 --2- 172.21.15.77:0/2237650081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9dc101f90 0x7fd9dc073180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:49.634 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.630+0000 7fd9e146b700 1 -- 172.21.15.77:0/2237650081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd9dc102930 msgr2=0x7fd9dc0736c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:49.634 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.630+0000 7fd9e146b700 1 --2- 172.21.15.77:0/2237650081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd9dc102930 0x7fd9dc0736c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:49.634 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.630+0000 7fd9e146b700 1 -- 172.21.15.77:0/2237650081 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d80095f0 con 0x7fd9dc100da0 2023-12-01T01:17:49.634 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.630+0000 7fd9e1c6c700 1 --2- 172.21.15.77:0/2237650081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd9dc102930 0x7fd9dc0736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:17:49.634 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.630+0000 7fd9e146b700 1 --2- 172.21.15.77:0/2237650081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9dc100da0 0x7fd9dc074b40 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8009990 tx=0x7fd9d800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:49.634 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.630+0000 7fd9d27fc700 1 -- 172.21.15.77:0/2237650081 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d801d070 con 0x7fd9dc100da0 2023-12-01T01:17:49.634 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.630+0000 7fd9d27fc700 1 -- 172.21.15.77:0/2237650081 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9d80054f0 con 0x7fd9dc100da0 2023-12-01T01:17:49.634 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.630+0000 7fd9e36cf700 1 -- 172.21.15.77:0/2237650081 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9dc1a0d40 con 0x7fd9dc100da0 2023-12-01T01:17:49.634 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.630+0000 7fd9d27fc700 1 -- 172.21.15.77:0/2237650081 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8004b00 con 0x7fd9dc100da0 2023-12-01T01:17:49.634 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.630+0000 7fd9e36cf700 1 -- 172.21.15.77:0/2237650081 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9dc1a11c0 con 0x7fd9dc100da0 2023-12-01T01:17:49.634 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.631+0000 7fd9e36cf700 1 -- 172.21.15.77:0/2237650081 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9dc10c600 con 0x7fd9dc100da0 2023-12-01T01:17:49.635 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.632+0000 7fd9d27fc700 1 -- 172.21.15.77:0/2237650081 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fd9d800fb80 con 0x7fd9dc100da0 2023-12-01T01:17:49.635 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.632+0000 7fd9d27fc700 1 --2- 172.21.15.77:0/2237650081 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd9c80a0670 0x7fd9c80a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:49.635 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.633+0000 7fd9e0c6a700 1 --2- 172.21.15.77:0/2237650081 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd9c80a0670 0x7fd9c80a2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:49.635 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.633+0000 7fd9d27fc700 1 -- 172.21.15.77:0/2237650081 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fd9d80ccc70 con 0x7fd9dc100da0 2023-12-01T01:17:49.636 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.633+0000 7fd9e0c6a700 1 --2- 172.21.15.77:0/2237650081 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd9c80a0670 0x7fd9c80a2b30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc009c60 tx=0x7fd9cc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:49.637 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.636+0000 7fd9d27fc700 1 -- 172.21.15.77:0/2237650081 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd9d8096860 con 0x7fd9dc100da0 2023-12-01T01:17:49.832 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.831+0000 7fd9e36cf700 1 -- 172.21.15.77:0/2237650081 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9dc107ca0 con 0x7fd9dc100da0 2023-12-01T01:17:49.833 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.832+0000 7fd9d27fc700 1 -- 172.21.15.77:0/2237650081 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22472 (secure 0 0 0) 0x7fd9d8026090 con 0x7fd9dc100da0 2023-12-01T01:17:49.834 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:17:49.834 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":84,"fsid":"2a06e29a-8fe6-11ee-95a2-87774f69a715","created":"2023-12-01T01:09:30.210398+0000","modified":"2023-12-01T01:17:25.609766+0000","last_up_change":"2023-12-01T01:15:44.586092+0000","last_in_change":"2023-12-01T01:15:26.305439+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:12:30.514229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:16:23.568860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-01T01:16:24.531137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fc651e62-a103-4ab8-a76d-d3708ea6aeec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6803","nonce":2244799094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6805","nonce":2244799094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6809","nonce":2244799094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6807","nonce":2244799094}]},"public_addr":"172.21.15.77:6803/2244799094","cluster_addr":"172.21.15.77:6805/2244799094","heartbeat_back_addr":"172.21.15.77:6809/2244799094","heartbeat_front_addr":"172.21.15.77:6807/2244799094","state":["exists","up"]},{"osd":1,"uuid":"0adf8c54-49ec-49a0-8bf1-af1f20e60729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6801","nonce":224194890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6803","nonce":224194890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6807","nonce":224194890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6805","nonce":224194890}]},"public_addr":"172.21.15.137:6801/224194890","cluster_addr":"172.21.15.137:6803/224194890","heartbeat_back_addr":"172.21.15.137:6807/224194890","heartbeat_front_addr":"172.21.15.137:6805/224194890","state":["exists","up"]},{"osd":2,"uuid":"5a69802c-dede-4544-9a12-436f5c1c0038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6801","nonce":4078459110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6803","nonce":4078459110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6807","nonce":4078459110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6805","nonce":4078459110}]},"public_addr":"172.21.15.203:6801/4078459110","cluster_addr":"172.21.15.203:6803/4078459110","heartbeat_back_addr":"172.21.15.203:6807/4078459110","heartbeat_front_addr":"172.21.15.203:6805/4078459110","state":["exists","up"]},{"osd":3,"uuid":"ad325a97-580b-4c99-bf12-cbb0d92a6d0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6811","nonce":1454440477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6813","nonce":1454440477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6817","nonce":1454440477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6815","nonce":1454440477}]},"public_addr":"172.21.15.77:6811/1454440477","cluster_addr":"172.21.15.77:6813/1454440477","heartbeat_back_addr":"172.21.15.77:6817/1454440477","heartbeat_front_addr":"172.21.15.77:6815/1454440477","state":["exists","up"]},{"osd":4,"uuid":"1e3ac1ba-98bb-4b8b-8e26-50064c66bf2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6809","nonce":1103241451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6811","nonce":1103241451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6815","nonce":1103241451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6813","nonce":1103241451}]},"public_addr":"172.21.15.137:6809/1103241451","cluster_addr":"172.21.15.137:6811/1103241451","heartbeat_back_addr":"172.21.15.137:6815/1103241451","heartbeat_front_addr":"172.21.15.137:6813/1103241451","state":["exists","up"]},{"osd":5,"uuid":"eb32c016-abeb-44ab-abf0-19dd09235736","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6809","nonce":2648569482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6811","nonce":2648569482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6815","nonce":2648569482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6813","nonce":2648569482}]},"public_addr":"172.21.15.203:6809/2648569482","cluster_addr":"172.21.15.203:6811/2648569482","heartbeat_back_addr":"172.21.15.203:6815/2648569482","heartbeat_front_addr":"172.21.15.203:6813/2648569482","state":["exists","up"]},{"osd":6,"uuid":"0e3fa97e-2ca6-418c-9e8d-a6fe3c56adb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6819","nonce":2463414651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6821","nonce":2463414651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6825","nonce":2463414651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6823","nonce":2463414651}]},"public_addr":"172.21.15.77:6819/2463414651","cluster_addr":"172.21.15.77:6821/2463414651","heartbeat_back_addr":"172.21.15.77:6825/2463414651","heartbeat_front_addr":"172.21.15.77:6823/2463414651","state":["exists","up"]},{"osd":7,"uuid":"d7398c01-0f98-4138-8a49-c3ff0aae7e88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6817","nonce":3108330766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6819","nonce":3108330766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6823","nonce":3108330766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6821","nonce":3108330766}]},"public_addr":"172.21.15.137:6817/3108330766","cluster_addr":"172.21.15.137:6819/3108330766","heartbeat_back_addr":"172.21.15.137:6823/3108330766","heartbeat_front_addr":"172.21.15.137:6821/3108330766","state":["exists","up"]},{"osd":8,"uuid":"0c09050d-3e43-410c-8cd7-bcb891af56b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6817","nonce":1161158776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6819","nonce":1161158776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6823","nonce":1161158776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6821","nonce":1161158776}]},"public_addr":"172.21.15.203:6817/1161158776","cluster_addr":"172.21.15.203:6819/1161158776","heartbeat_back_addr":"172.21.15.203:6823/1161158776","heartbeat_front_addr":"172.21.15.203:6821/1161158776","state":["exists","up"]},{"osd":9,"uuid":"527ecb49-95ec-48bd-9872-3b07fae5c5ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6827","nonce":2409880685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6829","nonce":2409880685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6833","nonce":2409880685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6831","nonce":2409880685}]},"public_addr":"172.21.15.77:6827/2409880685","cluster_addr":"172.21.15.77:6829/2409880685","heartbeat_back_addr":"172.21.15.77:6833/2409880685","heartbeat_front_addr":"172.21.15.77:6831/2409880685","state":["exists","up"]},{"osd":10,"uuid":"1680bb3e-a3c8-427b-af68-e2214f32bed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6825","nonce":374920146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6827","nonce":374920146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6831","nonce":374920146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6829","nonce":374920146}]},"public_addr":"172.21.15.137:6825/374920146","cluster_addr":"172.21.15.137:6827/374920146","heartbeat_back_addr":"172.21.15.137:6831/374920146","heartbeat_front_addr":"172.21.15.137:6829/374920146","state":["exists","up"]},{"osd":11,"uuid":"8a533466-023d-4803-ae36-9a9f1715e49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6825","nonce":2161588962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6827","nonce":2161588962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6831","nonce":2161588962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6829","nonce":2161588962}]},"public_addr":"172.21.15.203:6825/2161588962","cluster_addr":"172.21.15.203:6827/2161588962","heartbeat_back_addr":"172.21.15.203:6831/2161588962","heartbeat_front_addr":"172.21.15.203:6829/2161588962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:11:45.420509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:06.210115+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:28.191307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:49.594116+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:11.261689+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:33.376541+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:54.968336+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:16.775310+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:38.733792+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:59.456699+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:15:20.924059+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:15:43.094773+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/2260035254":"2023-12-02T01:10:24.191835+0000","172.21.15.77:6800/1456621747":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/219838828":"2023-12-02T01:10:24.191835+0000","172.21.15.77:6801/1456621747":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/2937149266":"2023-12-02T01:10:02.232948+0000","172.21.15.77:0/807520018":"2023-12-02T01:10:02.232948+0000","172.21.15.77:6801/1661788026":"2023-12-02T01:10:02.232948+0000","172.21.15.77:6800/1661788026":"2023-12-02T01:10:02.232948+0000","172.21.15.77:0/3551282473":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/1202222558":"2023-12-02T01:10:02.232948+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:17:49.836 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.834+0000 7fd9e36cf700 1 -- 172.21.15.77:0/2237650081 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd9c80a0670 msgr2=0x7fd9c80a2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:49.836 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.834+0000 7fd9e36cf700 1 --2- 172.21.15.77:0/2237650081 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd9c80a0670 0x7fd9c80a2b30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc009c60 tx=0x7fd9cc009260 comp rx=0 tx=0).stop 2023-12-01T01:17:49.836 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.834+0000 7fd9e36cf700 1 -- 172.21.15.77:0/2237650081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9dc100da0 msgr2=0x7fd9dc074b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:49.836 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.834+0000 7fd9e36cf700 1 --2- 172.21.15.77:0/2237650081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9dc100da0 0x7fd9dc074b40 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8009990 tx=0x7fd9d800f690 comp rx=0 tx=0).stop 2023-12-01T01:17:49.836 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.834+0000 7fd9e36cf700 1 -- 172.21.15.77:0/2237650081 shutdown_connections 2023-12-01T01:17:49.836 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.834+0000 7fd9e36cf700 1 --2- 172.21.15.77:0/2237650081 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd9c80a0670 0x7fd9c80a2b30 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:49.836 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.834+0000 7fd9e36cf700 1 --2- 172.21.15.77:0/2237650081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd9dc102930 0x7fd9dc0736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:49.836 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.834+0000 7fd9e36cf700 1 --2- 172.21.15.77:0/2237650081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9dc100da0 0x7fd9dc074b40 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:49.836 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.834+0000 7fd9e36cf700 1 --2- 172.21.15.77:0/2237650081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9dc101f90 0x7fd9dc073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:49.836 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.834+0000 7fd9e36cf700 1 -- 172.21.15.77:0/2237650081 >> 172.21.15.77:0/2237650081 conn(0x7fd9dc0fc2d0 msgr2=0x7fd9dc0fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:49.836 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.834+0000 7fd9e36cf700 1 -- 172.21.15.77:0/2237650081 shutdown_connections 2023-12-01T01:17:49.836 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.834+0000 7fd9e36cf700 1 -- 172.21.15.77:0/2237650081 wait complete. 2023-12-01T01:17:49.846 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:49 smithi077 ceph-mon[98867]: pgmap v296: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 11 op/s 2023-12-01T01:17:49.848 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:12:30.514229+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T01:16:23.568860+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T01:16:24.531137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:17:49.848 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:17:49.974 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.971+0000 7fad16053700 1 -- 172.21.15.77:0/1119260546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad10101840 msgr2=0x7fad101080f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:49.974 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.971+0000 7fad16053700 1 --2- 172.21.15.77:0/1119260546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad10101840 0x7fad101080f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fad0800b0d0 tx=0x7fad0800b3e0 comp rx=0 tx=0).stop 2023-12-01T01:17:49.974 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.971+0000 7fad16053700 1 -- 172.21.15.77:0/1119260546 shutdown_connections 2023-12-01T01:17:49.974 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.971+0000 7fad16053700 1 --2- 172.21.15.77:0/1119260546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad10101840 0x7fad101080f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:49.974 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.971+0000 7fad16053700 1 --2- 172.21.15.77:0/1119260546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad100ffcb0 0x7fad101000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:49.974 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.971+0000 7fad16053700 1 --2- 172.21.15.77:0/1119260546 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad10100ea0 0x7fad10101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:49.974 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.971+0000 7fad16053700 1 -- 172.21.15.77:0/1119260546 >> 172.21.15.77:0/1119260546 conn(0x7fad100fb220 msgr2=0x7fad100fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:49.975 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.971+0000 7fad16053700 1 -- 172.21.15.77:0/1119260546 shutdown_connections 2023-12-01T01:17:49.975 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.971+0000 7fad16053700 1 -- 172.21.15.77:0/1119260546 wait complete. 2023-12-01T01:17:49.975 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.971+0000 7fad16053700 1 Processor -- start 2023-12-01T01:17:49.975 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.971+0000 7fad16053700 1 -- start start 2023-12-01T01:17:49.975 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.972+0000 7fad16053700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad100ffcb0 0x7fad10069a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:49.975 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.972+0000 7fad16053700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad10100ea0 0x7fad1019b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:49.975 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.972+0000 7fad16053700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad10101840 0x7fad1019df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:49.975 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.972+0000 7fad16053700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1019e6f0 con 0x7fad100ffcb0 2023-12-01T01:17:49.975 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.972+0000 7fad16053700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1019e860 con 0x7fad10100ea0 2023-12-01T01:17:49.975 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.972+0000 7fad16053700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1019ea20 con 0x7fad10101840 2023-12-01T01:17:49.975 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.972+0000 7fad0f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad100ffcb0 0x7fad10069a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:49.975 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.972+0000 7fad0f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad100ffcb0 0x7fad10069a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59188/0 (socket says 172.21.15.77:59188) 2023-12-01T01:17:49.975 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.972+0000 7fad0f7fe700 1 -- 172.21.15.77:0/2081573446 learned_addr learned my addr 172.21.15.77:0/2081573446 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:49.975 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.972+0000 7fad0effd700 1 --2- 172.21.15.77:0/2081573446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad10100ea0 0x7fad1019b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:49.976 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.972+0000 7fad0ffff700 1 --2- 172.21.15.77:0/2081573446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad10101840 0x7fad1019df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:49.976 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.972+0000 7fad0f7fe700 1 -- 172.21.15.77:0/2081573446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad10101840 msgr2=0x7fad1019df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:49.976 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.972+0000 7fad0f7fe700 1 --2- 172.21.15.77:0/2081573446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad10101840 0x7fad1019df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:49.976 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.972+0000 7fad0f7fe700 1 -- 172.21.15.77:0/2081573446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad10100ea0 msgr2=0x7fad1019b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:49.976 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.972+0000 7fad0f7fe700 1 --2- 172.21.15.77:0/2081573446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad10100ea0 0x7fad1019b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:49.976 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.972+0000 7fad0f7fe700 1 -- 172.21.15.77:0/2081573446 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad08009d00 con 0x7fad100ffcb0 2023-12-01T01:17:49.976 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.972+0000 7fad0f7fe700 1 --2- 172.21.15.77:0/2081573446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad100ffcb0 0x7fad10069a50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fad0000d980 tx=0x7fad0000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:49.976 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.972+0000 7fad0cff9700 1 -- 172.21.15.77:0/2081573446 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad00004020 con 0x7fad100ffcb0 2023-12-01T01:17:49.976 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.972+0000 7fad0cff9700 1 -- 172.21.15.77:0/2081573446 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad00009730 con 0x7fad100ffcb0 2023-12-01T01:17:49.976 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.972+0000 7fad16053700 1 -- 172.21.15.77:0/2081573446 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad1019ed00 con 0x7fad100ffcb0 2023-12-01T01:17:49.976 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.972+0000 7fad0cff9700 1 -- 172.21.15.77:0/2081573446 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad00003a80 con 0x7fad100ffcb0 2023-12-01T01:17:49.976 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.972+0000 7fad16053700 1 -- 172.21.15.77:0/2081573446 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad1019f1c0 con 0x7fad100ffcb0 2023-12-01T01:17:49.977 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.973+0000 7fad16053700 1 -- 172.21.15.77:0/2081573446 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad1010b520 con 0x7fad100ffcb0 2023-12-01T01:17:49.977 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.974+0000 7fad0cff9700 1 -- 172.21.15.77:0/2081573446 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fad000098a0 con 0x7fad100ffcb0 2023-12-01T01:17:49.977 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.975+0000 7fad0cff9700 1 --2- 172.21.15.77:0/2081573446 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7facf80a05b0 0x7facf80a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:49.977 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.975+0000 7fad0effd700 1 --2- 172.21.15.77:0/2081573446 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7facf80a05b0 0x7facf80a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:49.977 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.975+0000 7fad0cff9700 1 -- 172.21.15.77:0/2081573446 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fad000cba60 con 0x7fad100ffcb0 2023-12-01T01:17:49.978 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.976+0000 7fad0effd700 1 --2- 172.21.15.77:0/2081573446 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7facf80a05b0 0x7facf80a2a70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fad100697c0 tx=0x7fad0400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:49.980 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:49.979+0000 7fad0cff9700 1 -- 172.21.15.77:0/2081573446 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fad000955a0 con 0x7fad100ffcb0 2023-12-01T01:17:50.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:49 smithi203 ceph-mon[103581]: pgmap v296: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 11 op/s 2023-12-01T01:17:50.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:49 smithi137 ceph-mon[103312]: pgmap v296: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 11 op/s 2023-12-01T01:17:50.172 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.171+0000 7fad16053700 1 -- 172.21.15.77:0/2081573446 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fad1010b710 con 0x7fad100ffcb0 2023-12-01T01:17:50.173 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.171+0000 7fad0cff9700 1 -- 172.21.15.77:0/2081573446 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7fad00017020 con 0x7fad100ffcb0 2023-12-01T01:17:50.173 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 1 2023-12-01T01:17:50.175 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.173+0000 7fad16053700 1 -- 172.21.15.77:0/2081573446 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7facf80a05b0 msgr2=0x7facf80a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:50.175 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.173+0000 7fad16053700 1 --2- 172.21.15.77:0/2081573446 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7facf80a05b0 0x7facf80a2a70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fad100697c0 tx=0x7fad0400b410 comp rx=0 tx=0).stop 2023-12-01T01:17:50.175 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.173+0000 7fad16053700 1 -- 172.21.15.77:0/2081573446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad100ffcb0 msgr2=0x7fad10069a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:50.175 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.173+0000 7fad16053700 1 --2- 172.21.15.77:0/2081573446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad100ffcb0 0x7fad10069a50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fad0000d980 tx=0x7fad0000dc90 comp rx=0 tx=0).stop 2023-12-01T01:17:50.175 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.173+0000 7fad16053700 1 -- 172.21.15.77:0/2081573446 shutdown_connections 2023-12-01T01:17:50.175 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.173+0000 7fad16053700 1 --2- 172.21.15.77:0/2081573446 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7facf80a05b0 0x7facf80a2a70 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:50.175 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.173+0000 7fad16053700 1 --2- 172.21.15.77:0/2081573446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad10101840 0x7fad1019df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:50.175 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.173+0000 7fad16053700 1 --2- 172.21.15.77:0/2081573446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad100ffcb0 0x7fad10069a50 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:50.175 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.173+0000 7fad16053700 1 --2- 172.21.15.77:0/2081573446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad10100ea0 0x7fad1019b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:50.175 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.173+0000 7fad16053700 1 -- 172.21.15.77:0/2081573446 >> 172.21.15.77:0/2081573446 conn(0x7fad100fb220 msgr2=0x7fad100fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:50.175 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.174+0000 7fad16053700 1 -- 172.21.15.77:0/2081573446 shutdown_connections 2023-12-01T01:17:50.176 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.174+0000 7fad16053700 1 -- 172.21.15.77:0/2081573446 wait complete. 2023-12-01T01:17:50.187 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T01:17:50.313 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.310+0000 7fece06d2700 1 -- 172.21.15.77:0/4172947003 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fecd81041a0 msgr2=0x7fecd8106590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:50.313 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.310+0000 7fece06d2700 1 --2- 172.21.15.77:0/4172947003 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fecd81041a0 0x7fecd8106590 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fecc8009990 tx=0x7fecc8009ca0 comp rx=0 tx=0).stop 2023-12-01T01:17:50.313 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.310+0000 7fece06d2700 1 -- 172.21.15.77:0/4172947003 shutdown_connections 2023-12-01T01:17:50.313 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.310+0000 7fece06d2700 1 --2- 172.21.15.77:0/4172947003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecd8106ad0 0x7fecd8108ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:50.313 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.310+0000 7fece06d2700 1 --2- 172.21.15.77:0/4172947003 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fecd8101870 0x7fecd8103c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:50.313 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.310+0000 7fece06d2700 1 --2- 172.21.15.77:0/4172947003 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fecd81041a0 0x7fecd8106590 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:50.314 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.310+0000 7fece06d2700 1 -- 172.21.15.77:0/4172947003 >> 172.21.15.77:0/4172947003 conn(0x7fecd80fb220 msgr2=0x7fecd80fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:50.314 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.310+0000 7fece06d2700 1 -- 172.21.15.77:0/4172947003 shutdown_connections 2023-12-01T01:17:50.314 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.310+0000 7fece06d2700 1 -- 172.21.15.77:0/4172947003 wait complete. 2023-12-01T01:17:50.314 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.310+0000 7fece06d2700 1 Processor -- start 2023-12-01T01:17:50.314 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.310+0000 7fece06d2700 1 -- start start 2023-12-01T01:17:50.314 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.310+0000 7fece06d2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fecd8101870 0x7fecd8196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:50.315 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.310+0000 7fece06d2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fecd81041a0 0x7fecd8196f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:50.315 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.310+0000 7fece06d2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecd8106ad0 0x7fecd819dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:50.315 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.310+0000 7fece06d2700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd819e5a0 con 0x7fecd8101870 2023-12-01T01:17:50.315 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.310+0000 7fece06d2700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd819e6e0 con 0x7fecd8106ad0 2023-12-01T01:17:50.315 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.310+0000 7fece06d2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd819e870 con 0x7fecd81041a0 2023-12-01T01:17:50.315 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.311+0000 7fecde46e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fecd8101870 0x7fecd8196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:50.315 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.311+0000 7fecde46e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fecd8101870 0x7fecd8196a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59202/0 (socket says 172.21.15.77:59202) 2023-12-01T01:17:50.315 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.311+0000 7fecde46e700 1 -- 172.21.15.77:0/795714547 learned_addr learned my addr 172.21.15.77:0/795714547 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:50.315 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.311+0000 7fecddc6d700 1 --2- 172.21.15.77:0/795714547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fecd81041a0 0x7fecd8196f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:50.315 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.311+0000 7fecdec6f700 1 --2- 172.21.15.77:0/795714547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecd8106ad0 0x7fecd819dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:50.315 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.311+0000 7fecde46e700 1 -- 172.21.15.77:0/795714547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fecd81041a0 msgr2=0x7fecd8196f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:50.315 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.311+0000 7fecde46e700 1 --2- 172.21.15.77:0/795714547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fecd81041a0 0x7fecd8196f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:50.316 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.311+0000 7fecde46e700 1 -- 172.21.15.77:0/795714547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecd8106ad0 msgr2=0x7fecd819dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:50.316 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.311+0000 7fecde46e700 1 --2- 172.21.15.77:0/795714547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecd8106ad0 0x7fecd819dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:50.316 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.311+0000 7fecde46e700 1 -- 172.21.15.77:0/795714547 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc80095f0 con 0x7fecd8101870 2023-12-01T01:17:50.316 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.311+0000 7fecde46e700 1 --2- 172.21.15.77:0/795714547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fecd8101870 0x7fecd8196a20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fecd400e9a0 tx=0x7fecd400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:50.316 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.311+0000 7feccf7fe700 1 -- 172.21.15.77:0/795714547 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd4009790 con 0x7fecd8101870 2023-12-01T01:17:50.316 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.311+0000 7feccf7fe700 1 -- 172.21.15.77:0/795714547 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fecd4010430 con 0x7fecd8101870 2023-12-01T01:17:50.316 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.311+0000 7fece06d2700 1 -- 172.21.15.77:0/795714547 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecd819eb50 con 0x7fecd8101870 2023-12-01T01:17:50.316 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.311+0000 7feccf7fe700 1 -- 172.21.15.77:0/795714547 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd4018680 con 0x7fecd8101870 2023-12-01T01:17:50.316 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.311+0000 7fece06d2700 1 -- 172.21.15.77:0/795714547 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecd819f060 con 0x7fecd8101870 2023-12-01T01:17:50.317 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.312+0000 7fece06d2700 1 -- 172.21.15.77:0/795714547 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecd80fcda0 con 0x7fecd8101870 2023-12-01T01:17:50.317 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.313+0000 7feccf7fe700 1 -- 172.21.15.77:0/795714547 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7fecd40105a0 con 0x7fecd8101870 2023-12-01T01:17:50.317 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.314+0000 7feccf7fe700 1 --2- 172.21.15.77:0/795714547 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fecc40a05b0 0x7fecc40a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:50.317 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.314+0000 7fecddc6d700 1 --2- 172.21.15.77:0/795714547 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fecc40a05b0 0x7fecc40a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:50.317 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.314+0000 7feccf7fe700 1 -- 172.21.15.77:0/795714547 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fecd4014070 con 0x7fecd8101870 2023-12-01T01:17:50.318 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.314+0000 7fecddc6d700 1 --2- 172.21.15.77:0/795714547 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fecc40a05b0 0x7fecc40a2a70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fecc800b2e0 tx=0x7fecc8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:50.318 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.317+0000 7feccf7fe700 1 -- 172.21.15.77:0/795714547 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fecd40958f0 con 0x7fecd8101870 2023-12-01T01:17:50.507 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.505+0000 7fece06d2700 1 -- 172.21.15.77:0/795714547 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fecd802cf20 con 0x7fecd8101870 2023-12-01T01:17:50.507 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.506+0000 7feccf7fe700 1 -- 172.21.15.77:0/795714547 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7fecd4098f10 con 0x7fecd8101870 2023-12-01T01:17:50.507 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 32 2023-12-01T01:17:50.509 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.508+0000 7fece06d2700 1 -- 172.21.15.77:0/795714547 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fecc40a05b0 msgr2=0x7fecc40a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:50.509 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.508+0000 7fece06d2700 1 --2- 172.21.15.77:0/795714547 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fecc40a05b0 0x7fecc40a2a70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fecc800b2e0 tx=0x7fecc8005fb0 comp rx=0 tx=0).stop 2023-12-01T01:17:50.510 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.508+0000 7fece06d2700 1 -- 172.21.15.77:0/795714547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fecd8101870 msgr2=0x7fecd8196a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:50.510 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.508+0000 7fece06d2700 1 --2- 172.21.15.77:0/795714547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fecd8101870 0x7fecd8196a20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fecd400e9a0 tx=0x7fecd400ecb0 comp rx=0 tx=0).stop 2023-12-01T01:17:50.510 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.508+0000 7fece06d2700 1 -- 172.21.15.77:0/795714547 shutdown_connections 2023-12-01T01:17:50.510 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.508+0000 7fece06d2700 1 --2- 172.21.15.77:0/795714547 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fecc40a05b0 0x7fecc40a2a70 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:50.510 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.508+0000 7fece06d2700 1 --2- 172.21.15.77:0/795714547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fecd8106ad0 0x7fecd819dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:50.510 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.508+0000 7fece06d2700 1 --2- 172.21.15.77:0/795714547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fecd8101870 0x7fecd8196a20 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:50.510 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.508+0000 7fece06d2700 1 --2- 172.21.15.77:0/795714547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fecd81041a0 0x7fecd8196f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:50.510 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.508+0000 7fece06d2700 1 -- 172.21.15.77:0/795714547 >> 172.21.15.77:0/795714547 conn(0x7fecd80fb220 msgr2=0x7fecd81024e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:50.510 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.508+0000 7fece06d2700 1 -- 172.21.15.77:0/795714547 shutdown_connections 2023-12-01T01:17:50.510 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.508+0000 7fece06d2700 1 -- 172.21.15.77:0/795714547 wait complete. 2023-12-01T01:17:50.522 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T01:17:50.646 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.645+0000 7f8c48d46700 1 -- 172.21.15.77:0/2369461032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c44104130 msgr2=0x7f8c44106520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:50.646 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.645+0000 7f8c48d46700 1 --2- 172.21.15.77:0/2369461032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c44104130 0x7f8c44106520 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8c34009990 tx=0x7f8c34009ca0 comp rx=0 tx=0).stop 2023-12-01T01:17:50.647 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.645+0000 7f8c48d46700 1 -- 172.21.15.77:0/2369461032 shutdown_connections 2023-12-01T01:17:50.647 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.645+0000 7f8c48d46700 1 --2- 172.21.15.77:0/2369461032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c44106a60 0x7f8c44108ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:50.647 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.645+0000 7f8c48d46700 1 --2- 172.21.15.77:0/2369461032 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c44101800 0x7f8c44103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:50.647 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.645+0000 7f8c48d46700 1 --2- 172.21.15.77:0/2369461032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c44104130 0x7f8c44106520 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:50.647 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.645+0000 7f8c48d46700 1 -- 172.21.15.77:0/2369461032 >> 172.21.15.77:0/2369461032 conn(0x7f8c440fb1f0 msgr2=0x7f8c440fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:50.647 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.645+0000 7f8c48d46700 1 -- 172.21.15.77:0/2369461032 shutdown_connections 2023-12-01T01:17:50.647 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.645+0000 7f8c48d46700 1 -- 172.21.15.77:0/2369461032 wait complete. 2023-12-01T01:17:50.649 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.646+0000 7f8c48d46700 1 Processor -- start 2023-12-01T01:17:50.649 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.646+0000 7f8c48d46700 1 -- start start 2023-12-01T01:17:50.649 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.646+0000 7f8c48d46700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c44101800 0x7f8c44196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:50.649 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.646+0000 7f8c48d46700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c44104130 0x7f8c44196f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:50.650 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.646+0000 7f8c48d46700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c44106a60 0x7f8c4419dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:50.650 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.646+0000 7f8c48d46700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4419e580 con 0x7f8c44106a60 2023-12-01T01:17:50.650 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.646+0000 7f8c48d46700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4419e6c0 con 0x7f8c44104130 2023-12-01T01:17:50.650 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.646+0000 7f8c48d46700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4419e850 con 0x7f8c44101800 2023-12-01T01:17:50.650 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.646+0000 7f8c42d9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c44106a60 0x7f8c4419dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:50.650 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.646+0000 7f8c42d9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c44106a60 0x7f8c4419dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59220/0 (socket says 172.21.15.77:59220) 2023-12-01T01:17:50.650 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.646+0000 7f8c42d9d700 1 -- 172.21.15.77:0/1237220698 learned_addr learned my addr 172.21.15.77:0/1237220698 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:50.650 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.646+0000 7f8c41d9b700 1 --2- 172.21.15.77:0/1237220698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c44104130 0x7f8c44196f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:50.650 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.646+0000 7f8c4259c700 1 --2- 172.21.15.77:0/1237220698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c44101800 0x7f8c44196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:50.650 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.646+0000 7f8c42d9d700 1 -- 172.21.15.77:0/1237220698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c44101800 msgr2=0x7f8c44196a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:50.650 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.646+0000 7f8c42d9d700 1 --2- 172.21.15.77:0/1237220698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c44101800 0x7f8c44196a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:50.650 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.646+0000 7f8c42d9d700 1 -- 172.21.15.77:0/1237220698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c44104130 msgr2=0x7f8c44196f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:50.650 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.646+0000 7f8c42d9d700 1 --2- 172.21.15.77:0/1237220698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c44104130 0x7f8c44196f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:50.650 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.646+0000 7f8c42d9d700 1 -- 172.21.15.77:0/1237220698 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c340095f0 con 0x7f8c44106a60 2023-12-01T01:17:50.651 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.646+0000 7f8c4259c700 1 --2- 172.21.15.77:0/1237220698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c44101800 0x7f8c44196a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:17:50.651 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.646+0000 7f8c41d9b700 1 --2- 172.21.15.77:0/1237220698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c44104130 0x7f8c44196f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:17:50.651 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.646+0000 7f8c42d9d700 1 --2- 172.21.15.77:0/1237220698 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c44106a60 0x7f8c4419dfb0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c007ee0 tx=0x7f8c3c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:50.651 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.647+0000 7f8c337fe700 1 -- 172.21.15.77:0/1237220698 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c017070 con 0x7f8c44106a60 2023-12-01T01:17:50.651 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.647+0000 7f8c337fe700 1 -- 172.21.15.77:0/1237220698 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c3c00f040 con 0x7f8c44106a60 2023-12-01T01:17:50.651 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.647+0000 7f8c48d46700 1 -- 172.21.15.77:0/1237220698 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c4419eb30 con 0x7f8c44106a60 2023-12-01T01:17:50.651 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.647+0000 7f8c337fe700 1 -- 172.21.15.77:0/1237220698 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c005740 con 0x7f8c44106a60 2023-12-01T01:17:50.651 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.647+0000 7f8c48d46700 1 -- 172.21.15.77:0/1237220698 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c4419eff0 con 0x7f8c44106a60 2023-12-01T01:17:50.651 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.648+0000 7f8c48d46700 1 -- 172.21.15.77:0/1237220698 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c440fcd70 con 0x7f8c44106a60 2023-12-01T01:17:50.652 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.649+0000 7f8c337fe700 1 -- 172.21.15.77:0/1237220698 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f8c3c0058a0 con 0x7f8c44106a60 2023-12-01T01:17:50.652 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.649+0000 7f8c337fe700 1 --2- 172.21.15.77:0/1237220698 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8c2c0a05b0 0x7f8c2c0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:50.652 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.649+0000 7f8c4259c700 1 --2- 172.21.15.77:0/1237220698 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8c2c0a05b0 0x7f8c2c0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:50.652 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.649+0000 7f8c337fe700 1 -- 172.21.15.77:0/1237220698 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f8c3c0cb960 con 0x7f8c44106a60 2023-12-01T01:17:50.652 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.650+0000 7f8c4259c700 1 --2- 172.21.15.77:0/1237220698 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8c2c0a05b0 0x7f8c2c0a2a70 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8c38009be0 tx=0x7f8c38009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:50.654 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.653+0000 7f8c337fe700 1 -- 172.21.15.77:0/1237220698 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8c3c0954a0 con 0x7f8c44106a60 2023-12-01T01:17:50.844 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.843+0000 7f8c48d46700 1 -- 172.21.15.77:0/1237220698 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8c4402cec0 con 0x7f8c44106a60 2023-12-01T01:17:50.845 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.844+0000 7f8c337fe700 1 -- 172.21.15.77:0/1237220698 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8c3c098ac0 con 0x7f8c44106a60 2023-12-01T01:17:50.845 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 64 2023-12-01T01:17:50.847 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.846+0000 7f8c48d46700 1 -- 172.21.15.77:0/1237220698 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8c2c0a05b0 msgr2=0x7f8c2c0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:50.848 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.846+0000 7f8c48d46700 1 --2- 172.21.15.77:0/1237220698 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8c2c0a05b0 0x7f8c2c0a2a70 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8c38009be0 tx=0x7f8c38009260 comp rx=0 tx=0).stop 2023-12-01T01:17:50.848 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.846+0000 7f8c48d46700 1 -- 172.21.15.77:0/1237220698 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c44106a60 msgr2=0x7f8c4419dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:50.848 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.846+0000 7f8c48d46700 1 --2- 172.21.15.77:0/1237220698 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c44106a60 0x7f8c4419dfb0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c007ee0 tx=0x7f8c3c00d3b0 comp rx=0 tx=0).stop 2023-12-01T01:17:50.848 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.846+0000 7f8c48d46700 1 -- 172.21.15.77:0/1237220698 shutdown_connections 2023-12-01T01:17:50.848 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.846+0000 7f8c48d46700 1 --2- 172.21.15.77:0/1237220698 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8c2c0a05b0 0x7f8c2c0a2a70 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:50.848 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.846+0000 7f8c48d46700 1 --2- 172.21.15.77:0/1237220698 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c44106a60 0x7f8c4419dfb0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:50.848 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.846+0000 7f8c48d46700 1 --2- 172.21.15.77:0/1237220698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c44101800 0x7f8c44196a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:50.848 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.846+0000 7f8c48d46700 1 --2- 172.21.15.77:0/1237220698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c44104130 0x7f8c44196f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:50.848 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.846+0000 7f8c48d46700 1 -- 172.21.15.77:0/1237220698 >> 172.21.15.77:0/1237220698 conn(0x7f8c440fb1f0 msgr2=0x7f8c44102470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:50.848 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.846+0000 7f8c48d46700 1 -- 172.21.15.77:0/1237220698 shutdown_connections 2023-12-01T01:17:50.848 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.846+0000 7f8c48d46700 1 -- 172.21.15.77:0/1237220698 wait complete. 2023-12-01T01:17:50.859 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:50 smithi077 ceph-mon[98867]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-01T01:17:50.859 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:50 smithi077 ceph-mon[98867]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-01T01:17:50.859 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:50 smithi077 ceph-mon[98867]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-01T01:17:50.859 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:50 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/2237650081' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:17:50.859 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:50 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/2081573446' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-01T01:17:50.859 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:50 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/795714547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-01T01:17:50.860 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:17:50.983 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.980+0000 7f195a2a3700 1 -- 172.21.15.77:0/3896585017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1954101840 msgr2=0x7f19541080f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:50.983 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.980+0000 7f195a2a3700 1 --2- 172.21.15.77:0/3896585017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1954101840 0x7f19541080f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f194c00b0d0 tx=0x7f194c00b3e0 comp rx=0 tx=0).stop 2023-12-01T01:17:50.984 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.980+0000 7f195a2a3700 1 -- 172.21.15.77:0/3896585017 shutdown_connections 2023-12-01T01:17:50.984 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.980+0000 7f195a2a3700 1 --2- 172.21.15.77:0/3896585017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1954101840 0x7f19541080f0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:50.984 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.980+0000 7f195a2a3700 1 --2- 172.21.15.77:0/3896585017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f19540ffcb0 0x7f19541000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:50.984 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.980+0000 7f195a2a3700 1 --2- 172.21.15.77:0/3896585017 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1954100ea0 0x7f1954101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:50.984 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.980+0000 7f195a2a3700 1 -- 172.21.15.77:0/3896585017 >> 172.21.15.77:0/3896585017 conn(0x7f19540fb220 msgr2=0x7f19540fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:50.984 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.980+0000 7f195a2a3700 1 -- 172.21.15.77:0/3896585017 shutdown_connections 2023-12-01T01:17:50.984 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.980+0000 7f195a2a3700 1 -- 172.21.15.77:0/3896585017 wait complete. 2023-12-01T01:17:50.984 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.980+0000 7f195a2a3700 1 Processor -- start 2023-12-01T01:17:50.984 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.980+0000 7f195a2a3700 1 -- start start 2023-12-01T01:17:50.984 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.981+0000 7f195a2a3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19540ffcb0 0x7f1954069a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:50.984 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.981+0000 7f195a2a3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1954100ea0 0x7f195419b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:50.984 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.981+0000 7f195a2a3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1954101840 0x7f195419ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:50.984 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.981+0000 7f195a2a3700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195419e590 con 0x7f19540ffcb0 2023-12-01T01:17:50.985 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.981+0000 7f195a2a3700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195419e700 con 0x7f1954101840 2023-12-01T01:17:50.985 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.981+0000 7f195a2a3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195419e8c0 con 0x7f1954100ea0 2023-12-01T01:17:50.985 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.981+0000 7f1953fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19540ffcb0 0x7f1954069a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:50.985 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.981+0000 7f1953fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19540ffcb0 0x7f1954069a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59230/0 (socket says 172.21.15.77:59230) 2023-12-01T01:17:50.985 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.981+0000 7f1953fff700 1 -- 172.21.15.77:0/4217245699 learned_addr learned my addr 172.21.15.77:0/4217245699 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:50.985 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.981+0000 7f1958840700 1 --2- 172.21.15.77:0/4217245699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1954101840 0x7f195419ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:50.985 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.981+0000 7f19537fe700 1 --2- 172.21.15.77:0/4217245699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1954100ea0 0x7f195419b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:50.985 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.981+0000 7f1953fff700 1 -- 172.21.15.77:0/4217245699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1954100ea0 msgr2=0x7f195419b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:50.985 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.981+0000 7f1953fff700 1 --2- 172.21.15.77:0/4217245699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1954100ea0 0x7f195419b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:50.985 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.981+0000 7f1953fff700 1 -- 172.21.15.77:0/4217245699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1954101840 msgr2=0x7f195419ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:50.985 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.981+0000 7f1953fff700 1 --2- 172.21.15.77:0/4217245699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1954101840 0x7f195419ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:50.985 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.981+0000 7f1953fff700 1 -- 172.21.15.77:0/4217245699 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194c009d00 con 0x7f19540ffcb0 2023-12-01T01:17:50.985 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.981+0000 7f1958840700 1 --2- 172.21.15.77:0/4217245699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1954101840 0x7f195419ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:17:50.985 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.981+0000 7f19537fe700 1 --2- 172.21.15.77:0/4217245699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1954100ea0 0x7f195419b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:17:50.985 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.981+0000 7f1953fff700 1 --2- 172.21.15.77:0/4217245699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19540ffcb0 0x7f1954069a90 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f194400d980 tx=0x7f194400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:50.986 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.981+0000 7f19517fa700 1 -- 172.21.15.77:0/4217245699 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194400bb60 con 0x7f19540ffcb0 2023-12-01T01:17:50.986 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.982+0000 7f19517fa700 1 -- 172.21.15.77:0/4217245699 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1944007620 con 0x7f19540ffcb0 2023-12-01T01:17:50.986 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.982+0000 7f195a2a3700 1 -- 172.21.15.77:0/4217245699 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f195419eba0 con 0x7f19540ffcb0 2023-12-01T01:17:50.986 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.982+0000 7f19517fa700 1 -- 172.21.15.77:0/4217245699 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194400f620 con 0x7f19540ffcb0 2023-12-01T01:17:50.986 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.982+0000 7f195a2a3700 1 -- 172.21.15.77:0/4217245699 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f195419f0b0 con 0x7f19540ffcb0 2023-12-01T01:17:50.986 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.982+0000 7f195a2a3700 1 -- 172.21.15.77:0/4217245699 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f195410b4d0 con 0x7f19540ffcb0 2023-12-01T01:17:50.987 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.984+0000 7f19517fa700 1 -- 172.21.15.77:0/4217245699 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f1944010460 con 0x7f19540ffcb0 2023-12-01T01:17:50.987 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.984+0000 7f19517fa700 1 --2- 172.21.15.77:0/4217245699 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f193c0a05f0 0x7f193c0a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:50.987 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.984+0000 7f19517fa700 1 -- 172.21.15.77:0/4217245699 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f19440cc2d0 con 0x7f19540ffcb0 2023-12-01T01:17:50.987 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.984+0000 7f19537fe700 1 --2- 172.21.15.77:0/4217245699 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f193c0a05f0 0x7f193c0a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:50.987 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.985+0000 7f19537fe700 1 --2- 172.21.15.77:0/4217245699 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f193c0a05f0 0x7f193c0a2ab0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f194800baa0 tx=0x7f1948005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:50.988 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:50.987+0000 7f19517fa700 1 -- 172.21.15.77:0/4217245699 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1944095e10 con 0x7f19540ffcb0 2023-12-01T01:17:51.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:50 smithi203 ceph-mon[103581]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-01T01:17:51.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:50 smithi203 ceph-mon[103581]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-01T01:17:51.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:50 smithi203 ceph-mon[103581]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-01T01:17:51.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:50 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/2237650081' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:17:51.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:50 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/2081573446' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-01T01:17:51.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:50 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/795714547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-01T01:17:51.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:50 smithi137 ceph-mon[103312]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-01T01:17:51.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:50 smithi137 ceph-mon[103312]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-01T01:17:51.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:50 smithi137 ceph-mon[103312]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-01T01:17:51.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:50 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/2237650081' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:17:51.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:50 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/2081573446' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-01T01:17:51.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:50 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/795714547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-01T01:17:51.227 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.226+0000 7f195a2a3700 1 -- 172.21.15.77:0/4217245699 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1954106a20 con 0x7f19540ffcb0 2023-12-01T01:17:51.228 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.227+0000 7f19517fa700 1 -- 172.21.15.77:0/4217245699 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10838 (secure 0 0 0) 0x7f194400de60 con 0x7f19540ffcb0 2023-12-01T01:17:51.228 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:17:51.228 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1435145620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1435145620},{"type":"v1","addr":"172.21.15.77:6835","nonce":1435145620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/841838908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":841838908},{"type":"v1","addr":"172.21.15.77:6837","nonce":841838908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-01T01:16:25.345276+0000","modified":"2023-12-01T01:16:32.596617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24496,"mds_2":24475,"mds_3":24473,"mds_4":14649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/3726913446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3726913446},{"type":"v1","addr":"172.21.15.77:6839","nonce":3726913446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14649":{"gid":14649,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":5,"addr":"172.21.15.77:6841/3023184965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3023184965},{"type":"v1","addr":"172.21.15.77:6841","nonce":3023184965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/2249638445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2249638445},{"type":"v1","addr":"172.21.15.137:6833","nonce":2249638445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6833/1201599774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1201599774},{"type":"v1","addr":"172.21.15.203:6833","nonce":1201599774}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3935354878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3935354878},{"type":"v1","addr":"172.21.15.137:6835","nonce":3935354878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3237538176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3237538176},{"type":"v1","addr":"172.21.15.203:6835","nonce":3237538176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3518881867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3518881867},{"type":"v1","addr":"172.21.15.137:6837","nonce":3518881867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.203:6837/2887080977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2887080977},{"type":"v1","addr":"172.21.15.203:6837","nonce":2887080977}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.203:6839/2888221613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2888221613},{"type":"v1","addr":"172.21.15.203:6839","nonce":2888221613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/1099966660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1099966660},{"type":"v1","addr":"172.21.15.137:6839","nonce":1099966660}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-01T01:17:51.230 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.229+0000 7f195a2a3700 1 -- 172.21.15.77:0/4217245699 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f193c0a05f0 msgr2=0x7f193c0a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:51.230 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.229+0000 7f195a2a3700 1 --2- 172.21.15.77:0/4217245699 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f193c0a05f0 0x7f193c0a2ab0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f194800baa0 tx=0x7f1948005db0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.230 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.229+0000 7f195a2a3700 1 -- 172.21.15.77:0/4217245699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19540ffcb0 msgr2=0x7f1954069a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:51.230 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.229+0000 7f195a2a3700 1 --2- 172.21.15.77:0/4217245699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19540ffcb0 0x7f1954069a90 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f194400d980 tx=0x7f194400dd40 comp rx=0 tx=0).stop 2023-12-01T01:17:51.230 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.229+0000 7f195a2a3700 1 -- 172.21.15.77:0/4217245699 shutdown_connections 2023-12-01T01:17:51.230 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.229+0000 7f195a2a3700 1 --2- 172.21.15.77:0/4217245699 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f193c0a05f0 0x7f193c0a2ab0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.231 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.229+0000 7f195a2a3700 1 --2- 172.21.15.77:0/4217245699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1954101840 0x7f195419ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.231 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.229+0000 7f195a2a3700 1 --2- 172.21.15.77:0/4217245699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19540ffcb0 0x7f1954069a90 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.231 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.229+0000 7f195a2a3700 1 --2- 172.21.15.77:0/4217245699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1954100ea0 0x7f195419b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.231 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.229+0000 7f195a2a3700 1 -- 172.21.15.77:0/4217245699 >> 172.21.15.77:0/4217245699 conn(0x7f19540fb220 msgr2=0x7f19540fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:51.231 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.229+0000 7f195a2a3700 1 -- 172.21.15.77:0/4217245699 shutdown_connections 2023-12-01T01:17:51.231 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.229+0000 7f195a2a3700 1 -- 172.21.15.77:0/4217245699 wait complete. 2023-12-01T01:17:51.231 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 27 2023-12-01T01:17:51.243 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:17:51.367 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.364+0000 7f5785354700 1 -- 172.21.15.77:0/1936721637 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f57801008a0 msgr2=0x7f5780108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:51.368 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.364+0000 7f5785354700 1 --2- 172.21.15.77:0/1936721637 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f57801008a0 0x7f5780108140 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f577800b0a0 tx=0x7f577800b3b0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.368 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.364+0000 7f5785354700 1 -- 172.21.15.77:0/1936721637 shutdown_connections 2023-12-01T01:17:51.368 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.364+0000 7f5785354700 1 --2- 172.21.15.77:0/1936721637 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f57801008a0 0x7f5780108140 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.368 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.364+0000 7f5785354700 1 --2- 172.21.15.77:0/1936721637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5780101290 0x7f57801016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.368 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.364+0000 7f5785354700 1 --2- 172.21.15.77:0/1936721637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57800ffee0 0x7f5780100360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.368 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.364+0000 7f5785354700 1 -- 172.21.15.77:0/1936721637 >> 172.21.15.77:0/1936721637 conn(0x7f57800fa950 msgr2=0x7f57800fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:51.368 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.364+0000 7f5785354700 1 -- 172.21.15.77:0/1936721637 shutdown_connections 2023-12-01T01:17:51.368 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.364+0000 7f5785354700 1 -- 172.21.15.77:0/1936721637 wait complete. 2023-12-01T01:17:51.368 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.364+0000 7f5785354700 1 Processor -- start 2023-12-01T01:17:51.368 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.364+0000 7f5785354700 1 -- start start 2023-12-01T01:17:51.368 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.365+0000 7f5785354700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57800ffee0 0x7f578010a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:51.368 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.365+0000 7f5785354700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f57801008a0 0x7f5780108c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:51.369 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.365+0000 7f5785354700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5780101290 0x7f5780109140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:51.369 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.365+0000 7f5785354700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57801098e0 con 0x7f57801008a0 2023-12-01T01:17:51.369 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.365+0000 7f5785354700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5780109a50 con 0x7f57800ffee0 2023-12-01T01:17:51.369 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.365+0000 7f5785354700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57801a0ba0 con 0x7f5780101290 2023-12-01T01:17:51.369 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.365+0000 7f577e7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f57801008a0 0x7f5780108c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:51.369 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.365+0000 7f577e7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f57801008a0 0x7f5780108c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59248/0 (socket says 172.21.15.77:59248) 2023-12-01T01:17:51.369 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.365+0000 7f577e7fc700 1 -- 172.21.15.77:0/688106184 learned_addr learned my addr 172.21.15.77:0/688106184 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:51.369 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.365+0000 7f577effd700 1 --2- 172.21.15.77:0/688106184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57800ffee0 0x7f578010a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:51.369 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.365+0000 7f577f7fe700 1 --2- 172.21.15.77:0/688106184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5780101290 0x7f5780109140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:51.369 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.365+0000 7f577e7fc700 1 -- 172.21.15.77:0/688106184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5780101290 msgr2=0x7f5780109140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:51.369 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.365+0000 7f577e7fc700 1 --2- 172.21.15.77:0/688106184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5780101290 0x7f5780109140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.369 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.365+0000 7f577e7fc700 1 -- 172.21.15.77:0/688106184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57800ffee0 msgr2=0x7f578010a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:51.370 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.365+0000 7f577e7fc700 1 --2- 172.21.15.77:0/688106184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57800ffee0 0x7f578010a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.370 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.365+0000 7f577e7fc700 1 -- 172.21.15.77:0/688106184 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5778009d00 con 0x7f57801008a0 2023-12-01T01:17:51.370 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.365+0000 7f577f7fe700 1 --2- 172.21.15.77:0/688106184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5780101290 0x7f5780109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:17:51.370 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.365+0000 7f577e7fc700 1 --2- 172.21.15.77:0/688106184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f57801008a0 0x7f5780108c00 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f577000d950 tx=0x7f577000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:51.370 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.365+0000 7f5767fff700 1 -- 172.21.15.77:0/688106184 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5770009940 con 0x7f57801008a0 2023-12-01T01:17:51.370 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.366+0000 7f5785354700 1 -- 172.21.15.77:0/688106184 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57801a0e80 con 0x7f57801008a0 2023-12-01T01:17:51.370 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.366+0000 7f5767fff700 1 -- 172.21.15.77:0/688106184 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5770010460 con 0x7f57801008a0 2023-12-01T01:17:51.370 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.366+0000 7f5767fff700 1 -- 172.21.15.77:0/688106184 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577000f620 con 0x7f57801008a0 2023-12-01T01:17:51.370 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.366+0000 7f5785354700 1 -- 172.21.15.77:0/688106184 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57801a1350 con 0x7f57801008a0 2023-12-01T01:17:51.370 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.367+0000 7f5785354700 1 -- 172.21.15.77:0/688106184 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5780106be0 con 0x7f57801008a0 2023-12-01T01:17:51.371 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.367+0000 7f5767fff700 1 -- 172.21.15.77:0/688106184 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f57700108d0 con 0x7f57801008a0 2023-12-01T01:17:51.371 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.368+0000 7f5767fff700 1 --2- 172.21.15.77:0/688106184 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f57680a05f0 0x7f57680a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:51.371 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.368+0000 7f577effd700 1 --2- 172.21.15.77:0/688106184 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f57680a05f0 0x7f57680a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:51.371 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.368+0000 7f5767fff700 1 -- 172.21.15.77:0/688106184 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f57700cb8a0 con 0x7f57801008a0 2023-12-01T01:17:51.371 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.368+0000 7f577effd700 1 --2- 172.21.15.77:0/688106184 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f57680a05f0 0x7f57680a2ab0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5780067cd0 tx=0x7f5774009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:51.373 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.372+0000 7f5767fff700 1 -- 172.21.15.77:0/688106184 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f57700953e0 con 0x7f57801008a0 2023-12-01T01:17:51.566 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.565+0000 7f5785354700 1 -- 172.21.15.77:0/688106184 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f578002cec0 con 0x7f57801008a0 2023-12-01T01:17:51.568 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.567+0000 7f5767fff700 1 -- 172.21.15.77:0/688106184 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22472 (secure 0 0 0) 0x7f5770016090 con 0x7f57801008a0 2023-12-01T01:17:51.568 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:17:51.568 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":84,"fsid":"2a06e29a-8fe6-11ee-95a2-87774f69a715","created":"2023-12-01T01:09:30.210398+0000","modified":"2023-12-01T01:17:25.609766+0000","last_up_change":"2023-12-01T01:15:44.586092+0000","last_in_change":"2023-12-01T01:15:26.305439+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:12:30.514229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:16:23.568860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-01T01:16:24.531137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fc651e62-a103-4ab8-a76d-d3708ea6aeec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6803","nonce":2244799094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6805","nonce":2244799094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6809","nonce":2244799094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6807","nonce":2244799094}]},"public_addr":"172.21.15.77:6803/2244799094","cluster_addr":"172.21.15.77:6805/2244799094","heartbeat_back_addr":"172.21.15.77:6809/2244799094","heartbeat_front_addr":"172.21.15.77:6807/2244799094","state":["exists","up"]},{"osd":1,"uuid":"0adf8c54-49ec-49a0-8bf1-af1f20e60729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6801","nonce":224194890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6803","nonce":224194890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6807","nonce":224194890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6805","nonce":224194890}]},"public_addr":"172.21.15.137:6801/224194890","cluster_addr":"172.21.15.137:6803/224194890","heartbeat_back_addr":"172.21.15.137:6807/224194890","heartbeat_front_addr":"172.21.15.137:6805/224194890","state":["exists","up"]},{"osd":2,"uuid":"5a69802c-dede-4544-9a12-436f5c1c0038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6801","nonce":4078459110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6803","nonce":4078459110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6807","nonce":4078459110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6805","nonce":4078459110}]},"public_addr":"172.21.15.203:6801/4078459110","cluster_addr":"172.21.15.203:6803/4078459110","heartbeat_back_addr":"172.21.15.203:6807/4078459110","heartbeat_front_addr":"172.21.15.203:6805/4078459110","state":["exists","up"]},{"osd":3,"uuid":"ad325a97-580b-4c99-bf12-cbb0d92a6d0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6811","nonce":1454440477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6813","nonce":1454440477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6817","nonce":1454440477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6815","nonce":1454440477}]},"public_addr":"172.21.15.77:6811/1454440477","cluster_addr":"172.21.15.77:6813/1454440477","heartbeat_back_addr":"172.21.15.77:6817/1454440477","heartbeat_front_addr":"172.21.15.77:6815/1454440477","state":["exists","up"]},{"osd":4,"uuid":"1e3ac1ba-98bb-4b8b-8e26-50064c66bf2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6809","nonce":1103241451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6811","nonce":1103241451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6815","nonce":1103241451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6813","nonce":1103241451}]},"public_addr":"172.21.15.137:6809/1103241451","cluster_addr":"172.21.15.137:6811/1103241451","heartbeat_back_addr":"172.21.15.137:6815/1103241451","heartbeat_front_addr":"172.21.15.137:6813/1103241451","state":["exists","up"]},{"osd":5,"uuid":"eb32c016-abeb-44ab-abf0-19dd09235736","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6809","nonce":2648569482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6811","nonce":2648569482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6815","nonce":2648569482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6813","nonce":2648569482}]},"public_addr":"172.21.15.203:6809/2648569482","cluster_addr":"172.21.15.203:6811/2648569482","heartbeat_back_addr":"172.21.15.203:6815/2648569482","heartbeat_front_addr":"172.21.15.203:6813/2648569482","state":["exists","up"]},{"osd":6,"uuid":"0e3fa97e-2ca6-418c-9e8d-a6fe3c56adb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6819","nonce":2463414651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6821","nonce":2463414651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6825","nonce":2463414651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6823","nonce":2463414651}]},"public_addr":"172.21.15.77:6819/2463414651","cluster_addr":"172.21.15.77:6821/2463414651","heartbeat_back_addr":"172.21.15.77:6825/2463414651","heartbeat_front_addr":"172.21.15.77:6823/2463414651","state":["exists","up"]},{"osd":7,"uuid":"d7398c01-0f98-4138-8a49-c3ff0aae7e88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6817","nonce":3108330766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6819","nonce":3108330766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6823","nonce":3108330766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6821","nonce":3108330766}]},"public_addr":"172.21.15.137:6817/3108330766","cluster_addr":"172.21.15.137:6819/3108330766","heartbeat_back_addr":"172.21.15.137:6823/3108330766","heartbeat_front_addr":"172.21.15.137:6821/3108330766","state":["exists","up"]},{"osd":8,"uuid":"0c09050d-3e43-410c-8cd7-bcb891af56b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6817","nonce":1161158776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6819","nonce":1161158776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6823","nonce":1161158776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6821","nonce":1161158776}]},"public_addr":"172.21.15.203:6817/1161158776","cluster_addr":"172.21.15.203:6819/1161158776","heartbeat_back_addr":"172.21.15.203:6823/1161158776","heartbeat_front_addr":"172.21.15.203:6821/1161158776","state":["exists","up"]},{"osd":9,"uuid":"527ecb49-95ec-48bd-9872-3b07fae5c5ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6827","nonce":2409880685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6829","nonce":2409880685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6833","nonce":2409880685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6831","nonce":2409880685}]},"public_addr":"172.21.15.77:6827/2409880685","cluster_addr":"172.21.15.77:6829/2409880685","heartbeat_back_addr":"172.21.15.77:6833/2409880685","heartbeat_front_addr":"172.21.15.77:6831/2409880685","state":["exists","up"]},{"osd":10,"uuid":"1680bb3e-a3c8-427b-af68-e2214f32bed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6825","nonce":374920146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6827","nonce":374920146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6831","nonce":374920146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6829","nonce":374920146}]},"public_addr":"172.21.15.137:6825/374920146","cluster_addr":"172.21.15.137:6827/374920146","heartbeat_back_addr":"172.21.15.137:6831/374920146","heartbeat_front_addr":"172.21.15.137:6829/374920146","state":["exists","up"]},{"osd":11,"uuid":"8a533466-023d-4803-ae36-9a9f1715e49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6825","nonce":2161588962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6827","nonce":2161588962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6831","nonce":2161588962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6829","nonce":2161588962}]},"public_addr":"172.21.15.203:6825/2161588962","cluster_addr":"172.21.15.203:6827/2161588962","heartbeat_back_addr":"172.21.15.203:6831/2161588962","heartbeat_front_addr":"172.21.15.203:6829/2161588962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:11:45.420509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:06.210115+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:28.191307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:49.594116+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:11.261689+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:33.376541+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:54.968336+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:16.775310+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:38.733792+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:59.456699+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:15:20.924059+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:15:43.094773+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/2260035254":"2023-12-02T01:10:24.191835+0000","172.21.15.77:6800/1456621747":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/219838828":"2023-12-02T01:10:24.191835+0000","172.21.15.77:6801/1456621747":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/2937149266":"2023-12-02T01:10:02.232948+0000","172.21.15.77:0/807520018":"2023-12-02T01:10:02.232948+0000","172.21.15.77:6801/1661788026":"2023-12-02T01:10:02.232948+0000","172.21.15.77:6800/1661788026":"2023-12-02T01:10:02.232948+0000","172.21.15.77:0/3551282473":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/1202222558":"2023-12-02T01:10:02.232948+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:17:51.570 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.569+0000 7f5785354700 1 -- 172.21.15.77:0/688106184 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f57680a05f0 msgr2=0x7f57680a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:51.570 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.569+0000 7f5785354700 1 --2- 172.21.15.77:0/688106184 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f57680a05f0 0x7f57680a2ab0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5780067cd0 tx=0x7f5774009260 comp rx=0 tx=0).stop 2023-12-01T01:17:51.570 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.569+0000 7f5785354700 1 -- 172.21.15.77:0/688106184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f57801008a0 msgr2=0x7f5780108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:51.571 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.569+0000 7f5785354700 1 --2- 172.21.15.77:0/688106184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f57801008a0 0x7f5780108c00 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f577000d950 tx=0x7f577000dc60 comp rx=0 tx=0).stop 2023-12-01T01:17:51.571 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.569+0000 7f5785354700 1 -- 172.21.15.77:0/688106184 shutdown_connections 2023-12-01T01:17:51.571 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.569+0000 7f5785354700 1 --2- 172.21.15.77:0/688106184 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f57680a05f0 0x7f57680a2ab0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.571 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.569+0000 7f5785354700 1 --2- 172.21.15.77:0/688106184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5780101290 0x7f5780109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.571 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.569+0000 7f5785354700 1 --2- 172.21.15.77:0/688106184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57800ffee0 0x7f578010a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.571 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.569+0000 7f5785354700 1 --2- 172.21.15.77:0/688106184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f57801008a0 0x7f5780108c00 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.571 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.569+0000 7f5785354700 1 -- 172.21.15.77:0/688106184 >> 172.21.15.77:0/688106184 conn(0x7f57800fa950 msgr2=0x7f57800fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:51.571 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.569+0000 7f5785354700 1 -- 172.21.15.77:0/688106184 shutdown_connections 2023-12-01T01:17:51.571 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.569+0000 7f5785354700 1 -- 172.21.15.77:0/688106184 wait complete. 2023-12-01T01:17:51.583 DEBUG:tasks.fwd_scrub:Yielding 2023-12-01T01:17:51.583 INFO:teuthology.run_tasks:Running task background_exec... 2023-12-01T01:17:51.597 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi077.front.sepia.ceph.com 2023-12-01T01:17:51.597 DEBUG:teuthology.orchestra.run.smithi077:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2023-12-01T01:17:51.597 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-12-01T01:17:51.597 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-12-01T01:17:51.598 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-12-01T01:17:51.600 INFO:teuthology.run_tasks:Running task workunit... 2023-12-01T01:17:51.614 INFO:tasks.workunit:Pulling workunits from ref a6f82997ac57b70a895455dfed6256360a1e4c32 2023-12-01T01:17:51.615 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-12-01T01:17:51.615 INFO:tasks.workunit:timeout=3h 2023-12-01T01:17:51.615 INFO:tasks.workunit:cleanup=True 2023-12-01T01:17:51.615 DEBUG:teuthology.orchestra.run.smithi077:> stat -- /home/ubuntu/cephtest/mnt.0 2023-12-01T01:17:51.639 INFO:teuthology.orchestra.run.smithi077.stdout: File: /home/ubuntu/cephtest/mnt.0 2023-12-01T01:17:51.639 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2023-12-01T01:17:51.639 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 7ch/124d Inode: 1099511628283 Links: 2 2023-12-01T01:17:51.639 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-01T01:17:51.639 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:cephfs_t:s0 2023-12-01T01:17:51.639 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2023-12-01 01:16:32.381394574 +0000 2023-12-01T01:17:51.639 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2023-12-01 01:16:32.381394574 +0000 2023-12-01T01:17:51.639 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2023-12-01 01:17:44.905496247 +0000 2023-12-01T01:17:51.639 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2023-12-01 01:16:32.381394574 +0000 2023-12-01T01:17:51.639 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2023-12-01T01:17:51.640 DEBUG:teuthology.orchestra.run.smithi077:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2023-12-01T01:17:51.736 DEBUG:teuthology.orchestra.run.smithi077:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout a6f82997ac57b70a895455dfed6256360a1e4c32 2023-12-01T01:17:51.738 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.735+0000 7f0682c18700 1 -- 172.21.15.77:0/2885050904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f067c0ffc80 msgr2=0x7f067c1000a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:51.738 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.735+0000 7f0682c18700 1 --2- 172.21.15.77:0/2885050904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f067c0ffc80 0x7f067c1000a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f06700099c0 tx=0x7f0670009cd0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.739 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.735+0000 7f0682c18700 1 -- 172.21.15.77:0/2885050904 shutdown_connections 2023-12-01T01:17:51.739 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.735+0000 7f0682c18700 1 --2- 172.21.15.77:0/2885050904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f067c101810 0x7f067c1080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.739 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.735+0000 7f0682c18700 1 --2- 172.21.15.77:0/2885050904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f067c0ffc80 0x7f067c1000a0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.739 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.735+0000 7f0682c18700 1 --2- 172.21.15.77:0/2885050904 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f067c100e70 0x7f067c1012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.739 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.735+0000 7f0682c18700 1 -- 172.21.15.77:0/2885050904 >> 172.21.15.77:0/2885050904 conn(0x7f067c0fb1f0 msgr2=0x7f067c0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:51.739 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.736+0000 7f0682c18700 1 -- 172.21.15.77:0/2885050904 shutdown_connections 2023-12-01T01:17:51.739 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.736+0000 7f0682c18700 1 -- 172.21.15.77:0/2885050904 wait complete. 2023-12-01T01:17:51.739 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.736+0000 7f0682c18700 1 Processor -- start 2023-12-01T01:17:51.739 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.736+0000 7f0682c18700 1 -- start start 2023-12-01T01:17:51.739 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.736+0000 7f0682c18700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f067c0ffc80 0x7f067c068a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:51.739 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.736+0000 7f0682c18700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f067c100e70 0x7f067c19b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:51.739 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.736+0000 7f0682c18700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f067c101810 0x7f067c19de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:51.739 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.736+0000 7f0682c18700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067c19e5b0 con 0x7f067c101810 2023-12-01T01:17:51.739 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.736+0000 7f0682c18700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067c19e720 con 0x7f067c100e70 2023-12-01T01:17:51.740 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.736+0000 7f0682c18700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067c19e8e0 con 0x7f067c0ffc80 2023-12-01T01:17:51.740 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.736+0000 7f06811b5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f067c101810 0x7f067c19de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:51.740 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.736+0000 7f06811b5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f067c101810 0x7f067c19de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59264/0 (socket says 172.21.15.77:59264) 2023-12-01T01:17:51.740 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.736+0000 7f06811b5700 1 -- 172.21.15.77:0/617438846 learned_addr learned my addr 172.21.15.77:0/617438846 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:51.740 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.736+0000 7f06809b4700 1 --2- 172.21.15.77:0/617438846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f067c0ffc80 0x7f067c068a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:51.740 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.737+0000 7f067bfff700 1 --2- 172.21.15.77:0/617438846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f067c100e70 0x7f067c19b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:51.740 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.737+0000 7f06811b5700 1 -- 172.21.15.77:0/617438846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f067c0ffc80 msgr2=0x7f067c068a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:51.740 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.737+0000 7f06811b5700 1 --2- 172.21.15.77:0/617438846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f067c0ffc80 0x7f067c068a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.740 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.737+0000 7f06811b5700 1 -- 172.21.15.77:0/617438846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f067c100e70 msgr2=0x7f067c19b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:51.740 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.737+0000 7f06811b5700 1 --2- 172.21.15.77:0/617438846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f067c100e70 0x7f067c19b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.744 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.737+0000 7f06811b5700 1 -- 172.21.15.77:0/617438846 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06700095f0 con 0x7f067c101810 2023-12-01T01:17:51.744 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.737+0000 7f06811b5700 1 --2- 172.21.15.77:0/617438846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f067c101810 0x7f067c19de10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0674007ee0 tx=0x7f067400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:51.744 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.737+0000 7f0679ffb700 1 -- 172.21.15.77:0/617438846 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0674017070 con 0x7f067c101810 2023-12-01T01:17:51.744 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.737+0000 7f0679ffb700 1 -- 172.21.15.77:0/617438846 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f067400f040 con 0x7f067c101810 2023-12-01T01:17:51.744 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.737+0000 7f0679ffb700 1 -- 172.21.15.77:0/617438846 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0674005580 con 0x7f067c101810 2023-12-01T01:17:51.744 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.737+0000 7f0682c18700 1 -- 172.21.15.77:0/617438846 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f067c19ebc0 con 0x7f067c101810 2023-12-01T01:17:51.744 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.737+0000 7f0682c18700 1 -- 172.21.15.77:0/617438846 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f067c19f080 con 0x7f067c101810 2023-12-01T01:17:51.744 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.739+0000 7f0679ffb700 1 -- 172.21.15.77:0/617438846 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f067400db20 con 0x7f067c101810 2023-12-01T01:17:51.744 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.740+0000 7f0679ffb700 1 --2- 172.21.15.77:0/617438846 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f066c0a05b0 0x7f066c0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:51.744 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.740+0000 7f0679ffb700 1 -- 172.21.15.77:0/617438846 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f06740cb1a0 con 0x7f067c101810 2023-12-01T01:17:51.744 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.740+0000 7f06809b4700 1 --2- 172.21.15.77:0/617438846 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f066c0a05b0 0x7f066c0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:51.745 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.740+0000 7f06809b4700 1 --2- 172.21.15.77:0/617438846 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f066c0a05b0 0x7f066c0a2a70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0670009990 tx=0x7f0670005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:51.745 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.741+0000 7f068eda8b80 1 Processor -- start 2023-12-01T01:17:51.745 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.741+0000 7f068eda8b80 1 -- start start 2023-12-01T01:17:51.745 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.741+0000 7f068eda8b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee0559a170 0x55ee0559a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:51.745 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.741+0000 7f068eda8b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee0559aaf0 0x55ee05598ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:51.745 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.741+0000 7f068eda8b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee05599470 0x55ee055998f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:51.745 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.741+0000 7f068eda8b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee054f0a90 con 0x55ee0559aaf0 2023-12-01T01:17:51.745 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.741+0000 7f068eda8b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee054ee450 con 0x55ee05599470 2023-12-01T01:17:51.745 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.741+0000 7f068eda8b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee054e9d40 con 0x55ee0559a170 2023-12-01T01:17:51.745 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.742+0000 7f067bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee0559aaf0 0x55ee05598ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:51.745 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.742+0000 7f067bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee0559aaf0 0x55ee05598ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59270/0 (socket says 172.21.15.77:59270) 2023-12-01T01:17:51.745 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.742+0000 7f067bfff700 1 -- 172.21.15.77:0/3115801986 learned_addr learned my addr 172.21.15.77:0/3115801986 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:51.745 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.742+0000 7f06809b4700 1 --2- 172.21.15.77:0/3115801986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee0559a170 0x55ee0559a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:51.745 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.742+0000 7f06811b5700 1 --2- 172.21.15.77:0/3115801986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee05599470 0x55ee055998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:51.746 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.742+0000 7f067bfff700 1 -- 172.21.15.77:0/3115801986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee0559a170 msgr2=0x55ee0559a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:51.746 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.742+0000 7f067bfff700 1 --2- 172.21.15.77:0/3115801986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee0559a170 0x55ee0559a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.746 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.742+0000 7f067bfff700 1 -- 172.21.15.77:0/3115801986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee05599470 msgr2=0x55ee055998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:51.746 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.742+0000 7f067bfff700 1 --2- 172.21.15.77:0/3115801986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee05599470 0x55ee055998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.746 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.742+0000 7f067bfff700 1 -- 172.21.15.77:0/3115801986 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee05317e60 con 0x55ee0559aaf0 2023-12-01T01:17:51.746 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.742+0000 7f06811b5700 1 --2- 172.21.15.77:0/3115801986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee05599470 0x55ee055998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:17:51.746 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.742+0000 7f067bfff700 1 --2- 172.21.15.77:0/3115801986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee0559aaf0 0x55ee05598ee0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0668009d80 tx=0x7f0668011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=834c68060d56cb7 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:51.746 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.742+0000 7f0666ffd700 1 -- 172.21.15.77:0/3115801986 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0668004230 con 0x55ee0559aaf0 2023-12-01T01:17:51.746 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.742+0000 7f0666ffd700 1 -- 172.21.15.77:0/3115801986 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0668004390 con 0x55ee0559aaf0 2023-12-01T01:17:51.746 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.742+0000 7f0666ffd700 1 -- 172.21.15.77:0/3115801986 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0668019860 con 0x55ee0559aaf0 2023-12-01T01:17:51.750 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.743+0000 7f068eda8b80 1 -- 172.21.15.77:0/3115801986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee0559aaf0 msgr2=0x55ee05598ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:51.751 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.743+0000 7f068eda8b80 1 --2- 172.21.15.77:0/3115801986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee0559aaf0 0x55ee05598ee0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0668009d80 tx=0x7f0668011720 comp rx=0 tx=0).stop 2023-12-01T01:17:51.751 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.743+0000 7f068eda8b80 1 -- 172.21.15.77:0/3115801986 shutdown_connections 2023-12-01T01:17:51.751 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.743+0000 7f068eda8b80 1 --2- 172.21.15.77:0/3115801986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee05599470 0x55ee055998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.751 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.743+0000 7f068eda8b80 1 --2- 172.21.15.77:0/3115801986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee0559a170 0x55ee0559a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.751 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.743+0000 7f068eda8b80 1 --2- 172.21.15.77:0/3115801986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee0559aaf0 0x55ee05598ee0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.751 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.743+0000 7f068eda8b80 1 -- 172.21.15.77:0/3115801986 >> 172.21.15.77:0/3115801986 conn(0x55ee054f8150 msgr2=0x55ee055939c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:51.751 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.743+0000 7f068eda8b80 1 -- 172.21.15.77:0/3115801986 shutdown_connections 2023-12-01T01:17:51.751 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.743+0000 7f068eda8b80 1 -- 172.21.15.77:0/3115801986 wait complete. 2023-12-01T01:17:51.751 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.743+0000 7f068eda8b80 1 -- start start 2023-12-01T01:17:51.751 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.744+0000 7f068eda8b80 1 Processor -- start 2023-12-01T01:17:51.752 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.744+0000 7f068eda8b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee05599470 0x55ee055a29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:51.752 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.744+0000 7f068eda8b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee0559a170 0x55ee055a2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:51.752 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.744+0000 7f068eda8b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee0559aaf0 0x55ee055a9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:51.752 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.744+0000 7f068eda8b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee05486590 con 0x55ee05599470 2023-12-01T01:17:51.752 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.744+0000 7f068eda8b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee05486230 con 0x55ee0559a170 2023-12-01T01:17:51.752 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.744+0000 7f068eda8b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee0536d9a0 con 0x55ee0559aaf0 2023-12-01T01:17:51.752 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.744+0000 7f068eda8b80 10 client.0 _ll_register_callbacks cb 0x55ee053d3b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:17:51.752 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.744+0000 7f06809b4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee05599470 0x55ee055a29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:51.752 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.744+0000 7f06809b4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee05599470 0x55ee055a29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59274/0 (socket says 172.21.15.77:59274) 2023-12-01T01:17:51.752 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.744+0000 7f06809b4700 1 -- 172.21.15.77:0/2392753978 learned_addr learned my addr 172.21.15.77:0/2392753978 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:51.752 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.744+0000 7f067bfff700 1 --2- 172.21.15.77:0/2392753978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee0559a170 0x55ee055a2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:51.752 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.744+0000 7f06811b5700 1 --2- 172.21.15.77:0/2392753978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee0559aaf0 0x55ee055a9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:51.752 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.744+0000 7f06809b4700 1 -- 172.21.15.77:0/2392753978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee0559aaf0 msgr2=0x55ee055a9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:51.753 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.744+0000 7f06809b4700 1 --2- 172.21.15.77:0/2392753978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee0559aaf0 0x55ee055a9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.753 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.744+0000 7f06809b4700 1 -- 172.21.15.77:0/2392753978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee0559a170 msgr2=0x55ee055a2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:51.753 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.744+0000 7f06809b4700 1 --2- 172.21.15.77:0/2392753978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee0559a170 0x55ee055a2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.753 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.744+0000 7f06809b4700 1 -- 172.21.15.77:0/2392753978 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06680099e0 con 0x55ee05599470 2023-12-01T01:17:51.753 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.744+0000 7f06811b5700 1 --2- 172.21.15.77:0/2392753978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee0559aaf0 0x55ee055a9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:17:51.753 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.745+0000 7f06809b4700 1 --2- 172.21.15.77:0/2392753978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee05599470 0x55ee055a29a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f0670000c00 tx=0x7f0670009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d05486b7797472d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:51.753 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.745+0000 7f065bfff700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:17:51.753 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.745+0000 7f065bfff700 1 -- 172.21.15.77:0/2392753978 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0670029050 con 0x55ee05599470 2023-12-01T01:17:51.753 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.745+0000 7f065bfff700 1 -- 172.21.15.77:0/2392753978 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f067000f660 con 0x55ee05599470 2023-12-01T01:17:51.753 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.745+0000 7f068eda8b80 1 -- 172.21.15.77:0/2392753978 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ee05514d90 con 0x55ee05599470 2023-12-01T01:17:51.753 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.745+0000 7f065bfff700 1 -- 172.21.15.77:0/2392753978 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f067000f860 con 0x55ee05599470 2023-12-01T01:17:51.753 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.745+0000 7f068eda8b80 10 client.15462 fetch_fsmap learned FSMap version 27 2023-12-01T01:17:51.753 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.745+0000 7f068eda8b80 1 -- 172.21.15.77:0/2392753978 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ee05300ce0 con 0x55ee05599470 2023-12-01T01:17:51.753 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.745+0000 7f065bfff700 1 -- 172.21.15.77:0/2392753978 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f067003d020 con 0x55ee05599470 2023-12-01T01:17:51.754 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.745+0000 7f065bfff700 1 -- 172.21.15.77:0/2392753978 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f0670022a10 con 0x55ee05599470 2023-12-01T01:17:51.754 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.745+0000 7f068eda8b80 10 client.15462 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:17:51.754 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.745+0000 7f068eda8b80 10 client.15462 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-01T01:17:51.754 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.745+0000 7f068eda8b80 20 client.15462 populate_metadata read hostname 'smithi077' 2023-12-01T01:17:51.754 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.745+0000 7f068eda8b80 1 --2- 172.21.15.77:0/2392753978 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55ee055ae730 0x55ee055b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:51.754 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.745+0000 7f068eda8b80 4 client.15462 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:17:51.754 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.745+0000 7f068eda8b80 1 -- 172.21.15.77:0/2392753978 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee05567e90 con 0x55ee055ae730 2023-12-01T01:17:51.754 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.746+0000 7f067bfff700 1 --2- 172.21.15.77:0/2392753978 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55ee055ae730 0x55ee055b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:51.754 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.746+0000 7f067bfff700 1 --2- 172.21.15.77:0/2392753978 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55ee055ae730 0x55ee055b0b50 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df16eaad36ba13fa server_cookie=b5c2585f7d6931bc in_seq=0 out_seq=0 2023-12-01T01:17:51.754 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.746+0000 7f065bfff700 10 client.15462 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:17:51.754 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.747+0000 7f065bfff700 1 -- 172.21.15.77:0/2392753978 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55ee05567e90 con 0x55ee055ae730 2023-12-01T01:17:51.754 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.747+0000 7f065bfff700 10 client.15462 handle_command_reply: tid=0 2023-12-01T01:17:51.754 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.747+0000 7f068eda8b80 1 client.15462 shutdown 2023-12-01T01:17:51.755 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.747+0000 7f068eda8b80 1 -- 172.21.15.77:0/2392753978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee05599470 msgr2=0x55ee055a29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:51.755 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.747+0000 7f068eda8b80 1 --2- 172.21.15.77:0/2392753978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee05599470 0x55ee055a29a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f0670000c00 tx=0x7f0670009cd0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.755 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.747+0000 7f068eda8b80 1 -- 172.21.15.77:0/2392753978 shutdown_connections 2023-12-01T01:17:51.755 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.747+0000 7f068eda8b80 1 --2- 172.21.15.77:0/2392753978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee0559aaf0 0x55ee055a9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.755 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.747+0000 7f068eda8b80 1 --2- 172.21.15.77:0/2392753978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee05599470 0x55ee055a29a0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.755 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.747+0000 7f068eda8b80 1 --2- 172.21.15.77:0/2392753978 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55ee055ae730 0x55ee055b0b50 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.755 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.747+0000 7f068eda8b80 1 --2- 172.21.15.77:0/2392753978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee0559a170 0x55ee055a2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.755 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.747+0000 7f065bfff700 0 client.15462 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:17:51.755 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.747+0000 7f068eda8b80 1 -- 172.21.15.77:0/2392753978 >> 172.21.15.77:0/2392753978 conn(0x55ee054f8150 msgr2=0x55ee05593130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:51.755 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.747+0000 7f068eda8b80 1 -- 172.21.15.77:0/2392753978 shutdown_connections 2023-12-01T01:17:51.755 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.747+0000 7f068eda8b80 1 -- 172.21.15.77:0/2392753978 wait complete. 2023-12-01T01:17:51.755 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.747+0000 7f068eda8b80 20 client.15462 trim_cache size 0 max 16384 2023-12-01T01:17:51.756 INFO:tasks.workunit.client.0.smithi077.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-12-01T01:17:51.771 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.764+0000 7f068eda8b80 1 Processor -- start 2023-12-01T01:17:51.771 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.764+0000 7f068eda8b80 1 -- start start 2023-12-01T01:17:51.771 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.764+0000 7f068eda8b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee05599470 0x55ee055a7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:51.771 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.764+0000 7f068eda8b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee0559a170 0x55ee055a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:51.771 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.764+0000 7f068eda8b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee0559aaf0 0x55ee055a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:51.772 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.764+0000 7f068eda8b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee055a1e50 con 0x55ee0559aaf0 2023-12-01T01:17:51.772 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.764+0000 7f068eda8b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee055a0560 con 0x55ee0559a170 2023-12-01T01:17:51.772 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.764+0000 7f068eda8b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee0559dc20 con 0x55ee05599470 2023-12-01T01:17:51.772 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.764+0000 7f06811b5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee0559aaf0 0x55ee055a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:51.772 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.764+0000 7f06811b5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee0559aaf0 0x55ee055a0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59286/0 (socket says 172.21.15.77:59286) 2023-12-01T01:17:51.772 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.764+0000 7f06811b5700 1 -- 172.21.15.77:0/2557438700 learned_addr learned my addr 172.21.15.77:0/2557438700 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:51.772 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.764+0000 7f067bfff700 1 --2- 172.21.15.77:0/2557438700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee0559a170 0x55ee055a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:51.772 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.764+0000 7f06809b4700 1 --2- 172.21.15.77:0/2557438700 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee05599470 0x55ee055a7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:51.772 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.764+0000 7f06811b5700 1 -- 172.21.15.77:0/2557438700 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee05599470 msgr2=0x55ee055a7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:51.772 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.764+0000 7f06811b5700 1 --2- 172.21.15.77:0/2557438700 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee05599470 0x55ee055a7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.772 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.764+0000 7f06811b5700 1 -- 172.21.15.77:0/2557438700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee0559a170 msgr2=0x55ee055a07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:51.772 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.764+0000 7f06811b5700 1 --2- 172.21.15.77:0/2557438700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee0559a170 0x55ee055a07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.772 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.764+0000 7f06811b5700 1 -- 172.21.15.77:0/2557438700 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee055a1a00 con 0x55ee0559aaf0 2023-12-01T01:17:51.772 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.764+0000 7f06809b4700 1 --2- 172.21.15.77:0/2557438700 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee05599470 0x55ee055a7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:17:51.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.764+0000 7f067bfff700 1 --2- 172.21.15.77:0/2557438700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee0559a170 0x55ee055a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:17:51.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.765+0000 7f06811b5700 1 --2- 172.21.15.77:0/2557438700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee0559aaf0 0x55ee055a0cf0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0674094ea0 tx=0x7f06740b8040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f094f427c402d47a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:51.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.765+0000 7f06677fe700 1 -- 172.21.15.77:0/2557438700 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06740a0a10 con 0x55ee0559aaf0 2023-12-01T01:17:51.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.765+0000 7f06677fe700 1 -- 172.21.15.77:0/2557438700 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0674096d40 con 0x55ee0559aaf0 2023-12-01T01:17:51.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.765+0000 7f06677fe700 1 -- 172.21.15.77:0/2557438700 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06740224c0 con 0x55ee0559aaf0 2023-12-01T01:17:51.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.765+0000 7f068eda8b80 1 -- 172.21.15.77:0/2557438700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee0559aaf0 msgr2=0x55ee055a0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:51.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.765+0000 7f068eda8b80 1 --2- 172.21.15.77:0/2557438700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee0559aaf0 0x55ee055a0cf0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0674094ea0 tx=0x7f06740b8040 comp rx=0 tx=0).stop 2023-12-01T01:17:51.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.765+0000 7f068eda8b80 1 -- 172.21.15.77:0/2557438700 shutdown_connections 2023-12-01T01:17:51.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.765+0000 7f068eda8b80 1 --2- 172.21.15.77:0/2557438700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee0559aaf0 0x55ee055a0cf0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.765+0000 7f068eda8b80 1 --2- 172.21.15.77:0/2557438700 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee05599470 0x55ee055a7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.765+0000 7f068eda8b80 1 --2- 172.21.15.77:0/2557438700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee0559a170 0x55ee055a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.765+0000 7f068eda8b80 1 -- 172.21.15.77:0/2557438700 >> 172.21.15.77:0/2557438700 conn(0x55ee054f8150 msgr2=0x55ee05593990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:51.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.765+0000 7f068eda8b80 1 -- 172.21.15.77:0/2557438700 shutdown_connections 2023-12-01T01:17:51.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.766+0000 7f068eda8b80 1 -- 172.21.15.77:0/2557438700 wait complete. 2023-12-01T01:17:51.774 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.766+0000 7f068eda8b80 1 -- start start 2023-12-01T01:17:51.774 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.766+0000 7f068eda8b80 1 Processor -- start 2023-12-01T01:17:51.774 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.766+0000 7f068eda8b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee05599470 0x55ee055a6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:51.774 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.766+0000 7f068eda8b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee0559a170 0x55ee055a7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:51.774 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.766+0000 7f068eda8b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee0559aaf0 0x55ee056046b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:51.774 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.766+0000 7f068eda8b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee05604c20 con 0x55ee05599470 2023-12-01T01:17:51.774 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.766+0000 7f068eda8b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee05604d90 con 0x55ee0559a170 2023-12-01T01:17:51.774 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.766+0000 7f068eda8b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee05604f50 con 0x55ee0559aaf0 2023-12-01T01:17:51.774 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.766+0000 7f068eda8b80 10 client.0 _ll_register_callbacks cb 0x55ee053d3b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:17:51.774 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.766+0000 7f06809b4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee05599470 0x55ee055a6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:51.774 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.766+0000 7f06809b4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee05599470 0x55ee055a6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59288/0 (socket says 172.21.15.77:59288) 2023-12-01T01:17:51.774 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.766+0000 7f06809b4700 1 -- 172.21.15.77:0/3378994859 learned_addr learned my addr 172.21.15.77:0/3378994859 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:51.774 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.766+0000 7f06811b5700 1 --2- 172.21.15.77:0/3378994859 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee0559aaf0 0x55ee056046b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:51.775 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.767+0000 7f067bfff700 1 --2- 172.21.15.77:0/3378994859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee0559a170 0x55ee055a7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:51.775 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.767+0000 7f06809b4700 1 -- 172.21.15.77:0/3378994859 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee0559aaf0 msgr2=0x55ee056046b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:51.775 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.767+0000 7f06809b4700 1 --2- 172.21.15.77:0/3378994859 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee0559aaf0 0x55ee056046b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.775 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.767+0000 7f06809b4700 1 -- 172.21.15.77:0/3378994859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee0559a170 msgr2=0x55ee055a7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:51.775 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.767+0000 7f06809b4700 1 --2- 172.21.15.77:0/3378994859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee0559a170 0x55ee055a7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.775 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.767+0000 7f06809b4700 1 -- 172.21.15.77:0/3378994859 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f067c19f080 con 0x55ee05599470 2023-12-01T01:17:51.775 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.767+0000 7f067bfff700 1 --2- 172.21.15.77:0/3378994859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee0559a170 0x55ee055a7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:17:51.775 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.767+0000 7f06809b4700 1 --2- 172.21.15.77:0/3378994859 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee05599470 0x55ee055a6ef0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f06700058e0 tx=0x7f0670037c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e206943eb3c68f8c server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:51.775 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.767+0000 7f065bfff700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:17:51.775 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.767+0000 7f065bfff700 1 -- 172.21.15.77:0/3378994859 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0670029050 con 0x55ee05599470 2023-12-01T01:17:51.775 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.767+0000 7f065bfff700 1 -- 172.21.15.77:0/3378994859 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f067003e080 con 0x55ee05599470 2023-12-01T01:17:51.775 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.767+0000 7f068eda8b80 1 -- 172.21.15.77:0/3378994859 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0668004390 con 0x55ee05599470 2023-12-01T01:17:51.775 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.767+0000 7f065bfff700 1 -- 172.21.15.77:0/3378994859 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f067000bb60 con 0x55ee05599470 2023-12-01T01:17:51.776 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.767+0000 7f068eda8b80 10 client.15468 fetch_fsmap learned FSMap version 27 2023-12-01T01:17:51.776 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.767+0000 7f068eda8b80 1 -- 172.21.15.77:0/3378994859 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ee0533e2d0 con 0x55ee05599470 2023-12-01T01:17:51.776 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.767+0000 7f065bfff700 1 -- 172.21.15.77:0/3378994859 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0668004390 con 0x55ee05599470 2023-12-01T01:17:51.776 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.768+0000 7f065bfff700 1 -- 172.21.15.77:0/3378994859 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f06700427e0 con 0x55ee05599470 2023-12-01T01:17:51.776 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.768+0000 7f068eda8b80 10 client.15468 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:17:51.776 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.768+0000 7f068eda8b80 10 client.15468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-01T01:17:51.776 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.768+0000 7f068eda8b80 20 client.15468 populate_metadata read hostname 'smithi077' 2023-12-01T01:17:51.776 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.768+0000 7f068eda8b80 1 --2- 172.21.15.77:0/3378994859 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55ee05606410 0x55ee05608800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:51.776 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.768+0000 7f068eda8b80 4 client.15468 mds_command: new command op to 24500 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-12-01T01:17:51.776 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.768+0000 7f068eda8b80 1 -- 172.21.15.77:0/3378994859 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ee054e2dc0 con 0x55ee05606410 2023-12-01T01:17:51.776 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.768+0000 7f067bfff700 1 --2- 172.21.15.77:0/3378994859 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55ee05606410 0x55ee05608800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:51.776 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.768+0000 7f065bfff700 10 client.15468 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:17:51.776 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.768+0000 7f067bfff700 1 --2- 172.21.15.77:0/3378994859 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55ee05606410 0x55ee05608800 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de47af536f73dd21 server_cookie=6d96d9eb5f20393b in_seq=0 out_seq=0 2023-12-01T01:17:51.776 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.769+0000 7f065bfff700 1 -- 172.21.15.77:0/3378994859 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ee054e2dc0 con 0x55ee05606410 2023-12-01T01:17:51.776 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.769+0000 7f065bfff700 10 client.15468 handle_command_reply: tid=0 2023-12-01T01:17:51.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.769+0000 7f068eda8b80 1 client.15468 shutdown 2023-12-01T01:17:51.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.770+0000 7f068eda8b80 1 -- 172.21.15.77:0/3378994859 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee05599470 msgr2=0x55ee055a6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:51.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.770+0000 7f068eda8b80 1 --2- 172.21.15.77:0/3378994859 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee05599470 0x55ee055a6ef0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f06700058e0 tx=0x7f0670037c60 comp rx=0 tx=0).stop 2023-12-01T01:17:51.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.770+0000 7f068eda8b80 1 -- 172.21.15.77:0/3378994859 shutdown_connections 2023-12-01T01:17:51.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.770+0000 7f068eda8b80 1 --2- 172.21.15.77:0/3378994859 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee0559aaf0 0x55ee056046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.770+0000 7f068eda8b80 1 --2- 172.21.15.77:0/3378994859 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee05599470 0x55ee055a6ef0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.770+0000 7f068eda8b80 1 --2- 172.21.15.77:0/3378994859 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55ee05606410 0x55ee05608800 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.770+0000 7f068eda8b80 1 --2- 172.21.15.77:0/3378994859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee0559a170 0x55ee055a7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.770+0000 7f065bfff700 0 client.15468 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:17:51.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.770+0000 7f068eda8b80 1 -- 172.21.15.77:0/3378994859 >> 172.21.15.77:0/3378994859 conn(0x55ee054f8150 msgr2=0x55ee055a07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:51.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.770+0000 7f068eda8b80 1 -- 172.21.15.77:0/3378994859 shutdown_connections 2023-12-01T01:17:51.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.770+0000 7f068eda8b80 1 -- 172.21.15.77:0/3378994859 wait complete. 2023-12-01T01:17:51.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.770+0000 7f068eda8b80 20 client.15468 trim_cache size 0 max 16384 2023-12-01T01:17:51.778 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2023-12-01T01:17:51.778 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2023-12-01T01:17:51.778 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "7b4b436d-2a99-44ea-9dae-f1d1076178b4", 2023-12-01T01:17:51.778 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2023-12-01T01:17:51.778 INFO:teuthology.orchestra.run.smithi077.stdout:} 2023-12-01T01:17:51.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.770+0000 7f0682c18700 1 -- 172.21.15.77:0/617438846 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f066c0a05b0 msgr2=0x7f066c0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:51.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.770+0000 7f0682c18700 1 --2- 172.21.15.77:0/617438846 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f066c0a05b0 0x7f066c0a2a70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0670009990 tx=0x7f0670005fb0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.770+0000 7f0682c18700 1 -- 172.21.15.77:0/617438846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f067c101810 msgr2=0x7f067c19de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:51.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.770+0000 7f0682c18700 1 --2- 172.21.15.77:0/617438846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f067c101810 0x7f067c19de10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0674007ee0 tx=0x7f067400d3b0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.770+0000 7f0682c18700 1 -- 172.21.15.77:0/617438846 shutdown_connections 2023-12-01T01:17:51.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.770+0000 7f0682c18700 1 --2- 172.21.15.77:0/617438846 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f066c0a05b0 0x7f066c0a2a70 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.770+0000 7f0682c18700 1 --2- 172.21.15.77:0/617438846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f067c101810 0x7f067c19de10 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.770+0000 7f0682c18700 1 --2- 172.21.15.77:0/617438846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f067c0ffc80 0x7f067c068a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.770+0000 7f0682c18700 1 --2- 172.21.15.77:0/617438846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f067c100e70 0x7f067c19b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.770+0000 7f0682c18700 1 -- 172.21.15.77:0/617438846 >> 172.21.15.77:0/617438846 conn(0x7f067c0fb1f0 msgr2=0x7f067c0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:51.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.771+0000 7f0682c18700 1 -- 172.21.15.77:0/617438846 shutdown_connections 2023-12-01T01:17:51.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.771+0000 7f0682c18700 1 -- 172.21.15.77:0/617438846 wait complete. 2023-12-01T01:17:51.784 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-01T01:17:51.847 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:51 smithi077 ceph-mon[98867]: pgmap v297: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 10 op/s 2023-12-01T01:17:51.848 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:51 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/1237220698' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-01T01:17:51.848 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:51 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/4217245699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:17:51.848 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:51 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/688106184' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:17:51.947 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.946+0000 7f029a0b0700 1 -- 172.21.15.77:0/926699402 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0294104160 msgr2=0x7f0294106550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:51.948 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.946+0000 7f029a0b0700 1 --2- 172.21.15.77:0/926699402 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0294104160 0x7f0294106550 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0284009990 tx=0x7f0284009ca0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.948 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.946+0000 7f029a0b0700 1 -- 172.21.15.77:0/926699402 shutdown_connections 2023-12-01T01:17:51.948 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.946+0000 7f029a0b0700 1 --2- 172.21.15.77:0/926699402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0294106a90 0x7f0294108f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.948 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.946+0000 7f029a0b0700 1 --2- 172.21.15.77:0/926699402 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0294101830 0x7f0294103c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.948 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.946+0000 7f029a0b0700 1 --2- 172.21.15.77:0/926699402 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0294104160 0x7f0294106550 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.948 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.946+0000 7f029a0b0700 1 -- 172.21.15.77:0/926699402 >> 172.21.15.77:0/926699402 conn(0x7f02940fb220 msgr2=0x7f02940fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:51.948 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.946+0000 7f029a0b0700 1 -- 172.21.15.77:0/926699402 shutdown_connections 2023-12-01T01:17:51.948 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.946+0000 7f029a0b0700 1 -- 172.21.15.77:0/926699402 wait complete. 2023-12-01T01:17:51.948 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.947+0000 7f029a0b0700 1 Processor -- start 2023-12-01T01:17:51.950 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.947+0000 7f029a0b0700 1 -- start start 2023-12-01T01:17:51.950 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.947+0000 7f029a0b0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0294101830 0x7f0294196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:51.950 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.947+0000 7f029a0b0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0294104160 0x7f0294196f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:51.950 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.947+0000 7f029a0b0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0294106a90 0x7f029419dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:51.950 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.947+0000 7f029a0b0700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029419e590 con 0x7f0294106a90 2023-12-01T01:17:51.950 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.947+0000 7f029a0b0700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029419e6d0 con 0x7f0294101830 2023-12-01T01:17:51.950 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.947+0000 7f029a0b0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029419e860 con 0x7f0294104160 2023-12-01T01:17:51.950 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.947+0000 7f0293fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0294106a90 0x7f029419dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:51.951 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.947+0000 7f0293fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0294106a90 0x7f029419dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59308/0 (socket says 172.21.15.77:59308) 2023-12-01T01:17:51.951 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.947+0000 7f0293fff700 1 -- 172.21.15.77:0/3061681871 learned_addr learned my addr 172.21.15.77:0/3061681871 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:51.951 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.947+0000 7f02937fe700 1 --2- 172.21.15.77:0/3061681871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0294101830 0x7f0294196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:51.951 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.947+0000 7f0292ffd700 1 --2- 172.21.15.77:0/3061681871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0294104160 0x7f0294196f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:51.951 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.947+0000 7f0293fff700 1 -- 172.21.15.77:0/3061681871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0294104160 msgr2=0x7f0294196f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:51.951 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.947+0000 7f0293fff700 1 --2- 172.21.15.77:0/3061681871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0294104160 0x7f0294196f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.951 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.947+0000 7f0293fff700 1 -- 172.21.15.77:0/3061681871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0294101830 msgr2=0x7f0294196a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:51.951 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.947+0000 7f0293fff700 1 --2- 172.21.15.77:0/3061681871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0294101830 0x7f0294196a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.951 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.947+0000 7f0293fff700 1 -- 172.21.15.77:0/3061681871 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02840095f0 con 0x7f0294106a90 2023-12-01T01:17:51.951 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.948+0000 7f0293fff700 1 --2- 172.21.15.77:0/3061681871 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0294106a90 0x7f029419dfc0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f028c007d70 tx=0x7f028c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:51.951 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.948+0000 7f0290ff9700 1 -- 172.21.15.77:0/3061681871 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028c00ac90 con 0x7f0294106a90 2023-12-01T01:17:51.951 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.948+0000 7f0290ff9700 1 -- 172.21.15.77:0/3061681871 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f028c00adf0 con 0x7f0294106a90 2023-12-01T01:17:51.951 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.948+0000 7f029a0b0700 1 -- 172.21.15.77:0/3061681871 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f029419eb40 con 0x7f0294106a90 2023-12-01T01:17:51.952 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.948+0000 7f0290ff9700 1 -- 172.21.15.77:0/3061681871 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028c0115e0 con 0x7f0294106a90 2023-12-01T01:17:51.952 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.948+0000 7f029a0b0700 1 -- 172.21.15.77:0/3061681871 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f029419f050 con 0x7f0294106a90 2023-12-01T01:17:51.954 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.950+0000 7f0290ff9700 1 -- 172.21.15.77:0/3061681871 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) v1 ==== 135121+0+0 (secure 0 0 0) 0x7f028c012420 con 0x7f0294106a90 2023-12-01T01:17:51.954 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.950+0000 7f0290ff9700 1 --2- 172.21.15.77:0/3061681871 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f027c0a05b0 0x7f027c0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:51.954 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.951+0000 7f0290ff9700 1 -- 172.21.15.77:0/3061681871 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f028c0cbb80 con 0x7f0294106a90 2023-12-01T01:17:51.955 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.951+0000 7f02937fe700 1 --2- 172.21.15.77:0/3061681871 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f027c0a05b0 0x7f027c0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:51.955 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.951+0000 7f02937fe700 1 --2- 172.21.15.77:0/3061681871 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f027c0a05b0 0x7f027c0a2a70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0288009c60 tx=0x7f0288009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:51.955 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.951+0000 7f02a6240b80 1 Processor -- start 2023-12-01T01:17:51.955 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.951+0000 7f02a6240b80 1 -- start start 2023-12-01T01:17:51.955 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.952+0000 7f02a6240b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ceee970f0 0x556ceee97530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:51.955 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.952+0000 7f02a6240b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ceee97a70 0x556ceee95e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:51.955 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.952+0000 7f02a6240b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ceee963f0 0x556ceee96870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:51.955 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.952+0000 7f02a6240b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ceededa10 con 0x556ceee97a70 2023-12-01T01:17:51.955 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.952+0000 7f02a6240b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ceedeb3d0 con 0x556ceee963f0 2023-12-01T01:17:51.955 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.952+0000 7f02a6240b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ceede6cc0 con 0x556ceee970f0 2023-12-01T01:17:51.955 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.952+0000 7f0292ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ceee97a70 0x556ceee95e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:51.955 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.952+0000 7f0292ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ceee97a70 0x556ceee95e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59314/0 (socket says 172.21.15.77:59314) 2023-12-01T01:17:51.955 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.952+0000 7f0292ffd700 1 -- 172.21.15.77:0/3040887870 learned_addr learned my addr 172.21.15.77:0/3040887870 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:51.955 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.952+0000 7f0293fff700 1 --2- 172.21.15.77:0/3040887870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ceee963f0 0x556ceee96870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:51.956 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.952+0000 7f02937fe700 1 --2- 172.21.15.77:0/3040887870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ceee970f0 0x556ceee97530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:51.956 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.952+0000 7f0292ffd700 1 -- 172.21.15.77:0/3040887870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ceee970f0 msgr2=0x556ceee97530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:51.956 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.952+0000 7f0292ffd700 1 --2- 172.21.15.77:0/3040887870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ceee970f0 0x556ceee97530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.956 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.952+0000 7f0292ffd700 1 -- 172.21.15.77:0/3040887870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ceee963f0 msgr2=0x556ceee96870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:51.956 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.952+0000 7f0292ffd700 1 --2- 172.21.15.77:0/3040887870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ceee963f0 0x556ceee96870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.956 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.952+0000 7f0292ffd700 1 -- 172.21.15.77:0/3040887870 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ceec14de0 con 0x556ceee97a70 2023-12-01T01:17:51.956 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.952+0000 7f0292ffd700 1 --2- 172.21.15.77:0/3040887870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ceee97a70 0x556ceee95e60 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0284015000 tx=0x7f0284005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b768990b8aa6bbe server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:51.956 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.952+0000 7f0279ffb700 1 -- 172.21.15.77:0/3040887870 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028401d070 con 0x556ceee97a70 2023-12-01T01:17:51.956 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.952+0000 7f0279ffb700 1 -- 172.21.15.77:0/3040887870 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f028401e040 con 0x556ceee97a70 2023-12-01T01:17:51.956 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.952+0000 7f0279ffb700 1 -- 172.21.15.77:0/3040887870 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0284004b40 con 0x556ceee97a70 2023-12-01T01:17:51.956 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.953+0000 7f02a6240b80 1 -- 172.21.15.77:0/3040887870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ceee97a70 msgr2=0x556ceee95e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:51.956 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.953+0000 7f02a6240b80 1 --2- 172.21.15.77:0/3040887870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ceee97a70 0x556ceee95e60 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0284015000 tx=0x7f0284005c70 comp rx=0 tx=0).stop 2023-12-01T01:17:51.956 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.953+0000 7f02a6240b80 1 -- 172.21.15.77:0/3040887870 shutdown_connections 2023-12-01T01:17:51.956 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.953+0000 7f02a6240b80 1 --2- 172.21.15.77:0/3040887870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ceee963f0 0x556ceee96870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.957 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.953+0000 7f02a6240b80 1 --2- 172.21.15.77:0/3040887870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ceee970f0 0x556ceee97530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.957 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.953+0000 7f02a6240b80 1 --2- 172.21.15.77:0/3040887870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ceee97a70 0x556ceee95e60 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.957 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.953+0000 7f02a6240b80 1 -- 172.21.15.77:0/3040887870 >> 172.21.15.77:0/3040887870 conn(0x556ceedf50d0 msgr2=0x556ceee90940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:51.957 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.953+0000 7f02a6240b80 1 -- 172.21.15.77:0/3040887870 shutdown_connections 2023-12-01T01:17:51.957 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.953+0000 7f02a6240b80 1 -- 172.21.15.77:0/3040887870 wait complete. 2023-12-01T01:17:51.957 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.953+0000 7f02a6240b80 1 -- start start 2023-12-01T01:17:51.960 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.954+0000 7f02a6240b80 1 Processor -- start 2023-12-01T01:17:51.961 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.954+0000 7f02a6240b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ceee963f0 0x556ceee9f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:51.961 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.954+0000 7f02a6240b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ceee970f0 0x556ceee9fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:51.961 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.954+0000 7f02a6240b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ceee97a70 0x556ceeea6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:51.961 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.954+0000 7f02a6240b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ceed83510 con 0x556ceee970f0 2023-12-01T01:17:51.961 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.954+0000 7f02a6240b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ceed831b0 con 0x556ceee963f0 2023-12-01T01:17:51.961 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.954+0000 7f02a6240b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ceec6a920 con 0x556ceee97a70 2023-12-01T01:17:51.961 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.954+0000 7f02a6240b80 10 client.0 _ll_register_callbacks cb 0x556ceecd0a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:17:51.961 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.954+0000 7f0292ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ceee970f0 0x556ceee9fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:51.961 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.954+0000 7f0292ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ceee970f0 0x556ceee9fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59320/0 (socket says 172.21.15.77:59320) 2023-12-01T01:17:51.961 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.954+0000 7f0292ffd700 1 -- 172.21.15.77:0/508280144 learned_addr learned my addr 172.21.15.77:0/508280144 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:51.961 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.954+0000 7f0293fff700 1 --2- 172.21.15.77:0/508280144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ceee97a70 0x556ceeea6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:51.961 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.954+0000 7f02937fe700 1 --2- 172.21.15.77:0/508280144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ceee963f0 0x556ceee9f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:51.961 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.954+0000 7f0292ffd700 1 -- 172.21.15.77:0/508280144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ceee97a70 msgr2=0x556ceeea6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:51.962 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.954+0000 7f0292ffd700 1 --2- 172.21.15.77:0/508280144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ceee97a70 0x556ceeea6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.962 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.954+0000 7f0292ffd700 1 -- 172.21.15.77:0/508280144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ceee963f0 msgr2=0x556ceee9f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:51.962 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.954+0000 7f0292ffd700 1 --2- 172.21.15.77:0/508280144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ceee963f0 0x556ceee9f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.962 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.954+0000 7f0292ffd700 1 -- 172.21.15.77:0/508280144 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0294109af0 con 0x556ceee970f0 2023-12-01T01:17:51.962 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.954+0000 7f0292ffd700 1 --2- 172.21.15.77:0/508280144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ceee970f0 0x556ceee9fe30 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0284009350 tx=0x7f0284022bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cabc5825fb904d6e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:51.962 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.954+0000 7f0272ffd700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:17:51.962 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.955+0000 7f0272ffd700 1 -- 172.21.15.77:0/508280144 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028401d030 con 0x556ceee970f0 2023-12-01T01:17:51.962 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.955+0000 7f0272ffd700 1 -- 172.21.15.77:0/508280144 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f028400f660 con 0x556ceee970f0 2023-12-01T01:17:51.962 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.955+0000 7f02a6240b80 1 -- 172.21.15.77:0/508280144 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556ceee11d10 con 0x556ceee970f0 2023-12-01T01:17:51.962 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.955+0000 7f0272ffd700 1 -- 172.21.15.77:0/508280144 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0284022e50 con 0x556ceee970f0 2023-12-01T01:17:51.962 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.955+0000 7f02a6240b80 10 client.15480 fetch_fsmap learned FSMap version 27 2023-12-01T01:17:51.962 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.955+0000 7f02a6240b80 1 -- 172.21.15.77:0/508280144 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556ceebfdc60 con 0x556ceee970f0 2023-12-01T01:17:51.962 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.955+0000 7f0272ffd700 1 -- 172.21.15.77:0/508280144 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556ceee11d10 con 0x556ceee970f0 2023-12-01T01:17:51.963 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.955+0000 7f0272ffd700 1 -- 172.21.15.77:0/508280144 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f028402c520 con 0x556ceee970f0 2023-12-01T01:17:51.963 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.955+0000 7f02a6240b80 10 client.15480 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:17:51.963 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.955+0000 7f02a6240b80 10 client.15480 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-01T01:17:51.963 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.955+0000 7f02a6240b80 20 client.15480 populate_metadata read hostname 'smithi077' 2023-12-01T01:17:51.963 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.955+0000 7f02a6240b80 1 --2- 172.21.15.77:0/508280144 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x556ceeeab1a0 0x556ceeead5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:51.963 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.955+0000 7f02a6240b80 4 client.15480 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:17:51.963 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.955+0000 7f02a6240b80 1 -- 172.21.15.77:0/508280144 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556ceee64e10 con 0x556ceeeab1a0 2023-12-01T01:17:51.963 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.955+0000 7f02937fe700 1 --2- 172.21.15.77:0/508280144 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x556ceeeab1a0 0x556ceeead5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:51.963 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.956+0000 7f0272ffd700 10 client.15480 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:17:51.963 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.956+0000 7f02937fe700 1 --2- 172.21.15.77:0/508280144 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x556ceeeab1a0 0x556ceeead5c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2509013a503bb20d server_cookie=2554817e08b19203 in_seq=0 out_seq=0 2023-12-01T01:17:51.963 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.957+0000 7f0272ffd700 1 -- 172.21.15.77:0/508280144 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x556ceee64e10 con 0x556ceeeab1a0 2023-12-01T01:17:51.963 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.957+0000 7f0272ffd700 10 client.15480 handle_command_reply: tid=0 2023-12-01T01:17:51.963 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.957+0000 7f02a6240b80 1 client.15480 shutdown 2023-12-01T01:17:51.964 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.957+0000 7f02a6240b80 1 -- 172.21.15.77:0/508280144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ceee970f0 msgr2=0x556ceee9fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:51.964 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.957+0000 7f02a6240b80 1 --2- 172.21.15.77:0/508280144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ceee970f0 0x556ceee9fe30 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0284009350 tx=0x7f0284022bb0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.964 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.957+0000 7f02a6240b80 1 -- 172.21.15.77:0/508280144 shutdown_connections 2023-12-01T01:17:51.964 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.957+0000 7f02a6240b80 1 --2- 172.21.15.77:0/508280144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ceee97a70 0x556ceeea6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.964 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.957+0000 7f02a6240b80 1 --2- 172.21.15.77:0/508280144 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x556ceeeab1a0 0x556ceeead5c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.964 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.957+0000 7f02a6240b80 1 --2- 172.21.15.77:0/508280144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ceee963f0 0x556ceee9f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.964 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.957+0000 7f02a6240b80 1 --2- 172.21.15.77:0/508280144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ceee970f0 0x556ceee9fe30 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.964 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.957+0000 7f0272ffd700 0 client.15480 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:17:51.964 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.957+0000 7f02a6240b80 1 -- 172.21.15.77:0/508280144 >> 172.21.15.77:0/508280144 conn(0x556ceedf50d0 msgr2=0x556ceee900b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:51.964 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.957+0000 7f02a6240b80 1 -- 172.21.15.77:0/508280144 shutdown_connections 2023-12-01T01:17:51.964 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.957+0000 7f02a6240b80 1 -- 172.21.15.77:0/508280144 wait complete. 2023-12-01T01:17:51.964 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.957+0000 7f02a6240b80 20 client.15480 trim_cache size 0 max 16384 2023-12-01T01:17:51.979 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.973+0000 7f02a6240b80 1 Processor -- start 2023-12-01T01:17:51.979 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.973+0000 7f02a6240b80 1 -- start start 2023-12-01T01:17:51.980 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.973+0000 7f02a6240b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ceee963f0 0x556ceee95e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:51.980 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.973+0000 7f02a6240b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ceee970f0 0x556ceeea4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:51.980 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.973+0000 7f02a6240b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ceee97a70 0x556ceee9d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:51.980 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.973+0000 7f02a6240b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ceee9f460 con 0x556ceee970f0 2023-12-01T01:17:51.980 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.973+0000 7f02a6240b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ceee91240 con 0x556ceee963f0 2023-12-01T01:17:51.980 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.973+0000 7f02a6240b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ceee92170 con 0x556ceee97a70 2023-12-01T01:17:51.980 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.974+0000 7f0292ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ceee970f0 0x556ceeea4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:51.980 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.974+0000 7f0292ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ceee970f0 0x556ceeea4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59328/0 (socket says 172.21.15.77:59328) 2023-12-01T01:17:51.980 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.974+0000 7f0292ffd700 1 -- 172.21.15.77:0/3041705858 learned_addr learned my addr 172.21.15.77:0/3041705858 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:51.980 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.974+0000 7f0293fff700 1 --2- 172.21.15.77:0/3041705858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ceee97a70 0x556ceee9d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:51.980 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.974+0000 7f02937fe700 1 --2- 172.21.15.77:0/3041705858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ceee963f0 0x556ceee95e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:51.980 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.974+0000 7f0292ffd700 1 -- 172.21.15.77:0/3041705858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ceee97a70 msgr2=0x556ceee9d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:51.981 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.974+0000 7f0292ffd700 1 --2- 172.21.15.77:0/3041705858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ceee97a70 0x556ceee9d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.981 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.974+0000 7f0292ffd700 1 -- 172.21.15.77:0/3041705858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ceee963f0 msgr2=0x556ceee95e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:51.981 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.974+0000 7f0292ffd700 1 --2- 172.21.15.77:0/3041705858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ceee963f0 0x556ceee95e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.981 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.974+0000 7f0292ffd700 1 -- 172.21.15.77:0/3041705858 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ceec3b250 con 0x556ceee970f0 2023-12-01T01:17:51.981 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.974+0000 7f0293fff700 1 --2- 172.21.15.77:0/3041705858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ceee97a70 0x556ceee9d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:17:51.981 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.974+0000 7f0292ffd700 1 --2- 172.21.15.77:0/3041705858 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ceee970f0 0x556ceeea4e20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0284009990 tx=0x7f028402cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=779f8e37b9e78723 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:51.981 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.974+0000 7f027a7fc700 1 -- 172.21.15.77:0/3041705858 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028401d030 con 0x556ceee970f0 2023-12-01T01:17:51.981 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.974+0000 7f027a7fc700 1 -- 172.21.15.77:0/3041705858 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f028402cbf0 con 0x556ceee970f0 2023-12-01T01:17:51.981 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.974+0000 7f027a7fc700 1 -- 172.21.15.77:0/3041705858 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028400fb80 con 0x556ceee970f0 2023-12-01T01:17:51.981 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.975+0000 7f02a6240b80 1 -- 172.21.15.77:0/3041705858 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ceee970f0 msgr2=0x556ceeea4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:51.981 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.975+0000 7f02a6240b80 1 --2- 172.21.15.77:0/3041705858 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ceee970f0 0x556ceeea4e20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0284009990 tx=0x7f028402cb20 comp rx=0 tx=0).stop 2023-12-01T01:17:51.981 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.975+0000 7f02a6240b80 1 -- 172.21.15.77:0/3041705858 shutdown_connections 2023-12-01T01:17:51.981 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.975+0000 7f02a6240b80 1 --2- 172.21.15.77:0/3041705858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ceee97a70 0x556ceee9d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.981 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.975+0000 7f02a6240b80 1 --2- 172.21.15.77:0/3041705858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ceee963f0 0x556ceee95e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.982 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.975+0000 7f02a6240b80 1 --2- 172.21.15.77:0/3041705858 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ceee970f0 0x556ceeea4e20 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.982 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.975+0000 7f02a6240b80 1 -- 172.21.15.77:0/3041705858 >> 172.21.15.77:0/3041705858 conn(0x556ceedf50d0 msgr2=0x556ceeea7700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:51.982 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.975+0000 7f02a6240b80 1 -- 172.21.15.77:0/3041705858 shutdown_connections 2023-12-01T01:17:51.982 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.975+0000 7f02a6240b80 1 -- 172.21.15.77:0/3041705858 wait complete. 2023-12-01T01:17:51.982 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.975+0000 7f02a6240b80 1 -- start start 2023-12-01T01:17:51.982 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.975+0000 7f02a6240b80 1 Processor -- start 2023-12-01T01:17:51.982 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.975+0000 7f02a6240b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ceee963f0 0x556ceeea2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:51.982 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.975+0000 7f02a6240b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ceee970f0 0x556ceeea3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:51.982 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.975+0000 7f02a6240b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ceee97a70 0x556ceeea4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:51.982 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.976+0000 7f02a6240b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ceeea4880 con 0x556ceee97a70 2023-12-01T01:17:51.982 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.976+0000 7f02a6240b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ceeea49f0 con 0x556ceee963f0 2023-12-01T01:17:51.982 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.976+0000 7f02a6240b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ceeea4bb0 con 0x556ceee970f0 2023-12-01T01:17:51.982 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.976+0000 7f02a6240b80 10 client.0 _ll_register_callbacks cb 0x7f028c012bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:17:51.983 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.976+0000 7f0293fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ceee97a70 0x556ceeea4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:51.983 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.976+0000 7f0293fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ceee97a70 0x556ceeea4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59336/0 (socket says 172.21.15.77:59336) 2023-12-01T01:17:51.983 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.976+0000 7f0293fff700 1 -- 172.21.15.77:0/553409162 learned_addr learned my addr 172.21.15.77:0/553409162 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:17:51.983 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.976+0000 7f02937fe700 1 --2- 172.21.15.77:0/553409162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ceee963f0 0x556ceeea2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:51.983 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.976+0000 7f0292ffd700 1 --2- 172.21.15.77:0/553409162 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ceee970f0 0x556ceeea3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:51.983 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.976+0000 7f0293fff700 1 -- 172.21.15.77:0/553409162 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ceee970f0 msgr2=0x556ceeea3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:51.983 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.976+0000 7f0293fff700 1 --2- 172.21.15.77:0/553409162 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ceee970f0 0x556ceeea3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.983 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.976+0000 7f0293fff700 1 -- 172.21.15.77:0/553409162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ceee963f0 msgr2=0x556ceeea2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:17:51.983 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.976+0000 7f0293fff700 1 --2- 172.21.15.77:0/553409162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ceee963f0 0x556ceeea2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.983 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.976+0000 7f0293fff700 1 -- 172.21.15.77:0/553409162 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556ceebfdc60 con 0x556ceee97a70 2023-12-01T01:17:51.983 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.976+0000 7f0293fff700 1 --2- 172.21.15.77:0/553409162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ceee97a70 0x556ceeea4340 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f028c012c60 tx=0x7f028c098cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=775d64324b4690c8 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:17:51.983 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.976+0000 7f0272ffd700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:17:51.983 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.976+0000 7f0272ffd700 1 -- 172.21.15.77:0/553409162 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028c082de0 con 0x556ceee97a70 2023-12-01T01:17:51.984 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.976+0000 7f0272ffd700 1 -- 172.21.15.77:0/553409162 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f028c099390 con 0x556ceee97a70 2023-12-01T01:17:51.984 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.976+0000 7f02a6240b80 1 -- 172.21.15.77:0/553409162 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f028401e040 con 0x556ceee97a70 2023-12-01T01:17:51.984 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.976+0000 7f0272ffd700 1 -- 172.21.15.77:0/553409162 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f028c09d1c0 con 0x556ceee97a70 2023-12-01T01:17:51.984 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.976+0000 7f02a6240b80 10 client.15486 fetch_fsmap learned FSMap version 27 2023-12-01T01:17:51.984 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.976+0000 7f02a6240b80 1 -- 172.21.15.77:0/553409162 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556ceec132b0 con 0x556ceee97a70 2023-12-01T01:17:51.984 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.977+0000 7f0272ffd700 1 -- 172.21.15.77:0/553409162 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f028401e040 con 0x556ceee97a70 2023-12-01T01:17:51.984 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.977+0000 7f0272ffd700 1 -- 172.21.15.77:0/553409162 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f028c064f80 con 0x556ceee97a70 2023-12-01T01:17:51.984 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.977+0000 7f02a6240b80 10 client.15486 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:17:51.984 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.977+0000 7f02a6240b80 10 client.15486 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-01T01:17:51.984 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.977+0000 7f02a6240b80 20 client.15486 populate_metadata read hostname 'smithi077' 2023-12-01T01:17:51.984 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.977+0000 7f02a6240b80 1 --2- 172.21.15.77:0/553409162 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x556ceef02270 0x556ceef04690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:17:51.984 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.977+0000 7f02a6240b80 4 client.15486 mds_command: new command op to 24500 tid=0[{"prefix": "scrub status"}] 2023-12-01T01:17:51.984 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.977+0000 7f02a6240b80 1 -- 172.21.15.77:0/553409162 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556ceeddfd40 con 0x556ceef02270 2023-12-01T01:17:51.984 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.977+0000 7f02937fe700 1 --2- 172.21.15.77:0/553409162 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x556ceef02270 0x556ceef04690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:17:51.985 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.978+0000 7f0272ffd700 10 client.15486 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:17:51.985 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.978+0000 7f02937fe700 1 --2- 172.21.15.77:0/553409162 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x556ceef02270 0x556ceef04690 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4e59d5789e6aca9 server_cookie=625e4a2caf37e382 in_seq=0 out_seq=0 2023-12-01T01:17:51.985 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.978+0000 7f0272ffd700 1 -- 172.21.15.77:0/553409162 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x556ceeddfd40 con 0x556ceef02270 2023-12-01T01:17:51.985 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.978+0000 7f0272ffd700 10 client.15486 handle_command_reply: tid=0 2023-12-01T01:17:51.985 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.978+0000 7f02a6240b80 1 client.15486 shutdown 2023-12-01T01:17:51.985 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.978+0000 7f02a6240b80 1 -- 172.21.15.77:0/553409162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ceee97a70 msgr2=0x556ceeea4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:51.985 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.978+0000 7f02a6240b80 1 --2- 172.21.15.77:0/553409162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ceee97a70 0x556ceeea4340 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f028c012c60 tx=0x7f028c098cf0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.985 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.978+0000 7f02a6240b80 1 -- 172.21.15.77:0/553409162 shutdown_connections 2023-12-01T01:17:51.985 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.978+0000 7f02a6240b80 1 --2- 172.21.15.77:0/553409162 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ceee97a70 0x556ceeea4340 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.985 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.978+0000 7f02a6240b80 1 --2- 172.21.15.77:0/553409162 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x556ceef02270 0x556ceef04690 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.985 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.978+0000 7f02a6240b80 1 --2- 172.21.15.77:0/553409162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ceee963f0 0x556ceeea2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.985 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.978+0000 7f0272ffd700 0 client.15486 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:17:51.985 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.978+0000 7f02a6240b80 1 --2- 172.21.15.77:0/553409162 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ceee970f0 0x556ceeea3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.986 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.978+0000 7f02a6240b80 1 -- 172.21.15.77:0/553409162 >> 172.21.15.77:0/553409162 conn(0x556ceedf50d0 msgr2=0x556ceee9d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:51.986 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.978+0000 7f02a6240b80 1 -- 172.21.15.77:0/553409162 shutdown_connections 2023-12-01T01:17:51.986 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.978+0000 7f02a6240b80 1 -- 172.21.15.77:0/553409162 wait complete. 2023-12-01T01:17:51.986 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.978+0000 7f02a6240b80 20 client.15486 trim_cache size 0 max 16384 2023-12-01T01:17:51.986 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2023-12-01T01:17:51.986 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2023-12-01T01:17:51.986 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2023-12-01T01:17:51.986 INFO:teuthology.orchestra.run.smithi077.stdout: "7b4b436d-2a99-44ea-9dae-f1d1076178b4": { 2023-12-01T01:17:51.986 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2023-12-01T01:17:51.986 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "7b4b436d-2a99-44ea-9dae-f1d1076178b4", 2023-12-01T01:17:51.987 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2023-12-01T01:17:51.987 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:17:51.987 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:17:51.987 INFO:teuthology.orchestra.run.smithi077.stdout:} 2023-12-01T01:17:51.987 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.979+0000 7f029a0b0700 1 -- 172.21.15.77:0/3061681871 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f027c0a05b0 msgr2=0x7f027c0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:51.987 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.979+0000 7f029a0b0700 1 --2- 172.21.15.77:0/3061681871 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f027c0a05b0 0x7f027c0a2a70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0288009c60 tx=0x7f0288009260 comp rx=0 tx=0).stop 2023-12-01T01:17:51.988 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.979+0000 7f029a0b0700 1 -- 172.21.15.77:0/3061681871 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0294106a90 msgr2=0x7f029419dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:17:51.988 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.979+0000 7f029a0b0700 1 --2- 172.21.15.77:0/3061681871 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0294106a90 0x7f029419dfc0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f028c007d70 tx=0x7f028c00a340 comp rx=0 tx=0).stop 2023-12-01T01:17:51.988 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.979+0000 7f029a0b0700 1 -- 172.21.15.77:0/3061681871 shutdown_connections 2023-12-01T01:17:51.988 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.979+0000 7f029a0b0700 1 --2- 172.21.15.77:0/3061681871 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f027c0a05b0 0x7f027c0a2a70 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.988 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.979+0000 7f029a0b0700 1 --2- 172.21.15.77:0/3061681871 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0294106a90 0x7f029419dfc0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.988 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.979+0000 7f029a0b0700 1 --2- 172.21.15.77:0/3061681871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0294101830 0x7f0294196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.988 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.979+0000 7f029a0b0700 1 --2- 172.21.15.77:0/3061681871 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0294104160 0x7f0294196f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:17:51.988 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.979+0000 7f029a0b0700 1 -- 172.21.15.77:0/3061681871 >> 172.21.15.77:0/3061681871 conn(0x7f02940fb220 msgr2=0x7f02941024a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:17:51.988 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.979+0000 7f029a0b0700 1 -- 172.21.15.77:0/3061681871 shutdown_connections 2023-12-01T01:17:51.988 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:17:51.979+0000 7f029a0b0700 1 -- 172.21.15.77:0/3061681871 wait complete. 2023-12-01T01:17:51.993 INFO:tasks.cephfs.filesystem:scrub status for tag:7b4b436d-2a99-44ea-9dae-f1d1076178b4 - {'path': '/', 'tag': '7b4b436d-2a99-44ea-9dae-f1d1076178b4', 'options': 'recursive,force'} 2023-12-01T01:17:52.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:51 smithi203 ceph-mon[103581]: pgmap v297: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 10 op/s 2023-12-01T01:17:52.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:51 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/1237220698' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-01T01:17:52.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:51 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/4217245699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:17:52.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:51 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/688106184' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:17:52.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:51 smithi137 ceph-mon[103312]: pgmap v297: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 10 op/s 2023-12-01T01:17:52.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:51 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/1237220698' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-01T01:17:52.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:51 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/4217245699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:17:52.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:51 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/688106184' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:17:52.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:52 smithi077 ceph-mon[98867]: scrub queued for path: / 2023-12-01T01:17:52.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:52 smithi077 ceph-mon[98867]: scrub summary: idle+waiting paths [/] 2023-12-01T01:17:52.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:52 smithi077 ceph-mon[98867]: scrub summary: active paths [/] 2023-12-01T01:17:52.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:52 smithi077 ceph-mon[98867]: scrub summary: idle+waiting paths [/] 2023-12-01T01:17:53.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:52 smithi203 ceph-mon[103581]: scrub queued for path: / 2023-12-01T01:17:53.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:52 smithi203 ceph-mon[103581]: scrub summary: idle+waiting paths [/] 2023-12-01T01:17:53.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:52 smithi203 ceph-mon[103581]: scrub summary: active paths [/] 2023-12-01T01:17:53.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:52 smithi203 ceph-mon[103581]: scrub summary: idle+waiting paths [/] 2023-12-01T01:17:53.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:52 smithi137 ceph-mon[103312]: scrub queued for path: / 2023-12-01T01:17:53.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:52 smithi137 ceph-mon[103312]: scrub summary: idle+waiting paths [/] 2023-12-01T01:17:53.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:52 smithi137 ceph-mon[103312]: scrub summary: active paths [/] 2023-12-01T01:17:53.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:52 smithi137 ceph-mon[103312]: scrub summary: idle+waiting paths [/] 2023-12-01T01:17:53.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:53 smithi077 ceph-mon[98867]: pgmap v298: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 9 op/s 2023-12-01T01:17:54.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:53 smithi203 ceph-mon[103581]: pgmap v298: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 9 op/s 2023-12-01T01:17:54.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:53 smithi137 ceph-mon[103312]: pgmap v298: 97 pgs: 97 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 9 op/s 2023-12-01T01:17:55.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:55 smithi077 ceph-mon[98867]: pgmap v299: 97 pgs: 97 active+clean; 651 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2023-12-01T01:17:55.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:55 smithi077 ceph-mon[98867]: mgrmap e18: x(active, since 7m), standbys: y, z 2023-12-01T01:17:55.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:55 smithi077 ceph-mon[98867]: scrub summary: idle 2023-12-01T01:17:56.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:55 smithi203 ceph-mon[103581]: pgmap v299: 97 pgs: 97 active+clean; 651 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2023-12-01T01:17:56.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:55 smithi203 ceph-mon[103581]: mgrmap e18: x(active, since 7m), standbys: y, z 2023-12-01T01:17:56.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:55 smithi203 ceph-mon[103581]: scrub summary: idle 2023-12-01T01:17:56.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:55 smithi137 ceph-mon[103312]: pgmap v299: 97 pgs: 97 active+clean; 651 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2023-12-01T01:17:56.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:55 smithi137 ceph-mon[103312]: mgrmap e18: x(active, since 7m), standbys: y, z 2023-12-01T01:17:56.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:55 smithi137 ceph-mon[103312]: scrub summary: idle 2023-12-01T01:17:57.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:57 smithi077 ceph-mon[98867]: pgmap v300: 97 pgs: 97 active+clean; 651 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2023-12-01T01:17:58.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:57 smithi203 ceph-mon[103581]: pgmap v300: 97 pgs: 97 active+clean; 651 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2023-12-01T01:17:58.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:57 smithi137 ceph-mon[103312]: pgmap v300: 97 pgs: 97 active+clean; 651 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2023-12-01T01:17:59.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:17:58 smithi203 ceph-mon[103581]: pgmap v301: 97 pgs: 97 active+clean; 651 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2023-12-01T01:17:59.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:17:58 smithi137 ceph-mon[103312]: pgmap v301: 97 pgs: 97 active+clean; 651 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2023-12-01T01:17:59.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:17:58 smithi077 ceph-mon[98867]: pgmap v301: 97 pgs: 97 active+clean; 651 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2023-12-01T01:18:01.653 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:01 smithi077 ceph-mon[98867]: pgmap v302: 97 pgs: 97 active+clean; 651 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2023-12-01T01:18:01.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:01 smithi203 ceph-mon[103581]: pgmap v302: 97 pgs: 97 active+clean; 651 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2023-12-01T01:18:01.778 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:01 smithi137 ceph-mon[103312]: pgmap v302: 97 pgs: 97 active+clean; 651 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2023-12-01T01:18:03.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:03 smithi077 ceph-mon[98867]: pgmap v303: 97 pgs: 97 active+clean; 651 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2023-12-01T01:18:04.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:03 smithi203 ceph-mon[103581]: pgmap v303: 97 pgs: 97 active+clean; 651 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2023-12-01T01:18:04.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:03 smithi137 ceph-mon[103312]: pgmap v303: 97 pgs: 97 active+clean; 651 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2023-12-01T01:18:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.758+0000 7f7bb56f2700 1 -- 172.21.15.77:0/1933679940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7bb0100ed0 msgr2=0x7f7bb0101330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.758+0000 7f7bb56f2700 1 --2- 172.21.15.77:0/1933679940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7bb0100ed0 0x7f7bb0101330 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7ba40099c0 tx=0x7f7ba4009cd0 comp rx=0 tx=0).stop 2023-12-01T01:18:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.758+0000 7f7bb56f2700 1 -- 172.21.15.77:0/1933679940 shutdown_connections 2023-12-01T01:18:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.758+0000 7f7bb56f2700 1 --2- 172.21.15.77:0/1933679940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bb0101870 0x7f7bb0108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.758+0000 7f7bb56f2700 1 --2- 172.21.15.77:0/1933679940 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7bb00ffce0 0x7f7bb0100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.758+0000 7f7bb56f2700 1 --2- 172.21.15.77:0/1933679940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7bb0100ed0 0x7f7bb0101330 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.758+0000 7f7bb56f2700 1 -- 172.21.15.77:0/1933679940 >> 172.21.15.77:0/1933679940 conn(0x7f7bb00fb250 msgr2=0x7f7bb00fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.758+0000 7f7bb56f2700 1 -- 172.21.15.77:0/1933679940 shutdown_connections 2023-12-01T01:18:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.758+0000 7f7bb56f2700 1 -- 172.21.15.77:0/1933679940 wait complete. 2023-12-01T01:18:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.759+0000 7f7bb56f2700 1 Processor -- start 2023-12-01T01:18:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.759+0000 7f7bb56f2700 1 -- start start 2023-12-01T01:18:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.759+0000 7f7bb56f2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7bb00ffce0 0x7f7bb0195a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.759+0000 7f7bb56f2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7bb0100ed0 0x7f7bb0195fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.759+0000 7f7bb56f2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bb0101870 0x7f7bb0196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.759+0000 7f7bb56f2700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb01974b0 con 0x7f7bb00ffce0 2023-12-01T01:18:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.759+0000 7f7bb56f2700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb0197620 con 0x7f7bb0101870 2023-12-01T01:18:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.759+0000 7f7bb56f2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb01977e0 con 0x7f7bb0100ed0 2023-12-01T01:18:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.760+0000 7f7baeffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7bb00ffce0 0x7f7bb0195a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.760+0000 7f7baeffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7bb00ffce0 0x7f7bb0195a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59344/0 (socket says 172.21.15.77:59344) 2023-12-01T01:18:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.760+0000 7f7baeffd700 1 -- 172.21.15.77:0/3906179455 learned_addr learned my addr 172.21.15.77:0/3906179455 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.760+0000 7f7bae7fc700 1 --2- 172.21.15.77:0/3906179455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7bb0100ed0 0x7f7bb0195fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:04.764 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.760+0000 7f7baf7fe700 1 --2- 172.21.15.77:0/3906179455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bb0101870 0x7f7bb0196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:04.764 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.760+0000 7f7baeffd700 1 -- 172.21.15.77:0/3906179455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7bb0100ed0 msgr2=0x7f7bb0195fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:04.764 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.760+0000 7f7baeffd700 1 --2- 172.21.15.77:0/3906179455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7bb0100ed0 0x7f7bb0195fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:04.764 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.760+0000 7f7baeffd700 1 -- 172.21.15.77:0/3906179455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bb0101870 msgr2=0x7f7bb0196e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:04.764 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.760+0000 7f7baeffd700 1 --2- 172.21.15.77:0/3906179455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bb0101870 0x7f7bb0196e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:04.764 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.760+0000 7f7baeffd700 1 -- 172.21.15.77:0/3906179455 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba40095f0 con 0x7f7bb00ffce0 2023-12-01T01:18:04.764 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.760+0000 7f7baf7fe700 1 --2- 172.21.15.77:0/3906179455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bb0101870 0x7f7bb0196e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:18:04.764 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.760+0000 7f7baeffd700 1 --2- 172.21.15.77:0/3906179455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7bb00ffce0 0x7f7bb0195a90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7ba000d980 tx=0x7f7ba000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:04.764 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.760+0000 7f7b97fff700 1 -- 172.21.15.77:0/3906179455 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba0004020 con 0x7f7bb00ffce0 2023-12-01T01:18:04.764 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.760+0000 7f7bb56f2700 1 -- 172.21.15.77:0/3906179455 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bb0197ac0 con 0x7f7bb00ffce0 2023-12-01T01:18:04.764 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.760+0000 7f7b97fff700 1 -- 172.21.15.77:0/3906179455 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ba0009730 con 0x7f7bb00ffce0 2023-12-01T01:18:04.764 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.760+0000 7f7b97fff700 1 -- 172.21.15.77:0/3906179455 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba0003a80 con 0x7f7bb00ffce0 2023-12-01T01:18:04.764 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.760+0000 7f7bb56f2700 1 -- 172.21.15.77:0/3906179455 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb019e280 con 0x7f7bb00ffce0 2023-12-01T01:18:04.767 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.763+0000 7f7b97fff700 1 -- 172.21.15.77:0/3906179455 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f7ba00098a0 con 0x7f7bb00ffce0 2023-12-01T01:18:04.767 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.763+0000 7f7b97fff700 1 --2- 172.21.15.77:0/3906179455 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7b980a0520 0x7f7b980a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:04.767 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.763+0000 7f7bae7fc700 1 --2- 172.21.15.77:0/3906179455 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7b980a0520 0x7f7b980a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:04.767 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.763+0000 7f7b97fff700 1 -- 172.21.15.77:0/3906179455 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f7ba00cb2b0 con 0x7f7bb00ffce0 2023-12-01T01:18:04.768 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.764+0000 7f7bae7fc700 1 --2- 172.21.15.77:0/3906179455 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7b980a0520 0x7f7b980a29e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7ba400b580 tx=0x7f7ba4005bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:04.768 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.764+0000 7f7bc1882b80 1 Processor -- start 2023-12-01T01:18:04.768 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.764+0000 7f7bc1882b80 1 -- start start 2023-12-01T01:18:04.768 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.764+0000 7f7bc1882b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc24641890 0x55bc24641cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:04.768 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.764+0000 7f7bc1882b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc246421f0 0x55bc24642b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:04.768 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.764+0000 7f7bc1882b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc246430e0 0x55bc24643560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:04.768 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.764+0000 7f7bc1882b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc245979b0 con 0x55bc24641890 2023-12-01T01:18:04.768 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.764+0000 7f7bc1882b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc24595370 con 0x55bc246421f0 2023-12-01T01:18:04.768 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.764+0000 7f7bc1882b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc24590c60 con 0x55bc246430e0 2023-12-01T01:18:04.768 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.765+0000 7f7baeffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc24641890 0x55bc24641cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:04.769 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.765+0000 7f7baeffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc24641890 0x55bc24641cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59352/0 (socket says 172.21.15.77:59352) 2023-12-01T01:18:04.769 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.765+0000 7f7baeffd700 1 -- 172.21.15.77:0/2970259776 learned_addr learned my addr 172.21.15.77:0/2970259776 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:04.769 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.765+0000 7f7bae7fc700 1 --2- 172.21.15.77:0/2970259776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc246421f0 0x55bc24642b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:04.769 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.765+0000 7f7baf7fe700 1 --2- 172.21.15.77:0/2970259776 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc246430e0 0x55bc24643560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:04.769 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.765+0000 7f7baeffd700 1 -- 172.21.15.77:0/2970259776 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc246430e0 msgr2=0x55bc24643560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:04.769 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.765+0000 7f7baeffd700 1 --2- 172.21.15.77:0/2970259776 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc246430e0 0x55bc24643560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:04.769 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.765+0000 7f7baeffd700 1 -- 172.21.15.77:0/2970259776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc246421f0 msgr2=0x55bc24642b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:04.769 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.765+0000 7f7baeffd700 1 --2- 172.21.15.77:0/2970259776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc246421f0 0x55bc24642b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:04.769 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.765+0000 7f7baeffd700 1 -- 172.21.15.77:0/2970259776 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc243bed80 con 0x55bc24641890 2023-12-01T01:18:04.769 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.765+0000 7f7baf7fe700 1 --2- 172.21.15.77:0/2970259776 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc246430e0 0x55bc24643560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:18:04.769 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.765+0000 7f7baeffd700 1 --2- 172.21.15.77:0/2970259776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc24641890 0x55bc24641cb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7ba00a06f0 tx=0x7f7ba0098590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7424c0d1614cb070 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:04.769 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.765+0000 7f7b957fa700 1 -- 172.21.15.77:0/2970259776 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba0098bb0 con 0x55bc24641890 2023-12-01T01:18:04.769 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.765+0000 7f7b957fa700 1 -- 172.21.15.77:0/2970259776 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ba009c9e0 con 0x55bc24641890 2023-12-01T01:18:04.769 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.765+0000 7f7b957fa700 1 -- 172.21.15.77:0/2970259776 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba0098e10 con 0x55bc24641890 2023-12-01T01:18:04.770 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.766+0000 7f7bc1882b80 1 -- 172.21.15.77:0/2970259776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc24641890 msgr2=0x55bc24641cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:04.770 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.766+0000 7f7bc1882b80 1 --2- 172.21.15.77:0/2970259776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc24641890 0x55bc24641cb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7ba00a06f0 tx=0x7f7ba0098590 comp rx=0 tx=0).stop 2023-12-01T01:18:04.770 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.766+0000 7f7bc1882b80 1 -- 172.21.15.77:0/2970259776 shutdown_connections 2023-12-01T01:18:04.770 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.766+0000 7f7bc1882b80 1 --2- 172.21.15.77:0/2970259776 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc246430e0 0x55bc24643560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:04.770 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.766+0000 7f7bc1882b80 1 --2- 172.21.15.77:0/2970259776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc24641890 0x55bc24641cb0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:04.770 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.766+0000 7f7bc1882b80 1 --2- 172.21.15.77:0/2970259776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc246421f0 0x55bc24642b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:04.770 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.766+0000 7f7bc1882b80 1 -- 172.21.15.77:0/2970259776 >> 172.21.15.77:0/2970259776 conn(0x55bc2459f070 msgr2=0x55bc2463c040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:04.770 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.766+0000 7f7bc1882b80 1 -- 172.21.15.77:0/2970259776 shutdown_connections 2023-12-01T01:18:04.770 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.766+0000 7f7bc1882b80 1 -- 172.21.15.77:0/2970259776 wait complete. 2023-12-01T01:18:04.770 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.766+0000 7f7bc1882b80 1 -- start start 2023-12-01T01:18:04.770 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.766+0000 7f7bc1882b80 1 Processor -- start 2023-12-01T01:18:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.767+0000 7f7bc1882b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc24641890 0x55bc2464b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.767+0000 7f7bc1882b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc246421f0 0x55bc2464bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.767+0000 7f7bc1882b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc246430e0 0x55bc24652bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.767+0000 7f7bc1882b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc2452d4b0 con 0x55bc246421f0 2023-12-01T01:18:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.767+0000 7f7bc1882b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc2452d150 con 0x55bc246430e0 2023-12-01T01:18:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.767+0000 7f7bc1882b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc244148c0 con 0x55bc24641890 2023-12-01T01:18:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.767+0000 7f7bc1882b80 10 client.0 _ll_register_callbacks cb 0x55bc2447aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:18:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.767+0000 7f7bae7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc246421f0 0x55bc2464bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.767+0000 7f7bae7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc246421f0 0x55bc2464bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59360/0 (socket says 172.21.15.77:59360) 2023-12-01T01:18:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.767+0000 7f7bae7fc700 1 -- 172.21.15.77:0/3250850638 learned_addr learned my addr 172.21.15.77:0/3250850638 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.767+0000 7f7baf7fe700 1 --2- 172.21.15.77:0/3250850638 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc246430e0 0x55bc24652bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.767+0000 7f7baeffd700 1 --2- 172.21.15.77:0/3250850638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc24641890 0x55bc2464b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.767+0000 7f7bae7fc700 1 -- 172.21.15.77:0/3250850638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc24641890 msgr2=0x55bc2464b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.767+0000 7f7bae7fc700 1 --2- 172.21.15.77:0/3250850638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc24641890 0x55bc2464b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.767+0000 7f7bae7fc700 1 -- 172.21.15.77:0/3250850638 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc246430e0 msgr2=0x55bc24652bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.767+0000 7f7bae7fc700 1 --2- 172.21.15.77:0/3250850638 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc246430e0 0x55bc24652bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.767+0000 7f7bae7fc700 1 -- 172.21.15.77:0/3250850638 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb019e280 con 0x55bc246421f0 2023-12-01T01:18:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.767+0000 7f7baeffd700 1 --2- 172.21.15.77:0/3250850638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc24641890 0x55bc2464b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:18:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.767+0000 7f7bae7fc700 1 --2- 172.21.15.77:0/3250850638 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc246421f0 0x55bc2464bb90 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4015000 tx=0x7f7ba4009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=103d14a6f339bf95 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.767+0000 7f7b92ff5700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:18:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.767+0000 7f7b92ff5700 1 -- 172.21.15.77:0/3250850638 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba40056b0 con 0x55bc246421f0 2023-12-01T01:18:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.767+0000 7f7b92ff5700 1 -- 172.21.15.77:0/3250850638 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f7ba4004500 con 0x55bc246421f0 2023-12-01T01:18:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.767+0000 7f7bc1882b80 1 -- 172.21.15.77:0/3250850638 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bc245bbcb0 con 0x55bc246421f0 2023-12-01T01:18:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.768+0000 7f7b92ff5700 1 -- 172.21.15.77:0/3250850638 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ba40046c0 con 0x55bc246421f0 2023-12-01T01:18:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.768+0000 7f7bc1882b80 10 client.15498 fetch_fsmap learned FSMap version 27 2023-12-01T01:18:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.768+0000 7f7bc1882b80 1 -- 172.21.15.77:0/3250850638 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55bc243a7c00 con 0x55bc246421f0 2023-12-01T01:18:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.768+0000 7f7b92ff5700 1 -- 172.21.15.77:0/3250850638 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ba403f020 con 0x55bc246421f0 2023-12-01T01:18:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.768+0000 7f7b92ff5700 1 -- 172.21.15.77:0/3250850638 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f7ba402f070 con 0x55bc246421f0 2023-12-01T01:18:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.768+0000 7f7bc1882b80 10 client.15498 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:18:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.768+0000 7f7bc1882b80 10 client.15498 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-01T01:18:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.768+0000 7f7bc1882b80 20 client.15498 populate_metadata read hostname 'smithi077' 2023-12-01T01:18:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.768+0000 7f7bc1882b80 1 --2- 172.21.15.77:0/3250850638 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55bc246573f0 0x55bc24659810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.768+0000 7f7bc1882b80 4 client.15498 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:18:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.768+0000 7f7bc1882b80 1 -- 172.21.15.77:0/3250850638 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc2460edb0 con 0x55bc246573f0 2023-12-01T01:18:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.768+0000 7f7baeffd700 1 --2- 172.21.15.77:0/3250850638 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55bc246573f0 0x55bc24659810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.769+0000 7f7b92ff5700 10 client.15498 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.769+0000 7f7baeffd700 1 --2- 172.21.15.77:0/3250850638 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55bc246573f0 0x55bc24659810 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4041daee4648b8d server_cookie=956429edf06b4d94 in_seq=0 out_seq=0 2023-12-01T01:18:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.769+0000 7f7b92ff5700 1 -- 172.21.15.77:0/3250850638 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55bc2460edb0 con 0x55bc246573f0 2023-12-01T01:18:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.769+0000 7f7b92ff5700 10 client.15498 handle_command_reply: tid=0 2023-12-01T01:18:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.769+0000 7f7bc1882b80 1 client.15498 shutdown 2023-12-01T01:18:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.770+0000 7f7bc1882b80 1 -- 172.21.15.77:0/3250850638 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc246421f0 msgr2=0x55bc2464bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.770+0000 7f7bc1882b80 1 --2- 172.21.15.77:0/3250850638 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc246421f0 0x55bc2464bb90 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4015000 tx=0x7f7ba4009cd0 comp rx=0 tx=0).stop 2023-12-01T01:18:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.770+0000 7f7bc1882b80 1 -- 172.21.15.77:0/3250850638 shutdown_connections 2023-12-01T01:18:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.770+0000 7f7bc1882b80 1 --2- 172.21.15.77:0/3250850638 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55bc246573f0 0x55bc24659810 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.770+0000 7f7bc1882b80 1 --2- 172.21.15.77:0/3250850638 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc246430e0 0x55bc24652bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.770+0000 7f7bc1882b80 1 --2- 172.21.15.77:0/3250850638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc24641890 0x55bc2464b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.770+0000 7f7b92ff5700 0 client.15498 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.770+0000 7f7bc1882b80 1 --2- 172.21.15.77:0/3250850638 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc246421f0 0x55bc2464bb90 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.770+0000 7f7bc1882b80 1 -- 172.21.15.77:0/3250850638 >> 172.21.15.77:0/3250850638 conn(0x55bc2459f070 msgr2=0x55bc2463a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.770+0000 7f7bc1882b80 1 -- 172.21.15.77:0/3250850638 shutdown_connections 2023-12-01T01:18:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.770+0000 7f7bc1882b80 1 -- 172.21.15.77:0/3250850638 wait complete. 2023-12-01T01:18:04.777 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.770+0000 7f7bc1882b80 20 client.15498 trim_cache size 0 max 16384 2023-12-01T01:18:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.786+0000 7f7bc1882b80 1 Processor -- start 2023-12-01T01:18:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.786+0000 7f7bc1882b80 1 -- start start 2023-12-01T01:18:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.786+0000 7f7bc1882b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc24641890 0x55bc246434c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.786+0000 7f7bc1882b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc246421f0 0x55bc2464a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.786+0000 7f7bc1882b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc246430e0 0x55bc246462b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.786+0000 7f7bc1882b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc24652fb0 con 0x55bc246421f0 2023-12-01T01:18:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.786+0000 7f7bc1882b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc24632ef0 con 0x55bc24641890 2023-12-01T01:18:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.786+0000 7f7bc1882b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc24642670 con 0x55bc246430e0 2023-12-01T01:18:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.786+0000 7f7bae7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc246421f0 0x55bc2464a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.786+0000 7f7bae7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc246421f0 0x55bc2464a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59368/0 (socket says 172.21.15.77:59368) 2023-12-01T01:18:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.786+0000 7f7bae7fc700 1 -- 172.21.15.77:0/3054281785 learned_addr learned my addr 172.21.15.77:0/3054281785 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.786+0000 7f7baeffd700 1 --2- 172.21.15.77:0/3054281785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc24641890 0x55bc246434c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.786+0000 7f7baf7fe700 1 --2- 172.21.15.77:0/3054281785 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc246430e0 0x55bc246462b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.786+0000 7f7bae7fc700 1 -- 172.21.15.77:0/3054281785 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc246430e0 msgr2=0x55bc246462b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.786+0000 7f7bae7fc700 1 --2- 172.21.15.77:0/3054281785 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc246430e0 0x55bc246462b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.786+0000 7f7bae7fc700 1 -- 172.21.15.77:0/3054281785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc24641890 msgr2=0x55bc246434c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.786+0000 7f7bae7fc700 1 --2- 172.21.15.77:0/3054281785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc24641890 0x55bc246434c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.786+0000 7f7bae7fc700 1 -- 172.21.15.77:0/3054281785 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc243e51f0 con 0x55bc246421f0 2023-12-01T01:18:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.787+0000 7f7bae7fc700 1 --2- 172.21.15.77:0/3054281785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc246421f0 0x55bc2464a340 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7ba400f950 tx=0x7f7ba4005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ca00fac0e6cba9 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.787+0000 7f7b95ffb700 1 -- 172.21.15.77:0/3054281785 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba400fd70 con 0x55bc246421f0 2023-12-01T01:18:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.787+0000 7f7b95ffb700 1 -- 172.21.15.77:0/3054281785 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ba403f070 con 0x55bc246421f0 2023-12-01T01:18:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.787+0000 7f7b95ffb700 1 -- 172.21.15.77:0/3054281785 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba4004cf0 con 0x55bc246421f0 2023-12-01T01:18:04.793 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.787+0000 7f7bc1882b80 1 -- 172.21.15.77:0/3054281785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc246421f0 msgr2=0x55bc2464a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:04.793 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.787+0000 7f7bc1882b80 1 --2- 172.21.15.77:0/3054281785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc246421f0 0x55bc2464a340 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7ba400f950 tx=0x7f7ba4005f00 comp rx=0 tx=0).stop 2023-12-01T01:18:04.793 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.788+0000 7f7bc1882b80 1 -- 172.21.15.77:0/3054281785 shutdown_connections 2023-12-01T01:18:04.793 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.788+0000 7f7bc1882b80 1 --2- 172.21.15.77:0/3054281785 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc246430e0 0x55bc246462b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:04.793 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.788+0000 7f7bc1882b80 1 --2- 172.21.15.77:0/3054281785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc24641890 0x55bc246434c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:04.793 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.788+0000 7f7bc1882b80 1 --2- 172.21.15.77:0/3054281785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc246421f0 0x55bc2464a340 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:04.793 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.788+0000 7f7bc1882b80 1 -- 172.21.15.77:0/3054281785 >> 172.21.15.77:0/3054281785 conn(0x55bc2459f070 msgr2=0x55bc2463bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:04.793 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.788+0000 7f7bc1882b80 1 -- 172.21.15.77:0/3054281785 shutdown_connections 2023-12-01T01:18:04.793 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.788+0000 7f7bc1882b80 1 -- 172.21.15.77:0/3054281785 wait complete. 2023-12-01T01:18:04.793 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.788+0000 7f7bc1882b80 1 -- start start 2023-12-01T01:18:04.793 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.788+0000 7f7bc1882b80 1 Processor -- start 2023-12-01T01:18:04.793 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.788+0000 7f7bc1882b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc24641890 0x55bc246a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:04.794 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.788+0000 7f7bc1882b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc246421f0 0x55bc246a7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:04.794 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.788+0000 7f7bc1882b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc246430e0 0x55bc246a9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:04.794 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.788+0000 7f7bc1882b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc246aa3d0 con 0x55bc246430e0 2023-12-01T01:18:04.794 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.788+0000 7f7bc1882b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc246aa540 con 0x55bc24641890 2023-12-01T01:18:04.794 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.788+0000 7f7bc1882b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc246aa700 con 0x55bc246421f0 2023-12-01T01:18:04.794 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.788+0000 7f7bc1882b80 10 client.0 _ll_register_callbacks cb 0x55bc2447aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:18:04.794 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.788+0000 7f7baf7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc246430e0 0x55bc246a9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:04.794 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.788+0000 7f7baf7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc246430e0 0x55bc246a9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59376/0 (socket says 172.21.15.77:59376) 2023-12-01T01:18:04.794 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.788+0000 7f7baf7fe700 1 -- 172.21.15.77:0/2495552821 learned_addr learned my addr 172.21.15.77:0/2495552821 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:04.794 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.788+0000 7f7bae7fc700 1 --2- 172.21.15.77:0/2495552821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc246421f0 0x55bc246a7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:04.794 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.789+0000 7f7baeffd700 1 --2- 172.21.15.77:0/2495552821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc24641890 0x55bc246a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:04.794 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.789+0000 7f7baf7fe700 1 -- 172.21.15.77:0/2495552821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc246421f0 msgr2=0x55bc246a7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:04.794 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.789+0000 7f7baf7fe700 1 --2- 172.21.15.77:0/2495552821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc246421f0 0x55bc246a7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.789+0000 7f7baf7fe700 1 -- 172.21.15.77:0/2495552821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc24641890 msgr2=0x55bc246a50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.789+0000 7f7baf7fe700 1 --2- 172.21.15.77:0/2495552821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc24641890 0x55bc246a50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.789+0000 7f7baf7fe700 1 -- 172.21.15.77:0/2495552821 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc243a7c00 con 0x55bc246430e0 2023-12-01T01:18:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.789+0000 7f7bae7fc700 1 --2- 172.21.15.77:0/2495552821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc246421f0 0x55bc246a7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:18:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.789+0000 7f7baf7fe700 1 --2- 172.21.15.77:0/2495552821 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc246430e0 0x55bc246a9c90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8007a00 tx=0x7f7ba8007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a34a2b122794f59 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.789+0000 7f7b92ff5700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:18:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.789+0000 7f7b92ff5700 1 -- 172.21.15.77:0/2495552821 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8007e30 con 0x55bc246430e0 2023-12-01T01:18:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.789+0000 7f7b92ff5700 1 -- 172.21.15.77:0/2495552821 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f7ba80149c0 con 0x55bc246430e0 2023-12-01T01:18:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.789+0000 7f7bc1882b80 1 -- 172.21.15.77:0/2495552821 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7ba009c9e0 con 0x55bc246430e0 2023-12-01T01:18:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.789+0000 7f7b92ff5700 1 -- 172.21.15.77:0/2495552821 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ba8014b80 con 0x55bc246430e0 2023-12-01T01:18:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.789+0000 7f7bc1882b80 10 client.15504 fetch_fsmap learned FSMap version 27 2023-12-01T01:18:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.789+0000 7f7bc1882b80 1 -- 172.21.15.77:0/2495552821 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55bc243bd250 con 0x55bc246430e0 2023-12-01T01:18:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.789+0000 7f7b92ff5700 1 -- 172.21.15.77:0/2495552821 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ba009c9e0 con 0x55bc246430e0 2023-12-01T01:18:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.790+0000 7f7b92ff5700 1 -- 172.21.15.77:0/2495552821 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f7ba8020410 con 0x55bc246430e0 2023-12-01T01:18:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.790+0000 7f7bc1882b80 10 client.15504 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:18:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.790+0000 7f7bc1882b80 10 client.15504 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-01T01:18:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.790+0000 7f7bc1882b80 20 client.15504 populate_metadata read hostname 'smithi077' 2023-12-01T01:18:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.790+0000 7f7bc1882b80 1 --2- 172.21.15.77:0/2495552821 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55bc246ac1f0 0x55bc246ae610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.790+0000 7f7bc1882b80 4 client.15504 mds_command: new command op to 24500 tid=0[{"prefix": "flush journal"}] 2023-12-01T01:18:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.790+0000 7f7bc1882b80 1 -- 172.21.15.77:0/2495552821 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bc24589ce0 con 0x55bc246ac1f0 2023-12-01T01:18:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.790+0000 7f7baeffd700 1 --2- 172.21.15.77:0/2495552821 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55bc246ac1f0 0x55bc246ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.790+0000 7f7b92ff5700 10 client.15504 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.790+0000 7f7baeffd700 1 --2- 172.21.15.77:0/2495552821 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55bc246ac1f0 0x55bc246ae610 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bbb4ab4bb60508c server_cookie=6a99ce1804ed9087 in_seq=0 out_seq=0 2023-12-01T01:18:04.803 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.801+0000 7f7b92ff5700 1 -- 172.21.15.77:0/2495552821 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bc24589ce0 con 0x55bc246ac1f0 2023-12-01T01:18:04.803 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.802+0000 7f7b92ff5700 10 client.15504 handle_command_reply: tid=0 2023-12-01T01:18:04.803 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.802+0000 7f7bc1882b80 1 client.15504 shutdown 2023-12-01T01:18:04.803 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.802+0000 7f7bc1882b80 1 -- 172.21.15.77:0/2495552821 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc246430e0 msgr2=0x55bc246a9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:04.803 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.802+0000 7f7bc1882b80 1 --2- 172.21.15.77:0/2495552821 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc246430e0 0x55bc246a9c90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8007a00 tx=0x7f7ba8007d10 comp rx=0 tx=0).stop 2023-12-01T01:18:04.803 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.802+0000 7f7bc1882b80 1 -- 172.21.15.77:0/2495552821 shutdown_connections 2023-12-01T01:18:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.802+0000 7f7bc1882b80 1 --2- 172.21.15.77:0/2495552821 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc246430e0 0x55bc246a9c90 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.802+0000 7f7bc1882b80 1 --2- 172.21.15.77:0/2495552821 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55bc246ac1f0 0x55bc246ae610 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.802+0000 7f7bc1882b80 1 --2- 172.21.15.77:0/2495552821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bc24641890 0x55bc246a50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.802+0000 7f7bc1882b80 1 --2- 172.21.15.77:0/2495552821 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bc246421f0 0x55bc246a7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.802+0000 7f7b92ff5700 0 client.15504 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.802+0000 7f7bc1882b80 1 -- 172.21.15.77:0/2495552821 >> 172.21.15.77:0/2495552821 conn(0x55bc2459f070 msgr2=0x55bc2464a340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.802+0000 7f7bc1882b80 1 -- 172.21.15.77:0/2495552821 shutdown_connections 2023-12-01T01:18:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.802+0000 7f7bc1882b80 1 -- 172.21.15.77:0/2495552821 wait complete. 2023-12-01T01:18:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.802+0000 7f7bc1882b80 20 client.15504 trim_cache size 0 max 16384 2023-12-01T01:18:04.805 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout:{ 2023-12-01T01:18:04.805 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout: "message": "", 2023-12-01T01:18:04.805 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout: "return_code": 0 2023-12-01T01:18:04.805 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout:} 2023-12-01T01:18:04.805 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.802+0000 7f7bb56f2700 1 -- 172.21.15.77:0/3906179455 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7b980a0520 msgr2=0x7f7b980a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:04.805 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.802+0000 7f7bb56f2700 1 --2- 172.21.15.77:0/3906179455 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7b980a0520 0x7f7b980a29e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7ba400b580 tx=0x7f7ba4005bc0 comp rx=0 tx=0).stop 2023-12-01T01:18:04.806 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.802+0000 7f7bb56f2700 1 -- 172.21.15.77:0/3906179455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7bb00ffce0 msgr2=0x7f7bb0195a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:04.806 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.802+0000 7f7bb56f2700 1 --2- 172.21.15.77:0/3906179455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7bb00ffce0 0x7f7bb0195a90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7ba000d980 tx=0x7f7ba000dc90 comp rx=0 tx=0).stop 2023-12-01T01:18:04.806 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.803+0000 7f7bb56f2700 1 -- 172.21.15.77:0/3906179455 shutdown_connections 2023-12-01T01:18:04.806 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.803+0000 7f7bb56f2700 1 --2- 172.21.15.77:0/3906179455 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7b980a0520 0x7f7b980a29e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:04.806 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.803+0000 7f7bb56f2700 1 --2- 172.21.15.77:0/3906179455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bb0101870 0x7f7bb0196e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:04.806 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.803+0000 7f7bb56f2700 1 --2- 172.21.15.77:0/3906179455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7bb00ffce0 0x7f7bb0195a90 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:04.806 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.803+0000 7f7bb56f2700 1 --2- 172.21.15.77:0/3906179455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7bb0100ed0 0x7f7bb0195fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:04.806 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.803+0000 7f7bb56f2700 1 -- 172.21.15.77:0/3906179455 >> 172.21.15.77:0/3906179455 conn(0x7f7bb00fb250 msgr2=0x7f7bb01043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:04.806 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.803+0000 7f7bb56f2700 1 -- 172.21.15.77:0/3906179455 shutdown_connections 2023-12-01T01:18:04.806 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:04.803+0000 7f7bb56f2700 1 -- 172.21.15.77:0/3906179455 wait complete. 2023-12-01T01:18:05.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:05 smithi077 ceph-mon[98867]: pgmap v304: 97 pgs: 97 active+clean; 651 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2023-12-01T01:18:06.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:05 smithi203 ceph-mon[103581]: pgmap v304: 97 pgs: 97 active+clean; 651 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2023-12-01T01:18:06.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:05 smithi137 ceph-mon[103312]: pgmap v304: 97 pgs: 97 active+clean; 651 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2023-12-01T01:18:07.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:07 smithi077 ceph-mon[98867]: pgmap v305: 97 pgs: 97 active+clean; 651 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1023 B/s wr, 11 op/s 2023-12-01T01:18:08.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:07 smithi203 ceph-mon[103581]: pgmap v305: 97 pgs: 97 active+clean; 651 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1023 B/s wr, 11 op/s 2023-12-01T01:18:08.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:07 smithi137 ceph-mon[103312]: pgmap v305: 97 pgs: 97 active+clean; 651 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1023 B/s wr, 11 op/s 2023-12-01T01:18:09.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:09 smithi077 ceph-mon[98867]: pgmap v306: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2023-12-01T01:18:10.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:09 smithi203 ceph-mon[103581]: pgmap v306: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2023-12-01T01:18:10.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:09 smithi137 ceph-mon[103312]: pgmap v306: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2023-12-01T01:18:11.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:11 smithi077 ceph-mon[98867]: pgmap v307: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2023-12-01T01:18:12.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:11 smithi203 ceph-mon[103581]: pgmap v307: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2023-12-01T01:18:12.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:11 smithi137 ceph-mon[103312]: pgmap v307: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2023-12-01T01:18:13.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:13 smithi077 ceph-mon[98867]: pgmap v308: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2023-12-01T01:18:14.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:13 smithi203 ceph-mon[103581]: pgmap v308: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2023-12-01T01:18:14.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:13 smithi137 ceph-mon[103312]: pgmap v308: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2023-12-01T01:18:15.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:15 smithi077 ceph-mon[98867]: pgmap v309: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.4 KiB/s wr, 15 op/s 2023-12-01T01:18:16.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:15 smithi203 ceph-mon[103581]: pgmap v309: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.4 KiB/s wr, 15 op/s 2023-12-01T01:18:16.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:15 smithi137 ceph-mon[103312]: pgmap v309: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.4 KiB/s wr, 15 op/s 2023-12-01T01:18:17.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:17 smithi077 ceph-mon[98867]: pgmap v310: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2023-12-01T01:18:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.966+0000 7f1876883700 1 -- 172.21.15.77:0/569028876 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f18700ffe90 msgr2=0x7f1870100310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.966+0000 7f1876883700 1 --2- 172.21.15.77:0/569028876 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f18700ffe90 0x7f1870100310 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f1864009a10 tx=0x7f1864009d20 comp rx=0 tx=0).stop 2023-12-01T01:18:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.966+0000 7f1876883700 1 -- 172.21.15.77:0/569028876 shutdown_connections 2023-12-01T01:18:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.966+0000 7f1876883700 1 --2- 172.21.15.77:0/569028876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1870100850 0x7f18701080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.966+0000 7f1876883700 1 --2- 172.21.15.77:0/569028876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1870101240 0x7f1870101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.966+0000 7f1876883700 1 --2- 172.21.15.77:0/569028876 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f18700ffe90 0x7f1870100310 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.966+0000 7f1876883700 1 -- 172.21.15.77:0/569028876 >> 172.21.15.77:0/569028876 conn(0x7f18700fa960 msgr2=0x7f18700fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.966+0000 7f1876883700 1 -- 172.21.15.77:0/569028876 shutdown_connections 2023-12-01T01:18:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.966+0000 7f1876883700 1 -- 172.21.15.77:0/569028876 wait complete. 2023-12-01T01:18:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.967+0000 7f1876883700 1 Processor -- start 2023-12-01T01:18:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.967+0000 7f1876883700 1 -- start start 2023-12-01T01:18:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.967+0000 7f1876883700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18700ffe90 0x7f1870198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.967+0000 7f1876883700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1870100850 0x7f1870199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.967+0000 7f1876883700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1870101240 0x7f187019a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.967+0000 7f1876883700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187019a830 con 0x7f1870101240 2023-12-01T01:18:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.967+0000 7f1876883700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187019a9a0 con 0x7f1870100850 2023-12-01T01:18:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.967+0000 7f1876883700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187019ab60 con 0x7f18700ffe90 2023-12-01T01:18:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.967+0000 7f186ffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18700ffe90 0x7f1870198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.967+0000 7f186ffff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18700ffe90 0x7f1870198e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:45574/0 (socket says 172.21.15.77:45574) 2023-12-01T01:18:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.967+0000 7f186ffff700 1 -- 172.21.15.77:0/929113724 learned_addr learned my addr 172.21.15.77:0/929113724 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:17.972 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.967+0000 7f1874e20700 1 --2- 172.21.15.77:0/929113724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1870101240 0x7f187019a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.968+0000 7f186f7fe700 1 --2- 172.21.15.77:0/929113724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1870100850 0x7f1870199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.968+0000 7f186ffff700 1 -- 172.21.15.77:0/929113724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1870100850 msgr2=0x7f1870199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.968+0000 7f186ffff700 1 --2- 172.21.15.77:0/929113724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1870100850 0x7f1870199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.968+0000 7f186ffff700 1 -- 172.21.15.77:0/929113724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1870101240 msgr2=0x7f187019a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.968+0000 7f186ffff700 1 --2- 172.21.15.77:0/929113724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1870101240 0x7f187019a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.968+0000 7f186ffff700 1 -- 172.21.15.77:0/929113724 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18640095f0 con 0x7f18700ffe90 2023-12-01T01:18:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.968+0000 7f1874e20700 1 --2- 172.21.15.77:0/929113724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1870101240 0x7f187019a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:18:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.968+0000 7f186ffff700 1 --2- 172.21.15.77:0/929113724 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18700ffe90 0x7f1870198e40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f186000d950 tx=0x7f186000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.968+0000 7f186d7fa700 1 -- 172.21.15.77:0/929113724 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186000f890 con 0x7f18700ffe90 2023-12-01T01:18:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.968+0000 7f186d7fa700 1 -- 172.21.15.77:0/929113724 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1860010460 con 0x7f18700ffe90 2023-12-01T01:18:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.968+0000 7f1876883700 1 -- 172.21.15.77:0/929113724 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f187019ae40 con 0x7f18700ffe90 2023-12-01T01:18:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.968+0000 7f186d7fa700 1 -- 172.21.15.77:0/929113724 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186000e610 con 0x7f18700ffe90 2023-12-01T01:18:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.968+0000 7f1876883700 1 -- 172.21.15.77:0/929113724 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18701a1620 con 0x7f18700ffe90 2023-12-01T01:18:17.974 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.971+0000 7f186d7fa700 1 -- 172.21.15.77:0/929113724 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f18600108d0 con 0x7f18700ffe90 2023-12-01T01:18:17.974 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.972+0000 7f186d7fa700 1 --2- 172.21.15.77:0/929113724 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f18580a0520 0x7f18580a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:17.974 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.972+0000 7f186d7fa700 1 -- 172.21.15.77:0/929113724 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f18600cb6a0 con 0x7f18700ffe90 2023-12-01T01:18:17.974 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.973+0000 7f1882a13b80 1 Processor -- start 2023-12-01T01:18:17.974 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.973+0000 7f186f7fe700 1 --2- 172.21.15.77:0/929113724 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f18580a0520 0x7f18580a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.973+0000 7f1882a13b80 1 -- start start 2023-12-01T01:18:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.973+0000 7f1882a13b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564868d2be90 0x564868d2c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.973+0000 7f1882a13b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564868d2c7f0 0x564868d2d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.973+0000 7f1882a13b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564868d2d6e0 0x564868d2db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.973+0000 7f1882a13b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x564868c7b9b0 con 0x564868d2be90 2023-12-01T01:18:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.974+0000 7f1882a13b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564868c79370 con 0x564868d2c7f0 2023-12-01T01:18:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.974+0000 7f1882a13b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x564868c74c60 con 0x564868d2d6e0 2023-12-01T01:18:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.974+0000 7f186ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564868d2be90 0x564868d2c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.974+0000 7f186ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564868d2be90 0x564868d2c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59394/0 (socket says 172.21.15.77:59394) 2023-12-01T01:18:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.974+0000 7f186ffff700 1 -- 172.21.15.77:0/3482014613 learned_addr learned my addr 172.21.15.77:0/3482014613 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.974+0000 7f1874e20700 1 --2- 172.21.15.77:0/3482014613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564868d2d6e0 0x564868d2db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.974+0000 7f186f7fe700 1 --2- 172.21.15.77:0/3482014613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564868d2c7f0 0x564868d2d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.974+0000 7f186f7fe700 1 --2- 172.21.15.77:0/929113724 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f18580a0520 0x7f18580a29e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1864000c00 tx=0x7f1864005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.974+0000 7f186ffff700 1 -- 172.21.15.77:0/3482014613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564868d2d6e0 msgr2=0x564868d2db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.974+0000 7f186ffff700 1 --2- 172.21.15.77:0/3482014613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564868d2d6e0 0x564868d2db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.974+0000 7f186ffff700 1 -- 172.21.15.77:0/3482014613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564868d2c7f0 msgr2=0x564868d2d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.974+0000 7f186ffff700 1 --2- 172.21.15.77:0/3482014613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564868d2c7f0 0x564868d2d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.974+0000 7f186ffff700 1 -- 172.21.15.77:0/3482014613 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564868aa2d80 con 0x564868d2be90 2023-12-01T01:18:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.974+0000 7f1874e20700 1 --2- 172.21.15.77:0/3482014613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564868d2d6e0 0x564868d2db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:18:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.974+0000 7f186ffff700 1 --2- 172.21.15.77:0/3482014613 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564868d2be90 0x564868d2c2b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f18600a0ae0 tx=0x7f18600996a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85177487f96b9b60 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.975+0000 7f18567fc700 1 -- 172.21.15.77:0/3482014613 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18600a1010 con 0x564868d2be90 2023-12-01T01:18:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.975+0000 7f18567fc700 1 -- 172.21.15.77:0/3482014613 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1860097340 con 0x564868d2be90 2023-12-01T01:18:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.975+0000 7f18567fc700 1 -- 172.21.15.77:0/3482014613 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18600a0b80 con 0x564868d2be90 2023-12-01T01:18:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.975+0000 7f1882a13b80 1 -- 172.21.15.77:0/3482014613 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564868d2be90 msgr2=0x564868d2c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.975+0000 7f1882a13b80 1 --2- 172.21.15.77:0/3482014613 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564868d2be90 0x564868d2c2b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f18600a0ae0 tx=0x7f18600996a0 comp rx=0 tx=0).stop 2023-12-01T01:18:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.976+0000 7f1882a13b80 1 -- 172.21.15.77:0/3482014613 shutdown_connections 2023-12-01T01:18:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.976+0000 7f1882a13b80 1 --2- 172.21.15.77:0/3482014613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564868d2d6e0 0x564868d2db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.976+0000 7f1882a13b80 1 --2- 172.21.15.77:0/3482014613 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564868d2be90 0x564868d2c2b0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.976+0000 7f1882a13b80 1 --2- 172.21.15.77:0/3482014613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564868d2c7f0 0x564868d2d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.976+0000 7f1882a13b80 1 -- 172.21.15.77:0/3482014613 >> 172.21.15.77:0/3482014613 conn(0x564868c83070 msgr2=0x564868d20040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.976+0000 7f1882a13b80 1 -- 172.21.15.77:0/3482014613 shutdown_connections 2023-12-01T01:18:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.976+0000 7f1882a13b80 1 -- 172.21.15.77:0/3482014613 wait complete. 2023-12-01T01:18:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.976+0000 7f1882a13b80 1 -- start start 2023-12-01T01:18:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.976+0000 7f1882a13b80 1 Processor -- start 2023-12-01T01:18:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.977+0000 7f1882a13b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564868d2be90 0x564868d24bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.977+0000 7f1882a13b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564868d2c7f0 0x564868d250f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.977+0000 7f1882a13b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564868d2d6e0 0x564868d2a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.977+0000 7f1882a13b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x564868c114b0 con 0x564868d2d6e0 2023-12-01T01:18:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.977+0000 7f1882a13b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564868c11150 con 0x564868d2be90 2023-12-01T01:18:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.977+0000 7f1882a13b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x564868af88c0 con 0x564868d2c7f0 2023-12-01T01:18:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.977+0000 7f1882a13b80 10 client.0 _ll_register_callbacks cb 0x564868b5ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:18:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.977+0000 7f186ffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564868d2be90 0x564868d24bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.977+0000 7f186ffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564868d2be90 0x564868d24bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:38322/0 (socket says 172.21.15.77:38322) 2023-12-01T01:18:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.977+0000 7f186ffff700 1 -- 172.21.15.77:0/2217208769 learned_addr learned my addr 172.21.15.77:0/2217208769 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.977+0000 7f1874e20700 1 --2- 172.21.15.77:0/2217208769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564868d2d6e0 0x564868d2a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.977+0000 7f186f7fe700 1 --2- 172.21.15.77:0/2217208769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564868d2c7f0 0x564868d250f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.977+0000 7f186ffff700 1 -- 172.21.15.77:0/2217208769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564868d2c7f0 msgr2=0x564868d250f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.977+0000 7f186ffff700 1 --2- 172.21.15.77:0/2217208769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564868d2c7f0 0x564868d250f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.977+0000 7f186ffff700 1 -- 172.21.15.77:0/2217208769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564868d2d6e0 msgr2=0x564868d2a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.977+0000 7f186ffff700 1 --2- 172.21.15.77:0/2217208769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564868d2d6e0 0x564868d2a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.977+0000 7f186ffff700 1 -- 172.21.15.77:0/2217208769 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18701a1620 con 0x564868d2be90 2023-12-01T01:18:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.977+0000 7f186f7fe700 1 --2- 172.21.15.77:0/2217208769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564868d2c7f0 0x564868d250f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:18:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.978+0000 7f186ffff700 1 --2- 172.21.15.77:0/2217208769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564868d2be90 0x564868d24bb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1860098f30 tx=0x7f186009ce90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9881f28d14b0abbc server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.978+0000 7f184f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2023-12-01T01:18:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.978+0000 7f184f7fe700 1 -- 172.21.15.77:0/2217208769 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18600a0b80 con 0x564868d2be90 2023-12-01T01:18:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.978+0000 7f184f7fe700 1 -- 172.21.15.77:0/2217208769 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f18600a0ce0 con 0x564868d2be90 2023-12-01T01:18:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.978+0000 7f1882a13b80 1 -- 172.21.15.77:0/2217208769 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564868c9fcb0 con 0x564868d2be90 2023-12-01T01:18:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.978+0000 7f184f7fe700 1 -- 172.21.15.77:0/2217208769 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1860093bb0 con 0x564868d2be90 2023-12-01T01:18:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.978+0000 7f1882a13b80 10 client.25127 fetch_fsmap learned FSMap version 27 2023-12-01T01:18:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.978+0000 7f1882a13b80 1 -- 172.21.15.77:0/2217208769 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564868a8bc00 con 0x564868d2be90 2023-12-01T01:18:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.978+0000 7f184f7fe700 1 -- 172.21.15.77:0/2217208769 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564868c9fcb0 con 0x564868d2be90 2023-12-01T01:18:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.978+0000 7f184f7fe700 1 -- 172.21.15.77:0/2217208769 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f1860060070 con 0x564868d2be90 2023-12-01T01:18:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.978+0000 7f1882a13b80 10 client.25127 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:18:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.978+0000 7f1882a13b80 10 client.25127 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-01T01:18:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.978+0000 7f1882a13b80 20 client.25127 populate_metadata read hostname 'smithi077' 2023-12-01T01:18:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.979+0000 7f1882a13b80 1 --2- 172.21.15.77:0/2217208769 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x564868d49e60 0x564868d4c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.979+0000 7f1882a13b80 4 client.25127 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:18:17.984 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.979+0000 7f1882a13b80 1 -- 172.21.15.77:0/2217208769 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564868cf2db0 con 0x564868d49e60 2023-12-01T01:18:17.984 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.979+0000 7f186f7fe700 1 --2- 172.21.15.77:0/2217208769 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x564868d49e60 0x564868d4c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:17.984 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.979+0000 7f184f7fe700 10 client.25127 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:17.984 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.979+0000 7f186f7fe700 1 --2- 172.21.15.77:0/2217208769 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x564868d49e60 0x564868d4c250 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8da752fc91d78f55 server_cookie=8a4d3823fe0effd0 in_seq=0 out_seq=0 2023-12-01T01:18:17.985 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.980+0000 7f184f7fe700 1 -- 172.21.15.77:0/2217208769 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x564868cf2db0 con 0x564868d49e60 2023-12-01T01:18:17.985 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.980+0000 7f184f7fe700 10 client.25127 handle_command_reply: tid=0 2023-12-01T01:18:17.985 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.980+0000 7f1882a13b80 1 client.25127 shutdown 2023-12-01T01:18:17.985 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.980+0000 7f1882a13b80 1 -- 172.21.15.77:0/2217208769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564868d2be90 msgr2=0x564868d24bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:17.985 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.980+0000 7f1882a13b80 1 --2- 172.21.15.77:0/2217208769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564868d2be90 0x564868d24bb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1860098f30 tx=0x7f186009ce90 comp rx=0 tx=0).stop 2023-12-01T01:18:17.985 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.980+0000 7f1882a13b80 1 -- 172.21.15.77:0/2217208769 shutdown_connections 2023-12-01T01:18:17.985 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.980+0000 7f1882a13b80 1 --2- 172.21.15.77:0/2217208769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564868d2d6e0 0x564868d2a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:17.985 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.980+0000 7f1882a13b80 1 --2- 172.21.15.77:0/2217208769 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x564868d49e60 0x564868d4c250 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:17.985 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.980+0000 7f1882a13b80 1 --2- 172.21.15.77:0/2217208769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564868d2be90 0x564868d24bb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:17.986 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.980+0000 7f1882a13b80 1 --2- 172.21.15.77:0/2217208769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564868d2c7f0 0x564868d250f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:17.986 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.980+0000 7f1882a13b80 1 -- 172.21.15.77:0/2217208769 >> 172.21.15.77:0/2217208769 conn(0x564868c83070 msgr2=0x564868d1e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:17.986 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.980+0000 7f184f7fe700 0 client.25127 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:17.986 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.980+0000 7f1882a13b80 1 -- 172.21.15.77:0/2217208769 shutdown_connections 2023-12-01T01:18:17.986 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.980+0000 7f1882a13b80 1 -- 172.21.15.77:0/2217208769 wait complete. 2023-12-01T01:18:17.986 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:17.980+0000 7f1882a13b80 20 client.25127 trim_cache size 0 max 16384 2023-12-01T01:18:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.008+0000 7f1882a13b80 1 Processor -- start 2023-12-01T01:18:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.008+0000 7f1882a13b80 1 -- start start 2023-12-01T01:18:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.008+0000 7f1882a13b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564868d2be90 0x564868d2a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.008+0000 7f1882a13b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564868d2c7f0 0x564868d237d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.008+0000 7f1882a13b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564868d2d6e0 0x564868d33b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.008+0000 7f1882a13b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x564868d25da0 con 0x564868d2d6e0 2023-12-01T01:18:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.008+0000 7f1882a13b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564868d2b920 con 0x564868d2be90 2023-12-01T01:18:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.008+0000 7f1882a13b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x564868d167a0 con 0x564868d2c7f0 2023-12-01T01:18:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.008+0000 7f1874e20700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564868d2d6e0 0x564868d33b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.008+0000 7f1874e20700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564868d2d6e0 0x564868d33b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59410/0 (socket says 172.21.15.77:59410) 2023-12-01T01:18:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.008+0000 7f1874e20700 1 -- 172.21.15.77:0/15208551 learned_addr learned my addr 172.21.15.77:0/15208551 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.008+0000 7f186ffff700 1 --2- 172.21.15.77:0/15208551 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564868d2be90 0x564868d2a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.009+0000 7f186f7fe700 1 --2- 172.21.15.77:0/15208551 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564868d2c7f0 0x564868d237d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.009+0000 7f1874e20700 1 -- 172.21.15.77:0/15208551 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564868d2c7f0 msgr2=0x564868d237d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.009+0000 7f1874e20700 1 --2- 172.21.15.77:0/15208551 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564868d2c7f0 0x564868d237d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.009+0000 7f1874e20700 1 -- 172.21.15.77:0/15208551 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564868d2be90 msgr2=0x564868d2a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.009+0000 7f1874e20700 1 --2- 172.21.15.77:0/15208551 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564868d2be90 0x564868d2a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.009+0000 7f1874e20700 1 -- 172.21.15.77:0/15208551 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564868ac91f0 con 0x564868d2d6e0 2023-12-01T01:18:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.009+0000 7f186ffff700 1 --2- 172.21.15.77:0/15208551 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564868d2be90 0x564868d2a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:18:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.009+0000 7f186f7fe700 1 --2- 172.21.15.77:0/15208551 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564868d2c7f0 0x564868d237d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:18:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.009+0000 7f1874e20700 1 --2- 172.21.15.77:0/15208551 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564868d2d6e0 0x564868d33b00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f186800a8d0 tx=0x7f1868009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caadd6e40b3a2a4b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.009+0000 7f1856ffd700 1 -- 172.21.15.77:0/15208551 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1868010050 con 0x564868d2d6e0 2023-12-01T01:18:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.009+0000 7f1856ffd700 1 -- 172.21.15.77:0/15208551 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1868011040 con 0x564868d2d6e0 2023-12-01T01:18:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.009+0000 7f1856ffd700 1 -- 172.21.15.77:0/15208551 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1868017d50 con 0x564868d2d6e0 2023-12-01T01:18:18.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:17 smithi203 ceph-mon[103581]: pgmap v310: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2023-12-01T01:18:18.015 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.010+0000 7f1882a13b80 1 -- 172.21.15.77:0/15208551 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564868d2d6e0 msgr2=0x564868d33b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:18.015 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.010+0000 7f1882a13b80 1 --2- 172.21.15.77:0/15208551 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564868d2d6e0 0x564868d33b00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f186800a8d0 tx=0x7f1868009650 comp rx=0 tx=0).stop 2023-12-01T01:18:18.015 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.010+0000 7f1882a13b80 1 -- 172.21.15.77:0/15208551 shutdown_connections 2023-12-01T01:18:18.015 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.010+0000 7f1882a13b80 1 --2- 172.21.15.77:0/15208551 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564868d2d6e0 0x564868d33b00 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:18.015 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.010+0000 7f1882a13b80 1 --2- 172.21.15.77:0/15208551 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564868d2be90 0x564868d2a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:18.015 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.010+0000 7f1882a13b80 1 --2- 172.21.15.77:0/15208551 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564868d2c7f0 0x564868d237d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:18.015 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.010+0000 7f1882a13b80 1 -- 172.21.15.77:0/15208551 >> 172.21.15.77:0/15208551 conn(0x564868c83070 msgr2=0x564868d1fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:18.016 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.010+0000 7f1882a13b80 1 -- 172.21.15.77:0/15208551 shutdown_connections 2023-12-01T01:18:18.016 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.010+0000 7f1882a13b80 1 -- 172.21.15.77:0/15208551 wait complete. 2023-12-01T01:18:18.016 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.011+0000 7f1882a13b80 1 -- start start 2023-12-01T01:18:18.016 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.011+0000 7f1882a13b80 1 Processor -- start 2023-12-01T01:18:18.016 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.011+0000 7f1882a13b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564868d2be90 0x564868d95960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:18.016 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.011+0000 7f1882a13b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564868d2c7f0 0x564868d97eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:18.016 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.011+0000 7f1882a13b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564868d2d6e0 0x564868d9a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:18.016 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.011+0000 7f1882a13b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x564868d9a980 con 0x564868d2d6e0 2023-12-01T01:18:18.016 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.011+0000 7f1882a13b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564868d9aaf0 con 0x564868d2be90 2023-12-01T01:18:18.016 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.011+0000 7f1882a13b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x564868d9acb0 con 0x564868d2c7f0 2023-12-01T01:18:18.016 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.011+0000 7f1882a13b80 10 client.0 _ll_register_callbacks cb 0x7f1860098900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:18:18.016 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.011+0000 7f1874e20700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564868d2d6e0 0x564868d9a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:18.016 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.011+0000 7f1874e20700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564868d2d6e0 0x564868d9a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59416/0 (socket says 172.21.15.77:59416) 2023-12-01T01:18:18.016 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.011+0000 7f1874e20700 1 -- 172.21.15.77:0/693356484 learned_addr learned my addr 172.21.15.77:0/693356484 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:18.016 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.012+0000 7f186f7fe700 1 --2- 172.21.15.77:0/693356484 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564868d2c7f0 0x564868d97eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:18.017 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.012+0000 7f186ffff700 1 --2- 172.21.15.77:0/693356484 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564868d2be90 0x564868d95960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:18.017 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.012+0000 7f1874e20700 1 -- 172.21.15.77:0/693356484 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564868d2c7f0 msgr2=0x564868d97eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:18.017 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.012+0000 7f1874e20700 1 --2- 172.21.15.77:0/693356484 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564868d2c7f0 0x564868d97eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:18.017 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.012+0000 7f1874e20700 1 -- 172.21.15.77:0/693356484 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564868d2be90 msgr2=0x564868d95960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:18.017 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.012+0000 7f1874e20700 1 --2- 172.21.15.77:0/693356484 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564868d2be90 0x564868d95960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:18.017 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.012+0000 7f1874e20700 1 -- 172.21.15.77:0/693356484 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f186800a580 con 0x564868d2d6e0 2023-12-01T01:18:18.017 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.012+0000 7f186ffff700 1 --2- 172.21.15.77:0/693356484 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564868d2be90 0x564868d95960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:18:18.017 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.012+0000 7f1874e20700 1 --2- 172.21.15.77:0/693356484 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564868d2d6e0 0x564868d9a400 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1868000f80 tx=0x7f1868017cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a93199794c44da server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:18.017 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.012+0000 7f184f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:18:18.017 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.012+0000 7f184f7fe700 1 -- 172.21.15.77:0/693356484 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1868025070 con 0x564868d2d6e0 2023-12-01T01:18:18.017 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.012+0000 7f184f7fe700 1 -- 172.21.15.77:0/693356484 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f1868006e40 con 0x564868d2d6e0 2023-12-01T01:18:18.017 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.012+0000 7f1882a13b80 1 -- 172.21.15.77:0/693356484 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1860097340 con 0x564868d2d6e0 2023-12-01T01:18:18.017 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.013+0000 7f184f7fe700 1 -- 172.21.15.77:0/693356484 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1868010020 con 0x564868d2d6e0 2023-12-01T01:18:18.017 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.013+0000 7f1882a13b80 10 client.15522 fetch_fsmap learned FSMap version 27 2023-12-01T01:18:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.013+0000 7f1882a13b80 1 -- 172.21.15.77:0/693356484 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564868aa1250 con 0x564868d2d6e0 2023-12-01T01:18:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.013+0000 7f184f7fe700 1 -- 172.21.15.77:0/693356484 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1860097340 con 0x564868d2d6e0 2023-12-01T01:18:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.013+0000 7f184f7fe700 1 -- 172.21.15.77:0/693356484 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f1868020ad0 con 0x564868d2d6e0 2023-12-01T01:18:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.013+0000 7f1882a13b80 10 client.15522 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:18:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.013+0000 7f1882a13b80 10 client.15522 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-01T01:18:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.013+0000 7f1882a13b80 20 client.15522 populate_metadata read hostname 'smithi077' 2023-12-01T01:18:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.013+0000 7f1882a13b80 1 --2- 172.21.15.77:0/693356484 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x564868d9c5e0 0x564868d9ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.013+0000 7f1882a13b80 4 client.15522 mds_command: new command op to 24500 tid=0[{"prefix": "flush journal"}] 2023-12-01T01:18:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.013+0000 7f1882a13b80 1 -- 172.21.15.77:0/693356484 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564868c6dce0 con 0x564868d9c5e0 2023-12-01T01:18:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.013+0000 7f186ffff700 1 --2- 172.21.15.77:0/693356484 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x564868d9c5e0 0x564868d9ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.014+0000 7f184f7fe700 10 client.15522 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.014+0000 7f186ffff700 1 --2- 172.21.15.77:0/693356484 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x564868d9c5e0 0x564868d9ea00 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9da7801ec8fd5338 server_cookie=f9ec02bb03a24d8c in_seq=0 out_seq=0 2023-12-01T01:18:18.019 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.017+0000 7f184f7fe700 1 -- 172.21.15.77:0/693356484 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564868c6dce0 con 0x564868d9c5e0 2023-12-01T01:18:18.019 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.017+0000 7f184f7fe700 10 client.15522 handle_command_reply: tid=0 2023-12-01T01:18:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.018+0000 7f1882a13b80 1 client.15522 shutdown 2023-12-01T01:18:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.018+0000 7f1882a13b80 1 -- 172.21.15.77:0/693356484 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564868d2d6e0 msgr2=0x564868d9a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.018+0000 7f1882a13b80 1 --2- 172.21.15.77:0/693356484 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564868d2d6e0 0x564868d9a400 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1868000f80 tx=0x7f1868017cc0 comp rx=0 tx=0).stop 2023-12-01T01:18:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.018+0000 7f1882a13b80 1 -- 172.21.15.77:0/693356484 shutdown_connections 2023-12-01T01:18:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.018+0000 7f1882a13b80 1 --2- 172.21.15.77:0/693356484 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564868d2d6e0 0x564868d9a400 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.018+0000 7f1882a13b80 1 --2- 172.21.15.77:0/693356484 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x564868d9c5e0 0x564868d9ea00 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.018+0000 7f1882a13b80 1 --2- 172.21.15.77:0/693356484 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564868d2be90 0x564868d95960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.018+0000 7f1882a13b80 1 --2- 172.21.15.77:0/693356484 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564868d2c7f0 0x564868d97eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.018+0000 7f1882a13b80 1 -- 172.21.15.77:0/693356484 >> 172.21.15.77:0/693356484 conn(0x564868c83070 msgr2=0x564868d2a890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.018+0000 7f184f7fe700 0 client.15522 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.018+0000 7f1882a13b80 1 -- 172.21.15.77:0/693356484 shutdown_connections 2023-12-01T01:18:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.018+0000 7f1882a13b80 1 -- 172.21.15.77:0/693356484 wait complete. 2023-12-01T01:18:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.018+0000 7f1882a13b80 20 client.15522 trim_cache size 0 max 16384 2023-12-01T01:18:18.021 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout:{ 2023-12-01T01:18:18.021 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout: "message": "", 2023-12-01T01:18:18.021 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout: "return_code": 0 2023-12-01T01:18:18.021 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout:} 2023-12-01T01:18:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.019+0000 7f1876883700 1 -- 172.21.15.77:0/929113724 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f18580a0520 msgr2=0x7f18580a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.019+0000 7f1876883700 1 --2- 172.21.15.77:0/929113724 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f18580a0520 0x7f18580a29e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1864000c00 tx=0x7f1864005fb0 comp rx=0 tx=0).stop 2023-12-01T01:18:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.019+0000 7f1876883700 1 -- 172.21.15.77:0/929113724 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18700ffe90 msgr2=0x7f1870198e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.019+0000 7f1876883700 1 --2- 172.21.15.77:0/929113724 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18700ffe90 0x7f1870198e40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f186000d950 tx=0x7f186000dc60 comp rx=0 tx=0).stop 2023-12-01T01:18:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.019+0000 7f1876883700 1 -- 172.21.15.77:0/929113724 shutdown_connections 2023-12-01T01:18:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.019+0000 7f1876883700 1 --2- 172.21.15.77:0/929113724 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f18580a0520 0x7f18580a29e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.019+0000 7f1876883700 1 --2- 172.21.15.77:0/929113724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1870101240 0x7f187019a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.019+0000 7f1876883700 1 --2- 172.21.15.77:0/929113724 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18700ffe90 0x7f1870198e40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.019+0000 7f1876883700 1 --2- 172.21.15.77:0/929113724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1870100850 0x7f1870199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.019+0000 7f1876883700 1 -- 172.21.15.77:0/929113724 >> 172.21.15.77:0/929113724 conn(0x7f18700fa960 msgr2=0x7f18701067f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.019+0000 7f1876883700 1 -- 172.21.15.77:0/929113724 shutdown_connections 2023-12-01T01:18:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:18.019+0000 7f1876883700 1 -- 172.21.15.77:0/929113724 wait complete. 2023-12-01T01:18:18.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:17 smithi137 ceph-mon[103312]: pgmap v310: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2023-12-01T01:18:19.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:19 smithi077 ceph-mon[98867]: pgmap v311: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1023 B/s wr, 14 op/s 2023-12-01T01:18:20.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:19 smithi203 ceph-mon[103581]: pgmap v311: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1023 B/s wr, 14 op/s 2023-12-01T01:18:20.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:19 smithi137 ceph-mon[103312]: pgmap v311: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1023 B/s wr, 14 op/s 2023-12-01T01:18:21.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:21 smithi077 ceph-mon[98867]: pgmap v312: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 11 op/s 2023-12-01T01:18:21.995 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-01T01:18:22.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:21 smithi203 ceph-mon[103581]: pgmap v312: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 11 op/s 2023-12-01T01:18:22.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:21 smithi137 ceph-mon[103312]: pgmap v312: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 11 op/s 2023-12-01T01:18:22.189 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.187+0000 7ff9c934b700 1 -- 172.21.15.77:0/3500351305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9c40ffc80 msgr2=0x7ff9c41000a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:22.189 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.187+0000 7ff9c934b700 1 --2- 172.21.15.77:0/3500351305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9c40ffc80 0x7ff9c41000a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8009990 tx=0x7ff9b8009ca0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.189 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.187+0000 7ff9c934b700 1 -- 172.21.15.77:0/3500351305 shutdown_connections 2023-12-01T01:18:22.189 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.187+0000 7ff9c934b700 1 --2- 172.21.15.77:0/3500351305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9c4101810 0x7ff9c41080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.190 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.187+0000 7ff9c934b700 1 --2- 172.21.15.77:0/3500351305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9c40ffc80 0x7ff9c41000a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.190 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.187+0000 7ff9c934b700 1 --2- 172.21.15.77:0/3500351305 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9c4100e70 0x7ff9c41012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.190 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.187+0000 7ff9c934b700 1 -- 172.21.15.77:0/3500351305 >> 172.21.15.77:0/3500351305 conn(0x7ff9c40fb1f0 msgr2=0x7ff9c40fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:22.190 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.187+0000 7ff9c934b700 1 -- 172.21.15.77:0/3500351305 shutdown_connections 2023-12-01T01:18:22.190 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.187+0000 7ff9c934b700 1 -- 172.21.15.77:0/3500351305 wait complete. 2023-12-01T01:18:22.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.188+0000 7ff9c934b700 1 Processor -- start 2023-12-01T01:18:22.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.189+0000 7ff9c934b700 1 -- start start 2023-12-01T01:18:22.193 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.189+0000 7ff9c934b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9c40ffc80 0x7ff9c410a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:22.194 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.189+0000 7ff9c934b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9c4100e70 0x7ff9c4108a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:22.194 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.189+0000 7ff9c934b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9c4101810 0x7ff9c4108f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:22.194 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.189+0000 7ff9c934b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c4109720 con 0x7ff9c4101810 2023-12-01T01:18:22.194 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.189+0000 7ff9c934b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c4109890 con 0x7ff9c4100e70 2023-12-01T01:18:22.194 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.189+0000 7ff9c934b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c41a0b40 con 0x7ff9c40ffc80 2023-12-01T01:18:22.194 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.189+0000 7ff9c37fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9c4101810 0x7ff9c4108f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:22.194 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.189+0000 7ff9c37fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9c4101810 0x7ff9c4108f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59430/0 (socket says 172.21.15.77:59430) 2023-12-01T01:18:22.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.189+0000 7ff9c37fe700 1 -- 172.21.15.77:0/2837025127 learned_addr learned my addr 172.21.15.77:0/2837025127 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:22.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.189+0000 7ff9c2ffd700 1 --2- 172.21.15.77:0/2837025127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9c40ffc80 0x7ff9c410a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:22.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.189+0000 7ff9c27fc700 1 --2- 172.21.15.77:0/2837025127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9c4100e70 0x7ff9c4108a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:22.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.189+0000 7ff9c37fe700 1 -- 172.21.15.77:0/2837025127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9c40ffc80 msgr2=0x7ff9c410a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:22.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.189+0000 7ff9c37fe700 1 --2- 172.21.15.77:0/2837025127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9c40ffc80 0x7ff9c410a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.189+0000 7ff9c37fe700 1 -- 172.21.15.77:0/2837025127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9c4100e70 msgr2=0x7ff9c4108a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:22.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.189+0000 7ff9c37fe700 1 --2- 172.21.15.77:0/2837025127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9c4100e70 0x7ff9c4108a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.189+0000 7ff9c37fe700 1 -- 172.21.15.77:0/2837025127 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b80095f0 con 0x7ff9c4101810 2023-12-01T01:18:22.196 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.189+0000 7ff9c27fc700 1 --2- 172.21.15.77:0/2837025127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9c4100e70 0x7ff9c4108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:18:22.196 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.190+0000 7ff9c37fe700 1 --2- 172.21.15.77:0/2837025127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9c4101810 0x7ff9c4108f80 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc00b970 tx=0x7ff9bc00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:22.196 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.190+0000 7ff9abfff700 1 -- 172.21.15.77:0/2837025127 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc00c930 con 0x7ff9c4101810 2023-12-01T01:18:22.196 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.190+0000 7ff9c934b700 1 -- 172.21.15.77:0/2837025127 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9c41a0e20 con 0x7ff9c4101810 2023-12-01T01:18:22.196 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.190+0000 7ff9abfff700 1 -- 172.21.15.77:0/2837025127 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9bc014440 con 0x7ff9c4101810 2023-12-01T01:18:22.196 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.190+0000 7ff9c934b700 1 -- 172.21.15.77:0/2837025127 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9c41a1320 con 0x7ff9c4101810 2023-12-01T01:18:22.196 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.190+0000 7ff9abfff700 1 -- 172.21.15.77:0/2837025127 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc0075d0 con 0x7ff9c4101810 2023-12-01T01:18:22.205 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.193+0000 7ff9abfff700 1 -- 172.21.15.77:0/2837025127 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7ff9bc00ca90 con 0x7ff9c4101810 2023-12-01T01:18:22.205 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.194+0000 7ff9abfff700 1 --2- 172.21.15.77:0/2837025127 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff9ac0a0520 0x7ff9ac0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:22.205 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.194+0000 7ff9abfff700 1 -- 172.21.15.77:0/2837025127 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7ff9bc006e40 con 0x7ff9c4101810 2023-12-01T01:18:22.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.195+0000 7ff9c2ffd700 1 --2- 172.21.15.77:0/2837025127 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff9ac0a0520 0x7ff9ac0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:22.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.195+0000 7ff9d54dbb80 1 Processor -- start 2023-12-01T01:18:22.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.195+0000 7ff9d54dbb80 1 -- start start 2023-12-01T01:18:22.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.195+0000 7ff9d54dbb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ca8662960 0x556ca8662d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:22.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.195+0000 7ff9d54dbb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca8661670 0x556ca8661af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:22.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.195+0000 7ff9d54dbb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ca8662030 0x556ca86698d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:22.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.195+0000 7ff9d54dbb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ca85b0a10 con 0x556ca8662030 2023-12-01T01:18:22.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.195+0000 7ff9d54dbb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ca85ae3d0 con 0x556ca8662960 2023-12-01T01:18:22.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.195+0000 7ff9c2ffd700 1 --2- 172.21.15.77:0/2837025127 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff9ac0a0520 0x7ff9ac0a29e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff9b800b2e0 tx=0x7ff9b8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:22.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.195+0000 7ff9d54dbb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ca85a9cc0 con 0x556ca8661670 2023-12-01T01:18:22.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.195+0000 7ff9c37fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ca8662030 0x556ca86698d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:22.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.195+0000 7ff9c37fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ca8662030 0x556ca86698d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59438/0 (socket says 172.21.15.77:59438) 2023-12-01T01:18:22.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.195+0000 7ff9c37fe700 1 -- 172.21.15.77:0/3059817028 learned_addr learned my addr 172.21.15.77:0/3059817028 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:22.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.195+0000 7ff9c2ffd700 1 --2- 172.21.15.77:0/3059817028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ca8662960 0x556ca8662d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:22.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.196+0000 7ff9c27fc700 1 --2- 172.21.15.77:0/3059817028 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca8661670 0x556ca8661af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:22.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.196+0000 7ff9c37fe700 1 -- 172.21.15.77:0/3059817028 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca8661670 msgr2=0x556ca8661af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:22.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.196+0000 7ff9c37fe700 1 --2- 172.21.15.77:0/3059817028 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca8661670 0x556ca8661af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.196+0000 7ff9c37fe700 1 -- 172.21.15.77:0/3059817028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ca8662960 msgr2=0x556ca8662d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:22.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.196+0000 7ff9c37fe700 1 --2- 172.21.15.77:0/3059817028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ca8662960 0x556ca8662d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.196+0000 7ff9c37fe700 1 -- 172.21.15.77:0/3059817028 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ca83d7de0 con 0x556ca8662030 2023-12-01T01:18:22.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.196+0000 7ff9c37fe700 1 --2- 172.21.15.77:0/3059817028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ca8662030 0x556ca86698d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc095250 tx=0x7ff9bc004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f79576c68cffb8f9 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:22.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.196+0000 7ff9a97fa700 1 -- 172.21.15.77:0/3059817028 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc0c8ea0 con 0x556ca8662030 2023-12-01T01:18:22.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.196+0000 7ff9a97fa700 1 -- 172.21.15.77:0/3059817028 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9bc09c610 con 0x556ca8662030 2023-12-01T01:18:22.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.196+0000 7ff9a97fa700 1 -- 172.21.15.77:0/3059817028 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc097f30 con 0x556ca8662030 2023-12-01T01:18:22.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.197+0000 7ff9d54dbb80 1 -- 172.21.15.77:0/3059817028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ca8662030 msgr2=0x556ca86698d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:22.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.197+0000 7ff9d54dbb80 1 --2- 172.21.15.77:0/3059817028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ca8662030 0x556ca86698d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc095250 tx=0x7ff9bc004be0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.197+0000 7ff9d54dbb80 1 -- 172.21.15.77:0/3059817028 shutdown_connections 2023-12-01T01:18:22.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.197+0000 7ff9d54dbb80 1 --2- 172.21.15.77:0/3059817028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ca8662030 0x556ca86698d0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.197+0000 7ff9d54dbb80 1 --2- 172.21.15.77:0/3059817028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ca8662960 0x556ca8662d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.210 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.197+0000 7ff9d54dbb80 1 --2- 172.21.15.77:0/3059817028 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca8661670 0x556ca8661af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.210 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.197+0000 7ff9d54dbb80 1 -- 172.21.15.77:0/3059817028 >> 172.21.15.77:0/3059817028 conn(0x556ca85b80d0 msgr2=0x556ca8653940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:22.210 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.197+0000 7ff9d54dbb80 1 -- 172.21.15.77:0/3059817028 shutdown_connections 2023-12-01T01:18:22.210 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.197+0000 7ff9d54dbb80 1 -- 172.21.15.77:0/3059817028 wait complete. 2023-12-01T01:18:22.210 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.198+0000 7ff9d54dbb80 1 -- start start 2023-12-01T01:18:22.210 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.198+0000 7ff9d54dbb80 1 Processor -- start 2023-12-01T01:18:22.210 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.198+0000 7ff9d54dbb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca8661670 0x556ca865a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:22.211 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.198+0000 7ff9d54dbb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ca8662030 0x556ca865a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:22.211 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.198+0000 7ff9d54dbb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ca8662960 0x556ca865f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:22.211 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.198+0000 7ff9d54dbb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ca8546510 con 0x556ca8662030 2023-12-01T01:18:22.211 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.198+0000 7ff9d54dbb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ca85461b0 con 0x556ca8662960 2023-12-01T01:18:22.211 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.198+0000 7ff9d54dbb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ca842d920 con 0x556ca8661670 2023-12-01T01:18:22.211 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.198+0000 7ff9d54dbb80 10 client.0 _ll_register_callbacks cb 0x556ca8493a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:18:22.211 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.198+0000 7ff9c27fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ca8662030 0x556ca865a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:22.212 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.198+0000 7ff9c27fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ca8662030 0x556ca865a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59444/0 (socket says 172.21.15.77:59444) 2023-12-01T01:18:22.212 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.198+0000 7ff9c27fc700 1 -- 172.21.15.77:0/2121994010 learned_addr learned my addr 172.21.15.77:0/2121994010 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:22.212 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.198+0000 7ff9c2ffd700 1 --2- 172.21.15.77:0/2121994010 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca8661670 0x556ca865a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:22.212 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.198+0000 7ff9c37fe700 1 --2- 172.21.15.77:0/2121994010 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ca8662960 0x556ca865f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:22.212 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.199+0000 7ff9c27fc700 1 -- 172.21.15.77:0/2121994010 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca8661670 msgr2=0x556ca865a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:22.212 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.199+0000 7ff9c27fc700 1 --2- 172.21.15.77:0/2121994010 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca8661670 0x556ca865a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.212 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.199+0000 7ff9c27fc700 1 -- 172.21.15.77:0/2121994010 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ca8662960 msgr2=0x556ca865f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:22.212 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.199+0000 7ff9c27fc700 1 --2- 172.21.15.77:0/2121994010 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ca8662960 0x556ca865f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.213 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.199+0000 7ff9c27fc700 1 -- 172.21.15.77:0/2121994010 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9c41a1320 con 0x556ca8662030 2023-12-01T01:18:22.213 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.199+0000 7ff9c37fe700 1 --2- 172.21.15.77:0/2121994010 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ca8662960 0x556ca865f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:18:22.213 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.199+0000 7ff9c27fc700 1 --2- 172.21.15.77:0/2121994010 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ca8662030 0x556ca865a740 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff9b40126a0 tx=0x7ff9b40129b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b333cb6cce145984 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:22.213 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.199+0000 7ff9a6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:18:22.213 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.199+0000 7ff9a6ff5700 1 -- 172.21.15.77:0/2121994010 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4010810 con 0x556ca8662030 2023-12-01T01:18:22.213 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.199+0000 7ff9d54dbb80 1 -- 172.21.15.77:0/2121994010 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556ca85d4d10 con 0x556ca8662030 2023-12-01T01:18:22.213 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.199+0000 7ff9a6ff5700 1 -- 172.21.15.77:0/2121994010 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7ff9b4010970 con 0x556ca8662030 2023-12-01T01:18:22.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.199+0000 7ff9a6ff5700 1 -- 172.21.15.77:0/2121994010 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9b4010b30 con 0x556ca8662030 2023-12-01T01:18:22.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.199+0000 7ff9d54dbb80 10 client.15531 fetch_fsmap learned FSMap version 27 2023-12-01T01:18:22.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.199+0000 7ff9d54dbb80 1 -- 172.21.15.77:0/2121994010 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556ca83c0c60 con 0x556ca8662030 2023-12-01T01:18:22.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.200+0000 7ff9a6ff5700 1 -- 172.21.15.77:0/2121994010 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556ca85d4d10 con 0x556ca8662030 2023-12-01T01:18:22.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.200+0000 7ff9a6ff5700 1 -- 172.21.15.77:0/2121994010 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7ff9b400e3c0 con 0x556ca8662030 2023-12-01T01:18:22.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.200+0000 7ff9d54dbb80 10 client.15531 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:18:22.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.200+0000 7ff9d54dbb80 10 client.15531 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-01T01:18:22.215 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.200+0000 7ff9d54dbb80 20 client.15531 populate_metadata read hostname 'smithi077' 2023-12-01T01:18:22.215 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.200+0000 7ff9d54dbb80 1 --2- 172.21.15.77:0/2121994010 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x556ca867f880 0x556ca8681ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:22.215 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.200+0000 7ff9d54dbb80 4 client.15531 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:18:22.215 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.200+0000 7ff9d54dbb80 1 -- 172.21.15.77:0/2121994010 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556ca8627e10 con 0x556ca867f880 2023-12-01T01:18:22.215 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.200+0000 7ff9c2ffd700 1 --2- 172.21.15.77:0/2121994010 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x556ca867f880 0x556ca8681ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:22.215 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.201+0000 7ff9a6ff5700 10 client.15531 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:22.215 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.201+0000 7ff9c2ffd700 1 --2- 172.21.15.77:0/2121994010 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x556ca867f880 0x556ca8681ca0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88db13fa2a8f0c30 server_cookie=4dcff55be8ac5702 in_seq=0 out_seq=0 2023-12-01T01:18:22.216 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.201+0000 7ff9a6ff5700 1 -- 172.21.15.77:0/2121994010 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x556ca8627e10 con 0x556ca867f880 2023-12-01T01:18:22.216 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.201+0000 7ff9a6ff5700 10 client.15531 handle_command_reply: tid=0 2023-12-01T01:18:22.216 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.201+0000 7ff9d54dbb80 1 client.15531 shutdown 2023-12-01T01:18:22.216 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.201+0000 7ff9d54dbb80 1 -- 172.21.15.77:0/2121994010 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ca8662030 msgr2=0x556ca865a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:22.216 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.201+0000 7ff9d54dbb80 1 --2- 172.21.15.77:0/2121994010 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ca8662030 0x556ca865a740 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff9b40126a0 tx=0x7ff9b40129b0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.216 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.201+0000 7ff9d54dbb80 1 -- 172.21.15.77:0/2121994010 shutdown_connections 2023-12-01T01:18:22.216 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.201+0000 7ff9d54dbb80 1 --2- 172.21.15.77:0/2121994010 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x556ca867f880 0x556ca8681ca0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.217 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.201+0000 7ff9d54dbb80 1 --2- 172.21.15.77:0/2121994010 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ca8662960 0x556ca865f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.217 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.201+0000 7ff9d54dbb80 1 --2- 172.21.15.77:0/2121994010 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca8661670 0x556ca865a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.217 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.201+0000 7ff9a6ff5700 0 client.15531 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:22.217 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.201+0000 7ff9d54dbb80 1 --2- 172.21.15.77:0/2121994010 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ca8662030 0x556ca865a740 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.217 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.201+0000 7ff9d54dbb80 1 -- 172.21.15.77:0/2121994010 >> 172.21.15.77:0/2121994010 conn(0x556ca85b80d0 msgr2=0x556ca86530b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:22.217 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.202+0000 7ff9d54dbb80 1 -- 172.21.15.77:0/2121994010 shutdown_connections 2023-12-01T01:18:22.217 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.202+0000 7ff9d54dbb80 1 -- 172.21.15.77:0/2121994010 wait complete. 2023-12-01T01:18:22.217 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.202+0000 7ff9d54dbb80 20 client.15531 trim_cache size 0 max 16384 2023-12-01T01:18:22.243 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.233+0000 7ff9d54dbb80 1 Processor -- start 2023-12-01T01:18:22.243 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.233+0000 7ff9d54dbb80 1 -- start start 2023-12-01T01:18:22.243 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.233+0000 7ff9d54dbb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca8661670 0x556ca8686600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:22.243 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.233+0000 7ff9d54dbb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ca8662030 0x556ca865f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:22.244 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.233+0000 7ff9d54dbb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ca8662960 0x556ca865a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:22.244 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.233+0000 7ff9d54dbb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ca85d4d10 con 0x556ca8662030 2023-12-01T01:18:22.244 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.233+0000 7ff9d54dbb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ca8659d70 con 0x556ca8662960 2023-12-01T01:18:22.244 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.233+0000 7ff9d54dbb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ca864bb50 con 0x556ca8661670 2023-12-01T01:18:22.244 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.234+0000 7ff9c27fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ca8662030 0x556ca865f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:22.244 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.234+0000 7ff9c27fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ca8662030 0x556ca865f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59452/0 (socket says 172.21.15.77:59452) 2023-12-01T01:18:22.244 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.234+0000 7ff9c27fc700 1 -- 172.21.15.77:0/1964022572 learned_addr learned my addr 172.21.15.77:0/1964022572 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:22.244 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.234+0000 7ff9c2ffd700 1 --2- 172.21.15.77:0/1964022572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca8661670 0x556ca8686600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:22.245 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.234+0000 7ff9c37fe700 1 --2- 172.21.15.77:0/1964022572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ca8662960 0x556ca865a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:22.245 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.234+0000 7ff9c27fc700 1 -- 172.21.15.77:0/1964022572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca8661670 msgr2=0x556ca8686600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:22.245 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.234+0000 7ff9c27fc700 1 --2- 172.21.15.77:0/1964022572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca8661670 0x556ca8686600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.245 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.234+0000 7ff9c27fc700 1 -- 172.21.15.77:0/1964022572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ca8662960 msgr2=0x556ca865a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:22.245 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.234+0000 7ff9c27fc700 1 --2- 172.21.15.77:0/1964022572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ca8662960 0x556ca865a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.245 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.234+0000 7ff9c27fc700 1 -- 172.21.15.77:0/1964022572 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ca83fe250 con 0x556ca8662030 2023-12-01T01:18:22.245 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.234+0000 7ff9c27fc700 1 --2- 172.21.15.77:0/1964022572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ca8662030 0x556ca865f690 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4001eb0 tx=0x7ff9b4012db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a398177d9dfeb7c6 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:22.246 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.234+0000 7ff9a9ffb700 1 -- 172.21.15.77:0/1964022572 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4005d60 con 0x556ca8662030 2023-12-01T01:18:22.246 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.234+0000 7ff9a9ffb700 1 -- 172.21.15.77:0/1964022572 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9b400ee00 con 0x556ca8662030 2023-12-01T01:18:22.246 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.234+0000 7ff9a9ffb700 1 -- 172.21.15.77:0/1964022572 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b40038e0 con 0x556ca8662030 2023-12-01T01:18:22.246 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.235+0000 7ff9d54dbb80 1 -- 172.21.15.77:0/1964022572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ca8662030 msgr2=0x556ca865f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:22.246 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.235+0000 7ff9d54dbb80 1 --2- 172.21.15.77:0/1964022572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ca8662030 0x556ca865f690 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4001eb0 tx=0x7ff9b4012db0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.246 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.235+0000 7ff9d54dbb80 1 -- 172.21.15.77:0/1964022572 shutdown_connections 2023-12-01T01:18:22.246 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.235+0000 7ff9d54dbb80 1 --2- 172.21.15.77:0/1964022572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ca8662960 0x556ca865a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.247 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.235+0000 7ff9d54dbb80 1 --2- 172.21.15.77:0/1964022572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca8661670 0x556ca8686600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.247 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.235+0000 7ff9d54dbb80 1 --2- 172.21.15.77:0/1964022572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ca8662030 0x556ca865f690 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.247 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.235+0000 7ff9d54dbb80 1 -- 172.21.15.77:0/1964022572 >> 172.21.15.77:0/1964022572 conn(0x556ca85b80d0 msgr2=0x556ca86693b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:22.247 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.236+0000 7ff9d54dbb80 1 -- 172.21.15.77:0/1964022572 shutdown_connections 2023-12-01T01:18:22.247 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.236+0000 7ff9d54dbb80 1 -- 172.21.15.77:0/1964022572 wait complete. 2023-12-01T01:18:22.247 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.236+0000 7ff9d54dbb80 1 -- start start 2023-12-01T01:18:22.247 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.236+0000 7ff9d54dbb80 1 Processor -- start 2023-12-01T01:18:22.247 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.236+0000 7ff9d54dbb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ca8661670 0x556ca86c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:22.248 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.236+0000 7ff9d54dbb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ca8662030 0x556ca86c9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:22.248 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.236+0000 7ff9d54dbb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca8662960 0x556ca86d0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:22.248 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.236+0000 7ff9d54dbb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ca86d0a70 con 0x556ca8661670 2023-12-01T01:18:22.248 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.236+0000 7ff9d54dbb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ca86d0be0 con 0x556ca8662030 2023-12-01T01:18:22.248 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.236+0000 7ff9d54dbb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ca86d0da0 con 0x556ca8662960 2023-12-01T01:18:22.248 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.236+0000 7ff9d54dbb80 10 client.0 _ll_register_callbacks cb 0x7ff9b4009c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:18:22.248 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.236+0000 7ff9c2ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ca8661670 0x556ca86c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:22.249 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.236+0000 7ff9c2ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ca8661670 0x556ca86c95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59456/0 (socket says 172.21.15.77:59456) 2023-12-01T01:18:22.249 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.237+0000 7ff9c2ffd700 1 -- 172.21.15.77:0/3775973385 learned_addr learned my addr 172.21.15.77:0/3775973385 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:22.249 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.237+0000 7ff9c27fc700 1 --2- 172.21.15.77:0/3775973385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ca8662030 0x556ca86c9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:22.249 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.237+0000 7ff9c37fe700 1 --2- 172.21.15.77:0/3775973385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca8662960 0x556ca86d0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:22.249 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.237+0000 7ff9c2ffd700 1 -- 172.21.15.77:0/3775973385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca8662960 msgr2=0x556ca86d0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:22.249 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.237+0000 7ff9c2ffd700 1 --2- 172.21.15.77:0/3775973385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca8662960 0x556ca86d0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.249 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.237+0000 7ff9c2ffd700 1 -- 172.21.15.77:0/3775973385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ca8662030 msgr2=0x556ca86c9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:22.250 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.237+0000 7ff9c2ffd700 1 --2- 172.21.15.77:0/3775973385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ca8662030 0x556ca86c9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.250 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.237+0000 7ff9c2ffd700 1 -- 172.21.15.77:0/3775973385 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556ca83c0c60 con 0x556ca8661670 2023-12-01T01:18:22.250 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.237+0000 7ff9c27fc700 1 --2- 172.21.15.77:0/3775973385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ca8662030 0x556ca86c9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:18:22.250 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.237+0000 7ff9c2ffd700 1 --2- 172.21.15.77:0/3775973385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ca8661670 0x556ca86c95a0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8009990 tx=0x7ff9b8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=899979b34fa61f6d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:22.250 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.237+0000 7ff9a6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:18:22.250 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.237+0000 7ff9a6ff5700 1 -- 172.21.15.77:0/3775973385 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b800f460 con 0x556ca8661670 2023-12-01T01:18:22.250 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.237+0000 7ff9a6ff5700 1 -- 172.21.15.77:0/3775973385 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7ff9b800f5c0 con 0x556ca8661670 2023-12-01T01:18:22.251 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.237+0000 7ff9d54dbb80 1 -- 172.21.15.77:0/3775973385 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff9bc09c610 con 0x556ca8661670 2023-12-01T01:18:22.251 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.238+0000 7ff9a6ff5700 1 -- 172.21.15.77:0/3775973385 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9b800f780 con 0x556ca8661670 2023-12-01T01:18:22.251 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.238+0000 7ff9d54dbb80 10 client.15537 fetch_fsmap learned FSMap version 27 2023-12-01T01:18:22.251 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.238+0000 7ff9d54dbb80 1 -- 172.21.15.77:0/3775973385 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556ca83d62b0 con 0x556ca8661670 2023-12-01T01:18:22.251 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.238+0000 7ff9a6ff5700 1 -- 172.21.15.77:0/3775973385 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9bc09c610 con 0x556ca8661670 2023-12-01T01:18:22.251 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.238+0000 7ff9a6ff5700 1 -- 172.21.15.77:0/3775973385 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7ff9b8004ad0 con 0x556ca8661670 2023-12-01T01:18:22.251 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.238+0000 7ff9d54dbb80 10 client.15537 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:18:22.251 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.238+0000 7ff9d54dbb80 10 client.15537 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-01T01:18:22.252 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.238+0000 7ff9d54dbb80 20 client.15537 populate_metadata read hostname 'smithi077' 2023-12-01T01:18:22.252 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.238+0000 7ff9d54dbb80 1 --2- 172.21.15.77:0/3775973385 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x556ca867f880 0x556ca86d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:22.252 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.238+0000 7ff9d54dbb80 4 client.15537 mds_command: new command op to 24500 tid=0[{"prefix": "scrub status"}] 2023-12-01T01:18:22.252 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.238+0000 7ff9d54dbb80 1 -- 172.21.15.77:0/3775973385 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556ca85a2d40 con 0x556ca867f880 2023-12-01T01:18:22.252 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.238+0000 7ff9c27fc700 1 --2- 172.21.15.77:0/3775973385 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x556ca867f880 0x556ca86d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:22.252 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.239+0000 7ff9a6ff5700 10 client.15537 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:22.252 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.239+0000 7ff9c27fc700 1 --2- 172.21.15.77:0/3775973385 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x556ca867f880 0x556ca86d47f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34e640c19db275fe server_cookie=8f1fc514a10ad3c7 in_seq=0 out_seq=0 2023-12-01T01:18:22.253 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.239+0000 7ff9a6ff5700 1 -- 172.21.15.77:0/3775973385 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x556ca85a2d40 con 0x556ca867f880 2023-12-01T01:18:22.253 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.239+0000 7ff9a6ff5700 10 client.15537 handle_command_reply: tid=0 2023-12-01T01:18:22.253 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.239+0000 7ff9d54dbb80 1 client.15537 shutdown 2023-12-01T01:18:22.253 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.239+0000 7ff9d54dbb80 1 -- 172.21.15.77:0/3775973385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ca8661670 msgr2=0x556ca86c95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:22.253 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.239+0000 7ff9d54dbb80 1 --2- 172.21.15.77:0/3775973385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ca8661670 0x556ca86c95a0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8009990 tx=0x7ff9b8009ca0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.253 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.239+0000 7ff9d54dbb80 1 -- 172.21.15.77:0/3775973385 shutdown_connections 2023-12-01T01:18:22.253 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.239+0000 7ff9d54dbb80 1 --2- 172.21.15.77:0/3775973385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ca8662960 0x556ca86d0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.253 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.239+0000 7ff9d54dbb80 1 --2- 172.21.15.77:0/3775973385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556ca8661670 0x556ca86c95a0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.254 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.239+0000 7ff9d54dbb80 1 --2- 172.21.15.77:0/3775973385 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x556ca867f880 0x556ca86d47f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.254 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.239+0000 7ff9d54dbb80 1 --2- 172.21.15.77:0/3775973385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ca8662030 0x556ca86c9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.254 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.239+0000 7ff9d54dbb80 1 -- 172.21.15.77:0/3775973385 >> 172.21.15.77:0/3775973385 conn(0x556ca85b80d0 msgr2=0x556ca8665ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:22.254 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.240+0000 7ff9a6ff5700 0 client.15537 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:22.254 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.240+0000 7ff9d54dbb80 1 -- 172.21.15.77:0/3775973385 shutdown_connections 2023-12-01T01:18:22.254 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.240+0000 7ff9d54dbb80 1 -- 172.21.15.77:0/3775973385 wait complete. 2023-12-01T01:18:22.254 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.240+0000 7ff9d54dbb80 20 client.15537 trim_cache size 0 max 16384 2023-12-01T01:18:22.255 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2023-12-01T01:18:22.255 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2023-12-01T01:18:22.255 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2023-12-01T01:18:22.255 INFO:teuthology.orchestra.run.smithi077.stdout:} 2023-12-01T01:18:22.256 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.240+0000 7ff9c934b700 1 -- 172.21.15.77:0/2837025127 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff9ac0a0520 msgr2=0x7ff9ac0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:22.257 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.240+0000 7ff9c934b700 1 --2- 172.21.15.77:0/2837025127 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff9ac0a0520 0x7ff9ac0a29e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff9b800b2e0 tx=0x7ff9b8005fb0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.257 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.240+0000 7ff9c934b700 1 -- 172.21.15.77:0/2837025127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9c4101810 msgr2=0x7ff9c4108f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:22.257 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.240+0000 7ff9c934b700 1 --2- 172.21.15.77:0/2837025127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9c4101810 0x7ff9c4108f80 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc00b970 tx=0x7ff9bc00bc80 comp rx=0 tx=0).stop 2023-12-01T01:18:22.257 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.240+0000 7ff9c934b700 1 -- 172.21.15.77:0/2837025127 shutdown_connections 2023-12-01T01:18:22.257 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.240+0000 7ff9c934b700 1 --2- 172.21.15.77:0/2837025127 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff9ac0a0520 0x7ff9ac0a29e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.257 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.240+0000 7ff9c934b700 1 --2- 172.21.15.77:0/2837025127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9c4101810 0x7ff9c4108f80 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.257 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.240+0000 7ff9c934b700 1 --2- 172.21.15.77:0/2837025127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9c40ffc80 0x7ff9c410a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.257 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.240+0000 7ff9c934b700 1 --2- 172.21.15.77:0/2837025127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9c4100e70 0x7ff9c4108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.257 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.240+0000 7ff9c934b700 1 -- 172.21.15.77:0/2837025127 >> 172.21.15.77:0/2837025127 conn(0x7ff9c40fb1f0 msgr2=0x7ff9c40fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:22.257 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.241+0000 7ff9c934b700 1 -- 172.21.15.77:0/2837025127 shutdown_connections 2023-12-01T01:18:22.257 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.241+0000 7ff9c934b700 1 -- 172.21.15.77:0/2837025127 wait complete. 2023-12-01T01:18:22.261 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-12-01T01:18:22.261 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:18:22.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.393+0000 7f89edb54700 1 -- 172.21.15.77:0/2875764076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89e8100e60 msgr2=0x7f89e81012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:22.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.393+0000 7f89edb54700 1 --2- 172.21.15.77:0/2875764076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89e8100e60 0x7f89e81012c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f89dc009990 tx=0x7f89dc009ca0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.393+0000 7f89edb54700 1 -- 172.21.15.77:0/2875764076 shutdown_connections 2023-12-01T01:18:22.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.393+0000 7f89edb54700 1 --2- 172.21.15.77:0/2875764076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89e8101800 0x7f89e81080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.393+0000 7f89edb54700 1 --2- 172.21.15.77:0/2875764076 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89e80ffc70 0x7f89e8100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.393+0000 7f89edb54700 1 --2- 172.21.15.77:0/2875764076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89e8100e60 0x7f89e81012c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.393+0000 7f89edb54700 1 -- 172.21.15.77:0/2875764076 >> 172.21.15.77:0/2875764076 conn(0x7f89e80fb1c0 msgr2=0x7f89e80fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:22.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.393+0000 7f89edb54700 1 -- 172.21.15.77:0/2875764076 shutdown_connections 2023-12-01T01:18:22.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.393+0000 7f89edb54700 1 -- 172.21.15.77:0/2875764076 wait complete. 2023-12-01T01:18:22.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.394+0000 7f89edb54700 1 Processor -- start 2023-12-01T01:18:22.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.394+0000 7f89edb54700 1 -- start start 2023-12-01T01:18:22.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.394+0000 7f89edb54700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89e80ffc70 0x7f89e810a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:22.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.394+0000 7f89edb54700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89e8100e60 0x7f89e8108a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:22.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.394+0000 7f89edb54700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89e8101800 0x7f89e8108f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:22.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.394+0000 7f89edb54700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e8109710 con 0x7f89e8101800 2023-12-01T01:18:22.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.394+0000 7f89edb54700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e8109880 con 0x7f89e8100e60 2023-12-01T01:18:22.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.394+0000 7f89edb54700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e81a09c0 con 0x7f89e80ffc70 2023-12-01T01:18:22.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.394+0000 7f89e7fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89e8101800 0x7f89e8108f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:22.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.394+0000 7f89e7fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89e8101800 0x7f89e8108f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59472/0 (socket says 172.21.15.77:59472) 2023-12-01T01:18:22.398 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.394+0000 7f89e7fff700 1 -- 172.21.15.77:0/1915570610 learned_addr learned my addr 172.21.15.77:0/1915570610 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:22.398 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.394+0000 7f89e77fe700 1 --2- 172.21.15.77:0/1915570610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89e80ffc70 0x7f89e810a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:22.398 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.395+0000 7f89e7fff700 1 -- 172.21.15.77:0/1915570610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89e80ffc70 msgr2=0x7f89e810a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:22.398 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.395+0000 7f89e7fff700 1 --2- 172.21.15.77:0/1915570610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89e80ffc70 0x7f89e810a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.398 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.395+0000 7f89e7fff700 1 -- 172.21.15.77:0/1915570610 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89e8100e60 msgr2=0x7f89e8108a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:22.398 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.395+0000 7f89e7fff700 1 --2- 172.21.15.77:0/1915570610 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89e8100e60 0x7f89e8108a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.398 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.395+0000 7f89e7fff700 1 -- 172.21.15.77:0/1915570610 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89dc0095f0 con 0x7f89e8101800 2023-12-01T01:18:22.398 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.395+0000 7f89e7fff700 1 --2- 172.21.15.77:0/1915570610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89e8101800 0x7f89e8108f70 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f89e000ef30 tx=0x7f89e000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:22.398 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.395+0000 7f89e4ff9700 1 -- 172.21.15.77:0/1915570610 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e0018070 con 0x7f89e8101800 2023-12-01T01:18:22.398 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.395+0000 7f89edb54700 1 -- 172.21.15.77:0/1915570610 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e81a0c40 con 0x7f89e8101800 2023-12-01T01:18:22.398 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.395+0000 7f89edb54700 1 -- 172.21.15.77:0/1915570610 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e81a1150 con 0x7f89e8101800 2023-12-01T01:18:22.399 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.398+0000 7f89e4ff9700 1 -- 172.21.15.77:0/1915570610 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89e000f040 con 0x7f89e8101800 2023-12-01T01:18:22.399 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.398+0000 7f89e4ff9700 1 -- 172.21.15.77:0/1915570610 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e000cd60 con 0x7f89e8101800 2023-12-01T01:18:22.399 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.398+0000 7f89e4ff9700 1 -- 172.21.15.77:0/1915570610 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f89e0004020 con 0x7f89e8101800 2023-12-01T01:18:22.400 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.399+0000 7f89edb54700 1 -- 172.21.15.77:0/1915570610 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89e810c5e0 con 0x7f89e8101800 2023-12-01T01:18:22.400 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.399+0000 7f89e4ff9700 1 --2- 172.21.15.77:0/1915570610 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f89d00a06f0 0x7f89d00a2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:22.400 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.399+0000 7f89e4ff9700 1 -- 172.21.15.77:0/1915570610 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f89e00cc360 con 0x7f89e8101800 2023-12-01T01:18:22.400 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.400+0000 7f89e77fe700 1 --2- 172.21.15.77:0/1915570610 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f89d00a06f0 0x7f89d00a2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:22.401 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.400+0000 7f89e77fe700 1 --2- 172.21.15.77:0/1915570610 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f89d00a06f0 0x7f89d00a2bb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f89e8100cc0 tx=0x7f89d800a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:22.404 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.404+0000 7f89e4ff9700 1 -- 172.21.15.77:0/1915570610 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f89e0096010 con 0x7f89e8101800 2023-12-01T01:18:22.669 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.668+0000 7f89edb54700 1 -- 172.21.15.77:0/1915570610 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89e8106ac0 con 0x7f89e8101800 2023-12-01T01:18:22.670 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.669+0000 7f89e4ff9700 1 -- 172.21.15.77:0/1915570610 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10838 (secure 0 0 0) 0x7f89e0099630 con 0x7f89e8101800 2023-12-01T01:18:22.670 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:18:22.670 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1435145620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1435145620},{"type":"v1","addr":"172.21.15.77:6835","nonce":1435145620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/841838908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":841838908},{"type":"v1","addr":"172.21.15.77:6837","nonce":841838908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-01T01:16:25.345276+0000","modified":"2023-12-01T01:16:32.596617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24496,"mds_2":24475,"mds_3":24473,"mds_4":14649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/3726913446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3726913446},{"type":"v1","addr":"172.21.15.77:6839","nonce":3726913446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14649":{"gid":14649,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":5,"addr":"172.21.15.77:6841/3023184965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3023184965},{"type":"v1","addr":"172.21.15.77:6841","nonce":3023184965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/2249638445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2249638445},{"type":"v1","addr":"172.21.15.137:6833","nonce":2249638445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6833/1201599774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1201599774},{"type":"v1","addr":"172.21.15.203:6833","nonce":1201599774}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3935354878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3935354878},{"type":"v1","addr":"172.21.15.137:6835","nonce":3935354878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3237538176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3237538176},{"type":"v1","addr":"172.21.15.203:6835","nonce":3237538176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3518881867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3518881867},{"type":"v1","addr":"172.21.15.137:6837","nonce":3518881867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.203:6837/2887080977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2887080977},{"type":"v1","addr":"172.21.15.203:6837","nonce":2887080977}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.203:6839/2888221613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2888221613},{"type":"v1","addr":"172.21.15.203:6839","nonce":2888221613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/1099966660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1099966660},{"type":"v1","addr":"172.21.15.137:6839","nonce":1099966660}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-01T01:18:22.673 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.672+0000 7f89edb54700 1 -- 172.21.15.77:0/1915570610 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f89d00a06f0 msgr2=0x7f89d00a2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:22.673 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.672+0000 7f89edb54700 1 --2- 172.21.15.77:0/1915570610 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f89d00a06f0 0x7f89d00a2bb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f89e8100cc0 tx=0x7f89d800a3b0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.673 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.672+0000 7f89edb54700 1 -- 172.21.15.77:0/1915570610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89e8101800 msgr2=0x7f89e8108f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:22.673 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.672+0000 7f89edb54700 1 --2- 172.21.15.77:0/1915570610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89e8101800 0x7f89e8108f70 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f89e000ef30 tx=0x7f89e000c3a0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.674 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.672+0000 7f89edb54700 1 -- 172.21.15.77:0/1915570610 shutdown_connections 2023-12-01T01:18:22.674 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.672+0000 7f89edb54700 1 --2- 172.21.15.77:0/1915570610 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f89d00a06f0 0x7f89d00a2bb0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.674 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.672+0000 7f89edb54700 1 --2- 172.21.15.77:0/1915570610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89e8101800 0x7f89e8108f70 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.674 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.672+0000 7f89edb54700 1 --2- 172.21.15.77:0/1915570610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89e80ffc70 0x7f89e810a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.674 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.672+0000 7f89edb54700 1 --2- 172.21.15.77:0/1915570610 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89e8100e60 0x7f89e8108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.674 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.672+0000 7f89edb54700 1 -- 172.21.15.77:0/1915570610 >> 172.21.15.77:0/1915570610 conn(0x7f89e80fb1c0 msgr2=0x7f89e80fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:22.674 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.672+0000 7f89edb54700 1 -- 172.21.15.77:0/1915570610 shutdown_connections 2023-12-01T01:18:22.674 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.672+0000 7f89edb54700 1 -- 172.21.15.77:0/1915570610 wait complete. 2023-12-01T01:18:22.674 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 27 2023-12-01T01:18:22.690 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-12-01T01:18:22.828 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.826+0000 7f1e22da3700 1 -- 172.21.15.77:0/1752245185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e1c106ad0 msgr2=0x7f1e1c108ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:22.828 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.826+0000 7f1e22da3700 1 --2- 172.21.15.77:0/1752245185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e1c106ad0 0x7f1e1c108ee0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f1e1400b0a0 tx=0x7f1e1400b3b0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.828 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.826+0000 7f1e22da3700 1 -- 172.21.15.77:0/1752245185 shutdown_connections 2023-12-01T01:18:22.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.827+0000 7f1e22da3700 1 --2- 172.21.15.77:0/1752245185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e1c106ad0 0x7f1e1c108ee0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.827+0000 7f1e22da3700 1 --2- 172.21.15.77:0/1752245185 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e1c101870 0x7f1e1c103c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.827+0000 7f1e22da3700 1 --2- 172.21.15.77:0/1752245185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e1c1041a0 0x7f1e1c106590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.827+0000 7f1e22da3700 1 -- 172.21.15.77:0/1752245185 >> 172.21.15.77:0/1752245185 conn(0x7f1e1c0fb220 msgr2=0x7f1e1c0fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:22.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.827+0000 7f1e22da3700 1 -- 172.21.15.77:0/1752245185 shutdown_connections 2023-12-01T01:18:22.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.827+0000 7f1e22da3700 1 -- 172.21.15.77:0/1752245185 wait complete. 2023-12-01T01:18:22.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.827+0000 7f1e22da3700 1 Processor -- start 2023-12-01T01:18:22.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.827+0000 7f1e22da3700 1 -- start start 2023-12-01T01:18:22.831 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.828+0000 7f1e22da3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e1c101870 0x7f1e1c196b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:22.831 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.828+0000 7f1e22da3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e1c1041a0 0x7f1e1c197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:22.831 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.828+0000 7f1e22da3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e1c106ad0 0x7f1e1c19e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:22.831 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.828+0000 7f1e22da3700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c19e690 con 0x7f1e1c106ad0 2023-12-01T01:18:22.831 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.828+0000 7f1e22da3700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c19e7d0 con 0x7f1e1c101870 2023-12-01T01:18:22.831 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.828+0000 7f1e22da3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c19e960 con 0x7f1e1c1041a0 2023-12-01T01:18:22.831 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.828+0000 7f1e21340700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e1c106ad0 0x7f1e1c19e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:22.831 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.828+0000 7f1e21340700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e1c106ad0 0x7f1e1c19e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59486/0 (socket says 172.21.15.77:59486) 2023-12-01T01:18:22.831 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.828+0000 7f1e21340700 1 -- 172.21.15.77:0/569001691 learned_addr learned my addr 172.21.15.77:0/569001691 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:22.832 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.828+0000 7f1e20b3f700 1 --2- 172.21.15.77:0/569001691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e1c101870 0x7f1e1c196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:22.832 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.828+0000 7f1e1bfff700 1 --2- 172.21.15.77:0/569001691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e1c1041a0 0x7f1e1c197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:22.832 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.828+0000 7f1e21340700 1 -- 172.21.15.77:0/569001691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e1c1041a0 msgr2=0x7f1e1c197050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:22.832 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.828+0000 7f1e21340700 1 --2- 172.21.15.77:0/569001691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e1c1041a0 0x7f1e1c197050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.832 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.828+0000 7f1e21340700 1 -- 172.21.15.77:0/569001691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e1c101870 msgr2=0x7f1e1c196b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:22.832 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.828+0000 7f1e21340700 1 --2- 172.21.15.77:0/569001691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e1c101870 0x7f1e1c196b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.832 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.828+0000 7f1e21340700 1 -- 172.21.15.77:0/569001691 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e14009d00 con 0x7f1e1c106ad0 2023-12-01T01:18:22.832 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.828+0000 7f1e21340700 1 --2- 172.21.15.77:0/569001691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e1c106ad0 0x7f1e1c19e0c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f1e140062a0 tx=0x7f1e14004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:22.832 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.828+0000 7f1e19ffb700 1 -- 172.21.15.77:0/569001691 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1400e040 con 0x7f1e1c106ad0 2023-12-01T01:18:22.832 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.828+0000 7f1e22da3700 1 -- 172.21.15.77:0/569001691 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e1c19ebe0 con 0x7f1e1c106ad0 2023-12-01T01:18:22.832 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.828+0000 7f1e19ffb700 1 -- 172.21.15.77:0/569001691 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e1400cec0 con 0x7f1e1c106ad0 2023-12-01T01:18:22.832 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.828+0000 7f1e19ffb700 1 -- 172.21.15.77:0/569001691 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e14004020 con 0x7f1e1c106ad0 2023-12-01T01:18:22.832 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.829+0000 7f1e22da3700 1 -- 172.21.15.77:0/569001691 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e1c19f0b0 con 0x7f1e1c106ad0 2023-12-01T01:18:22.833 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.831+0000 7f1e19ffb700 1 -- 172.21.15.77:0/569001691 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f1e1401d690 con 0x7f1e1c106ad0 2023-12-01T01:18:22.833 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.831+0000 7f1e19ffb700 1 --2- 172.21.15.77:0/569001691 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f1e0c0a0520 0x7f1e0c0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:22.833 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.831+0000 7f1e20b3f700 1 --2- 172.21.15.77:0/569001691 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f1e0c0a0520 0x7f1e0c0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:22.833 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.831+0000 7f1e19ffb700 1 -- 172.21.15.77:0/569001691 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f1e140cd1d0 con 0x7f1e1c106ad0 2023-12-01T01:18:22.833 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.832+0000 7f1e20b3f700 1 --2- 172.21.15.77:0/569001691 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f1e0c0a0520 0x7f1e0c0a29e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1e10007c60 tx=0x7f1e100073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:22.836 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.832+0000 7f1e2ef33b80 1 Processor -- start 2023-12-01T01:18:22.836 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.833+0000 7f1e2ef33b80 1 -- start start 2023-12-01T01:18:22.836 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.833+0000 7f1e2ef33b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8fdc410f0 0x55f8fdc41530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:22.836 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.833+0000 7f1e2ef33b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8fdc41a70 0x55f8fdc3fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:22.836 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.833+0000 7f1e2ef33b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8fdc403f0 0x55f8fdc40870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:22.836 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.833+0000 7f1e2ef33b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8fdb97a10 con 0x55f8fdc41a70 2023-12-01T01:18:22.836 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.833+0000 7f1e2ef33b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8fdb953d0 con 0x55f8fdc410f0 2023-12-01T01:18:22.836 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.833+0000 7f1e2ef33b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8fdb90cc0 con 0x55f8fdc403f0 2023-12-01T01:18:22.836 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.833+0000 7f1e1bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8fdc41a70 0x55f8fdc3fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:22.836 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.833+0000 7f1e1bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8fdc41a70 0x55f8fdc3fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59492/0 (socket says 172.21.15.77:59492) 2023-12-01T01:18:22.837 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.833+0000 7f1e1bfff700 1 -- 172.21.15.77:0/3823808911 learned_addr learned my addr 172.21.15.77:0/3823808911 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:22.837 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.833+0000 7f1e20b3f700 1 --2- 172.21.15.77:0/3823808911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8fdc410f0 0x55f8fdc41530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:22.837 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.833+0000 7f1e1bfff700 1 -- 172.21.15.77:0/3823808911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8fdc403f0 msgr2=0x55f8fdc40870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:22.837 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.833+0000 7f1e21340700 1 --2- 172.21.15.77:0/3823808911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8fdc403f0 0x55f8fdc40870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:22.837 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.833+0000 7f1e1bfff700 1 --2- 172.21.15.77:0/3823808911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8fdc403f0 0x55f8fdc40870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.837 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.833+0000 7f1e1bfff700 1 -- 172.21.15.77:0/3823808911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8fdc410f0 msgr2=0x55f8fdc41530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:22.837 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.833+0000 7f1e1bfff700 1 --2- 172.21.15.77:0/3823808911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8fdc410f0 0x55f8fdc41530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.837 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.833+0000 7f1e1bfff700 1 -- 172.21.15.77:0/3823808911 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8fd9bede0 con 0x55f8fdc41a70 2023-12-01T01:18:22.837 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.833+0000 7f1e20b3f700 1 --2- 172.21.15.77:0/3823808911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8fdc410f0 0x55f8fdc41530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:18:22.837 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.833+0000 7f1e1bfff700 1 --2- 172.21.15.77:0/3823808911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8fdc41a70 0x55f8fdc3fe60 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1e0800bcf0 tx=0x7f1e0800f360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b912c8071bcc73b3 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:22.837 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.834+0000 7f1e06ffd700 1 -- 172.21.15.77:0/3823808911 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0800fc90 con 0x55f8fdc41a70 2023-12-01T01:18:22.837 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.834+0000 7f1e06ffd700 1 -- 172.21.15.77:0/3823808911 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e0800fdf0 con 0x55f8fdc41a70 2023-12-01T01:18:22.837 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.834+0000 7f1e06ffd700 1 -- 172.21.15.77:0/3823808911 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e08011760 con 0x55f8fdc41a70 2023-12-01T01:18:22.837 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.834+0000 7f1e2ef33b80 1 -- 172.21.15.77:0/3823808911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8fdc41a70 msgr2=0x55f8fdc3fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:22.837 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.834+0000 7f1e2ef33b80 1 --2- 172.21.15.77:0/3823808911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8fdc41a70 0x55f8fdc3fe60 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1e0800bcf0 tx=0x7f1e0800f360 comp rx=0 tx=0).stop 2023-12-01T01:18:22.838 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.834+0000 7f1e2ef33b80 1 -- 172.21.15.77:0/3823808911 shutdown_connections 2023-12-01T01:18:22.838 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.834+0000 7f1e2ef33b80 1 --2- 172.21.15.77:0/3823808911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8fdc403f0 0x55f8fdc40870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.838 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.834+0000 7f1e2ef33b80 1 --2- 172.21.15.77:0/3823808911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8fdc410f0 0x55f8fdc41530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.838 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.834+0000 7f1e2ef33b80 1 --2- 172.21.15.77:0/3823808911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8fdc41a70 0x55f8fdc3fe60 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.838 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.834+0000 7f1e2ef33b80 1 -- 172.21.15.77:0/3823808911 >> 172.21.15.77:0/3823808911 conn(0x55f8fdb9f0d0 msgr2=0x55f8fdc3a940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:22.838 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.834+0000 7f1e2ef33b80 1 -- 172.21.15.77:0/3823808911 shutdown_connections 2023-12-01T01:18:22.838 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.834+0000 7f1e2ef33b80 1 -- 172.21.15.77:0/3823808911 wait complete. 2023-12-01T01:18:22.838 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.835+0000 7f1e2ef33b80 1 -- start start 2023-12-01T01:18:22.838 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.835+0000 7f1e2ef33b80 1 Processor -- start 2023-12-01T01:18:22.842 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.835+0000 7f1e2ef33b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8fdc403f0 0x55f8fdc49a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:22.842 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.835+0000 7f1e2ef33b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8fdc410f0 0x55f8fdc49f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:22.842 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.835+0000 7f1e2ef33b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8fdc41a70 0x55f8fdc50f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:22.842 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.835+0000 7f1e2ef33b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8fdb2d510 con 0x55f8fdc410f0 2023-12-01T01:18:22.842 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.835+0000 7f1e2ef33b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8fdb2d1b0 con 0x55f8fdc403f0 2023-12-01T01:18:22.842 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.835+0000 7f1e2ef33b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8fda14920 con 0x55f8fdc41a70 2023-12-01T01:18:22.842 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.835+0000 7f1e2ef33b80 10 client.0 _ll_register_callbacks cb 0x55f8fda7aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:18:22.842 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.835+0000 7f1e1bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8fdc410f0 0x55f8fdc49f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:22.842 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.835+0000 7f1e1bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8fdc410f0 0x55f8fdc49f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59498/0 (socket says 172.21.15.77:59498) 2023-12-01T01:18:22.842 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.835+0000 7f1e1bfff700 1 -- 172.21.15.77:0/786440347 learned_addr learned my addr 172.21.15.77:0/786440347 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:22.843 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.835+0000 7f1e20b3f700 1 --2- 172.21.15.77:0/786440347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8fdc403f0 0x55f8fdc49a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:22.843 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.836+0000 7f1e21340700 1 --2- 172.21.15.77:0/786440347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8fdc41a70 0x55f8fdc50f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:22.843 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.836+0000 7f1e1bfff700 1 -- 172.21.15.77:0/786440347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8fdc41a70 msgr2=0x55f8fdc50f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:22.843 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.836+0000 7f1e1bfff700 1 --2- 172.21.15.77:0/786440347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8fdc41a70 0x55f8fdc50f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.843 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.836+0000 7f1e1bfff700 1 -- 172.21.15.77:0/786440347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8fdc403f0 msgr2=0x55f8fdc49a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:22.843 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.836+0000 7f1e1bfff700 1 --2- 172.21.15.77:0/786440347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8fdc403f0 0x55f8fdc49a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.843 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.836+0000 7f1e1bfff700 1 -- 172.21.15.77:0/786440347 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e0800b920 con 0x55f8fdc410f0 2023-12-01T01:18:22.843 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.836+0000 7f1e20b3f700 1 --2- 172.21.15.77:0/786440347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8fdc403f0 0x55f8fdc49a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:18:22.843 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.836+0000 7f1e1bfff700 1 --2- 172.21.15.77:0/786440347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8fdc410f0 0x55f8fdc49f40 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1e0800fcc0 tx=0x7f1e08005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afd118efd236c6e9 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:22.843 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.836+0000 7f1dfbfff700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:18:22.843 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.836+0000 7f1dfbfff700 1 -- 172.21.15.77:0/786440347 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0801a8f0 con 0x55f8fdc410f0 2023-12-01T01:18:22.843 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.836+0000 7f1dfbfff700 1 -- 172.21.15.77:0/786440347 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f1e08005040 con 0x55f8fdc410f0 2023-12-01T01:18:22.843 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.836+0000 7f1e2ef33b80 1 -- 172.21.15.77:0/786440347 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f8fdbbbd10 con 0x55f8fdc410f0 2023-12-01T01:18:22.843 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.836+0000 7f1dfbfff700 1 -- 172.21.15.77:0/786440347 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e0801aa50 con 0x55f8fdc410f0 2023-12-01T01:18:22.843 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.836+0000 7f1e2ef33b80 10 client.15552 fetch_fsmap learned FSMap version 27 2023-12-01T01:18:22.844 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.836+0000 7f1e2ef33b80 1 -- 172.21.15.77:0/786440347 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f8fd9a7c60 con 0x55f8fdc410f0 2023-12-01T01:18:22.844 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.836+0000 7f1dfbfff700 1 -- 172.21.15.77:0/786440347 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f8fdbbbd10 con 0x55f8fdc410f0 2023-12-01T01:18:22.844 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.836+0000 7f1dfbfff700 1 -- 172.21.15.77:0/786440347 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f1e08023030 con 0x55f8fdc410f0 2023-12-01T01:18:22.844 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.837+0000 7f1e2ef33b80 10 client.15552 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:18:22.844 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.837+0000 7f1e2ef33b80 10 client.15552 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2023-12-01T01:18:22.844 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.837+0000 7f1e2ef33b80 20 client.15552 populate_metadata read hostname 'smithi077' 2023-12-01T01:18:22.844 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.837+0000 7f1e2ef33b80 1 --2- 172.21.15.77:0/786440347 >> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] conn(0x55f8fdc55650 0x55f8fdc57a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:22.844 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.837+0000 7f1e2ef33b80 4 client.15552 mds_command: new command op to 14649 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:18:22.844 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.837+0000 7f1e2ef33b80 1 -- 172.21.15.77:0/786440347 --> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f8fdc0ee10 con 0x55f8fdc55650 2023-12-01T01:18:22.844 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.837+0000 7f1e20b3f700 1 --2- 172.21.15.77:0/786440347 >> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] conn(0x55f8fdc55650 0x55f8fdc57a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:22.844 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.837+0000 7f1dfbfff700 10 client.15552 ms_handle_connect on v2:172.21.15.77:6840/3023184965 2023-12-01T01:18:22.844 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.837+0000 7f1e20b3f700 1 --2- 172.21.15.77:0/786440347 >> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] conn(0x55f8fdc55650 0x55f8fdc57a70 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=302a7ce72a33047a server_cookie=7463b8472c215927 in_seq=0 out_seq=0 2023-12-01T01:18:22.844 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.837+0000 7f1dfbfff700 1 -- 172.21.15.77:0/786440347 <== mds.4 v2:172.21.15.77:6840/3023184965 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55f8fdc0ee10 con 0x55f8fdc55650 2023-12-01T01:18:22.844 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.837+0000 7f1dfbfff700 10 client.15552 handle_command_reply: tid=0 2023-12-01T01:18:22.845 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.838+0000 7f1e2ef33b80 1 client.15552 shutdown 2023-12-01T01:18:22.845 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.838+0000 7f1e2ef33b80 1 -- 172.21.15.77:0/786440347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8fdc410f0 msgr2=0x55f8fdc49f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:22.845 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.838+0000 7f1e2ef33b80 1 --2- 172.21.15.77:0/786440347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8fdc410f0 0x55f8fdc49f40 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1e0800fcc0 tx=0x7f1e08005d50 comp rx=0 tx=0).stop 2023-12-01T01:18:22.845 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.838+0000 7f1e2ef33b80 1 -- 172.21.15.77:0/786440347 shutdown_connections 2023-12-01T01:18:22.845 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.838+0000 7f1e2ef33b80 1 --2- 172.21.15.77:0/786440347 >> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] conn(0x55f8fdc55650 0x55f8fdc57a70 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.845 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.838+0000 7f1e2ef33b80 1 --2- 172.21.15.77:0/786440347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8fdc41a70 0x55f8fdc50f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.845 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.838+0000 7f1dfbfff700 0 client.15552 ms_handle_reset on v2:172.21.15.77:6840/3023184965 2023-12-01T01:18:22.845 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.838+0000 7f1e2ef33b80 1 --2- 172.21.15.77:0/786440347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8fdc403f0 0x55f8fdc49a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.845 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.838+0000 7f1e2ef33b80 1 --2- 172.21.15.77:0/786440347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8fdc410f0 0x55f8fdc49f40 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.845 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.838+0000 7f1e2ef33b80 1 -- 172.21.15.77:0/786440347 >> 172.21.15.77:0/786440347 conn(0x55f8fdb9f0d0 msgr2=0x55f8fdc3a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:22.845 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.838+0000 7f1e2ef33b80 1 -- 172.21.15.77:0/786440347 shutdown_connections 2023-12-01T01:18:22.845 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.838+0000 7f1e2ef33b80 1 -- 172.21.15.77:0/786440347 wait complete. 2023-12-01T01:18:22.845 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.838+0000 7f1e2ef33b80 20 client.15552 trim_cache size 0 max 16384 2023-12-01T01:18:22.854 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.850+0000 7f1e2ef33b80 1 Processor -- start 2023-12-01T01:18:22.854 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.850+0000 7f1e2ef33b80 1 -- start start 2023-12-01T01:18:22.854 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.851+0000 7f1e2ef33b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8fdc403f0 0x55f8fdc3fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:22.854 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.851+0000 7f1e2ef33b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8fdc410f0 0x55f8fdc4ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:22.854 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.851+0000 7f1e2ef33b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8fdc41a70 0x55f8fdc499c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:22.854 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.851+0000 7f1e2ef33b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8fdc3b280 con 0x55f8fdc41a70 2023-12-01T01:18:22.854 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.851+0000 7f1e2ef33b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8fdc35660 con 0x55f8fdc403f0 2023-12-01T01:18:22.854 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.851+0000 7f1e2ef33b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8fdc3ae60 con 0x55f8fdc410f0 2023-12-01T01:18:22.855 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.851+0000 7f1e21340700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8fdc41a70 0x55f8fdc499c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:22.855 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.851+0000 7f1e21340700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8fdc41a70 0x55f8fdc499c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59508/0 (socket says 172.21.15.77:59508) 2023-12-01T01:18:22.855 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.851+0000 7f1e21340700 1 -- 172.21.15.77:0/2999651455 learned_addr learned my addr 172.21.15.77:0/2999651455 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:22.855 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.851+0000 7f1e20b3f700 1 --2- 172.21.15.77:0/2999651455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8fdc403f0 0x55f8fdc3fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:22.855 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.851+0000 7f1e1bfff700 1 --2- 172.21.15.77:0/2999651455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8fdc410f0 0x55f8fdc4ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:22.855 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.851+0000 7f1e21340700 1 -- 172.21.15.77:0/2999651455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8fdc410f0 msgr2=0x55f8fdc4ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:22.855 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.851+0000 7f1e21340700 1 --2- 172.21.15.77:0/2999651455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8fdc410f0 0x55f8fdc4ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.855 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.851+0000 7f1e21340700 1 -- 172.21.15.77:0/2999651455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8fdc403f0 msgr2=0x55f8fdc3fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:22.855 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.851+0000 7f1e21340700 1 --2- 172.21.15.77:0/2999651455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8fdc403f0 0x55f8fdc3fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.855 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.851+0000 7f1e21340700 1 -- 172.21.15.77:0/2999651455 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8fdc48980 con 0x55f8fdc41a70 2023-12-01T01:18:22.855 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.851+0000 7f1e21340700 1 --2- 172.21.15.77:0/2999651455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8fdc41a70 0x55f8fdc499c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1e140a2570 tx=0x7f1e14009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82415ad898635d3c server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:22.855 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.852+0000 7f1e077fe700 1 -- 172.21.15.77:0/2999651455 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e140cc040 con 0x55f8fdc41a70 2023-12-01T01:18:22.855 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.852+0000 7f1e077fe700 1 -- 172.21.15.77:0/2999651455 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e1409a420 con 0x55f8fdc41a70 2023-12-01T01:18:22.855 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.852+0000 7f1e077fe700 1 -- 172.21.15.77:0/2999651455 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e140a2b30 con 0x55f8fdc41a70 2023-12-01T01:18:22.855 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.852+0000 7f1e2ef33b80 1 -- 172.21.15.77:0/2999651455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8fdc41a70 msgr2=0x55f8fdc499c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:22.856 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.852+0000 7f1e2ef33b80 1 --2- 172.21.15.77:0/2999651455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8fdc41a70 0x55f8fdc499c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1e140a2570 tx=0x7f1e14009040 comp rx=0 tx=0).stop 2023-12-01T01:18:22.856 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.852+0000 7f1e2ef33b80 1 -- 172.21.15.77:0/2999651455 shutdown_connections 2023-12-01T01:18:22.856 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.852+0000 7f1e2ef33b80 1 --2- 172.21.15.77:0/2999651455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8fdc41a70 0x55f8fdc499c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.856 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.852+0000 7f1e2ef33b80 1 --2- 172.21.15.77:0/2999651455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8fdc403f0 0x55f8fdc3fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.856 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.852+0000 7f1e2ef33b80 1 --2- 172.21.15.77:0/2999651455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8fdc410f0 0x55f8fdc4ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.856 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.852+0000 7f1e2ef33b80 1 -- 172.21.15.77:0/2999651455 >> 172.21.15.77:0/2999651455 conn(0x55f8fdb9f0d0 msgr2=0x55f8fdc44350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:22.856 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.852+0000 7f1e2ef33b80 1 -- 172.21.15.77:0/2999651455 shutdown_connections 2023-12-01T01:18:22.856 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.852+0000 7f1e2ef33b80 1 -- 172.21.15.77:0/2999651455 wait complete. 2023-12-01T01:18:22.856 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.853+0000 7f1e2ef33b80 1 -- start start 2023-12-01T01:18:22.856 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.853+0000 7f1e2ef33b80 1 Processor -- start 2023-12-01T01:18:22.860 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.853+0000 7f1e2ef33b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8fdc403f0 0x55f8fdc4e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:22.860 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.853+0000 7f1e2ef33b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8fdc410f0 0x55f8fdc4eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:22.860 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.853+0000 7f1e2ef33b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8fdc41a70 0x55f8fdc4d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:22.860 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.853+0000 7f1e2ef33b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8fdc4d780 con 0x55f8fdc403f0 2023-12-01T01:18:22.860 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.853+0000 7f1e2ef33b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8fdc4d8f0 con 0x55f8fdc41a70 2023-12-01T01:18:22.860 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.853+0000 7f1e2ef33b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8fdc4dab0 con 0x55f8fdc410f0 2023-12-01T01:18:22.860 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.853+0000 7f1e2ef33b80 10 client.0 _ll_register_callbacks cb 0x7f1e0800bb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:18:22.860 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.853+0000 7f1e20b3f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8fdc403f0 0x55f8fdc4e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:22.860 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.853+0000 7f1e20b3f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8fdc403f0 0x55f8fdc4e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59510/0 (socket says 172.21.15.77:59510) 2023-12-01T01:18:22.860 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.853+0000 7f1e20b3f700 1 -- 172.21.15.77:0/2374373725 learned_addr learned my addr 172.21.15.77:0/2374373725 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:22.860 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.853+0000 7f1e1bfff700 1 --2- 172.21.15.77:0/2374373725 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8fdc410f0 0x55f8fdc4eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:22.860 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.854+0000 7f1e21340700 1 --2- 172.21.15.77:0/2374373725 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8fdc41a70 0x55f8fdc4d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:22.861 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.854+0000 7f1e20b3f700 1 -- 172.21.15.77:0/2374373725 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8fdc410f0 msgr2=0x55f8fdc4eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:22.861 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.854+0000 7f1e20b3f700 1 --2- 172.21.15.77:0/2374373725 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8fdc410f0 0x55f8fdc4eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.861 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.854+0000 7f1e20b3f700 1 -- 172.21.15.77:0/2374373725 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8fdc41a70 msgr2=0x55f8fdc4d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:22.861 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.854+0000 7f1e20b3f700 1 --2- 172.21.15.77:0/2374373725 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8fdc41a70 0x55f8fdc4d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.861 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.854+0000 7f1e20b3f700 1 -- 172.21.15.77:0/2374373725 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e1c19f0b0 con 0x55f8fdc403f0 2023-12-01T01:18:22.861 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.854+0000 7f1e21340700 1 --2- 172.21.15.77:0/2374373725 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8fdc41a70 0x55f8fdc4d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:18:22.861 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.854+0000 7f1e20b3f700 1 --2- 172.21.15.77:0/2374373725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8fdc403f0 0x55f8fdc4e990 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x55f8fdc58340 tx=0x7f1e10013070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9ccae43925a497f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:22.861 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.854+0000 7f1dfbfff700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:18:22.861 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.854+0000 7f1dfbfff700 1 -- 172.21.15.77:0/2374373725 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e10026040 con 0x55f8fdc403f0 2023-12-01T01:18:22.861 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.854+0000 7f1dfbfff700 1 -- 172.21.15.77:0/2374373725 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f1e10017070 con 0x55f8fdc403f0 2023-12-01T01:18:22.861 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.854+0000 7f1e2ef33b80 1 -- 172.21.15.77:0/2374373725 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1e0800fdf0 con 0x55f8fdc403f0 2023-12-01T01:18:22.861 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.854+0000 7f1dfbfff700 1 -- 172.21.15.77:0/2374373725 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e10011e90 con 0x55f8fdc403f0 2023-12-01T01:18:22.861 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.854+0000 7f1e2ef33b80 10 client.15558 fetch_fsmap learned FSMap version 27 2023-12-01T01:18:22.861 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.854+0000 7f1e2ef33b80 1 -- 172.21.15.77:0/2374373725 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f8fd9e5250 con 0x55f8fdc403f0 2023-12-01T01:18:22.861 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.854+0000 7f1dfbfff700 1 -- 172.21.15.77:0/2374373725 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e0800fdf0 con 0x55f8fdc403f0 2023-12-01T01:18:22.862 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.854+0000 7f1dfbfff700 1 -- 172.21.15.77:0/2374373725 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f1e1002c020 con 0x55f8fdc403f0 2023-12-01T01:18:22.862 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.854+0000 7f1e2ef33b80 10 client.15558 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:18:22.862 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.854+0000 7f1e2ef33b80 10 client.15558 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2023-12-01T01:18:22.862 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.854+0000 7f1e2ef33b80 20 client.15558 populate_metadata read hostname 'smithi077' 2023-12-01T01:18:22.862 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.854+0000 7f1e2ef33b80 1 --2- 172.21.15.77:0/2374373725 >> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] conn(0x55f8fdca85f0 0x55f8fdcaaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:22.862 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.855+0000 7f1e2ef33b80 4 client.15558 mds_command: new command op to 14649 tid=0[{"prefix": "damage ls"}] 2023-12-01T01:18:22.862 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.855+0000 7f1e2ef33b80 1 -- 172.21.15.77:0/2374373725 --> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f8fdb89d40 con 0x55f8fdca85f0 2023-12-01T01:18:22.862 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.855+0000 7f1e1bfff700 1 --2- 172.21.15.77:0/2374373725 >> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] conn(0x55f8fdca85f0 0x55f8fdcaaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:22.862 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.855+0000 7f1dfbfff700 10 client.15558 ms_handle_connect on v2:172.21.15.77:6840/3023184965 2023-12-01T01:18:22.862 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.855+0000 7f1e1bfff700 1 --2- 172.21.15.77:0/2374373725 >> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] conn(0x55f8fdca85f0 0x55f8fdcaaa10 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e75cc5f05a1ec6d4 server_cookie=8cd4e63da75e6247 in_seq=0 out_seq=0 2023-12-01T01:18:22.862 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.855+0000 7f1dfbfff700 1 -- 172.21.15.77:0/2374373725 <== mds.4 v2:172.21.15.77:6840/3023184965 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f8fdb89d40 con 0x55f8fdca85f0 2023-12-01T01:18:22.862 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.855+0000 7f1dfbfff700 10 client.15558 handle_command_reply: tid=0 2023-12-01T01:18:22.862 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.855+0000 7f1e2ef33b80 1 client.15558 shutdown 2023-12-01T01:18:22.862 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.855+0000 7f1e2ef33b80 1 -- 172.21.15.77:0/2374373725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8fdc403f0 msgr2=0x55f8fdc4e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:22.863 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.855+0000 7f1e2ef33b80 1 --2- 172.21.15.77:0/2374373725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8fdc403f0 0x55f8fdc4e990 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x55f8fdc58340 tx=0x7f1e10013070 comp rx=0 tx=0).stop 2023-12-01T01:18:22.863 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.855+0000 7f1e2ef33b80 1 -- 172.21.15.77:0/2374373725 shutdown_connections 2023-12-01T01:18:22.863 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.855+0000 7f1e2ef33b80 1 --2- 172.21.15.77:0/2374373725 >> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] conn(0x55f8fdca85f0 0x55f8fdcaaa10 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.863 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.855+0000 7f1e2ef33b80 1 --2- 172.21.15.77:0/2374373725 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8fdc41a70 0x55f8fdc4d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.863 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.855+0000 7f1e2ef33b80 1 --2- 172.21.15.77:0/2374373725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8fdc403f0 0x55f8fdc4e990 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.863 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.855+0000 7f1e2ef33b80 1 --2- 172.21.15.77:0/2374373725 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8fdc410f0 0x55f8fdc4eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.863 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.855+0000 7f1dfbfff700 0 client.15558 ms_handle_reset on v2:172.21.15.77:6840/3023184965 2023-12-01T01:18:22.863 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.855+0000 7f1e2ef33b80 1 -- 172.21.15.77:0/2374373725 >> 172.21.15.77:0/2374373725 conn(0x55f8fdb9f0d0 msgr2=0x55f8fdc499a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:22.863 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.856+0000 7f1e2ef33b80 1 -- 172.21.15.77:0/2374373725 shutdown_connections 2023-12-01T01:18:22.863 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.856+0000 7f1e2ef33b80 1 -- 172.21.15.77:0/2374373725 wait complete. 2023-12-01T01:18:22.863 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.856+0000 7f1e2ef33b80 20 client.15558 trim_cache size 0 max 16384 2023-12-01T01:18:22.863 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2023-12-01T01:18:22.864 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.856+0000 7f1e22da3700 1 -- 172.21.15.77:0/569001691 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f1e0c0a0520 msgr2=0x7f1e0c0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:22.864 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.856+0000 7f1e22da3700 1 --2- 172.21.15.77:0/569001691 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f1e0c0a0520 0x7f1e0c0a29e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1e10007c60 tx=0x7f1e100073d0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.864 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.856+0000 7f1e22da3700 1 -- 172.21.15.77:0/569001691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e1c106ad0 msgr2=0x7f1e1c19e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:22.864 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.856+0000 7f1e22da3700 1 --2- 172.21.15.77:0/569001691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e1c106ad0 0x7f1e1c19e0c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f1e140062a0 tx=0x7f1e14004d00 comp rx=0 tx=0).stop 2023-12-01T01:18:22.865 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.856+0000 7f1e22da3700 1 -- 172.21.15.77:0/569001691 shutdown_connections 2023-12-01T01:18:22.865 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.856+0000 7f1e22da3700 1 --2- 172.21.15.77:0/569001691 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f1e0c0a0520 0x7f1e0c0a29e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.865 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.856+0000 7f1e22da3700 1 --2- 172.21.15.77:0/569001691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1e1c106ad0 0x7f1e1c19e0c0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.865 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.856+0000 7f1e22da3700 1 --2- 172.21.15.77:0/569001691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e1c101870 0x7f1e1c196b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.865 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.856+0000 7f1e22da3700 1 --2- 172.21.15.77:0/569001691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1e1c1041a0 0x7f1e1c197050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:22.865 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.856+0000 7f1e22da3700 1 -- 172.21.15.77:0/569001691 >> 172.21.15.77:0/569001691 conn(0x7f1e1c0fb220 msgr2=0x7f1e1c1024e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:22.865 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.856+0000 7f1e22da3700 1 -- 172.21.15.77:0/569001691 shutdown_connections 2023-12-01T01:18:22.865 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:22.856+0000 7f1e22da3700 1 -- 172.21.15.77:0/569001691 wait complete. 2023-12-01T01:18:22.869 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-12-01T01:18:23.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.009+0000 7fd528922700 1 -- 172.21.15.77:0/2601415554 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5241041d0 msgr2=0x7fd5241065c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:23.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.009+0000 7fd528922700 1 --2- 172.21.15.77:0/2601415554 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5241041d0 0x7fd5241065c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd518009990 tx=0x7fd518009ca0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.010+0000 7fd528922700 1 -- 172.21.15.77:0/2601415554 shutdown_connections 2023-12-01T01:18:23.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.010+0000 7fd528922700 1 --2- 172.21.15.77:0/2601415554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd524106b00 0x7fd524108f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.010+0000 7fd528922700 1 --2- 172.21.15.77:0/2601415554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5241018a0 0x7fd524103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.010+0000 7fd528922700 1 --2- 172.21.15.77:0/2601415554 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5241041d0 0x7fd5241065c0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.010+0000 7fd528922700 1 -- 172.21.15.77:0/2601415554 >> 172.21.15.77:0/2601415554 conn(0x7fd5240fb210 msgr2=0x7fd5240fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:23.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.010+0000 7fd528922700 1 -- 172.21.15.77:0/2601415554 shutdown_connections 2023-12-01T01:18:23.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.010+0000 7fd528922700 1 -- 172.21.15.77:0/2601415554 wait complete. 2023-12-01T01:18:23.013 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.010+0000 7fd528922700 1 Processor -- start 2023-12-01T01:18:23.014 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.010+0000 7fd528922700 1 -- start start 2023-12-01T01:18:23.014 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.011+0000 7fd528922700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5241018a0 0x7fd524101210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.014 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.011+0000 7fd528922700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5241041d0 0x7fd5240ff850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.014 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.011+0000 7fd528922700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd524106b00 0x7fd5240ffd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.014 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.011+0000 7fd528922700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd524100530 con 0x7fd5241018a0 2023-12-01T01:18:23.014 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.011+0000 7fd528922700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5241006a0 con 0x7fd5241041d0 2023-12-01T01:18:23.014 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.011+0000 7fd528922700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5241a0a90 con 0x7fd524106b00 2023-12-01T01:18:23.014 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.011+0000 7fd52259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5241018a0 0x7fd524101210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.014 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.011+0000 7fd52259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5241018a0 0x7fd524101210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59524/0 (socket says 172.21.15.77:59524) 2023-12-01T01:18:23.015 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.011+0000 7fd52259c700 1 -- 172.21.15.77:0/3289262729 learned_addr learned my addr 172.21.15.77:0/3289262729 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:23.015 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.011+0000 7fd521d9b700 1 --2- 172.21.15.77:0/3289262729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5241041d0 0x7fd5240ff850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.015 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.011+0000 7fd522d9d700 1 --2- 172.21.15.77:0/3289262729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd524106b00 0x7fd5240ffd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.015 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.011+0000 7fd52259c700 1 -- 172.21.15.77:0/3289262729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd524106b00 msgr2=0x7fd5240ffd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:23.015 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.011+0000 7fd52259c700 1 --2- 172.21.15.77:0/3289262729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd524106b00 0x7fd5240ffd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.015 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.011+0000 7fd52259c700 1 -- 172.21.15.77:0/3289262729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5241041d0 msgr2=0x7fd5240ff850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:23.015 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.011+0000 7fd52259c700 1 --2- 172.21.15.77:0/3289262729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5241041d0 0x7fd5240ff850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.015 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.011+0000 7fd52259c700 1 -- 172.21.15.77:0/3289262729 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5180095f0 con 0x7fd5241018a0 2023-12-01T01:18:23.015 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.011+0000 7fd521d9b700 1 --2- 172.21.15.77:0/3289262729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5241041d0 0x7fd5240ff850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:18:23.015 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.011+0000 7fd52259c700 1 --2- 172.21.15.77:0/3289262729 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5241018a0 0x7fd524101210 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd51400d950 tx=0x7fd51400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:23.015 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.011+0000 7fd5137fe700 1 -- 172.21.15.77:0/3289262729 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51400f890 con 0x7fd5241018a0 2023-12-01T01:18:23.015 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.012+0000 7fd5137fe700 1 -- 172.21.15.77:0/3289262729 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd514010460 con 0x7fd5241018a0 2023-12-01T01:18:23.015 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.012+0000 7fd528922700 1 -- 172.21.15.77:0/3289262729 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5241a0d70 con 0x7fd5241018a0 2023-12-01T01:18:23.015 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.012+0000 7fd5137fe700 1 -- 172.21.15.77:0/3289262729 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51400e610 con 0x7fd5241018a0 2023-12-01T01:18:23.016 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.012+0000 7fd528922700 1 -- 172.21.15.77:0/3289262729 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5241a11d0 con 0x7fd5241018a0 2023-12-01T01:18:23.016 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.015+0000 7fd5137fe700 1 -- 172.21.15.77:0/3289262729 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fd5140108d0 con 0x7fd5241018a0 2023-12-01T01:18:23.016 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.015+0000 7fd5137fe700 1 --2- 172.21.15.77:0/3289262729 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd50c0a0520 0x7fd50c0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.017 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.016+0000 7fd521d9b700 1 --2- 172.21.15.77:0/3289262729 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd50c0a0520 0x7fd50c0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.017 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.016+0000 7fd5137fe700 1 -- 172.21.15.77:0/3289262729 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fd5140cb690 con 0x7fd5241018a0 2023-12-01T01:18:23.017 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.016+0000 7fd534ab2b80 1 Processor -- start 2023-12-01T01:18:23.019 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.016+0000 7fd534ab2b80 1 -- start start 2023-12-01T01:18:23.019 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.016+0000 7fd521d9b700 1 --2- 172.21.15.77:0/3289262729 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd50c0a0520 0x7fd50c0a29e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd51800b2e0 tx=0x7fd518005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:23.019 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.016+0000 7fd534ab2b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56553ef220f0 0x56553ef22530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.019 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.016+0000 7fd534ab2b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56553ef22a70 0x56553ef20e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.019 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.016+0000 7fd534ab2b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56553ef213f0 0x56553ef21870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.019 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.016+0000 7fd534ab2b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x56553ee78a10 con 0x56553ef220f0 2023-12-01T01:18:23.019 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.016+0000 7fd534ab2b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56553ee763d0 con 0x56553ef22a70 2023-12-01T01:18:23.019 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.016+0000 7fd534ab2b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56553ee71cc0 con 0x56553ef213f0 2023-12-01T01:18:23.019 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.017+0000 7fd522d9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56553ef213f0 0x56553ef21870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.019 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.017+0000 7fd522d9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56553ef213f0 0x56553ef21870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:45724/0 (socket says 172.21.15.77:45724) 2023-12-01T01:18:23.019 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.017+0000 7fd522d9d700 1 -- 172.21.15.77:0/1013316404 learned_addr learned my addr 172.21.15.77:0/1013316404 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:23.019 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.017+0000 7fd521d9b700 1 --2- 172.21.15.77:0/1013316404 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56553ef22a70 0x56553ef20e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.020 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.017+0000 7fd52259c700 1 --2- 172.21.15.77:0/1013316404 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56553ef220f0 0x56553ef22530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.020 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.017+0000 7fd522d9d700 1 -- 172.21.15.77:0/1013316404 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56553ef22a70 msgr2=0x56553ef20e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:23.020 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.017+0000 7fd522d9d700 1 --2- 172.21.15.77:0/1013316404 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56553ef22a70 0x56553ef20e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.020 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.017+0000 7fd522d9d700 1 -- 172.21.15.77:0/1013316404 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56553ef220f0 msgr2=0x56553ef22530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:23.020 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.017+0000 7fd522d9d700 1 --2- 172.21.15.77:0/1013316404 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56553ef220f0 0x56553ef22530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.020 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.017+0000 7fd522d9d700 1 -- 172.21.15.77:0/1013316404 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56553ec9fde0 con 0x56553ef213f0 2023-12-01T01:18:23.020 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.017+0000 7fd522d9d700 1 --2- 172.21.15.77:0/1013316404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56553ef213f0 0x56553ef21870 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd51c011600 tx=0x7fd51c009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d76f10de1bc37ec server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:23.020 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.017+0000 7fd510ff9700 1 -- 172.21.15.77:0/1013316404 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c00e040 con 0x56553ef213f0 2023-12-01T01:18:23.020 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.018+0000 7fd510ff9700 1 -- 172.21.15.77:0/1013316404 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd51c00f040 con 0x56553ef213f0 2023-12-01T01:18:23.020 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.018+0000 7fd510ff9700 1 -- 172.21.15.77:0/1013316404 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c004790 con 0x56553ef213f0 2023-12-01T01:18:23.023 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.018+0000 7fd534ab2b80 1 -- 172.21.15.77:0/1013316404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56553ef213f0 msgr2=0x56553ef21870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:23.023 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.018+0000 7fd534ab2b80 1 --2- 172.21.15.77:0/1013316404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56553ef213f0 0x56553ef21870 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd51c011600 tx=0x7fd51c009920 comp rx=0 tx=0).stop 2023-12-01T01:18:23.023 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.018+0000 7fd534ab2b80 1 -- 172.21.15.77:0/1013316404 shutdown_connections 2023-12-01T01:18:23.023 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.018+0000 7fd534ab2b80 1 --2- 172.21.15.77:0/1013316404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56553ef213f0 0x56553ef21870 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.024 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.018+0000 7fd534ab2b80 1 --2- 172.21.15.77:0/1013316404 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56553ef220f0 0x56553ef22530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.024 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.018+0000 7fd534ab2b80 1 --2- 172.21.15.77:0/1013316404 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56553ef22a70 0x56553ef20e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.024 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.018+0000 7fd534ab2b80 1 -- 172.21.15.77:0/1013316404 >> 172.21.15.77:0/1013316404 conn(0x56553ee800d0 msgr2=0x56553ef1b940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:23.024 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.019+0000 7fd534ab2b80 1 -- 172.21.15.77:0/1013316404 shutdown_connections 2023-12-01T01:18:23.024 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.019+0000 7fd534ab2b80 1 -- 172.21.15.77:0/1013316404 wait complete. 2023-12-01T01:18:23.024 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.019+0000 7fd534ab2b80 1 -- start start 2023-12-01T01:18:23.024 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.019+0000 7fd534ab2b80 1 Processor -- start 2023-12-01T01:18:23.024 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.019+0000 7fd534ab2b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56553ef213f0 0x56553ef2a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.024 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.019+0000 7fd534ab2b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56553ef220f0 0x56553ef2ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.024 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.019+0000 7fd534ab2b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56553ef22a70 0x56553ef31e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.024 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.019+0000 7fd534ab2b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x56553ee0e510 con 0x56553ef220f0 2023-12-01T01:18:23.024 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.019+0000 7fd534ab2b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56553ee0e1b0 con 0x56553ef22a70 2023-12-01T01:18:23.024 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.019+0000 7fd534ab2b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56553ecf5920 con 0x56553ef213f0 2023-12-01T01:18:23.024 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.019+0000 7fd534ab2b80 10 client.0 _ll_register_callbacks cb 0x56553ed5ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:18:23.025 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.019+0000 7fd521d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56553ef220f0 0x56553ef2ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.025 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.019+0000 7fd521d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56553ef220f0 0x56553ef2ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59540/0 (socket says 172.21.15.77:59540) 2023-12-01T01:18:23.025 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.019+0000 7fd521d9b700 1 -- 172.21.15.77:0/3475216765 learned_addr learned my addr 172.21.15.77:0/3475216765 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:23.025 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.019+0000 7fd52259c700 1 --2- 172.21.15.77:0/3475216765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56553ef213f0 0x56553ef2a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.025 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.020+0000 7fd522d9d700 1 --2- 172.21.15.77:0/3475216765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56553ef22a70 0x56553ef31e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.025 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.020+0000 7fd521d9b700 1 -- 172.21.15.77:0/3475216765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56553ef213f0 msgr2=0x56553ef2a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:23.025 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.020+0000 7fd521d9b700 1 --2- 172.21.15.77:0/3475216765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56553ef213f0 0x56553ef2a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.025 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.020+0000 7fd521d9b700 1 -- 172.21.15.77:0/3475216765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56553ef22a70 msgr2=0x56553ef31e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:23.025 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.020+0000 7fd521d9b700 1 --2- 172.21.15.77:0/3475216765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56553ef22a70 0x56553ef31e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.025 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.020+0000 7fd521d9b700 1 -- 172.21.15.77:0/3475216765 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd51c0112b0 con 0x56553ef220f0 2023-12-01T01:18:23.025 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.020+0000 7fd522d9d700 1 --2- 172.21.15.77:0/3475216765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56553ef22a70 0x56553ef31e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:18:23.025 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.020+0000 7fd521d9b700 1 --2- 172.21.15.77:0/3475216765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56553ef220f0 0x56553ef2ae50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd518009990 tx=0x7fd51801a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1acfdf459f57b964 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:23.025 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.020+0000 7fd509ffb700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:18:23.025 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.020+0000 7fd509ffb700 1 -- 172.21.15.77:0/3475216765 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51802b070 con 0x56553ef220f0 2023-12-01T01:18:23.025 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.020+0000 7fd509ffb700 1 -- 172.21.15.77:0/3475216765 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fd518005250 con 0x56553ef220f0 2023-12-01T01:18:23.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.020+0000 7fd534ab2b80 1 -- 172.21.15.77:0/3475216765 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56553ee9cd10 con 0x56553ef220f0 2023-12-01T01:18:23.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.020+0000 7fd509ffb700 1 -- 172.21.15.77:0/3475216765 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd518005410 con 0x56553ef220f0 2023-12-01T01:18:23.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.020+0000 7fd534ab2b80 10 client.15570 fetch_fsmap learned FSMap version 27 2023-12-01T01:18:23.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.020+0000 7fd534ab2b80 1 -- 172.21.15.77:0/3475216765 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56553ec88c60 con 0x56553ef220f0 2023-12-01T01:18:23.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.021+0000 7fd509ffb700 1 -- 172.21.15.77:0/3475216765 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56553ee9cd10 con 0x56553ef220f0 2023-12-01T01:18:23.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.021+0000 7fd509ffb700 1 -- 172.21.15.77:0/3475216765 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fd518004880 con 0x56553ef220f0 2023-12-01T01:18:23.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.021+0000 7fd534ab2b80 10 client.15570 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:18:23.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.021+0000 7fd534ab2b80 10 client.15570 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2023-12-01T01:18:23.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.021+0000 7fd534ab2b80 20 client.15570 populate_metadata read hostname 'smithi077' 2023-12-01T01:18:23.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.021+0000 7fd534ab2b80 1 --2- 172.21.15.77:0/3475216765 >> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] conn(0x56553ef366c0 0x56553ef38ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.021+0000 7fd534ab2b80 4 client.15570 mds_command: new command op to 24473 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:18:23.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.021+0000 7fd534ab2b80 1 -- 172.21.15.77:0/3475216765 --> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56553eeefe10 con 0x56553ef366c0 2023-12-01T01:18:23.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.021+0000 7fd52259c700 1 --2- 172.21.15.77:0/3475216765 >> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] conn(0x56553ef366c0 0x56553ef38ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.022+0000 7fd52259c700 1 --2- 172.21.15.77:0/3475216765 >> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] conn(0x56553ef366c0 0x56553ef38ae0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1ade40187f0b6dcb server_cookie=6d93275479ab5491 in_seq=0 out_seq=0 2023-12-01T01:18:23.027 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.022+0000 7fd509ffb700 10 client.15570 ms_handle_connect on v2:172.21.15.137:6836/3518881867 2023-12-01T01:18:23.027 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.022+0000 7fd509ffb700 1 -- 172.21.15.77:0/3475216765 <== mds.3 v2:172.21.15.137:6836/3518881867 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56553eeefe10 con 0x56553ef366c0 2023-12-01T01:18:23.027 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.022+0000 7fd509ffb700 10 client.15570 handle_command_reply: tid=0 2023-12-01T01:18:23.027 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.022+0000 7fd534ab2b80 1 client.15570 shutdown 2023-12-01T01:18:23.027 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.022+0000 7fd534ab2b80 1 -- 172.21.15.77:0/3475216765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56553ef220f0 msgr2=0x56553ef2ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:23.027 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.022+0000 7fd534ab2b80 1 --2- 172.21.15.77:0/3475216765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56553ef220f0 0x56553ef2ae50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd518009990 tx=0x7fd51801a050 comp rx=0 tx=0).stop 2023-12-01T01:18:23.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.023+0000 7fd534ab2b80 1 -- 172.21.15.77:0/3475216765 shutdown_connections 2023-12-01T01:18:23.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.023+0000 7fd534ab2b80 1 --2- 172.21.15.77:0/3475216765 >> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] conn(0x56553ef366c0 0x56553ef38ae0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.023+0000 7fd534ab2b80 1 --2- 172.21.15.77:0/3475216765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56553ef22a70 0x56553ef31e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.023+0000 7fd509ffb700 0 client.15570 ms_handle_reset on v2:172.21.15.137:6836/3518881867 2023-12-01T01:18:23.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.023+0000 7fd534ab2b80 1 --2- 172.21.15.77:0/3475216765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56553ef213f0 0x56553ef2a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.023+0000 7fd534ab2b80 1 --2- 172.21.15.77:0/3475216765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56553ef220f0 0x56553ef2ae50 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.023+0000 7fd534ab2b80 1 -- 172.21.15.77:0/3475216765 >> 172.21.15.77:0/3475216765 conn(0x56553ee800d0 msgr2=0x56553ef1b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:23.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.023+0000 7fd534ab2b80 1 -- 172.21.15.77:0/3475216765 shutdown_connections 2023-12-01T01:18:23.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.023+0000 7fd534ab2b80 1 -- 172.21.15.77:0/3475216765 wait complete. 2023-12-01T01:18:23.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.023+0000 7fd534ab2b80 20 client.15570 trim_cache size 0 max 16384 2023-12-01T01:18:23.047 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.045+0000 7fd534ab2b80 1 Processor -- start 2023-12-01T01:18:23.047 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.045+0000 7fd534ab2b80 1 -- start start 2023-12-01T01:18:23.048 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.045+0000 7fd534ab2b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56553ef213f0 0x56553ef2fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.048 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.045+0000 7fd534ab2b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56553ef220f0 0x56553ef25370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.048 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.045+0000 7fd534ab2b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56553ef22a70 0x56553ef258b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.048 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.045+0000 7fd534ab2b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x56553ee9cd10 con 0x56553ef213f0 2023-12-01T01:18:23.048 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.045+0000 7fd534ab2b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56553ef2bb00 con 0x56553ef220f0 2023-12-01T01:18:23.048 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.045+0000 7fd534ab2b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56553ef1c830 con 0x56553ef22a70 2023-12-01T01:18:23.048 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.045+0000 7fd52259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56553ef213f0 0x56553ef2fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.048 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.045+0000 7fd52259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56553ef213f0 0x56553ef2fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59546/0 (socket says 172.21.15.77:59546) 2023-12-01T01:18:23.048 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.045+0000 7fd52259c700 1 -- 172.21.15.77:0/2645638050 learned_addr learned my addr 172.21.15.77:0/2645638050 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:23.048 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.046+0000 7fd522d9d700 1 --2- 172.21.15.77:0/2645638050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56553ef22a70 0x56553ef258b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.048 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.046+0000 7fd52259c700 1 -- 172.21.15.77:0/2645638050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56553ef22a70 msgr2=0x56553ef258b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:23.048 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.046+0000 7fd52259c700 1 --2- 172.21.15.77:0/2645638050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56553ef22a70 0x56553ef258b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.048 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.046+0000 7fd52259c700 1 -- 172.21.15.77:0/2645638050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56553ef220f0 msgr2=0x56553ef25370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-01T01:18:23.048 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.046+0000 7fd52259c700 1 --2- 172.21.15.77:0/2645638050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56553ef220f0 0x56553ef25370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.049 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.046+0000 7fd52259c700 1 -- 172.21.15.77:0/2645638050 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56553ecc6250 con 0x56553ef213f0 2023-12-01T01:18:23.049 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.046+0000 7fd52259c700 1 --2- 172.21.15.77:0/2645638050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56553ef213f0 0x56553ef2fe40 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd514098c90 tx=0x7fd514005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75ee0962e59f692 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:23.049 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.046+0000 7fd5117fa700 1 -- 172.21.15.77:0/2645638050 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5140a1000 con 0x56553ef213f0 2023-12-01T01:18:23.049 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.046+0000 7fd5117fa700 1 -- 172.21.15.77:0/2645638050 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5140970d0 con 0x56553ef213f0 2023-12-01T01:18:23.049 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.046+0000 7fd5117fa700 1 -- 172.21.15.77:0/2645638050 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5140986e0 con 0x56553ef213f0 2023-12-01T01:18:23.052 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.047+0000 7fd534ab2b80 1 -- 172.21.15.77:0/2645638050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56553ef213f0 msgr2=0x56553ef2fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:23.053 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.047+0000 7fd534ab2b80 1 --2- 172.21.15.77:0/2645638050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56553ef213f0 0x56553ef2fe40 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd514098c90 tx=0x7fd514005910 comp rx=0 tx=0).stop 2023-12-01T01:18:23.053 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.047+0000 7fd534ab2b80 1 -- 172.21.15.77:0/2645638050 shutdown_connections 2023-12-01T01:18:23.053 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.047+0000 7fd534ab2b80 1 --2- 172.21.15.77:0/2645638050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56553ef22a70 0x56553ef258b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.053 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.047+0000 7fd534ab2b80 1 --2- 172.21.15.77:0/2645638050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56553ef213f0 0x56553ef2fe40 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.053 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.047+0000 7fd534ab2b80 1 --2- 172.21.15.77:0/2645638050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56553ef220f0 0x56553ef25370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.053 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.047+0000 7fd534ab2b80 1 -- 172.21.15.77:0/2645638050 >> 172.21.15.77:0/2645638050 conn(0x56553ee800d0 msgr2=0x56553ef29640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:23.053 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.047+0000 7fd534ab2b80 1 -- 172.21.15.77:0/2645638050 shutdown_connections 2023-12-01T01:18:23.053 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.047+0000 7fd534ab2b80 1 -- 172.21.15.77:0/2645638050 wait complete. 2023-12-01T01:18:23.053 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.047+0000 7fd534ab2b80 1 -- start start 2023-12-01T01:18:23.053 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.047+0000 7fd534ab2b80 1 Processor -- start 2023-12-01T01:18:23.053 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.048+0000 7fd534ab2b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56553ef213f0 0x56553ef2f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.053 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.048+0000 7fd534ab2b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56553ef220f0 0x56553ef2fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.053 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.048+0000 7fd534ab2b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56553ef22a70 0x56553ef2dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.054 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.048+0000 7fd534ab2b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x56553ef2e140 con 0x56553ef213f0 2023-12-01T01:18:23.054 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.048+0000 7fd534ab2b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56553ef2e2b0 con 0x56553ef22a70 2023-12-01T01:18:23.054 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.048+0000 7fd534ab2b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56553ef2e420 con 0x56553ef220f0 2023-12-01T01:18:23.054 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.048+0000 7fd534ab2b80 10 client.0 _ll_register_callbacks cb 0x7fd51800f870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:18:23.054 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.048+0000 7fd522d9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56553ef22a70 0x56553ef2dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.054 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.048+0000 7fd522d9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56553ef22a70 0x56553ef2dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:38474/0 (socket says 172.21.15.77:38474) 2023-12-01T01:18:23.054 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.048+0000 7fd522d9d700 1 -- 172.21.15.77:0/4015860747 learned_addr learned my addr 172.21.15.77:0/4015860747 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:23.054 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.048+0000 7fd52259c700 1 --2- 172.21.15.77:0/4015860747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56553ef213f0 0x56553ef2f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.054 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.048+0000 7fd522d9d700 1 -- 172.21.15.77:0/4015860747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56553ef220f0 msgr2=0x56553ef2fb30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-01T01:18:23.054 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.048+0000 7fd522d9d700 1 --2- 172.21.15.77:0/4015860747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56553ef220f0 0x56553ef2fb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.054 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.048+0000 7fd522d9d700 1 -- 172.21.15.77:0/4015860747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56553ef213f0 msgr2=0x56553ef2f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:23.054 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.048+0000 7fd522d9d700 1 --2- 172.21.15.77:0/4015860747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56553ef213f0 0x56553ef2f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.054 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.048+0000 7fd522d9d700 1 -- 172.21.15.77:0/4015860747 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5241a11d0 con 0x56553ef22a70 2023-12-01T01:18:23.054 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.049+0000 7fd522d9d700 1 --2- 172.21.15.77:0/4015860747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56553ef22a70 0x56553ef2dc00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd51c011bf0 tx=0x7fd51c004110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10fd44fe83848025 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:23.054 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.049+0000 7fd509ffb700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2023-12-01T01:18:23.055 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.049+0000 7fd509ffb700 1 -- 172.21.15.77:0/4015860747 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c00e070 con 0x56553ef22a70 2023-12-01T01:18:23.055 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.049+0000 7fd534ab2b80 1 -- 172.21.15.77:0/4015860747 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd51c00f040 con 0x56553ef22a70 2023-12-01T01:18:23.055 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.049+0000 7fd509ffb700 1 -- 172.21.15.77:0/4015860747 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fd51c007500 con 0x56553ef22a70 2023-12-01T01:18:23.055 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.049+0000 7fd509ffb700 1 -- 172.21.15.77:0/4015860747 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd51c006000 con 0x56553ef22a70 2023-12-01T01:18:23.055 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.049+0000 7fd534ab2b80 10 client.25175 fetch_fsmap learned FSMap version 27 2023-12-01T01:18:23.055 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.049+0000 7fd534ab2b80 1 -- 172.21.15.77:0/4015860747 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56553ec9e2b0 con 0x56553ef22a70 2023-12-01T01:18:23.055 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.049+0000 7fd509ffb700 1 -- 172.21.15.77:0/4015860747 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd51c00f040 con 0x56553ef22a70 2023-12-01T01:18:23.055 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.050+0000 7fd509ffb700 1 -- 172.21.15.77:0/4015860747 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fd51c015820 con 0x56553ef22a70 2023-12-01T01:18:23.055 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.050+0000 7fd534ab2b80 10 client.25175 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:18:23.055 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.050+0000 7fd534ab2b80 10 client.25175 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2023-12-01T01:18:23.055 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.050+0000 7fd534ab2b80 20 client.25175 populate_metadata read hostname 'smithi077' 2023-12-01T01:18:23.055 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.050+0000 7fd534ab2b80 1 --2- 172.21.15.77:0/4015860747 >> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] conn(0x56553ef89450 0x56553ef8b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.055 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.050+0000 7fd534ab2b80 4 client.25175 mds_command: new command op to 24473 tid=0[{"prefix": "damage ls"}] 2023-12-01T01:18:23.055 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.050+0000 7fd534ab2b80 1 -- 172.21.15.77:0/4015860747 --> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56553ee6ad40 con 0x56553ef89450 2023-12-01T01:18:23.056 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.050+0000 7fd52259c700 1 --2- 172.21.15.77:0/4015860747 >> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] conn(0x56553ef89450 0x56553ef8b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.056 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.050+0000 7fd509ffb700 10 client.25175 ms_handle_connect on v2:172.21.15.137:6836/3518881867 2023-12-01T01:18:23.056 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.050+0000 7fd52259c700 1 --2- 172.21.15.77:0/4015860747 >> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] conn(0x56553ef89450 0x56553ef8b870 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=883c4bfe3c569e3 server_cookie=92276695e8e4cad2 in_seq=0 out_seq=0 2023-12-01T01:18:23.056 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.050+0000 7fd509ffb700 1 -- 172.21.15.77:0/4015860747 <== mds.3 v2:172.21.15.137:6836/3518881867 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56553ee6ad40 con 0x56553ef89450 2023-12-01T01:18:23.056 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.050+0000 7fd509ffb700 10 client.25175 handle_command_reply: tid=0 2023-12-01T01:18:23.056 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.051+0000 7fd534ab2b80 1 client.25175 shutdown 2023-12-01T01:18:23.056 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.051+0000 7fd534ab2b80 1 -- 172.21.15.77:0/4015860747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56553ef22a70 msgr2=0x56553ef2dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:23.056 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.051+0000 7fd534ab2b80 1 --2- 172.21.15.77:0/4015860747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56553ef22a70 0x56553ef2dc00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd51c011bf0 tx=0x7fd51c004110 comp rx=0 tx=0).stop 2023-12-01T01:18:23.056 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.051+0000 7fd534ab2b80 1 -- 172.21.15.77:0/4015860747 shutdown_connections 2023-12-01T01:18:23.056 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.051+0000 7fd534ab2b80 1 --2- 172.21.15.77:0/4015860747 >> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] conn(0x56553ef89450 0x56553ef8b870 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.056 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.051+0000 7fd534ab2b80 1 --2- 172.21.15.77:0/4015860747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56553ef22a70 0x56553ef2dc00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.056 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.051+0000 7fd534ab2b80 1 --2- 172.21.15.77:0/4015860747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56553ef213f0 0x56553ef2f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.056 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.051+0000 7fd534ab2b80 1 --2- 172.21.15.77:0/4015860747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56553ef220f0 0x56553ef2fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.056 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.051+0000 7fd534ab2b80 1 -- 172.21.15.77:0/4015860747 >> 172.21.15.77:0/4015860747 conn(0x56553ee800d0 msgr2=0x56553ef25370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:23.056 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.051+0000 7fd509ffb700 0 client.25175 ms_handle_reset on v2:172.21.15.137:6836/3518881867 2023-12-01T01:18:23.057 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.051+0000 7fd534ab2b80 1 -- 172.21.15.77:0/4015860747 shutdown_connections 2023-12-01T01:18:23.057 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.051+0000 7fd534ab2b80 1 -- 172.21.15.77:0/4015860747 wait complete. 2023-12-01T01:18:23.057 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.051+0000 7fd534ab2b80 20 client.25175 trim_cache size 0 max 16384 2023-12-01T01:18:23.057 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2023-12-01T01:18:23.058 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.051+0000 7fd528922700 1 -- 172.21.15.77:0/3289262729 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd50c0a0520 msgr2=0x7fd50c0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:23.058 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.051+0000 7fd528922700 1 --2- 172.21.15.77:0/3289262729 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd50c0a0520 0x7fd50c0a29e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd51800b2e0 tx=0x7fd518005fb0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.058 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.051+0000 7fd528922700 1 -- 172.21.15.77:0/3289262729 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5241018a0 msgr2=0x7fd524101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:23.058 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.051+0000 7fd528922700 1 --2- 172.21.15.77:0/3289262729 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5241018a0 0x7fd524101210 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd51400d950 tx=0x7fd51400dc60 comp rx=0 tx=0).stop 2023-12-01T01:18:23.058 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.056+0000 7fd528922700 1 -- 172.21.15.77:0/3289262729 shutdown_connections 2023-12-01T01:18:23.058 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.056+0000 7fd528922700 1 --2- 172.21.15.77:0/3289262729 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd50c0a0520 0x7fd50c0a29e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.058 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.056+0000 7fd528922700 1 --2- 172.21.15.77:0/3289262729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd524106b00 0x7fd5240ffd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.058 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.056+0000 7fd528922700 1 --2- 172.21.15.77:0/3289262729 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd5241018a0 0x7fd524101210 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.058 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.056+0000 7fd528922700 1 --2- 172.21.15.77:0/3289262729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5241041d0 0x7fd5240ff850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.058 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.056+0000 7fd528922700 1 -- 172.21.15.77:0/3289262729 >> 172.21.15.77:0/3289262729 conn(0x7fd5240fb210 msgr2=0x7fd524102510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:23.058 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.056+0000 7fd528922700 1 -- 172.21.15.77:0/3289262729 shutdown_connections 2023-12-01T01:18:23.059 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.056+0000 7fd528922700 1 -- 172.21.15.77:0/3289262729 wait complete. 2023-12-01T01:18:23.069 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-12-01T01:18:23.249 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.247+0000 7f587f578700 1 -- 172.21.15.77:0/1777211600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58780ff430 msgr2=0x7f58780ff850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:23.249 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.247+0000 7f587f578700 1 --2- 172.21.15.77:0/1777211600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58780ff430 0x7f58780ff850 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5874009990 tx=0x7f5874009ca0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.249 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.247+0000 7f587f578700 1 -- 172.21.15.77:0/1777211600 shutdown_connections 2023-12-01T01:18:23.249 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.247+0000 7f587f578700 1 --2- 172.21.15.77:0/1777211600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5878100fc0 0x7f5878107870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.249 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.247+0000 7f587f578700 1 --2- 172.21.15.77:0/1777211600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58780ff430 0x7f58780ff850 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.249 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.247+0000 7f587f578700 1 --2- 172.21.15.77:0/1777211600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5878100620 0x7f5878100a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.249 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.247+0000 7f587f578700 1 -- 172.21.15.77:0/1777211600 >> 172.21.15.77:0/1777211600 conn(0x7f58780fa980 msgr2=0x7f58780fcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:23.249 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.248+0000 7f587f578700 1 -- 172.21.15.77:0/1777211600 shutdown_connections 2023-12-01T01:18:23.249 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.248+0000 7f587f578700 1 -- 172.21.15.77:0/1777211600 wait complete. 2023-12-01T01:18:23.251 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.248+0000 7f587f578700 1 Processor -- start 2023-12-01T01:18:23.251 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.248+0000 7f587f578700 1 -- start start 2023-12-01T01:18:23.251 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.249+0000 7f587f578700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58780ff430 0x7f5878194840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.251 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.249+0000 7f587f578700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5878100620 0x7f5878194d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.251 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.249+0000 7f587f578700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5878100fc0 0x7f587819bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.251 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.249+0000 7f587f578700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f587819c3c0 con 0x7f5878100fc0 2023-12-01T01:18:23.251 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.249+0000 7f587f578700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f587819c500 con 0x7f5878100620 2023-12-01T01:18:23.251 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.249+0000 7f587f578700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f587819c690 con 0x7f58780ff430 2023-12-01T01:18:23.252 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.249+0000 7f587db15700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5878100fc0 0x7f587819bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.252 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.249+0000 7f587db15700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5878100fc0 0x7f587819bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59566/0 (socket says 172.21.15.77:59566) 2023-12-01T01:18:23.252 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.249+0000 7f587db15700 1 -- 172.21.15.77:0/2347454291 learned_addr learned my addr 172.21.15.77:0/2347454291 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:23.252 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.249+0000 7f587d314700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58780ff430 0x7f5878194840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.252 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.249+0000 7f587cb13700 1 --2- 172.21.15.77:0/2347454291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5878100620 0x7f5878194d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.252 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.249+0000 7f587db15700 1 -- 172.21.15.77:0/2347454291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58780ff430 msgr2=0x7f5878194840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:23.252 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.249+0000 7f587db15700 1 --2- 172.21.15.77:0/2347454291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58780ff430 0x7f5878194840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.252 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.249+0000 7f587db15700 1 -- 172.21.15.77:0/2347454291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5878100620 msgr2=0x7f5878194d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:23.252 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.249+0000 7f587db15700 1 --2- 172.21.15.77:0/2347454291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5878100620 0x7f5878194d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.252 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.249+0000 7f587db15700 1 -- 172.21.15.77:0/2347454291 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58740095f0 con 0x7f5878100fc0 2023-12-01T01:18:23.252 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.249+0000 7f587cb13700 1 --2- 172.21.15.77:0/2347454291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5878100620 0x7f5878194d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:18:23.252 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.249+0000 7f587db15700 1 --2- 172.21.15.77:0/2347454291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5878100fc0 0x7f587819bdf0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5870012370 tx=0x7f587000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:23.252 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.250+0000 7f586e7fc700 1 -- 172.21.15.77:0/2347454291 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587001c070 con 0x7f5878100fc0 2023-12-01T01:18:23.252 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.250+0000 7f587f578700 1 -- 172.21.15.77:0/2347454291 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f587819c910 con 0x7f5878100fc0 2023-12-01T01:18:23.253 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.250+0000 7f586e7fc700 1 -- 172.21.15.77:0/2347454291 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5870012ea0 con 0x7f5878100fc0 2023-12-01T01:18:23.253 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.250+0000 7f586e7fc700 1 -- 172.21.15.77:0/2347454291 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5870014040 con 0x7f5878100fc0 2023-12-01T01:18:23.253 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.250+0000 7f587f578700 1 -- 172.21.15.77:0/2347454291 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f587819ce20 con 0x7f5878100fc0 2023-12-01T01:18:23.254 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.253+0000 7f586e7fc700 1 -- 172.21.15.77:0/2347454291 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f5870004cb0 con 0x7f5878100fc0 2023-12-01T01:18:23.254 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.253+0000 7f586e7fc700 1 --2- 172.21.15.77:0/2347454291 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f58640a0520 0x7f58640a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.255 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.254+0000 7f586e7fc700 1 -- 172.21.15.77:0/2347454291 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f58700cf380 con 0x7f5878100fc0 2023-12-01T01:18:23.255 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.254+0000 7f588b708b80 1 Processor -- start 2023-12-01T01:18:23.255 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.255+0000 7f587d314700 1 --2- 172.21.15.77:0/2347454291 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f58640a0520 0x7f58640a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.257 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.255+0000 7f588b708b80 1 -- start start 2023-12-01T01:18:23.257 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.255+0000 7f588b708b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c846f9f960 0x55c846f9fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.257 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.255+0000 7f588b708b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c846f9e670 0x55c846f9eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.257 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.255+0000 7f588b708b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c846f9f030 0x55c846fa68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.257 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.255+0000 7f588b708b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c846eeda10 con 0x55c846f9f030 2023-12-01T01:18:23.257 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.255+0000 7f588b708b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c846eeb3d0 con 0x55c846f9f960 2023-12-01T01:18:23.257 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.255+0000 7f588b708b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c846ee6cc0 con 0x55c846f9e670 2023-12-01T01:18:23.258 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.255+0000 7f587d314700 1 --2- 172.21.15.77:0/2347454291 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f58640a0520 0x7f58640a29e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f587400b2e0 tx=0x7f5874005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:23.258 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.255+0000 7f587cb13700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c846f9e670 0x55c846f9eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.258 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.255+0000 7f587db15700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c846f9f030 0x55c846fa68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.258 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.255+0000 7f587cb13700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c846f9e670 0x55c846f9eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:45758/0 (socket says 172.21.15.77:45758) 2023-12-01T01:18:23.258 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.255+0000 7f587db15700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c846f9f030 0x55c846fa68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59574/0 (socket says 172.21.15.77:59574) 2023-12-01T01:18:23.258 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.255+0000 7f587cb13700 1 -- 172.21.15.77:0/3558736703 learned_addr learned my addr 172.21.15.77:0/3558736703 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:23.258 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.255+0000 7f587d314700 1 --2- 172.21.15.77:0/3558736703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c846f9f960 0x55c846f9fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.258 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.255+0000 7f587cb13700 1 -- 172.21.15.77:0/3558736703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c846f9f960 msgr2=0x55c846f9fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:23.258 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.256+0000 7f587cb13700 1 --2- 172.21.15.77:0/3558736703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c846f9f960 0x55c846f9fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.258 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.256+0000 7f587cb13700 1 -- 172.21.15.77:0/3558736703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c846f9f030 msgr2=0x55c846fa68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:23.259 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.256+0000 7f587cb13700 1 --2- 172.21.15.77:0/3558736703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c846f9f030 0x55c846fa68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.259 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.256+0000 7f587cb13700 1 -- 172.21.15.77:0/3558736703 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c846d14de0 con 0x55c846f9e670 2023-12-01T01:18:23.259 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.256+0000 7f587d314700 1 --2- 172.21.15.77:0/3558736703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c846f9f960 0x55c846f9fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:18:23.259 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.256+0000 7f587db15700 1 --2- 172.21.15.77:0/3558736703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c846f9f030 0x55c846fa68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:18:23.259 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.256+0000 7f587cb13700 1 --2- 172.21.15.77:0/3558736703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c846f9e670 0x55c846f9eaf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5868010530 tx=0x7f5868010840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c97724e3b4547367 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:23.259 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.256+0000 7f58637fe700 1 -- 172.21.15.77:0/3558736703 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5868004220 con 0x55c846f9e670 2023-12-01T01:18:23.259 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.256+0000 7f58637fe700 1 -- 172.21.15.77:0/3558736703 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5868015070 con 0x55c846f9e670 2023-12-01T01:18:23.259 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.256+0000 7f58637fe700 1 -- 172.21.15.77:0/3558736703 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58680116a0 con 0x55c846f9e670 2023-12-01T01:18:23.262 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.257+0000 7f588b708b80 1 -- 172.21.15.77:0/3558736703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c846f9e670 msgr2=0x55c846f9eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:23.262 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.257+0000 7f588b708b80 1 --2- 172.21.15.77:0/3558736703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c846f9e670 0x55c846f9eaf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5868010530 tx=0x7f5868010840 comp rx=0 tx=0).stop 2023-12-01T01:18:23.262 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.257+0000 7f588b708b80 1 -- 172.21.15.77:0/3558736703 shutdown_connections 2023-12-01T01:18:23.262 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.257+0000 7f588b708b80 1 --2- 172.21.15.77:0/3558736703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c846f9f030 0x55c846fa68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.262 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.257+0000 7f588b708b80 1 --2- 172.21.15.77:0/3558736703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c846f9f960 0x55c846f9fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.262 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.257+0000 7f588b708b80 1 --2- 172.21.15.77:0/3558736703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c846f9e670 0x55c846f9eaf0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.262 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.257+0000 7f588b708b80 1 -- 172.21.15.77:0/3558736703 >> 172.21.15.77:0/3558736703 conn(0x55c846ef50d0 msgr2=0x55c846f90940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:23.262 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.257+0000 7f588b708b80 1 -- 172.21.15.77:0/3558736703 shutdown_connections 2023-12-01T01:18:23.263 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.257+0000 7f588b708b80 1 -- 172.21.15.77:0/3558736703 wait complete. 2023-12-01T01:18:23.263 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.258+0000 7f588b708b80 1 -- start start 2023-12-01T01:18:23.263 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.258+0000 7f588b708b80 1 Processor -- start 2023-12-01T01:18:23.263 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.258+0000 7f588b708b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c846f9e670 0x55c846f971e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.263 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.258+0000 7f588b708b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c846f9f030 0x55c846f97720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.263 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.258+0000 7f588b708b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c846f9f960 0x55c846f9c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.263 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.258+0000 7f588b708b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c846e83510 con 0x55c846f9e670 2023-12-01T01:18:23.263 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.258+0000 7f588b708b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c846e831b0 con 0x55c846f9f960 2023-12-01T01:18:23.263 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.258+0000 7f588b708b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c846d6a920 con 0x55c846f9f030 2023-12-01T01:18:23.263 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.258+0000 7f588b708b80 10 client.0 _ll_register_callbacks cb 0x55c846dd0a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:18:23.263 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.258+0000 7f587d314700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c846f9e670 0x55c846f971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.263 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.259+0000 7f587d314700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c846f9e670 0x55c846f971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59576/0 (socket says 172.21.15.77:59576) 2023-12-01T01:18:23.264 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.259+0000 7f587d314700 1 -- 172.21.15.77:0/3310119479 learned_addr learned my addr 172.21.15.77:0/3310119479 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:23.264 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.259+0000 7f587db15700 1 --2- 172.21.15.77:0/3310119479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c846f9f960 0x55c846f9c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.264 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.259+0000 7f587d314700 1 -- 172.21.15.77:0/3310119479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c846f9f030 msgr2=0x55c846f97720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-01T01:18:23.264 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.259+0000 7f587d314700 1 --2- 172.21.15.77:0/3310119479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c846f9f030 0x55c846f97720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.264 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.259+0000 7f587d314700 1 -- 172.21.15.77:0/3310119479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c846f9f960 msgr2=0x55c846f9c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:23.264 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.259+0000 7f587d314700 1 --2- 172.21.15.77:0/3310119479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c846f9f960 0x55c846f9c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.264 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.259+0000 7f587d314700 1 -- 172.21.15.77:0/3310119479 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f587819ce20 con 0x55c846f9e670 2023-12-01T01:18:23.264 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.259+0000 7f587d314700 1 --2- 172.21.15.77:0/3310119479 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c846f9e670 0x55c846f971e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5874009990 tx=0x7f5874009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1841097ecaa9163e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:23.264 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.259+0000 7f5860ff9700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:18:23.264 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.259+0000 7f5860ff9700 1 -- 172.21.15.77:0/3310119479 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587400f660 con 0x55c846f9e670 2023-12-01T01:18:23.264 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.259+0000 7f5860ff9700 1 -- 172.21.15.77:0/3310119479 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f587400f7c0 con 0x55c846f9e670 2023-12-01T01:18:23.264 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.259+0000 7f588b708b80 1 -- 172.21.15.77:0/3310119479 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c846f11d10 con 0x55c846f9e670 2023-12-01T01:18:23.264 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.260+0000 7f5860ff9700 1 -- 172.21.15.77:0/3310119479 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5874004e80 con 0x55c846f9e670 2023-12-01T01:18:23.264 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.260+0000 7f588b708b80 10 client.15588 fetch_fsmap learned FSMap version 27 2023-12-01T01:18:23.265 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.260+0000 7f588b708b80 1 -- 172.21.15.77:0/3310119479 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f586800bd90 con 0x55c846f9e670 2023-12-01T01:18:23.265 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.260+0000 7f5860ff9700 1 -- 172.21.15.77:0/3310119479 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c846f11d10 con 0x55c846f9e670 2023-12-01T01:18:23.265 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.260+0000 7f5860ff9700 1 -- 172.21.15.77:0/3310119479 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f587402fb40 con 0x55c846f9e670 2023-12-01T01:18:23.265 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.260+0000 7f588b708b80 10 client.15588 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:18:23.265 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.260+0000 7f588b708b80 10 client.15588 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-01T01:18:23.265 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.260+0000 7f588b708b80 20 client.15588 populate_metadata read hostname 'smithi077' 2023-12-01T01:18:23.265 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.260+0000 7f588b708b80 1 --2- 172.21.15.77:0/3310119479 >> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] conn(0x55c846fbc7e0 0x55c846fbec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.265 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.260+0000 7f588b708b80 4 client.15588 mds_command: new command op to 24475 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:18:23.265 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.260+0000 7f588b708b80 1 -- 172.21.15.77:0/3310119479 --> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c846f64e10 con 0x55c846fbc7e0 2023-12-01T01:18:23.265 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.260+0000 7f587cb13700 1 --2- 172.21.15.77:0/3310119479 >> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] conn(0x55c846fbc7e0 0x55c846fbec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.265 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.261+0000 7f5860ff9700 10 client.15588 ms_handle_connect on v2:172.21.15.203:6836/2887080977 2023-12-01T01:18:23.265 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.261+0000 7f587cb13700 1 --2- 172.21.15.77:0/3310119479 >> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] conn(0x55c846fbc7e0 0x55c846fbec00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bd387322953f0067 server_cookie=696d6167f2781473 in_seq=0 out_seq=0 2023-12-01T01:18:23.267 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.262+0000 7f5860ff9700 1 -- 172.21.15.77:0/3310119479 <== mds.2 v2:172.21.15.203:6836/2887080977 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55c846f64e10 con 0x55c846fbc7e0 2023-12-01T01:18:23.267 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.262+0000 7f5860ff9700 10 client.15588 handle_command_reply: tid=0 2023-12-01T01:18:23.267 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.262+0000 7f588b708b80 1 client.15588 shutdown 2023-12-01T01:18:23.267 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.262+0000 7f588b708b80 1 -- 172.21.15.77:0/3310119479 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c846f9e670 msgr2=0x55c846f971e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:23.267 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.262+0000 7f588b708b80 1 --2- 172.21.15.77:0/3310119479 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c846f9e670 0x55c846f971e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5874009990 tx=0x7f5874009ca0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.267 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.262+0000 7f588b708b80 1 -- 172.21.15.77:0/3310119479 shutdown_connections 2023-12-01T01:18:23.267 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.262+0000 7f588b708b80 1 --2- 172.21.15.77:0/3310119479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c846f9f960 0x55c846f9c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.267 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.262+0000 7f588b708b80 1 --2- 172.21.15.77:0/3310119479 >> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] conn(0x55c846fbc7e0 0x55c846fbec00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.267 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.262+0000 7f588b708b80 1 --2- 172.21.15.77:0/3310119479 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c846f9e670 0x55c846f971e0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.267 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.262+0000 7f588b708b80 1 --2- 172.21.15.77:0/3310119479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c846f9f030 0x55c846f97720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.267 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.262+0000 7f588b708b80 1 -- 172.21.15.77:0/3310119479 >> 172.21.15.77:0/3310119479 conn(0x55c846ef50d0 msgr2=0x55c846f900b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:23.267 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.262+0000 7f5860ff9700 0 client.15588 ms_handle_reset on v2:172.21.15.203:6836/2887080977 2023-12-01T01:18:23.267 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.262+0000 7f588b708b80 1 -- 172.21.15.77:0/3310119479 shutdown_connections 2023-12-01T01:18:23.268 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.262+0000 7f588b708b80 1 -- 172.21.15.77:0/3310119479 wait complete. 2023-12-01T01:18:23.268 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.262+0000 7f588b708b80 20 client.15588 trim_cache size 0 max 16384 2023-12-01T01:18:23.289 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.286+0000 7f588b708b80 1 Processor -- start 2023-12-01T01:18:23.289 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.286+0000 7f588b708b80 1 -- start start 2023-12-01T01:18:23.289 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.286+0000 7f588b708b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c846f9e670 0x55c846fbebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.289 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.286+0000 7f588b708b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c846f9f030 0x55c846f9c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.289 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.286+0000 7f588b708b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c846f9f960 0x55c846f96890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.289 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.286+0000 7f588b708b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c846f11d10 con 0x55c846f9f960 2023-12-01T01:18:23.289 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.286+0000 7f588b708b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c846f983d0 con 0x55c846f9e670 2023-12-01T01:18:23.289 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.286+0000 7f588b708b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c846fa6ca0 con 0x55c846f9f030 2023-12-01T01:18:23.289 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.287+0000 7f587db15700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c846f9f960 0x55c846f96890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.289 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.287+0000 7f587db15700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c846f9f960 0x55c846f96890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59586/0 (socket says 172.21.15.77:59586) 2023-12-01T01:18:23.290 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.287+0000 7f587db15700 1 -- 172.21.15.77:0/310554636 learned_addr learned my addr 172.21.15.77:0/310554636 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:23.290 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.287+0000 7f587cb13700 1 --2- 172.21.15.77:0/310554636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c846f9f030 0x55c846f9c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.290 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.287+0000 7f587d314700 1 --2- 172.21.15.77:0/310554636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c846f9e670 0x55c846fbebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.290 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.287+0000 7f587cb13700 1 -- 172.21.15.77:0/310554636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c846f9e670 msgr2=0x55c846fbebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:23.290 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.287+0000 7f587cb13700 1 --2- 172.21.15.77:0/310554636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c846f9e670 0x55c846fbebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.290 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.287+0000 7f587cb13700 1 -- 172.21.15.77:0/310554636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c846f9f960 msgr2=0x55c846f96890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:23.290 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.287+0000 7f587cb13700 1 --2- 172.21.15.77:0/310554636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c846f9f960 0x55c846f96890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.290 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.287+0000 7f587cb13700 1 -- 172.21.15.77:0/310554636 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c846cfdc60 con 0x55c846f9f030 2023-12-01T01:18:23.290 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.287+0000 7f587db15700 1 --2- 172.21.15.77:0/310554636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c846f9f960 0x55c846f96890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:18:23.290 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.287+0000 7f587d314700 1 --2- 172.21.15.77:0/310554636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c846f9e670 0x55c846fbebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:18:23.290 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.287+0000 7f587cb13700 1 --2- 172.21.15.77:0/310554636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c846f9f030 0x55c846f9c710 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x55c846fbf550 tx=0x7f586800bcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d66622cdbd58e5f5 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:23.290 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.288+0000 7f5863fff700 1 -- 172.21.15.77:0/310554636 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5868020040 con 0x55c846f9f030 2023-12-01T01:18:23.290 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.288+0000 7f5863fff700 1 -- 172.21.15.77:0/310554636 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5868010ec0 con 0x55c846f9f030 2023-12-01T01:18:23.291 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.288+0000 7f5863fff700 1 -- 172.21.15.77:0/310554636 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58680052a0 con 0x55c846f9f030 2023-12-01T01:18:23.294 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.289+0000 7f588b708b80 1 -- 172.21.15.77:0/310554636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c846f9f030 msgr2=0x55c846f9c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:23.294 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.289+0000 7f588b708b80 1 --2- 172.21.15.77:0/310554636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c846f9f030 0x55c846f9c710 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x55c846fbf550 tx=0x7f586800bcf0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.294 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.289+0000 7f588b708b80 1 -- 172.21.15.77:0/310554636 shutdown_connections 2023-12-01T01:18:23.294 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.289+0000 7f588b708b80 1 --2- 172.21.15.77:0/310554636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c846f9f960 0x55c846f96890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.294 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.289+0000 7f588b708b80 1 --2- 172.21.15.77:0/310554636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c846f9e670 0x55c846fbebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.294 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.289+0000 7f588b708b80 1 --2- 172.21.15.77:0/310554636 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c846f9f030 0x55c846f9c710 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.294 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.289+0000 7f588b708b80 1 -- 172.21.15.77:0/310554636 >> 172.21.15.77:0/310554636 conn(0x55c846ef50d0 msgr2=0x55c846f8cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:23.294 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.289+0000 7f588b708b80 1 -- 172.21.15.77:0/310554636 shutdown_connections 2023-12-01T01:18:23.294 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.289+0000 7f588b708b80 1 -- 172.21.15.77:0/310554636 wait complete. 2023-12-01T01:18:23.294 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.289+0000 7f588b708b80 1 -- start start 2023-12-01T01:18:23.294 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.289+0000 7f588b708b80 1 Processor -- start 2023-12-01T01:18:23.295 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.290+0000 7f588b708b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c846f9e670 0x55c846f9c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.295 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.290+0000 7f588b708b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c846f9f030 0x55c84700a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.295 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.290+0000 7f588b708b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c846f9f960 0x55c84700c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.295 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.290+0000 7f588b708b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c84700d060 con 0x55c846f9f030 2023-12-01T01:18:23.295 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.290+0000 7f588b708b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c84700d1d0 con 0x55c846f9e670 2023-12-01T01:18:23.295 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.290+0000 7f588b708b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c84700d390 con 0x55c846f9f960 2023-12-01T01:18:23.295 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.290+0000 7f588b708b80 10 client.0 _ll_register_callbacks cb 0x7f5874005350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:18:23.295 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.290+0000 7f587cb13700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c846f9f030 0x55c84700a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.295 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.290+0000 7f587cb13700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c846f9f030 0x55c84700a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59590/0 (socket says 172.21.15.77:59590) 2023-12-01T01:18:23.295 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.290+0000 7f587cb13700 1 -- 172.21.15.77:0/510860687 learned_addr learned my addr 172.21.15.77:0/510860687 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:23.295 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.290+0000 7f587d314700 1 --2- 172.21.15.77:0/510860687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c846f9e670 0x55c846f9c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.295 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.290+0000 7f587db15700 1 --2- 172.21.15.77:0/510860687 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c846f9f960 0x55c84700c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.295 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.290+0000 7f587cb13700 1 -- 172.21.15.77:0/510860687 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c846f9f960 msgr2=0x55c84700c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:23.295 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.290+0000 7f587cb13700 1 --2- 172.21.15.77:0/510860687 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c846f9f960 0x55c84700c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.296 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.290+0000 7f587cb13700 1 -- 172.21.15.77:0/510860687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c846f9e670 msgr2=0x55c846f9c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:23.296 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.290+0000 7f587cb13700 1 --2- 172.21.15.77:0/510860687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c846f9e670 0x55c846f9c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.296 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.291+0000 7f587cb13700 1 -- 172.21.15.77:0/510860687 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f586800bd90 con 0x55c846f9f030 2023-12-01T01:18:23.296 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.291+0000 7f587d314700 1 --2- 172.21.15.77:0/510860687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c846f9e670 0x55c846f9c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:18:23.296 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.291+0000 7f587db15700 1 --2- 172.21.15.77:0/510860687 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c846f9f960 0x55c84700c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:18:23.296 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.291+0000 7f587cb13700 1 --2- 172.21.15.77:0/510860687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c846f9f030 0x55c84700a310 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f5868005a40 tx=0x7f5868010ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d56b5b4ba0c3302 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:23.296 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.291+0000 7f5860ff9700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:18:23.296 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.291+0000 7f5860ff9700 1 -- 172.21.15.77:0/510860687 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5868020030 con 0x55c846f9f030 2023-12-01T01:18:23.296 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.291+0000 7f5860ff9700 1 -- 172.21.15.77:0/510860687 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f5868017070 con 0x55c846f9f030 2023-12-01T01:18:23.296 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.291+0000 7f588b708b80 1 -- 172.21.15.77:0/510860687 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c846edfd40 con 0x55c846f9f030 2023-12-01T01:18:23.296 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.291+0000 7f5860ff9700 1 -- 172.21.15.77:0/510860687 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5868004000 con 0x55c846f9f030 2023-12-01T01:18:23.296 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.291+0000 7f588b708b80 10 client.15594 fetch_fsmap learned FSMap version 27 2023-12-01T01:18:23.296 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.291+0000 7f588b708b80 1 -- 172.21.15.77:0/510860687 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f587819c910 con 0x55c846f9f030 2023-12-01T01:18:23.296 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.291+0000 7f5860ff9700 1 -- 172.21.15.77:0/510860687 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f586803f020 con 0x55c846f9f030 2023-12-01T01:18:23.296 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.292+0000 7f5860ff9700 1 -- 172.21.15.77:0/510860687 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f58680037a0 con 0x55c846f9f030 2023-12-01T01:18:23.297 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.292+0000 7f588b708b80 10 client.15594 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:18:23.297 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.292+0000 7f588b708b80 10 client.15594 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-01T01:18:23.297 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.292+0000 7f588b708b80 20 client.15594 populate_metadata read hostname 'smithi077' 2023-12-01T01:18:23.297 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.293+0000 7f588b708b80 1 --2- 172.21.15.77:0/510860687 >> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] conn(0x55c84700e820 0x55c847010c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.297 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.293+0000 7f588b708b80 4 client.15594 mds_command: new command op to 24475 tid=0[{"prefix": "damage ls"}] 2023-12-01T01:18:23.297 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.293+0000 7f588b708b80 1 -- 172.21.15.77:0/510860687 --> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c846edfa60 con 0x55c84700e820 2023-12-01T01:18:23.297 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.293+0000 7f587d314700 1 --2- 172.21.15.77:0/510860687 >> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] conn(0x55c84700e820 0x55c847010c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.298 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.293+0000 7f5860ff9700 10 client.15594 ms_handle_connect on v2:172.21.15.203:6836/2887080977 2023-12-01T01:18:23.298 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.293+0000 7f587d314700 1 --2- 172.21.15.77:0/510860687 >> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] conn(0x55c84700e820 0x55c847010c40 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=49b7ac21cc2e3865 server_cookie=fdf5f30e447c8454 in_seq=0 out_seq=0 2023-12-01T01:18:23.298 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.293+0000 7f5860ff9700 1 -- 172.21.15.77:0/510860687 <== mds.2 v2:172.21.15.203:6836/2887080977 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c846edfa60 con 0x55c84700e820 2023-12-01T01:18:23.298 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.293+0000 7f5860ff9700 10 client.15594 handle_command_reply: tid=0 2023-12-01T01:18:23.298 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.293+0000 7f588b708b80 1 client.15594 shutdown 2023-12-01T01:18:23.299 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.294+0000 7f588b708b80 1 -- 172.21.15.77:0/510860687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c846f9f030 msgr2=0x55c84700a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:23.299 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.294+0000 7f588b708b80 1 --2- 172.21.15.77:0/510860687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c846f9f030 0x55c84700a310 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f5868005a40 tx=0x7f5868010ec0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.299 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.294+0000 7f588b708b80 1 -- 172.21.15.77:0/510860687 shutdown_connections 2023-12-01T01:18:23.299 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.294+0000 7f588b708b80 1 --2- 172.21.15.77:0/510860687 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c846f9f960 0x55c84700c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.299 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.294+0000 7f588b708b80 1 --2- 172.21.15.77:0/510860687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c846f9e670 0x55c846f9c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.299 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.294+0000 7f588b708b80 1 --2- 172.21.15.77:0/510860687 >> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] conn(0x55c84700e820 0x55c847010c40 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.299 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.294+0000 7f588b708b80 1 --2- 172.21.15.77:0/510860687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c846f9f030 0x55c84700a310 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.299 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.294+0000 7f588b708b80 1 -- 172.21.15.77:0/510860687 >> 172.21.15.77:0/510860687 conn(0x55c846ef50d0 msgr2=0x55c846fa2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:23.299 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.294+0000 7f5860ff9700 0 client.15594 ms_handle_reset on v2:172.21.15.203:6836/2887080977 2023-12-01T01:18:23.299 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.294+0000 7f588b708b80 1 -- 172.21.15.77:0/510860687 shutdown_connections 2023-12-01T01:18:23.299 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.294+0000 7f588b708b80 1 -- 172.21.15.77:0/510860687 wait complete. 2023-12-01T01:18:23.299 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.294+0000 7f588b708b80 20 client.15594 trim_cache size 0 max 16384 2023-12-01T01:18:23.300 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2023-12-01T01:18:23.300 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.295+0000 7f587f578700 1 -- 172.21.15.77:0/2347454291 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f58640a0520 msgr2=0x7f58640a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:23.300 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.295+0000 7f587f578700 1 --2- 172.21.15.77:0/2347454291 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f58640a0520 0x7f58640a29e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f587400b2e0 tx=0x7f5874005fb0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.300 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.295+0000 7f587f578700 1 -- 172.21.15.77:0/2347454291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5878100fc0 msgr2=0x7f587819bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:23.301 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.295+0000 7f587f578700 1 --2- 172.21.15.77:0/2347454291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5878100fc0 0x7f587819bdf0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5870012370 tx=0x7f587000ef90 comp rx=0 tx=0).stop 2023-12-01T01:18:23.301 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.295+0000 7f587f578700 1 -- 172.21.15.77:0/2347454291 shutdown_connections 2023-12-01T01:18:23.301 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.295+0000 7f587f578700 1 --2- 172.21.15.77:0/2347454291 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f58640a0520 0x7f58640a29e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.301 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.295+0000 7f587f578700 1 --2- 172.21.15.77:0/2347454291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5878100fc0 0x7f587819bdf0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.301 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.295+0000 7f587f578700 1 --2- 172.21.15.77:0/2347454291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58780ff430 0x7f5878194840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.301 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.295+0000 7f587f578700 1 --2- 172.21.15.77:0/2347454291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5878100620 0x7f5878194d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.301 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.295+0000 7f587f578700 1 -- 172.21.15.77:0/2347454291 >> 172.21.15.77:0/2347454291 conn(0x7f58780fa980 msgr2=0x7f58780fcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:23.301 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.295+0000 7f587f578700 1 -- 172.21.15.77:0/2347454291 shutdown_connections 2023-12-01T01:18:23.301 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.295+0000 7f587f578700 1 -- 172.21.15.77:0/2347454291 wait complete. 2023-12-01T01:18:23.312 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-12-01T01:18:23.469 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.465+0000 7fb64aa82700 1 -- 172.21.15.77:0/596382806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb644100750 msgr2=0x7fb644100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:23.469 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.465+0000 7fb64aa82700 1 --2- 172.21.15.77:0/596382806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb644100750 0x7fb644100fd0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb63c00b0a0 tx=0x7fb63c00b3b0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.469 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.465+0000 7fb64aa82700 1 -- 172.21.15.77:0/596382806 shutdown_connections 2023-12-01T01:18:23.469 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.465+0000 7fb64aa82700 1 --2- 172.21.15.77:0/596382806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb644100750 0x7fb644100fd0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.469 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.465+0000 7fb64aa82700 1 --2- 172.21.15.77:0/596382806 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6440ff030 0x7fb6440ff450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.469 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.465+0000 7fb64aa82700 1 --2- 172.21.15.77:0/596382806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6440ffd90 0x7fb644100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.469 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.465+0000 7fb64aa82700 1 -- 172.21.15.77:0/596382806 >> 172.21.15.77:0/596382806 conn(0x7fb6440fa960 msgr2=0x7fb6440fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:23.469 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.466+0000 7fb64aa82700 1 -- 172.21.15.77:0/596382806 shutdown_connections 2023-12-01T01:18:23.469 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.466+0000 7fb64aa82700 1 -- 172.21.15.77:0/596382806 wait complete. 2023-12-01T01:18:23.469 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.466+0000 7fb64aa82700 1 Processor -- start 2023-12-01T01:18:23.469 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.466+0000 7fb64aa82700 1 -- start start 2023-12-01T01:18:23.469 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.466+0000 7fb64aa82700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6440ff030 0x7fb644198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.470 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.466+0000 7fb64aa82700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6440ffd90 0x7fb644199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.470 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.466+0000 7fb64aa82700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb644100750 0x7fb6441a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.470 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.466+0000 7fb64aa82700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6441a0830 con 0x7fb6440ff030 2023-12-01T01:18:23.470 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.466+0000 7fb64aa82700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6441a09a0 con 0x7fb6440ffd90 2023-12-01T01:18:23.470 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.466+0000 7fb64aa82700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6441a0b60 con 0x7fb644100750 2023-12-01T01:18:23.470 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.467+0000 7fb64881e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6440ff030 0x7fb644198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.470 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.467+0000 7fb64881e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6440ff030 0x7fb644198cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59602/0 (socket says 172.21.15.77:59602) 2023-12-01T01:18:23.470 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.467+0000 7fb64881e700 1 -- 172.21.15.77:0/1124428187 learned_addr learned my addr 172.21.15.77:0/1124428187 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:23.470 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.467+0000 7fb643fff700 1 --2- 172.21.15.77:0/1124428187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6440ffd90 0x7fb644199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.470 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.467+0000 7fb64881e700 1 -- 172.21.15.77:0/1124428187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb644100750 msgr2=0x7fb6441a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:23.470 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.467+0000 7fb64881e700 1 --2- 172.21.15.77:0/1124428187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb644100750 0x7fb6441a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.470 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.467+0000 7fb64881e700 1 -- 172.21.15.77:0/1124428187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6440ffd90 msgr2=0x7fb644199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:23.470 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.467+0000 7fb64881e700 1 --2- 172.21.15.77:0/1124428187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6440ffd90 0x7fb644199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.467+0000 7fb64881e700 1 -- 172.21.15.77:0/1124428187 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb63c009d00 con 0x7fb6440ff030 2023-12-01T01:18:23.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.467+0000 7fb64881e700 1 --2- 172.21.15.77:0/1124428187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6440ff030 0x7fb644198cf0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb63800d950 tx=0x7fb63800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:23.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.467+0000 7fb641ffb700 1 -- 172.21.15.77:0/1124428187 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb638009940 con 0x7fb6440ff030 2023-12-01T01:18:23.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.467+0000 7fb64aa82700 1 -- 172.21.15.77:0/1124428187 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6441a0e40 con 0x7fb6440ff030 2023-12-01T01:18:23.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.467+0000 7fb641ffb700 1 -- 172.21.15.77:0/1124428187 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb638010460 con 0x7fb6440ff030 2023-12-01T01:18:23.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.467+0000 7fb641ffb700 1 -- 172.21.15.77:0/1124428187 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63800f620 con 0x7fb6440ff030 2023-12-01T01:18:23.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.468+0000 7fb64aa82700 1 -- 172.21.15.77:0/1124428187 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6441a1270 con 0x7fb6440ff030 2023-12-01T01:18:23.472 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.470+0000 7fb641ffb700 1 -- 172.21.15.77:0/1124428187 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fb6380108d0 con 0x7fb6440ff030 2023-12-01T01:18:23.472 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.471+0000 7fb641ffb700 1 --2- 172.21.15.77:0/1124428187 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fb62c0a0520 0x7fb62c0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.472 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.471+0000 7fb641ffb700 1 -- 172.21.15.77:0/1124428187 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fb6380cb800 con 0x7fb6440ff030 2023-12-01T01:18:23.472 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.471+0000 7fb643fff700 1 --2- 172.21.15.77:0/1124428187 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fb62c0a0520 0x7fb62c0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.472 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.471+0000 7fb643fff700 1 --2- 172.21.15.77:0/1124428187 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fb62c0a0520 0x7fb62c0a29e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb634005950 tx=0x7fb63400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:23.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.472+0000 7fb656c12b80 1 Processor -- start 2023-12-01T01:18:23.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.472+0000 7fb656c12b80 1 -- start start 2023-12-01T01:18:23.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.472+0000 7fb656c12b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56441cda50f0 0x56441cda5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.472+0000 7fb656c12b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56441cda5a70 0x56441cda3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.474 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.472+0000 7fb656c12b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56441cda43f0 0x56441cda4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.475 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.472+0000 7fb656c12b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x56441ccfba10 con 0x56441cda5a70 2023-12-01T01:18:23.475 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.472+0000 7fb656c12b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56441ccf93d0 con 0x56441cda43f0 2023-12-01T01:18:23.475 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.472+0000 7fb656c12b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56441ccf4cc0 con 0x56441cda50f0 2023-12-01T01:18:23.475 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.472+0000 7fb643fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56441cda5a70 0x56441cda3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.475 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.472+0000 7fb643fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56441cda5a70 0x56441cda3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59614/0 (socket says 172.21.15.77:59614) 2023-12-01T01:18:23.475 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.472+0000 7fb643fff700 1 -- 172.21.15.77:0/3840388338 learned_addr learned my addr 172.21.15.77:0/3840388338 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:23.475 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.472+0000 7fb64881e700 1 --2- 172.21.15.77:0/3840388338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56441cda50f0 0x56441cda5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.475 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.472+0000 7fb64901f700 1 --2- 172.21.15.77:0/3840388338 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56441cda43f0 0x56441cda4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.475 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.472+0000 7fb643fff700 1 -- 172.21.15.77:0/3840388338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56441cda50f0 msgr2=0x56441cda5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:23.475 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.472+0000 7fb643fff700 1 --2- 172.21.15.77:0/3840388338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56441cda50f0 0x56441cda5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.475 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.472+0000 7fb643fff700 1 -- 172.21.15.77:0/3840388338 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56441cda43f0 msgr2=0x56441cda4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:23.475 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.472+0000 7fb643fff700 1 --2- 172.21.15.77:0/3840388338 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56441cda43f0 0x56441cda4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.476 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.472+0000 7fb643fff700 1 -- 172.21.15.77:0/3840388338 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56441cb22de0 con 0x56441cda5a70 2023-12-01T01:18:23.476 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.472+0000 7fb64901f700 1 --2- 172.21.15.77:0/3840388338 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56441cda43f0 0x56441cda4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:18:23.476 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.473+0000 7fb643fff700 1 --2- 172.21.15.77:0/3840388338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56441cda5a70 0x56441cda3e60 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb634010fa0 tx=0x7fb634013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c14d07124b06ee5 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:23.476 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.473+0000 7fb62affd700 1 -- 172.21.15.77:0/3840388338 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb634011db0 con 0x56441cda5a70 2023-12-01T01:18:23.476 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.473+0000 7fb62affd700 1 -- 172.21.15.77:0/3840388338 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb634015070 con 0x56441cda5a70 2023-12-01T01:18:23.476 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.473+0000 7fb62affd700 1 -- 172.21.15.77:0/3840388338 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63401c610 con 0x56441cda5a70 2023-12-01T01:18:23.480 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.474+0000 7fb656c12b80 1 -- 172.21.15.77:0/3840388338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56441cda5a70 msgr2=0x56441cda3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:23.480 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.474+0000 7fb656c12b80 1 --2- 172.21.15.77:0/3840388338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56441cda5a70 0x56441cda3e60 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb634010fa0 tx=0x7fb634013000 comp rx=0 tx=0).stop 2023-12-01T01:18:23.480 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.474+0000 7fb656c12b80 1 -- 172.21.15.77:0/3840388338 shutdown_connections 2023-12-01T01:18:23.480 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.474+0000 7fb656c12b80 1 --2- 172.21.15.77:0/3840388338 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56441cda43f0 0x56441cda4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.480 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.474+0000 7fb656c12b80 1 --2- 172.21.15.77:0/3840388338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56441cda50f0 0x56441cda5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.480 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.474+0000 7fb656c12b80 1 --2- 172.21.15.77:0/3840388338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56441cda5a70 0x56441cda3e60 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.480 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.474+0000 7fb656c12b80 1 -- 172.21.15.77:0/3840388338 >> 172.21.15.77:0/3840388338 conn(0x56441cd030d0 msgr2=0x56441cd9e940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:23.480 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.474+0000 7fb656c12b80 1 -- 172.21.15.77:0/3840388338 shutdown_connections 2023-12-01T01:18:23.480 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.474+0000 7fb656c12b80 1 -- 172.21.15.77:0/3840388338 wait complete. 2023-12-01T01:18:23.480 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.474+0000 7fb656c12b80 1 -- start start 2023-12-01T01:18:23.480 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.474+0000 7fb656c12b80 1 Processor -- start 2023-12-01T01:18:23.481 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.475+0000 7fb656c12b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56441cda43f0 0x56441cdad980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.481 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.475+0000 7fb656c12b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56441cda50f0 0x56441cdadec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.481 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.475+0000 7fb656c12b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56441cda5a70 0x56441cdb4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.481 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.475+0000 7fb656c12b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x56441cc91510 con 0x56441cda43f0 2023-12-01T01:18:23.481 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.475+0000 7fb656c12b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56441cc911b0 con 0x56441cda5a70 2023-12-01T01:18:23.481 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.475+0000 7fb656c12b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56441cb78920 con 0x56441cda50f0 2023-12-01T01:18:23.481 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.475+0000 7fb656c12b80 10 client.0 _ll_register_callbacks cb 0x56441cbdea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:18:23.481 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.475+0000 7fb64881e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56441cda43f0 0x56441cdad980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.481 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.475+0000 7fb64881e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56441cda43f0 0x56441cdad980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59616/0 (socket says 172.21.15.77:59616) 2023-12-01T01:18:23.481 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.475+0000 7fb64881e700 1 -- 172.21.15.77:0/3148462024 learned_addr learned my addr 172.21.15.77:0/3148462024 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:23.481 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.475+0000 7fb643fff700 1 --2- 172.21.15.77:0/3148462024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56441cda50f0 0x56441cdadec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.481 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.475+0000 7fb64881e700 1 -- 172.21.15.77:0/3148462024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56441cda50f0 msgr2=0x56441cdadec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:23.481 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.475+0000 7fb64881e700 1 --2- 172.21.15.77:0/3148462024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56441cda50f0 0x56441cdadec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.481 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.475+0000 7fb64881e700 1 -- 172.21.15.77:0/3148462024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56441cda5a70 msgr2=0x56441cdb4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:23.482 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.475+0000 7fb64881e700 1 --2- 172.21.15.77:0/3148462024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56441cda5a70 0x56441cdb4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.482 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.475+0000 7fb64881e700 1 -- 172.21.15.77:0/3148462024 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb634010c50 con 0x56441cda43f0 2023-12-01T01:18:23.482 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.475+0000 7fb643fff700 1 --2- 172.21.15.77:0/3148462024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56441cda50f0 0x56441cdadec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:18:23.482 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.475+0000 7fb64881e700 1 --2- 172.21.15.77:0/3148462024 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56441cda43f0 0x56441cdad980 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb638010fd0 tx=0x7fb63809ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8c8ec9b3b2ac22f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:23.482 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.475+0000 7fb623fff700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:18:23.482 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.475+0000 7fb623fff700 1 -- 172.21.15.77:0/3148462024 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6380974c0 con 0x56441cda43f0 2023-12-01T01:18:23.482 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.475+0000 7fb656c12b80 1 -- 172.21.15.77:0/3148462024 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56441cd1fd10 con 0x56441cda43f0 2023-12-01T01:18:23.482 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.475+0000 7fb623fff700 1 -- 172.21.15.77:0/3148462024 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fb638098550 con 0x56441cda43f0 2023-12-01T01:18:23.482 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.476+0000 7fb623fff700 1 -- 172.21.15.77:0/3148462024 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb638098710 con 0x56441cda43f0 2023-12-01T01:18:23.482 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.476+0000 7fb656c12b80 10 client.15606 fetch_fsmap learned FSMap version 27 2023-12-01T01:18:23.482 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.476+0000 7fb656c12b80 1 -- 172.21.15.77:0/3148462024 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56441cb0bc60 con 0x56441cda43f0 2023-12-01T01:18:23.482 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.476+0000 7fb623fff700 1 -- 172.21.15.77:0/3148462024 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56441cd1fd10 con 0x56441cda43f0 2023-12-01T01:18:23.482 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.476+0000 7fb623fff700 1 -- 172.21.15.77:0/3148462024 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fb638097fc0 con 0x56441cda43f0 2023-12-01T01:18:23.482 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.476+0000 7fb656c12b80 10 client.15606 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:18:23.483 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.476+0000 7fb656c12b80 10 client.15606 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-12-01T01:18:23.483 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.476+0000 7fb656c12b80 20 client.15606 populate_metadata read hostname 'smithi077' 2023-12-01T01:18:23.483 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.476+0000 7fb656c12b80 1 --2- 172.21.15.77:0/3148462024 >> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] conn(0x56441cdb95d0 0x56441cdbb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.483 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.476+0000 7fb656c12b80 4 client.15606 mds_command: new command op to 24496 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:18:23.483 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.476+0000 7fb656c12b80 1 -- 172.21.15.77:0/3148462024 --> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56441cd72e10 con 0x56441cdb95d0 2023-12-01T01:18:23.483 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.476+0000 7fb643fff700 1 --2- 172.21.15.77:0/3148462024 >> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] conn(0x56441cdb95d0 0x56441cdbb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.483 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.477+0000 7fb623fff700 10 client.15606 ms_handle_connect on v2:172.21.15.203:6838/2888221613 2023-12-01T01:18:23.483 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.477+0000 7fb643fff700 1 --2- 172.21.15.77:0/3148462024 >> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] conn(0x56441cdb95d0 0x56441cdbb9f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d6d317fd9e4f149f server_cookie=3bbf0d0088e1c43a in_seq=0 out_seq=0 2023-12-01T01:18:23.483 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.477+0000 7fb623fff700 1 -- 172.21.15.77:0/3148462024 <== mds.1 v2:172.21.15.203:6838/2888221613 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56441cd72e10 con 0x56441cdb95d0 2023-12-01T01:18:23.483 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.477+0000 7fb623fff700 10 client.15606 handle_command_reply: tid=0 2023-12-01T01:18:23.483 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.478+0000 7fb656c12b80 1 client.15606 shutdown 2023-12-01T01:18:23.484 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.478+0000 7fb656c12b80 1 -- 172.21.15.77:0/3148462024 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56441cda43f0 msgr2=0x56441cdad980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:23.484 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.478+0000 7fb656c12b80 1 --2- 172.21.15.77:0/3148462024 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56441cda43f0 0x56441cdad980 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb638010fd0 tx=0x7fb63809ce90 comp rx=0 tx=0).stop 2023-12-01T01:18:23.484 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.478+0000 7fb656c12b80 1 -- 172.21.15.77:0/3148462024 shutdown_connections 2023-12-01T01:18:23.484 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.478+0000 7fb656c12b80 1 --2- 172.21.15.77:0/3148462024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56441cda5a70 0x56441cdb4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.484 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.478+0000 7fb656c12b80 1 --2- 172.21.15.77:0/3148462024 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56441cda43f0 0x56441cdad980 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.484 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.478+0000 7fb656c12b80 1 --2- 172.21.15.77:0/3148462024 >> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] conn(0x56441cdb95d0 0x56441cdbb9f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.484 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.478+0000 7fb656c12b80 1 --2- 172.21.15.77:0/3148462024 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56441cda50f0 0x56441cdadec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.484 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.478+0000 7fb623fff700 0 client.15606 ms_handle_reset on v2:172.21.15.203:6838/2888221613 2023-12-01T01:18:23.484 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.478+0000 7fb656c12b80 1 -- 172.21.15.77:0/3148462024 >> 172.21.15.77:0/3148462024 conn(0x56441cd030d0 msgr2=0x56441cd9e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:23.484 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.478+0000 7fb656c12b80 1 -- 172.21.15.77:0/3148462024 shutdown_connections 2023-12-01T01:18:23.484 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.478+0000 7fb656c12b80 1 -- 172.21.15.77:0/3148462024 wait complete. 2023-12-01T01:18:23.484 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.478+0000 7fb656c12b80 20 client.15606 trim_cache size 0 max 16384 2023-12-01T01:18:23.494 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.491+0000 7fb656c12b80 1 Processor -- start 2023-12-01T01:18:23.494 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.491+0000 7fb656c12b80 1 -- start start 2023-12-01T01:18:23.494 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.491+0000 7fb656c12b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56441cda43f0 0x56441cdb2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.494 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.491+0000 7fb656c12b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56441cda50f0 0x56441cdad030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.494 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.491+0000 7fb656c12b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56441cda5a70 0x56441cdac6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.495 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.491+0000 7fb656c12b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x56441cd72e10 con 0x56441cda43f0 2023-12-01T01:18:23.495 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.491+0000 7fb656c12b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56441cdace30 con 0x56441cda50f0 2023-12-01T01:18:23.495 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.491+0000 7fb656c12b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56441cda0470 con 0x56441cda5a70 2023-12-01T01:18:23.495 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.491+0000 7fb64881e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56441cda43f0 0x56441cdb2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.495 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.491+0000 7fb64881e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56441cda43f0 0x56441cdb2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59624/0 (socket says 172.21.15.77:59624) 2023-12-01T01:18:23.495 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.491+0000 7fb64881e700 1 -- 172.21.15.77:0/1710536170 learned_addr learned my addr 172.21.15.77:0/1710536170 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:23.495 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.491+0000 7fb64901f700 1 --2- 172.21.15.77:0/1710536170 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56441cda5a70 0x56441cdac6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.495 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.491+0000 7fb643fff700 1 --2- 172.21.15.77:0/1710536170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56441cda50f0 0x56441cdad030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.495 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.492+0000 7fb64901f700 1 -- 172.21.15.77:0/1710536170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56441cda50f0 msgr2=0x56441cdad030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:23.495 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.492+0000 7fb64901f700 1 --2- 172.21.15.77:0/1710536170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56441cda50f0 0x56441cdad030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.495 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.492+0000 7fb64901f700 1 -- 172.21.15.77:0/1710536170 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56441cda43f0 msgr2=0x56441cdb2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:23.495 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.492+0000 7fb64901f700 1 --2- 172.21.15.77:0/1710536170 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56441cda43f0 0x56441cdb2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.495 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.492+0000 7fb64901f700 1 -- 172.21.15.77:0/1710536170 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56441cb49250 con 0x56441cda5a70 2023-12-01T01:18:23.495 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.492+0000 7fb64881e700 1 --2- 172.21.15.77:0/1710536170 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56441cda43f0 0x56441cdb2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:18:23.496 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.492+0000 7fb64901f700 1 --2- 172.21.15.77:0/1710536170 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56441cda5a70 0x56441cdac6b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb63c007ab0 tx=0x7fb63c0074e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd91c10c21eea6e1 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:23.496 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.492+0000 7fb62b7fe700 1 -- 172.21.15.77:0/1710536170 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c00e070 con 0x56441cda5a70 2023-12-01T01:18:23.496 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.492+0000 7fb62b7fe700 1 -- 172.21.15.77:0/1710536170 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb63c007850 con 0x56441cda5a70 2023-12-01T01:18:23.496 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.492+0000 7fb62b7fe700 1 -- 172.21.15.77:0/1710536170 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c0054a0 con 0x56441cda5a70 2023-12-01T01:18:23.496 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.493+0000 7fb656c12b80 1 -- 172.21.15.77:0/1710536170 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56441cda5a70 msgr2=0x56441cdac6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:23.496 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.493+0000 7fb656c12b80 1 --2- 172.21.15.77:0/1710536170 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56441cda5a70 0x56441cdac6b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb63c007ab0 tx=0x7fb63c0074e0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.496 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.493+0000 7fb656c12b80 1 -- 172.21.15.77:0/1710536170 shutdown_connections 2023-12-01T01:18:23.496 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.493+0000 7fb656c12b80 1 --2- 172.21.15.77:0/1710536170 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56441cda5a70 0x56441cdac6b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.496 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.493+0000 7fb656c12b80 1 --2- 172.21.15.77:0/1710536170 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56441cda43f0 0x56441cdb2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.496 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.493+0000 7fb656c12b80 1 --2- 172.21.15.77:0/1710536170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56441cda50f0 0x56441cdad030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.496 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.493+0000 7fb656c12b80 1 -- 172.21.15.77:0/1710536170 >> 172.21.15.77:0/1710536170 conn(0x56441cd030d0 msgr2=0x56441cd9e910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:23.496 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.493+0000 7fb656c12b80 1 -- 172.21.15.77:0/1710536170 shutdown_connections 2023-12-01T01:18:23.496 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.493+0000 7fb656c12b80 1 -- 172.21.15.77:0/1710536170 wait complete. 2023-12-01T01:18:23.499 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.493+0000 7fb656c12b80 1 -- start start 2023-12-01T01:18:23.499 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.494+0000 7fb656c12b80 1 Processor -- start 2023-12-01T01:18:23.499 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.494+0000 7fb656c12b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56441cda43f0 0x56441cdb26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.500 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.494+0000 7fb656c12b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56441cda50f0 0x56441cdb2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.501 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.494+0000 7fb656c12b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56441cda5a70 0x56441cdb0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.501 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.494+0000 7fb656c12b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x56441cdb1250 con 0x56441cda50f0 2023-12-01T01:18:23.501 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.494+0000 7fb656c12b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56441cdb13c0 con 0x56441cda5a70 2023-12-01T01:18:23.501 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.494+0000 7fb656c12b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56441cdb1580 con 0x56441cda43f0 2023-12-01T01:18:23.501 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.494+0000 7fb656c12b80 10 client.0 _ll_register_callbacks cb 0x56441cbdea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:18:23.501 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.494+0000 7fb64881e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56441cda43f0 0x56441cdb26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.501 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.494+0000 7fb643fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56441cda50f0 0x56441cdb2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.501 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.494+0000 7fb64881e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56441cda43f0 0x56441cdb26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:45818/0 (socket says 172.21.15.77:45818) 2023-12-01T01:18:23.501 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.494+0000 7fb64901f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56441cda5a70 0x56441cdb0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.501 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.494+0000 7fb64881e700 1 -- 172.21.15.77:0/2798289115 learned_addr learned my addr 172.21.15.77:0/2798289115 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:23.501 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.495+0000 7fb64881e700 1 -- 172.21.15.77:0/2798289115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56441cda5a70 msgr2=0x56441cdb0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:23.501 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.495+0000 7fb64881e700 1 --2- 172.21.15.77:0/2798289115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56441cda5a70 0x56441cdb0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.502 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.495+0000 7fb64881e700 1 -- 172.21.15.77:0/2798289115 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56441cda50f0 msgr2=0x56441cdb2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:23.502 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.495+0000 7fb64881e700 1 --2- 172.21.15.77:0/2798289115 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56441cda50f0 0x56441cdb2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.502 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.495+0000 7fb64881e700 1 -- 172.21.15.77:0/2798289115 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56441cb0bc60 con 0x56441cda43f0 2023-12-01T01:18:23.502 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.495+0000 7fb643fff700 1 --2- 172.21.15.77:0/2798289115 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56441cda50f0 0x56441cdb2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:18:23.502 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.495+0000 7fb64881e700 1 --2- 172.21.15.77:0/2798289115 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56441cda43f0 0x56441cdb26c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb6380a1290 tx=0x7fb63809cfb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adf3c6e1073f5fa9 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:23.502 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.495+0000 7fb623fff700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2023-12-01T01:18:23.502 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.495+0000 7fb623fff700 1 -- 172.21.15.77:0/2798289115 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb638014040 con 0x56441cda43f0 2023-12-01T01:18:23.502 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.495+0000 7fb623fff700 1 -- 172.21.15.77:0/2798289115 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fb638079030 con 0x56441cda43f0 2023-12-01T01:18:23.502 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.495+0000 7fb656c12b80 1 -- 172.21.15.77:0/2798289115 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb634015070 con 0x56441cda43f0 2023-12-01T01:18:23.502 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.495+0000 7fb623fff700 1 -- 172.21.15.77:0/2798289115 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb638098ba0 con 0x56441cda43f0 2023-12-01T01:18:23.502 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.495+0000 7fb656c12b80 10 client.25222 fetch_fsmap learned FSMap version 27 2023-12-01T01:18:23.502 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.495+0000 7fb656c12b80 1 -- 172.21.15.77:0/2798289115 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fb644101a00 con 0x56441cda43f0 2023-12-01T01:18:23.502 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.495+0000 7fb623fff700 1 -- 172.21.15.77:0/2798289115 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb634015070 con 0x56441cda43f0 2023-12-01T01:18:23.502 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.496+0000 7fb623fff700 1 -- 172.21.15.77:0/2798289115 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7fb638009aa0 con 0x56441cda43f0 2023-12-01T01:18:23.503 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.496+0000 7fb656c12b80 10 client.25222 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:18:23.503 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.496+0000 7fb656c12b80 10 client.25222 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-12-01T01:18:23.503 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.496+0000 7fb656c12b80 20 client.25222 populate_metadata read hostname 'smithi077' 2023-12-01T01:18:23.503 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.496+0000 7fb656c12b80 1 --2- 172.21.15.77:0/2798289115 >> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] conn(0x56441ce0bfe0 0x56441ce0e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.503 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.496+0000 7fb656c12b80 4 client.25222 mds_command: new command op to 24496 tid=0[{"prefix": "damage ls"}] 2023-12-01T01:18:23.503 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.496+0000 7fb656c12b80 1 -- 172.21.15.77:0/2798289115 --> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56441ccedd40 con 0x56441ce0bfe0 2023-12-01T01:18:23.503 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.496+0000 7fb643fff700 1 --2- 172.21.15.77:0/2798289115 >> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] conn(0x56441ce0bfe0 0x56441ce0e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.503 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.496+0000 7fb623fff700 10 client.25222 ms_handle_connect on v2:172.21.15.203:6838/2888221613 2023-12-01T01:18:23.503 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.496+0000 7fb643fff700 1 --2- 172.21.15.77:0/2798289115 >> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] conn(0x56441ce0bfe0 0x56441ce0e400 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=78cd2c13e8df82f server_cookie=ae68dd9213b588f in_seq=0 out_seq=0 2023-12-01T01:18:23.503 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.496+0000 7fb623fff700 1 -- 172.21.15.77:0/2798289115 <== mds.1 v2:172.21.15.203:6838/2888221613 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56441ccedd40 con 0x56441ce0bfe0 2023-12-01T01:18:23.503 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.496+0000 7fb623fff700 10 client.25222 handle_command_reply: tid=0 2023-12-01T01:18:23.503 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.496+0000 7fb656c12b80 1 client.25222 shutdown 2023-12-01T01:18:23.503 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.497+0000 7fb656c12b80 1 -- 172.21.15.77:0/2798289115 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56441cda43f0 msgr2=0x56441cdb26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:23.503 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.497+0000 7fb656c12b80 1 --2- 172.21.15.77:0/2798289115 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56441cda43f0 0x56441cdb26c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb6380a1290 tx=0x7fb63809cfb0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.504 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.497+0000 7fb656c12b80 1 -- 172.21.15.77:0/2798289115 shutdown_connections 2023-12-01T01:18:23.504 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.497+0000 7fb656c12b80 1 --2- 172.21.15.77:0/2798289115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56441cda5a70 0x56441cdb0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.504 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.497+0000 7fb656c12b80 1 --2- 172.21.15.77:0/2798289115 >> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] conn(0x56441ce0bfe0 0x56441ce0e400 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.504 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.497+0000 7fb656c12b80 1 --2- 172.21.15.77:0/2798289115 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56441cda43f0 0x56441cdb26c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.504 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.497+0000 7fb623fff700 0 client.25222 ms_handle_reset on v2:172.21.15.203:6838/2888221613 2023-12-01T01:18:23.504 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.497+0000 7fb656c12b80 1 --2- 172.21.15.77:0/2798289115 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56441cda50f0 0x56441cdb2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.504 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.497+0000 7fb656c12b80 1 -- 172.21.15.77:0/2798289115 >> 172.21.15.77:0/2798289115 conn(0x56441cd030d0 msgr2=0x56441cda8350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:23.504 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.497+0000 7fb656c12b80 1 -- 172.21.15.77:0/2798289115 shutdown_connections 2023-12-01T01:18:23.504 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.497+0000 7fb656c12b80 1 -- 172.21.15.77:0/2798289115 wait complete. 2023-12-01T01:18:23.504 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.497+0000 7fb656c12b80 20 client.25222 trim_cache size 0 max 16384 2023-12-01T01:18:23.505 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2023-12-01T01:18:23.505 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.498+0000 7fb64aa82700 1 -- 172.21.15.77:0/1124428187 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fb62c0a0520 msgr2=0x7fb62c0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:23.505 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.498+0000 7fb64aa82700 1 --2- 172.21.15.77:0/1124428187 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fb62c0a0520 0x7fb62c0a29e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb634005950 tx=0x7fb63400b410 comp rx=0 tx=0).stop 2023-12-01T01:18:23.506 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.498+0000 7fb64aa82700 1 -- 172.21.15.77:0/1124428187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6440ff030 msgr2=0x7fb644198cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:23.506 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.498+0000 7fb64aa82700 1 --2- 172.21.15.77:0/1124428187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6440ff030 0x7fb644198cf0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb63800d950 tx=0x7fb63800dc60 comp rx=0 tx=0).stop 2023-12-01T01:18:23.506 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.498+0000 7fb64aa82700 1 -- 172.21.15.77:0/1124428187 shutdown_connections 2023-12-01T01:18:23.506 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.498+0000 7fb64aa82700 1 --2- 172.21.15.77:0/1124428187 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fb62c0a0520 0x7fb62c0a29e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.506 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.498+0000 7fb64aa82700 1 --2- 172.21.15.77:0/1124428187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb644100750 0x7fb6441a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.506 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.498+0000 7fb64aa82700 1 --2- 172.21.15.77:0/1124428187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6440ff030 0x7fb644198cf0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.506 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.498+0000 7fb64aa82700 1 --2- 172.21.15.77:0/1124428187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6440ffd90 0x7fb644199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.506 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.498+0000 7fb64aa82700 1 -- 172.21.15.77:0/1124428187 >> 172.21.15.77:0/1124428187 conn(0x7fb6440fa960 msgr2=0x7fb6440fc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:23.506 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.498+0000 7fb64aa82700 1 -- 172.21.15.77:0/1124428187 shutdown_connections 2023-12-01T01:18:23.506 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.498+0000 7fb64aa82700 1 -- 172.21.15.77:0/1124428187 wait complete. 2023-12-01T01:18:23.515 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-12-01T01:18:23.646 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.644+0000 7f834ba8d700 1 -- 172.21.15.77:0/3783779266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83441005e0 msgr2=0x7f8344100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:23.646 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.644+0000 7f834ba8d700 1 --2- 172.21.15.77:0/3783779266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83441005e0 0x7f8344100a40 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f83340099c0 tx=0x7f8334009cd0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.646 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.644+0000 7f834ba8d700 1 -- 172.21.15.77:0/3783779266 shutdown_connections 2023-12-01T01:18:23.646 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.644+0000 7f834ba8d700 1 --2- 172.21.15.77:0/3783779266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8344100f80 0x7f8344107830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.646 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.644+0000 7f834ba8d700 1 --2- 172.21.15.77:0/3783779266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83440ff3f0 0x7f83440ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.646 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.644+0000 7f834ba8d700 1 --2- 172.21.15.77:0/3783779266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83441005e0 0x7f8344100a40 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.646 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.644+0000 7f834ba8d700 1 -- 172.21.15.77:0/3783779266 >> 172.21.15.77:0/3783779266 conn(0x7f83440fa960 msgr2=0x7f83440fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:23.646 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.644+0000 7f834ba8d700 1 -- 172.21.15.77:0/3783779266 shutdown_connections 2023-12-01T01:18:23.646 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.645+0000 7f834ba8d700 1 -- 172.21.15.77:0/3783779266 wait complete. 2023-12-01T01:18:23.648 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.645+0000 7f834ba8d700 1 Processor -- start 2023-12-01T01:18:23.648 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.645+0000 7f834ba8d700 1 -- start start 2023-12-01T01:18:23.648 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.646+0000 7f834ba8d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83440ff3f0 0x7f83441959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.648 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.646+0000 7f834ba8d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83441005e0 0x7f8344195ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.648 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.646+0000 7f834ba8d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8344100f80 0x7f834419cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.648 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.646+0000 7f834ba8d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834419d520 con 0x7f83441005e0 2023-12-01T01:18:23.649 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.646+0000 7f834ba8d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834419d660 con 0x7f8344100f80 2023-12-01T01:18:23.649 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.646+0000 7f834ba8d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834419d7f0 con 0x7f83440ff3f0 2023-12-01T01:18:23.649 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.646+0000 7f8349028700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83441005e0 0x7f8344195ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.649 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.646+0000 7f8349028700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83441005e0 0x7f8344195ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59646/0 (socket says 172.21.15.77:59646) 2023-12-01T01:18:23.649 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.646+0000 7f8349028700 1 -- 172.21.15.77:0/668149127 learned_addr learned my addr 172.21.15.77:0/668149127 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:23.649 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.646+0000 7f834a02a700 1 --2- 172.21.15.77:0/668149127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8344100f80 0x7f834419cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.649 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.646+0000 7f8349829700 1 --2- 172.21.15.77:0/668149127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83440ff3f0 0x7f83441959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.649 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.646+0000 7f8349028700 1 -- 172.21.15.77:0/668149127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83440ff3f0 msgr2=0x7f83441959a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:23.649 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.646+0000 7f8349028700 1 --2- 172.21.15.77:0/668149127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83440ff3f0 0x7f83441959a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.649 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.646+0000 7f8349028700 1 -- 172.21.15.77:0/668149127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8344100f80 msgr2=0x7f834419cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:23.649 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.646+0000 7f8349028700 1 --2- 172.21.15.77:0/668149127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8344100f80 0x7f834419cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.649 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.646+0000 7f8349028700 1 -- 172.21.15.77:0/668149127 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83340095f0 con 0x7f83441005e0 2023-12-01T01:18:23.649 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.646+0000 7f8349028700 1 --2- 172.21.15.77:0/668149127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83441005e0 0x7f8344195ee0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8334009990 tx=0x7f833400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:23.649 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.647+0000 7f833affd700 1 -- 172.21.15.77:0/668149127 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833401d070 con 0x7f83441005e0 2023-12-01T01:18:23.649 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.647+0000 7f834ba8d700 1 -- 172.21.15.77:0/668149127 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f834410b5a0 con 0x7f83441005e0 2023-12-01T01:18:23.650 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.647+0000 7f833affd700 1 -- 172.21.15.77:0/668149127 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8334005030 con 0x7f83441005e0 2023-12-01T01:18:23.650 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.647+0000 7f833affd700 1 -- 172.21.15.77:0/668149127 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8334004a70 con 0x7f83441005e0 2023-12-01T01:18:23.650 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.647+0000 7f834ba8d700 1 -- 172.21.15.77:0/668149127 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f834410baa0 con 0x7f83441005e0 2023-12-01T01:18:23.651 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.650+0000 7f833affd700 1 -- 172.21.15.77:0/668149127 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f8334005410 con 0x7f83441005e0 2023-12-01T01:18:23.651 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.650+0000 7f833affd700 1 --2- 172.21.15.77:0/668149127 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f83300a0520 0x7f83300a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.652 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.651+0000 7f833affd700 1 -- 172.21.15.77:0/668149127 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f83340cc5d0 con 0x7f83441005e0 2023-12-01T01:18:23.652 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.651+0000 7f8349829700 1 --2- 172.21.15.77:0/668149127 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f83300a0520 0x7f83300a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.652 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.651+0000 7f8349829700 1 --2- 172.21.15.77:0/668149127 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f83300a0520 0x7f83300a29e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8344100440 tx=0x7f8340009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:23.652 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.651+0000 7f8357c1db80 1 Processor -- start 2023-12-01T01:18:23.654 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.652+0000 7f8357c1db80 1 -- start start 2023-12-01T01:18:23.654 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.652+0000 7f8357c1db80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eb62ff500 0x557eb62ff940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.654 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.652+0000 7f8357c1db80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557eb62ffe80 0x557eb62fe270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.654 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.652+0000 7f8357c1db80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557eb62fe800 0x557eb62fec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.654 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.652+0000 7f8357c1db80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eb6251a10 con 0x557eb62ffe80 2023-12-01T01:18:23.654 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.652+0000 7f8357c1db80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eb624f3d0 con 0x557eb62ff500 2023-12-01T01:18:23.654 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.652+0000 7f8357c1db80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eb624acc0 con 0x557eb62fe800 2023-12-01T01:18:23.654 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.652+0000 7f8349028700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557eb62ffe80 0x557eb62fe270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.654 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.652+0000 7f8349028700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557eb62ffe80 0x557eb62fe270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59654/0 (socket says 172.21.15.77:59654) 2023-12-01T01:18:23.654 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.652+0000 7f8349028700 1 -- 172.21.15.77:0/2853909325 learned_addr learned my addr 172.21.15.77:0/2853909325 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:23.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.652+0000 7f8349829700 1 --2- 172.21.15.77:0/2853909325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eb62ff500 0x557eb62ff940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.652+0000 7f834a02a700 1 --2- 172.21.15.77:0/2853909325 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557eb62fe800 0x557eb62fec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.652+0000 7f8349028700 1 -- 172.21.15.77:0/2853909325 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557eb62fe800 msgr2=0x557eb62fec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:23.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.652+0000 7f8349028700 1 --2- 172.21.15.77:0/2853909325 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557eb62fe800 0x557eb62fec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.652+0000 7f8349028700 1 -- 172.21.15.77:0/2853909325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eb62ff500 msgr2=0x557eb62ff940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:23.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.652+0000 7f8349028700 1 --2- 172.21.15.77:0/2853909325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eb62ff500 0x557eb62ff940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.652+0000 7f8349028700 1 -- 172.21.15.77:0/2853909325 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557eb6078de0 con 0x557eb62ffe80 2023-12-01T01:18:23.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.652+0000 7f834a02a700 1 --2- 172.21.15.77:0/2853909325 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557eb62fe800 0x557eb62fec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:18:23.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.653+0000 7f8349028700 1 --2- 172.21.15.77:0/2853909325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557eb62ffe80 0x557eb62fe270 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8334096460 tx=0x7f833409a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e45c3fe7b761abcb server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:23.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.653+0000 7f832ffff700 1 -- 172.21.15.77:0/2853909325 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833409dd90 con 0x557eb62ffe80 2023-12-01T01:18:23.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.653+0000 7f832ffff700 1 -- 172.21.15.77:0/2853909325 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83340a1f40 con 0x557eb62ffe80 2023-12-01T01:18:23.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.653+0000 7f832ffff700 1 -- 172.21.15.77:0/2853909325 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8334099690 con 0x557eb62ffe80 2023-12-01T01:18:23.658 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.653+0000 7f8357c1db80 1 -- 172.21.15.77:0/2853909325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557eb62ffe80 msgr2=0x557eb62fe270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:23.658 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.653+0000 7f8357c1db80 1 --2- 172.21.15.77:0/2853909325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557eb62ffe80 0x557eb62fe270 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8334096460 tx=0x7f833409a660 comp rx=0 tx=0).stop 2023-12-01T01:18:23.658 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.654+0000 7f8357c1db80 1 -- 172.21.15.77:0/2853909325 shutdown_connections 2023-12-01T01:18:23.658 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.654+0000 7f8357c1db80 1 --2- 172.21.15.77:0/2853909325 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557eb62fe800 0x557eb62fec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.658 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.654+0000 7f8357c1db80 1 --2- 172.21.15.77:0/2853909325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eb62ff500 0x557eb62ff940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.659 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.654+0000 7f8357c1db80 1 --2- 172.21.15.77:0/2853909325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557eb62ffe80 0x557eb62fe270 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.659 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.654+0000 7f8357c1db80 1 -- 172.21.15.77:0/2853909325 >> 172.21.15.77:0/2853909325 conn(0x557eb62590d0 msgr2=0x557eb62f4940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:23.659 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.654+0000 7f8357c1db80 1 -- 172.21.15.77:0/2853909325 shutdown_connections 2023-12-01T01:18:23.659 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.654+0000 7f8357c1db80 1 -- 172.21.15.77:0/2853909325 wait complete. 2023-12-01T01:18:23.659 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.654+0000 7f8357c1db80 1 -- start start 2023-12-01T01:18:23.659 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.654+0000 7f8357c1db80 1 Processor -- start 2023-12-01T01:18:23.659 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.655+0000 7f8357c1db80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557eb62fe800 0x557eb6307a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.659 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.655+0000 7f8357c1db80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eb62ff500 0x557eb6307fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.659 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.655+0000 7f8357c1db80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557eb62ffe80 0x557eb630f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.659 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.655+0000 7f8357c1db80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eb61e7510 con 0x557eb62fe800 2023-12-01T01:18:23.659 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.655+0000 7f8357c1db80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eb61e71b0 con 0x557eb62ff500 2023-12-01T01:18:23.659 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.655+0000 7f8357c1db80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eb60ce920 con 0x557eb62ffe80 2023-12-01T01:18:23.659 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.655+0000 7f8357c1db80 10 client.0 _ll_register_callbacks cb 0x557eb6134a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:18:23.659 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.655+0000 7f8349829700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557eb62fe800 0x557eb6307a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.659 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.655+0000 7f8349829700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557eb62fe800 0x557eb6307a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59658/0 (socket says 172.21.15.77:59658) 2023-12-01T01:18:23.660 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.655+0000 7f8349829700 1 -- 172.21.15.77:0/1118461362 learned_addr learned my addr 172.21.15.77:0/1118461362 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:23.660 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.655+0000 7f8349028700 1 --2- 172.21.15.77:0/1118461362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eb62ff500 0x557eb6307fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.660 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.655+0000 7f834a02a700 1 --2- 172.21.15.77:0/1118461362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557eb62ffe80 0x557eb630f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.660 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.655+0000 7f8349829700 1 -- 172.21.15.77:0/1118461362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557eb62ffe80 msgr2=0x557eb630f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:23.660 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.655+0000 7f8349829700 1 --2- 172.21.15.77:0/1118461362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557eb62ffe80 0x557eb630f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.660 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.655+0000 7f8349829700 1 -- 172.21.15.77:0/1118461362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eb62ff500 msgr2=0x557eb6307fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:23.660 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.655+0000 7f8349829700 1 --2- 172.21.15.77:0/1118461362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eb62ff500 0x557eb6307fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.660 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.655+0000 7f8349829700 1 -- 172.21.15.77:0/1118461362 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f834410baa0 con 0x557eb62fe800 2023-12-01T01:18:23.660 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.655+0000 7f834a02a700 1 --2- 172.21.15.77:0/1118461362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557eb62ffe80 0x557eb630f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:18:23.660 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.656+0000 7f8349829700 1 --2- 172.21.15.77:0/1118461362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557eb62fe800 0x557eb6307a80 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f834001d8d0 tx=0x7f83400047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f35a82d7aba488 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:23.660 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.656+0000 7f832d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:18:23.660 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.656+0000 7f832d7fa700 1 -- 172.21.15.77:0/1118461362 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8340022070 con 0x557eb62fe800 2023-12-01T01:18:23.660 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.656+0000 7f8357c1db80 1 -- 172.21.15.77:0/1118461362 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557eb6275d10 con 0x557eb62fe800 2023-12-01T01:18:23.660 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.656+0000 7f832d7fa700 1 -- 172.21.15.77:0/1118461362 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f8340004020 con 0x557eb62fe800 2023-12-01T01:18:23.661 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.656+0000 7f832d7fa700 1 -- 172.21.15.77:0/1118461362 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83400041e0 con 0x557eb62fe800 2023-12-01T01:18:23.661 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.656+0000 7f8357c1db80 10 client.15624 fetch_fsmap learned FSMap version 27 2023-12-01T01:18:23.661 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.656+0000 7f8357c1db80 1 -- 172.21.15.77:0/1118461362 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557eb6061c60 con 0x557eb62fe800 2023-12-01T01:18:23.661 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.656+0000 7f832d7fa700 1 -- 172.21.15.77:0/1118461362 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557eb6275d10 con 0x557eb62fe800 2023-12-01T01:18:23.661 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.656+0000 7f832d7fa700 1 -- 172.21.15.77:0/1118461362 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f834001c720 con 0x557eb62fe800 2023-12-01T01:18:23.661 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.656+0000 7f8357c1db80 10 client.15624 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:18:23.661 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.656+0000 7f8357c1db80 10 client.15624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-01T01:18:23.661 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.656+0000 7f8357c1db80 20 client.15624 populate_metadata read hostname 'smithi077' 2023-12-01T01:18:23.661 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.656+0000 7f8357c1db80 1 --2- 172.21.15.77:0/1118461362 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x557eb6313770 0x557eb6315b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.661 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.656+0000 7f8357c1db80 4 client.15624 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:18:23.661 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.656+0000 7f8357c1db80 1 -- 172.21.15.77:0/1118461362 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557eb62c8e10 con 0x557eb6313770 2023-12-01T01:18:23.661 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.657+0000 7f8349028700 1 --2- 172.21.15.77:0/1118461362 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x557eb6313770 0x557eb6315b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.661 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.657+0000 7f832d7fa700 10 client.15624 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:23.661 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.657+0000 7f8349028700 1 --2- 172.21.15.77:0/1118461362 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x557eb6313770 0x557eb6315b90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9ce5bf0ceb6b90a server_cookie=3aa765da1d830020 in_seq=0 out_seq=0 2023-12-01T01:18:23.662 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.658+0000 7f832d7fa700 1 -- 172.21.15.77:0/1118461362 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x557eb62c8e10 con 0x557eb6313770 2023-12-01T01:18:23.662 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.658+0000 7f832d7fa700 10 client.15624 handle_command_reply: tid=0 2023-12-01T01:18:23.663 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.658+0000 7f8357c1db80 1 client.15624 shutdown 2023-12-01T01:18:23.663 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.658+0000 7f8357c1db80 1 -- 172.21.15.77:0/1118461362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557eb62fe800 msgr2=0x557eb6307a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:23.663 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.658+0000 7f8357c1db80 1 --2- 172.21.15.77:0/1118461362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557eb62fe800 0x557eb6307a80 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f834001d8d0 tx=0x7f83400047c0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.663 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.658+0000 7f8357c1db80 1 -- 172.21.15.77:0/1118461362 shutdown_connections 2023-12-01T01:18:23.663 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.658+0000 7f8357c1db80 1 --2- 172.21.15.77:0/1118461362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557eb62ffe80 0x557eb630f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.663 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.658+0000 7f8357c1db80 1 --2- 172.21.15.77:0/1118461362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557eb62fe800 0x557eb6307a80 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.663 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.658+0000 7f8357c1db80 1 --2- 172.21.15.77:0/1118461362 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x557eb6313770 0x557eb6315b90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.663 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.658+0000 7f8357c1db80 1 --2- 172.21.15.77:0/1118461362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eb62ff500 0x557eb6307fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.663 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.658+0000 7f832d7fa700 0 client.15624 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:23.663 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.658+0000 7f8357c1db80 1 -- 172.21.15.77:0/1118461362 >> 172.21.15.77:0/1118461362 conn(0x557eb62590d0 msgr2=0x557eb62f40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:23.663 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.658+0000 7f8357c1db80 1 -- 172.21.15.77:0/1118461362 shutdown_connections 2023-12-01T01:18:23.663 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.658+0000 7f8357c1db80 1 -- 172.21.15.77:0/1118461362 wait complete. 2023-12-01T01:18:23.663 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.658+0000 7f8357c1db80 20 client.15624 trim_cache size 0 max 16384 2023-12-01T01:18:23.684 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.676+0000 7f8357c1db80 1 Processor -- start 2023-12-01T01:18:23.684 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.676+0000 7f8357c1db80 1 -- start start 2023-12-01T01:18:23.684 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.676+0000 7f8357c1db80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eb62fe800 0x557eb62f59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.684 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.676+0000 7f8357c1db80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557eb62ff500 0x557eb630cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.684 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.676+0000 7f8357c1db80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557eb62ffe80 0x557eb6307130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.684 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.676+0000 7f8357c1db80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eb6275d10 con 0x557eb62ffe80 2023-12-01T01:18:23.684 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.676+0000 7f8357c1db80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eb6308c70 con 0x557eb62fe800 2023-12-01T01:18:23.684 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.676+0000 7f8357c1db80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eb62f6450 con 0x557eb62ff500 2023-12-01T01:18:23.685 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.676+0000 7f834a02a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557eb62ffe80 0x557eb6307130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.685 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.676+0000 7f834a02a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557eb62ffe80 0x557eb6307130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59670/0 (socket says 172.21.15.77:59670) 2023-12-01T01:18:23.685 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.676+0000 7f834a02a700 1 -- 172.21.15.77:0/468647908 learned_addr learned my addr 172.21.15.77:0/468647908 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:23.685 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.676+0000 7f8349028700 1 --2- 172.21.15.77:0/468647908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557eb62ff500 0x557eb630cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.685 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.676+0000 7f8349829700 1 --2- 172.21.15.77:0/468647908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eb62fe800 0x557eb62f59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.685 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.677+0000 7f834a02a700 1 -- 172.21.15.77:0/468647908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557eb62ff500 msgr2=0x557eb630cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:23.685 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.677+0000 7f834a02a700 1 --2- 172.21.15.77:0/468647908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557eb62ff500 0x557eb630cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.685 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.677+0000 7f834a02a700 1 -- 172.21.15.77:0/468647908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eb62fe800 msgr2=0x557eb62f59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:23.685 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.677+0000 7f834a02a700 1 --2- 172.21.15.77:0/468647908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eb62fe800 0x557eb62f59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.685 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.677+0000 7f834a02a700 1 -- 172.21.15.77:0/468647908 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557eb609f250 con 0x557eb62ffe80 2023-12-01T01:18:23.685 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.677+0000 7f8349028700 1 --2- 172.21.15.77:0/468647908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557eb62ff500 0x557eb630cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:18:23.685 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.677+0000 7f834a02a700 1 --2- 172.21.15.77:0/468647908 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557eb62ffe80 0x557eb6307130 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f833c0060b0 tx=0x7f833c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c0bb583397dddb server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:23.685 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.677+0000 7f8338da9700 1 -- 172.21.15.77:0/468647908 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833c00e070 con 0x557eb62ffe80 2023-12-01T01:18:23.685 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.677+0000 7f8338da9700 1 -- 172.21.15.77:0/468647908 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f833c012b00 con 0x557eb62ffe80 2023-12-01T01:18:23.686 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.677+0000 7f8338da9700 1 -- 172.21.15.77:0/468647908 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833c01bbc0 con 0x557eb62ffe80 2023-12-01T01:18:23.686 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.678+0000 7f8357c1db80 1 -- 172.21.15.77:0/468647908 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557eb62ffe80 msgr2=0x557eb6307130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:23.686 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.678+0000 7f8357c1db80 1 --2- 172.21.15.77:0/468647908 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557eb62ffe80 0x557eb6307130 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f833c0060b0 tx=0x7f833c00b040 comp rx=0 tx=0).stop 2023-12-01T01:18:23.686 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.678+0000 7f8357c1db80 1 -- 172.21.15.77:0/468647908 shutdown_connections 2023-12-01T01:18:23.686 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.678+0000 7f8357c1db80 1 --2- 172.21.15.77:0/468647908 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557eb62ffe80 0x557eb6307130 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.686 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.678+0000 7f8357c1db80 1 --2- 172.21.15.77:0/468647908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eb62fe800 0x557eb62f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.686 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.678+0000 7f8357c1db80 1 --2- 172.21.15.77:0/468647908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557eb62ff500 0x557eb630cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.686 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.678+0000 7f8357c1db80 1 -- 172.21.15.77:0/468647908 >> 172.21.15.77:0/468647908 conn(0x557eb62590d0 msgr2=0x557eb6305c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:23.686 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.678+0000 7f8357c1db80 1 -- 172.21.15.77:0/468647908 shutdown_connections 2023-12-01T01:18:23.686 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.678+0000 7f8357c1db80 1 -- 172.21.15.77:0/468647908 wait complete. 2023-12-01T01:18:23.686 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.678+0000 7f8357c1db80 1 -- start start 2023-12-01T01:18:23.687 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.679+0000 7f8357c1db80 1 Processor -- start 2023-12-01T01:18:23.687 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.679+0000 7f8357c1db80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557eb62fe800 0x557eb630ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.687 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.679+0000 7f8357c1db80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eb62ff500 0x557eb630b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.687 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.679+0000 7f8357c1db80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557eb62ffe80 0x557eb630c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.687 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.679+0000 7f8357c1db80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eb630c6a0 con 0x557eb62fe800 2023-12-01T01:18:23.687 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.679+0000 7f8357c1db80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eb630c810 con 0x557eb62ff500 2023-12-01T01:18:23.687 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.679+0000 7f8357c1db80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x557eb630c9d0 con 0x557eb62ffe80 2023-12-01T01:18:23.687 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.679+0000 7f8357c1db80 10 client.0 _ll_register_callbacks cb 0x557eb6134a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:18:23.687 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.679+0000 7f8349829700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557eb62fe800 0x557eb630ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.688 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.679+0000 7f8349829700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557eb62fe800 0x557eb630ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59672/0 (socket says 172.21.15.77:59672) 2023-12-01T01:18:23.688 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.679+0000 7f8349829700 1 -- 172.21.15.77:0/3645351565 learned_addr learned my addr 172.21.15.77:0/3645351565 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:23.688 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.679+0000 7f834a02a700 1 --2- 172.21.15.77:0/3645351565 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557eb62ffe80 0x557eb630c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.688 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.679+0000 7f8349829700 1 -- 172.21.15.77:0/3645351565 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557eb62ffe80 msgr2=0x557eb630c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:23.688 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.679+0000 7f8349829700 1 --2- 172.21.15.77:0/3645351565 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557eb62ffe80 0x557eb630c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.688 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.679+0000 7f8349829700 1 -- 172.21.15.77:0/3645351565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eb62ff500 msgr2=0x557eb630b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:23.688 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.679+0000 7f8349829700 1 --2- 172.21.15.77:0/3645351565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eb62ff500 0x557eb630b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.688 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.679+0000 7f8349829700 1 -- 172.21.15.77:0/3645351565 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557eb6078de0 con 0x557eb62fe800 2023-12-01T01:18:23.688 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.680+0000 7f8349829700 1 --2- 172.21.15.77:0/3645351565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557eb62fe800 0x557eb630ac10 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f834001d8a0 tx=0x7f834001cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5afe134dd153e27b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:23.688 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.680+0000 7f832d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:18:23.688 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.680+0000 7f832d7fa700 1 -- 172.21.15.77:0/3645351565 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8340022040 con 0x557eb62fe800 2023-12-01T01:18:23.688 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.680+0000 7f832d7fa700 1 -- 172.21.15.77:0/3645351565 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f8340031890 con 0x557eb62fe800 2023-12-01T01:18:23.688 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.680+0000 7f8357c1db80 1 -- 172.21.15.77:0/3645351565 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f83340a1f40 con 0x557eb62fe800 2023-12-01T01:18:23.688 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.680+0000 7f832d7fa700 1 -- 172.21.15.77:0/3645351565 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8340031a50 con 0x557eb62fe800 2023-12-01T01:18:23.689 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.680+0000 7f8357c1db80 10 client.15630 fetch_fsmap learned FSMap version 27 2023-12-01T01:18:23.689 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.680+0000 7f8357c1db80 1 -- 172.21.15.77:0/3645351565 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f833c009280 con 0x557eb62fe800 2023-12-01T01:18:23.689 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.680+0000 7f832d7fa700 1 -- 172.21.15.77:0/3645351565 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83340a1f40 con 0x557eb62fe800 2023-12-01T01:18:23.689 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.680+0000 7f832d7fa700 1 -- 172.21.15.77:0/3645351565 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f834003bca0 con 0x557eb62fe800 2023-12-01T01:18:23.689 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.680+0000 7f8357c1db80 10 client.15630 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:18:23.689 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.680+0000 7f8357c1db80 10 client.15630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-01T01:18:23.689 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.680+0000 7f8357c1db80 20 client.15630 populate_metadata read hostname 'smithi077' 2023-12-01T01:18:23.689 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.680+0000 7f8357c1db80 1 --2- 172.21.15.77:0/3645351565 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x557eb63660f0 0x557eb6368510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:23.689 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.681+0000 7f8357c1db80 4 client.15630 mds_command: new command op to 24500 tid=0[{"prefix": "damage ls"}] 2023-12-01T01:18:23.689 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.681+0000 7f8357c1db80 1 -- 172.21.15.77:0/3645351565 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557eb6243d40 con 0x557eb63660f0 2023-12-01T01:18:23.689 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.681+0000 7f8349028700 1 --2- 172.21.15.77:0/3645351565 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x557eb63660f0 0x557eb6368510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:23.689 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.681+0000 7f832d7fa700 10 client.15630 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:23.689 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.681+0000 7f8349028700 1 --2- 172.21.15.77:0/3645351565 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x557eb63660f0 0x557eb6368510 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98110d89aeddef6b server_cookie=3c8f07fab26e80f9 in_seq=0 out_seq=0 2023-12-01T01:18:23.689 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.682+0000 7f832d7fa700 1 -- 172.21.15.77:0/3645351565 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557eb6243d40 con 0x557eb63660f0 2023-12-01T01:18:23.689 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.682+0000 7f832d7fa700 10 client.15630 handle_command_reply: tid=0 2023-12-01T01:18:23.690 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.682+0000 7f8357c1db80 1 client.15630 shutdown 2023-12-01T01:18:23.690 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.682+0000 7f8357c1db80 1 -- 172.21.15.77:0/3645351565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557eb62fe800 msgr2=0x557eb630ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:23.690 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.682+0000 7f8357c1db80 1 --2- 172.21.15.77:0/3645351565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557eb62fe800 0x557eb630ac10 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f834001d8a0 tx=0x7f834001cf70 comp rx=0 tx=0).stop 2023-12-01T01:18:23.690 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.682+0000 7f8357c1db80 1 -- 172.21.15.77:0/3645351565 shutdown_connections 2023-12-01T01:18:23.690 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.682+0000 7f8357c1db80 1 --2- 172.21.15.77:0/3645351565 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557eb62ffe80 0x557eb630c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.690 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.682+0000 7f8357c1db80 1 --2- 172.21.15.77:0/3645351565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557eb62fe800 0x557eb630ac10 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.690 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.682+0000 7f8357c1db80 1 --2- 172.21.15.77:0/3645351565 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x557eb63660f0 0x557eb6368510 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.690 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.682+0000 7f8357c1db80 1 --2- 172.21.15.77:0/3645351565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557eb62ff500 0x557eb630b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.690 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.682+0000 7f832d7fa700 0 client.15630 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:23.690 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.682+0000 7f8357c1db80 1 -- 172.21.15.77:0/3645351565 >> 172.21.15.77:0/3645351565 conn(0x557eb62590d0 msgr2=0x557eb6307830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:23.690 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.682+0000 7f8357c1db80 1 -- 172.21.15.77:0/3645351565 shutdown_connections 2023-12-01T01:18:23.690 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.682+0000 7f8357c1db80 1 -- 172.21.15.77:0/3645351565 wait complete. 2023-12-01T01:18:23.690 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.682+0000 7f8357c1db80 20 client.15630 trim_cache size 0 max 16384 2023-12-01T01:18:23.691 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2023-12-01T01:18:23.693 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.683+0000 7f834ba8d700 1 -- 172.21.15.77:0/668149127 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f83300a0520 msgr2=0x7f83300a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:23.693 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.683+0000 7f834ba8d700 1 --2- 172.21.15.77:0/668149127 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f83300a0520 0x7f83300a29e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8344100440 tx=0x7f8340009260 comp rx=0 tx=0).stop 2023-12-01T01:18:23.694 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.683+0000 7f834ba8d700 1 -- 172.21.15.77:0/668149127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83441005e0 msgr2=0x7f8344195ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:23.694 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.683+0000 7f834ba8d700 1 --2- 172.21.15.77:0/668149127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83441005e0 0x7f8344195ee0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8334009990 tx=0x7f833400f690 comp rx=0 tx=0).stop 2023-12-01T01:18:23.694 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.683+0000 7f834ba8d700 1 -- 172.21.15.77:0/668149127 shutdown_connections 2023-12-01T01:18:23.694 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.683+0000 7f834ba8d700 1 --2- 172.21.15.77:0/668149127 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f83300a0520 0x7f83300a29e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.694 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.683+0000 7f834ba8d700 1 --2- 172.21.15.77:0/668149127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8344100f80 0x7f834419cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.694 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.683+0000 7f834ba8d700 1 --2- 172.21.15.77:0/668149127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83440ff3f0 0x7f83441959a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.694 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.683+0000 7f834ba8d700 1 --2- 172.21.15.77:0/668149127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83441005e0 0x7f8344195ee0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:23.694 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.683+0000 7f834ba8d700 1 -- 172.21.15.77:0/668149127 >> 172.21.15.77:0/668149127 conn(0x7f83440fa960 msgr2=0x7f83440fc3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:23.694 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.683+0000 7f834ba8d700 1 -- 172.21.15.77:0/668149127 shutdown_connections 2023-12-01T01:18:23.694 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:23.683+0000 7f834ba8d700 1 -- 172.21.15.77:0/668149127 wait complete. 2023-12-01T01:18:23.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:23 smithi077 ceph-mon[98867]: pgmap v313: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2023-12-01T01:18:23.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:23 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/1915570610' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:18:24.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:23 smithi203 ceph-mon[103581]: pgmap v313: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2023-12-01T01:18:24.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:23 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/1915570610' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:18:24.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:23 smithi137 ceph-mon[103312]: pgmap v313: 97 pgs: 97 active+clean; 652 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2023-12-01T01:18:24.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:23 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/1915570610' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:18:24.700 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-12-01T01:18:24.701 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-12-01T01:18:24.897 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.895+0000 7efe43a85700 1 -- 172.21.15.77:0/931007446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe3c0ffe90 msgr2=0x7efe3c100310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:24.897 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.895+0000 7efe43a85700 1 --2- 172.21.15.77:0/931007446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe3c0ffe90 0x7efe3c100310 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efe2c009990 tx=0x7efe2c009ca0 comp rx=0 tx=0).stop 2023-12-01T01:18:24.897 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.896+0000 7efe43a85700 1 -- 172.21.15.77:0/931007446 shutdown_connections 2023-12-01T01:18:24.897 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.896+0000 7efe43a85700 1 --2- 172.21.15.77:0/931007446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe3c100850 0x7efe3c1080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:24.898 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.896+0000 7efe43a85700 1 --2- 172.21.15.77:0/931007446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe3c101240 0x7efe3c101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:24.898 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.896+0000 7efe43a85700 1 --2- 172.21.15.77:0/931007446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe3c0ffe90 0x7efe3c100310 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:24.898 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.896+0000 7efe43a85700 1 -- 172.21.15.77:0/931007446 >> 172.21.15.77:0/931007446 conn(0x7efe3c0fa960 msgr2=0x7efe3c0fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:24.898 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.896+0000 7efe43a85700 1 -- 172.21.15.77:0/931007446 shutdown_connections 2023-12-01T01:18:24.898 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.896+0000 7efe43a85700 1 -- 172.21.15.77:0/931007446 wait complete. 2023-12-01T01:18:24.900 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.896+0000 7efe43a85700 1 Processor -- start 2023-12-01T01:18:24.900 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.896+0000 7efe43a85700 1 -- start start 2023-12-01T01:18:24.900 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.897+0000 7efe43a85700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe3c0ffe90 0x7efe3c196ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:24.900 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.897+0000 7efe43a85700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe3c100850 0x7efe3c196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:24.900 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.897+0000 7efe43a85700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe3c101240 0x7efe3c19e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:24.901 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.897+0000 7efe43a85700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c19e630 con 0x7efe3c100850 2023-12-01T01:18:24.901 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.897+0000 7efe43a85700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c19e770 con 0x7efe3c101240 2023-12-01T01:18:24.901 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.897+0000 7efe43a85700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c1088b0 con 0x7efe3c0ffe90 2023-12-01T01:18:24.901 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.897+0000 7efe41821700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe3c0ffe90 0x7efe3c196ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:24.901 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.897+0000 7efe41821700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe3c0ffe90 0x7efe3c196ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:45874/0 (socket says 172.21.15.77:45874) 2023-12-01T01:18:24.901 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.897+0000 7efe42022700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe3c101240 0x7efe3c19e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:24.901 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.897+0000 7efe41821700 1 -- 172.21.15.77:0/907909679 learned_addr learned my addr 172.21.15.77:0/907909679 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:24.901 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.897+0000 7efe41020700 1 --2- 172.21.15.77:0/907909679 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe3c100850 0x7efe3c196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:24.901 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.897+0000 7efe41821700 1 -- 172.21.15.77:0/907909679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe3c101240 msgr2=0x7efe3c19e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:24.901 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.897+0000 7efe41821700 1 --2- 172.21.15.77:0/907909679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe3c101240 0x7efe3c19e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:24.901 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.897+0000 7efe41821700 1 -- 172.21.15.77:0/907909679 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe3c100850 msgr2=0x7efe3c196ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:24.901 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.897+0000 7efe41821700 1 --2- 172.21.15.77:0/907909679 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe3c100850 0x7efe3c196ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:24.902 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.897+0000 7efe41821700 1 -- 172.21.15.77:0/907909679 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe2c0095f0 con 0x7efe3c0ffe90 2023-12-01T01:18:24.902 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.897+0000 7efe41020700 1 --2- 172.21.15.77:0/907909679 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe3c100850 0x7efe3c196ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:18:24.902 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.898+0000 7efe41821700 1 --2- 172.21.15.77:0/907909679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe3c0ffe90 0x7efe3c196ab0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efe3800d950 tx=0x7efe3800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:24.902 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.898+0000 7efe32ffd700 1 -- 172.21.15.77:0/907909679 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe38009940 con 0x7efe3c0ffe90 2023-12-01T01:18:24.902 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.898+0000 7efe32ffd700 1 -- 172.21.15.77:0/907909679 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe38010460 con 0x7efe3c0ffe90 2023-12-01T01:18:24.902 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.898+0000 7efe43a85700 1 -- 172.21.15.77:0/907909679 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe3c108b90 con 0x7efe3c0ffe90 2023-12-01T01:18:24.902 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.898+0000 7efe43a85700 1 -- 172.21.15.77:0/907909679 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe3c1090f0 con 0x7efe3c0ffe90 2023-12-01T01:18:24.903 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.898+0000 7efe32ffd700 1 -- 172.21.15.77:0/907909679 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3800f620 con 0x7efe3c0ffe90 2023-12-01T01:18:24.903 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.901+0000 7efe32ffd700 1 -- 172.21.15.77:0/907909679 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7efe380105d0 con 0x7efe3c0ffe90 2023-12-01T01:18:24.903 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.902+0000 7efe32ffd700 1 --2- 172.21.15.77:0/907909679 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7efe280a05a0 0x7efe280a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:24.903 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.902+0000 7efe32ffd700 1 -- 172.21.15.77:0/907909679 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7efe380cb9f0 con 0x7efe3c0ffe90 2023-12-01T01:18:24.904 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.902+0000 7efe41020700 1 --2- 172.21.15.77:0/907909679 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7efe280a05a0 0x7efe280a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:24.904 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.903+0000 7efe41020700 1 --2- 172.21.15.77:0/907909679 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7efe280a05a0 0x7efe280a2a60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efe2c000c00 tx=0x7efe2c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:24.907 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.906+0000 7efe4fc15b80 1 Processor -- start 2023-12-01T01:18:24.909 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.906+0000 7efe4fc15b80 1 -- start start 2023-12-01T01:18:24.909 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.907+0000 7efe4fc15b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562733c3170 0x5562733c35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:24.909 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.907+0000 7efe4fc15b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562733c3af0 0x5562733c1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:24.909 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.907+0000 7efe4fc15b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5562733c2470 0x5562733c28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:24.909 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.907+0000 7efe4fc15b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x556273319a90 con 0x5562733c3af0 2023-12-01T01:18:24.909 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.907+0000 7efe4fc15b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556273317450 con 0x5562733c3170 2023-12-01T01:18:24.909 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.907+0000 7efe4fc15b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x556273312d40 con 0x5562733c2470 2023-12-01T01:18:24.909 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.907+0000 7efe41821700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562733c3170 0x5562733c35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:24.910 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.907+0000 7efe41821700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562733c3170 0x5562733c35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:38616/0 (socket says 172.21.15.77:38616) 2023-12-01T01:18:24.910 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.907+0000 7efe41821700 1 -- 172.21.15.77:0/1890067028 learned_addr learned my addr 172.21.15.77:0/1890067028 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:24.910 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.907+0000 7efe41020700 1 --2- 172.21.15.77:0/1890067028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562733c3af0 0x5562733c1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:24.910 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.907+0000 7efe42022700 1 --2- 172.21.15.77:0/1890067028 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5562733c2470 0x5562733c28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:24.910 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.907+0000 7efe41821700 1 -- 172.21.15.77:0/1890067028 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5562733c2470 msgr2=0x5562733c28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:24.910 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.907+0000 7efe41821700 1 --2- 172.21.15.77:0/1890067028 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5562733c2470 0x5562733c28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:24.910 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.907+0000 7efe41821700 1 -- 172.21.15.77:0/1890067028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562733c3af0 msgr2=0x5562733c1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:24.910 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.907+0000 7efe41821700 1 --2- 172.21.15.77:0/1890067028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562733c3af0 0x5562733c1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:24.910 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.907+0000 7efe41821700 1 -- 172.21.15.77:0/1890067028 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556273140e60 con 0x5562733c3170 2023-12-01T01:18:24.910 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.907+0000 7efe42022700 1 --2- 172.21.15.77:0/1890067028 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5562733c2470 0x5562733c28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:18:24.910 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.907+0000 7efe41821700 1 --2- 172.21.15.77:0/1890067028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562733c3170 0x5562733c35b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efe380957f0 tx=0x7efe38099b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a705eae44efdc2 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:24.910 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.908+0000 7efe27fff700 1 -- 172.21.15.77:0/1890067028 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3809d120 con 0x5562733c3170 2023-12-01T01:18:24.910 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.908+0000 7efe27fff700 1 -- 172.21.15.77:0/1890067028 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe380a1360 con 0x5562733c3170 2023-12-01T01:18:24.911 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.908+0000 7efe27fff700 1 -- 172.21.15.77:0/1890067028 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe380987e0 con 0x5562733c3170 2023-12-01T01:18:24.913 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.909+0000 7efe4fc15b80 1 -- 172.21.15.77:0/1890067028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562733c3170 msgr2=0x5562733c35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:24.914 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.909+0000 7efe4fc15b80 1 --2- 172.21.15.77:0/1890067028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562733c3170 0x5562733c35b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efe380957f0 tx=0x7efe38099b20 comp rx=0 tx=0).stop 2023-12-01T01:18:24.914 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.909+0000 7efe4fc15b80 1 -- 172.21.15.77:0/1890067028 shutdown_connections 2023-12-01T01:18:24.914 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.909+0000 7efe4fc15b80 1 --2- 172.21.15.77:0/1890067028 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5562733c2470 0x5562733c28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:24.914 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.909+0000 7efe4fc15b80 1 --2- 172.21.15.77:0/1890067028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562733c3170 0x5562733c35b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:24.915 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.909+0000 7efe4fc15b80 1 --2- 172.21.15.77:0/1890067028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562733c3af0 0x5562733c1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:24.915 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.909+0000 7efe4fc15b80 1 -- 172.21.15.77:0/1890067028 >> 172.21.15.77:0/1890067028 conn(0x556273321150 msgr2=0x5562733bc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:24.915 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.909+0000 7efe4fc15b80 1 -- 172.21.15.77:0/1890067028 shutdown_connections 2023-12-01T01:18:24.915 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.909+0000 7efe4fc15b80 1 -- 172.21.15.77:0/1890067028 wait complete. 2023-12-01T01:18:24.915 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.909+0000 7efe4fc15b80 1 -- start start 2023-12-01T01:18:24.915 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.909+0000 7efe4fc15b80 1 Processor -- start 2023-12-01T01:18:24.915 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.910+0000 7efe4fc15b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5562733c2470 0x5562733cba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:24.915 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.910+0000 7efe4fc15b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562733c3170 0x5562733cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:24.915 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.910+0000 7efe4fc15b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562733c3af0 0x5562733d2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:24.915 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.910+0000 7efe4fc15b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562732af590 con 0x5562733c3af0 2023-12-01T01:18:24.915 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.910+0000 7efe4fc15b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562732af230 con 0x5562733c3170 2023-12-01T01:18:24.915 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.910+0000 7efe4fc15b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562731969a0 con 0x5562733c2470 2023-12-01T01:18:24.915 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.910+0000 7efe4fc15b80 10 client.0 _ll_register_callbacks cb 0x5562731fcb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:18:24.915 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.910+0000 7efe42022700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562733c3af0 0x5562733d2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:24.916 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.910+0000 7efe42022700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562733c3af0 0x5562733d2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59702/0 (socket says 172.21.15.77:59702) 2023-12-01T01:18:24.916 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.910+0000 7efe42022700 1 -- 172.21.15.77:0/704522945 learned_addr learned my addr 172.21.15.77:0/704522945 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:24.916 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.910+0000 7efe41020700 1 --2- 172.21.15.77:0/704522945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562733c3170 0x5562733cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:24.916 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.910+0000 7efe41821700 1 --2- 172.21.15.77:0/704522945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5562733c2470 0x5562733cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:24.916 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.911+0000 7efe42022700 1 -- 172.21.15.77:0/704522945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5562733c2470 msgr2=0x5562733cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:24.916 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.911+0000 7efe42022700 1 --2- 172.21.15.77:0/704522945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5562733c2470 0x5562733cba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:24.916 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.911+0000 7efe42022700 1 -- 172.21.15.77:0/704522945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562733c3170 msgr2=0x5562733cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:24.916 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.911+0000 7efe42022700 1 --2- 172.21.15.77:0/704522945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562733c3170 0x5562733cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:24.916 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.911+0000 7efe42022700 1 -- 172.21.15.77:0/704522945 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe3c1090f0 con 0x5562733c3af0 2023-12-01T01:18:24.916 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.911+0000 7efe41020700 1 --2- 172.21.15.77:0/704522945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562733c3170 0x5562733cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:18:24.916 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.911+0000 7efe42022700 1 --2- 172.21.15.77:0/704522945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562733c3af0 0x5562733d2f80 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7efe34007f00 tx=0x7efe34007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=881127ae1c45d969 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:24.916 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.911+0000 7efe257fa700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:18:24.916 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.911+0000 7efe257fa700 1 -- 172.21.15.77:0/704522945 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3400e040 con 0x5562733c3af0 2023-12-01T01:18:24.916 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.911+0000 7efe257fa700 1 -- 172.21.15.77:0/704522945 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7efe34009320 con 0x5562733c3af0 2023-12-01T01:18:24.917 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.911+0000 7efe4fc15b80 1 -- 172.21.15.77:0/704522945 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55627333dd90 con 0x5562733c3af0 2023-12-01T01:18:24.917 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.911+0000 7efe257fa700 1 -- 172.21.15.77:0/704522945 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe34013890 con 0x5562733c3af0 2023-12-01T01:18:24.917 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.911+0000 7efe4fc15b80 10 client.15642 fetch_fsmap learned FSMap version 27 2023-12-01T01:18:24.917 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.911+0000 7efe4fc15b80 1 -- 172.21.15.77:0/704522945 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556273129ce0 con 0x5562733c3af0 2023-12-01T01:18:24.917 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.911+0000 7efe257fa700 1 -- 172.21.15.77:0/704522945 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55627333dd90 con 0x5562733c3af0 2023-12-01T01:18:24.917 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.912+0000 7efe257fa700 1 -- 172.21.15.77:0/704522945 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7efe34008ba0 con 0x5562733c3af0 2023-12-01T01:18:24.917 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.912+0000 7efe4fc15b80 10 client.15642 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:18:24.917 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.912+0000 7efe4fc15b80 10 client.15642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-01T01:18:24.917 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.912+0000 7efe4fc15b80 20 client.15642 populate_metadata read hostname 'smithi077' 2023-12-01T01:18:24.917 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.912+0000 7efe4fc15b80 1 --2- 172.21.15.77:0/704522945 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5562733d7920 0x5562733d9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:24.917 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.912+0000 7efe4fc15b80 4 client.15642 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:18:24.917 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.912+0000 7efe4fc15b80 1 -- 172.21.15.77:0/704522945 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556273390e90 con 0x5562733d7920 2023-12-01T01:18:24.917 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.912+0000 7efe41821700 1 --2- 172.21.15.77:0/704522945 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5562733d7920 0x5562733d9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:24.917 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.912+0000 7efe257fa700 10 client.15642 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:24.918 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.912+0000 7efe41821700 1 --2- 172.21.15.77:0/704522945 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5562733d7920 0x5562733d9d40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baa841740e24142f server_cookie=918140d9338e2a85 in_seq=0 out_seq=0 2023-12-01T01:18:24.919 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.913+0000 7efe257fa700 1 -- 172.21.15.77:0/704522945 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x556273390e90 con 0x5562733d7920 2023-12-01T01:18:24.919 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.913+0000 7efe257fa700 10 client.15642 handle_command_reply: tid=0 2023-12-01T01:18:24.919 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.913+0000 7efe4fc15b80 1 client.15642 shutdown 2023-12-01T01:18:24.920 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.913+0000 7efe4fc15b80 1 -- 172.21.15.77:0/704522945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562733c3af0 msgr2=0x5562733d2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:24.920 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.913+0000 7efe4fc15b80 1 --2- 172.21.15.77:0/704522945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562733c3af0 0x5562733d2f80 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7efe34007f00 tx=0x7efe34007f30 comp rx=0 tx=0).stop 2023-12-01T01:18:24.920 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.913+0000 7efe4fc15b80 1 -- 172.21.15.77:0/704522945 shutdown_connections 2023-12-01T01:18:24.920 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.913+0000 7efe4fc15b80 1 --2- 172.21.15.77:0/704522945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562733c3af0 0x5562733d2f80 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:24.920 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.913+0000 7efe4fc15b80 1 --2- 172.21.15.77:0/704522945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5562733c2470 0x5562733cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:24.920 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.913+0000 7efe4fc15b80 1 --2- 172.21.15.77:0/704522945 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5562733d7920 0x5562733d9d40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:24.920 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.914+0000 7efe4fc15b80 1 --2- 172.21.15.77:0/704522945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562733c3170 0x5562733cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:24.920 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.914+0000 7efe257fa700 0 client.15642 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:24.921 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.914+0000 7efe4fc15b80 1 -- 172.21.15.77:0/704522945 >> 172.21.15.77:0/704522945 conn(0x556273321150 msgr2=0x5562733bc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:24.921 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.914+0000 7efe4fc15b80 1 -- 172.21.15.77:0/704522945 shutdown_connections 2023-12-01T01:18:24.921 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.914+0000 7efe4fc15b80 1 -- 172.21.15.77:0/704522945 wait complete. 2023-12-01T01:18:24.921 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.914+0000 7efe4fc15b80 20 client.15642 trim_cache size 0 max 16384 2023-12-01T01:18:24.947 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.945+0000 7efe4fc15b80 1 Processor -- start 2023-12-01T01:18:24.947 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.945+0000 7efe4fc15b80 1 -- start start 2023-12-01T01:18:24.947 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.945+0000 7efe4fc15b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562733c2470 0x5562733c1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:24.947 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.945+0000 7efe4fc15b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5562733c3170 0x5562733d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:24.947 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.945+0000 7efe4fc15b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562733c3af0 0x5562733bb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:24.947 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.946+0000 7efe4fc15b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562733b76e0 con 0x5562733c2470 2023-12-01T01:18:24.947 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.946+0000 7efe4fc15b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562733c95e0 con 0x5562733c3af0 2023-12-01T01:18:24.947 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.946+0000 7efe4fc15b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562733b4bd0 con 0x5562733c3170 2023-12-01T01:18:24.947 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.946+0000 7efe41821700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562733c2470 0x5562733c1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:24.947 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.946+0000 7efe41821700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562733c2470 0x5562733c1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59708/0 (socket says 172.21.15.77:59708) 2023-12-01T01:18:24.948 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.946+0000 7efe41821700 1 -- 172.21.15.77:0/584210392 learned_addr learned my addr 172.21.15.77:0/584210392 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:24.949 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.946+0000 7efe41821700 1 -- 172.21.15.77:0/584210392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5562733c3170 msgr2=0x5562733d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:24.949 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.946+0000 7efe41821700 1 --2- 172.21.15.77:0/584210392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5562733c3170 0x5562733d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:24.949 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.946+0000 7efe41821700 1 -- 172.21.15.77:0/584210392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562733c3af0 msgr2=0x5562733bb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:24.949 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.946+0000 7efe41821700 1 --2- 172.21.15.77:0/584210392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562733c3af0 0x5562733bb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:24.949 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.946+0000 7efe41821700 1 -- 172.21.15.77:0/584210392 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562731672d0 con 0x5562733c2470 2023-12-01T01:18:24.949 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.946+0000 7efe41821700 1 --2- 172.21.15.77:0/584210392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562733c2470 0x5562733c1ee0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7efe38095d60 tx=0x7efe38084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace222fab1470a6f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:24.949 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.946+0000 7efe30da9700 1 -- 172.21.15.77:0/584210392 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe380b7070 con 0x5562733c2470 2023-12-01T01:18:24.949 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.947+0000 7efe30da9700 1 -- 172.21.15.77:0/584210392 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe380b0070 con 0x5562733c2470 2023-12-01T01:18:24.949 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.947+0000 7efe30da9700 1 -- 172.21.15.77:0/584210392 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe38099820 con 0x5562733c2470 2023-12-01T01:18:24.949 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.948+0000 7efe4fc15b80 1 -- 172.21.15.77:0/584210392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562733c2470 msgr2=0x5562733c1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:24.949 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.948+0000 7efe4fc15b80 1 --2- 172.21.15.77:0/584210392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562733c2470 0x5562733c1ee0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7efe38095d60 tx=0x7efe38084040 comp rx=0 tx=0).stop 2023-12-01T01:18:24.950 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.948+0000 7efe4fc15b80 1 -- 172.21.15.77:0/584210392 shutdown_connections 2023-12-01T01:18:24.950 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.948+0000 7efe4fc15b80 1 --2- 172.21.15.77:0/584210392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562733c3af0 0x5562733bb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:24.950 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.948+0000 7efe4fc15b80 1 --2- 172.21.15.77:0/584210392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562733c2470 0x5562733c1ee0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:24.950 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.948+0000 7efe4fc15b80 1 --2- 172.21.15.77:0/584210392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5562733c3170 0x5562733d2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:24.950 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.948+0000 7efe4fc15b80 1 -- 172.21.15.77:0/584210392 >> 172.21.15.77:0/584210392 conn(0x556273321150 msgr2=0x5562733cba00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:24.950 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.948+0000 7efe4fc15b80 1 -- 172.21.15.77:0/584210392 shutdown_connections 2023-12-01T01:18:24.950 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.948+0000 7efe4fc15b80 1 -- 172.21.15.77:0/584210392 wait complete. 2023-12-01T01:18:24.952 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.948+0000 7efe4fc15b80 1 -- start start 2023-12-01T01:18:24.952 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.948+0000 7efe4fc15b80 1 Processor -- start 2023-12-01T01:18:24.952 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.949+0000 7efe4fc15b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5562733c2470 0x5562733d0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:24.952 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.949+0000 7efe4fc15b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562733c3170 0x5562733d0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:24.953 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.949+0000 7efe4fc15b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562733c3af0 0x5562733d1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:24.953 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.949+0000 7efe4fc15b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562733d1dc0 con 0x5562733c3170 2023-12-01T01:18:24.953 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.949+0000 7efe4fc15b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562733d1f30 con 0x5562733c3af0 2023-12-01T01:18:24.953 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.949+0000 7efe4fc15b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562733d20f0 con 0x5562733c2470 2023-12-01T01:18:24.953 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.949+0000 7efe4fc15b80 10 client.0 _ll_register_callbacks cb 0x7efe34007860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:18:24.953 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.949+0000 7efe41020700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562733c3170 0x5562733d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:24.953 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.949+0000 7efe41020700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562733c3170 0x5562733d0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59718/0 (socket says 172.21.15.77:59718) 2023-12-01T01:18:24.953 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.949+0000 7efe41020700 1 -- 172.21.15.77:0/1548853986 learned_addr learned my addr 172.21.15.77:0/1548853986 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:24.953 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.949+0000 7efe42022700 1 --2- 172.21.15.77:0/1548853986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562733c3af0 0x5562733d1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:24.953 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.949+0000 7efe41821700 1 --2- 172.21.15.77:0/1548853986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5562733c2470 0x5562733d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:24.953 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.949+0000 7efe42022700 1 -- 172.21.15.77:0/1548853986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5562733c2470 msgr2=0x5562733d0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:24.953 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.949+0000 7efe42022700 1 --2- 172.21.15.77:0/1548853986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5562733c2470 0x5562733d0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:24.953 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.950+0000 7efe42022700 1 -- 172.21.15.77:0/1548853986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562733c3170 msgr2=0x5562733d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:24.953 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.950+0000 7efe42022700 1 --2- 172.21.15.77:0/1548853986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562733c3170 0x5562733d0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:24.954 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.950+0000 7efe42022700 1 -- 172.21.15.77:0/1548853986 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556273140e60 con 0x5562733c3af0 2023-12-01T01:18:24.954 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.950+0000 7efe41020700 1 --2- 172.21.15.77:0/1548853986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562733c3170 0x5562733d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:18:24.954 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.950+0000 7efe42022700 1 --2- 172.21.15.77:0/1548853986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562733c3af0 0x5562733d1880 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7efe34016480 tx=0x7efe34013640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f2833b594178369 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:24.954 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.950+0000 7efe257fa700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2023-12-01T01:18:24.954 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.950+0000 7efe257fa700 1 -- 172.21.15.77:0/1548853986 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3400e040 con 0x5562733c3af0 2023-12-01T01:18:24.954 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.950+0000 7efe257fa700 1 -- 172.21.15.77:0/1548853986 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7efe34003c10 con 0x5562733c3af0 2023-12-01T01:18:24.954 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.950+0000 7efe4fc15b80 1 -- 172.21.15.77:0/1548853986 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe380a1360 con 0x5562733c3af0 2023-12-01T01:18:24.954 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.950+0000 7efe257fa700 1 -- 172.21.15.77:0/1548853986 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe34003dd0 con 0x5562733c3af0 2023-12-01T01:18:24.954 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.950+0000 7efe4fc15b80 10 client.25238 fetch_fsmap learned FSMap version 27 2023-12-01T01:18:24.954 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.950+0000 7efe4fc15b80 1 -- 172.21.15.77:0/1548853986 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55627313f330 con 0x5562733c3af0 2023-12-01T01:18:24.954 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.950+0000 7efe257fa700 1 -- 172.21.15.77:0/1548853986 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe34039020 con 0x5562733c3af0 2023-12-01T01:18:24.954 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.951+0000 7efe257fa700 1 -- 172.21.15.77:0/1548853986 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7efe34031060 con 0x5562733c3af0 2023-12-01T01:18:24.954 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.951+0000 7efe4fc15b80 10 client.25238 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:18:24.954 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.951+0000 7efe4fc15b80 10 client.25238 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-01T01:18:24.955 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.951+0000 7efe4fc15b80 20 client.25238 populate_metadata read hostname 'smithi077' 2023-12-01T01:18:24.955 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.951+0000 7efe4fc15b80 1 --2- 172.21.15.77:0/1548853986 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x556273434350 0x556273436770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:24.955 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.951+0000 7efe4fc15b80 4 client.25238 mds_command: new command op to 24500 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-12-01T01:18:24.955 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.951+0000 7efe4fc15b80 1 -- 172.21.15.77:0/1548853986 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55627330bdc0 con 0x556273434350 2023-12-01T01:18:24.956 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.952+0000 7efe41821700 1 --2- 172.21.15.77:0/1548853986 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x556273434350 0x556273436770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:24.956 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.952+0000 7efe257fa700 10 client.25238 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:24.956 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.952+0000 7efe41821700 1 --2- 172.21.15.77:0/1548853986 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x556273434350 0x556273436770 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2b2449b40a6e079 server_cookie=69b55cd25cee4106 in_seq=0 out_seq=0 2023-12-01T01:18:24.956 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.953+0000 7efe257fa700 1 -- 172.21.15.77:0/1548853986 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55627330bdc0 con 0x556273434350 2023-12-01T01:18:24.956 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.953+0000 7efe257fa700 10 client.25238 handle_command_reply: tid=0 2023-12-01T01:18:24.956 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.953+0000 7efe4fc15b80 1 client.25238 shutdown 2023-12-01T01:18:24.956 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.954+0000 7efe4fc15b80 1 -- 172.21.15.77:0/1548853986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562733c3af0 msgr2=0x5562733d1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:24.956 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.954+0000 7efe4fc15b80 1 --2- 172.21.15.77:0/1548853986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562733c3af0 0x5562733d1880 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7efe34016480 tx=0x7efe34013640 comp rx=0 tx=0).stop 2023-12-01T01:18:24.956 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.954+0000 7efe4fc15b80 1 -- 172.21.15.77:0/1548853986 shutdown_connections 2023-12-01T01:18:24.956 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.954+0000 7efe4fc15b80 1 --2- 172.21.15.77:0/1548853986 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x556273434350 0x556273436770 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:24.956 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.954+0000 7efe4fc15b80 1 --2- 172.21.15.77:0/1548853986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562733c3af0 0x5562733d1880 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:24.957 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.954+0000 7efe4fc15b80 1 --2- 172.21.15.77:0/1548853986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5562733c2470 0x5562733d0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:24.957 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.954+0000 7efe257fa700 0 client.25238 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:24.957 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.954+0000 7efe4fc15b80 1 --2- 172.21.15.77:0/1548853986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5562733c3170 0x5562733d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:24.957 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.954+0000 7efe4fc15b80 1 -- 172.21.15.77:0/1548853986 >> 172.21.15.77:0/1548853986 conn(0x556273321150 msgr2=0x5562733bb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:24.957 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.954+0000 7efe4fc15b80 1 -- 172.21.15.77:0/1548853986 shutdown_connections 2023-12-01T01:18:24.957 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.954+0000 7efe4fc15b80 1 -- 172.21.15.77:0/1548853986 wait complete. 2023-12-01T01:18:24.957 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.954+0000 7efe4fc15b80 20 client.25238 trim_cache size 0 max 16384 2023-12-01T01:18:24.957 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2023-12-01T01:18:24.957 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2023-12-01T01:18:24.957 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "affe082b-75ae-4a4c-bc4c-8ae568688646", 2023-12-01T01:18:24.957 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2023-12-01T01:18:24.958 INFO:teuthology.orchestra.run.smithi077.stdout:} 2023-12-01T01:18:24.958 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.955+0000 7efe43a85700 1 -- 172.21.15.77:0/907909679 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7efe280a05a0 msgr2=0x7efe280a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:24.958 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.955+0000 7efe43a85700 1 --2- 172.21.15.77:0/907909679 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7efe280a05a0 0x7efe280a2a60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efe2c000c00 tx=0x7efe2c005fb0 comp rx=0 tx=0).stop 2023-12-01T01:18:24.958 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.955+0000 7efe43a85700 1 -- 172.21.15.77:0/907909679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe3c0ffe90 msgr2=0x7efe3c196ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:24.959 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.955+0000 7efe43a85700 1 --2- 172.21.15.77:0/907909679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe3c0ffe90 0x7efe3c196ab0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efe3800d950 tx=0x7efe3800dc60 comp rx=0 tx=0).stop 2023-12-01T01:18:24.959 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.955+0000 7efe43a85700 1 -- 172.21.15.77:0/907909679 shutdown_connections 2023-12-01T01:18:24.959 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.955+0000 7efe43a85700 1 --2- 172.21.15.77:0/907909679 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7efe280a05a0 0x7efe280a2a60 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:24.959 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.955+0000 7efe43a85700 1 --2- 172.21.15.77:0/907909679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe3c101240 0x7efe3c19e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:24.959 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.955+0000 7efe43a85700 1 --2- 172.21.15.77:0/907909679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe3c0ffe90 0x7efe3c196ab0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:24.959 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.955+0000 7efe43a85700 1 --2- 172.21.15.77:0/907909679 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efe3c100850 0x7efe3c196ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:24.959 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.955+0000 7efe43a85700 1 -- 172.21.15.77:0/907909679 >> 172.21.15.77:0/907909679 conn(0x7efe3c0fa960 msgr2=0x7efe3c0fcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:24.959 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.955+0000 7efe43a85700 1 -- 172.21.15.77:0/907909679 shutdown_connections 2023-12-01T01:18:24.959 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:24.955+0000 7efe43a85700 1 -- 172.21.15.77:0/907909679 wait complete. 2023-12-01T01:18:24.973 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-01T01:18:25.164 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.162+0000 7f66d4f65700 1 -- 172.21.15.77:0/596449006 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f66d00fff70 msgr2=0x7f66d01003d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:25.164 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.162+0000 7f66d4f65700 1 --2- 172.21.15.77:0/596449006 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f66d00fff70 0x7f66d01003d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f66c4009990 tx=0x7f66c4009ca0 comp rx=0 tx=0).stop 2023-12-01T01:18:25.164 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.162+0000 7f66d4f65700 1 -- 172.21.15.77:0/596449006 shutdown_connections 2023-12-01T01:18:25.164 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.162+0000 7f66d4f65700 1 --2- 172.21.15.77:0/596449006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66d0100910 0x7f66d01081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:25.164 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.162+0000 7f66d4f65700 1 --2- 172.21.15.77:0/596449006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66d0101240 0x7f66d0101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:25.164 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.162+0000 7f66d4f65700 1 --2- 172.21.15.77:0/596449006 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f66d00fff70 0x7f66d01003d0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:25.164 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.162+0000 7f66d4f65700 1 -- 172.21.15.77:0/596449006 >> 172.21.15.77:0/596449006 conn(0x7f66d00fa980 msgr2=0x7f66d00fcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:25.164 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.163+0000 7f66d4f65700 1 -- 172.21.15.77:0/596449006 shutdown_connections 2023-12-01T01:18:25.165 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.163+0000 7f66d4f65700 1 -- 172.21.15.77:0/596449006 wait complete. 2023-12-01T01:18:25.166 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.164+0000 7f66d4f65700 1 Processor -- start 2023-12-01T01:18:25.166 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.164+0000 7f66d4f65700 1 -- start start 2023-12-01T01:18:25.166 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.164+0000 7f66d4f65700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f66d00fff70 0x7f66d0194840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:25.166 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.164+0000 7f66d4f65700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66d0100910 0x7f66d0194d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:25.166 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.164+0000 7f66d4f65700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66d0101240 0x7f66d019bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:25.166 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.164+0000 7f66d4f65700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d019c3c0 con 0x7f66d00fff70 2023-12-01T01:18:25.167 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.164+0000 7f66d4f65700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d019c500 con 0x7f66d0100910 2023-12-01T01:18:25.167 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.164+0000 7f66d4f65700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d019c690 con 0x7f66d0101240 2023-12-01T01:18:25.167 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.164+0000 7f66ce59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f66d00fff70 0x7f66d0194840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:25.167 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.164+0000 7f66ce59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f66d00fff70 0x7f66d0194840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59728/0 (socket says 172.21.15.77:59728) 2023-12-01T01:18:25.167 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.164+0000 7f66ce59c700 1 -- 172.21.15.77:0/3952071416 learned_addr learned my addr 172.21.15.77:0/3952071416 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:25.167 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.164+0000 7f66cdd9b700 1 --2- 172.21.15.77:0/3952071416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66d0100910 0x7f66d0194d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:25.167 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.164+0000 7f66ced9d700 1 --2- 172.21.15.77:0/3952071416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66d0101240 0x7f66d019bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:25.167 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.164+0000 7f66ce59c700 1 -- 172.21.15.77:0/3952071416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66d0101240 msgr2=0x7f66d019bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:25.167 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.164+0000 7f66ce59c700 1 --2- 172.21.15.77:0/3952071416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66d0101240 0x7f66d019bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:25.167 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.164+0000 7f66ce59c700 1 -- 172.21.15.77:0/3952071416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66d0100910 msgr2=0x7f66d0194d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:25.167 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.164+0000 7f66ce59c700 1 --2- 172.21.15.77:0/3952071416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66d0100910 0x7f66d0194d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:25.167 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.164+0000 7f66ce59c700 1 -- 172.21.15.77:0/3952071416 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c40095f0 con 0x7f66d00fff70 2023-12-01T01:18:25.167 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.165+0000 7f66ce59c700 1 --2- 172.21.15.77:0/3952071416 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f66d00fff70 0x7f66d0194840 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f66c000e970 tx=0x7f66c000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:25.167 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.165+0000 7f66bf7fe700 1 -- 172.21.15.77:0/3952071416 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c0009790 con 0x7f66d00fff70 2023-12-01T01:18:25.167 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.165+0000 7f66d4f65700 1 -- 172.21.15.77:0/3952071416 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66d019c970 con 0x7f66d00fff70 2023-12-01T01:18:25.168 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.165+0000 7f66bf7fe700 1 -- 172.21.15.77:0/3952071416 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66c000eec0 con 0x7f66d00fff70 2023-12-01T01:18:25.168 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.165+0000 7f66bf7fe700 1 -- 172.21.15.77:0/3952071416 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c0003cf0 con 0x7f66d00fff70 2023-12-01T01:18:25.168 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.165+0000 7f66d4f65700 1 -- 172.21.15.77:0/3952071416 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66d019ced0 con 0x7f66d00fff70 2023-12-01T01:18:25.169 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.168+0000 7f66bf7fe700 1 -- 172.21.15.77:0/3952071416 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f66c000ccd0 con 0x7f66d00fff70 2023-12-01T01:18:25.170 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.169+0000 7f66bf7fe700 1 --2- 172.21.15.77:0/3952071416 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f66b80a0520 0x7f66b80a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:25.170 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.169+0000 7f66cdd9b700 1 --2- 172.21.15.77:0/3952071416 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f66b80a0520 0x7f66b80a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:25.171 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.169+0000 7f66bf7fe700 1 -- 172.21.15.77:0/3952071416 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f66c0014070 con 0x7f66d00fff70 2023-12-01T01:18:25.171 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.170+0000 7f66cdd9b700 1 --2- 172.21.15.77:0/3952071416 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f66b80a0520 0x7f66b80a29e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f66c400b2e0 tx=0x7f66c4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:25.171 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.170+0000 7f66e10f5b80 1 Processor -- start 2023-12-01T01:18:25.171 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.171+0000 7f66e10f5b80 1 -- start start 2023-12-01T01:18:25.172 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.171+0000 7f66e10f5b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b47270f0 0x5610b4727530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:25.172 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.171+0000 7f66e10f5b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5610b4727a70 0x5610b4725e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:25.172 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.171+0000 7f66e10f5b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610b47263f0 0x5610b4726870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:25.172 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.171+0000 7f66e10f5b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610b467da10 con 0x5610b47270f0 2023-12-01T01:18:25.172 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.171+0000 7f66e10f5b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610b467b3d0 con 0x5610b47263f0 2023-12-01T01:18:25.172 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.171+0000 7f66e10f5b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610b4676cc0 con 0x5610b4727a70 2023-12-01T01:18:25.173 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.171+0000 7f66ce59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b47270f0 0x5610b4727530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:25.173 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.171+0000 7f66ced9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610b47263f0 0x5610b4726870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:25.173 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.171+0000 7f66ce59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b47270f0 0x5610b4727530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59736/0 (socket says 172.21.15.77:59736) 2023-12-01T01:18:25.173 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.171+0000 7f66cdd9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5610b4727a70 0x5610b4725e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:25.173 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.171+0000 7f66ce59c700 1 -- 172.21.15.77:0/938025715 learned_addr learned my addr 172.21.15.77:0/938025715 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:25.174 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.172+0000 7f66ced9d700 1 -- 172.21.15.77:0/938025715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5610b4727a70 msgr2=0x5610b4725e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:25.174 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.172+0000 7f66ced9d700 1 --2- 172.21.15.77:0/938025715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5610b4727a70 0x5610b4725e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:25.174 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.172+0000 7f66ced9d700 1 -- 172.21.15.77:0/938025715 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b47270f0 msgr2=0x5610b4727530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:25.174 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.172+0000 7f66ced9d700 1 --2- 172.21.15.77:0/938025715 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b47270f0 0x5610b4727530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:25.174 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.172+0000 7f66ced9d700 1 -- 172.21.15.77:0/938025715 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610b44a4de0 con 0x5610b47263f0 2023-12-01T01:18:25.174 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.172+0000 7f66ced9d700 1 --2- 172.21.15.77:0/938025715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610b47263f0 0x5610b4726870 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f66c800b4e0 tx=0x7f66c80099f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b859e476c9e7b9b0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:25.174 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.172+0000 7f66bcff9700 1 -- 172.21.15.77:0/938025715 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c801a070 con 0x5610b47263f0 2023-12-01T01:18:25.174 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.172+0000 7f66bcff9700 1 -- 172.21.15.77:0/938025715 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66c8011050 con 0x5610b47263f0 2023-12-01T01:18:25.174 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.172+0000 7f66bcff9700 1 -- 172.21.15.77:0/938025715 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c8004420 con 0x5610b47263f0 2023-12-01T01:18:25.176 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.173+0000 7f66e10f5b80 1 -- 172.21.15.77:0/938025715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610b47263f0 msgr2=0x5610b4726870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:25.176 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.173+0000 7f66e10f5b80 1 --2- 172.21.15.77:0/938025715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610b47263f0 0x5610b4726870 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f66c800b4e0 tx=0x7f66c80099f0 comp rx=0 tx=0).stop 2023-12-01T01:18:25.176 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.173+0000 7f66e10f5b80 1 -- 172.21.15.77:0/938025715 shutdown_connections 2023-12-01T01:18:25.176 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.173+0000 7f66e10f5b80 1 --2- 172.21.15.77:0/938025715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610b47263f0 0x5610b4726870 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:25.176 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.173+0000 7f66e10f5b80 1 --2- 172.21.15.77:0/938025715 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b47270f0 0x5610b4727530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:25.176 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.173+0000 7f66e10f5b80 1 --2- 172.21.15.77:0/938025715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5610b4727a70 0x5610b4725e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:25.176 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.173+0000 7f66e10f5b80 1 -- 172.21.15.77:0/938025715 >> 172.21.15.77:0/938025715 conn(0x5610b46850d0 msgr2=0x5610b4720940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:25.176 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.173+0000 7f66e10f5b80 1 -- 172.21.15.77:0/938025715 shutdown_connections 2023-12-01T01:18:25.176 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.173+0000 7f66e10f5b80 1 -- 172.21.15.77:0/938025715 wait complete. 2023-12-01T01:18:25.176 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.174+0000 7f66e10f5b80 1 -- start start 2023-12-01T01:18:25.176 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.174+0000 7f66e10f5b80 1 Processor -- start 2023-12-01T01:18:25.176 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.174+0000 7f66e10f5b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610b47263f0 0x5610b472f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:25.176 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.174+0000 7f66e10f5b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b47270f0 0x5610b472fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:25.177 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.174+0000 7f66e10f5b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5610b4727a70 0x5610b4736ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:25.177 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.174+0000 7f66e10f5b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610b4613510 con 0x5610b47270f0 2023-12-01T01:18:25.177 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.174+0000 7f66e10f5b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610b46131b0 con 0x5610b47263f0 2023-12-01T01:18:25.177 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.174+0000 7f66e10f5b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610b44fa920 con 0x5610b4727a70 2023-12-01T01:18:25.177 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.174+0000 7f66e10f5b80 10 client.0 _ll_register_callbacks cb 0x5610b4560a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:18:25.177 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.174+0000 7f66ce59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610b47263f0 0x5610b472f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:25.177 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.174+0000 7f66ce59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610b47263f0 0x5610b472f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:38664/0 (socket says 172.21.15.77:38664) 2023-12-01T01:18:25.177 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.174+0000 7f66ce59c700 1 -- 172.21.15.77:0/4169737160 learned_addr learned my addr 172.21.15.77:0/4169737160 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:25.177 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.174+0000 7f66cdd9b700 1 --2- 172.21.15.77:0/4169737160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b47270f0 0x5610b472fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:25.177 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.175+0000 7f66ced9d700 1 --2- 172.21.15.77:0/4169737160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5610b4727a70 0x5610b4736ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:25.177 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.175+0000 7f66ce59c700 1 -- 172.21.15.77:0/4169737160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5610b4727a70 msgr2=0x5610b4736ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:25.177 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.175+0000 7f66ce59c700 1 --2- 172.21.15.77:0/4169737160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5610b4727a70 0x5610b4736ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:25.177 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.175+0000 7f66ce59c700 1 -- 172.21.15.77:0/4169737160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b47270f0 msgr2=0x5610b472fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:25.177 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.175+0000 7f66ce59c700 1 --2- 172.21.15.77:0/4169737160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b47270f0 0x5610b472fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:25.177 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.175+0000 7f66ce59c700 1 -- 172.21.15.77:0/4169737160 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66c800b110 con 0x5610b47263f0 2023-12-01T01:18:25.180 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.175+0000 7f66ce59c700 1 --2- 172.21.15.77:0/4169737160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610b47263f0 0x5610b472f940 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f66c0009230 tx=0x7f66c0098870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42c8cdfd8f758ee1 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:25.180 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.175+0000 7f66b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2023-12-01T01:18:25.180 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.175+0000 7f66b5ffb700 1 -- 172.21.15.77:0/4169737160 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c00b1220 con 0x5610b47263f0 2023-12-01T01:18:25.180 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.175+0000 7f66e10f5b80 1 -- 172.21.15.77:0/4169737160 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5610b46a1d10 con 0x5610b47263f0 2023-12-01T01:18:25.180 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.175+0000 7f66b5ffb700 1 -- 172.21.15.77:0/4169737160 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f66c00993b0 con 0x5610b47263f0 2023-12-01T01:18:25.180 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.175+0000 7f66b5ffb700 1 -- 172.21.15.77:0/4169737160 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66c0098cb0 con 0x5610b47263f0 2023-12-01T01:18:25.180 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.176+0000 7f66e10f5b80 10 client.25250 fetch_fsmap learned FSMap version 27 2023-12-01T01:18:25.180 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.176+0000 7f66e10f5b80 1 -- 172.21.15.77:0/4169737160 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5610b448dc60 con 0x5610b47263f0 2023-12-01T01:18:25.180 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.176+0000 7f66b5ffb700 1 -- 172.21.15.77:0/4169737160 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5610b46a1d10 con 0x5610b47263f0 2023-12-01T01:18:25.180 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.176+0000 7f66b5ffb700 1 -- 172.21.15.77:0/4169737160 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f66c00a0890 con 0x5610b47263f0 2023-12-01T01:18:25.180 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.176+0000 7f66e10f5b80 10 client.25250 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:18:25.181 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.176+0000 7f66e10f5b80 10 client.25250 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-01T01:18:25.181 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.176+0000 7f66e10f5b80 20 client.25250 populate_metadata read hostname 'smithi077' 2023-12-01T01:18:25.181 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.176+0000 7f66e10f5b80 1 --2- 172.21.15.77:0/4169737160 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5610b4739f80 0x5610b473a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:25.181 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.176+0000 7f66e10f5b80 4 client.25250 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:18:25.181 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.176+0000 7f66e10f5b80 1 -- 172.21.15.77:0/4169737160 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5610b46f4e10 con 0x5610b4739f80 2023-12-01T01:18:25.181 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.176+0000 7f66cdd9b700 1 --2- 172.21.15.77:0/4169737160 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5610b4739f80 0x5610b473a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:25.181 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.177+0000 7f66b5ffb700 10 client.25250 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:25.181 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.177+0000 7f66cdd9b700 1 --2- 172.21.15.77:0/4169737160 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5610b4739f80 0x5610b473a390 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b053d782df8716ed server_cookie=413d8234d569ca31 in_seq=0 out_seq=0 2023-12-01T01:18:25.181 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.177+0000 7f66b5ffb700 1 -- 172.21.15.77:0/4169737160 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5610b46f4e10 con 0x5610b4739f80 2023-12-01T01:18:25.181 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.177+0000 7f66b5ffb700 10 client.25250 handle_command_reply: tid=0 2023-12-01T01:18:25.181 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.177+0000 7f66e10f5b80 1 client.25250 shutdown 2023-12-01T01:18:25.181 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.177+0000 7f66e10f5b80 1 -- 172.21.15.77:0/4169737160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610b47263f0 msgr2=0x5610b472f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:25.181 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.177+0000 7f66e10f5b80 1 --2- 172.21.15.77:0/4169737160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610b47263f0 0x5610b472f940 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f66c0009230 tx=0x7f66c0098870 comp rx=0 tx=0).stop 2023-12-01T01:18:25.181 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.178+0000 7f66e10f5b80 1 -- 172.21.15.77:0/4169737160 shutdown_connections 2023-12-01T01:18:25.181 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.178+0000 7f66e10f5b80 1 --2- 172.21.15.77:0/4169737160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5610b4727a70 0x5610b4736ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:25.181 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.178+0000 7f66e10f5b80 1 --2- 172.21.15.77:0/4169737160 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5610b4739f80 0x5610b473a390 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:25.182 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.178+0000 7f66e10f5b80 1 --2- 172.21.15.77:0/4169737160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610b47263f0 0x5610b472f940 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:25.182 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.178+0000 7f66e10f5b80 1 --2- 172.21.15.77:0/4169737160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b47270f0 0x5610b472fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:25.182 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.178+0000 7f66e10f5b80 1 -- 172.21.15.77:0/4169737160 >> 172.21.15.77:0/4169737160 conn(0x5610b46850d0 msgr2=0x5610b47200b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:25.182 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.178+0000 7f66b5ffb700 0 client.25250 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:25.182 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.178+0000 7f66e10f5b80 1 -- 172.21.15.77:0/4169737160 shutdown_connections 2023-12-01T01:18:25.182 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.178+0000 7f66e10f5b80 1 -- 172.21.15.77:0/4169737160 wait complete. 2023-12-01T01:18:25.182 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.178+0000 7f66e10f5b80 20 client.25250 trim_cache size 0 max 16384 2023-12-01T01:18:25.211 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.209+0000 7f66e10f5b80 1 Processor -- start 2023-12-01T01:18:25.211 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.209+0000 7f66e10f5b80 1 -- start start 2023-12-01T01:18:25.212 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.209+0000 7f66e10f5b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610b47263f0 0x5610b4725e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:25.212 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.209+0000 7f66e10f5b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5610b47270f0 0x5610b4736e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:25.212 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.209+0000 7f66e10f5b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b4727a70 0x5610b472a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:25.212 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.209+0000 7f66e10f5b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610b472f510 con 0x5610b4727a70 2023-12-01T01:18:25.212 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.209+0000 7f66e10f5b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610b4721240 con 0x5610b47263f0 2023-12-01T01:18:25.212 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.209+0000 7f66e10f5b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610b47216b0 con 0x5610b47270f0 2023-12-01T01:18:25.212 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.210+0000 7f66ced9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b4727a70 0x5610b472a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:25.212 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.210+0000 7f66ced9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b4727a70 0x5610b472a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59754/0 (socket says 172.21.15.77:59754) 2023-12-01T01:18:25.212 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.210+0000 7f66cdd9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5610b47270f0 0x5610b4736e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:25.212 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.210+0000 7f66ced9d700 1 -- 172.21.15.77:0/4211570971 learned_addr learned my addr 172.21.15.77:0/4211570971 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:25.212 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.210+0000 7f66ce59c700 1 --2- 172.21.15.77:0/4211570971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610b47263f0 0x5610b4725e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:25.212 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.210+0000 7f66ced9d700 1 -- 172.21.15.77:0/4211570971 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5610b47270f0 msgr2=0x5610b4736e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:25.212 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.210+0000 7f66ced9d700 1 --2- 172.21.15.77:0/4211570971 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5610b47270f0 0x5610b4736e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:25.212 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.210+0000 7f66ced9d700 1 -- 172.21.15.77:0/4211570971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610b47263f0 msgr2=0x5610b4725e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:25.213 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.210+0000 7f66ced9d700 1 --2- 172.21.15.77:0/4211570971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610b47263f0 0x5610b4725e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:25.213 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.210+0000 7f66ced9d700 1 -- 172.21.15.77:0/4211570971 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610b44cb250 con 0x5610b4727a70 2023-12-01T01:18:25.213 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.210+0000 7f66cdd9b700 1 --2- 172.21.15.77:0/4211570971 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5610b47270f0 0x5610b4736e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:18:25.213 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.210+0000 7f66ced9d700 1 --2- 172.21.15.77:0/4211570971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b4727a70 0x5610b472a370 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f66c8009d40 tx=0x7f66c8006080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3075056c639acbde server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:25.213 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.210+0000 7f66bd7fa700 1 -- 172.21.15.77:0/4211570971 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c801a040 con 0x5610b4727a70 2023-12-01T01:18:25.213 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.211+0000 7f66bd7fa700 1 -- 172.21.15.77:0/4211570971 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66c800b5d0 con 0x5610b4727a70 2023-12-01T01:18:25.213 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.211+0000 7f66bd7fa700 1 -- 172.21.15.77:0/4211570971 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c80045f0 con 0x5610b4727a70 2023-12-01T01:18:25.216 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.211+0000 7f66e10f5b80 1 -- 172.21.15.77:0/4211570971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b4727a70 msgr2=0x5610b472a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:25.216 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.211+0000 7f66e10f5b80 1 --2- 172.21.15.77:0/4211570971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b4727a70 0x5610b472a370 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f66c8009d40 tx=0x7f66c8006080 comp rx=0 tx=0).stop 2023-12-01T01:18:25.216 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.212+0000 7f66e10f5b80 1 -- 172.21.15.77:0/4211570971 shutdown_connections 2023-12-01T01:18:25.216 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.212+0000 7f66e10f5b80 1 --2- 172.21.15.77:0/4211570971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b4727a70 0x5610b472a370 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:25.216 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.212+0000 7f66e10f5b80 1 --2- 172.21.15.77:0/4211570971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610b47263f0 0x5610b4725e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:25.216 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.212+0000 7f66e10f5b80 1 --2- 172.21.15.77:0/4211570971 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5610b47270f0 0x5610b4736e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:25.216 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.212+0000 7f66e10f5b80 1 -- 172.21.15.77:0/4211570971 >> 172.21.15.77:0/4211570971 conn(0x5610b46850d0 msgr2=0x5610b472d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:25.217 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.212+0000 7f66e10f5b80 1 -- 172.21.15.77:0/4211570971 shutdown_connections 2023-12-01T01:18:25.217 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.212+0000 7f66e10f5b80 1 -- 172.21.15.77:0/4211570971 wait complete. 2023-12-01T01:18:25.217 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.212+0000 7f66e10f5b80 1 -- start start 2023-12-01T01:18:25.217 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.212+0000 7f66e10f5b80 1 Processor -- start 2023-12-01T01:18:25.217 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.212+0000 7f66e10f5b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610b47263f0 0x5610b4732e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:25.217 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.212+0000 7f66e10f5b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5610b47270f0 0x5610b47353b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:25.217 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.212+0000 7f66e10f5b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b4727a70 0x5610b4735950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:25.218 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.212+0000 7f66e10f5b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610b4736060 con 0x5610b4727a70 2023-12-01T01:18:25.218 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.212+0000 7f66e10f5b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610b47361d0 con 0x5610b47263f0 2023-12-01T01:18:25.218 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.212+0000 7f66e10f5b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610b4736390 con 0x5610b47270f0 2023-12-01T01:18:25.218 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.212+0000 7f66e10f5b80 10 client.0 _ll_register_callbacks cb 0x7f66c0094ff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:18:25.218 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.213+0000 7f66ced9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b4727a70 0x5610b4735950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:25.218 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.213+0000 7f66ced9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b4727a70 0x5610b4735950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59760/0 (socket says 172.21.15.77:59760) 2023-12-01T01:18:25.218 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.213+0000 7f66ced9d700 1 -- 172.21.15.77:0/2626131927 learned_addr learned my addr 172.21.15.77:0/2626131927 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:25.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.213+0000 7f66cdd9b700 1 --2- 172.21.15.77:0/2626131927 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5610b47270f0 0x5610b47353b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:25.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.213+0000 7f66ce59c700 1 --2- 172.21.15.77:0/2626131927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610b47263f0 0x5610b4732e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:25.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.213+0000 7f66ced9d700 1 -- 172.21.15.77:0/2626131927 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5610b47270f0 msgr2=0x5610b47353b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:25.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.213+0000 7f66ced9d700 1 --2- 172.21.15.77:0/2626131927 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5610b47270f0 0x5610b47353b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:25.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.213+0000 7f66ced9d700 1 -- 172.21.15.77:0/2626131927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610b47263f0 msgr2=0x5610b4732e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:25.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.213+0000 7f66ced9d700 1 --2- 172.21.15.77:0/2626131927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610b47263f0 0x5610b4732e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:25.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.213+0000 7f66ced9d700 1 -- 172.21.15.77:0/2626131927 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66d0108a40 con 0x5610b4727a70 2023-12-01T01:18:25.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.213+0000 7f66ced9d700 1 --2- 172.21.15.77:0/2626131927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b4727a70 0x5610b4735950 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f66c800b920 tx=0x7f66c8004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3955d65aff81456a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:25.219 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.213+0000 7f66b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:18:25.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.213+0000 7f66b5ffb700 1 -- 172.21.15.77:0/2626131927 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c801a040 con 0x5610b4727a70 2023-12-01T01:18:25.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.213+0000 7f66b5ffb700 1 -- 172.21.15.77:0/2626131927 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f66c8004890 con 0x5610b4727a70 2023-12-01T01:18:25.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.213+0000 7f66e10f5b80 1 -- 172.21.15.77:0/2626131927 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f66c8011050 con 0x5610b4727a70 2023-12-01T01:18:25.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.214+0000 7f66b5ffb700 1 -- 172.21.15.77:0/2626131927 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66c8009480 con 0x5610b4727a70 2023-12-01T01:18:25.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.214+0000 7f66e10f5b80 10 client.15666 fetch_fsmap learned FSMap version 27 2023-12-01T01:18:25.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.214+0000 7f66e10f5b80 1 -- 172.21.15.77:0/2626131927 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5610b44a4de0 con 0x5610b4727a70 2023-12-01T01:18:25.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.214+0000 7f66b5ffb700 1 -- 172.21.15.77:0/2626131927 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66c8011050 con 0x5610b4727a70 2023-12-01T01:18:25.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.214+0000 7f66b5ffb700 1 -- 172.21.15.77:0/2626131927 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f66c8036460 con 0x5610b4727a70 2023-12-01T01:18:25.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.214+0000 7f66e10f5b80 10 client.15666 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:18:25.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.214+0000 7f66e10f5b80 10 client.15666 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-01T01:18:25.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.214+0000 7f66e10f5b80 20 client.15666 populate_metadata read hostname 'smithi077' 2023-12-01T01:18:25.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.214+0000 7f66e10f5b80 1 --2- 172.21.15.77:0/2626131927 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5610b47a7b70 0x5610b47a9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:25.220 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.214+0000 7f66e10f5b80 4 client.15666 mds_command: new command op to 24500 tid=0[{"prefix": "scrub status"}] 2023-12-01T01:18:25.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.214+0000 7f66e10f5b80 1 -- 172.21.15.77:0/2626131927 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5610b466fd40 con 0x5610b47a7b70 2023-12-01T01:18:25.221 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.215+0000 7f66ce59c700 1 --2- 172.21.15.77:0/2626131927 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5610b47a7b70 0x5610b47a9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:25.222 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.215+0000 7f66b5ffb700 10 client.15666 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:25.223 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.215+0000 7f66ce59c700 1 --2- 172.21.15.77:0/2626131927 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5610b47a7b70 0x5610b47a9f90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4441ea1575ff339 server_cookie=96e13de05fc82ab4 in_seq=0 out_seq=0 2023-12-01T01:18:25.223 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.215+0000 7f66b5ffb700 1 -- 172.21.15.77:0/2626131927 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5610b466fd40 con 0x5610b47a7b70 2023-12-01T01:18:25.223 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.215+0000 7f66b5ffb700 10 client.15666 handle_command_reply: tid=0 2023-12-01T01:18:25.223 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.215+0000 7f66e10f5b80 1 client.15666 shutdown 2023-12-01T01:18:25.223 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.215+0000 7f66e10f5b80 1 -- 172.21.15.77:0/2626131927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b4727a70 msgr2=0x5610b4735950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:25.223 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.215+0000 7f66e10f5b80 1 --2- 172.21.15.77:0/2626131927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b4727a70 0x5610b4735950 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f66c800b920 tx=0x7f66c8004620 comp rx=0 tx=0).stop 2023-12-01T01:18:25.223 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.216+0000 7f66e10f5b80 1 -- 172.21.15.77:0/2626131927 shutdown_connections 2023-12-01T01:18:25.223 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.216+0000 7f66e10f5b80 1 --2- 172.21.15.77:0/2626131927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b4727a70 0x5610b4735950 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:25.223 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.216+0000 7f66e10f5b80 1 --2- 172.21.15.77:0/2626131927 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5610b47a7b70 0x5610b47a9f90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:25.223 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.216+0000 7f66e10f5b80 1 --2- 172.21.15.77:0/2626131927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610b47263f0 0x5610b4732e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:25.223 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.216+0000 7f66e10f5b80 1 --2- 172.21.15.77:0/2626131927 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5610b47270f0 0x5610b47353b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:25.223 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.216+0000 7f66e10f5b80 1 -- 172.21.15.77:0/2626131927 >> 172.21.15.77:0/2626131927 conn(0x5610b46850d0 msgr2=0x5610b472a370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:25.224 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.216+0000 7f66b5ffb700 0 client.15666 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:25.224 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.216+0000 7f66e10f5b80 1 -- 172.21.15.77:0/2626131927 shutdown_connections 2023-12-01T01:18:25.224 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.216+0000 7f66e10f5b80 1 -- 172.21.15.77:0/2626131927 wait complete. 2023-12-01T01:18:25.224 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.216+0000 7f66e10f5b80 20 client.15666 trim_cache size 0 max 16384 2023-12-01T01:18:25.224 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2023-12-01T01:18:25.224 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2023-12-01T01:18:25.225 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2023-12-01T01:18:25.225 INFO:teuthology.orchestra.run.smithi077.stdout: "affe082b-75ae-4a4c-bc4c-8ae568688646": { 2023-12-01T01:18:25.225 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2023-12-01T01:18:25.225 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "affe082b-75ae-4a4c-bc4c-8ae568688646", 2023-12-01T01:18:25.225 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2023-12-01T01:18:25.225 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:18:25.226 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:18:25.226 INFO:teuthology.orchestra.run.smithi077.stdout:} 2023-12-01T01:18:25.227 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.216+0000 7f66d4f65700 1 -- 172.21.15.77:0/3952071416 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f66b80a0520 msgr2=0x7f66b80a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:25.227 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.216+0000 7f66d4f65700 1 --2- 172.21.15.77:0/3952071416 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f66b80a0520 0x7f66b80a29e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f66c400b2e0 tx=0x7f66c4005fb0 comp rx=0 tx=0).stop 2023-12-01T01:18:25.228 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.216+0000 7f66d4f65700 1 -- 172.21.15.77:0/3952071416 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f66d00fff70 msgr2=0x7f66d0194840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:25.228 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.216+0000 7f66d4f65700 1 --2- 172.21.15.77:0/3952071416 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f66d00fff70 0x7f66d0194840 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f66c000e970 tx=0x7f66c000ec80 comp rx=0 tx=0).stop 2023-12-01T01:18:25.228 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.216+0000 7f66d4f65700 1 -- 172.21.15.77:0/3952071416 shutdown_connections 2023-12-01T01:18:25.228 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.216+0000 7f66d4f65700 1 --2- 172.21.15.77:0/3952071416 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f66b80a0520 0x7f66b80a29e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:25.228 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.216+0000 7f66d4f65700 1 --2- 172.21.15.77:0/3952071416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66d0101240 0x7f66d019bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:25.228 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.216+0000 7f66d4f65700 1 --2- 172.21.15.77:0/3952071416 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f66d00fff70 0x7f66d0194840 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:25.229 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.217+0000 7f66d4f65700 1 --2- 172.21.15.77:0/3952071416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66d0100910 0x7f66d0194d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:25.229 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.217+0000 7f66d4f65700 1 -- 172.21.15.77:0/3952071416 >> 172.21.15.77:0/3952071416 conn(0x7f66d00fa980 msgr2=0x7f66d01043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:25.229 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.217+0000 7f66d4f65700 1 -- 172.21.15.77:0/3952071416 shutdown_connections 2023-12-01T01:18:25.230 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:25.217+0000 7f66d4f65700 1 -- 172.21.15.77:0/3952071416 wait complete. 2023-12-01T01:18:25.234 INFO:tasks.cephfs.filesystem:scrub status for tag:affe082b-75ae-4a4c-bc4c-8ae568688646 - {'path': '/', 'tag': 'affe082b-75ae-4a4c-bc4c-8ae568688646', 'options': 'recursive,force'} 2023-12-01T01:18:25.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:25 smithi077 ceph-mon[98867]: pgmap v314: 97 pgs: 97 active+clean; 653 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2023-12-01T01:18:26.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:25 smithi203 ceph-mon[103581]: pgmap v314: 97 pgs: 97 active+clean; 653 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2023-12-01T01:18:26.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:25 smithi137 ceph-mon[103312]: pgmap v314: 97 pgs: 97 active+clean; 653 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2023-12-01T01:18:26.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:26 smithi077 ceph-mon[98867]: scrub queued for path: / 2023-12-01T01:18:26.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:26 smithi077 ceph-mon[98867]: scrub summary: idle+waiting paths [/] 2023-12-01T01:18:26.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:26 smithi077 ceph-mon[98867]: scrub summary: active paths [/] 2023-12-01T01:18:26.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:26 smithi077 ceph-mon[98867]: scrub summary: idle+waiting paths [/] 2023-12-01T01:18:27.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:26 smithi203 ceph-mon[103581]: scrub queued for path: / 2023-12-01T01:18:27.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:26 smithi203 ceph-mon[103581]: scrub summary: idle+waiting paths [/] 2023-12-01T01:18:27.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:26 smithi203 ceph-mon[103581]: scrub summary: active paths [/] 2023-12-01T01:18:27.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:26 smithi203 ceph-mon[103581]: scrub summary: idle+waiting paths [/] 2023-12-01T01:18:27.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:26 smithi137 ceph-mon[103312]: scrub queued for path: / 2023-12-01T01:18:27.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:26 smithi137 ceph-mon[103312]: scrub summary: idle+waiting paths [/] 2023-12-01T01:18:27.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:26 smithi137 ceph-mon[103312]: scrub summary: active paths [/] 2023-12-01T01:18:27.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:26 smithi137 ceph-mon[103312]: scrub summary: idle+waiting paths [/] 2023-12-01T01:18:27.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:27 smithi077 ceph-mon[98867]: pgmap v315: 97 pgs: 97 active+clean; 653 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 255 B/s wr, 9 op/s 2023-12-01T01:18:28.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:27 smithi203 ceph-mon[103581]: pgmap v315: 97 pgs: 97 active+clean; 653 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 255 B/s wr, 9 op/s 2023-12-01T01:18:28.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:27 smithi137 ceph-mon[103312]: pgmap v315: 97 pgs: 97 active+clean; 653 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 255 B/s wr, 9 op/s 2023-12-01T01:18:29.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:29 smithi077 ceph-mon[98867]: pgmap v316: 97 pgs: 97 active+clean; 663 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2023-12-01T01:18:30.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:29 smithi203 ceph-mon[103581]: pgmap v316: 97 pgs: 97 active+clean; 663 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2023-12-01T01:18:30.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:29 smithi137 ceph-mon[103312]: pgmap v316: 97 pgs: 97 active+clean; 663 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2023-12-01T01:18:31.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:30 smithi203 ceph-mon[103581]: scrub summary: idle 2023-12-01T01:18:31.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:30 smithi203 ceph-mon[103581]: pgmap v317: 97 pgs: 97 active+clean; 663 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-12-01T01:18:31.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:30 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:18:31.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:30 smithi137 ceph-mon[103312]: scrub summary: idle 2023-12-01T01:18:31.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:30 smithi137 ceph-mon[103312]: pgmap v317: 97 pgs: 97 active+clean; 663 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-12-01T01:18:31.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:30 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:18:31.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:30 smithi077 ceph-mon[98867]: scrub summary: idle 2023-12-01T01:18:31.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:30 smithi077 ceph-mon[98867]: pgmap v317: 97 pgs: 97 active+clean; 663 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-12-01T01:18:31.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:30 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:18:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.162+0000 7f7a6912f700 1 -- 172.21.15.77:0/2998876539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a64100ef0 msgr2=0x7f7a64101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.162+0000 7f7a6912f700 1 --2- 172.21.15.77:0/2998876539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a64100ef0 0x7f7a64101350 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7a58009990 tx=0x7f7a58009ca0 comp rx=0 tx=0).stop 2023-12-01T01:18:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.162+0000 7f7a6912f700 1 -- 172.21.15.77:0/2998876539 shutdown_connections 2023-12-01T01:18:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.162+0000 7f7a6912f700 1 --2- 172.21.15.77:0/2998876539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a64101890 0x7f7a64068c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.162+0000 7f7a6912f700 1 --2- 172.21.15.77:0/2998876539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a640ffd00 0x7f7a64100120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.162+0000 7f7a6912f700 1 --2- 172.21.15.77:0/2998876539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a64100ef0 0x7f7a64101350 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.162+0000 7f7a6912f700 1 -- 172.21.15.77:0/2998876539 >> 172.21.15.77:0/2998876539 conn(0x7f7a640fb250 msgr2=0x7f7a640fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.162+0000 7f7a6912f700 1 -- 172.21.15.77:0/2998876539 shutdown_connections 2023-12-01T01:18:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.162+0000 7f7a6912f700 1 -- 172.21.15.77:0/2998876539 wait complete. 2023-12-01T01:18:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.163+0000 7f7a6912f700 1 Processor -- start 2023-12-01T01:18:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.163+0000 7f7a6912f700 1 -- start start 2023-12-01T01:18:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.163+0000 7f7a6912f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a640ffd00 0x7f7a64198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.163+0000 7f7a6912f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a64100ef0 0x7f7a64199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.163+0000 7f7a6912f700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a64101890 0x7f7a6419a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.163+0000 7f7a6912f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6419a7b0 con 0x7f7a64100ef0 2023-12-01T01:18:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.163+0000 7f7a6912f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6419a920 con 0x7f7a640ffd00 2023-12-01T01:18:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.163+0000 7f7a6912f700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6419aae0 con 0x7f7a64101890 2023-12-01T01:18:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.163+0000 7f7a6359e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a64101890 0x7f7a6419a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.163+0000 7f7a62d9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a640ffd00 0x7f7a64198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.163+0000 7f7a6359e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a64101890 0x7f7a6419a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:45962/0 (socket says 172.21.15.77:45962) 2023-12-01T01:18:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.163+0000 7f7a6359e700 1 -- 172.21.15.77:0/4215532927 learned_addr learned my addr 172.21.15.77:0/4215532927 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.163+0000 7f7a6259c700 1 --2- 172.21.15.77:0/4215532927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a64100ef0 0x7f7a64199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.164+0000 7f7a6359e700 1 -- 172.21.15.77:0/4215532927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a640ffd00 msgr2=0x7f7a64198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.164+0000 7f7a6359e700 1 --2- 172.21.15.77:0/4215532927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a640ffd00 0x7f7a64198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.164+0000 7f7a6359e700 1 -- 172.21.15.77:0/4215532927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a64100ef0 msgr2=0x7f7a64199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.164+0000 7f7a6359e700 1 --2- 172.21.15.77:0/4215532927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a64100ef0 0x7f7a64199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.164+0000 7f7a6359e700 1 -- 172.21.15.77:0/4215532927 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a580095f0 con 0x7f7a64101890 2023-12-01T01:18:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.164+0000 7f7a6359e700 1 --2- 172.21.15.77:0/4215532927 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a64101890 0x7f7a6419a0d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c00eb80 tx=0x7f7a5c00ee90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.164+0000 7f7a4bfff700 1 -- 172.21.15.77:0/4215532927 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c018070 con 0x7f7a64101890 2023-12-01T01:18:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.164+0000 7f7a6912f700 1 -- 172.21.15.77:0/4215532927 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a6419ad60 con 0x7f7a64101890 2023-12-01T01:18:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.164+0000 7f7a6912f700 1 -- 172.21.15.77:0/4215532927 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a641a1500 con 0x7f7a64101890 2023-12-01T01:18:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.164+0000 7f7a4bfff700 1 -- 172.21.15.77:0/4215532927 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a5c00f040 con 0x7f7a64101890 2023-12-01T01:18:31.167 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.164+0000 7f7a4bfff700 1 -- 172.21.15.77:0/4215532927 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c00cce0 con 0x7f7a64101890 2023-12-01T01:18:31.168 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.167+0000 7f7a4bfff700 1 -- 172.21.15.77:0/4215532927 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f7a5c003a50 con 0x7f7a64101890 2023-12-01T01:18:31.169 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.168+0000 7f7a4bfff700 1 --2- 172.21.15.77:0/4215532927 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7a4c0a05a0 0x7f7a4c0a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.168+0000 7f7a62d9d700 1 --2- 172.21.15.77:0/4215532927 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7a4c0a05a0 0x7f7a4c0a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.168+0000 7f7a4bfff700 1 -- 172.21.15.77:0/4215532927 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f7a5c0cb3b0 con 0x7f7a64101890 2023-12-01T01:18:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.169+0000 7f7a62d9d700 1 --2- 172.21.15.77:0/4215532927 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7a4c0a05a0 0x7f7a4c0a2a60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7a64199b60 tx=0x7f7a54009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.169+0000 7f7a752bfb80 1 Processor -- start 2023-12-01T01:18:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.169+0000 7f7a752bfb80 1 -- start start 2023-12-01T01:18:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.169+0000 7f7a752bfb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56525f3d3680 0x56525f3d3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.169+0000 7f7a752bfb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56525f3d3fe0 0x56525f3d48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.169+0000 7f7a752bfb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56525f3d4ed0 0x56525f3d5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.169+0000 7f7a752bfb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x56525f32b9b0 con 0x56525f3d4ed0 2023-12-01T01:18:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.169+0000 7f7a752bfb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56525f329370 con 0x56525f3d3680 2023-12-01T01:18:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.169+0000 7f7a752bfb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56525f324c60 con 0x56525f3d3fe0 2023-12-01T01:18:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.169+0000 7f7a6359e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56525f3d4ed0 0x56525f3d5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.169+0000 7f7a62d9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56525f3d3680 0x56525f3d3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.169+0000 7f7a62d9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56525f3d3680 0x56525f3d3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:38700/0 (socket says 172.21.15.77:38700) 2023-12-01T01:18:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.169+0000 7f7a6359e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56525f3d4ed0 0x56525f3d5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59782/0 (socket says 172.21.15.77:59782) 2023-12-01T01:18:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.169+0000 7f7a62d9d700 1 -- 172.21.15.77:0/4216764806 learned_addr learned my addr 172.21.15.77:0/4216764806 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.170+0000 7f7a6259c700 1 --2- 172.21.15.77:0/4216764806 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56525f3d3fe0 0x56525f3d48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.170+0000 7f7a62d9d700 1 -- 172.21.15.77:0/4216764806 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56525f3d3fe0 msgr2=0x56525f3d48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.170+0000 7f7a62d9d700 1 --2- 172.21.15.77:0/4216764806 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56525f3d3fe0 0x56525f3d48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.170+0000 7f7a62d9d700 1 -- 172.21.15.77:0/4216764806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56525f3d4ed0 msgr2=0x56525f3d5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.170+0000 7f7a62d9d700 1 --2- 172.21.15.77:0/4216764806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56525f3d4ed0 0x56525f3d5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.170+0000 7f7a62d9d700 1 -- 172.21.15.77:0/4216764806 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56525f152d80 con 0x56525f3d3680 2023-12-01T01:18:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.170+0000 7f7a6359e700 1 --2- 172.21.15.77:0/4216764806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56525f3d4ed0 0x56525f3d5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:18:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.170+0000 7f7a62d9d700 1 --2- 172.21.15.77:0/4216764806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56525f3d3680 0x56525f3d3aa0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7a54011d30 tx=0x7f7a54014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9c057407e42fc0d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.170+0000 7f7a497fa700 1 -- 172.21.15.77:0/4216764806 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5400eb20 con 0x56525f3d3680 2023-12-01T01:18:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.170+0000 7f7a497fa700 1 -- 172.21.15.77:0/4216764806 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a54021070 con 0x56525f3d3680 2023-12-01T01:18:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.170+0000 7f7a497fa700 1 -- 172.21.15.77:0/4216764806 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5401cea0 con 0x56525f3d3680 2023-12-01T01:18:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.171+0000 7f7a752bfb80 1 -- 172.21.15.77:0/4216764806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56525f3d3680 msgr2=0x56525f3d3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.171+0000 7f7a752bfb80 1 --2- 172.21.15.77:0/4216764806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56525f3d3680 0x56525f3d3aa0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7a54011d30 tx=0x7f7a54014000 comp rx=0 tx=0).stop 2023-12-01T01:18:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.171+0000 7f7a752bfb80 1 -- 172.21.15.77:0/4216764806 shutdown_connections 2023-12-01T01:18:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.171+0000 7f7a752bfb80 1 --2- 172.21.15.77:0/4216764806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56525f3d4ed0 0x56525f3d5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.171+0000 7f7a752bfb80 1 --2- 172.21.15.77:0/4216764806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56525f3d3680 0x56525f3d3aa0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.171+0000 7f7a752bfb80 1 --2- 172.21.15.77:0/4216764806 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56525f3d3fe0 0x56525f3d48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.171+0000 7f7a752bfb80 1 -- 172.21.15.77:0/4216764806 >> 172.21.15.77:0/4216764806 conn(0x56525f333070 msgr2=0x56525f3d0040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.171+0000 7f7a752bfb80 1 -- 172.21.15.77:0/4216764806 shutdown_connections 2023-12-01T01:18:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.171+0000 7f7a752bfb80 1 -- 172.21.15.77:0/4216764806 wait complete. 2023-12-01T01:18:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.172+0000 7f7a752bfb80 1 -- start start 2023-12-01T01:18:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.172+0000 7f7a752bfb80 1 Processor -- start 2023-12-01T01:18:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.172+0000 7f7a752bfb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56525f3d3680 0x56525f3dd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.172+0000 7f7a752bfb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56525f3d3fe0 0x56525f3dd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.172+0000 7f7a752bfb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56525f3d4ed0 0x56525f3e4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.172+0000 7f7a752bfb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x56525f2c14b0 con 0x56525f3d4ed0 2023-12-01T01:18:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.172+0000 7f7a752bfb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56525f2c1150 con 0x56525f3d3fe0 2023-12-01T01:18:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.172+0000 7f7a752bfb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56525f1a88c0 con 0x56525f3d3680 2023-12-01T01:18:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.172+0000 7f7a752bfb80 10 client.0 _ll_register_callbacks cb 0x56525f20ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:18:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.172+0000 7f7a62d9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56525f3d3680 0x56525f3dd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.172+0000 7f7a62d9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56525f3d3680 0x56525f3dd400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:45972/0 (socket says 172.21.15.77:45972) 2023-12-01T01:18:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.172+0000 7f7a62d9d700 1 -- 172.21.15.77:0/3846235381 learned_addr learned my addr 172.21.15.77:0/3846235381 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.172+0000 7f7a6259c700 1 --2- 172.21.15.77:0/3846235381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56525f3d3fe0 0x56525f3dd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.172+0000 7f7a6359e700 1 --2- 172.21.15.77:0/3846235381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56525f3d4ed0 0x56525f3e4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.173+0000 7f7a62d9d700 1 -- 172.21.15.77:0/3846235381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56525f3d3fe0 msgr2=0x56525f3dd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.173+0000 7f7a62d9d700 1 --2- 172.21.15.77:0/3846235381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56525f3d3fe0 0x56525f3dd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.173+0000 7f7a62d9d700 1 -- 172.21.15.77:0/3846235381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56525f3d4ed0 msgr2=0x56525f3e4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.173+0000 7f7a62d9d700 1 --2- 172.21.15.77:0/3846235381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56525f3d4ed0 0x56525f3e4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.173+0000 7f7a62d9d700 1 -- 172.21.15.77:0/3846235381 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a641a1500 con 0x56525f3d3680 2023-12-01T01:18:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.173+0000 7f7a62d9d700 1 --2- 172.21.15.77:0/3846235381 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56525f3d3680 0x56525f3dd400 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7a5400e970 tx=0x7f7a5401ce90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64358c70de59d90d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.173+0000 7f7a46ff5700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2023-12-01T01:18:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.173+0000 7f7a46ff5700 1 -- 172.21.15.77:0/3846235381 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a54011de0 con 0x56525f3d3680 2023-12-01T01:18:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.173+0000 7f7a752bfb80 1 -- 172.21.15.77:0/3846235381 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56525f34fcb0 con 0x56525f3d3680 2023-12-01T01:18:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.173+0000 7f7a46ff5700 1 -- 172.21.15.77:0/3846235381 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f7a5400ed30 con 0x56525f3d3680 2023-12-01T01:18:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.173+0000 7f7a46ff5700 1 -- 172.21.15.77:0/3846235381 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a5401c410 con 0x56525f3d3680 2023-12-01T01:18:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.173+0000 7f7a752bfb80 10 client.25282 fetch_fsmap learned FSMap version 27 2023-12-01T01:18:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.173+0000 7f7a752bfb80 1 -- 172.21.15.77:0/3846235381 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f7a54017070 con 0x56525f3d3680 2023-12-01T01:18:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.173+0000 7f7a46ff5700 1 -- 172.21.15.77:0/3846235381 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56525f34fcb0 con 0x56525f3d3680 2023-12-01T01:18:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.174+0000 7f7a46ff5700 1 -- 172.21.15.77:0/3846235381 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f7a540047a0 con 0x56525f3d3680 2023-12-01T01:18:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.174+0000 7f7a752bfb80 10 client.25282 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:18:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.174+0000 7f7a752bfb80 10 client.25282 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-01T01:18:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.174+0000 7f7a752bfb80 20 client.25282 populate_metadata read hostname 'smithi077' 2023-12-01T01:18:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.174+0000 7f7a752bfb80 1 --2- 172.21.15.77:0/3846235381 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x56525f3e9180 0x56525f3eb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.174+0000 7f7a752bfb80 4 client.25282 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:18:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.174+0000 7f7a752bfb80 1 -- 172.21.15.77:0/3846235381 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56525f3a2db0 con 0x56525f3e9180 2023-12-01T01:18:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.174+0000 7f7a6259c700 1 --2- 172.21.15.77:0/3846235381 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x56525f3e9180 0x56525f3eb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.174+0000 7f7a46ff5700 10 client.25282 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.174+0000 7f7a6259c700 1 --2- 172.21.15.77:0/3846235381 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x56525f3e9180 0x56525f3eb5a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de6000a2e8170cb5 server_cookie=aa80166fd48446cf in_seq=0 out_seq=0 2023-12-01T01:18:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.175+0000 7f7a46ff5700 1 -- 172.21.15.77:0/3846235381 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56525f3a2db0 con 0x56525f3e9180 2023-12-01T01:18:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.175+0000 7f7a46ff5700 10 client.25282 handle_command_reply: tid=0 2023-12-01T01:18:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.175+0000 7f7a752bfb80 1 client.25282 shutdown 2023-12-01T01:18:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.175+0000 7f7a752bfb80 1 -- 172.21.15.77:0/3846235381 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56525f3d3680 msgr2=0x56525f3dd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.175+0000 7f7a752bfb80 1 --2- 172.21.15.77:0/3846235381 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56525f3d3680 0x56525f3dd400 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7a5400e970 tx=0x7f7a5401ce90 comp rx=0 tx=0).stop 2023-12-01T01:18:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.175+0000 7f7a752bfb80 1 -- 172.21.15.77:0/3846235381 shutdown_connections 2023-12-01T01:18:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.175+0000 7f7a752bfb80 1 --2- 172.21.15.77:0/3846235381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56525f3d4ed0 0x56525f3e4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.175+0000 7f7a752bfb80 1 --2- 172.21.15.77:0/3846235381 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56525f3d3680 0x56525f3dd400 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.175+0000 7f7a752bfb80 1 --2- 172.21.15.77:0/3846235381 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x56525f3e9180 0x56525f3eb5a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.175+0000 7f7a752bfb80 1 --2- 172.21.15.77:0/3846235381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56525f3d3fe0 0x56525f3dd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.175+0000 7f7a46ff5700 0 client.25282 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.175+0000 7f7a752bfb80 1 -- 172.21.15.77:0/3846235381 >> 172.21.15.77:0/3846235381 conn(0x56525f333070 msgr2=0x56525f3ce3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.175+0000 7f7a752bfb80 1 -- 172.21.15.77:0/3846235381 shutdown_connections 2023-12-01T01:18:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.175+0000 7f7a752bfb80 1 -- 172.21.15.77:0/3846235381 wait complete. 2023-12-01T01:18:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.175+0000 7f7a752bfb80 20 client.25282 trim_cache size 0 max 16384 2023-12-01T01:18:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.199+0000 7f7a752bfb80 1 Processor -- start 2023-12-01T01:18:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.199+0000 7f7a752bfb80 1 -- start start 2023-12-01T01:18:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.199+0000 7f7a752bfb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56525f3d3680 0x56525f3e2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.199+0000 7f7a752bfb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56525f3d3fe0 0x56525f3dcab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.199+0000 7f7a752bfb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56525f3d4ed0 0x56525f3db220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.199+0000 7f7a752bfb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x56525f3c9500 con 0x56525f3d3fe0 2023-12-01T01:18:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.199+0000 7f7a752bfb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56525f3d0770 con 0x56525f3d3680 2023-12-01T01:18:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.200+0000 7f7a752bfb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56525f3dafd0 con 0x56525f3d4ed0 2023-12-01T01:18:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.200+0000 7f7a6259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56525f3d3fe0 0x56525f3dcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.200+0000 7f7a6259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56525f3d3fe0 0x56525f3dcab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59794/0 (socket says 172.21.15.77:59794) 2023-12-01T01:18:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.200+0000 7f7a62d9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56525f3d3680 0x56525f3e2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.200+0000 7f7a6259c700 1 -- 172.21.15.77:0/496449941 learned_addr learned my addr 172.21.15.77:0/496449941 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.200+0000 7f7a6359e700 1 --2- 172.21.15.77:0/496449941 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56525f3d4ed0 0x56525f3db220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.200+0000 7f7a6259c700 1 -- 172.21.15.77:0/496449941 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56525f3d4ed0 msgr2=0x56525f3db220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.200+0000 7f7a6259c700 1 --2- 172.21.15.77:0/496449941 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56525f3d4ed0 0x56525f3db220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.200+0000 7f7a6259c700 1 -- 172.21.15.77:0/496449941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56525f3d3680 msgr2=0x56525f3e2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.200+0000 7f7a6259c700 1 --2- 172.21.15.77:0/496449941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56525f3d3680 0x56525f3e2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.200+0000 7f7a6259c700 1 -- 172.21.15.77:0/496449941 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56525f13bc00 con 0x56525f3d3fe0 2023-12-01T01:18:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.200+0000 7f7a62d9d700 1 --2- 172.21.15.77:0/496449941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56525f3d3680 0x56525f3e2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:18:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.200+0000 7f7a6359e700 1 --2- 172.21.15.77:0/496449941 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56525f3d4ed0 0x56525f3db220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:18:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.200+0000 7f7a6259c700 1 --2- 172.21.15.77:0/496449941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56525f3d3fe0 0x56525f3dcab0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f7a58009990 tx=0x7f7a58005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10307bfb7d75db75 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.201+0000 7f7a49ffb700 1 -- 172.21.15.77:0/496449941 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5801d040 con 0x56525f3d3fe0 2023-12-01T01:18:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.201+0000 7f7a49ffb700 1 -- 172.21.15.77:0/496449941 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a5801e040 con 0x56525f3d3fe0 2023-12-01T01:18:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.201+0000 7f7a49ffb700 1 -- 172.21.15.77:0/496449941 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a58028e40 con 0x56525f3d3fe0 2023-12-01T01:18:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.201+0000 7f7a752bfb80 1 -- 172.21.15.77:0/496449941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56525f3d3fe0 msgr2=0x56525f3dcab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.201+0000 7f7a752bfb80 1 --2- 172.21.15.77:0/496449941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56525f3d3fe0 0x56525f3dcab0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f7a58009990 tx=0x7f7a58005fb0 comp rx=0 tx=0).stop 2023-12-01T01:18:31.205 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.201+0000 7f7a752bfb80 1 -- 172.21.15.77:0/496449941 shutdown_connections 2023-12-01T01:18:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.201+0000 7f7a752bfb80 1 --2- 172.21.15.77:0/496449941 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56525f3d4ed0 0x56525f3db220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.201+0000 7f7a752bfb80 1 --2- 172.21.15.77:0/496449941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56525f3d3680 0x56525f3e2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.201+0000 7f7a752bfb80 1 --2- 172.21.15.77:0/496449941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56525f3d3fe0 0x56525f3dcab0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.201+0000 7f7a752bfb80 1 -- 172.21.15.77:0/496449941 >> 172.21.15.77:0/496449941 conn(0x56525f333070 msgr2=0x56525f3cd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.201+0000 7f7a752bfb80 1 -- 172.21.15.77:0/496449941 shutdown_connections 2023-12-01T01:18:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.202+0000 7f7a752bfb80 1 -- 172.21.15.77:0/496449941 wait complete. 2023-12-01T01:18:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.202+0000 7f7a752bfb80 1 -- start start 2023-12-01T01:18:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.202+0000 7f7a752bfb80 1 Processor -- start 2023-12-01T01:18:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.202+0000 7f7a752bfb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56525f3d3680 0x56525f432e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.202+0000 7f7a752bfb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56525f3d3fe0 0x56525f4333a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.202+0000 7f7a752bfb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56525f3d4ed0 0x56525f439e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.202+0000 7f7a752bfb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x56525f43a380 con 0x56525f3d4ed0 2023-12-01T01:18:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.202+0000 7f7a752bfb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56525f43a4f0 con 0x56525f3d3680 2023-12-01T01:18:31.206 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.202+0000 7f7a752bfb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56525f43a6b0 con 0x56525f3d3fe0 2023-12-01T01:18:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.202+0000 7f7a752bfb80 10 client.0 _ll_register_callbacks cb 0x7f7a5c009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:18:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.202+0000 7f7a62d9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56525f3d3680 0x56525f432e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.202+0000 7f7a62d9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56525f3d3680 0x56525f432e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:38720/0 (socket says 172.21.15.77:38720) 2023-12-01T01:18:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.202+0000 7f7a62d9d700 1 -- 172.21.15.77:0/1766439168 learned_addr learned my addr 172.21.15.77:0/1766439168 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.202+0000 7f7a6359e700 1 --2- 172.21.15.77:0/1766439168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56525f3d4ed0 0x56525f439e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.203+0000 7f7a6259c700 1 --2- 172.21.15.77:0/1766439168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56525f3d3fe0 0x56525f4333a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.203+0000 7f7a62d9d700 1 -- 172.21.15.77:0/1766439168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56525f3d3fe0 msgr2=0x56525f4333a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.203+0000 7f7a62d9d700 1 --2- 172.21.15.77:0/1766439168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56525f3d3fe0 0x56525f4333a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.203+0000 7f7a62d9d700 1 -- 172.21.15.77:0/1766439168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56525f3d4ed0 msgr2=0x56525f439e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.203+0000 7f7a62d9d700 1 --2- 172.21.15.77:0/1766439168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56525f3d4ed0 0x56525f439e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.203+0000 7f7a62d9d700 1 -- 172.21.15.77:0/1766439168 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a6419ad60 con 0x56525f3d3680 2023-12-01T01:18:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.203+0000 7f7a6259c700 1 --2- 172.21.15.77:0/1766439168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56525f3d3fe0 0x56525f4333a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:18:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.203+0000 7f7a62d9d700 1 --2- 172.21.15.77:0/1766439168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56525f3d3680 0x56525f432e60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7a5400e970 tx=0x7f7a54009350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5812bc590bcd5daa server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.203+0000 7f7a46ff5700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2023-12-01T01:18:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.203+0000 7f7a46ff5700 1 -- 172.21.15.77:0/1766439168 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a54004710 con 0x56525f3d3680 2023-12-01T01:18:31.208 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.203+0000 7f7a46ff5700 1 -- 172.21.15.77:0/1766439168 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f7a54004870 con 0x56525f3d3680 2023-12-01T01:18:31.208 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.203+0000 7f7a752bfb80 1 -- 172.21.15.77:0/1766439168 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a54021070 con 0x56525f3d3680 2023-12-01T01:18:31.209 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.204+0000 7f7a46ff5700 1 -- 172.21.15.77:0/1766439168 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a5401c410 con 0x56525f3d3680 2023-12-01T01:18:31.209 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.204+0000 7f7a752bfb80 10 client.25274 fetch_fsmap learned FSMap version 27 2023-12-01T01:18:31.209 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.204+0000 7f7a752bfb80 1 -- 172.21.15.77:0/1766439168 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f7a64069670 con 0x56525f3d3680 2023-12-01T01:18:31.209 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.204+0000 7f7a46ff5700 1 -- 172.21.15.77:0/1766439168 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a54021070 con 0x56525f3d3680 2023-12-01T01:18:31.209 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.204+0000 7f7a46ff5700 1 -- 172.21.15.77:0/1766439168 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f7a540050d0 con 0x56525f3d3680 2023-12-01T01:18:31.209 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.204+0000 7f7a752bfb80 10 client.25274 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:18:31.209 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.204+0000 7f7a752bfb80 10 client.25274 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-01T01:18:31.210 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.204+0000 7f7a752bfb80 20 client.25274 populate_metadata read hostname 'smithi077' 2023-12-01T01:18:31.210 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.204+0000 7f7a752bfb80 1 --2- 172.21.15.77:0/1766439168 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x56525f43bd90 0x56525f43e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:31.210 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.204+0000 7f7a752bfb80 4 client.25274 mds_command: new command op to 24500 tid=0[{"prefix": "flush journal"}] 2023-12-01T01:18:31.210 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.204+0000 7f7a752bfb80 1 -- 172.21.15.77:0/1766439168 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56525f31dce0 con 0x56525f43bd90 2023-12-01T01:18:31.210 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.204+0000 7f7a6259c700 1 --2- 172.21.15.77:0/1766439168 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x56525f43bd90 0x56525f43e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:31.210 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.205+0000 7f7a46ff5700 10 client.25274 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:31.210 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.205+0000 7f7a6259c700 1 --2- 172.21.15.77:0/1766439168 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x56525f43bd90 0x56525f43e1b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9c600c185a2082e server_cookie=b30ddc16d9a42881 in_seq=0 out_seq=0 2023-12-01T01:18:31.213 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.211+0000 7f7a46ff5700 1 -- 172.21.15.77:0/1766439168 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56525f31dce0 con 0x56525f43bd90 2023-12-01T01:18:31.213 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.211+0000 7f7a46ff5700 10 client.25274 handle_command_reply: tid=0 2023-12-01T01:18:31.213 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.211+0000 7f7a752bfb80 1 client.25274 shutdown 2023-12-01T01:18:31.213 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.211+0000 7f7a752bfb80 1 -- 172.21.15.77:0/1766439168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56525f3d3680 msgr2=0x56525f432e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:31.213 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.212+0000 7f7a752bfb80 1 --2- 172.21.15.77:0/1766439168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56525f3d3680 0x56525f432e60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7a5400e970 tx=0x7f7a54009350 comp rx=0 tx=0).stop 2023-12-01T01:18:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.212+0000 7f7a752bfb80 1 -- 172.21.15.77:0/1766439168 shutdown_connections 2023-12-01T01:18:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.212+0000 7f7a752bfb80 1 --2- 172.21.15.77:0/1766439168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56525f3d4ed0 0x56525f439e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.212+0000 7f7a752bfb80 1 --2- 172.21.15.77:0/1766439168 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x56525f43bd90 0x56525f43e1b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.212+0000 7f7a752bfb80 1 --2- 172.21.15.77:0/1766439168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56525f3d3680 0x56525f432e60 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.212+0000 7f7a752bfb80 1 --2- 172.21.15.77:0/1766439168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56525f3d3fe0 0x56525f4333a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.212+0000 7f7a752bfb80 1 -- 172.21.15.77:0/1766439168 >> 172.21.15.77:0/1766439168 conn(0x56525f333070 msgr2=0x56525f3cfc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.212+0000 7f7a46ff5700 0 client.25274 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.212+0000 7f7a752bfb80 1 -- 172.21.15.77:0/1766439168 shutdown_connections 2023-12-01T01:18:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.212+0000 7f7a752bfb80 1 -- 172.21.15.77:0/1766439168 wait complete. 2023-12-01T01:18:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.212+0000 7f7a752bfb80 20 client.25274 trim_cache size 0 max 16384 2023-12-01T01:18:31.214 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout:{ 2023-12-01T01:18:31.215 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout: "message": "", 2023-12-01T01:18:31.215 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout: "return_code": 0 2023-12-01T01:18:31.215 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout:} 2023-12-01T01:18:31.215 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.212+0000 7f7a6912f700 1 -- 172.21.15.77:0/4215532927 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7a4c0a05a0 msgr2=0x7f7a4c0a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:31.215 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.212+0000 7f7a6912f700 1 --2- 172.21.15.77:0/4215532927 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7a4c0a05a0 0x7f7a4c0a2a60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7a64199b60 tx=0x7f7a54009260 comp rx=0 tx=0).stop 2023-12-01T01:18:31.215 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.212+0000 7f7a6912f700 1 -- 172.21.15.77:0/4215532927 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a64101890 msgr2=0x7f7a6419a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:31.215 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.212+0000 7f7a6912f700 1 --2- 172.21.15.77:0/4215532927 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a64101890 0x7f7a6419a0d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c00eb80 tx=0x7f7a5c00ee90 comp rx=0 tx=0).stop 2023-12-01T01:18:31.215 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.213+0000 7f7a6912f700 1 -- 172.21.15.77:0/4215532927 shutdown_connections 2023-12-01T01:18:31.215 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.213+0000 7f7a6912f700 1 --2- 172.21.15.77:0/4215532927 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7a4c0a05a0 0x7f7a4c0a2a60 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.213+0000 7f7a6912f700 1 --2- 172.21.15.77:0/4215532927 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a64101890 0x7f7a6419a0d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.213+0000 7f7a6912f700 1 --2- 172.21.15.77:0/4215532927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a640ffd00 0x7f7a64198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.213+0000 7f7a6912f700 1 --2- 172.21.15.77:0/4215532927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a64100ef0 0x7f7a64199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:31.216 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.213+0000 7f7a6912f700 1 -- 172.21.15.77:0/4215532927 >> 172.21.15.77:0/4215532927 conn(0x7f7a640fb250 msgr2=0x7f7a640fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.216+0000 7f7a6912f700 1 -- 172.21.15.77:0/4215532927 shutdown_connections 2023-12-01T01:18:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:31.216+0000 7f7a6912f700 1 -- 172.21.15.77:0/4215532927 wait complete. 2023-12-01T01:18:33.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:33 smithi077 ceph-mon[98867]: pgmap v318: 97 pgs: 97 active+clean; 663 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2023-12-01T01:18:33.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:33 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:18:33.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:33 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:18:33.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:33 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:18:33.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:33 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:18:33.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:33 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:18:33.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:33 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:18:34.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:33 smithi203 ceph-mon[103581]: pgmap v318: 97 pgs: 97 active+clean; 663 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2023-12-01T01:18:34.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:33 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:18:34.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:33 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:18:34.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:33 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:18:34.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:33 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:18:34.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:33 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:18:34.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:33 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:18:34.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:33 smithi137 ceph-mon[103312]: pgmap v318: 97 pgs: 97 active+clean; 663 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2023-12-01T01:18:34.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:33 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:18:34.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:33 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:18:34.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:33 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:18:34.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:33 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:18:34.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:33 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:18:34.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:33 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:18:34.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:34 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:18:34.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:34 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:18:34.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:34 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:18:35.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:34 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:18:35.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:34 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:18:35.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:34 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:18:35.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:34 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:18:35.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:34 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:18:35.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:34 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:18:35.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:35 smithi077 ceph-mon[98867]: pgmap v319: 97 pgs: 97 active+clean; 664 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2023-12-01T01:18:36.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:35 smithi203 ceph-mon[103581]: pgmap v319: 97 pgs: 97 active+clean; 664 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2023-12-01T01:18:36.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:35 smithi137 ceph-mon[103312]: pgmap v319: 97 pgs: 97 active+clean; 664 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2023-12-01T01:18:36.776 INFO:tasks.workunit.client.0.smithi077.stderr:Note: switching to 'a6f82997ac57b70a895455dfed6256360a1e4c32'. 2023-12-01T01:18:36.776 INFO:tasks.workunit.client.0.smithi077.stderr: 2023-12-01T01:18:36.777 INFO:tasks.workunit.client.0.smithi077.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-12-01T01:18:36.777 INFO:tasks.workunit.client.0.smithi077.stderr:changes and commit them, and you can discard any commits you make in this 2023-12-01T01:18:36.777 INFO:tasks.workunit.client.0.smithi077.stderr:state without impacting any branches by switching back to a branch. 2023-12-01T01:18:36.777 INFO:tasks.workunit.client.0.smithi077.stderr: 2023-12-01T01:18:36.777 INFO:tasks.workunit.client.0.smithi077.stderr:If you want to create a new branch to retain commits you create, you may 2023-12-01T01:18:36.777 INFO:tasks.workunit.client.0.smithi077.stderr:do so (now or later) by using -c with the switch command. Example: 2023-12-01T01:18:36.777 INFO:tasks.workunit.client.0.smithi077.stderr: 2023-12-01T01:18:36.777 INFO:tasks.workunit.client.0.smithi077.stderr: git switch -c 2023-12-01T01:18:36.777 INFO:tasks.workunit.client.0.smithi077.stderr: 2023-12-01T01:18:36.777 INFO:tasks.workunit.client.0.smithi077.stderr:Or undo this operation with: 2023-12-01T01:18:36.777 INFO:tasks.workunit.client.0.smithi077.stderr: 2023-12-01T01:18:36.777 INFO:tasks.workunit.client.0.smithi077.stderr: git switch - 2023-12-01T01:18:36.777 INFO:tasks.workunit.client.0.smithi077.stderr: 2023-12-01T01:18:36.777 INFO:tasks.workunit.client.0.smithi077.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-12-01T01:18:36.777 INFO:tasks.workunit.client.0.smithi077.stderr: 2023-12-01T01:18:36.778 INFO:tasks.workunit.client.0.smithi077.stderr:HEAD is now at a6f82997ac5 qa/cephfs: try usual umount if client blocklisting cannot be inferred 2023-12-01T01:18:36.781 DEBUG:teuthology.orchestra.run.smithi077:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-12-01T01:18:36.803 INFO:tasks.workunit.client.0.smithi077.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-12-01T01:18:36.805 INFO:tasks.workunit.client.0.smithi077.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-12-01T01:18:36.805 INFO:tasks.workunit.client.0.smithi077.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-12-01T01:18:37.371 INFO:tasks.workunit.client.0.smithi077.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-12-01T01:18:37.425 INFO:tasks.workunit.client.0.smithi077.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-12-01T01:18:37.462 INFO:tasks.workunit.client.0.smithi077.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-12-01T01:18:37.464 INFO:tasks.workunit.client.0.smithi077.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-12-01T01:18:37.464 INFO:tasks.workunit.client.0.smithi077.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-12-01T01:18:37.500 INFO:tasks.workunit.client.0.smithi077.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-12-01T01:18:37.503 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-12-01T01:18:37.503 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-12-01T01:18:37.561 INFO:tasks.workunit:Running workunits matching suites/fsx.sh on client.0... 2023-12-01T01:18:37.562 INFO:tasks.workunit:Running workunit suites/fsx.sh... 2023-12-01T01:18:37.562 DEBUG:teuthology.orchestra.run.smithi077:workunit test suites/fsx.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a6f82997ac57b70a895455dfed6256360a1e4c32 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh 2023-12-01T01:18:37.624 INFO:tasks.workunit.client.0.smithi077.stderr:+ set -e 2023-12-01T01:18:37.624 INFO:tasks.workunit.client.0.smithi077.stderr:+ git clone https://git.ceph.com/xfstests-dev.git 2023-12-01T01:18:37.625 INFO:tasks.workunit.client.0.smithi077.stderr:Cloning into 'xfstests-dev'... 2023-12-01T01:18:37.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:37 smithi077 ceph-mon[98867]: pgmap v320: 97 pgs: 97 active+clean; 664 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2023-12-01T01:18:38.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:37 smithi203 ceph-mon[103581]: pgmap v320: 97 pgs: 97 active+clean; 664 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2023-12-01T01:18:38.071 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:37 smithi137 ceph-mon[103312]: pgmap v320: 97 pgs: 97 active+clean; 664 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2023-12-01T01:18:39.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:39 smithi077 ceph-mon[98867]: pgmap v321: 97 pgs: 97 active+clean; 664 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2023-12-01T01:18:40.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:39 smithi203 ceph-mon[103581]: pgmap v321: 97 pgs: 97 active+clean; 664 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2023-12-01T01:18:40.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:39 smithi137 ceph-mon[103312]: pgmap v321: 97 pgs: 97 active+clean; 664 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2023-12-01T01:18:41.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:41 smithi077 ceph-mon[98867]: pgmap v322: 97 pgs: 97 active+clean; 1.9 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 12 op/s 2023-12-01T01:18:42.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:41 smithi203 ceph-mon[103581]: pgmap v322: 97 pgs: 97 active+clean; 1.9 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 12 op/s 2023-12-01T01:18:42.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:41 smithi137 ceph-mon[103312]: pgmap v322: 97 pgs: 97 active+clean; 1.9 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 105 KiB/s wr, 12 op/s 2023-12-01T01:18:43.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:43 smithi077 ceph-mon[98867]: pgmap v323: 97 pgs: 97 active+clean; 1.9 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-12-01T01:18:44.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:43 smithi203 ceph-mon[103581]: pgmap v323: 97 pgs: 97 active+clean; 1.9 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-12-01T01:18:44.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:43 smithi137 ceph-mon[103312]: pgmap v323: 97 pgs: 97 active+clean; 1.9 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 110 KiB/s wr, 11 op/s 2023-12-01T01:18:44.336 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.335+0000 7f61aef06700 1 -- 172.21.15.77:0/725325513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61a8106ad0 msgr2=0x7f61a8108ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.335+0000 7f61aef06700 1 --2- 172.21.15.77:0/725325513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61a8106ad0 0x7f61a8108ee0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f61a000b0a0 tx=0x7f61a000b3b0 comp rx=0 tx=0).stop 2023-12-01T01:18:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.335+0000 7f61aef06700 1 -- 172.21.15.77:0/725325513 shutdown_connections 2023-12-01T01:18:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.335+0000 7f61aef06700 1 --2- 172.21.15.77:0/725325513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61a8106ad0 0x7f61a8108ee0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.335+0000 7f61aef06700 1 --2- 172.21.15.77:0/725325513 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61a8101870 0x7f61a8103c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.335+0000 7f61aef06700 1 --2- 172.21.15.77:0/725325513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61a81041a0 0x7f61a8106590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.335+0000 7f61aef06700 1 -- 172.21.15.77:0/725325513 >> 172.21.15.77:0/725325513 conn(0x7f61a80fb1e0 msgr2=0x7f61a80fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.335+0000 7f61aef06700 1 -- 172.21.15.77:0/725325513 shutdown_connections 2023-12-01T01:18:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.335+0000 7f61aef06700 1 -- 172.21.15.77:0/725325513 wait complete. 2023-12-01T01:18:44.339 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.336+0000 7f61aef06700 1 Processor -- start 2023-12-01T01:18:44.339 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.336+0000 7f61aef06700 1 -- start start 2023-12-01T01:18:44.339 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.336+0000 7f61aef06700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61a8101870 0x7f61a8196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:44.339 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.336+0000 7f61aef06700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61a81041a0 0x7f61a8197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.336+0000 7f61aef06700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61a8106ad0 0x7f61a8197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.336+0000 7f61aef06700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a8198620 con 0x7f61a81041a0 2023-12-01T01:18:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.336+0000 7f61aef06700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a8198790 con 0x7f61a8106ad0 2023-12-01T01:18:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.336+0000 7f61aef06700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a8198950 con 0x7f61a8101870 2023-12-01T01:18:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.336+0000 7f61a7fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61a81041a0 0x7f61a8197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.336+0000 7f61acca2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61a8101870 0x7f61a8196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.336+0000 7f61a7fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61a81041a0 0x7f61a8197190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59826/0 (socket says 172.21.15.77:59826) 2023-12-01T01:18:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.336+0000 7f61acca2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61a8101870 0x7f61a8196c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:46012/0 (socket says 172.21.15.77:46012) 2023-12-01T01:18:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.336+0000 7f61a7fff700 1 -- 172.21.15.77:0/2986268312 learned_addr learned my addr 172.21.15.77:0/2986268312 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.337+0000 7f61ad4a3700 1 --2- 172.21.15.77:0/2986268312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61a8106ad0 0x7f61a8197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.337+0000 7f61a7fff700 1 -- 172.21.15.77:0/2986268312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61a8101870 msgr2=0x7f61a8196c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.337+0000 7f61a7fff700 1 --2- 172.21.15.77:0/2986268312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61a8101870 0x7f61a8196c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.337+0000 7f61a7fff700 1 -- 172.21.15.77:0/2986268312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61a8106ad0 msgr2=0x7f61a8197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.337+0000 7f61a7fff700 1 --2- 172.21.15.77:0/2986268312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61a8106ad0 0x7f61a8197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.337+0000 7f61a7fff700 1 -- 172.21.15.77:0/2986268312 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a0009d00 con 0x7f61a81041a0 2023-12-01T01:18:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.337+0000 7f61a7fff700 1 --2- 172.21.15.77:0/2986268312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61a81041a0 0x7f61a8197190 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f619400e9a0 tx=0x7f619400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.337+0000 7f61a5ffb700 1 -- 172.21.15.77:0/2986268312 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6194009790 con 0x7f61a81041a0 2023-12-01T01:18:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.337+0000 7f61aef06700 1 -- 172.21.15.77:0/2986268312 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61a8198c30 con 0x7f61a81041a0 2023-12-01T01:18:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.337+0000 7f61a5ffb700 1 -- 172.21.15.77:0/2986268312 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f619400cbd0 con 0x7f61a81041a0 2023-12-01T01:18:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.337+0000 7f61a5ffb700 1 -- 172.21.15.77:0/2986268312 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6194018680 con 0x7f61a81041a0 2023-12-01T01:18:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.337+0000 7f61aef06700 1 -- 172.21.15.77:0/2986268312 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61a819f3f0 con 0x7f61a81041a0 2023-12-01T01:18:44.344 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.339+0000 7f61a5ffb700 1 -- 172.21.15.77:0/2986268312 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f619400cd40 con 0x7f61a81041a0 2023-12-01T01:18:44.344 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.340+0000 7f61a5ffb700 1 --2- 172.21.15.77:0/2986268312 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f61980a05a0 0x7f61980a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:44.344 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.340+0000 7f61a5ffb700 1 -- 172.21.15.77:0/2986268312 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f6194014070 con 0x7f61a81041a0 2023-12-01T01:18:44.344 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.340+0000 7f61acca2700 1 --2- 172.21.15.77:0/2986268312 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f61980a05a0 0x7f61980a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:44.344 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.340+0000 7f61acca2700 1 --2- 172.21.15.77:0/2986268312 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f61980a05a0 0x7f61980a2a60 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f619c009c60 tx=0x7f619c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:44.344 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.341+0000 7f61bb096b80 1 Processor -- start 2023-12-01T01:18:44.344 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.341+0000 7f61bb096b80 1 -- start start 2023-12-01T01:18:44.345 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.341+0000 7f61bb096b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc60c11680 0x55dc60c11aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:44.345 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.341+0000 7f61bb096b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc60c11fe0 0x55dc60c128f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:44.345 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.341+0000 7f61bb096b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dc60c12ed0 0x55dc60c13350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:44.345 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.341+0000 7f61bb096b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc60b699b0 con 0x55dc60c11680 2023-12-01T01:18:44.345 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.341+0000 7f61bb096b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc60b67370 con 0x55dc60c11fe0 2023-12-01T01:18:44.345 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.341+0000 7f61bb096b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc60b62c60 con 0x55dc60c12ed0 2023-12-01T01:18:44.345 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.341+0000 7f61acca2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc60c11680 0x55dc60c11aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:44.345 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.341+0000 7f61acca2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc60c11680 0x55dc60c11aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59832/0 (socket says 172.21.15.77:59832) 2023-12-01T01:18:44.345 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.341+0000 7f61acca2700 1 -- 172.21.15.77:0/585871208 learned_addr learned my addr 172.21.15.77:0/585871208 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:44.345 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.341+0000 7f61a7fff700 1 --2- 172.21.15.77:0/585871208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc60c11fe0 0x55dc60c128f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:44.345 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.341+0000 7f61ad4a3700 1 --2- 172.21.15.77:0/585871208 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dc60c12ed0 0x55dc60c13350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:44.345 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.341+0000 7f61acca2700 1 -- 172.21.15.77:0/585871208 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dc60c12ed0 msgr2=0x55dc60c13350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:44.345 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.341+0000 7f61acca2700 1 --2- 172.21.15.77:0/585871208 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dc60c12ed0 0x55dc60c13350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:44.345 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.341+0000 7f61acca2700 1 -- 172.21.15.77:0/585871208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc60c11fe0 msgr2=0x55dc60c128f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:44.345 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.341+0000 7f61acca2700 1 --2- 172.21.15.77:0/585871208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc60c11fe0 0x55dc60c128f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:44.346 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.342+0000 7f61acca2700 1 -- 172.21.15.77:0/585871208 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc60990d80 con 0x55dc60c11680 2023-12-01T01:18:44.346 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.342+0000 7f61a7fff700 1 --2- 172.21.15.77:0/585871208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc60c11fe0 0x55dc60c128f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:18:44.346 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.342+0000 7f61acca2700 1 --2- 172.21.15.77:0/585871208 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc60c11680 0x55dc60c11aa0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f619c00e4e0 tx=0x7f619c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d25dad44bed6512 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:44.346 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.342+0000 7f6192ffd700 1 -- 172.21.15.77:0/585871208 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619c026430 con 0x55dc60c11680 2023-12-01T01:18:44.346 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.342+0000 7f6192ffd700 1 -- 172.21.15.77:0/585871208 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f619c021070 con 0x55dc60c11680 2023-12-01T01:18:44.346 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.342+0000 7f6192ffd700 1 -- 172.21.15.77:0/585871208 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619c01d6e0 con 0x55dc60c11680 2023-12-01T01:18:44.346 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.343+0000 7f61bb096b80 1 -- 172.21.15.77:0/585871208 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc60c11680 msgr2=0x55dc60c11aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:44.346 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.343+0000 7f61bb096b80 1 --2- 172.21.15.77:0/585871208 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc60c11680 0x55dc60c11aa0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f619c00e4e0 tx=0x7f619c014000 comp rx=0 tx=0).stop 2023-12-01T01:18:44.346 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.343+0000 7f61bb096b80 1 -- 172.21.15.77:0/585871208 shutdown_connections 2023-12-01T01:18:44.346 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.343+0000 7f61bb096b80 1 --2- 172.21.15.77:0/585871208 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dc60c12ed0 0x55dc60c13350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:44.346 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.343+0000 7f61bb096b80 1 --2- 172.21.15.77:0/585871208 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc60c11680 0x55dc60c11aa0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:44.346 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.343+0000 7f61bb096b80 1 --2- 172.21.15.77:0/585871208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc60c11fe0 0x55dc60c128f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:44.346 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.343+0000 7f61bb096b80 1 -- 172.21.15.77:0/585871208 >> 172.21.15.77:0/585871208 conn(0x55dc60b71070 msgr2=0x55dc60c0e040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:44.347 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.343+0000 7f61bb096b80 1 -- 172.21.15.77:0/585871208 shutdown_connections 2023-12-01T01:18:44.347 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.343+0000 7f61bb096b80 1 -- 172.21.15.77:0/585871208 wait complete. 2023-12-01T01:18:44.347 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.343+0000 7f61bb096b80 1 -- start start 2023-12-01T01:18:44.347 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.343+0000 7f61bb096b80 1 Processor -- start 2023-12-01T01:18:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.343+0000 7f61bb096b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc60c11680 0x55dc60c1b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.343+0000 7f61bb096b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc60c11fe0 0x55dc60c1ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.343+0000 7f61bb096b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dc60c12ed0 0x55dc60c22a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.343+0000 7f61bb096b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc60aff4b0 con 0x55dc60c11680 2023-12-01T01:18:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.343+0000 7f61bb096b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc60aff150 con 0x55dc60c11fe0 2023-12-01T01:18:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.344+0000 7f61bb096b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc609e68c0 con 0x55dc60c12ed0 2023-12-01T01:18:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.344+0000 7f61bb096b80 10 client.0 _ll_register_callbacks cb 0x55dc60a4ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:18:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.344+0000 7f61acca2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc60c11680 0x55dc60c1b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.344+0000 7f61acca2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc60c11680 0x55dc60c1b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59838/0 (socket says 172.21.15.77:59838) 2023-12-01T01:18:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.344+0000 7f61acca2700 1 -- 172.21.15.77:0/4080124241 learned_addr learned my addr 172.21.15.77:0/4080124241 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.344+0000 7f61a7fff700 1 --2- 172.21.15.77:0/4080124241 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc60c11fe0 0x55dc60c1ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.344+0000 7f61ad4a3700 1 --2- 172.21.15.77:0/4080124241 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dc60c12ed0 0x55dc60c22a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.344+0000 7f61acca2700 1 -- 172.21.15.77:0/4080124241 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dc60c12ed0 msgr2=0x55dc60c22a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.344+0000 7f61acca2700 1 --2- 172.21.15.77:0/4080124241 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dc60c12ed0 0x55dc60c22a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.344+0000 7f61acca2700 1 -- 172.21.15.77:0/4080124241 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc60c11fe0 msgr2=0x55dc60c1ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.344+0000 7f61acca2700 1 --2- 172.21.15.77:0/4080124241 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc60c11fe0 0x55dc60c1ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.344+0000 7f61acca2700 1 -- 172.21.15.77:0/4080124241 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f619c017070 con 0x55dc60c11680 2023-12-01T01:18:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.344+0000 7f61ad4a3700 1 --2- 172.21.15.77:0/4080124241 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dc60c12ed0 0x55dc60c22a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:18:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.344+0000 7f61acca2700 1 --2- 172.21.15.77:0/4080124241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc60c11680 0x55dc60c1b4d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f619c00ee10 tx=0x7f619c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed59e5681c644da9 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.344+0000 7f6187fff700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:18:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.344+0000 7f6187fff700 1 -- 172.21.15.77:0/4080124241 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619c0053d0 con 0x55dc60c11680 2023-12-01T01:18:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.344+0000 7f6187fff700 1 -- 172.21.15.77:0/4080124241 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f619c005530 con 0x55dc60c11680 2023-12-01T01:18:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.344+0000 7f61bb096b80 1 -- 172.21.15.77:0/4080124241 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dc60b8dcb0 con 0x55dc60c11680 2023-12-01T01:18:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.344+0000 7f6187fff700 1 -- 172.21.15.77:0/4080124241 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f619c0056f0 con 0x55dc60c11680 2023-12-01T01:18:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.344+0000 7f61bb096b80 10 client.15696 fetch_fsmap learned FSMap version 27 2023-12-01T01:18:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.344+0000 7f61bb096b80 1 -- 172.21.15.77:0/4080124241 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55dc60979c00 con 0x55dc60c11680 2023-12-01T01:18:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.345+0000 7f6187fff700 1 -- 172.21.15.77:0/4080124241 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dc60b8dcb0 con 0x55dc60c11680 2023-12-01T01:18:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.345+0000 7f6187fff700 1 -- 172.21.15.77:0/4080124241 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f619c003c10 con 0x55dc60c11680 2023-12-01T01:18:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.345+0000 7f61bb096b80 10 client.15696 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:18:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.345+0000 7f61bb096b80 10 client.15696 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-01T01:18:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.345+0000 7f61bb096b80 20 client.15696 populate_metadata read hostname 'smithi077' 2023-12-01T01:18:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.345+0000 7f61bb096b80 1 --2- 172.21.15.77:0/4080124241 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55dc60c27140 0x55dc60c29560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.345+0000 7f61bb096b80 4 client.15696 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:18:44.353 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.345+0000 7f61bb096b80 1 -- 172.21.15.77:0/4080124241 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc60be0db0 con 0x55dc60c27140 2023-12-01T01:18:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.345+0000 7f61a7fff700 1 --2- 172.21.15.77:0/4080124241 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55dc60c27140 0x55dc60c29560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.345+0000 7f61a7fff700 1 --2- 172.21.15.77:0/4080124241 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55dc60c27140 0x55dc60c29560 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5266ad35dc520ebb server_cookie=41f0fba4e06f3549 in_seq=0 out_seq=0 2023-12-01T01:18:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.345+0000 7f6187fff700 10 client.15696 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.346+0000 7f6187fff700 1 -- 172.21.15.77:0/4080124241 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55dc60be0db0 con 0x55dc60c27140 2023-12-01T01:18:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.346+0000 7f6187fff700 10 client.15696 handle_command_reply: tid=0 2023-12-01T01:18:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.346+0000 7f61bb096b80 1 client.15696 shutdown 2023-12-01T01:18:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.346+0000 7f61bb096b80 1 -- 172.21.15.77:0/4080124241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc60c11680 msgr2=0x55dc60c1b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.346+0000 7f61bb096b80 1 --2- 172.21.15.77:0/4080124241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc60c11680 0x55dc60c1b4d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f619c00ee10 tx=0x7f619c00ee40 comp rx=0 tx=0).stop 2023-12-01T01:18:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.346+0000 7f61bb096b80 1 -- 172.21.15.77:0/4080124241 shutdown_connections 2023-12-01T01:18:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.346+0000 7f61bb096b80 1 --2- 172.21.15.77:0/4080124241 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dc60c12ed0 0x55dc60c22a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:44.354 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.346+0000 7f61bb096b80 1 --2- 172.21.15.77:0/4080124241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc60c11680 0x55dc60c1b4d0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.346+0000 7f61bb096b80 1 --2- 172.21.15.77:0/4080124241 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55dc60c27140 0x55dc60c29560 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.346+0000 7f61bb096b80 1 --2- 172.21.15.77:0/4080124241 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc60c11fe0 0x55dc60c1ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.346+0000 7f6187fff700 0 client.15696 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.346+0000 7f61bb096b80 1 -- 172.21.15.77:0/4080124241 >> 172.21.15.77:0/4080124241 conn(0x55dc60b71070 msgr2=0x55dc60c0c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.346+0000 7f61bb096b80 1 -- 172.21.15.77:0/4080124241 shutdown_connections 2023-12-01T01:18:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.346+0000 7f61bb096b80 1 -- 172.21.15.77:0/4080124241 wait complete. 2023-12-01T01:18:44.355 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.347+0000 7f61bb096b80 20 client.15696 trim_cache size 0 max 16384 2023-12-01T01:18:44.366 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.361+0000 7f61bb096b80 1 Processor -- start 2023-12-01T01:18:44.366 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.361+0000 7f61bb096b80 1 -- start start 2023-12-01T01:18:44.366 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.361+0000 7f61bb096b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc60c11680 0x55dc60c20a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:44.367 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.361+0000 7f61bb096b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dc60c11fe0 0x55dc60c1ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:44.367 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.361+0000 7f61bb096b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc60c12ed0 0x55dc60c0dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:44.367 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.361+0000 7f61bb096b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc60c0d660 con 0x55dc60c11680 2023-12-01T01:18:44.367 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.361+0000 7f61bb096b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc60c04a20 con 0x55dc60c12ed0 2023-12-01T01:18:44.367 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.361+0000 7f61bb096b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc60c07500 con 0x55dc60c11fe0 2023-12-01T01:18:44.367 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.361+0000 7f61acca2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc60c11680 0x55dc60c20a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:44.367 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.361+0000 7f61acca2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc60c11680 0x55dc60c20a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59846/0 (socket says 172.21.15.77:59846) 2023-12-01T01:18:44.367 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.361+0000 7f61acca2700 1 -- 172.21.15.77:0/821392240 learned_addr learned my addr 172.21.15.77:0/821392240 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:44.367 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.361+0000 7f61a7fff700 1 --2- 172.21.15.77:0/821392240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dc60c11fe0 0x55dc60c1ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:44.367 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.361+0000 7f61ad4a3700 1 --2- 172.21.15.77:0/821392240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc60c12ed0 0x55dc60c0dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:44.367 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.361+0000 7f61acca2700 1 -- 172.21.15.77:0/821392240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dc60c11fe0 msgr2=0x55dc60c1ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.361+0000 7f61acca2700 1 --2- 172.21.15.77:0/821392240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dc60c11fe0 0x55dc60c1ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.361+0000 7f61acca2700 1 -- 172.21.15.77:0/821392240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc60c12ed0 msgr2=0x55dc60c0dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.361+0000 7f61acca2700 1 --2- 172.21.15.77:0/821392240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc60c12ed0 0x55dc60c0dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.361+0000 7f61acca2700 1 -- 172.21.15.77:0/821392240 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc609b71f0 con 0x55dc60c11680 2023-12-01T01:18:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.361+0000 7f61ad4a3700 1 --2- 172.21.15.77:0/821392240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc60c12ed0 0x55dc60c0dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:18:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.361+0000 7f61a7fff700 1 --2- 172.21.15.77:0/821392240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dc60c11fe0 0x55dc60c1ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:18:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.361+0000 7f61acca2700 1 --2- 172.21.15.77:0/821392240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc60c11680 0x55dc60c20a00 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f619c00efd0 tx=0x7f619c009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a23dca2c8453a86 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.362+0000 7f61937fe700 1 -- 172.21.15.77:0/821392240 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619c01ceb0 con 0x55dc60c11680 2023-12-01T01:18:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.362+0000 7f61937fe700 1 -- 172.21.15.77:0/821392240 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f619c01d6f0 con 0x55dc60c11680 2023-12-01T01:18:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.362+0000 7f61937fe700 1 -- 172.21.15.77:0/821392240 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619c01c650 con 0x55dc60c11680 2023-12-01T01:18:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.362+0000 7f61bb096b80 1 -- 172.21.15.77:0/821392240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc60c11680 msgr2=0x55dc60c20a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.362+0000 7f61bb096b80 1 --2- 172.21.15.77:0/821392240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc60c11680 0x55dc60c20a00 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f619c00efd0 tx=0x7f619c009f80 comp rx=0 tx=0).stop 2023-12-01T01:18:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.362+0000 7f61bb096b80 1 -- 172.21.15.77:0/821392240 shutdown_connections 2023-12-01T01:18:44.368 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.362+0000 7f61bb096b80 1 --2- 172.21.15.77:0/821392240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc60c12ed0 0x55dc60c0dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.362+0000 7f61bb096b80 1 --2- 172.21.15.77:0/821392240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc60c11680 0x55dc60c20a00 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.362+0000 7f61bb096b80 1 --2- 172.21.15.77:0/821392240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dc60c11fe0 0x55dc60c1ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.362+0000 7f61bb096b80 1 -- 172.21.15.77:0/821392240 >> 172.21.15.77:0/821392240 conn(0x55dc60b71070 msgr2=0x55dc60c1a170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.362+0000 7f61bb096b80 1 -- 172.21.15.77:0/821392240 shutdown_connections 2023-12-01T01:18:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.362+0000 7f61bb096b80 1 -- 172.21.15.77:0/821392240 wait complete. 2023-12-01T01:18:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.363+0000 7f61bb096b80 1 -- start start 2023-12-01T01:18:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.363+0000 7f61bb096b80 1 Processor -- start 2023-12-01T01:18:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.363+0000 7f61bb096b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc60c11680 0x55dc60c72dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.363+0000 7f61bb096b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc60c11fe0 0x55dc60c75320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.363+0000 7f61bb096b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dc60c12ed0 0x55dc60c77940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.363+0000 7f61bb096b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc60c78080 con 0x55dc60c11fe0 2023-12-01T01:18:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.363+0000 7f61bb096b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc60c781f0 con 0x55dc60c11680 2023-12-01T01:18:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.363+0000 7f61bb096b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc60c783b0 con 0x55dc60c12ed0 2023-12-01T01:18:44.369 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.363+0000 7f61bb096b80 10 client.0 _ll_register_callbacks cb 0x7f619c00e3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:18:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.363+0000 7f61a7fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc60c11fe0 0x55dc60c75320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.363+0000 7f61a7fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc60c11fe0 0x55dc60c75320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59854/0 (socket says 172.21.15.77:59854) 2023-12-01T01:18:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.363+0000 7f61a7fff700 1 -- 172.21.15.77:0/3432254076 learned_addr learned my addr 172.21.15.77:0/3432254076 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.363+0000 7f61ad4a3700 1 --2- 172.21.15.77:0/3432254076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dc60c12ed0 0x55dc60c77940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.363+0000 7f61acca2700 1 --2- 172.21.15.77:0/3432254076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc60c11680 0x55dc60c72dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.363+0000 7f61a7fff700 1 -- 172.21.15.77:0/3432254076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dc60c12ed0 msgr2=0x55dc60c77940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.363+0000 7f61a7fff700 1 --2- 172.21.15.77:0/3432254076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dc60c12ed0 0x55dc60c77940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.363+0000 7f61a7fff700 1 -- 172.21.15.77:0/3432254076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc60c11680 msgr2=0x55dc60c72dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.363+0000 7f61a7fff700 1 --2- 172.21.15.77:0/3432254076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc60c11680 0x55dc60c72dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.363+0000 7f61a7fff700 1 -- 172.21.15.77:0/3432254076 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dc60979c00 con 0x55dc60c11fe0 2023-12-01T01:18:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.364+0000 7f61a7fff700 1 --2- 172.21.15.77:0/3432254076 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc60c11fe0 0x55dc60c75320 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f619400c5b0 tx=0x7f6194084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d113e3459d0fa98a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:44.370 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.364+0000 7f6187fff700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:18:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.364+0000 7f6187fff700 1 -- 172.21.15.77:0/3432254076 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619409d2d0 con 0x55dc60c11fe0 2023-12-01T01:18:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.364+0000 7f6187fff700 1 -- 172.21.15.77:0/3432254076 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f61940a1510 con 0x55dc60c11fe0 2023-12-01T01:18:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.364+0000 7f61bb096b80 1 -- 172.21.15.77:0/3432254076 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f619c021070 con 0x55dc60c11fe0 2023-12-01T01:18:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.364+0000 7f6187fff700 1 -- 172.21.15.77:0/3432254076 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61940840b0 con 0x55dc60c11fe0 2023-12-01T01:18:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.364+0000 7f61bb096b80 10 client.15702 fetch_fsmap learned FSMap version 27 2023-12-01T01:18:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.364+0000 7f61bb096b80 1 -- 172.21.15.77:0/3432254076 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55dc6098f250 con 0x55dc60c11fe0 2023-12-01T01:18:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.364+0000 7f6187fff700 1 -- 172.21.15.77:0/3432254076 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f619c021070 con 0x55dc60c11fe0 2023-12-01T01:18:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.364+0000 7f6187fff700 1 -- 172.21.15.77:0/3432254076 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 27) v1 ==== 7454+0+0 (secure 0 0 0) 0x7f6194099700 con 0x55dc60c11fe0 2023-12-01T01:18:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.364+0000 7f61bb096b80 10 client.15702 fetch_fsmap finished waiting for FSMap version 27 2023-12-01T01:18:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.364+0000 7f61bb096b80 10 client.15702 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-01T01:18:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.364+0000 7f61bb096b80 20 client.15702 populate_metadata read hostname 'smithi077' 2023-12-01T01:18:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.364+0000 7f61bb096b80 1 --2- 172.21.15.77:0/3432254076 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55dc60c79c10 0x55dc60c7c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.364+0000 7f61bb096b80 4 client.15702 mds_command: new command op to 24500 tid=0[{"prefix": "flush journal"}] 2023-12-01T01:18:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.364+0000 7f61bb096b80 1 -- 172.21.15.77:0/3432254076 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55dc60b5bce0 con 0x55dc60c79c10 2023-12-01T01:18:44.371 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.365+0000 7f61acca2700 1 --2- 172.21.15.77:0/3432254076 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55dc60c79c10 0x55dc60c7c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.365+0000 7f61acca2700 1 --2- 172.21.15.77:0/3432254076 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55dc60c79c10 0x55dc60c7c030 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50d1fc9950f6ea5f server_cookie=8843f28328bf7a2f in_seq=0 out_seq=0 2023-12-01T01:18:44.372 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.365+0000 7f6187fff700 10 client.15702 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.539+0000 7f6187fff700 1 -- 172.21.15.77:0/3432254076 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55dc60b5bce0 con 0x55dc60c79c10 2023-12-01T01:18:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.539+0000 7f6187fff700 10 client.15702 handle_command_reply: tid=0 2023-12-01T01:18:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.539+0000 7f61bb096b80 1 client.15702 shutdown 2023-12-01T01:18:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.539+0000 7f61bb096b80 1 -- 172.21.15.77:0/3432254076 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc60c11fe0 msgr2=0x55dc60c75320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.539+0000 7f61bb096b80 1 --2- 172.21.15.77:0/3432254076 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc60c11fe0 0x55dc60c75320 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f619400c5b0 tx=0x7f6194084040 comp rx=0 tx=0).stop 2023-12-01T01:18:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.539+0000 7f61bb096b80 1 -- 172.21.15.77:0/3432254076 shutdown_connections 2023-12-01T01:18:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.539+0000 7f61bb096b80 1 --2- 172.21.15.77:0/3432254076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dc60c12ed0 0x55dc60c77940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.539+0000 7f61bb096b80 1 --2- 172.21.15.77:0/3432254076 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55dc60c79c10 0x55dc60c7c030 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.539+0000 7f61bb096b80 1 --2- 172.21.15.77:0/3432254076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc60c11680 0x55dc60c72dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.539+0000 7f61bb096b80 1 --2- 172.21.15.77:0/3432254076 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dc60c11fe0 0x55dc60c75320 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.539+0000 7f6187fff700 0 client.15702 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.539+0000 7f61bb096b80 1 -- 172.21.15.77:0/3432254076 >> 172.21.15.77:0/3432254076 conn(0x55dc60b71070 msgr2=0x55dc60c0dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.539+0000 7f61bb096b80 1 -- 172.21.15.77:0/3432254076 shutdown_connections 2023-12-01T01:18:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.539+0000 7f61bb096b80 1 -- 172.21.15.77:0/3432254076 wait complete. 2023-12-01T01:18:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.539+0000 7f61bb096b80 20 client.15702 trim_cache size 0 max 16384 2023-12-01T01:18:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout:{ 2023-12-01T01:18:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout: "message": "", 2023-12-01T01:18:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout: "return_code": 0 2023-12-01T01:18:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout:} 2023-12-01T01:18:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.540+0000 7f61aef06700 1 -- 172.21.15.77:0/2986268312 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f61980a05a0 msgr2=0x7f61980a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.540+0000 7f61aef06700 1 --2- 172.21.15.77:0/2986268312 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f61980a05a0 0x7f61980a2a60 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f619c009c60 tx=0x7f619c009260 comp rx=0 tx=0).stop 2023-12-01T01:18:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.540+0000 7f61aef06700 1 -- 172.21.15.77:0/2986268312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61a81041a0 msgr2=0x7f61a8197190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.540+0000 7f61aef06700 1 --2- 172.21.15.77:0/2986268312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61a81041a0 0x7f61a8197190 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f619400e9a0 tx=0x7f619400ecb0 comp rx=0 tx=0).stop 2023-12-01T01:18:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.540+0000 7f61aef06700 1 -- 172.21.15.77:0/2986268312 shutdown_connections 2023-12-01T01:18:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.540+0000 7f61aef06700 1 --2- 172.21.15.77:0/2986268312 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f61980a05a0 0x7f61980a2a60 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.540+0000 7f61aef06700 1 --2- 172.21.15.77:0/2986268312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61a8106ad0 0x7f61a8197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.540+0000 7f61aef06700 1 --2- 172.21.15.77:0/2986268312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61a8101870 0x7f61a8196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.540+0000 7f61aef06700 1 --2- 172.21.15.77:0/2986268312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61a81041a0 0x7f61a8197190 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.540+0000 7f61aef06700 1 -- 172.21.15.77:0/2986268312 >> 172.21.15.77:0/2986268312 conn(0x7f61a80fb1e0 msgr2=0x7f61a8102500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.540+0000 7f61aef06700 1 -- 172.21.15.77:0/2986268312 shutdown_connections 2023-12-01T01:18:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:44.540+0000 7f61aef06700 1 -- 172.21.15.77:0/2986268312 wait complete. 2023-12-01T01:18:45.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:45 smithi077 ceph-mon[98867]: pgmap v324: 97 pgs: 97 active+clean; 14 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 1.1 MiB/s wr, 17 op/s 2023-12-01T01:18:46.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:45 smithi203 ceph-mon[103581]: pgmap v324: 97 pgs: 97 active+clean; 14 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 1.1 MiB/s wr, 17 op/s 2023-12-01T01:18:46.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:45 smithi137 ceph-mon[103312]: pgmap v324: 97 pgs: 97 active+clean; 14 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 1.1 MiB/s wr, 17 op/s 2023-12-01T01:18:47.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:47 smithi077 ceph-mon[98867]: pgmap v325: 97 pgs: 97 active+clean; 14 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 1.1 MiB/s wr, 49 op/s 2023-12-01T01:18:48.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:47 smithi203 ceph-mon[103581]: pgmap v325: 97 pgs: 97 active+clean; 14 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 1.1 MiB/s wr, 49 op/s 2023-12-01T01:18:48.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:47 smithi137 ceph-mon[103312]: pgmap v325: 97 pgs: 97 active+clean; 14 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 1.1 MiB/s wr, 49 op/s 2023-12-01T01:18:49.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:49 smithi077 ceph-mon[98867]: pgmap v326: 97 pgs: 97 active+clean; 15 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 1.2 MiB/s wr, 98 op/s 2023-12-01T01:18:50.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:49 smithi203 ceph-mon[103581]: pgmap v326: 97 pgs: 97 active+clean; 15 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 1.2 MiB/s wr, 98 op/s 2023-12-01T01:18:50.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:49 smithi137 ceph-mon[103312]: pgmap v326: 97 pgs: 97 active+clean; 15 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 1.2 MiB/s wr, 98 op/s 2023-12-01T01:18:52.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:51 smithi203 ceph-mon[103581]: pgmap v327: 97 pgs: 97 active+clean; 15 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 1.2 MiB/s wr, 123 op/s 2023-12-01T01:18:52.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:51 smithi203 ceph-mon[103581]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:18:52.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:51 smithi137 ceph-mon[103312]: pgmap v327: 97 pgs: 97 active+clean; 15 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 1.2 MiB/s wr, 123 op/s 2023-12-01T01:18:52.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:51 smithi137 ceph-mon[103312]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:18:52.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:51 smithi077 ceph-mon[98867]: pgmap v327: 97 pgs: 97 active+clean; 15 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 1.2 MiB/s wr, 123 op/s 2023-12-01T01:18:52.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:51 smithi077 ceph-mon[98867]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:18:53.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:52 smithi203 ceph-mon[103581]: pgmap v328: 97 pgs: 97 active+clean; 15 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2023-12-01T01:18:53.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:52 smithi137 ceph-mon[103312]: pgmap v328: 97 pgs: 97 active+clean; 15 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2023-12-01T01:18:53.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:52 smithi077 ceph-mon[98867]: pgmap v328: 97 pgs: 97 active+clean; 15 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2023-12-01T01:18:55.235 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-01T01:18:55.384 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.381+0000 7f116791b700 1 -- 172.21.15.77:0/1040171924 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1160100e70 msgr2=0x7f11601012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:55.385 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.381+0000 7f116791b700 1 --2- 172.21.15.77:0/1040171924 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1160100e70 0x7f11601012d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f11500099c0 tx=0x7f1150009cd0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.385 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.381+0000 7f116791b700 1 -- 172.21.15.77:0/1040171924 shutdown_connections 2023-12-01T01:18:55.385 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.381+0000 7f116791b700 1 --2- 172.21.15.77:0/1040171924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1160101810 0x7f11601080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.385 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.381+0000 7f116791b700 1 --2- 172.21.15.77:0/1040171924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11600ffc80 0x7f11601000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.385 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.381+0000 7f116791b700 1 --2- 172.21.15.77:0/1040171924 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1160100e70 0x7f11601012d0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.385 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.381+0000 7f116791b700 1 -- 172.21.15.77:0/1040171924 >> 172.21.15.77:0/1040171924 conn(0x7f11600fb1f0 msgr2=0x7f11600fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:55.385 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.381+0000 7f116791b700 1 -- 172.21.15.77:0/1040171924 shutdown_connections 2023-12-01T01:18:55.385 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.381+0000 7f116791b700 1 -- 172.21.15.77:0/1040171924 wait complete. 2023-12-01T01:18:55.385 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.382+0000 7f116791b700 1 Processor -- start 2023-12-01T01:18:55.385 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.382+0000 7f116791b700 1 -- start start 2023-12-01T01:18:55.386 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.382+0000 7f116791b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11600ffc80 0x7f1160068ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:55.386 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.382+0000 7f116791b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1160100e70 0x7f116019b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:55.386 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.382+0000 7f116791b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1160101810 0x7f116019ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:55.386 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.382+0000 7f116791b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116019e590 con 0x7f1160101810 2023-12-01T01:18:55.386 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.382+0000 7f116791b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116019e700 con 0x7f1160100e70 2023-12-01T01:18:55.386 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.382+0000 7f116791b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116019e8c0 con 0x7f11600ffc80 2023-12-01T01:18:55.386 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.382+0000 7f1165eb8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1160101810 0x7f116019ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:55.386 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.382+0000 7f1164eb6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1160100e70 0x7f116019b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:55.386 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.382+0000 7f1164eb6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1160100e70 0x7f116019b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:38790/0 (socket says 172.21.15.77:38790) 2023-12-01T01:18:55.386 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.382+0000 7f1164eb6700 1 -- 172.21.15.77:0/2659906925 learned_addr learned my addr 172.21.15.77:0/2659906925 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:55.386 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.382+0000 7f11656b7700 1 --2- 172.21.15.77:0/2659906925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11600ffc80 0x7f1160068ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:55.386 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.382+0000 7f1165eb8700 1 -- 172.21.15.77:0/2659906925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11600ffc80 msgr2=0x7f1160068ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:55.386 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.382+0000 7f1165eb8700 1 --2- 172.21.15.77:0/2659906925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11600ffc80 0x7f1160068ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.386 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.382+0000 7f1165eb8700 1 -- 172.21.15.77:0/2659906925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1160100e70 msgr2=0x7f116019b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:55.387 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.382+0000 7f1165eb8700 1 --2- 172.21.15.77:0/2659906925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1160100e70 0x7f116019b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.387 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.382+0000 7f1165eb8700 1 -- 172.21.15.77:0/2659906925 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11500095f0 con 0x7f1160101810 2023-12-01T01:18:55.387 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.382+0000 7f1165eb8700 1 --2- 172.21.15.77:0/2659906925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1160101810 0x7f116019ddf0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1158007f10 tx=0x7f115800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:55.387 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.382+0000 7f11567fc700 1 -- 172.21.15.77:0/2659906925 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115800ddd0 con 0x7f1160101810 2023-12-01T01:18:55.387 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.383+0000 7f11567fc700 1 -- 172.21.15.77:0/2659906925 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f115800f040 con 0x7f1160101810 2023-12-01T01:18:55.387 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.383+0000 7f116791b700 1 -- 172.21.15.77:0/2659906925 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f116019eba0 con 0x7f1160101810 2023-12-01T01:18:55.387 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.383+0000 7f11567fc700 1 -- 172.21.15.77:0/2659906925 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1158005a90 con 0x7f1160101810 2023-12-01T01:18:55.387 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.383+0000 7f116791b700 1 -- 172.21.15.77:0/2659906925 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f116019f0b0 con 0x7f1160101810 2023-12-01T01:18:55.394 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.385+0000 7f11567fc700 1 -- 172.21.15.77:0/2659906925 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f1158004ad0 con 0x7f1160101810 2023-12-01T01:18:55.394 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.385+0000 7f11567fc700 1 --2- 172.21.15.77:0/2659906925 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f114c0a05a0 0x7f114c0a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:55.394 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.385+0000 7f11656b7700 1 --2- 172.21.15.77:0/2659906925 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f114c0a05a0 0x7f114c0a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:55.394 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.385+0000 7f11567fc700 1 -- 172.21.15.77:0/2659906925 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f11580cb130 con 0x7f1160101810 2023-12-01T01:18:55.394 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.386+0000 7f11656b7700 1 --2- 172.21.15.77:0/2659906925 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f114c0a05a0 0x7f114c0a2a60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1160100cd0 tx=0x7f115c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:55.394 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.386+0000 7f1173aabb80 1 Processor -- start 2023-12-01T01:18:55.394 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.386+0000 7f1173aabb80 1 -- start start 2023-12-01T01:18:55.394 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.387+0000 7f1173aabb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566db5960f0 0x5566db596530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:55.394 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.387+0000 7f1173aabb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566db596a70 0x5566db594e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:55.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.387+0000 7f1173aabb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566db5953f0 0x5566db595870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:55.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.387+0000 7f1173aabb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566db4eca10 con 0x5566db596a70 2023-12-01T01:18:55.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.387+0000 7f1173aabb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566db4ea3d0 con 0x5566db5953f0 2023-12-01T01:18:55.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.387+0000 7f1173aabb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566db4e5cc0 con 0x5566db5960f0 2023-12-01T01:18:55.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.387+0000 7f1164eb6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566db596a70 0x5566db594e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:55.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.387+0000 7f1164eb6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566db596a70 0x5566db594e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59876/0 (socket says 172.21.15.77:59876) 2023-12-01T01:18:55.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.387+0000 7f1164eb6700 1 -- 172.21.15.77:0/2047696245 learned_addr learned my addr 172.21.15.77:0/2047696245 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:55.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.387+0000 7f1165eb8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566db5953f0 0x5566db595870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:55.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.387+0000 7f11656b7700 1 --2- 172.21.15.77:0/2047696245 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566db5960f0 0x5566db596530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:55.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.387+0000 7f1164eb6700 1 -- 172.21.15.77:0/2047696245 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566db5960f0 msgr2=0x5566db596530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:55.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.387+0000 7f1164eb6700 1 --2- 172.21.15.77:0/2047696245 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566db5960f0 0x5566db596530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.387+0000 7f1164eb6700 1 -- 172.21.15.77:0/2047696245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566db5953f0 msgr2=0x5566db595870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:55.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.387+0000 7f1164eb6700 1 --2- 172.21.15.77:0/2047696245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566db5953f0 0x5566db595870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.387+0000 7f1164eb6700 1 -- 172.21.15.77:0/2047696245 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566db313de0 con 0x5566db596a70 2023-12-01T01:18:55.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.387+0000 7f1165eb8700 1 --2- 172.21.15.77:0/2047696245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566db5953f0 0x5566db595870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:18:55.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.387+0000 7f1164eb6700 1 --2- 172.21.15.77:0/2047696245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566db596a70 0x5566db594e60 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f1150005c30 tx=0x7f1150005c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11adba8602aad085 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:55.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.387+0000 7f114b7fe700 1 -- 172.21.15.77:0/2047696245 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115000bb10 con 0x5566db596a70 2023-12-01T01:18:55.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.387+0000 7f114b7fe700 1 -- 172.21.15.77:0/2047696245 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f115001e040 con 0x5566db596a70 2023-12-01T01:18:55.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.387+0000 7f114b7fe700 1 -- 172.21.15.77:0/2047696245 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1150003b30 con 0x5566db596a70 2023-12-01T01:18:55.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.388+0000 7f1173aabb80 1 -- 172.21.15.77:0/2047696245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566db596a70 msgr2=0x5566db594e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:55.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.388+0000 7f1173aabb80 1 --2- 172.21.15.77:0/2047696245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566db596a70 0x5566db594e60 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f1150005c30 tx=0x7f1150005c60 comp rx=0 tx=0).stop 2023-12-01T01:18:55.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.388+0000 7f1173aabb80 1 -- 172.21.15.77:0/2047696245 shutdown_connections 2023-12-01T01:18:55.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.388+0000 7f1173aabb80 1 --2- 172.21.15.77:0/2047696245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566db5953f0 0x5566db595870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.388+0000 7f1173aabb80 1 --2- 172.21.15.77:0/2047696245 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566db5960f0 0x5566db596530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.388+0000 7f1173aabb80 1 --2- 172.21.15.77:0/2047696245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566db596a70 0x5566db594e60 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.388+0000 7f1173aabb80 1 -- 172.21.15.77:0/2047696245 >> 172.21.15.77:0/2047696245 conn(0x5566db4f40d0 msgr2=0x5566db58f940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:55.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.388+0000 7f1173aabb80 1 -- 172.21.15.77:0/2047696245 shutdown_connections 2023-12-01T01:18:55.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.388+0000 7f1173aabb80 1 -- 172.21.15.77:0/2047696245 wait complete. 2023-12-01T01:18:55.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.388+0000 7f1173aabb80 1 -- start start 2023-12-01T01:18:55.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.388+0000 7f1173aabb80 1 Processor -- start 2023-12-01T01:18:55.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.389+0000 7f1173aabb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566db5953f0 0x5566db59e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:55.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.389+0000 7f1173aabb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566db5960f0 0x5566db59eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:55.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.389+0000 7f1173aabb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566db596a70 0x5566db5a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:55.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.389+0000 7f1173aabb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566db482510 con 0x5566db596a70 2023-12-01T01:18:55.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.389+0000 7f1173aabb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566db4821b0 con 0x5566db5960f0 2023-12-01T01:18:55.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.389+0000 7f1173aabb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566db369920 con 0x5566db5953f0 2023-12-01T01:18:55.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.389+0000 7f1173aabb80 10 client.0 _ll_register_callbacks cb 0x5566db3cfa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:18:55.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.389+0000 7f1165eb8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566db596a70 0x5566db5a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:55.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.389+0000 7f1164eb6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566db5960f0 0x5566db59eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:55.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.389+0000 7f1165eb8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566db596a70 0x5566db5a5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59884/0 (socket says 172.21.15.77:59884) 2023-12-01T01:18:55.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.389+0000 7f1165eb8700 1 -- 172.21.15.77:0/821004879 learned_addr learned my addr 172.21.15.77:0/821004879 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:55.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.389+0000 7f1164eb6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566db5960f0 0x5566db59eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:38804/0 (socket says 172.21.15.77:38804) 2023-12-01T01:18:55.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.389+0000 7f11656b7700 1 --2- 172.21.15.77:0/821004879 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566db5953f0 0x5566db59e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:55.398 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.389+0000 7f1165eb8700 1 -- 172.21.15.77:0/821004879 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566db5953f0 msgr2=0x5566db59e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:55.398 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.389+0000 7f1165eb8700 1 --2- 172.21.15.77:0/821004879 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566db5953f0 0x5566db59e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.398 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.389+0000 7f1165eb8700 1 -- 172.21.15.77:0/821004879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566db5960f0 msgr2=0x5566db59eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:55.398 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.389+0000 7f1165eb8700 1 --2- 172.21.15.77:0/821004879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566db5960f0 0x5566db59eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.398 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.389+0000 7f1165eb8700 1 -- 172.21.15.77:0/821004879 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f116019f0b0 con 0x5566db596a70 2023-12-01T01:18:55.398 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.390+0000 7f1165eb8700 1 --2- 172.21.15.77:0/821004879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566db596a70 0x5566db5a5f20 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1158098730 tx=0x7f11580b80b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e82cca8208a77b1 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:55.398 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.390+0000 7f1148ff9700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:18:55.398 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.390+0000 7f1148ff9700 1 -- 172.21.15.77:0/821004879 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11580b0070 con 0x5566db596a70 2023-12-01T01:18:55.398 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.390+0000 7f1148ff9700 1 -- 172.21.15.77:0/821004879 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f1158099130 con 0x5566db596a70 2023-12-01T01:18:55.398 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.390+0000 7f1173aabb80 1 -- 172.21.15.77:0/821004879 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5566db510d10 con 0x5566db596a70 2023-12-01T01:18:55.398 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.390+0000 7f1148ff9700 1 -- 172.21.15.77:0/821004879 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1158098a30 con 0x5566db596a70 2023-12-01T01:18:55.398 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.390+0000 7f1173aabb80 10 client.15714 fetch_fsmap learned FSMap version 29 2023-12-01T01:18:55.398 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.390+0000 7f1173aabb80 1 -- 172.21.15.77:0/821004879 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f1160108a10 con 0x5566db596a70 2023-12-01T01:18:55.399 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.390+0000 7f1148ff9700 1 -- 172.21.15.77:0/821004879 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f115807b020 con 0x5566db596a70 2023-12-01T01:18:55.399 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.390+0000 7f1148ff9700 1 -- 172.21.15.77:0/821004879 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f11580a0610 con 0x5566db596a70 2023-12-01T01:18:55.399 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.390+0000 7f1173aabb80 10 client.15714 fetch_fsmap finished waiting for FSMap version 29 2023-12-01T01:18:55.399 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.390+0000 7f1173aabb80 10 client.15714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-01T01:18:55.399 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.390+0000 7f1173aabb80 20 client.15714 populate_metadata read hostname 'smithi077' 2023-12-01T01:18:55.399 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.390+0000 7f1173aabb80 1 --2- 172.21.15.77:0/821004879 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5566db5aa850 0x5566db5acc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:55.399 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.390+0000 7f1173aabb80 4 client.15714 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:18:55.399 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.390+0000 7f1173aabb80 1 -- 172.21.15.77:0/821004879 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5566db563e10 con 0x5566db5aa850 2023-12-01T01:18:55.399 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.390+0000 7f11656b7700 1 --2- 172.21.15.77:0/821004879 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5566db5aa850 0x5566db5acc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:55.399 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.391+0000 7f1148ff9700 10 client.15714 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:55.399 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.391+0000 7f11656b7700 1 --2- 172.21.15.77:0/821004879 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5566db5aa850 0x5566db5acc70 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d57bbdf207fb296c server_cookie=9da6f10a7f9c79ca in_seq=0 out_seq=0 2023-12-01T01:18:55.399 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.392+0000 7f1148ff9700 1 -- 172.21.15.77:0/821004879 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5566db563e10 con 0x5566db5aa850 2023-12-01T01:18:55.399 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.392+0000 7f1148ff9700 10 client.15714 handle_command_reply: tid=0 2023-12-01T01:18:55.399 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.392+0000 7f1173aabb80 1 client.15714 shutdown 2023-12-01T01:18:55.400 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.392+0000 7f1173aabb80 1 -- 172.21.15.77:0/821004879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566db596a70 msgr2=0x5566db5a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:55.400 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.392+0000 7f1173aabb80 1 --2- 172.21.15.77:0/821004879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566db596a70 0x5566db5a5f20 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1158098730 tx=0x7f11580b80b0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.400 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.392+0000 7f1173aabb80 1 -- 172.21.15.77:0/821004879 shutdown_connections 2023-12-01T01:18:55.400 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.392+0000 7f1173aabb80 1 --2- 172.21.15.77:0/821004879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566db596a70 0x5566db5a5f20 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.400 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.392+0000 7f1173aabb80 1 --2- 172.21.15.77:0/821004879 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566db5953f0 0x5566db59e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.400 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.392+0000 7f1173aabb80 1 --2- 172.21.15.77:0/821004879 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5566db5aa850 0x5566db5acc70 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.400 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.392+0000 7f1173aabb80 1 --2- 172.21.15.77:0/821004879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566db5960f0 0x5566db59eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.400 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.392+0000 7f1148ff9700 0 client.15714 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:55.400 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.392+0000 7f1173aabb80 1 -- 172.21.15.77:0/821004879 >> 172.21.15.77:0/821004879 conn(0x5566db4f40d0 msgr2=0x5566db58f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:55.400 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.392+0000 7f1173aabb80 1 -- 172.21.15.77:0/821004879 shutdown_connections 2023-12-01T01:18:55.400 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.392+0000 7f1173aabb80 1 -- 172.21.15.77:0/821004879 wait complete. 2023-12-01T01:18:55.400 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.392+0000 7f1173aabb80 20 client.15714 trim_cache size 0 max 16384 2023-12-01T01:18:55.415 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.409+0000 7f1173aabb80 1 Processor -- start 2023-12-01T01:18:55.415 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.410+0000 7f1173aabb80 1 -- start start 2023-12-01T01:18:55.415 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.410+0000 7f1173aabb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566db5953f0 0x5566db594e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:55.415 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.410+0000 7f1173aabb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566db5960f0 0x5566db5a3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:55.415 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.410+0000 7f1173aabb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566db596a70 0x5566db599310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:55.415 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.410+0000 7f1173aabb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566db5914a0 con 0x5566db5960f0 2023-12-01T01:18:55.415 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.410+0000 7f1173aabb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566db59fb50 con 0x5566db596a70 2023-12-01T01:18:55.415 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.410+0000 7f1173aabb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566db59d070 con 0x5566db5953f0 2023-12-01T01:18:55.415 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.410+0000 7f1164eb6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566db5960f0 0x5566db5a3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:55.416 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.410+0000 7f1164eb6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566db5960f0 0x5566db5a3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59890/0 (socket says 172.21.15.77:59890) 2023-12-01T01:18:55.416 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.410+0000 7f1164eb6700 1 -- 172.21.15.77:0/3407750194 learned_addr learned my addr 172.21.15.77:0/3407750194 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:55.416 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.410+0000 7f1165eb8700 1 --2- 172.21.15.77:0/3407750194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566db596a70 0x5566db599310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:55.417 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.410+0000 7f11656b7700 1 --2- 172.21.15.77:0/3407750194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566db5953f0 0x5566db594e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:55.417 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.410+0000 7f1164eb6700 1 -- 172.21.15.77:0/3407750194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566db5953f0 msgr2=0x5566db594e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:55.417 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.410+0000 7f1164eb6700 1 --2- 172.21.15.77:0/3407750194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566db5953f0 0x5566db594e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.417 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.410+0000 7f1164eb6700 1 -- 172.21.15.77:0/3407750194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566db596a70 msgr2=0x5566db599310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:55.417 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.410+0000 7f1164eb6700 1 --2- 172.21.15.77:0/3407750194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566db596a70 0x5566db599310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.417 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.410+0000 7f1164eb6700 1 -- 172.21.15.77:0/3407750194 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566db2fcc60 con 0x5566db5960f0 2023-12-01T01:18:55.417 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.410+0000 7f1165eb8700 1 --2- 172.21.15.77:0/3407750194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566db596a70 0x5566db599310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:18:55.417 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.410+0000 7f1164eb6700 1 --2- 172.21.15.77:0/3407750194 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566db5960f0 0x5566db5a3ed0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x5566db596e50 tx=0x7f1150000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2bd3c2d8081722f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:55.417 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.410+0000 7f114bfff700 1 -- 172.21.15.77:0/3407750194 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115000bc50 con 0x5566db5960f0 2023-12-01T01:18:55.417 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.410+0000 7f114bfff700 1 -- 172.21.15.77:0/3407750194 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1150005570 con 0x5566db5960f0 2023-12-01T01:18:55.417 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.410+0000 7f114bfff700 1 -- 172.21.15.77:0/3407750194 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115002b880 con 0x5566db5960f0 2023-12-01T01:18:55.418 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.411+0000 7f1173aabb80 1 -- 172.21.15.77:0/3407750194 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566db5960f0 msgr2=0x5566db5a3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:55.418 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.411+0000 7f1173aabb80 1 --2- 172.21.15.77:0/3407750194 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566db5960f0 0x5566db5a3ed0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x5566db596e50 tx=0x7f1150000bc0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.418 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.411+0000 7f1173aabb80 1 -- 172.21.15.77:0/3407750194 shutdown_connections 2023-12-01T01:18:55.418 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.411+0000 7f1173aabb80 1 --2- 172.21.15.77:0/3407750194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566db596a70 0x5566db599310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.418 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.411+0000 7f1173aabb80 1 --2- 172.21.15.77:0/3407750194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566db5953f0 0x5566db594e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.418 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.411+0000 7f1173aabb80 1 --2- 172.21.15.77:0/3407750194 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566db5960f0 0x5566db5a3ed0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.418 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.411+0000 7f1173aabb80 1 -- 172.21.15.77:0/3407750194 >> 172.21.15.77:0/3407750194 conn(0x5566db4f40d0 msgr2=0x5566db58f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:55.418 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.411+0000 7f1173aabb80 1 -- 172.21.15.77:0/3407750194 shutdown_connections 2023-12-01T01:18:55.418 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.411+0000 7f1173aabb80 1 -- 172.21.15.77:0/3407750194 wait complete. 2023-12-01T01:18:55.418 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.411+0000 7f1173aabb80 1 -- start start 2023-12-01T01:18:55.418 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.411+0000 7f1173aabb80 1 Processor -- start 2023-12-01T01:18:55.418 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.412+0000 7f1173aabb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566db5953f0 0x5566db5a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:55.418 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.412+0000 7f1173aabb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566db5960f0 0x5566db5a2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:55.418 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.412+0000 7f1173aabb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566db596a70 0x5566db5a2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:55.418 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.412+0000 7f1173aabb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566db5a2c80 con 0x5566db5953f0 2023-12-01T01:18:55.419 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.412+0000 7f1173aabb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566db5a2df0 con 0x5566db5960f0 2023-12-01T01:18:55.419 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.412+0000 7f1173aabb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566db5a2fb0 con 0x5566db596a70 2023-12-01T01:18:55.419 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.412+0000 7f1173aabb80 10 client.0 _ll_register_callbacks cb 0x7f1158098570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:18:55.419 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.412+0000 7f11656b7700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566db5953f0 0x5566db5a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:55.419 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.412+0000 7f11656b7700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566db5953f0 0x5566db5a3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59894/0 (socket says 172.21.15.77:59894) 2023-12-01T01:18:55.419 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.412+0000 7f11656b7700 1 -- 172.21.15.77:0/3557639877 learned_addr learned my addr 172.21.15.77:0/3557639877 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:55.419 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.412+0000 7f1164eb6700 1 --2- 172.21.15.77:0/3557639877 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566db5960f0 0x5566db5a2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:55.419 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.412+0000 7f1165eb8700 1 --2- 172.21.15.77:0/3557639877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566db596a70 0x5566db5a2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:55.419 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.412+0000 7f11656b7700 1 -- 172.21.15.77:0/3557639877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566db596a70 msgr2=0x5566db5a2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:55.419 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.412+0000 7f11656b7700 1 --2- 172.21.15.77:0/3557639877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566db596a70 0x5566db5a2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.419 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.412+0000 7f11656b7700 1 -- 172.21.15.77:0/3557639877 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566db5960f0 msgr2=0x5566db5a2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:55.419 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.412+0000 7f11656b7700 1 --2- 172.21.15.77:0/3557639877 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566db5960f0 0x5566db5a2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.419 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.412+0000 7f11656b7700 1 -- 172.21.15.77:0/3557639877 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5566db313de0 con 0x5566db5953f0 2023-12-01T01:18:55.419 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.412+0000 7f11656b7700 1 --2- 172.21.15.77:0/3557639877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566db5953f0 0x5566db5a3bf0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x5566db595e90 tx=0x7f115c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef93f36df65bbe9 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:55.420 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.412+0000 7f1148ff9700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:18:55.420 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.412+0000 7f1148ff9700 1 -- 172.21.15.77:0/3557639877 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115c025040 con 0x5566db5953f0 2023-12-01T01:18:55.420 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.412+0000 7f1148ff9700 1 -- 172.21.15.77:0/3557639877 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f115c018070 con 0x5566db5953f0 2023-12-01T01:18:55.420 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.412+0000 7f1173aabb80 1 -- 172.21.15.77:0/3557639877 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f115001e040 con 0x5566db5953f0 2023-12-01T01:18:55.420 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.413+0000 7f1148ff9700 1 -- 172.21.15.77:0/3557639877 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f115c012ab0 con 0x5566db5953f0 2023-12-01T01:18:55.420 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.413+0000 7f1173aabb80 10 client.15720 fetch_fsmap learned FSMap version 29 2023-12-01T01:18:55.420 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.413+0000 7f1173aabb80 1 -- 172.21.15.77:0/3557639877 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5566db33a250 con 0x5566db5953f0 2023-12-01T01:18:55.420 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.413+0000 7f1148ff9700 1 -- 172.21.15.77:0/3557639877 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f115001e040 con 0x5566db5953f0 2023-12-01T01:18:55.420 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.413+0000 7f1148ff9700 1 -- 172.21.15.77:0/3557639877 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f115c033470 con 0x5566db5953f0 2023-12-01T01:18:55.420 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.413+0000 7f1173aabb80 10 client.15720 fetch_fsmap finished waiting for FSMap version 29 2023-12-01T01:18:55.420 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.413+0000 7f1173aabb80 10 client.15720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-01T01:18:55.420 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.413+0000 7f1173aabb80 20 client.15720 populate_metadata read hostname 'smithi077' 2023-12-01T01:18:55.420 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.413+0000 7f1173aabb80 1 --2- 172.21.15.77:0/3557639877 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5566db5fdc40 0x5566db600060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:55.420 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.413+0000 7f1173aabb80 4 client.15720 mds_command: new command op to 24500 tid=0[{"prefix": "scrub status"}] 2023-12-01T01:18:55.420 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.413+0000 7f1173aabb80 1 -- 172.21.15.77:0/3557639877 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5566db4ded40 con 0x5566db5fdc40 2023-12-01T01:18:55.421 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.413+0000 7f1164eb6700 1 --2- 172.21.15.77:0/3557639877 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5566db5fdc40 0x5566db600060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:55.421 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.413+0000 7f1164eb6700 1 --2- 172.21.15.77:0/3557639877 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5566db5fdc40 0x5566db600060 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=436525353d482c0c server_cookie=54c3571676090d8c in_seq=0 out_seq=0 2023-12-01T01:18:55.421 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.413+0000 7f1148ff9700 10 client.15720 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:55.422 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.414+0000 7f1148ff9700 1 -- 172.21.15.77:0/3557639877 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5566db4ded40 con 0x5566db5fdc40 2023-12-01T01:18:55.422 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.414+0000 7f1148ff9700 10 client.15720 handle_command_reply: tid=0 2023-12-01T01:18:55.422 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.414+0000 7f1173aabb80 1 client.15720 shutdown 2023-12-01T01:18:55.422 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.414+0000 7f1173aabb80 1 -- 172.21.15.77:0/3557639877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566db5953f0 msgr2=0x5566db5a3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:55.422 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.414+0000 7f1173aabb80 1 --2- 172.21.15.77:0/3557639877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566db5953f0 0x5566db5a3bf0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x5566db595e90 tx=0x7f115c0047c0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.422 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.414+0000 7f1173aabb80 1 -- 172.21.15.77:0/3557639877 shutdown_connections 2023-12-01T01:18:55.422 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.414+0000 7f1173aabb80 1 --2- 172.21.15.77:0/3557639877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5566db596a70 0x5566db5a2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.422 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.414+0000 7f1173aabb80 1 --2- 172.21.15.77:0/3557639877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566db5953f0 0x5566db5a3bf0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.422 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.414+0000 7f1173aabb80 1 --2- 172.21.15.77:0/3557639877 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5566db5fdc40 0x5566db600060 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.423 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.414+0000 7f1173aabb80 1 --2- 172.21.15.77:0/3557639877 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566db5960f0 0x5566db5a2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.423 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.414+0000 7f1148ff9700 0 client.15720 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:55.423 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.414+0000 7f1173aabb80 1 -- 172.21.15.77:0/3557639877 >> 172.21.15.77:0/3557639877 conn(0x5566db4f40d0 msgr2=0x5566db59e940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:55.423 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.415+0000 7f1173aabb80 1 -- 172.21.15.77:0/3557639877 shutdown_connections 2023-12-01T01:18:55.423 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.415+0000 7f1173aabb80 1 -- 172.21.15.77:0/3557639877 wait complete. 2023-12-01T01:18:55.423 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.415+0000 7f1173aabb80 20 client.15720 trim_cache size 0 max 16384 2023-12-01T01:18:55.423 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2023-12-01T01:18:55.423 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2023-12-01T01:18:55.423 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2023-12-01T01:18:55.423 INFO:teuthology.orchestra.run.smithi077.stdout:} 2023-12-01T01:18:55.424 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.415+0000 7f116791b700 1 -- 172.21.15.77:0/2659906925 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f114c0a05a0 msgr2=0x7f114c0a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:55.424 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.415+0000 7f116791b700 1 --2- 172.21.15.77:0/2659906925 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f114c0a05a0 0x7f114c0a2a60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1160100cd0 tx=0x7f115c009260 comp rx=0 tx=0).stop 2023-12-01T01:18:55.424 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.415+0000 7f116791b700 1 -- 172.21.15.77:0/2659906925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1160101810 msgr2=0x7f116019ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:55.424 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.415+0000 7f116791b700 1 --2- 172.21.15.77:0/2659906925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1160101810 0x7f116019ddf0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1158007f10 tx=0x7f115800d3b0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.424 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.415+0000 7f116791b700 1 -- 172.21.15.77:0/2659906925 shutdown_connections 2023-12-01T01:18:55.424 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.415+0000 7f116791b700 1 --2- 172.21.15.77:0/2659906925 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f114c0a05a0 0x7f114c0a2a60 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.425 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.415+0000 7f116791b700 1 --2- 172.21.15.77:0/2659906925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1160101810 0x7f116019ddf0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.425 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.415+0000 7f116791b700 1 --2- 172.21.15.77:0/2659906925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11600ffc80 0x7f1160068ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.425 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.415+0000 7f116791b700 1 --2- 172.21.15.77:0/2659906925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1160100e70 0x7f116019b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.425 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.415+0000 7f116791b700 1 -- 172.21.15.77:0/2659906925 >> 172.21.15.77:0/2659906925 conn(0x7f11600fb1f0 msgr2=0x7f11600fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:55.425 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.415+0000 7f116791b700 1 -- 172.21.15.77:0/2659906925 shutdown_connections 2023-12-01T01:18:55.425 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.415+0000 7f116791b700 1 -- 172.21.15.77:0/2659906925 wait complete. 2023-12-01T01:18:55.428 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-12-01T01:18:55.428 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:18:55.560 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.558+0000 7f6a43a47700 1 -- 172.21.15.77:0/4266735694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a3c101800 msgr2=0x7f6a3c1080b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:55.560 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.558+0000 7f6a43a47700 1 --2- 172.21.15.77:0/4266735694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a3c101800 0x7f6a3c1080b0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6a3400b0d0 tx=0x7f6a3400b3e0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.560 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.558+0000 7f6a43a47700 1 -- 172.21.15.77:0/4266735694 shutdown_connections 2023-12-01T01:18:55.560 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.558+0000 7f6a43a47700 1 --2- 172.21.15.77:0/4266735694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a3c101800 0x7f6a3c1080b0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.560 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.558+0000 7f6a43a47700 1 --2- 172.21.15.77:0/4266735694 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a3c0ffc70 0x7f6a3c100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.560 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.558+0000 7f6a43a47700 1 --2- 172.21.15.77:0/4266735694 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a3c100e60 0x7f6a3c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.560 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.558+0000 7f6a43a47700 1 -- 172.21.15.77:0/4266735694 >> 172.21.15.77:0/4266735694 conn(0x7f6a3c0fb1c0 msgr2=0x7f6a3c0fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:55.560 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.558+0000 7f6a43a47700 1 -- 172.21.15.77:0/4266735694 shutdown_connections 2023-12-01T01:18:55.560 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.558+0000 7f6a43a47700 1 -- 172.21.15.77:0/4266735694 wait complete. 2023-12-01T01:18:55.562 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.559+0000 7f6a43a47700 1 Processor -- start 2023-12-01T01:18:55.562 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.559+0000 7f6a43a47700 1 -- start start 2023-12-01T01:18:55.562 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.559+0000 7f6a43a47700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a3c0ffc70 0x7f6a3c196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:55.562 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.559+0000 7f6a43a47700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a3c100e60 0x7f6a3c197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:55.562 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.559+0000 7f6a43a47700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a3c101800 0x7f6a3c19e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:55.562 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.559+0000 7f6a43a47700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c19e6a0 con 0x7f6a3c100e60 2023-12-01T01:18:55.562 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.559+0000 7f6a43a47700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c19e7e0 con 0x7f6a3c101800 2023-12-01T01:18:55.562 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.559+0000 7f6a43a47700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c19e970 con 0x7f6a3c0ffc70 2023-12-01T01:18:55.562 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.559+0000 7f6a40fe2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a3c100e60 0x7f6a3c197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:55.562 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.559+0000 7f6a40fe2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a3c100e60 0x7f6a3c197060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59910/0 (socket says 172.21.15.77:59910) 2023-12-01T01:18:55.562 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.559+0000 7f6a40fe2700 1 -- 172.21.15.77:0/3787081855 learned_addr learned my addr 172.21.15.77:0/3787081855 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:55.562 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.559+0000 7f6a41fe4700 1 --2- 172.21.15.77:0/3787081855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a3c101800 0x7f6a3c19e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:55.562 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.560+0000 7f6a40fe2700 1 -- 172.21.15.77:0/3787081855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a3c0ffc70 msgr2=0x7f6a3c196b20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-01T01:18:55.563 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.560+0000 7f6a40fe2700 1 --2- 172.21.15.77:0/3787081855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a3c0ffc70 0x7f6a3c196b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.563 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.560+0000 7f6a40fe2700 1 -- 172.21.15.77:0/3787081855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a3c101800 msgr2=0x7f6a3c19e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:55.563 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.560+0000 7f6a40fe2700 1 --2- 172.21.15.77:0/3787081855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a3c101800 0x7f6a3c19e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.563 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.560+0000 7f6a40fe2700 1 -- 172.21.15.77:0/3787081855 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a34009d00 con 0x7f6a3c100e60 2023-12-01T01:18:55.563 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.560+0000 7f6a40fe2700 1 --2- 172.21.15.77:0/3787081855 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a3c100e60 0x7f6a3c197060 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6a3800d950 tx=0x7f6a3800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:55.563 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.560+0000 7f6a2e7fc700 1 -- 172.21.15.77:0/3787081855 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a38009940 con 0x7f6a3c100e60 2023-12-01T01:18:55.563 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.560+0000 7f6a43a47700 1 -- 172.21.15.77:0/3787081855 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a3c19ec50 con 0x7f6a3c100e60 2023-12-01T01:18:55.563 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.560+0000 7f6a2e7fc700 1 -- 172.21.15.77:0/3787081855 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a38010460 con 0x7f6a3c100e60 2023-12-01T01:18:55.563 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.560+0000 7f6a2e7fc700 1 -- 172.21.15.77:0/3787081855 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3800f620 con 0x7f6a3c100e60 2023-12-01T01:18:55.563 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.560+0000 7f6a43a47700 1 -- 172.21.15.77:0/3787081855 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a3c19f160 con 0x7f6a3c100e60 2023-12-01T01:18:55.564 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.561+0000 7f6a43a47700 1 -- 172.21.15.77:0/3787081855 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a3c107110 con 0x7f6a3c100e60 2023-12-01T01:18:55.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.563+0000 7f6a2e7fc700 1 -- 172.21.15.77:0/3787081855 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f6a380108d0 con 0x7f6a3c100e60 2023-12-01T01:18:55.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.563+0000 7f6a2e7fc700 1 --2- 172.21.15.77:0/3787081855 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f6a280a0570 0x7f6a280a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:55.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.564+0000 7f6a2e7fc700 1 -- 172.21.15.77:0/3787081855 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f6a380cbad0 con 0x7f6a3c100e60 2023-12-01T01:18:55.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.564+0000 7f6a417e3700 1 --2- 172.21.15.77:0/3787081855 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f6a280a0570 0x7f6a280a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:55.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.564+0000 7f6a417e3700 1 --2- 172.21.15.77:0/3787081855 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f6a280a0570 0x7f6a280a2a30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c100cc0 tx=0x7f6a30009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:55.570 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.569+0000 7f6a2e7fc700 1 -- 172.21.15.77:0/3787081855 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6a38095710 con 0x7f6a3c100e60 2023-12-01T01:18:55.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:55 smithi077 ceph-mon[98867]: pgmap v329: 97 pgs: 97 active+clean; 17 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 135 op/s 2023-12-01T01:18:55.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:55 smithi203 ceph-mon[103581]: pgmap v329: 97 pgs: 97 active+clean; 17 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 135 op/s 2023-12-01T01:18:55.778 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:55 smithi137 ceph-mon[103312]: pgmap v329: 97 pgs: 97 active+clean; 17 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 135 op/s 2023-12-01T01:18:55.852 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.851+0000 7f6a43a47700 1 -- 172.21.15.77:0/3787081855 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a3c02d0d0 con 0x7f6a3c100e60 2023-12-01T01:18:55.853 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.852+0000 7f6a2e7fc700 1 -- 172.21.15.77:0/3787081855 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10845 (secure 0 0 0) 0x7f6a38098d30 con 0x7f6a3c100e60 2023-12-01T01:18:55.853 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:18:55.853 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1435145620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1435145620},{"type":"v1","addr":"172.21.15.77:6835","nonce":1435145620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/841838908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":841838908},{"type":"v1","addr":"172.21.15.77:6837","nonce":841838908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-01T01:16:25.345276+0000","modified":"2023-12-01T01:18:55.356755+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24496,"mds_2":24475,"mds_3":24473,"mds_4":14649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/3726913446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3726913446},{"type":"v1","addr":"172.21.15.77:6839","nonce":3726913446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14649":{"gid":14649,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":5,"addr":"172.21.15.77:6841/3023184965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3023184965},{"type":"v1","addr":"172.21.15.77:6841","nonce":3023184965}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/2249638445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2249638445},{"type":"v1","addr":"172.21.15.137:6833","nonce":2249638445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6833/1201599774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1201599774},{"type":"v1","addr":"172.21.15.203:6833","nonce":1201599774}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3935354878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3935354878},{"type":"v1","addr":"172.21.15.137:6835","nonce":3935354878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3237538176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3237538176},{"type":"v1","addr":"172.21.15.203:6835","nonce":3237538176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3518881867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3518881867},{"type":"v1","addr":"172.21.15.137:6837","nonce":3518881867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.203:6837/2887080977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2887080977},{"type":"v1","addr":"172.21.15.203:6837","nonce":2887080977}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.203:6839/2888221613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2888221613},{"type":"v1","addr":"172.21.15.203:6839","nonce":2888221613}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/1099966660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1099966660},{"type":"v1","addr":"172.21.15.137:6839","nonce":1099966660}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-01T01:18:55.856 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.854+0000 7f6a43a47700 1 -- 172.21.15.77:0/3787081855 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f6a280a0570 msgr2=0x7f6a280a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:55.856 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.854+0000 7f6a43a47700 1 --2- 172.21.15.77:0/3787081855 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f6a280a0570 0x7f6a280a2a30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c100cc0 tx=0x7f6a30009260 comp rx=0 tx=0).stop 2023-12-01T01:18:55.856 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.854+0000 7f6a43a47700 1 -- 172.21.15.77:0/3787081855 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a3c100e60 msgr2=0x7f6a3c197060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:55.856 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.854+0000 7f6a43a47700 1 --2- 172.21.15.77:0/3787081855 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a3c100e60 0x7f6a3c197060 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6a3800d950 tx=0x7f6a3800dc60 comp rx=0 tx=0).stop 2023-12-01T01:18:55.856 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.854+0000 7f6a43a47700 1 -- 172.21.15.77:0/3787081855 shutdown_connections 2023-12-01T01:18:55.856 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.854+0000 7f6a43a47700 1 --2- 172.21.15.77:0/3787081855 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f6a280a0570 0x7f6a280a2a30 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.856 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.854+0000 7f6a43a47700 1 --2- 172.21.15.77:0/3787081855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a3c101800 0x7f6a3c19e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.856 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.854+0000 7f6a43a47700 1 --2- 172.21.15.77:0/3787081855 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a3c0ffc70 0x7f6a3c196b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.857 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.854+0000 7f6a43a47700 1 --2- 172.21.15.77:0/3787081855 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a3c100e60 0x7f6a3c197060 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.857 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.854+0000 7f6a43a47700 1 -- 172.21.15.77:0/3787081855 >> 172.21.15.77:0/3787081855 conn(0x7f6a3c0fb1c0 msgr2=0x7f6a3c104250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:55.857 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.854+0000 7f6a43a47700 1 -- 172.21.15.77:0/3787081855 shutdown_connections 2023-12-01T01:18:55.857 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.854+0000 7f6a43a47700 1 -- 172.21.15.77:0/3787081855 wait complete. 2023-12-01T01:18:55.857 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 29 2023-12-01T01:18:55.868 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-12-01T01:18:55.996 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.993+0000 7fdb95cda700 1 -- 172.21.15.77:0/931026254 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb900ffc80 msgr2=0x7fdb901000a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:55.996 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.993+0000 7fdb95cda700 1 --2- 172.21.15.77:0/931026254 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb900ffc80 0x7fdb901000a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fdb800099c0 tx=0x7fdb80009cd0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.997 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.993+0000 7fdb95cda700 1 -- 172.21.15.77:0/931026254 shutdown_connections 2023-12-01T01:18:55.997 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.993+0000 7fdb95cda700 1 --2- 172.21.15.77:0/931026254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb90101810 0x7fdb901080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.997 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.993+0000 7fdb95cda700 1 --2- 172.21.15.77:0/931026254 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb900ffc80 0x7fdb901000a0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.997 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.993+0000 7fdb95cda700 1 --2- 172.21.15.77:0/931026254 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb90100e70 0x7fdb901012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.997 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.993+0000 7fdb95cda700 1 -- 172.21.15.77:0/931026254 >> 172.21.15.77:0/931026254 conn(0x7fdb900fb1f0 msgr2=0x7fdb900fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:55.997 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.993+0000 7fdb95cda700 1 -- 172.21.15.77:0/931026254 shutdown_connections 2023-12-01T01:18:55.997 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.993+0000 7fdb95cda700 1 -- 172.21.15.77:0/931026254 wait complete. 2023-12-01T01:18:55.997 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.994+0000 7fdb95cda700 1 Processor -- start 2023-12-01T01:18:55.997 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.994+0000 7fdb95cda700 1 -- start start 2023-12-01T01:18:55.997 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.994+0000 7fdb95cda700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb900ffc80 0x7fdb90196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:55.997 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.994+0000 7fdb95cda700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb90100e70 0x7fdb90196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:55.997 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.994+0000 7fdb95cda700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb90101810 0x7fdb9019dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:55.997 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.994+0000 7fdb95cda700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9019e5c0 con 0x7fdb90100e70 2023-12-01T01:18:55.997 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.994+0000 7fdb95cda700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9019e700 con 0x7fdb90101810 2023-12-01T01:18:55.998 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.994+0000 7fdb95cda700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9019e890 con 0x7fdb900ffc80 2023-12-01T01:18:55.998 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.994+0000 7fdb8effd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb90100e70 0x7fdb90196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:55.998 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.994+0000 7fdb8effd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb90100e70 0x7fdb90196f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59924/0 (socket says 172.21.15.77:59924) 2023-12-01T01:18:55.998 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.994+0000 7fdb8effd700 1 -- 172.21.15.77:0/2248910028 learned_addr learned my addr 172.21.15.77:0/2248910028 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:55.998 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.994+0000 7fdb8f7fe700 1 --2- 172.21.15.77:0/2248910028 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb900ffc80 0x7fdb90196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:55.998 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.994+0000 7fdb8ffff700 1 --2- 172.21.15.77:0/2248910028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb90101810 0x7fdb9019dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:55.998 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.995+0000 7fdb8effd700 1 -- 172.21.15.77:0/2248910028 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb900ffc80 msgr2=0x7fdb90196a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:55.998 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.995+0000 7fdb8effd700 1 --2- 172.21.15.77:0/2248910028 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb900ffc80 0x7fdb90196a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.998 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.995+0000 7fdb8effd700 1 -- 172.21.15.77:0/2248910028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb90101810 msgr2=0x7fdb9019dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:55.998 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.995+0000 7fdb8effd700 1 --2- 172.21.15.77:0/2248910028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb90101810 0x7fdb9019dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:55.998 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.995+0000 7fdb8effd700 1 -- 172.21.15.77:0/2248910028 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb800095f0 con 0x7fdb90100e70 2023-12-01T01:18:55.998 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.995+0000 7fdb8f7fe700 1 --2- 172.21.15.77:0/2248910028 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb900ffc80 0x7fdb90196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:18:55.998 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.995+0000 7fdb8effd700 1 --2- 172.21.15.77:0/2248910028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb90100e70 0x7fdb90196f80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fdb7800e9a0 tx=0x7fdb7800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:55.998 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.995+0000 7fdb8cff9700 1 -- 172.21.15.77:0/2248910028 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb78009790 con 0x7fdb90100e70 2023-12-01T01:18:55.999 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.995+0000 7fdb8cff9700 1 -- 172.21.15.77:0/2248910028 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb78010430 con 0x7fdb90100e70 2023-12-01T01:18:55.999 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.995+0000 7fdb95cda700 1 -- 172.21.15.77:0/2248910028 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb9019eb70 con 0x7fdb90100e70 2023-12-01T01:18:55.999 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.995+0000 7fdb8cff9700 1 -- 172.21.15.77:0/2248910028 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb78018680 con 0x7fdb90100e70 2023-12-01T01:18:55.999 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.995+0000 7fdb95cda700 1 -- 172.21.15.77:0/2248910028 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb9019f080 con 0x7fdb90100e70 2023-12-01T01:18:56.001 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.997+0000 7fdb8cff9700 1 -- 172.21.15.77:0/2248910028 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fdb780105a0 con 0x7fdb90100e70 2023-12-01T01:18:56.001 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.998+0000 7fdb8cff9700 1 --2- 172.21.15.77:0/2248910028 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fdb7c0a0520 0x7fdb7c0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.001 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.998+0000 7fdb8cff9700 1 -- 172.21.15.77:0/2248910028 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fdb78014070 con 0x7fdb90100e70 2023-12-01T01:18:56.001 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.998+0000 7fdb8f7fe700 1 --2- 172.21.15.77:0/2248910028 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fdb7c0a0520 0x7fdb7c0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.001 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.998+0000 7fdb8f7fe700 1 --2- 172.21.15.77:0/2248910028 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fdb7c0a0520 0x7fdb7c0a29e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdb80009990 tx=0x7fdb8001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:56.001 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.999+0000 7fdba1e6ab80 1 Processor -- start 2023-12-01T01:18:56.001 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.999+0000 7fdba1e6ab80 1 -- start start 2023-12-01T01:18:56.001 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.999+0000 7fdba1e6ab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635da13d500 0x5635da13d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.999+0000 7fdba1e6ab80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635da13de80 0x5635da13c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.999+0000 7fdba1e6ab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635da13c800 0x5635da13cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.999+0000 7fdba1e6ab80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635da08fa10 con 0x5635da13de80 2023-12-01T01:18:56.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.999+0000 7fdba1e6ab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635da08d3d0 con 0x5635da13d500 2023-12-01T01:18:56.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.999+0000 7fdba1e6ab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635da088cc0 con 0x5635da13c800 2023-12-01T01:18:56.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.999+0000 7fdb8effd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635da13de80 0x5635da13c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.999+0000 7fdb8f7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635da13d500 0x5635da13d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.999+0000 7fdb8effd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635da13de80 0x5635da13c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59932/0 (socket says 172.21.15.77:59932) 2023-12-01T01:18:56.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.999+0000 7fdb8effd700 1 -- 172.21.15.77:0/1548927311 learned_addr learned my addr 172.21.15.77:0/1548927311 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:56.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.999+0000 7fdb8ffff700 1 --2- 172.21.15.77:0/1548927311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635da13c800 0x5635da13cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.999+0000 7fdb8effd700 1 -- 172.21.15.77:0/1548927311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635da13c800 msgr2=0x5635da13cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:56.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.999+0000 7fdb8effd700 1 --2- 172.21.15.77:0/1548927311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635da13c800 0x5635da13cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.999+0000 7fdb8effd700 1 -- 172.21.15.77:0/1548927311 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635da13d500 msgr2=0x5635da13d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:56.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.999+0000 7fdb8effd700 1 --2- 172.21.15.77:0/1548927311 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635da13d500 0x5635da13d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.999+0000 7fdb8effd700 1 -- 172.21.15.77:0/1548927311 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635d9eb6de0 con 0x5635da13de80 2023-12-01T01:18:56.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.999+0000 7fdb8f7fe700 1 --2- 172.21.15.77:0/1548927311 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635da13d500 0x5635da13d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:18:56.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:55.999+0000 7fdb8effd700 1 --2- 172.21.15.77:0/1548927311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635da13de80 0x5635da13c270 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fdb78095880 tx=0x7fdb78098fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5bb280ead9cbd79 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:56.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.000+0000 7fdb85ffb700 1 -- 172.21.15.77:0/1548927311 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb78084070 con 0x5635da13de80 2023-12-01T01:18:56.007 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.000+0000 7fdb85ffb700 1 -- 172.21.15.77:0/1548927311 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb78099380 con 0x5635da13de80 2023-12-01T01:18:56.007 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.000+0000 7fdb85ffb700 1 -- 172.21.15.77:0/1548927311 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb78098ad0 con 0x5635da13de80 2023-12-01T01:18:56.007 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.000+0000 7fdba1e6ab80 1 -- 172.21.15.77:0/1548927311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635da13de80 msgr2=0x5635da13c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:56.007 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.000+0000 7fdba1e6ab80 1 --2- 172.21.15.77:0/1548927311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635da13de80 0x5635da13c270 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fdb78095880 tx=0x7fdb78098fc0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.007 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.000+0000 7fdba1e6ab80 1 -- 172.21.15.77:0/1548927311 shutdown_connections 2023-12-01T01:18:56.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.000+0000 7fdba1e6ab80 1 --2- 172.21.15.77:0/1548927311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635da13c800 0x5635da13cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.000+0000 7fdba1e6ab80 1 --2- 172.21.15.77:0/1548927311 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635da13d500 0x5635da13d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.000+0000 7fdba1e6ab80 1 --2- 172.21.15.77:0/1548927311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635da13de80 0x5635da13c270 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.000+0000 7fdba1e6ab80 1 -- 172.21.15.77:0/1548927311 >> 172.21.15.77:0/1548927311 conn(0x5635da0970d0 msgr2=0x5635da132940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:56.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.000+0000 7fdba1e6ab80 1 -- 172.21.15.77:0/1548927311 shutdown_connections 2023-12-01T01:18:56.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.000+0000 7fdba1e6ab80 1 -- 172.21.15.77:0/1548927311 wait complete. 2023-12-01T01:18:56.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.001+0000 7fdba1e6ab80 1 -- start start 2023-12-01T01:18:56.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.001+0000 7fdba1e6ab80 1 Processor -- start 2023-12-01T01:18:56.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.001+0000 7fdba1e6ab80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635da13c800 0x5635da145e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.001+0000 7fdba1e6ab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635da13d500 0x5635da146370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.001+0000 7fdba1e6ab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635da13de80 0x5635da14d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.001+0000 7fdba1e6ab80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635da025510 con 0x5635da13c800 2023-12-01T01:18:56.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.001+0000 7fdba1e6ab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635da0251b0 con 0x5635da13d500 2023-12-01T01:18:56.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.001+0000 7fdba1e6ab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635d9f0c920 con 0x5635da13de80 2023-12-01T01:18:56.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.001+0000 7fdba1e6ab80 10 client.0 _ll_register_callbacks cb 0x5635d9f72a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:18:56.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.001+0000 7fdb8f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635da13c800 0x5635da145e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.001+0000 7fdb8f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635da13c800 0x5635da145e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59936/0 (socket says 172.21.15.77:59936) 2023-12-01T01:18:56.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.001+0000 7fdb8f7fe700 1 -- 172.21.15.77:0/706067318 learned_addr learned my addr 172.21.15.77:0/706067318 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:56.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.001+0000 7fdb8effd700 1 --2- 172.21.15.77:0/706067318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635da13d500 0x5635da146370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.001+0000 7fdb8ffff700 1 --2- 172.21.15.77:0/706067318 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635da13de80 0x5635da14d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.002+0000 7fdb8f7fe700 1 -- 172.21.15.77:0/706067318 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635da13de80 msgr2=0x5635da14d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:56.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.002+0000 7fdb8f7fe700 1 --2- 172.21.15.77:0/706067318 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635da13de80 0x5635da14d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.002+0000 7fdb8f7fe700 1 -- 172.21.15.77:0/706067318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635da13d500 msgr2=0x5635da146370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:56.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.002+0000 7fdb8f7fe700 1 --2- 172.21.15.77:0/706067318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635da13d500 0x5635da146370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.002+0000 7fdb8f7fe700 1 -- 172.21.15.77:0/706067318 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb90108a10 con 0x5635da13c800 2023-12-01T01:18:56.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.002+0000 7fdb8f7fe700 1 --2- 172.21.15.77:0/706067318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635da13c800 0x5635da145e30 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fdb800099c0 tx=0x7fdb80009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16b8f8b338e51f2 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:56.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.002+0000 7fdb6effd700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:18:56.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.002+0000 7fdb6effd700 1 -- 172.21.15.77:0/706067318 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8002a070 con 0x5635da13c800 2023-12-01T01:18:56.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.002+0000 7fdb6effd700 1 -- 172.21.15.77:0/706067318 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fdb800056a0 con 0x5635da13c800 2023-12-01T01:18:56.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.002+0000 7fdba1e6ab80 1 -- 172.21.15.77:0/706067318 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5635da0b3d10 con 0x5635da13c800 2023-12-01T01:18:56.010 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.002+0000 7fdb6effd700 1 -- 172.21.15.77:0/706067318 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb8000f460 con 0x5635da13c800 2023-12-01T01:18:56.010 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.002+0000 7fdba1e6ab80 10 client.15738 fetch_fsmap learned FSMap version 29 2023-12-01T01:18:56.010 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.002+0000 7fdba1e6ab80 1 -- 172.21.15.77:0/706067318 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fdb9019f080 con 0x5635da13c800 2023-12-01T01:18:56.010 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.002+0000 7fdb6effd700 1 -- 172.21.15.77:0/706067318 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb8003e020 con 0x5635da13c800 2023-12-01T01:18:56.010 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.002+0000 7fdb6effd700 1 -- 172.21.15.77:0/706067318 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdb80024720 con 0x5635da13c800 2023-12-01T01:18:56.010 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.002+0000 7fdba1e6ab80 10 client.15738 fetch_fsmap finished waiting for FSMap version 29 2023-12-01T01:18:56.010 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.002+0000 7fdba1e6ab80 10 client.15738 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2023-12-01T01:18:56.010 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.002+0000 7fdba1e6ab80 20 client.15738 populate_metadata read hostname 'smithi077' 2023-12-01T01:18:56.010 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.002+0000 7fdba1e6ab80 1 --2- 172.21.15.77:0/706067318 >> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] conn(0x5635da151c30 0x5635da154050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.010 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.002+0000 7fdba1e6ab80 4 client.15738 mds_command: new command op to 14649 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:18:56.010 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.002+0000 7fdba1e6ab80 1 -- 172.21.15.77:0/706067318 --> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5635da106e10 con 0x5635da151c30 2023-12-01T01:18:56.010 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.003+0000 7fdb8effd700 1 --2- 172.21.15.77:0/706067318 >> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] conn(0x5635da151c30 0x5635da154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.010 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.003+0000 7fdb6effd700 10 client.15738 ms_handle_connect on v2:172.21.15.77:6840/3023184965 2023-12-01T01:18:56.010 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.003+0000 7fdb8effd700 1 --2- 172.21.15.77:0/706067318 >> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] conn(0x5635da151c30 0x5635da154050 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ed1d95cd2e38f99d server_cookie=ce0b9c52005c898b in_seq=0 out_seq=0 2023-12-01T01:18:56.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.003+0000 7fdb6effd700 1 -- 172.21.15.77:0/706067318 <== mds.4 v2:172.21.15.77:6840/3023184965 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5635da106e10 con 0x5635da151c30 2023-12-01T01:18:56.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.003+0000 7fdb6effd700 10 client.15738 handle_command_reply: tid=0 2023-12-01T01:18:56.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.003+0000 7fdba1e6ab80 1 client.15738 shutdown 2023-12-01T01:18:56.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.003+0000 7fdba1e6ab80 1 -- 172.21.15.77:0/706067318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635da13c800 msgr2=0x5635da145e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:56.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.004+0000 7fdba1e6ab80 1 --2- 172.21.15.77:0/706067318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635da13c800 0x5635da145e30 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fdb800099c0 tx=0x7fdb80009310 comp rx=0 tx=0).stop 2023-12-01T01:18:56.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.004+0000 7fdba1e6ab80 1 -- 172.21.15.77:0/706067318 shutdown_connections 2023-12-01T01:18:56.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.004+0000 7fdba1e6ab80 1 --2- 172.21.15.77:0/706067318 >> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] conn(0x5635da151c30 0x5635da154050 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.004+0000 7fdba1e6ab80 1 --2- 172.21.15.77:0/706067318 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635da13de80 0x5635da14d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.004+0000 7fdba1e6ab80 1 --2- 172.21.15.77:0/706067318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635da13c800 0x5635da145e30 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.004+0000 7fdb6effd700 0 client.15738 ms_handle_reset on v2:172.21.15.77:6840/3023184965 2023-12-01T01:18:56.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.004+0000 7fdba1e6ab80 1 --2- 172.21.15.77:0/706067318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635da13d500 0x5635da146370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.004+0000 7fdba1e6ab80 1 -- 172.21.15.77:0/706067318 >> 172.21.15.77:0/706067318 conn(0x5635da0970d0 msgr2=0x5635da1320b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:56.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.004+0000 7fdba1e6ab80 1 -- 172.21.15.77:0/706067318 shutdown_connections 2023-12-01T01:18:56.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.004+0000 7fdba1e6ab80 1 -- 172.21.15.77:0/706067318 wait complete. 2023-12-01T01:18:56.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.004+0000 7fdba1e6ab80 20 client.15738 trim_cache size 0 max 16384 2023-12-01T01:18:56.020 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.017+0000 7fdba1e6ab80 1 Processor -- start 2023-12-01T01:18:56.020 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.017+0000 7fdba1e6ab80 1 -- start start 2023-12-01T01:18:56.020 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.017+0000 7fdba1e6ab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635da13c800 0x5635da14b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.020 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.017+0000 7fdba1e6ab80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635da13d500 0x5635da1454e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.020 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.017+0000 7fdba1e6ab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635da13de80 0x5635da132910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.020 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.017+0000 7fdba1e6ab80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635da1452e0 con 0x5635da13d500 2023-12-01T01:18:56.020 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.017+0000 7fdba1e6ab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635da1335d0 con 0x5635da13de80 2023-12-01T01:18:56.020 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.017+0000 7fdba1e6ab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635da1337d0 con 0x5635da13c800 2023-12-01T01:18:56.020 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.017+0000 7fdb8effd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635da13d500 0x5635da1454e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.020 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.017+0000 7fdb8effd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635da13d500 0x5635da1454e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59946/0 (socket says 172.21.15.77:59946) 2023-12-01T01:18:56.020 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.017+0000 7fdb8effd700 1 -- 172.21.15.77:0/3528260765 learned_addr learned my addr 172.21.15.77:0/3528260765 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:56.021 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.017+0000 7fdb8ffff700 1 --2- 172.21.15.77:0/3528260765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635da13de80 0x5635da132910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.021 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.017+0000 7fdb8f7fe700 1 --2- 172.21.15.77:0/3528260765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635da13c800 0x5635da14b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.021 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.017+0000 7fdb8effd700 1 -- 172.21.15.77:0/3528260765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635da13c800 msgr2=0x5635da14b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:56.021 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.017+0000 7fdb8effd700 1 --2- 172.21.15.77:0/3528260765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635da13c800 0x5635da14b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.021 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.017+0000 7fdb8effd700 1 -- 172.21.15.77:0/3528260765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635da13de80 msgr2=0x5635da132910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:56.021 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.017+0000 7fdb8effd700 1 --2- 172.21.15.77:0/3528260765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635da13de80 0x5635da132910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.021 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.017+0000 7fdb8effd700 1 -- 172.21.15.77:0/3528260765 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635d9e9fc60 con 0x5635da13d500 2023-12-01T01:18:56.021 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.017+0000 7fdb8effd700 1 --2- 172.21.15.77:0/3528260765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635da13d500 0x5635da1454e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fdb78095df0 tx=0x7fdb78093f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8c78e10c78d33f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:56.021 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.018+0000 7fdb867fc700 1 -- 172.21.15.77:0/3528260765 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb780b5070 con 0x5635da13d500 2023-12-01T01:18:56.021 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.018+0000 7fdb867fc700 1 -- 172.21.15.77:0/3528260765 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb780a13f0 con 0x5635da13d500 2023-12-01T01:18:56.021 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.018+0000 7fdb867fc700 1 -- 172.21.15.77:0/3528260765 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb78098870 con 0x5635da13d500 2023-12-01T01:18:56.021 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.018+0000 7fdba1e6ab80 1 -- 172.21.15.77:0/3528260765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635da13d500 msgr2=0x5635da1454e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:56.021 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.018+0000 7fdba1e6ab80 1 --2- 172.21.15.77:0/3528260765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635da13d500 0x5635da1454e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fdb78095df0 tx=0x7fdb78093f90 comp rx=0 tx=0).stop 2023-12-01T01:18:56.021 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.018+0000 7fdba1e6ab80 1 -- 172.21.15.77:0/3528260765 shutdown_connections 2023-12-01T01:18:56.021 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.018+0000 7fdba1e6ab80 1 --2- 172.21.15.77:0/3528260765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635da13de80 0x5635da132910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.022 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.018+0000 7fdba1e6ab80 1 --2- 172.21.15.77:0/3528260765 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635da13c800 0x5635da14b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.022 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.018+0000 7fdba1e6ab80 1 --2- 172.21.15.77:0/3528260765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635da13d500 0x5635da1454e0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.022 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.018+0000 7fdba1e6ab80 1 -- 172.21.15.77:0/3528260765 >> 172.21.15.77:0/3528260765 conn(0x5635da0970d0 msgr2=0x5635da13cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:56.022 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.018+0000 7fdba1e6ab80 1 -- 172.21.15.77:0/3528260765 shutdown_connections 2023-12-01T01:18:56.022 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.018+0000 7fdba1e6ab80 1 -- 172.21.15.77:0/3528260765 wait complete. 2023-12-01T01:18:56.022 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.019+0000 7fdba1e6ab80 1 -- start start 2023-12-01T01:18:56.022 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.019+0000 7fdba1e6ab80 1 Processor -- start 2023-12-01T01:18:56.025 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.019+0000 7fdba1e6ab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635da13c800 0x5635da149590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.025 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.019+0000 7fdba1e6ab80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635da13d500 0x5635da149ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.025 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.019+0000 7fdba1e6ab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635da13de80 0x5635da14a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.025 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.019+0000 7fdba1e6ab80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635da14ab20 con 0x5635da13d500 2023-12-01T01:18:56.025 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.019+0000 7fdba1e6ab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635da14ac90 con 0x5635da13de80 2023-12-01T01:18:56.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.019+0000 7fdba1e6ab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635da14ae50 con 0x5635da13c800 2023-12-01T01:18:56.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.019+0000 7fdba1e6ab80 10 client.0 _ll_register_callbacks cb 0x7fdb80004970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:18:56.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.019+0000 7fdb8effd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635da13d500 0x5635da149ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.019+0000 7fdb8f7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635da13c800 0x5635da149590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.019+0000 7fdb8effd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635da13d500 0x5635da149ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59952/0 (socket says 172.21.15.77:59952) 2023-12-01T01:18:56.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.019+0000 7fdb8effd700 1 -- 172.21.15.77:0/1748611907 learned_addr learned my addr 172.21.15.77:0/1748611907 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:56.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.019+0000 7fdb8ffff700 1 --2- 172.21.15.77:0/1748611907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635da13de80 0x5635da14a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.020+0000 7fdb8f7fe700 1 -- 172.21.15.77:0/1748611907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635da13de80 msgr2=0x5635da14a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:56.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.020+0000 7fdb8f7fe700 1 --2- 172.21.15.77:0/1748611907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635da13de80 0x5635da14a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.020+0000 7fdb8f7fe700 1 -- 172.21.15.77:0/1748611907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635da13d500 msgr2=0x5635da149ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:56.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.020+0000 7fdb8f7fe700 1 --2- 172.21.15.77:0/1748611907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635da13d500 0x5635da149ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.020+0000 7fdb8f7fe700 1 -- 172.21.15.77:0/1748611907 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5635d9eb6de0 con 0x5635da13c800 2023-12-01T01:18:56.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.020+0000 7fdb8effd700 1 --2- 172.21.15.77:0/1748611907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635da13d500 0x5635da149ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:18:56.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.020+0000 7fdb8f7fe700 1 --2- 172.21.15.77:0/1748611907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635da13c800 0x5635da149590 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdb80024a50 tx=0x7fdb800055d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ae5c5aa9a936abd server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:56.027 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.020+0000 7fdb6effd700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2023-12-01T01:18:56.027 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.020+0000 7fdb6effd700 1 -- 172.21.15.77:0/1748611907 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8002a040 con 0x5635da13c800 2023-12-01T01:18:56.027 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.020+0000 7fdba1e6ab80 1 -- 172.21.15.77:0/1748611907 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb78099380 con 0x5635da13c800 2023-12-01T01:18:56.027 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.020+0000 7fdb6effd700 1 -- 172.21.15.77:0/1748611907 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fdb8003f080 con 0x5635da13c800 2023-12-01T01:18:56.027 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.020+0000 7fdb6effd700 1 -- 172.21.15.77:0/1748611907 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb80024830 con 0x5635da13c800 2023-12-01T01:18:56.027 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.020+0000 7fdba1e6ab80 10 client.25339 fetch_fsmap learned FSMap version 29 2023-12-01T01:18:56.027 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.020+0000 7fdba1e6ab80 1 -- 172.21.15.77:0/1748611907 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5635d9edd250 con 0x5635da13c800 2023-12-01T01:18:56.027 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.020+0000 7fdb6effd700 1 -- 172.21.15.77:0/1748611907 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb78099380 con 0x5635da13c800 2023-12-01T01:18:56.027 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.021+0000 7fdb6effd700 1 -- 172.21.15.77:0/1748611907 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdb80041040 con 0x5635da13c800 2023-12-01T01:18:56.027 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.021+0000 7fdba1e6ab80 10 client.25339 fetch_fsmap finished waiting for FSMap version 29 2023-12-01T01:18:56.027 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.021+0000 7fdba1e6ab80 10 client.25339 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2023-12-01T01:18:56.027 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.021+0000 7fdba1e6ab80 20 client.25339 populate_metadata read hostname 'smithi077' 2023-12-01T01:18:56.027 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.021+0000 7fdba1e6ab80 1 --2- 172.21.15.77:0/1748611907 >> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] conn(0x5635da1a41a0 0x5635da1a6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.027 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.021+0000 7fdba1e6ab80 4 client.25339 mds_command: new command op to 14649 tid=0[{"prefix": "damage ls"}] 2023-12-01T01:18:56.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.021+0000 7fdba1e6ab80 1 -- 172.21.15.77:0/1748611907 --> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5635da081d40 con 0x5635da1a41a0 2023-12-01T01:18:56.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.021+0000 7fdb8effd700 1 --2- 172.21.15.77:0/1748611907 >> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] conn(0x5635da1a41a0 0x5635da1a6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.021+0000 7fdb6effd700 10 client.25339 ms_handle_connect on v2:172.21.15.77:6840/3023184965 2023-12-01T01:18:56.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.021+0000 7fdb8effd700 1 --2- 172.21.15.77:0/1748611907 >> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] conn(0x5635da1a41a0 0x5635da1a6590 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bc0f7a8cc83261da server_cookie=72c359d4a37172e3 in_seq=0 out_seq=0 2023-12-01T01:18:56.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.021+0000 7fdb6effd700 1 -- 172.21.15.77:0/1748611907 <== mds.4 v2:172.21.15.77:6840/3023184965 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5635da081d40 con 0x5635da1a41a0 2023-12-01T01:18:56.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.021+0000 7fdb6effd700 10 client.25339 handle_command_reply: tid=0 2023-12-01T01:18:56.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.021+0000 7fdba1e6ab80 1 client.25339 shutdown 2023-12-01T01:18:56.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.021+0000 7fdba1e6ab80 1 -- 172.21.15.77:0/1748611907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635da13c800 msgr2=0x5635da149590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:56.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.021+0000 7fdba1e6ab80 1 --2- 172.21.15.77:0/1748611907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635da13c800 0x5635da149590 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdb80024a50 tx=0x7fdb800055d0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.022+0000 7fdba1e6ab80 1 -- 172.21.15.77:0/1748611907 shutdown_connections 2023-12-01T01:18:56.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.022+0000 7fdba1e6ab80 1 --2- 172.21.15.77:0/1748611907 >> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] conn(0x5635da1a41a0 0x5635da1a6590 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.022+0000 7fdba1e6ab80 1 --2- 172.21.15.77:0/1748611907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635da13de80 0x5635da14a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.022+0000 7fdb6effd700 0 client.25339 ms_handle_reset on v2:172.21.15.77:6840/3023184965 2023-12-01T01:18:56.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.022+0000 7fdba1e6ab80 1 --2- 172.21.15.77:0/1748611907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635da13c800 0x5635da149590 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.022+0000 7fdba1e6ab80 1 --2- 172.21.15.77:0/1748611907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5635da13d500 0x5635da149ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.022+0000 7fdba1e6ab80 1 -- 172.21.15.77:0/1748611907 >> 172.21.15.77:0/1748611907 conn(0x5635da0970d0 msgr2=0x5635da144ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:56.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.022+0000 7fdba1e6ab80 1 -- 172.21.15.77:0/1748611907 shutdown_connections 2023-12-01T01:18:56.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.022+0000 7fdba1e6ab80 1 -- 172.21.15.77:0/1748611907 wait complete. 2023-12-01T01:18:56.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.022+0000 7fdba1e6ab80 20 client.25339 trim_cache size 0 max 16384 2023-12-01T01:18:56.029 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2023-12-01T01:18:56.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.022+0000 7fdb95cda700 1 -- 172.21.15.77:0/2248910028 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fdb7c0a0520 msgr2=0x7fdb7c0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:56.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.022+0000 7fdb95cda700 1 --2- 172.21.15.77:0/2248910028 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fdb7c0a0520 0x7fdb7c0a29e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdb80009990 tx=0x7fdb8001a040 comp rx=0 tx=0).stop 2023-12-01T01:18:56.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.022+0000 7fdb95cda700 1 -- 172.21.15.77:0/2248910028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb90100e70 msgr2=0x7fdb90196f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:56.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.022+0000 7fdb95cda700 1 --2- 172.21.15.77:0/2248910028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb90100e70 0x7fdb90196f80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fdb7800e9a0 tx=0x7fdb7800ecb0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.022+0000 7fdb95cda700 1 -- 172.21.15.77:0/2248910028 shutdown_connections 2023-12-01T01:18:56.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.022+0000 7fdb95cda700 1 --2- 172.21.15.77:0/2248910028 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fdb7c0a0520 0x7fdb7c0a29e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.022+0000 7fdb95cda700 1 --2- 172.21.15.77:0/2248910028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb90101810 0x7fdb9019dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.022+0000 7fdb95cda700 1 --2- 172.21.15.77:0/2248910028 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb900ffc80 0x7fdb90196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.022+0000 7fdb95cda700 1 --2- 172.21.15.77:0/2248910028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb90100e70 0x7fdb90196f80 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.022+0000 7fdb95cda700 1 -- 172.21.15.77:0/2248910028 >> 172.21.15.77:0/2248910028 conn(0x7fdb900fb1f0 msgr2=0x7fdb900fbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:56.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.022+0000 7fdb95cda700 1 -- 172.21.15.77:0/2248910028 shutdown_connections 2023-12-01T01:18:56.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.022+0000 7fdb95cda700 1 -- 172.21.15.77:0/2248910028 wait complete. 2023-12-01T01:18:56.036 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-12-01T01:18:56.196 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.192+0000 7f93f3562700 1 -- 172.21.15.77:0/3112708900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93ec0ffcb0 msgr2=0x7f93ec1000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:56.196 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.192+0000 7f93f3562700 1 --2- 172.21.15.77:0/3112708900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93ec0ffcb0 0x7f93ec1000d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f93e80099c0 tx=0x7f93e8009cd0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.196 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.193+0000 7f93f3562700 1 -- 172.21.15.77:0/3112708900 shutdown_connections 2023-12-01T01:18:56.196 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.193+0000 7f93f3562700 1 --2- 172.21.15.77:0/3112708900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93ec101840 0x7f93ec1080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.196 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.193+0000 7f93f3562700 1 --2- 172.21.15.77:0/3112708900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93ec0ffcb0 0x7f93ec1000d0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.196 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.193+0000 7f93f3562700 1 --2- 172.21.15.77:0/3112708900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93ec100ea0 0x7f93ec101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.196 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.193+0000 7f93f3562700 1 -- 172.21.15.77:0/3112708900 >> 172.21.15.77:0/3112708900 conn(0x7f93ec0fb220 msgr2=0x7f93ec0fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:56.196 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.193+0000 7f93f3562700 1 -- 172.21.15.77:0/3112708900 shutdown_connections 2023-12-01T01:18:56.196 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.193+0000 7f93f3562700 1 -- 172.21.15.77:0/3112708900 wait complete. 2023-12-01T01:18:56.196 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.193+0000 7f93f3562700 1 Processor -- start 2023-12-01T01:18:56.196 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.193+0000 7f93f3562700 1 -- start start 2023-12-01T01:18:56.196 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.193+0000 7f93f3562700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93ec0ffcb0 0x7f93ec069ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.197 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.193+0000 7f93f3562700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93ec100ea0 0x7f93ec19b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.197 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.193+0000 7f93f3562700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93ec101840 0x7f93ec19ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.197 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.193+0000 7f93f3562700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec19e590 con 0x7f93ec100ea0 2023-12-01T01:18:56.197 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.193+0000 7f93f3562700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec19e700 con 0x7f93ec101840 2023-12-01T01:18:56.197 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.193+0000 7f93f3562700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec19e8c0 con 0x7f93ec0ffcb0 2023-12-01T01:18:56.197 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.194+0000 7f93f12fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93ec0ffcb0 0x7f93ec069ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.197 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.194+0000 7f93f0afd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93ec100ea0 0x7f93ec19b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.197 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.194+0000 7f93f12fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93ec0ffcb0 0x7f93ec069ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:46152/0 (socket says 172.21.15.77:46152) 2023-12-01T01:18:56.197 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.194+0000 7f93f12fe700 1 -- 172.21.15.77:0/275018041 learned_addr learned my addr 172.21.15.77:0/275018041 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:56.197 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.194+0000 7f93f1aff700 1 --2- 172.21.15.77:0/275018041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93ec101840 0x7f93ec19ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.197 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.194+0000 7f93f12fe700 1 -- 172.21.15.77:0/275018041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93ec101840 msgr2=0x7f93ec19ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:56.197 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.194+0000 7f93f12fe700 1 --2- 172.21.15.77:0/275018041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93ec101840 0x7f93ec19ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.197 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.194+0000 7f93f12fe700 1 -- 172.21.15.77:0/275018041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93ec100ea0 msgr2=0x7f93ec19b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:56.197 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.194+0000 7f93f12fe700 1 --2- 172.21.15.77:0/275018041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93ec100ea0 0x7f93ec19b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.198 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.194+0000 7f93f12fe700 1 -- 172.21.15.77:0/275018041 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e80095f0 con 0x7f93ec0ffcb0 2023-12-01T01:18:56.198 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.194+0000 7f93f0afd700 1 --2- 172.21.15.77:0/275018041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93ec100ea0 0x7f93ec19b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:18:56.198 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.194+0000 7f93f12fe700 1 --2- 172.21.15.77:0/275018041 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93ec0ffcb0 0x7f93ec069ab0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f93e8009fd0 tx=0x7f93e8009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:56.198 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.194+0000 7f93e27fc700 1 -- 172.21.15.77:0/275018041 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e801d070 con 0x7f93ec0ffcb0 2023-12-01T01:18:56.198 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.194+0000 7f93f3562700 1 -- 172.21.15.77:0/275018041 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93ec19eb40 con 0x7f93ec0ffcb0 2023-12-01T01:18:56.198 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.194+0000 7f93e27fc700 1 -- 172.21.15.77:0/275018041 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93e8005030 con 0x7f93ec0ffcb0 2023-12-01T01:18:56.198 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.194+0000 7f93e27fc700 1 -- 172.21.15.77:0/275018041 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e8004a90 con 0x7f93ec0ffcb0 2023-12-01T01:18:56.198 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.195+0000 7f93f3562700 1 -- 172.21.15.77:0/275018041 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93ec19efc0 con 0x7f93ec0ffcb0 2023-12-01T01:18:56.200 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.196+0000 7f93e27fc700 1 -- 172.21.15.77:0/275018041 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f93e80051a0 con 0x7f93ec0ffcb0 2023-12-01T01:18:56.200 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.197+0000 7f93e27fc700 1 --2- 172.21.15.77:0/275018041 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f93d80a0520 0x7f93d80a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.200 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.197+0000 7f93f0afd700 1 --2- 172.21.15.77:0/275018041 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f93d80a0520 0x7f93d80a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.200 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.197+0000 7f93e27fc700 1 -- 172.21.15.77:0/275018041 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f93e80cc610 con 0x7f93ec0ffcb0 2023-12-01T01:18:56.201 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.197+0000 7f93f0afd700 1 --2- 172.21.15.77:0/275018041 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f93d80a0520 0x7f93d80a29e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f93dc011490 tx=0x7f93dc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:56.201 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.198+0000 7f93ff6f2b80 1 Processor -- start 2023-12-01T01:18:56.201 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.198+0000 7f93ff6f2b80 1 -- start start 2023-12-01T01:18:56.201 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.198+0000 7f93ff6f2b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d278aa0f0 0x556d278aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.201 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.198+0000 7f93ff6f2b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d278aaa70 0x556d278a8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.201 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.198+0000 7f93ff6f2b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d278a93f0 0x556d278a9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.201 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.198+0000 7f93ff6f2b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d27800a10 con 0x556d278a93f0 2023-12-01T01:18:56.201 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.198+0000 7f93ff6f2b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d277fe3d0 con 0x556d278aaa70 2023-12-01T01:18:56.201 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.198+0000 7f93ff6f2b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d277f9cc0 con 0x556d278aa0f0 2023-12-01T01:18:56.201 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.198+0000 7f93f12fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d278aa0f0 0x556d278aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.201 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.198+0000 7f93f1aff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d278a93f0 0x556d278a9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.201 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.198+0000 7f93f12fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d278aa0f0 0x556d278aa530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:46160/0 (socket says 172.21.15.77:46160) 2023-12-01T01:18:56.201 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.198+0000 7f93f12fe700 1 -- 172.21.15.77:0/4080629952 learned_addr learned my addr 172.21.15.77:0/4080629952 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:56.201 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.198+0000 7f93f0afd700 1 --2- 172.21.15.77:0/4080629952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d278aaa70 0x556d278a8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.201 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.199+0000 7f93f12fe700 1 -- 172.21.15.77:0/4080629952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d278aaa70 msgr2=0x556d278a8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:56.202 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.199+0000 7f93f12fe700 1 --2- 172.21.15.77:0/4080629952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d278aaa70 0x556d278a8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.202 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.199+0000 7f93f12fe700 1 -- 172.21.15.77:0/4080629952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d278a93f0 msgr2=0x556d278a9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:56.202 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.199+0000 7f93f12fe700 1 --2- 172.21.15.77:0/4080629952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d278a93f0 0x556d278a9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.202 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.199+0000 7f93f12fe700 1 -- 172.21.15.77:0/4080629952 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d27627de0 con 0x556d278aa0f0 2023-12-01T01:18:56.202 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.199+0000 7f93f1aff700 1 --2- 172.21.15.77:0/4080629952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d278a93f0 0x556d278a9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:18:56.202 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.199+0000 7f93f12fe700 1 --2- 172.21.15.77:0/4080629952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d278aa0f0 0x556d278aa530 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f93e80964a0 tx=0x7f93e809a770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e95478e9ac6f87d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:56.202 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.199+0000 7f93d77fe700 1 -- 172.21.15.77:0/4080629952 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e80a1f80 con 0x556d278aa0f0 2023-12-01T01:18:56.202 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.199+0000 7f93d77fe700 1 -- 172.21.15.77:0/4080629952 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93e8098340 con 0x556d278aa0f0 2023-12-01T01:18:56.202 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.199+0000 7f93d77fe700 1 -- 172.21.15.77:0/4080629952 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e80a1af0 con 0x556d278aa0f0 2023-12-01T01:18:56.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.200+0000 7f93ff6f2b80 1 -- 172.21.15.77:0/4080629952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d278aa0f0 msgr2=0x556d278aa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:56.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.200+0000 7f93ff6f2b80 1 --2- 172.21.15.77:0/4080629952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d278aa0f0 0x556d278aa530 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f93e80964a0 tx=0x7f93e809a770 comp rx=0 tx=0).stop 2023-12-01T01:18:56.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.200+0000 7f93ff6f2b80 1 -- 172.21.15.77:0/4080629952 shutdown_connections 2023-12-01T01:18:56.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.200+0000 7f93ff6f2b80 1 --2- 172.21.15.77:0/4080629952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d278a93f0 0x556d278a9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.200+0000 7f93ff6f2b80 1 --2- 172.21.15.77:0/4080629952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d278aa0f0 0x556d278aa530 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.200+0000 7f93ff6f2b80 1 --2- 172.21.15.77:0/4080629952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d278aaa70 0x556d278a8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.200+0000 7f93ff6f2b80 1 -- 172.21.15.77:0/4080629952 >> 172.21.15.77:0/4080629952 conn(0x556d278080d0 msgr2=0x556d278a3940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:56.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.200+0000 7f93ff6f2b80 1 -- 172.21.15.77:0/4080629952 shutdown_connections 2023-12-01T01:18:56.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.200+0000 7f93ff6f2b80 1 -- 172.21.15.77:0/4080629952 wait complete. 2023-12-01T01:18:56.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.200+0000 7f93ff6f2b80 1 -- start start 2023-12-01T01:18:56.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.200+0000 7f93ff6f2b80 1 Processor -- start 2023-12-01T01:18:56.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.201+0000 7f93ff6f2b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d278a93f0 0x556d278b2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.201+0000 7f93ff6f2b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d278aa0f0 0x556d278b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.201+0000 7f93ff6f2b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d278aaa70 0x556d278b9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.201+0000 7f93ff6f2b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d27796510 con 0x556d278aaa70 2023-12-01T01:18:56.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.201+0000 7f93ff6f2b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d277961b0 con 0x556d278aa0f0 2023-12-01T01:18:56.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.201+0000 7f93ff6f2b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d2767d920 con 0x556d278a93f0 2023-12-01T01:18:56.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.201+0000 7f93ff6f2b80 10 client.0 _ll_register_callbacks cb 0x556d276e3a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:18:56.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.201+0000 7f93f12fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d278a93f0 0x556d278b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.201+0000 7f93f1aff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d278aaa70 0x556d278b9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.201+0000 7f93f0afd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d278aa0f0 0x556d278b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.201+0000 7f93f12fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d278a93f0 0x556d278b2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:46166/0 (socket says 172.21.15.77:46166) 2023-12-01T01:18:56.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.201+0000 7f93f1aff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d278aaa70 0x556d278b9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59982/0 (socket says 172.21.15.77:59982) 2023-12-01T01:18:56.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.201+0000 7f93f12fe700 1 -- 172.21.15.77:0/704384159 learned_addr learned my addr 172.21.15.77:0/704384159 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:56.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.201+0000 7f93f12fe700 1 -- 172.21.15.77:0/704384159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d278aa0f0 msgr2=0x556d278b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:56.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.201+0000 7f93f12fe700 1 --2- 172.21.15.77:0/704384159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d278aa0f0 0x556d278b2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.201+0000 7f93f12fe700 1 -- 172.21.15.77:0/704384159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d278aaa70 msgr2=0x556d278b9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:56.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.201+0000 7f93f12fe700 1 --2- 172.21.15.77:0/704384159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d278aaa70 0x556d278b9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.201+0000 7f93f12fe700 1 -- 172.21.15.77:0/704384159 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e4011230 con 0x556d278a93f0 2023-12-01T01:18:56.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.201+0000 7f93f1aff700 1 --2- 172.21.15.77:0/704384159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d278aaa70 0x556d278b9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:18:56.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.201+0000 7f93f12fe700 1 --2- 172.21.15.77:0/704384159 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d278a93f0 0x556d278b2690 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f93e80a1a50 tx=0x7f93e8099b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=883688684792fd53 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:56.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.201+0000 7f93d4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2023-12-01T01:18:56.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.202+0000 7f93d4ff9700 1 -- 172.21.15.77:0/704384159 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e80b2270 con 0x556d278a93f0 2023-12-01T01:18:56.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.202+0000 7f93d4ff9700 1 -- 172.21.15.77:0/704384159 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f93e80b23d0 con 0x556d278a93f0 2023-12-01T01:18:56.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.202+0000 7f93ff6f2b80 1 -- 172.21.15.77:0/704384159 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556d27824d10 con 0x556d278a93f0 2023-12-01T01:18:56.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.202+0000 7f93d4ff9700 1 -- 172.21.15.77:0/704384159 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93e80a1af0 con 0x556d278a93f0 2023-12-01T01:18:56.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.202+0000 7f93ff6f2b80 10 client.25351 fetch_fsmap learned FSMap version 29 2023-12-01T01:18:56.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.202+0000 7f93ff6f2b80 1 -- 172.21.15.77:0/704384159 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f93ec19efc0 con 0x556d278a93f0 2023-12-01T01:18:56.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.202+0000 7f93d4ff9700 1 -- 172.21.15.77:0/704384159 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556d27824d10 con 0x556d278a93f0 2023-12-01T01:18:56.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.202+0000 7f93d4ff9700 1 -- 172.21.15.77:0/704384159 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f93e8061030 con 0x556d278a93f0 2023-12-01T01:18:56.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.202+0000 7f93ff6f2b80 10 client.25351 fetch_fsmap finished waiting for FSMap version 29 2023-12-01T01:18:56.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.202+0000 7f93ff6f2b80 10 client.25351 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2023-12-01T01:18:56.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.202+0000 7f93ff6f2b80 20 client.25351 populate_metadata read hostname 'smithi077' 2023-12-01T01:18:56.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.202+0000 7f93ff6f2b80 1 --2- 172.21.15.77:0/704384159 >> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] conn(0x556d278bdf20 0x556d278c0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.202+0000 7f93ff6f2b80 4 client.25351 mds_command: new command op to 24473 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:18:56.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.202+0000 7f93ff6f2b80 1 -- 172.21.15.77:0/704384159 --> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556d27877e10 con 0x556d278bdf20 2023-12-01T01:18:56.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.202+0000 7f93f0afd700 1 --2- 172.21.15.77:0/704384159 >> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] conn(0x556d278bdf20 0x556d278c0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.203+0000 7f93d4ff9700 10 client.25351 ms_handle_connect on v2:172.21.15.137:6836/3518881867 2023-12-01T01:18:56.210 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.203+0000 7f93f0afd700 1 --2- 172.21.15.77:0/704384159 >> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] conn(0x556d278bdf20 0x556d278c0310 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=19399564c21839b9 server_cookie=d999078c351c83ce in_seq=0 out_seq=0 2023-12-01T01:18:56.210 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.203+0000 7f93d4ff9700 1 -- 172.21.15.77:0/704384159 <== mds.3 v2:172.21.15.137:6836/3518881867 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x556d27877e10 con 0x556d278bdf20 2023-12-01T01:18:56.210 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.203+0000 7f93d4ff9700 10 client.25351 handle_command_reply: tid=0 2023-12-01T01:18:56.210 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.203+0000 7f93ff6f2b80 1 client.25351 shutdown 2023-12-01T01:18:56.210 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.204+0000 7f93ff6f2b80 1 -- 172.21.15.77:0/704384159 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d278a93f0 msgr2=0x556d278b2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:56.210 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.204+0000 7f93ff6f2b80 1 --2- 172.21.15.77:0/704384159 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d278a93f0 0x556d278b2690 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f93e80a1a50 tx=0x7f93e8099b70 comp rx=0 tx=0).stop 2023-12-01T01:18:56.210 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.204+0000 7f93ff6f2b80 1 -- 172.21.15.77:0/704384159 shutdown_connections 2023-12-01T01:18:56.210 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.204+0000 7f93ff6f2b80 1 --2- 172.21.15.77:0/704384159 >> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] conn(0x556d278bdf20 0x556d278c0310 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.210 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.204+0000 7f93ff6f2b80 1 --2- 172.21.15.77:0/704384159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d278aaa70 0x556d278b9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.210 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.204+0000 7f93d4ff9700 0 client.25351 ms_handle_reset on v2:172.21.15.137:6836/3518881867 2023-12-01T01:18:56.210 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.204+0000 7f93ff6f2b80 1 --2- 172.21.15.77:0/704384159 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d278a93f0 0x556d278b2690 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.210 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.204+0000 7f93ff6f2b80 1 --2- 172.21.15.77:0/704384159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d278aa0f0 0x556d278b2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.210 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.204+0000 7f93ff6f2b80 1 -- 172.21.15.77:0/704384159 >> 172.21.15.77:0/704384159 conn(0x556d278080d0 msgr2=0x556d278a30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:56.210 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.204+0000 7f93ff6f2b80 1 -- 172.21.15.77:0/704384159 shutdown_connections 2023-12-01T01:18:56.211 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.204+0000 7f93ff6f2b80 1 -- 172.21.15.77:0/704384159 wait complete. 2023-12-01T01:18:56.211 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.204+0000 7f93ff6f2b80 20 client.25351 trim_cache size 0 max 16384 2023-12-01T01:18:56.225 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.218+0000 7f93ff6f2b80 1 Processor -- start 2023-12-01T01:18:56.225 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.218+0000 7f93ff6f2b80 1 -- start start 2023-12-01T01:18:56.225 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.219+0000 7f93ff6f2b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d278a93f0 0x556d278a8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.225 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.219+0000 7f93ff6f2b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d278aa0f0 0x556d278b9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.225 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.219+0000 7f93ff6f2b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d278aaa70 0x556d278b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.225 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.219+0000 7f93ff6f2b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d278b2260 con 0x556d278a93f0 2023-12-01T01:18:56.226 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.219+0000 7f93ff6f2b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d278a4320 con 0x556d278aaa70 2023-12-01T01:18:56.226 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.219+0000 7f93ff6f2b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d278a4700 con 0x556d278aa0f0 2023-12-01T01:18:56.226 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.219+0000 7f93f12fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d278a93f0 0x556d278a8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.226 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.219+0000 7f93f12fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d278a93f0 0x556d278a8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59986/0 (socket says 172.21.15.77:59986) 2023-12-01T01:18:56.226 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.219+0000 7f93f12fe700 1 -- 172.21.15.77:0/3408922689 learned_addr learned my addr 172.21.15.77:0/3408922689 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:56.226 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.219+0000 7f93f0afd700 1 --2- 172.21.15.77:0/3408922689 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d278aa0f0 0x556d278b9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.226 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.219+0000 7f93f1aff700 1 --2- 172.21.15.77:0/3408922689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d278aaa70 0x556d278b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.226 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.219+0000 7f93f12fe700 1 -- 172.21.15.77:0/3408922689 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d278aa0f0 msgr2=0x556d278b9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:56.226 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.219+0000 7f93f12fe700 1 --2- 172.21.15.77:0/3408922689 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d278aa0f0 0x556d278b9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.226 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.219+0000 7f93f12fe700 1 -- 172.21.15.77:0/3408922689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d278aaa70 msgr2=0x556d278b07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:56.226 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.219+0000 7f93f12fe700 1 --2- 172.21.15.77:0/3408922689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d278aaa70 0x556d278b07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.226 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.219+0000 7f93f12fe700 1 -- 172.21.15.77:0/3408922689 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d27610c60 con 0x556d278a93f0 2023-12-01T01:18:56.226 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.219+0000 7f93f1aff700 1 --2- 172.21.15.77:0/3408922689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d278aaa70 0x556d278b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:18:56.226 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.219+0000 7f93f12fe700 1 --2- 172.21.15.77:0/3408922689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d278a93f0 0x556d278a8e60 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f93e8099f30 tx=0x7f93e809df50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad1e582e91e807b5 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:56.226 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.219+0000 7f93d7fff700 1 -- 172.21.15.77:0/3408922689 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e8094c00 con 0x556d278a93f0 2023-12-01T01:18:56.227 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.219+0000 7f93d7fff700 1 -- 172.21.15.77:0/3408922689 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93e8004bf0 con 0x556d278a93f0 2023-12-01T01:18:56.227 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.219+0000 7f93d7fff700 1 -- 172.21.15.77:0/3408922689 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e80a1d50 con 0x556d278a93f0 2023-12-01T01:18:56.227 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.220+0000 7f93ff6f2b80 1 -- 172.21.15.77:0/3408922689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d278a93f0 msgr2=0x556d278a8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:56.227 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.220+0000 7f93ff6f2b80 1 --2- 172.21.15.77:0/3408922689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d278a93f0 0x556d278a8e60 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f93e8099f30 tx=0x7f93e809df50 comp rx=0 tx=0).stop 2023-12-01T01:18:56.227 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.220+0000 7f93ff6f2b80 1 -- 172.21.15.77:0/3408922689 shutdown_connections 2023-12-01T01:18:56.227 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.220+0000 7f93ff6f2b80 1 --2- 172.21.15.77:0/3408922689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d278aaa70 0x556d278b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.227 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.220+0000 7f93ff6f2b80 1 --2- 172.21.15.77:0/3408922689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d278a93f0 0x556d278a8e60 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.227 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.220+0000 7f93ff6f2b80 1 --2- 172.21.15.77:0/3408922689 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d278aa0f0 0x556d278b9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.227 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.220+0000 7f93ff6f2b80 1 -- 172.21.15.77:0/3408922689 >> 172.21.15.77:0/3408922689 conn(0x556d278080d0 msgr2=0x556d278a37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:56.227 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.220+0000 7f93ff6f2b80 1 -- 172.21.15.77:0/3408922689 shutdown_connections 2023-12-01T01:18:56.227 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.220+0000 7f93ff6f2b80 1 -- 172.21.15.77:0/3408922689 wait complete. 2023-12-01T01:18:56.227 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.220+0000 7f93ff6f2b80 1 -- start start 2023-12-01T01:18:56.227 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.221+0000 7f93ff6f2b80 1 Processor -- start 2023-12-01T01:18:56.227 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.221+0000 7f93ff6f2b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d278a93f0 0x556d278b7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.227 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.221+0000 7f93ff6f2b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d278aa0f0 0x556d278b7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.228 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.221+0000 7f93ff6f2b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d278aaa70 0x556d278b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.228 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.221+0000 7f93ff6f2b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d278b8fc0 con 0x556d278a93f0 2023-12-01T01:18:56.228 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.221+0000 7f93ff6f2b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d278b9130 con 0x556d278aaa70 2023-12-01T01:18:56.228 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.221+0000 7f93ff6f2b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d278b92f0 con 0x556d278aa0f0 2023-12-01T01:18:56.228 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.221+0000 7f93ff6f2b80 10 client.0 _ll_register_callbacks cb 0x7f93e4009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:18:56.228 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.221+0000 7f93f12fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d278a93f0 0x556d278b7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.228 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.221+0000 7f93f12fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d278a93f0 0x556d278b7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59992/0 (socket says 172.21.15.77:59992) 2023-12-01T01:18:56.228 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.221+0000 7f93f12fe700 1 -- 172.21.15.77:0/3953136526 learned_addr learned my addr 172.21.15.77:0/3953136526 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:56.228 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.221+0000 7f93f0afd700 1 --2- 172.21.15.77:0/3953136526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d278aa0f0 0x556d278b7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.228 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.221+0000 7f93f1aff700 1 --2- 172.21.15.77:0/3953136526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d278aaa70 0x556d278b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.228 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.221+0000 7f93f12fe700 1 -- 172.21.15.77:0/3953136526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d278aa0f0 msgr2=0x556d278b7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:56.228 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.221+0000 7f93f12fe700 1 --2- 172.21.15.77:0/3953136526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d278aa0f0 0x556d278b7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.228 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.221+0000 7f93f12fe700 1 -- 172.21.15.77:0/3953136526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d278aaa70 msgr2=0x556d278b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:56.228 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.221+0000 7f93f12fe700 1 --2- 172.21.15.77:0/3953136526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d278aaa70 0x556d278b8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.229 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.222+0000 7f93f12fe700 1 -- 172.21.15.77:0/3953136526 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93ec19efc0 con 0x556d278a93f0 2023-12-01T01:18:56.229 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.222+0000 7f93f0afd700 1 --2- 172.21.15.77:0/3953136526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d278aa0f0 0x556d278b7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:18:56.229 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.222+0000 7f93f12fe700 1 --2- 172.21.15.77:0/3953136526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d278a93f0 0x556d278b7530 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f93e80964a0 tx=0x7f93e809a770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e6d110ddb477fb server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:56.229 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.222+0000 7f93d4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:18:56.229 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.222+0000 7f93d4ff9700 1 -- 172.21.15.77:0/3953136526 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e80b2240 con 0x556d278a93f0 2023-12-01T01:18:56.229 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.222+0000 7f93d4ff9700 1 -- 172.21.15.77:0/3953136526 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f93e800fc70 con 0x556d278a93f0 2023-12-01T01:18:56.229 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.222+0000 7f93ff6f2b80 1 -- 172.21.15.77:0/3953136526 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f93e8098340 con 0x556d278a93f0 2023-12-01T01:18:56.229 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.222+0000 7f93d4ff9700 1 -- 172.21.15.77:0/3953136526 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93e8004d60 con 0x556d278a93f0 2023-12-01T01:18:56.229 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.222+0000 7f93ff6f2b80 10 client.15762 fetch_fsmap learned FSMap version 29 2023-12-01T01:18:56.229 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.222+0000 7f93ff6f2b80 1 -- 172.21.15.77:0/3953136526 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x556d2764e250 con 0x556d278a93f0 2023-12-01T01:18:56.229 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.222+0000 7f93d4ff9700 1 -- 172.21.15.77:0/3953136526 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93e8098340 con 0x556d278a93f0 2023-12-01T01:18:56.229 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.222+0000 7f93d4ff9700 1 -- 172.21.15.77:0/3953136526 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f93e8061070 con 0x556d278a93f0 2023-12-01T01:18:56.229 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.222+0000 7f93ff6f2b80 10 client.15762 fetch_fsmap finished waiting for FSMap version 29 2023-12-01T01:18:56.229 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.222+0000 7f93ff6f2b80 10 client.15762 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2023-12-01T01:18:56.229 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.222+0000 7f93ff6f2b80 20 client.15762 populate_metadata read hostname 'smithi077' 2023-12-01T01:18:56.229 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.222+0000 7f93ff6f2b80 1 --2- 172.21.15.77:0/3953136526 >> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] conn(0x556d2790fe60 0x556d27912250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.230 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.222+0000 7f93ff6f2b80 4 client.15762 mds_command: new command op to 24473 tid=0[{"prefix": "damage ls"}] 2023-12-01T01:18:56.230 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.222+0000 7f93ff6f2b80 1 -- 172.21.15.77:0/3953136526 --> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556d277f2d40 con 0x556d2790fe60 2023-12-01T01:18:56.230 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.223+0000 7f93f0afd700 1 --2- 172.21.15.77:0/3953136526 >> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] conn(0x556d2790fe60 0x556d27912250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.230 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.223+0000 7f93d4ff9700 10 client.15762 ms_handle_connect on v2:172.21.15.137:6836/3518881867 2023-12-01T01:18:56.230 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.223+0000 7f93f0afd700 1 --2- 172.21.15.77:0/3953136526 >> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] conn(0x556d2790fe60 0x556d27912250 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a5fad90dd7975dd8 server_cookie=613ab0a0235a9a3a in_seq=0 out_seq=0 2023-12-01T01:18:56.230 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.223+0000 7f93d4ff9700 1 -- 172.21.15.77:0/3953136526 <== mds.3 v2:172.21.15.137:6836/3518881867 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556d277f2d40 con 0x556d2790fe60 2023-12-01T01:18:56.230 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.223+0000 7f93d4ff9700 10 client.15762 handle_command_reply: tid=0 2023-12-01T01:18:56.230 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.223+0000 7f93ff6f2b80 1 client.15762 shutdown 2023-12-01T01:18:56.230 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.223+0000 7f93ff6f2b80 1 -- 172.21.15.77:0/3953136526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d278a93f0 msgr2=0x556d278b7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:56.230 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.223+0000 7f93ff6f2b80 1 --2- 172.21.15.77:0/3953136526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d278a93f0 0x556d278b7530 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f93e80964a0 tx=0x7f93e809a770 comp rx=0 tx=0).stop 2023-12-01T01:18:56.230 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.223+0000 7f93ff6f2b80 1 -- 172.21.15.77:0/3953136526 shutdown_connections 2023-12-01T01:18:56.230 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.223+0000 7f93ff6f2b80 1 --2- 172.21.15.77:0/3953136526 >> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] conn(0x556d2790fe60 0x556d27912250 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.230 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.223+0000 7f93ff6f2b80 1 --2- 172.21.15.77:0/3953136526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d278aaa70 0x556d278b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.230 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.223+0000 7f93d4ff9700 0 client.15762 ms_handle_reset on v2:172.21.15.137:6836/3518881867 2023-12-01T01:18:56.231 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.223+0000 7f93ff6f2b80 1 --2- 172.21.15.77:0/3953136526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d278a93f0 0x556d278b7530 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.231 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.223+0000 7f93ff6f2b80 1 --2- 172.21.15.77:0/3953136526 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556d278aa0f0 0x556d278b7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.231 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.223+0000 7f93ff6f2b80 1 -- 172.21.15.77:0/3953136526 >> 172.21.15.77:0/3953136526 conn(0x556d278080d0 msgr2=0x556d278b2440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:56.231 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.223+0000 7f93ff6f2b80 1 -- 172.21.15.77:0/3953136526 shutdown_connections 2023-12-01T01:18:56.231 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.223+0000 7f93ff6f2b80 1 -- 172.21.15.77:0/3953136526 wait complete. 2023-12-01T01:18:56.231 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.224+0000 7f93ff6f2b80 20 client.15762 trim_cache size 0 max 16384 2023-12-01T01:18:56.231 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2023-12-01T01:18:56.232 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.224+0000 7f93f3562700 1 -- 172.21.15.77:0/275018041 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f93d80a0520 msgr2=0x7f93d80a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:56.232 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.224+0000 7f93f3562700 1 --2- 172.21.15.77:0/275018041 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f93d80a0520 0x7f93d80a29e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f93dc011490 tx=0x7f93dc009260 comp rx=0 tx=0).stop 2023-12-01T01:18:56.232 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.224+0000 7f93f3562700 1 -- 172.21.15.77:0/275018041 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93ec0ffcb0 msgr2=0x7f93ec069ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:56.232 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.224+0000 7f93f3562700 1 --2- 172.21.15.77:0/275018041 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93ec0ffcb0 0x7f93ec069ab0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f93e8009fd0 tx=0x7f93e8009580 comp rx=0 tx=0).stop 2023-12-01T01:18:56.232 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.224+0000 7f93f3562700 1 -- 172.21.15.77:0/275018041 shutdown_connections 2023-12-01T01:18:56.232 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.224+0000 7f93f3562700 1 --2- 172.21.15.77:0/275018041 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f93d80a0520 0x7f93d80a29e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.232 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.224+0000 7f93f3562700 1 --2- 172.21.15.77:0/275018041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93ec101840 0x7f93ec19ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.232 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.224+0000 7f93f3562700 1 --2- 172.21.15.77:0/275018041 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93ec0ffcb0 0x7f93ec069ab0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.232 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.224+0000 7f93f3562700 1 --2- 172.21.15.77:0/275018041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93ec100ea0 0x7f93ec19b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.233 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.224+0000 7f93f3562700 1 -- 172.21.15.77:0/275018041 >> 172.21.15.77:0/275018041 conn(0x7f93ec0fb220 msgr2=0x7f93ec0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:56.233 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.224+0000 7f93f3562700 1 -- 172.21.15.77:0/275018041 shutdown_connections 2023-12-01T01:18:56.233 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.224+0000 7f93f3562700 1 -- 172.21.15.77:0/275018041 wait complete. 2023-12-01T01:18:56.237 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-12-01T01:18:56.406 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.404+0000 7f1369ccb700 1 -- 172.21.15.77:0/348674187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1364101810 msgr2=0x7f13641080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:56.406 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.404+0000 7f1369ccb700 1 --2- 172.21.15.77:0/348674187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1364101810 0x7f13641080c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f135c00b0d0 tx=0x7f135c00b3e0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.406 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.404+0000 7f1369ccb700 1 -- 172.21.15.77:0/348674187 shutdown_connections 2023-12-01T01:18:56.406 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.404+0000 7f1369ccb700 1 --2- 172.21.15.77:0/348674187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1364101810 0x7f13641080c0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.406 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.404+0000 7f1369ccb700 1 --2- 172.21.15.77:0/348674187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13640ffc80 0x7f13641000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.406 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.404+0000 7f1369ccb700 1 --2- 172.21.15.77:0/348674187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1364100e70 0x7f13641012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.406 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.404+0000 7f1369ccb700 1 -- 172.21.15.77:0/348674187 >> 172.21.15.77:0/348674187 conn(0x7f13640fb1f0 msgr2=0x7f13640fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:56.408 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.404+0000 7f1369ccb700 1 -- 172.21.15.77:0/348674187 shutdown_connections 2023-12-01T01:18:56.408 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.404+0000 7f1369ccb700 1 -- 172.21.15.77:0/348674187 wait complete. 2023-12-01T01:18:56.408 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.405+0000 7f1369ccb700 1 Processor -- start 2023-12-01T01:18:56.408 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.405+0000 7f1369ccb700 1 -- start start 2023-12-01T01:18:56.408 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.405+0000 7f1369ccb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13640ffc80 0x7f1364068a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.408 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.405+0000 7f1369ccb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1364100e70 0x7f136419dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.409 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.405+0000 7f1369ccb700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1364101810 0x7f13641a0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.409 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.405+0000 7f1369ccb700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13641a08f0 con 0x7f1364100e70 2023-12-01T01:18:56.409 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.405+0000 7f1369ccb700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13641a0a60 con 0x7f13640ffc80 2023-12-01T01:18:56.409 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.405+0000 7f1369ccb700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13641a0c20 con 0x7f1364101810 2023-12-01T01:18:56.409 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.405+0000 7f1362ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1364100e70 0x7f136419dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.409 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.405+0000 7f1362ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1364100e70 0x7f136419dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60008/0 (socket says 172.21.15.77:60008) 2023-12-01T01:18:56.409 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.405+0000 7f1362ffd700 1 -- 172.21.15.77:0/1224106380 learned_addr learned my addr 172.21.15.77:0/1224106380 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:56.409 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.405+0000 7f13637fe700 1 --2- 172.21.15.77:0/1224106380 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13640ffc80 0x7f1364068a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.409 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.405+0000 7f1363fff700 1 --2- 172.21.15.77:0/1224106380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1364101810 0x7f13641a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.409 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.405+0000 7f1362ffd700 1 -- 172.21.15.77:0/1224106380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1364101810 msgr2=0x7f13641a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:56.410 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.405+0000 7f1362ffd700 1 --2- 172.21.15.77:0/1224106380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1364101810 0x7f13641a0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.410 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.405+0000 7f1362ffd700 1 -- 172.21.15.77:0/1224106380 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13640ffc80 msgr2=0x7f1364068a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:56.410 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.405+0000 7f1362ffd700 1 --2- 172.21.15.77:0/1224106380 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13640ffc80 0x7f1364068a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.410 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.405+0000 7f1362ffd700 1 -- 172.21.15.77:0/1224106380 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f135c009d00 con 0x7f1364100e70 2023-12-01T01:18:56.410 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.406+0000 7f1362ffd700 1 --2- 172.21.15.77:0/1224106380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1364100e70 0x7f136419dcf0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f135800e9a0 tx=0x7f135800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:56.410 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.406+0000 7f1360ff9700 1 -- 172.21.15.77:0/1224106380 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1358009790 con 0x7f1364100e70 2023-12-01T01:18:56.410 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.406+0000 7f1369ccb700 1 -- 172.21.15.77:0/1224106380 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13641a0f00 con 0x7f1364100e70 2023-12-01T01:18:56.411 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.406+0000 7f1360ff9700 1 -- 172.21.15.77:0/1224106380 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1358010430 con 0x7f1364100e70 2023-12-01T01:18:56.411 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.406+0000 7f1360ff9700 1 -- 172.21.15.77:0/1224106380 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1358018680 con 0x7f1364100e70 2023-12-01T01:18:56.411 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.406+0000 7f1369ccb700 1 -- 172.21.15.77:0/1224106380 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13641a1380 con 0x7f1364100e70 2023-12-01T01:18:56.418 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.409+0000 7f1360ff9700 1 -- 172.21.15.77:0/1224106380 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f13580105a0 con 0x7f1364100e70 2023-12-01T01:18:56.418 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.409+0000 7f1360ff9700 1 --2- 172.21.15.77:0/1224106380 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f134c0a0520 0x7f134c0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.418 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.409+0000 7f1360ff9700 1 -- 172.21.15.77:0/1224106380 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f1358014070 con 0x7f1364100e70 2023-12-01T01:18:56.418 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.409+0000 7f13637fe700 1 --2- 172.21.15.77:0/1224106380 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f134c0a0520 0x7f134c0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.418 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.410+0000 7f13637fe700 1 --2- 172.21.15.77:0/1224106380 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f134c0a0520 0x7f134c0a29e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1364100cd0 tx=0x7f1354009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:56.418 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.410+0000 7f1375e5bb80 1 Processor -- start 2023-12-01T01:18:56.418 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.410+0000 7f1375e5bb80 1 -- start start 2023-12-01T01:18:56.418 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.410+0000 7f1375e5bb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564345c260f0 0x564345c26530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.418 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.410+0000 7f1375e5bb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564345c26a70 0x564345c24e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.418 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.410+0000 7f1375e5bb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564345c253f0 0x564345c25870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.418 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.410+0000 7f1375e5bb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x564345b7ca10 con 0x564345c253f0 2023-12-01T01:18:56.419 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.410+0000 7f1375e5bb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564345b7a3d0 con 0x564345c260f0 2023-12-01T01:18:56.419 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.410+0000 7f1375e5bb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x564345b75cc0 con 0x564345c26a70 2023-12-01T01:18:56.419 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.410+0000 7f1363fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564345c253f0 0x564345c25870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.419 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.410+0000 7f1363fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564345c253f0 0x564345c25870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60018/0 (socket says 172.21.15.77:60018) 2023-12-01T01:18:56.419 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.410+0000 7f1363fff700 1 -- 172.21.15.77:0/4090460355 learned_addr learned my addr 172.21.15.77:0/4090460355 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:56.419 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.410+0000 7f13637fe700 1 --2- 172.21.15.77:0/4090460355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564345c260f0 0x564345c26530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.419 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.410+0000 7f1362ffd700 1 --2- 172.21.15.77:0/4090460355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564345c26a70 0x564345c24e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.419 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.410+0000 7f13637fe700 1 -- 172.21.15.77:0/4090460355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564345c26a70 msgr2=0x564345c24e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:56.419 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.410+0000 7f13637fe700 1 --2- 172.21.15.77:0/4090460355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564345c26a70 0x564345c24e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.419 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.410+0000 7f13637fe700 1 -- 172.21.15.77:0/4090460355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564345c253f0 msgr2=0x564345c25870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:56.419 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.410+0000 7f13637fe700 1 --2- 172.21.15.77:0/4090460355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564345c253f0 0x564345c25870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.419 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.410+0000 7f13637fe700 1 -- 172.21.15.77:0/4090460355 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643459a3de0 con 0x564345c260f0 2023-12-01T01:18:56.419 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.411+0000 7f13637fe700 1 --2- 172.21.15.77:0/4090460355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564345c260f0 0x564345c26530 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1354011c80 tx=0x7f1354014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31eb6628e353395b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:56.420 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.411+0000 7f1349ffb700 1 -- 172.21.15.77:0/4090460355 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1354004220 con 0x564345c260f0 2023-12-01T01:18:56.420 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.411+0000 7f1349ffb700 1 -- 172.21.15.77:0/4090460355 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1354021070 con 0x564345c260f0 2023-12-01T01:18:56.420 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.411+0000 7f1349ffb700 1 -- 172.21.15.77:0/4090460355 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1354003b50 con 0x564345c260f0 2023-12-01T01:18:56.420 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.411+0000 7f1375e5bb80 1 -- 172.21.15.77:0/4090460355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564345c260f0 msgr2=0x564345c26530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:56.420 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.411+0000 7f1375e5bb80 1 --2- 172.21.15.77:0/4090460355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564345c260f0 0x564345c26530 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1354011c80 tx=0x7f1354014000 comp rx=0 tx=0).stop 2023-12-01T01:18:56.420 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.411+0000 7f1375e5bb80 1 -- 172.21.15.77:0/4090460355 shutdown_connections 2023-12-01T01:18:56.420 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.411+0000 7f1375e5bb80 1 --2- 172.21.15.77:0/4090460355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564345c253f0 0x564345c25870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.420 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.411+0000 7f1375e5bb80 1 --2- 172.21.15.77:0/4090460355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564345c260f0 0x564345c26530 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.420 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.411+0000 7f1375e5bb80 1 --2- 172.21.15.77:0/4090460355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564345c26a70 0x564345c24e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.420 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.411+0000 7f1375e5bb80 1 -- 172.21.15.77:0/4090460355 >> 172.21.15.77:0/4090460355 conn(0x564345b840d0 msgr2=0x564345c1f940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:56.420 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.411+0000 7f1375e5bb80 1 -- 172.21.15.77:0/4090460355 shutdown_connections 2023-12-01T01:18:56.420 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.411+0000 7f1375e5bb80 1 -- 172.21.15.77:0/4090460355 wait complete. 2023-12-01T01:18:56.420 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.412+0000 7f1375e5bb80 1 -- start start 2023-12-01T01:18:56.420 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.412+0000 7f1375e5bb80 1 Processor -- start 2023-12-01T01:18:56.421 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.412+0000 7f1375e5bb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564345c253f0 0x564345c2e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.421 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.412+0000 7f1375e5bb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564345c260f0 0x564345c2ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.421 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.412+0000 7f1375e5bb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564345c26a70 0x564345c35f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.421 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.412+0000 7f1375e5bb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x564345b12510 con 0x564345c26a70 2023-12-01T01:18:56.421 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.412+0000 7f1375e5bb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564345b121b0 con 0x564345c253f0 2023-12-01T01:18:56.421 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.412+0000 7f1375e5bb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643459f9920 con 0x564345c260f0 2023-12-01T01:18:56.421 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.412+0000 7f1375e5bb80 10 client.0 _ll_register_callbacks cb 0x564345a5fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:18:56.421 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.412+0000 7f13637fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564345c253f0 0x564345c2e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.421 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.412+0000 7f1363fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564345c26a70 0x564345c35f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.421 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.412+0000 7f13637fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564345c253f0 0x564345c2e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:38942/0 (socket says 172.21.15.77:38942) 2023-12-01T01:18:56.421 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.412+0000 7f13637fe700 1 -- 172.21.15.77:0/282834689 learned_addr learned my addr 172.21.15.77:0/282834689 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:56.421 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.412+0000 7f1362ffd700 1 --2- 172.21.15.77:0/282834689 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564345c260f0 0x564345c2ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.421 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.413+0000 7f1363fff700 1 -- 172.21.15.77:0/282834689 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564345c260f0 msgr2=0x564345c2ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:56.421 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.413+0000 7f1363fff700 1 --2- 172.21.15.77:0/282834689 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564345c260f0 0x564345c2ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.422 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.413+0000 7f1363fff700 1 -- 172.21.15.77:0/282834689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564345c253f0 msgr2=0x564345c2e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:56.422 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.413+0000 7f1363fff700 1 --2- 172.21.15.77:0/282834689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564345c253f0 0x564345c2e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.422 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.413+0000 7f1363fff700 1 -- 172.21.15.77:0/282834689 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1354017070 con 0x564345c26a70 2023-12-01T01:18:56.422 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.413+0000 7f13637fe700 1 --2- 172.21.15.77:0/282834689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564345c253f0 0x564345c2e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:18:56.422 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.413+0000 7f1362ffd700 1 --2- 172.21.15.77:0/282834689 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564345c260f0 0x564345c2ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:18:56.422 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.413+0000 7f1363fff700 1 --2- 172.21.15.77:0/282834689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564345c26a70 0x564345c35f70 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f135c00b0d0 tx=0x7f135c0078e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f99356df0a0f0b99 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:56.422 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.413+0000 7f1342ffd700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:18:56.422 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.413+0000 7f1342ffd700 1 -- 172.21.15.77:0/282834689 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c00e040 con 0x564345c26a70 2023-12-01T01:18:56.422 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.413+0000 7f1342ffd700 1 -- 172.21.15.77:0/282834689 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f135c003c00 con 0x564345c26a70 2023-12-01T01:18:56.422 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.413+0000 7f1375e5bb80 1 -- 172.21.15.77:0/282834689 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564345ba0d10 con 0x564345c26a70 2023-12-01T01:18:56.422 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.413+0000 7f1342ffd700 1 -- 172.21.15.77:0/282834689 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f135c003dc0 con 0x564345c26a70 2023-12-01T01:18:56.422 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.413+0000 7f1375e5bb80 10 client.15774 fetch_fsmap learned FSMap version 29 2023-12-01T01:18:56.422 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.413+0000 7f1375e5bb80 1 -- 172.21.15.77:0/282834689 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56434598cc60 con 0x564345c26a70 2023-12-01T01:18:56.423 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.413+0000 7f1342ffd700 1 -- 172.21.15.77:0/282834689 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564345ba0d10 con 0x564345c26a70 2023-12-01T01:18:56.423 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.413+0000 7f1342ffd700 1 -- 172.21.15.77:0/282834689 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f135c00b100 con 0x564345c26a70 2023-12-01T01:18:56.423 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.414+0000 7f1375e5bb80 10 client.15774 fetch_fsmap finished waiting for FSMap version 29 2023-12-01T01:18:56.423 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.414+0000 7f1375e5bb80 10 client.15774 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-01T01:18:56.423 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.414+0000 7f1375e5bb80 20 client.15774 populate_metadata read hostname 'smithi077' 2023-12-01T01:18:56.423 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.414+0000 7f1375e5bb80 1 --2- 172.21.15.77:0/282834689 >> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] conn(0x564345c3a7d0 0x564345c3cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.423 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.414+0000 7f1375e5bb80 4 client.15774 mds_command: new command op to 24475 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:18:56.423 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.414+0000 7f1375e5bb80 1 -- 172.21.15.77:0/282834689 --> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564345bf3e10 con 0x564345c3a7d0 2023-12-01T01:18:56.423 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.414+0000 7f13637fe700 1 --2- 172.21.15.77:0/282834689 >> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] conn(0x564345c3a7d0 0x564345c3cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.423 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.414+0000 7f1342ffd700 10 client.15774 ms_handle_connect on v2:172.21.15.203:6836/2887080977 2023-12-01T01:18:56.423 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.414+0000 7f13637fe700 1 --2- 172.21.15.77:0/282834689 >> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] conn(0x564345c3a7d0 0x564345c3cbf0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fcd1bf7d69a2683e server_cookie=e03135847c9fbc41 in_seq=0 out_seq=0 2023-12-01T01:18:56.423 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.415+0000 7f1342ffd700 1 -- 172.21.15.77:0/282834689 <== mds.2 v2:172.21.15.203:6836/2887080977 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x564345bf3e10 con 0x564345c3a7d0 2023-12-01T01:18:56.423 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.415+0000 7f1342ffd700 10 client.15774 handle_command_reply: tid=0 2023-12-01T01:18:56.423 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.415+0000 7f1375e5bb80 1 client.15774 shutdown 2023-12-01T01:18:56.424 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.415+0000 7f1375e5bb80 1 -- 172.21.15.77:0/282834689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564345c26a70 msgr2=0x564345c35f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:56.424 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.415+0000 7f1375e5bb80 1 --2- 172.21.15.77:0/282834689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564345c26a70 0x564345c35f70 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f135c00b0d0 tx=0x7f135c0078e0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.424 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.415+0000 7f1375e5bb80 1 -- 172.21.15.77:0/282834689 shutdown_connections 2023-12-01T01:18:56.424 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.415+0000 7f1375e5bb80 1 --2- 172.21.15.77:0/282834689 >> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] conn(0x564345c3a7d0 0x564345c3cbf0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.424 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.415+0000 7f1375e5bb80 1 --2- 172.21.15.77:0/282834689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564345c26a70 0x564345c35f70 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.424 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.415+0000 7f1342ffd700 0 client.15774 ms_handle_reset on v2:172.21.15.203:6836/2887080977 2023-12-01T01:18:56.425 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.415+0000 7f1375e5bb80 1 --2- 172.21.15.77:0/282834689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564345c253f0 0x564345c2e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.425 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.415+0000 7f1375e5bb80 1 --2- 172.21.15.77:0/282834689 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564345c260f0 0x564345c2ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.425 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.415+0000 7f1375e5bb80 1 -- 172.21.15.77:0/282834689 >> 172.21.15.77:0/282834689 conn(0x564345b840d0 msgr2=0x564345c1f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:56.425 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.415+0000 7f1375e5bb80 1 -- 172.21.15.77:0/282834689 shutdown_connections 2023-12-01T01:18:56.425 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.415+0000 7f1375e5bb80 1 -- 172.21.15.77:0/282834689 wait complete. 2023-12-01T01:18:56.425 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.415+0000 7f1375e5bb80 20 client.15774 trim_cache size 0 max 16384 2023-12-01T01:18:56.434 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.428+0000 7f1375e5bb80 1 Processor -- start 2023-12-01T01:18:56.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.428+0000 7f1375e5bb80 1 -- start start 2023-12-01T01:18:56.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.428+0000 7f1375e5bb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564345c253f0 0x564345c24e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.428+0000 7f1375e5bb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564345c260f0 0x564345c33f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.428+0000 7f1375e5bb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564345c26a70 0x564345c29310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.428+0000 7f1375e5bb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x564345bf3e10 con 0x564345c260f0 2023-12-01T01:18:56.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.428+0000 7f1375e5bb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564345ba0d10 con 0x564345c26a70 2023-12-01T01:18:56.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.428+0000 7f1375e5bb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x564345c2dea0 con 0x564345c253f0 2023-12-01T01:18:56.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.428+0000 7f1362ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564345c260f0 0x564345c33f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.428+0000 7f1362ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564345c260f0 0x564345c33f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60030/0 (socket says 172.21.15.77:60030) 2023-12-01T01:18:56.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.428+0000 7f1362ffd700 1 -- 172.21.15.77:0/1235206134 learned_addr learned my addr 172.21.15.77:0/1235206134 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:56.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.428+0000 7f1363fff700 1 --2- 172.21.15.77:0/1235206134 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564345c26a70 0x564345c29310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.428+0000 7f13637fe700 1 --2- 172.21.15.77:0/1235206134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564345c253f0 0x564345c24e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.428+0000 7f1362ffd700 1 -- 172.21.15.77:0/1235206134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564345c253f0 msgr2=0x564345c24e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:56.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.428+0000 7f1362ffd700 1 --2- 172.21.15.77:0/1235206134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564345c253f0 0x564345c24e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.428+0000 7f1362ffd700 1 -- 172.21.15.77:0/1235206134 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564345c26a70 msgr2=0x564345c29310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:56.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.428+0000 7f1362ffd700 1 --2- 172.21.15.77:0/1235206134 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564345c26a70 0x564345c29310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.428+0000 7f1362ffd700 1 -- 172.21.15.77:0/1235206134 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643459ca250 con 0x564345c260f0 2023-12-01T01:18:56.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.429+0000 7f1362ffd700 1 --2- 172.21.15.77:0/1235206134 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564345c260f0 0x564345c33f20 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f1358095860 tx=0x7f13580cb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3fe4d8d09381ff server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:56.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.429+0000 7f134a7fc700 1 -- 172.21.15.77:0/1235206134 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1358099a60 con 0x564345c260f0 2023-12-01T01:18:56.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.429+0000 7f134a7fc700 1 -- 172.21.15.77:0/1235206134 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1358099360 con 0x564345c260f0 2023-12-01T01:18:56.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.429+0000 7f134a7fc700 1 -- 172.21.15.77:0/1235206134 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13580a0f40 con 0x564345c260f0 2023-12-01T01:18:56.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.429+0000 7f1375e5bb80 1 -- 172.21.15.77:0/1235206134 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564345c260f0 msgr2=0x564345c33f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:56.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.429+0000 7f1375e5bb80 1 --2- 172.21.15.77:0/1235206134 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564345c260f0 0x564345c33f20 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f1358095860 tx=0x7f13580cb040 comp rx=0 tx=0).stop 2023-12-01T01:18:56.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.429+0000 7f1375e5bb80 1 -- 172.21.15.77:0/1235206134 shutdown_connections 2023-12-01T01:18:56.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.429+0000 7f1375e5bb80 1 --2- 172.21.15.77:0/1235206134 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564345c26a70 0x564345c29310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.429+0000 7f1375e5bb80 1 --2- 172.21.15.77:0/1235206134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564345c253f0 0x564345c24e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.429+0000 7f1375e5bb80 1 --2- 172.21.15.77:0/1235206134 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564345c260f0 0x564345c33f20 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.429+0000 7f1375e5bb80 1 -- 172.21.15.77:0/1235206134 >> 172.21.15.77:0/1235206134 conn(0x564345b840d0 msgr2=0x564345c2d6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:56.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.429+0000 7f1375e5bb80 1 -- 172.21.15.77:0/1235206134 shutdown_connections 2023-12-01T01:18:56.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.430+0000 7f1375e5bb80 1 -- 172.21.15.77:0/1235206134 wait complete. 2023-12-01T01:18:56.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.430+0000 7f1375e5bb80 1 -- start start 2023-12-01T01:18:56.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.430+0000 7f1375e5bb80 1 Processor -- start 2023-12-01T01:18:56.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.430+0000 7f1375e5bb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564345c253f0 0x564345c338e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.430+0000 7f1375e5bb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564345c260f0 0x564345c33e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.430+0000 7f1375e5bb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564345c26a70 0x564345c31f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.430+0000 7f1375e5bb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x564345c32620 con 0x564345c253f0 2023-12-01T01:18:56.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.430+0000 7f1375e5bb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564345c32790 con 0x564345c260f0 2023-12-01T01:18:56.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.430+0000 7f1375e5bb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x564345c32950 con 0x564345c26a70 2023-12-01T01:18:56.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.430+0000 7f1375e5bb80 10 client.0 _ll_register_callbacks cb 0x7f135c009c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:18:56.438 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.430+0000 7f13637fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564345c253f0 0x564345c338e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.438 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.430+0000 7f1362ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564345c260f0 0x564345c33e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.438 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.430+0000 7f13637fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564345c253f0 0x564345c338e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60034/0 (socket says 172.21.15.77:60034) 2023-12-01T01:18:56.438 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.430+0000 7f13637fe700 1 -- 172.21.15.77:0/601031015 learned_addr learned my addr 172.21.15.77:0/601031015 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:56.438 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.430+0000 7f1363fff700 1 --2- 172.21.15.77:0/601031015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564345c26a70 0x564345c31f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.438 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.431+0000 7f13637fe700 1 -- 172.21.15.77:0/601031015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564345c26a70 msgr2=0x564345c31f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:56.438 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.431+0000 7f13637fe700 1 --2- 172.21.15.77:0/601031015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564345c26a70 0x564345c31f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.438 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.431+0000 7f13637fe700 1 -- 172.21.15.77:0/601031015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564345c260f0 msgr2=0x564345c33e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:56.438 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.431+0000 7f13637fe700 1 --2- 172.21.15.77:0/601031015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564345c260f0 0x564345c33e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.438 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.431+0000 7f13637fe700 1 -- 172.21.15.77:0/601031015 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13641a1380 con 0x564345c253f0 2023-12-01T01:18:56.438 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.431+0000 7f1362ffd700 1 --2- 172.21.15.77:0/601031015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564345c260f0 0x564345c33e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:18:56.438 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.431+0000 7f13637fe700 1 --2- 172.21.15.77:0/601031015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564345c253f0 0x564345c338e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x564345c3d500 tx=0x7f1354009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f766a8890bffd10 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:56.438 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.431+0000 7f1342ffd700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:18:56.439 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.431+0000 7f1342ffd700 1 -- 172.21.15.77:0/601031015 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1354036070 con 0x564345c253f0 2023-12-01T01:18:56.439 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.431+0000 7f1342ffd700 1 -- 172.21.15.77:0/601031015 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f1354031310 con 0x564345c253f0 2023-12-01T01:18:56.439 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.431+0000 7f1375e5bb80 1 -- 172.21.15.77:0/601031015 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1354021070 con 0x564345c253f0 2023-12-01T01:18:56.439 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.431+0000 7f1342ffd700 1 -- 172.21.15.77:0/601031015 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f135402c040 con 0x564345c253f0 2023-12-01T01:18:56.439 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.431+0000 7f1375e5bb80 10 client.15780 fetch_fsmap learned FSMap version 29 2023-12-01T01:18:56.439 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.431+0000 7f1375e5bb80 1 -- 172.21.15.77:0/601031015 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5643459a22b0 con 0x564345c253f0 2023-12-01T01:18:56.439 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.431+0000 7f1342ffd700 1 -- 172.21.15.77:0/601031015 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1354041020 con 0x564345c253f0 2023-12-01T01:18:56.439 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.431+0000 7f1342ffd700 1 -- 172.21.15.77:0/601031015 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1354039020 con 0x564345c253f0 2023-12-01T01:18:56.439 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.431+0000 7f1375e5bb80 10 client.15780 fetch_fsmap finished waiting for FSMap version 29 2023-12-01T01:18:56.439 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.431+0000 7f1375e5bb80 10 client.15780 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-01T01:18:56.439 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.431+0000 7f1375e5bb80 20 client.15780 populate_metadata read hostname 'smithi077' 2023-12-01T01:18:56.439 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.431+0000 7f1375e5bb80 1 --2- 172.21.15.77:0/601031015 >> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] conn(0x564345c8d750 0x564345c8fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.439 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.431+0000 7f1375e5bb80 4 client.15780 mds_command: new command op to 24475 tid=0[{"prefix": "damage ls"}] 2023-12-01T01:18:56.441 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.432+0000 7f1375e5bb80 1 -- 172.21.15.77:0/601031015 --> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564345b6ed40 con 0x564345c8d750 2023-12-01T01:18:56.441 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.432+0000 7f1362ffd700 1 --2- 172.21.15.77:0/601031015 >> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] conn(0x564345c8d750 0x564345c8fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.441 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.432+0000 7f1342ffd700 10 client.15780 ms_handle_connect on v2:172.21.15.203:6836/2887080977 2023-12-01T01:18:56.441 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.432+0000 7f1362ffd700 1 --2- 172.21.15.77:0/601031015 >> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] conn(0x564345c8d750 0x564345c8fb70 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3503be53bd7ab9f6 server_cookie=d85fe0c81bc0c03f in_seq=0 out_seq=0 2023-12-01T01:18:56.441 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.432+0000 7f1342ffd700 1 -- 172.21.15.77:0/601031015 <== mds.2 v2:172.21.15.203:6836/2887080977 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564345b6ed40 con 0x564345c8d750 2023-12-01T01:18:56.441 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.432+0000 7f1342ffd700 10 client.15780 handle_command_reply: tid=0 2023-12-01T01:18:56.441 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.432+0000 7f1375e5bb80 1 client.15780 shutdown 2023-12-01T01:18:56.441 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.432+0000 7f1375e5bb80 1 -- 172.21.15.77:0/601031015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564345c253f0 msgr2=0x564345c338e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:56.441 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.432+0000 7f1375e5bb80 1 --2- 172.21.15.77:0/601031015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564345c253f0 0x564345c338e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x564345c3d500 tx=0x7f1354009780 comp rx=0 tx=0).stop 2023-12-01T01:18:56.441 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.432+0000 7f1375e5bb80 1 -- 172.21.15.77:0/601031015 shutdown_connections 2023-12-01T01:18:56.441 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.432+0000 7f1375e5bb80 1 --2- 172.21.15.77:0/601031015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564345c26a70 0x564345c31f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.442 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.432+0000 7f1375e5bb80 1 --2- 172.21.15.77:0/601031015 >> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] conn(0x564345c8d750 0x564345c8fb70 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.442 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.432+0000 7f1375e5bb80 1 --2- 172.21.15.77:0/601031015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564345c253f0 0x564345c338e0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.442 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.432+0000 7f1342ffd700 0 client.15780 ms_handle_reset on v2:172.21.15.203:6836/2887080977 2023-12-01T01:18:56.442 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.432+0000 7f1375e5bb80 1 --2- 172.21.15.77:0/601031015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564345c260f0 0x564345c33e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.442 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.432+0000 7f1375e5bb80 1 -- 172.21.15.77:0/601031015 >> 172.21.15.77:0/601031015 conn(0x564345b840d0 msgr2=0x564345c29310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:56.442 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.433+0000 7f1375e5bb80 1 -- 172.21.15.77:0/601031015 shutdown_connections 2023-12-01T01:18:56.442 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.433+0000 7f1375e5bb80 1 -- 172.21.15.77:0/601031015 wait complete. 2023-12-01T01:18:56.442 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.433+0000 7f1375e5bb80 20 client.15780 trim_cache size 0 max 16384 2023-12-01T01:18:56.442 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2023-12-01T01:18:56.443 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.433+0000 7f1369ccb700 1 -- 172.21.15.77:0/1224106380 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f134c0a0520 msgr2=0x7f134c0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:56.443 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.433+0000 7f1369ccb700 1 --2- 172.21.15.77:0/1224106380 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f134c0a0520 0x7f134c0a29e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1364100cd0 tx=0x7f1354009260 comp rx=0 tx=0).stop 2023-12-01T01:18:56.443 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.433+0000 7f1369ccb700 1 -- 172.21.15.77:0/1224106380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1364100e70 msgr2=0x7f136419dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:56.443 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.433+0000 7f1369ccb700 1 --2- 172.21.15.77:0/1224106380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1364100e70 0x7f136419dcf0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f135800e9a0 tx=0x7f135800ecb0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.443 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.433+0000 7f1369ccb700 1 -- 172.21.15.77:0/1224106380 shutdown_connections 2023-12-01T01:18:56.443 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.433+0000 7f1369ccb700 1 --2- 172.21.15.77:0/1224106380 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f134c0a0520 0x7f134c0a29e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.443 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.433+0000 7f1369ccb700 1 --2- 172.21.15.77:0/1224106380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1364101810 0x7f13641a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.444 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.433+0000 7f1369ccb700 1 --2- 172.21.15.77:0/1224106380 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13640ffc80 0x7f1364068a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.444 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.433+0000 7f1369ccb700 1 --2- 172.21.15.77:0/1224106380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1364100e70 0x7f136419dcf0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.444 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.433+0000 7f1369ccb700 1 -- 172.21.15.77:0/1224106380 >> 172.21.15.77:0/1224106380 conn(0x7f13640fb1f0 msgr2=0x7f1364104300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:56.444 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.433+0000 7f1369ccb700 1 -- 172.21.15.77:0/1224106380 shutdown_connections 2023-12-01T01:18:56.444 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.433+0000 7f1369ccb700 1 -- 172.21.15.77:0/1224106380 wait complete. 2023-12-01T01:18:56.447 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-12-01T01:18:56.575 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.571+0000 7f852241b700 1 -- 172.21.15.77:0/200196463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f851c101f70 msgr2=0x7f851c1023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:56.575 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.571+0000 7f852241b700 1 --2- 172.21.15.77:0/200196463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f851c101f70 0x7f851c1023d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f850c0099c0 tx=0x7f850c009cd0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.576 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.572+0000 7f852241b700 1 -- 172.21.15.77:0/200196463 shutdown_connections 2023-12-01T01:18:56.576 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.572+0000 7f852241b700 1 --2- 172.21.15.77:0/200196463 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f851c102910 0x7f851c1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.576 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.572+0000 7f852241b700 1 --2- 172.21.15.77:0/200196463 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f851c100d80 0x7f851c1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.576 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.572+0000 7f852241b700 1 --2- 172.21.15.77:0/200196463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f851c101f70 0x7f851c1023d0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.576 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.572+0000 7f852241b700 1 -- 172.21.15.77:0/200196463 >> 172.21.15.77:0/200196463 conn(0x7f851c0fc310 msgr2=0x7f851c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:56.576 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.572+0000 7f852241b700 1 -- 172.21.15.77:0/200196463 shutdown_connections 2023-12-01T01:18:56.576 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.572+0000 7f852241b700 1 -- 172.21.15.77:0/200196463 wait complete. 2023-12-01T01:18:56.576 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.572+0000 7f852241b700 1 Processor -- start 2023-12-01T01:18:56.576 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.572+0000 7f852241b700 1 -- start start 2023-12-01T01:18:56.576 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.572+0000 7f852241b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f851c100d80 0x7f851c196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.576 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.572+0000 7f852241b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f851c101f70 0x7f851c196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.576 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.572+0000 7f852241b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f851c102910 0x7f851c19dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.577 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.572+0000 7f852241b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851c19e5b0 con 0x7f851c100d80 2023-12-01T01:18:56.577 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.572+0000 7f852241b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851c19e6f0 con 0x7f851c102910 2023-12-01T01:18:56.577 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.572+0000 7f852241b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851c19e880 con 0x7f851c101f70 2023-12-01T01:18:56.577 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.573+0000 7f851bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f851c100d80 0x7f851c196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.577 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.573+0000 7f851bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f851c100d80 0x7f851c196a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60048/0 (socket says 172.21.15.77:60048) 2023-12-01T01:18:56.577 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.573+0000 7f851bfff700 1 -- 172.21.15.77:0/2944989987 learned_addr learned my addr 172.21.15.77:0/2944989987 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:56.577 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.573+0000 7f85209b8700 1 --2- 172.21.15.77:0/2944989987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f851c102910 0x7f851c19dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.577 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.573+0000 7f851b7fe700 1 --2- 172.21.15.77:0/2944989987 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f851c101f70 0x7f851c196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.577 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.573+0000 7f851bfff700 1 -- 172.21.15.77:0/2944989987 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f851c101f70 msgr2=0x7f851c196f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:56.577 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.573+0000 7f851bfff700 1 --2- 172.21.15.77:0/2944989987 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f851c101f70 0x7f851c196f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.577 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.573+0000 7f851bfff700 1 -- 172.21.15.77:0/2944989987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f851c102910 msgr2=0x7f851c19dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:56.577 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.573+0000 7f851bfff700 1 --2- 172.21.15.77:0/2944989987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f851c102910 0x7f851c19dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.578 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.573+0000 7f851bfff700 1 -- 172.21.15.77:0/2944989987 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850c0095f0 con 0x7f851c100d80 2023-12-01T01:18:56.578 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.573+0000 7f851bfff700 1 --2- 172.21.15.77:0/2944989987 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f851c100d80 0x7f851c196a30 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f851000e970 tx=0x7f851000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:56.578 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.573+0000 7f85197fa700 1 -- 172.21.15.77:0/2944989987 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8510009790 con 0x7f851c100d80 2023-12-01T01:18:56.578 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.573+0000 7f852241b700 1 -- 172.21.15.77:0/2944989987 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f851c19eb60 con 0x7f851c100d80 2023-12-01T01:18:56.578 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.573+0000 7f85197fa700 1 -- 172.21.15.77:0/2944989987 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f851000eec0 con 0x7f851c100d80 2023-12-01T01:18:56.579 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.573+0000 7f85197fa700 1 -- 172.21.15.77:0/2944989987 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8510003cf0 con 0x7f851c100d80 2023-12-01T01:18:56.579 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.573+0000 7f852241b700 1 -- 172.21.15.77:0/2944989987 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f851c19f070 con 0x7f851c100d80 2023-12-01T01:18:56.585 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.575+0000 7f85197fa700 1 -- 172.21.15.77:0/2944989987 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f851000ccd0 con 0x7f851c100d80 2023-12-01T01:18:56.585 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.576+0000 7f85197fa700 1 --2- 172.21.15.77:0/2944989987 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f85040a0520 0x7f85040a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.585 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.576+0000 7f85197fa700 1 -- 172.21.15.77:0/2944989987 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f8510014070 con 0x7f851c100d80 2023-12-01T01:18:56.585 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.576+0000 7f851b7fe700 1 --2- 172.21.15.77:0/2944989987 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f85040a0520 0x7f85040a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.585 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.576+0000 7f851b7fe700 1 --2- 172.21.15.77:0/2944989987 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f85040a0520 0x7f85040a29e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f850c00b580 tx=0x7f850c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:56.586 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.577+0000 7f852e5abb80 1 Processor -- start 2023-12-01T01:18:56.586 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.577+0000 7f852e5abb80 1 -- start start 2023-12-01T01:18:56.586 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.577+0000 7f852e5abb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f09e00f0 0x5628f09e0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.586 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.577+0000 7f852e5abb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628f09e0a70 0x5628f09dee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.586 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.577+0000 7f852e5abb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628f09df3f0 0x5628f09df870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.586 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.577+0000 7f852e5abb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628f0936a10 con 0x5628f09e0a70 2023-12-01T01:18:56.586 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.577+0000 7f852e5abb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628f09343d0 con 0x5628f09df3f0 2023-12-01T01:18:56.586 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.577+0000 7f852e5abb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628f092fcc0 con 0x5628f09e00f0 2023-12-01T01:18:56.586 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.577+0000 7f851b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628f09e0a70 0x5628f09dee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.586 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.577+0000 7f851b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628f09e0a70 0x5628f09dee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60056/0 (socket says 172.21.15.77:60056) 2023-12-01T01:18:56.586 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.577+0000 7f851b7fe700 1 -- 172.21.15.77:0/197903382 learned_addr learned my addr 172.21.15.77:0/197903382 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:56.586 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.577+0000 7f85209b8700 1 --2- 172.21.15.77:0/197903382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628f09df3f0 0x5628f09df870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.586 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.577+0000 7f851bfff700 1 --2- 172.21.15.77:0/197903382 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f09e00f0 0x5628f09e0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.587 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.577+0000 7f85209b8700 1 -- 172.21.15.77:0/197903382 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f09e00f0 msgr2=0x5628f09e0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:56.587 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.577+0000 7f85209b8700 1 --2- 172.21.15.77:0/197903382 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f09e00f0 0x5628f09e0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.587 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.577+0000 7f85209b8700 1 -- 172.21.15.77:0/197903382 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628f09e0a70 msgr2=0x5628f09dee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:56.587 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.577+0000 7f85209b8700 1 --2- 172.21.15.77:0/197903382 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628f09e0a70 0x5628f09dee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.587 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.577+0000 7f85209b8700 1 -- 172.21.15.77:0/197903382 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628f075dde0 con 0x5628f09df3f0 2023-12-01T01:18:56.587 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.577+0000 7f851b7fe700 1 --2- 172.21.15.77:0/197903382 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628f09e0a70 0x5628f09dee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:18:56.587 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.578+0000 7f85209b8700 1 --2- 172.21.15.77:0/197903382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628f09df3f0 0x5628f09df870 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8514003f20 tx=0x7f8514009ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=512e0cd807e3984e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:56.587 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.578+0000 7f85027fc700 1 -- 172.21.15.77:0/197903382 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8514019070 con 0x5628f09df3f0 2023-12-01T01:18:56.587 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.578+0000 7f85027fc700 1 -- 172.21.15.77:0/197903382 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f851400f040 con 0x5628f09df3f0 2023-12-01T01:18:56.587 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.578+0000 7f85027fc700 1 -- 172.21.15.77:0/197903382 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8514007970 con 0x5628f09df3f0 2023-12-01T01:18:56.587 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.578+0000 7f852e5abb80 1 -- 172.21.15.77:0/197903382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628f09df3f0 msgr2=0x5628f09df870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:56.587 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.578+0000 7f852e5abb80 1 --2- 172.21.15.77:0/197903382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628f09df3f0 0x5628f09df870 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8514003f20 tx=0x7f8514009ba0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.587 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.579+0000 7f852e5abb80 1 -- 172.21.15.77:0/197903382 shutdown_connections 2023-12-01T01:18:56.587 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.579+0000 7f852e5abb80 1 --2- 172.21.15.77:0/197903382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628f09df3f0 0x5628f09df870 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.587 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.579+0000 7f852e5abb80 1 --2- 172.21.15.77:0/197903382 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f09e00f0 0x5628f09e0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.588 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.579+0000 7f852e5abb80 1 --2- 172.21.15.77:0/197903382 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628f09e0a70 0x5628f09dee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.588 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.579+0000 7f852e5abb80 1 -- 172.21.15.77:0/197903382 >> 172.21.15.77:0/197903382 conn(0x5628f093e0d0 msgr2=0x5628f09d9940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:56.588 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.579+0000 7f852e5abb80 1 -- 172.21.15.77:0/197903382 shutdown_connections 2023-12-01T01:18:56.588 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.579+0000 7f852e5abb80 1 -- 172.21.15.77:0/197903382 wait complete. 2023-12-01T01:18:56.588 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.579+0000 7f852e5abb80 1 -- start start 2023-12-01T01:18:56.588 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.579+0000 7f852e5abb80 1 Processor -- start 2023-12-01T01:18:56.588 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.579+0000 7f852e5abb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f09df3f0 0x5628f09e8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.588 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.579+0000 7f852e5abb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628f09e00f0 0x5628f09e8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.588 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.579+0000 7f852e5abb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628f09e0a70 0x5628f09efec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.588 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.579+0000 7f852e5abb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628f08cc510 con 0x5628f09e0a70 2023-12-01T01:18:56.588 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.579+0000 7f852e5abb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628f08cc1b0 con 0x5628f09e00f0 2023-12-01T01:18:56.588 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.579+0000 7f852e5abb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628f07b3920 con 0x5628f09df3f0 2023-12-01T01:18:56.588 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.580+0000 7f852e5abb80 10 client.0 _ll_register_callbacks cb 0x5628f0819a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:18:56.588 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.580+0000 7f851bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f09df3f0 0x5628f09e8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.589 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.580+0000 7f85209b8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628f09e0a70 0x5628f09efec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.589 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.580+0000 7f851b7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628f09e00f0 0x5628f09e8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.589 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.580+0000 7f851bfff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f09df3f0 0x5628f09e8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:46250/0 (socket says 172.21.15.77:46250) 2023-12-01T01:18:56.589 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.580+0000 7f85209b8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628f09e0a70 0x5628f09efec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60066/0 (socket says 172.21.15.77:60066) 2023-12-01T01:18:56.589 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.580+0000 7f851bfff700 1 -- 172.21.15.77:0/4284475158 learned_addr learned my addr 172.21.15.77:0/4284475158 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:56.589 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.580+0000 7f85209b8700 1 -- 172.21.15.77:0/4284475158 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f09df3f0 msgr2=0x5628f09e8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:56.589 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.580+0000 7f85209b8700 1 --2- 172.21.15.77:0/4284475158 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f09df3f0 0x5628f09e8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.589 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.580+0000 7f85209b8700 1 -- 172.21.15.77:0/4284475158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628f09e00f0 msgr2=0x5628f09e8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:56.589 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.580+0000 7f85209b8700 1 --2- 172.21.15.77:0/4284475158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628f09e00f0 0x5628f09e8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.589 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.580+0000 7f85209b8700 1 -- 172.21.15.77:0/4284475158 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f851c109b10 con 0x5628f09e0a70 2023-12-01T01:18:56.589 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.580+0000 7f851b7fe700 1 --2- 172.21.15.77:0/4284475158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628f09e00f0 0x5628f09e8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:18:56.589 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.580+0000 7f851bfff700 1 --2- 172.21.15.77:0/4284475158 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f09df3f0 0x5628f09e8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:18:56.589 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.580+0000 7f85209b8700 1 --2- 172.21.15.77:0/4284475158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628f09e0a70 0x5628f09efec0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f8514004010 tx=0x7f8514004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9ce1a02ece67dc5 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:56.590 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.580+0000 7f84fb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:18:56.590 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.580+0000 7f84fb7fe700 1 -- 172.21.15.77:0/4284475158 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8514019040 con 0x5628f09e0a70 2023-12-01T01:18:56.590 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.580+0000 7f84fb7fe700 1 -- 172.21.15.77:0/4284475158 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f8514007a80 con 0x5628f09e0a70 2023-12-01T01:18:56.590 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.580+0000 7f852e5abb80 1 -- 172.21.15.77:0/4284475158 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5628f095ad10 con 0x5628f09e0a70 2023-12-01T01:18:56.590 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.580+0000 7f84fb7fe700 1 -- 172.21.15.77:0/4284475158 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8514007c40 con 0x5628f09e0a70 2023-12-01T01:18:56.590 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.580+0000 7f852e5abb80 10 client.15792 fetch_fsmap learned FSMap version 29 2023-12-01T01:18:56.590 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.580+0000 7f852e5abb80 1 -- 172.21.15.77:0/4284475158 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f8514003b50 con 0x5628f09e0a70 2023-12-01T01:18:56.590 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.580+0000 7f84fb7fe700 1 -- 172.21.15.77:0/4284475158 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f851402e020 con 0x5628f09e0a70 2023-12-01T01:18:56.590 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.581+0000 7f84fb7fe700 1 -- 172.21.15.77:0/4284475158 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f851401f450 con 0x5628f09e0a70 2023-12-01T01:18:56.590 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.581+0000 7f852e5abb80 10 client.15792 fetch_fsmap finished waiting for FSMap version 29 2023-12-01T01:18:56.590 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.581+0000 7f852e5abb80 10 client.15792 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-12-01T01:18:56.590 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.581+0000 7f852e5abb80 20 client.15792 populate_metadata read hostname 'smithi077' 2023-12-01T01:18:56.591 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.581+0000 7f852e5abb80 1 --2- 172.21.15.77:0/4284475158 >> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] conn(0x5628f09f4580 0x5628f09f69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.591 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.581+0000 7f852e5abb80 4 client.15792 mds_command: new command op to 24496 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:18:56.591 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.581+0000 7f852e5abb80 1 -- 172.21.15.77:0/4284475158 --> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5628f09ade10 con 0x5628f09f4580 2023-12-01T01:18:56.591 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.581+0000 7f851bfff700 1 --2- 172.21.15.77:0/4284475158 >> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] conn(0x5628f09f4580 0x5628f09f69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.591 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.581+0000 7f84fb7fe700 10 client.15792 ms_handle_connect on v2:172.21.15.203:6838/2888221613 2023-12-01T01:18:56.591 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.581+0000 7f851bfff700 1 --2- 172.21.15.77:0/4284475158 >> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] conn(0x5628f09f4580 0x5628f09f69a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ff080f51d82f3b70 server_cookie=c6dfd6340753e384 in_seq=0 out_seq=0 2023-12-01T01:18:56.591 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.582+0000 7f84fb7fe700 1 -- 172.21.15.77:0/4284475158 <== mds.1 v2:172.21.15.203:6838/2888221613 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5628f09ade10 con 0x5628f09f4580 2023-12-01T01:18:56.591 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.582+0000 7f84fb7fe700 10 client.15792 handle_command_reply: tid=0 2023-12-01T01:18:56.591 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.582+0000 7f852e5abb80 1 client.15792 shutdown 2023-12-01T01:18:56.591 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.582+0000 7f852e5abb80 1 -- 172.21.15.77:0/4284475158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628f09e0a70 msgr2=0x5628f09efec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:56.591 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.582+0000 7f852e5abb80 1 --2- 172.21.15.77:0/4284475158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628f09e0a70 0x5628f09efec0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f8514004010 tx=0x7f8514004760 comp rx=0 tx=0).stop 2023-12-01T01:18:56.591 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.582+0000 7f852e5abb80 1 -- 172.21.15.77:0/4284475158 shutdown_connections 2023-12-01T01:18:56.591 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.582+0000 7f852e5abb80 1 --2- 172.21.15.77:0/4284475158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628f09e0a70 0x5628f09efec0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.591 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.582+0000 7f852e5abb80 1 --2- 172.21.15.77:0/4284475158 >> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] conn(0x5628f09f4580 0x5628f09f69a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.592 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.582+0000 7f852e5abb80 1 --2- 172.21.15.77:0/4284475158 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f09df3f0 0x5628f09e8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.592 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.582+0000 7f852e5abb80 1 --2- 172.21.15.77:0/4284475158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628f09e00f0 0x5628f09e8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.592 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.582+0000 7f852e5abb80 1 -- 172.21.15.77:0/4284475158 >> 172.21.15.77:0/4284475158 conn(0x5628f093e0d0 msgr2=0x5628f09d90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:56.592 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.582+0000 7f84fb7fe700 0 client.15792 ms_handle_reset on v2:172.21.15.203:6838/2888221613 2023-12-01T01:18:56.592 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.582+0000 7f852e5abb80 1 -- 172.21.15.77:0/4284475158 shutdown_connections 2023-12-01T01:18:56.592 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.582+0000 7f852e5abb80 1 -- 172.21.15.77:0/4284475158 wait complete. 2023-12-01T01:18:56.592 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.582+0000 7f852e5abb80 20 client.15792 trim_cache size 0 max 16384 2023-12-01T01:18:56.598 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.596+0000 7f852e5abb80 1 Processor -- start 2023-12-01T01:18:56.599 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.596+0000 7f852e5abb80 1 -- start start 2023-12-01T01:18:56.599 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.596+0000 7f852e5abb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628f09df3f0 0x5628f09dee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.599 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.596+0000 7f852e5abb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f09e00f0 0x5628f09efe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.599 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.596+0000 7f852e5abb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628f09e0a70 0x5628f09e67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.599 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.596+0000 7f852e5abb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628f09e7e20 con 0x5628f09e0a70 2023-12-01T01:18:56.599 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.596+0000 7f852e5abb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628f09db4a0 con 0x5628f09df3f0 2023-12-01T01:18:56.599 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.596+0000 7f852e5abb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628f09e8500 con 0x5628f09e00f0 2023-12-01T01:18:56.599 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.597+0000 7f85209b8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628f09e0a70 0x5628f09e67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.599 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.597+0000 7f85209b8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628f09e0a70 0x5628f09e67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60074/0 (socket says 172.21.15.77:60074) 2023-12-01T01:18:56.599 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.597+0000 7f851bfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628f09df3f0 0x5628f09dee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.599 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.597+0000 7f85209b8700 1 -- 172.21.15.77:0/2474681390 learned_addr learned my addr 172.21.15.77:0/2474681390 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:56.599 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.597+0000 7f851b7fe700 1 --2- 172.21.15.77:0/2474681390 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f09e00f0 0x5628f09efe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.600 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.597+0000 7f851bfff700 1 -- 172.21.15.77:0/2474681390 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f09e00f0 msgr2=0x5628f09efe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:56.600 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.597+0000 7f851bfff700 1 --2- 172.21.15.77:0/2474681390 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f09e00f0 0x5628f09efe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.600 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.597+0000 7f851bfff700 1 -- 172.21.15.77:0/2474681390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628f09e0a70 msgr2=0x5628f09e67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:56.600 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.597+0000 7f851bfff700 1 --2- 172.21.15.77:0/2474681390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628f09e0a70 0x5628f09e67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.600 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.597+0000 7f851bfff700 1 -- 172.21.15.77:0/2474681390 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628f0746c60 con 0x5628f09df3f0 2023-12-01T01:18:56.600 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.597+0000 7f851b7fe700 1 --2- 172.21.15.77:0/2474681390 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f09e00f0 0x5628f09efe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:18:56.600 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.597+0000 7f85209b8700 1 --2- 172.21.15.77:0/2474681390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628f09e0a70 0x5628f09e67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:18:56.600 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.597+0000 7f851bfff700 1 --2- 172.21.15.77:0/2474681390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628f09df3f0 0x5628f09dee60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f851000c5b0 tx=0x7f85100c95f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5df08ef240b0d08e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:56.600 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.597+0000 7f8502ffd700 1 -- 172.21.15.77:0/2474681390 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8510098de0 con 0x5628f09df3f0 2023-12-01T01:18:56.600 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.597+0000 7f8502ffd700 1 -- 172.21.15.77:0/2474681390 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85100b0070 con 0x5628f09df3f0 2023-12-01T01:18:56.601 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.597+0000 7f8502ffd700 1 -- 172.21.15.77:0/2474681390 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8510097930 con 0x5628f09df3f0 2023-12-01T01:18:56.605 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.598+0000 7f852e5abb80 1 -- 172.21.15.77:0/2474681390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628f09df3f0 msgr2=0x5628f09dee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:56.605 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.598+0000 7f852e5abb80 1 --2- 172.21.15.77:0/2474681390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628f09df3f0 0x5628f09dee60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f851000c5b0 tx=0x7f85100c95f0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.605 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.598+0000 7f852e5abb80 1 -- 172.21.15.77:0/2474681390 shutdown_connections 2023-12-01T01:18:56.605 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.598+0000 7f852e5abb80 1 --2- 172.21.15.77:0/2474681390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628f09e0a70 0x5628f09e67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.605 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.598+0000 7f852e5abb80 1 --2- 172.21.15.77:0/2474681390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628f09df3f0 0x5628f09dee60 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.605 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.598+0000 7f852e5abb80 1 --2- 172.21.15.77:0/2474681390 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f09e00f0 0x5628f09efe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.605 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.598+0000 7f852e5abb80 1 -- 172.21.15.77:0/2474681390 >> 172.21.15.77:0/2474681390 conn(0x5628f093e0d0 msgr2=0x5628f09d97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:56.605 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.598+0000 7f852e5abb80 1 -- 172.21.15.77:0/2474681390 shutdown_connections 2023-12-01T01:18:56.605 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.598+0000 7f852e5abb80 1 -- 172.21.15.77:0/2474681390 wait complete. 2023-12-01T01:18:56.605 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.598+0000 7f852e5abb80 1 -- start start 2023-12-01T01:18:56.605 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.599+0000 7f852e5abb80 1 Processor -- start 2023-12-01T01:18:56.606 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.599+0000 7f852e5abb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f09df3f0 0x5628f09ec070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.606 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.599+0000 7f852e5abb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628f09e00f0 0x5628f09ec5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.606 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.599+0000 7f852e5abb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628f09e0a70 0x5628f09ef5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.606 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.599+0000 7f852e5abb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628f09efb10 con 0x5628f09e0a70 2023-12-01T01:18:56.606 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.599+0000 7f852e5abb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628f09efc80 con 0x5628f09e00f0 2023-12-01T01:18:56.606 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.599+0000 7f852e5abb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628f09efe40 con 0x5628f09df3f0 2023-12-01T01:18:56.606 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.599+0000 7f852e5abb80 10 client.0 _ll_register_callbacks cb 0x7f850c005a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:18:56.606 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.599+0000 7f85209b8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628f09e0a70 0x5628f09ef5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.606 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.599+0000 7f851b7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628f09e00f0 0x5628f09ec5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.606 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.599+0000 7f85209b8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628f09e0a70 0x5628f09ef5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60080/0 (socket says 172.21.15.77:60080) 2023-12-01T01:18:56.606 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.599+0000 7f851b7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628f09e00f0 0x5628f09ec5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:39000/0 (socket says 172.21.15.77:39000) 2023-12-01T01:18:56.606 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.599+0000 7f85209b8700 1 -- 172.21.15.77:0/3622639239 learned_addr learned my addr 172.21.15.77:0/3622639239 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:56.606 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.599+0000 7f851bfff700 1 --2- 172.21.15.77:0/3622639239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f09df3f0 0x5628f09ec070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.606 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.599+0000 7f85209b8700 1 -- 172.21.15.77:0/3622639239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f09df3f0 msgr2=0x5628f09ec070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:56.606 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.599+0000 7f85209b8700 1 --2- 172.21.15.77:0/3622639239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f09df3f0 0x5628f09ec070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.607 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.599+0000 7f85209b8700 1 -- 172.21.15.77:0/3622639239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628f09e00f0 msgr2=0x5628f09ec5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:56.607 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.600+0000 7f85209b8700 1 --2- 172.21.15.77:0/3622639239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628f09e00f0 0x5628f09ec5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.607 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.600+0000 7f85209b8700 1 -- 172.21.15.77:0/3622639239 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8514003b50 con 0x5628f09e0a70 2023-12-01T01:18:56.607 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.600+0000 7f851b7fe700 1 --2- 172.21.15.77:0/3622639239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628f09e00f0 0x5628f09ec5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:18:56.607 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.600+0000 7f85209b8700 1 --2- 172.21.15.77:0/3622639239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628f09e0a70 0x5628f09ef5d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8514002230 tx=0x7f8514009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca0b8dc550e629ae server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:56.607 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.600+0000 7f84fb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:18:56.607 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.600+0000 7f84fb7fe700 1 -- 172.21.15.77:0/3622639239 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8514019040 con 0x5628f09e0a70 2023-12-01T01:18:56.607 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.600+0000 7f84fb7fe700 1 -- 172.21.15.77:0/3622639239 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f851402f080 con 0x5628f09e0a70 2023-12-01T01:18:56.607 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.600+0000 7f852e5abb80 1 -- 172.21.15.77:0/3622639239 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f851400f040 con 0x5628f09e0a70 2023-12-01T01:18:56.607 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.600+0000 7f84fb7fe700 1 -- 172.21.15.77:0/3622639239 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8514007a80 con 0x5628f09e0a70 2023-12-01T01:18:56.607 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.600+0000 7f852e5abb80 10 client.15798 fetch_fsmap learned FSMap version 29 2023-12-01T01:18:56.607 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.600+0000 7f852e5abb80 1 -- 172.21.15.77:0/3622639239 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f851c19f070 con 0x5628f09e0a70 2023-12-01T01:18:56.607 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.600+0000 7f84fb7fe700 1 -- 172.21.15.77:0/3622639239 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f851400f040 con 0x5628f09e0a70 2023-12-01T01:18:56.607 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.600+0000 7f84fb7fe700 1 -- 172.21.15.77:0/3622639239 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8514031040 con 0x5628f09e0a70 2023-12-01T01:18:56.608 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.600+0000 7f852e5abb80 10 client.15798 fetch_fsmap finished waiting for FSMap version 29 2023-12-01T01:18:56.608 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.600+0000 7f852e5abb80 10 client.15798 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-12-01T01:18:56.608 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.600+0000 7f852e5abb80 20 client.15798 populate_metadata read hostname 'smithi077' 2023-12-01T01:18:56.608 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.600+0000 7f852e5abb80 1 --2- 172.21.15.77:0/3622639239 >> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] conn(0x5628f0a47140 0x5628f0a49560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.608 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.600+0000 7f852e5abb80 4 client.15798 mds_command: new command op to 24496 tid=0[{"prefix": "damage ls"}] 2023-12-01T01:18:56.608 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.600+0000 7f852e5abb80 1 -- 172.21.15.77:0/3622639239 --> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5628f0928d40 con 0x5628f0a47140 2023-12-01T01:18:56.608 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.600+0000 7f851bfff700 1 --2- 172.21.15.77:0/3622639239 >> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] conn(0x5628f0a47140 0x5628f0a49560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.608 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.601+0000 7f84fb7fe700 10 client.15798 ms_handle_connect on v2:172.21.15.203:6838/2888221613 2023-12-01T01:18:56.608 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.601+0000 7f851bfff700 1 --2- 172.21.15.77:0/3622639239 >> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] conn(0x5628f0a47140 0x5628f0a49560 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=28b82b3e035de6fd server_cookie=d0af238871d28581 in_seq=0 out_seq=0 2023-12-01T01:18:56.608 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.601+0000 7f84fb7fe700 1 -- 172.21.15.77:0/3622639239 <== mds.1 v2:172.21.15.203:6838/2888221613 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5628f0928d40 con 0x5628f0a47140 2023-12-01T01:18:56.608 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.601+0000 7f84fb7fe700 10 client.15798 handle_command_reply: tid=0 2023-12-01T01:18:56.608 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.601+0000 7f852e5abb80 1 client.15798 shutdown 2023-12-01T01:18:56.608 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.601+0000 7f852e5abb80 1 -- 172.21.15.77:0/3622639239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628f09e0a70 msgr2=0x5628f09ef5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:56.608 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.601+0000 7f852e5abb80 1 --2- 172.21.15.77:0/3622639239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628f09e0a70 0x5628f09ef5d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8514002230 tx=0x7f8514009620 comp rx=0 tx=0).stop 2023-12-01T01:18:56.608 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.601+0000 7f852e5abb80 1 -- 172.21.15.77:0/3622639239 shutdown_connections 2023-12-01T01:18:56.609 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.601+0000 7f852e5abb80 1 --2- 172.21.15.77:0/3622639239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628f09e0a70 0x5628f09ef5d0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.609 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.601+0000 7f852e5abb80 1 --2- 172.21.15.77:0/3622639239 >> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] conn(0x5628f0a47140 0x5628f0a49560 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.609 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.601+0000 7f852e5abb80 1 --2- 172.21.15.77:0/3622639239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5628f09df3f0 0x5628f09ec070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.609 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.601+0000 7f852e5abb80 1 --2- 172.21.15.77:0/3622639239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628f09e00f0 0x5628f09ec5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.609 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.601+0000 7f84fb7fe700 0 client.15798 ms_handle_reset on v2:172.21.15.203:6838/2888221613 2023-12-01T01:18:56.609 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.601+0000 7f852e5abb80 1 -- 172.21.15.77:0/3622639239 >> 172.21.15.77:0/3622639239 conn(0x5628f093e0d0 msgr2=0x5628f09e3370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:56.609 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.601+0000 7f852e5abb80 1 -- 172.21.15.77:0/3622639239 shutdown_connections 2023-12-01T01:18:56.609 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.601+0000 7f852e5abb80 1 -- 172.21.15.77:0/3622639239 wait complete. 2023-12-01T01:18:56.609 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.601+0000 7f852e5abb80 20 client.15798 trim_cache size 0 max 16384 2023-12-01T01:18:56.609 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2023-12-01T01:18:56.610 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.602+0000 7f852241b700 1 -- 172.21.15.77:0/2944989987 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f85040a0520 msgr2=0x7f85040a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:56.610 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.602+0000 7f852241b700 1 --2- 172.21.15.77:0/2944989987 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f85040a0520 0x7f85040a29e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f850c00b580 tx=0x7f850c01a040 comp rx=0 tx=0).stop 2023-12-01T01:18:56.610 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.602+0000 7f852241b700 1 -- 172.21.15.77:0/2944989987 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f851c100d80 msgr2=0x7f851c196a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:56.610 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.602+0000 7f852241b700 1 --2- 172.21.15.77:0/2944989987 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f851c100d80 0x7f851c196a30 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f851000e970 tx=0x7f851000ec80 comp rx=0 tx=0).stop 2023-12-01T01:18:56.611 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.602+0000 7f852241b700 1 -- 172.21.15.77:0/2944989987 shutdown_connections 2023-12-01T01:18:56.611 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.602+0000 7f852241b700 1 --2- 172.21.15.77:0/2944989987 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f85040a0520 0x7f85040a29e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.611 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.602+0000 7f852241b700 1 --2- 172.21.15.77:0/2944989987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f851c102910 0x7f851c19dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.611 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.602+0000 7f852241b700 1 --2- 172.21.15.77:0/2944989987 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f851c100d80 0x7f851c196a30 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.611 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.602+0000 7f852241b700 1 --2- 172.21.15.77:0/2944989987 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f851c101f70 0x7f851c196f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.611 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.602+0000 7f852241b700 1 -- 172.21.15.77:0/2944989987 >> 172.21.15.77:0/2944989987 conn(0x7f851c0fc310 msgr2=0x7f851c0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:56.611 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.602+0000 7f852241b700 1 -- 172.21.15.77:0/2944989987 shutdown_connections 2023-12-01T01:18:56.611 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.602+0000 7f852241b700 1 -- 172.21.15.77:0/2944989987 wait complete. 2023-12-01T01:18:56.614 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:56 smithi077 ceph-mon[98867]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:18:56.614 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:56 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/3787081855' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:18:56.616 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-12-01T01:18:56.763 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.760+0000 7fed78f08700 1 -- 172.21.15.77:0/1734005494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed74101800 msgr2=0x7fed74103bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:56.763 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.760+0000 7fed78f08700 1 --2- 172.21.15.77:0/1734005494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed74101800 0x7fed74103bf0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fed64009990 tx=0x7fed64009ca0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.763 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.760+0000 7fed78f08700 1 -- 172.21.15.77:0/1734005494 shutdown_connections 2023-12-01T01:18:56.763 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.760+0000 7fed78f08700 1 --2- 172.21.15.77:0/1734005494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed74106a60 0x7fed74108ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.764 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.760+0000 7fed78f08700 1 --2- 172.21.15.77:0/1734005494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed74101800 0x7fed74103bf0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.764 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.760+0000 7fed78f08700 1 --2- 172.21.15.77:0/1734005494 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed74104130 0x7fed74106520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.764 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.760+0000 7fed78f08700 1 -- 172.21.15.77:0/1734005494 >> 172.21.15.77:0/1734005494 conn(0x7fed740fb1f0 msgr2=0x7fed740fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:56.764 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.760+0000 7fed78f08700 1 -- 172.21.15.77:0/1734005494 shutdown_connections 2023-12-01T01:18:56.764 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.760+0000 7fed78f08700 1 -- 172.21.15.77:0/1734005494 wait complete. 2023-12-01T01:18:56.764 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.761+0000 7fed78f08700 1 Processor -- start 2023-12-01T01:18:56.764 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.761+0000 7fed78f08700 1 -- start start 2023-12-01T01:18:56.764 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.761+0000 7fed78f08700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed74101800 0x7fed74196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.764 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.761+0000 7fed78f08700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed74104130 0x7fed74196fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.764 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.761+0000 7fed78f08700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed74106a60 0x7fed7419e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.764 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.761+0000 7fed78f08700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7419e5e0 con 0x7fed74106a60 2023-12-01T01:18:56.764 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.761+0000 7fed78f08700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7419e720 con 0x7fed74101800 2023-12-01T01:18:56.764 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.761+0000 7fed78f08700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7419e8b0 con 0x7fed74104130 2023-12-01T01:18:56.764 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.761+0000 7fed72d9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed74106a60 0x7fed7419e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.761+0000 7fed72d9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed74106a60 0x7fed7419e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60094/0 (socket says 172.21.15.77:60094) 2023-12-01T01:18:56.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.761+0000 7fed72d9d700 1 -- 172.21.15.77:0/3634810426 learned_addr learned my addr 172.21.15.77:0/3634810426 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:56.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.761+0000 7fed7259c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed74101800 0x7fed74196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.761+0000 7fed71d9b700 1 --2- 172.21.15.77:0/3634810426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed74104130 0x7fed74196fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.761+0000 7fed72d9d700 1 -- 172.21.15.77:0/3634810426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed74104130 msgr2=0x7fed74196fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:56.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.761+0000 7fed72d9d700 1 --2- 172.21.15.77:0/3634810426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed74104130 0x7fed74196fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.761+0000 7fed72d9d700 1 -- 172.21.15.77:0/3634810426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed74101800 msgr2=0x7fed74196a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:56.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.761+0000 7fed72d9d700 1 --2- 172.21.15.77:0/3634810426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed74101800 0x7fed74196a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.761+0000 7fed72d9d700 1 -- 172.21.15.77:0/3634810426 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed640095f0 con 0x7fed74106a60 2023-12-01T01:18:56.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.761+0000 7fed7259c700 1 --2- 172.21.15.77:0/3634810426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed74101800 0x7fed74196a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:18:56.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.761+0000 7fed72d9d700 1 --2- 172.21.15.77:0/3634810426 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed74106a60 0x7fed7419e010 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00ef30 tx=0x7fed6c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:56.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.761+0000 7fed637fe700 1 -- 172.21.15.77:0/3634810426 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6c018070 con 0x7fed74106a60 2023-12-01T01:18:56.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.762+0000 7fed637fe700 1 -- 172.21.15.77:0/3634810426 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed6c00f040 con 0x7fed74106a60 2023-12-01T01:18:56.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.762+0000 7fed78f08700 1 -- 172.21.15.77:0/3634810426 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed7419eb30 con 0x7fed74106a60 2023-12-01T01:18:56.766 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.762+0000 7fed637fe700 1 -- 172.21.15.77:0/3634810426 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6c00cd00 con 0x7fed74106a60 2023-12-01T01:18:56.766 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.762+0000 7fed78f08700 1 -- 172.21.15.77:0/3634810426 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed7419eff0 con 0x7fed74106a60 2023-12-01T01:18:56.766 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:56 smithi203 ceph-mon[103581]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:18:56.766 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:56 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/3787081855' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:18:56.768 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.763+0000 7fed637fe700 1 -- 172.21.15.77:0/3634810426 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fed6c003a00 con 0x7fed74106a60 2023-12-01T01:18:56.768 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.764+0000 7fed637fe700 1 --2- 172.21.15.77:0/3634810426 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fed5c0a0520 0x7fed5c0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.768 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.764+0000 7fed637fe700 1 -- 172.21.15.77:0/3634810426 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fed6c0cb470 con 0x7fed74106a60 2023-12-01T01:18:56.768 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.764+0000 7fed7259c700 1 --2- 172.21.15.77:0/3634810426 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fed5c0a0520 0x7fed5c0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.768 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.765+0000 7fed7259c700 1 --2- 172.21.15.77:0/3634810426 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fed5c0a0520 0x7fed5c0a29e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fed6400b2e0 tx=0x7fed64005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:56.768 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.765+0000 7fed85098b80 1 Processor -- start 2023-12-01T01:18:56.768 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.765+0000 7fed85098b80 1 -- start start 2023-12-01T01:18:56.768 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.766+0000 7fed85098b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bd2c6c0f0 0x562bd2c6c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.768 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.766+0000 7fed85098b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562bd2c6ca70 0x562bd2c6ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.766+0000 7fed85098b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bd2c6b3f0 0x562bd2c6b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.766+0000 7fed85098b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bd2bc2a10 con 0x562bd2c6c0f0 2023-12-01T01:18:56.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.766+0000 7fed85098b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bd2bc03d0 con 0x562bd2c6b3f0 2023-12-01T01:18:56.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.766+0000 7fed85098b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bd2bbbcc0 con 0x562bd2c6ca70 2023-12-01T01:18:56.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.766+0000 7fed7259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bd2c6c0f0 0x562bd2c6c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.766+0000 7fed7259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bd2c6c0f0 0x562bd2c6c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60098/0 (socket says 172.21.15.77:60098) 2023-12-01T01:18:56.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.766+0000 7fed71d9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562bd2c6ca70 0x562bd2c6ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.766+0000 7fed7259c700 1 -- 172.21.15.77:0/7774824 learned_addr learned my addr 172.21.15.77:0/7774824 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:56.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.766+0000 7fed72d9d700 1 --2- 172.21.15.77:0/7774824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bd2c6b3f0 0x562bd2c6b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.766+0000 7fed7259c700 1 -- 172.21.15.77:0/7774824 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562bd2c6ca70 msgr2=0x562bd2c6ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:56.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.766+0000 7fed7259c700 1 --2- 172.21.15.77:0/7774824 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562bd2c6ca70 0x562bd2c6ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.766+0000 7fed7259c700 1 -- 172.21.15.77:0/7774824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bd2c6b3f0 msgr2=0x562bd2c6b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:56.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.766+0000 7fed7259c700 1 --2- 172.21.15.77:0/7774824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bd2c6b3f0 0x562bd2c6b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.766+0000 7fed7259c700 1 -- 172.21.15.77:0/7774824 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bd29e9de0 con 0x562bd2c6c0f0 2023-12-01T01:18:56.770 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.767+0000 7fed7259c700 1 --2- 172.21.15.77:0/7774824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bd2c6c0f0 0x562bd2c6c530 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fed640057e0 tx=0x7fed64020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c56d3d8c78160b1d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:56.770 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.767+0000 7fed60ff9700 1 -- 172.21.15.77:0/7774824 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6402b070 con 0x562bd2c6c0f0 2023-12-01T01:18:56.770 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.767+0000 7fed60ff9700 1 -- 172.21.15.77:0/7774824 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed6401e040 con 0x562bd2c6c0f0 2023-12-01T01:18:56.770 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.767+0000 7fed60ff9700 1 -- 172.21.15.77:0/7774824 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6402f400 con 0x562bd2c6c0f0 2023-12-01T01:18:56.774 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.767+0000 7fed85098b80 1 -- 172.21.15.77:0/7774824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bd2c6c0f0 msgr2=0x562bd2c6c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:56.774 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.767+0000 7fed85098b80 1 --2- 172.21.15.77:0/7774824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bd2c6c0f0 0x562bd2c6c530 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fed640057e0 tx=0x7fed64020000 comp rx=0 tx=0).stop 2023-12-01T01:18:56.775 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.767+0000 7fed85098b80 1 -- 172.21.15.77:0/7774824 shutdown_connections 2023-12-01T01:18:56.775 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.767+0000 7fed85098b80 1 --2- 172.21.15.77:0/7774824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bd2c6b3f0 0x562bd2c6b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.775 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.767+0000 7fed85098b80 1 --2- 172.21.15.77:0/7774824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bd2c6c0f0 0x562bd2c6c530 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.775 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.767+0000 7fed85098b80 1 --2- 172.21.15.77:0/7774824 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562bd2c6ca70 0x562bd2c6ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.775 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.767+0000 7fed85098b80 1 -- 172.21.15.77:0/7774824 >> 172.21.15.77:0/7774824 conn(0x562bd2bca0d0 msgr2=0x562bd2c65940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:56.775 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.768+0000 7fed85098b80 1 -- 172.21.15.77:0/7774824 shutdown_connections 2023-12-01T01:18:56.775 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.768+0000 7fed85098b80 1 -- 172.21.15.77:0/7774824 wait complete. 2023-12-01T01:18:56.775 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.768+0000 7fed85098b80 1 -- start start 2023-12-01T01:18:56.775 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.768+0000 7fed85098b80 1 Processor -- start 2023-12-01T01:18:56.775 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.768+0000 7fed85098b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bd2c6b3f0 0x562bd2c85950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.775 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.768+0000 7fed85098b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562bd2c6c0f0 0x562bd2c85e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.776 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.768+0000 7fed85098b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bd2c6ca70 0x562bd2c8ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.776 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.768+0000 7fed85098b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bd2b58510 con 0x562bd2c6ca70 2023-12-01T01:18:56.776 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.768+0000 7fed85098b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bd2b581b0 con 0x562bd2c6b3f0 2023-12-01T01:18:56.776 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.768+0000 7fed85098b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bd2a3f920 con 0x562bd2c6c0f0 2023-12-01T01:18:56.776 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.768+0000 7fed85098b80 10 client.0 _ll_register_callbacks cb 0x562bd2aa5a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:18:56.776 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.768+0000 7fed72d9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bd2c6ca70 0x562bd2c8ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.776 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.768+0000 7fed72d9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bd2c6ca70 0x562bd2c8ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60108/0 (socket says 172.21.15.77:60108) 2023-12-01T01:18:56.776 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.768+0000 7fed72d9d700 1 -- 172.21.15.77:0/442914595 learned_addr learned my addr 172.21.15.77:0/442914595 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:56.776 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.768+0000 7fed71d9b700 1 --2- 172.21.15.77:0/442914595 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562bd2c6c0f0 0x562bd2c85e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.776 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.768+0000 7fed7259c700 1 --2- 172.21.15.77:0/442914595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bd2c6b3f0 0x562bd2c85950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.776 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.769+0000 7fed72d9d700 1 -- 172.21.15.77:0/442914595 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562bd2c6c0f0 msgr2=0x562bd2c85e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:56.776 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.769+0000 7fed72d9d700 1 --2- 172.21.15.77:0/442914595 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562bd2c6c0f0 0x562bd2c85e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.776 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.769+0000 7fed72d9d700 1 -- 172.21.15.77:0/442914595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bd2c6b3f0 msgr2=0x562bd2c85950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:56.776 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.769+0000 7fed72d9d700 1 --2- 172.21.15.77:0/442914595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bd2c6b3f0 0x562bd2c85950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.776 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.769+0000 7fed72d9d700 1 -- 172.21.15.77:0/442914595 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed74109ac0 con 0x562bd2c6ca70 2023-12-01T01:18:56.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.769+0000 7fed7259c700 1 --2- 172.21.15.77:0/442914595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bd2c6b3f0 0x562bd2c85950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:18:56.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.769+0000 7fed72d9d700 1 --2- 172.21.15.77:0/442914595 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bd2c6ca70 0x562bd2c8ced0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fed6c095870 tx=0x7fed6c004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b78cbb8b12a920a5 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:56.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.769+0000 7fed59ffb700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:18:56.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.769+0000 7fed59ffb700 1 -- 172.21.15.77:0/442914595 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6c0cb2b0 con 0x562bd2c6ca70 2023-12-01T01:18:56.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.769+0000 7fed59ffb700 1 -- 172.21.15.77:0/442914595 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fed6c099500 con 0x562bd2c6ca70 2023-12-01T01:18:56.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.769+0000 7fed85098b80 1 -- 172.21.15.77:0/442914595 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562bd2be6d10 con 0x562bd2c6ca70 2023-12-01T01:18:56.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.769+0000 7fed59ffb700 1 -- 172.21.15.77:0/442914595 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed6c00ce60 con 0x562bd2c6ca70 2023-12-01T01:18:56.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.769+0000 7fed85098b80 10 client.15810 fetch_fsmap learned FSMap version 29 2023-12-01T01:18:56.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.769+0000 7fed85098b80 1 -- 172.21.15.77:0/442914595 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x562bd29d2c60 con 0x562bd2c6ca70 2023-12-01T01:18:56.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.769+0000 7fed59ffb700 1 -- 172.21.15.77:0/442914595 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed6c0b5080 con 0x562bd2c6ca70 2023-12-01T01:18:56.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.769+0000 7fed59ffb700 1 -- 172.21.15.77:0/442914595 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fed6c0982f0 con 0x562bd2c6ca70 2023-12-01T01:18:56.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.769+0000 7fed85098b80 10 client.15810 fetch_fsmap finished waiting for FSMap version 29 2023-12-01T01:18:56.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.769+0000 7fed85098b80 10 client.15810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-01T01:18:56.777 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.769+0000 7fed85098b80 20 client.15810 populate_metadata read hostname 'smithi077' 2023-12-01T01:18:56.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.769+0000 7fed85098b80 1 --2- 172.21.15.77:0/442914595 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x562bd2c917b0 0x562bd2c93bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.769+0000 7fed85098b80 4 client.15810 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:18:56.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.769+0000 7fed85098b80 1 -- 172.21.15.77:0/442914595 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562bd2c39e10 con 0x562bd2c917b0 2023-12-01T01:18:56.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.770+0000 7fed7259c700 1 --2- 172.21.15.77:0/442914595 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x562bd2c917b0 0x562bd2c93bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.770+0000 7fed7259c700 1 --2- 172.21.15.77:0/442914595 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x562bd2c917b0 0x562bd2c93bd0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2726ad4a1bdba0d9 server_cookie=caa8eccd8604fc45 in_seq=0 out_seq=0 2023-12-01T01:18:56.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.770+0000 7fed59ffb700 10 client.15810 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:56.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.771+0000 7fed59ffb700 1 -- 172.21.15.77:0/442914595 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x562bd2c39e10 con 0x562bd2c917b0 2023-12-01T01:18:56.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.771+0000 7fed59ffb700 10 client.15810 handle_command_reply: tid=0 2023-12-01T01:18:56.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.771+0000 7fed85098b80 1 client.15810 shutdown 2023-12-01T01:18:56.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.771+0000 7fed85098b80 1 -- 172.21.15.77:0/442914595 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bd2c6ca70 msgr2=0x562bd2c8ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:56.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.771+0000 7fed85098b80 1 --2- 172.21.15.77:0/442914595 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bd2c6ca70 0x562bd2c8ced0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fed6c095870 tx=0x7fed6c004b50 comp rx=0 tx=0).stop 2023-12-01T01:18:56.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.771+0000 7fed85098b80 1 -- 172.21.15.77:0/442914595 shutdown_connections 2023-12-01T01:18:56.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.771+0000 7fed85098b80 1 --2- 172.21.15.77:0/442914595 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bd2c6ca70 0x562bd2c8ced0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.778 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.771+0000 7fed85098b80 1 --2- 172.21.15.77:0/442914595 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x562bd2c917b0 0x562bd2c93bd0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.771+0000 7fed85098b80 1 --2- 172.21.15.77:0/442914595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bd2c6b3f0 0x562bd2c85950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.771+0000 7fed85098b80 1 --2- 172.21.15.77:0/442914595 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562bd2c6c0f0 0x562bd2c85e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.771+0000 7fed85098b80 1 -- 172.21.15.77:0/442914595 >> 172.21.15.77:0/442914595 conn(0x562bd2bca0d0 msgr2=0x562bd2c650b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:56.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.771+0000 7fed59ffb700 0 client.15810 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:56.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.771+0000 7fed85098b80 1 -- 172.21.15.77:0/442914595 shutdown_connections 2023-12-01T01:18:56.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.771+0000 7fed85098b80 1 -- 172.21.15.77:0/442914595 wait complete. 2023-12-01T01:18:56.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.772+0000 7fed85098b80 20 client.15810 trim_cache size 0 max 16384 2023-12-01T01:18:56.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:56 smithi137 ceph-mon[103312]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:18:56.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:56 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/3787081855' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:18:56.791 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.784+0000 7fed85098b80 1 Processor -- start 2023-12-01T01:18:56.791 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.784+0000 7fed85098b80 1 -- start start 2023-12-01T01:18:56.791 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.784+0000 7fed85098b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bd2c6b3f0 0x562bd2c6ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.791 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.784+0000 7fed85098b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bd2c6c0f0 0x562bd2c8ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.791 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.784+0000 7fed85098b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562bd2c6ca70 0x562bd2c6f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.791 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.784+0000 7fed85098b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bd2c60660 con 0x562bd2c6b3f0 2023-12-01T01:18:56.791 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.784+0000 7fed85098b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bd2c72580 con 0x562bd2c6c0f0 2023-12-01T01:18:56.791 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.784+0000 7fed85098b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bd2c72ee0 con 0x562bd2c6ca70 2023-12-01T01:18:56.791 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.784+0000 7fed7259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bd2c6b3f0 0x562bd2c6ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.791 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.784+0000 7fed7259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bd2c6b3f0 0x562bd2c6ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60112/0 (socket says 172.21.15.77:60112) 2023-12-01T01:18:56.791 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.784+0000 7fed7259c700 1 -- 172.21.15.77:0/3610742818 learned_addr learned my addr 172.21.15.77:0/3610742818 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:56.791 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.784+0000 7fed72d9d700 1 --2- 172.21.15.77:0/3610742818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562bd2c6ca70 0x562bd2c6f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.792 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.784+0000 7fed71d9b700 1 --2- 172.21.15.77:0/3610742818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bd2c6c0f0 0x562bd2c8ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.792 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.785+0000 7fed7259c700 1 -- 172.21.15.77:0/3610742818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562bd2c6ca70 msgr2=0x562bd2c6f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:56.792 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.785+0000 7fed7259c700 1 --2- 172.21.15.77:0/3610742818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562bd2c6ca70 0x562bd2c6f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.792 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.785+0000 7fed7259c700 1 -- 172.21.15.77:0/3610742818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bd2c6c0f0 msgr2=0x562bd2c8ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:56.792 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.785+0000 7fed7259c700 1 --2- 172.21.15.77:0/3610742818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bd2c6c0f0 0x562bd2c8ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.792 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.785+0000 7fed7259c700 1 -- 172.21.15.77:0/3610742818 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bd2a10250 con 0x562bd2c6b3f0 2023-12-01T01:18:56.792 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.785+0000 7fed71d9b700 1 --2- 172.21.15.77:0/3610742818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bd2c6c0f0 0x562bd2c8ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:18:56.792 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.785+0000 7fed7259c700 1 --2- 172.21.15.77:0/3610742818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bd2c6b3f0 0x562bd2c6ae60 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fed64009ce0 tx=0x7fed6401a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85229e14b2e88388 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:56.792 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.785+0000 7fed617fa700 1 -- 172.21.15.77:0/3610742818 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6402b050 con 0x562bd2c6b3f0 2023-12-01T01:18:56.792 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.785+0000 7fed617fa700 1 -- 172.21.15.77:0/3610742818 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed6402c040 con 0x562bd2c6b3f0 2023-12-01T01:18:56.792 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.785+0000 7fed617fa700 1 -- 172.21.15.77:0/3610742818 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6402f620 con 0x562bd2c6b3f0 2023-12-01T01:18:56.792 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.785+0000 7fed85098b80 1 -- 172.21.15.77:0/3610742818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bd2c6b3f0 msgr2=0x562bd2c6ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:56.792 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.785+0000 7fed85098b80 1 --2- 172.21.15.77:0/3610742818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bd2c6b3f0 0x562bd2c6ae60 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fed64009ce0 tx=0x7fed6401a040 comp rx=0 tx=0).stop 2023-12-01T01:18:56.793 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.786+0000 7fed85098b80 1 -- 172.21.15.77:0/3610742818 shutdown_connections 2023-12-01T01:18:56.793 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.786+0000 7fed85098b80 1 --2- 172.21.15.77:0/3610742818 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562bd2c6ca70 0x562bd2c6f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.793 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.786+0000 7fed85098b80 1 --2- 172.21.15.77:0/3610742818 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bd2c6b3f0 0x562bd2c6ae60 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.793 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.786+0000 7fed85098b80 1 --2- 172.21.15.77:0/3610742818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bd2c6c0f0 0x562bd2c8ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.793 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.786+0000 7fed85098b80 1 -- 172.21.15.77:0/3610742818 >> 172.21.15.77:0/3610742818 conn(0x562bd2bca0d0 msgr2=0x562bd2c727d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:56.793 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.786+0000 7fed85098b80 1 -- 172.21.15.77:0/3610742818 shutdown_connections 2023-12-01T01:18:56.793 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.786+0000 7fed85098b80 1 -- 172.21.15.77:0/3610742818 wait complete. 2023-12-01T01:18:56.793 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.786+0000 7fed85098b80 1 -- start start 2023-12-01T01:18:56.793 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.786+0000 7fed85098b80 1 Processor -- start 2023-12-01T01:18:56.793 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.786+0000 7fed85098b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bd2c6b3f0 0x562bd2c889d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.793 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.786+0000 7fed85098b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bd2c6c0f0 0x562bd2c88f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.793 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.786+0000 7fed85098b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562bd2c6ca70 0x562bd2c89f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.793 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.786+0000 7fed85098b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bd2c8a460 con 0x562bd2c6c0f0 2023-12-01T01:18:56.793 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.786+0000 7fed85098b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bd2c8a5d0 con 0x562bd2c6b3f0 2023-12-01T01:18:56.794 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.786+0000 7fed85098b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bd2c8a790 con 0x562bd2c6ca70 2023-12-01T01:18:56.794 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.786+0000 7fed85098b80 10 client.0 _ll_register_callbacks cb 0x7fed6c098940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:18:56.794 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.786+0000 7fed71d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bd2c6c0f0 0x562bd2c88f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.794 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.786+0000 7fed71d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bd2c6c0f0 0x562bd2c88f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60120/0 (socket says 172.21.15.77:60120) 2023-12-01T01:18:56.794 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.786+0000 7fed71d9b700 1 -- 172.21.15.77:0/1347363716 learned_addr learned my addr 172.21.15.77:0/1347363716 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:56.794 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.786+0000 7fed7259c700 1 --2- 172.21.15.77:0/1347363716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bd2c6b3f0 0x562bd2c889d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.795 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.786+0000 7fed72d9d700 1 --2- 172.21.15.77:0/1347363716 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562bd2c6ca70 0x562bd2c89f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.795 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.787+0000 7fed71d9b700 1 -- 172.21.15.77:0/1347363716 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562bd2c6ca70 msgr2=0x562bd2c89f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:56.795 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.787+0000 7fed71d9b700 1 --2- 172.21.15.77:0/1347363716 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562bd2c6ca70 0x562bd2c89f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.795 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.787+0000 7fed71d9b700 1 -- 172.21.15.77:0/1347363716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bd2c6b3f0 msgr2=0x562bd2c889d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:56.795 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.787+0000 7fed71d9b700 1 --2- 172.21.15.77:0/1347363716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bd2c6b3f0 0x562bd2c889d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.795 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.787+0000 7fed71d9b700 1 -- 172.21.15.77:0/1347363716 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562bd29e9de0 con 0x562bd2c6c0f0 2023-12-01T01:18:56.795 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.787+0000 7fed72d9d700 1 --2- 172.21.15.77:0/1347363716 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562bd2c6ca70 0x562bd2c89f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:18:56.796 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.787+0000 7fed71d9b700 1 --2- 172.21.15.77:0/1347363716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bd2c6c0f0 0x562bd2c88f10 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fed68013ac0 tx=0x7fed68013dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ba36bcfbd83b40 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:56.796 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.787+0000 7fed59ffb700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:18:56.796 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.787+0000 7fed59ffb700 1 -- 172.21.15.77:0/1347363716 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed68018040 con 0x562bd2c6c0f0 2023-12-01T01:18:56.796 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.787+0000 7fed59ffb700 1 -- 172.21.15.77:0/1347363716 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fed68004520 con 0x562bd2c6c0f0 2023-12-01T01:18:56.796 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.787+0000 7fed85098b80 1 -- 172.21.15.77:0/1347363716 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed6401e040 con 0x562bd2c6c0f0 2023-12-01T01:18:56.796 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.787+0000 7fed59ffb700 1 -- 172.21.15.77:0/1347363716 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed6800e820 con 0x562bd2c6c0f0 2023-12-01T01:18:56.796 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.787+0000 7fed85098b80 10 client.15816 fetch_fsmap learned FSMap version 29 2023-12-01T01:18:56.796 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.787+0000 7fed85098b80 1 -- 172.21.15.77:0/1347363716 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x562bd29e82b0 con 0x562bd2c6c0f0 2023-12-01T01:18:56.796 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.787+0000 7fed59ffb700 1 -- 172.21.15.77:0/1347363716 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed6401e040 con 0x562bd2c6c0f0 2023-12-01T01:18:56.796 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.787+0000 7fed59ffb700 1 -- 172.21.15.77:0/1347363716 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fed6801f420 con 0x562bd2c6c0f0 2023-12-01T01:18:56.796 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.787+0000 7fed85098b80 10 client.15816 fetch_fsmap finished waiting for FSMap version 29 2023-12-01T01:18:56.796 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.787+0000 7fed85098b80 10 client.15816 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-01T01:18:56.797 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.787+0000 7fed85098b80 20 client.15816 populate_metadata read hostname 'smithi077' 2023-12-01T01:18:56.797 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.787+0000 7fed85098b80 1 --2- 172.21.15.77:0/1347363716 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x562bd2ce4220 0x562bd2ce6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:56.797 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.787+0000 7fed85098b80 4 client.15816 mds_command: new command op to 24500 tid=0[{"prefix": "damage ls"}] 2023-12-01T01:18:56.797 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.787+0000 7fed85098b80 1 -- 172.21.15.77:0/1347363716 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562bd2bb4d40 con 0x562bd2ce4220 2023-12-01T01:18:56.797 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.788+0000 7fed7259c700 1 --2- 172.21.15.77:0/1347363716 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x562bd2ce4220 0x562bd2ce6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:56.797 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.788+0000 7fed59ffb700 10 client.15816 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:56.797 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.788+0000 7fed7259c700 1 --2- 172.21.15.77:0/1347363716 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x562bd2ce4220 0x562bd2ce6640 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f62a682adb99d20 server_cookie=1be92d39e38c1f0c in_seq=0 out_seq=0 2023-12-01T01:18:56.797 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.789+0000 7fed59ffb700 1 -- 172.21.15.77:0/1347363716 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562bd2bb4d40 con 0x562bd2ce4220 2023-12-01T01:18:56.797 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.789+0000 7fed59ffb700 10 client.15816 handle_command_reply: tid=0 2023-12-01T01:18:56.797 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.789+0000 7fed85098b80 1 client.15816 shutdown 2023-12-01T01:18:56.797 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.789+0000 7fed85098b80 1 -- 172.21.15.77:0/1347363716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bd2c6c0f0 msgr2=0x562bd2c88f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:56.798 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.789+0000 7fed85098b80 1 --2- 172.21.15.77:0/1347363716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bd2c6c0f0 0x562bd2c88f10 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fed68013ac0 tx=0x7fed68013dd0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.798 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.789+0000 7fed85098b80 1 -- 172.21.15.77:0/1347363716 shutdown_connections 2023-12-01T01:18:56.798 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.789+0000 7fed85098b80 1 --2- 172.21.15.77:0/1347363716 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562bd2c6ca70 0x562bd2c89f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.798 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.789+0000 7fed85098b80 1 --2- 172.21.15.77:0/1347363716 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x562bd2ce4220 0x562bd2ce6640 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.798 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.789+0000 7fed85098b80 1 --2- 172.21.15.77:0/1347363716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562bd2c6b3f0 0x562bd2c889d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.798 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.789+0000 7fed59ffb700 0 client.15816 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:56.798 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.789+0000 7fed85098b80 1 --2- 172.21.15.77:0/1347363716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bd2c6c0f0 0x562bd2c88f10 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.798 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.789+0000 7fed85098b80 1 -- 172.21.15.77:0/1347363716 >> 172.21.15.77:0/1347363716 conn(0x562bd2bca0d0 msgr2=0x562bd2c6f310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:56.798 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.789+0000 7fed85098b80 1 -- 172.21.15.77:0/1347363716 shutdown_connections 2023-12-01T01:18:56.798 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.789+0000 7fed85098b80 1 -- 172.21.15.77:0/1347363716 wait complete. 2023-12-01T01:18:56.798 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.789+0000 7fed85098b80 20 client.15816 trim_cache size 0 max 16384 2023-12-01T01:18:56.799 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2023-12-01T01:18:56.799 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.789+0000 7fed78f08700 1 -- 172.21.15.77:0/3634810426 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fed5c0a0520 msgr2=0x7fed5c0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:56.799 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.789+0000 7fed78f08700 1 --2- 172.21.15.77:0/3634810426 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fed5c0a0520 0x7fed5c0a29e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fed6400b2e0 tx=0x7fed64005fb0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.799 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.789+0000 7fed78f08700 1 -- 172.21.15.77:0/3634810426 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed74106a60 msgr2=0x7fed7419e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:56.799 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.789+0000 7fed78f08700 1 --2- 172.21.15.77:0/3634810426 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed74106a60 0x7fed7419e010 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00ef30 tx=0x7fed6c00c3a0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.800 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.789+0000 7fed78f08700 1 -- 172.21.15.77:0/3634810426 shutdown_connections 2023-12-01T01:18:56.800 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.789+0000 7fed78f08700 1 --2- 172.21.15.77:0/3634810426 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fed5c0a0520 0x7fed5c0a29e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.800 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.789+0000 7fed78f08700 1 --2- 172.21.15.77:0/3634810426 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fed74106a60 0x7fed7419e010 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.800 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.789+0000 7fed78f08700 1 --2- 172.21.15.77:0/3634810426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed74101800 0x7fed74196a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.800 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.789+0000 7fed78f08700 1 --2- 172.21.15.77:0/3634810426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed74104130 0x7fed74196fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:56.800 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.789+0000 7fed78f08700 1 -- 172.21.15.77:0/3634810426 >> 172.21.15.77:0/3634810426 conn(0x7fed740fb1f0 msgr2=0x7fed74102470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:56.800 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.789+0000 7fed78f08700 1 -- 172.21.15.77:0/3634810426 shutdown_connections 2023-12-01T01:18:56.800 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:56.789+0000 7fed78f08700 1 -- 172.21.15.77:0/3634810426 wait complete. 2023-12-01T01:18:56.990 INFO:tasks.workunit.client.0.smithi077.stderr:Updating files: 11% (491/4217) Updating files: 12% (507/4217) Updating files: 13% (549/4217) Updating files: 13% (585/4217) Updating files: 14% (591/4217) Updating files: 15% (633/4217) Updating files: 16% (675/4217) Updating files: 17% (717/4217) Updating files: 18% (760/4217) Updating files: 19% (802/4217) Updating files: 20% (844/4217) Updating files: 21% (886/4217) Updating files: 22% (928/4217) Updating files: 23% (970/4217) Updating files: 24% (1013/4217) Updating files: 25% (1055/4217) Updating files: 26% (1097/4217) Updating files: 27% (1139/4217) Updating files: 28% (1181/4217) Updating files: 29% (1223/4217) Updating files: 29% (1254/4217) Updating files: 30% (1266/4217) Updating files: 31% (1308/4217) Updating files: 32% (1350/4217) Updating files: 33% (1392/4217) Updating files: 34% (1434/4217) Updating files: 35% (1476/4217) Updating files: 36% (1519/4217) Updating files: 37% (1561/4217) Updating files: 38% (1603/4217) Updating files: 39% (1645/4217) Updating files: 40% (1687/4217) Updating files: 41% (1729/4217) Updating files: 42% (1772/4217) Updating files: 43% (1814/4217) Updating files: 44% (1856/4217) Updating files: 45% (1898/4217) Updating files: 45% (1903/4217) Updating files: 46% (1940/4217) Updating files: 47% (1982/4217) Updating files: 48% (2025/4217) Updating files: 49% (2067/4217) Updating files: 50% (2109/4217) Updating files: 51% (2151/4217) Updating files: 52% (2193/4217) Updating files: 53% (2236/4217) Updating files: 54% (2278/4217) Updating files: 55% (2320/4217) Updating files: 56% (2362/4217) Updating files: 57% (2404/4217) Updating files: 58% (2446/4217) Updating files: 59% (2489/4217) Updating files: 60% (2531/4217) Updating files: 61% (2573/4217) Updating files: 62% (2615/4217) Updating files: 63% (2657/4217) Updating files: 64% (2699/4217) Updating files: 64% (2702/4217) Updating files: 64% (2730/4217) Updating files: 65% (2742/4217) Updating files: 66% (2784/4217) Updating files: 67% (2826/4217) Updating files: 68% (2868/4217) Updating files: 69% (2910/4217) Updating files: 70% (2952/4217) Updating files: 71% (2995/4217) Updating files: 71% (2997/4217) Updating files: 72% (3037/4217) Updating files: 73% (3079/4217) Updating files: 74% (3121/4217) Updating files: 75% (3163/4217) Updating files: 76% (3205/4217) Updating files: 77% (3248/4217) Updating files: 78% (3290/4217) Updating files: 79% (3332/4217) Updating files: 80% (3374/4217) Updating files: 81% (3416/4217) Updating files: 82% (3458/4217) Updating files: 83% (3501/4217) Updating files: 84% (3543/4217) Updating files: 85% (3585/4217) Updating files: 86% (3627/4217) Updating files: 87% (3669/4217) Updating files: 88% (3711/4217) Updating files: 89% (3754/4217) Updating files: 90% (3796/4217) Updating files: 91% (3838/4217) Updating files: 92% (3880/4217) Updating files: 93% (3922/4217) Updating files: 93% (3935/4217) Updating files: 94% (3964/4217) Updating files: 95% (4007/4217) Updating files: 96% (4049/4217) Updating files: 97% (4091/4217) Updating files: 98% (4133/4217) Updating files: 99% (4175/4217) Updating files: 100% (4217/4217) Updating files: 100% (4217/4217), done. 2023-12-01T01:18:56.999 INFO:tasks.workunit.client.0.smithi077.stderr:+ cd xfstests-dev 2023-12-01T01:18:56.999 INFO:tasks.workunit.client.0.smithi077.stderr:+ git checkout 12973fc04fd10d4af086901e10ffa8e48866b735 2023-12-01T01:18:57.652 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.651+0000 7fc0ca620700 1 -- 172.21.15.77:0/87717248 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0c4100e90 msgr2=0x7fc0c41012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.651+0000 7fc0ca620700 1 --2- 172.21.15.77:0/87717248 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0c4100e90 0x7fc0c41012f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fc0b80099c0 tx=0x7fc0b8009cd0 comp rx=0 tx=0).stop 2023-12-01T01:18:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.651+0000 7fc0ca620700 1 -- 172.21.15.77:0/87717248 shutdown_connections 2023-12-01T01:18:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.651+0000 7fc0ca620700 1 --2- 172.21.15.77:0/87717248 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0c4101830 0x7fc0c41080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.651+0000 7fc0ca620700 1 --2- 172.21.15.77:0/87717248 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0c40ffca0 0x7fc0c41000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.651+0000 7fc0ca620700 1 --2- 172.21.15.77:0/87717248 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0c4100e90 0x7fc0c41012f0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.651+0000 7fc0ca620700 1 -- 172.21.15.77:0/87717248 >> 172.21.15.77:0/87717248 conn(0x7fc0c40fb230 msgr2=0x7fc0c40fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.651+0000 7fc0ca620700 1 -- 172.21.15.77:0/87717248 shutdown_connections 2023-12-01T01:18:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.651+0000 7fc0ca620700 1 -- 172.21.15.77:0/87717248 wait complete. 2023-12-01T01:18:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.651+0000 7fc0ca620700 1 Processor -- start 2023-12-01T01:18:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.651+0000 7fc0ca620700 1 -- start start 2023-12-01T01:18:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.652+0000 7fc0ca620700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0c40ffca0 0x7fc0c4102270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.652+0000 7fc0ca620700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0c4100e90 0x7fc0c41027b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.652+0000 7fc0ca620700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0c4101830 0x7fc0c4102cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.652+0000 7fc0ca620700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c41a0ac0 con 0x7fc0c40ffca0 2023-12-01T01:18:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.652+0000 7fc0ca620700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c41a0c00 con 0x7fc0c4101830 2023-12-01T01:18:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.652+0000 7fc0ca620700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c41a0d40 con 0x7fc0c4100e90 2023-12-01T01:18:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.652+0000 7fc0c3fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0c40ffca0 0x7fc0c4102270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.652+0000 7fc0c3fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0c40ffca0 0x7fc0c4102270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60132/0 (socket says 172.21.15.77:60132) 2023-12-01T01:18:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.652+0000 7fc0c3fff700 1 -- 172.21.15.77:0/3355842111 learned_addr learned my addr 172.21.15.77:0/3355842111 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.652+0000 7fc0c8bbd700 1 --2- 172.21.15.77:0/3355842111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0c4101830 0x7fc0c4102cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.652+0000 7fc0c37fe700 1 --2- 172.21.15.77:0/3355842111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0c4100e90 0x7fc0c41027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.652+0000 7fc0c3fff700 1 -- 172.21.15.77:0/3355842111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0c4100e90 msgr2=0x7fc0c41027b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.652+0000 7fc0c3fff700 1 --2- 172.21.15.77:0/3355842111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0c4100e90 0x7fc0c41027b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.652+0000 7fc0c3fff700 1 -- 172.21.15.77:0/3355842111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0c4101830 msgr2=0x7fc0c4102cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.652+0000 7fc0c3fff700 1 --2- 172.21.15.77:0/3355842111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0c4101830 0x7fc0c4102cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.652+0000 7fc0c3fff700 1 -- 172.21.15.77:0/3355842111 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b80095f0 con 0x7fc0c40ffca0 2023-12-01T01:18:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.652+0000 7fc0c3fff700 1 --2- 172.21.15.77:0/3355842111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0c40ffca0 0x7fc0c4102270 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc0b400e950 tx=0x7fc0b400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.652+0000 7fc0c17fa700 1 -- 172.21.15.77:0/3355842111 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4009770 con 0x7fc0c40ffca0 2023-12-01T01:18:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.652+0000 7fc0c17fa700 1 -- 172.21.15.77:0/3355842111 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0b400c7e0 con 0x7fc0c40ffca0 2023-12-01T01:18:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.653+0000 7fc0ca620700 1 -- 172.21.15.77:0/3355842111 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0c41a0fd0 con 0x7fc0c40ffca0 2023-12-01T01:18:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.653+0000 7fc0c17fa700 1 -- 172.21.15.77:0/3355842111 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4003cf0 con 0x7fc0c40ffca0 2023-12-01T01:18:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.653+0000 7fc0ca620700 1 -- 172.21.15.77:0/3355842111 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0c41a13f0 con 0x7fc0c40ffca0 2023-12-01T01:18:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.655+0000 7fc0c17fa700 1 -- 172.21.15.77:0/3355842111 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fc0b400c950 con 0x7fc0c40ffca0 2023-12-01T01:18:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.655+0000 7fc0c17fa700 1 --2- 172.21.15.77:0/3355842111 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc0ac0a05a0 0x7fc0ac0a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.655+0000 7fc0c17fa700 1 -- 172.21.15.77:0/3355842111 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fc0b4014070 con 0x7fc0c40ffca0 2023-12-01T01:18:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.655+0000 7fc0c37fe700 1 --2- 172.21.15.77:0/3355842111 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc0ac0a05a0 0x7fc0ac0a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.656+0000 7fc0c37fe700 1 --2- 172.21.15.77:0/3355842111 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc0ac0a05a0 0x7fc0ac0a2a60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8009990 tx=0x7fc0b801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.656+0000 7fc0d67b0b80 1 Processor -- start 2023-12-01T01:18:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.656+0000 7fc0d67b0b80 1 -- start start 2023-12-01T01:18:57.663 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.656+0000 7fc0d67b0b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565043519680 0x565043519aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.656+0000 7fc0d67b0b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565043519fe0 0x56504351a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.656+0000 7fc0d67b0b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56504351aed0 0x56504351b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.656+0000 7fc0d67b0b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650434719b0 con 0x56504351aed0 2023-12-01T01:18:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.656+0000 7fc0d67b0b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56504346f370 con 0x565043519680 2023-12-01T01:18:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.656+0000 7fc0d67b0b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56504346ac60 con 0x565043519fe0 2023-12-01T01:18:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.656+0000 7fc0c37fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565043519fe0 0x56504351a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.656+0000 7fc0c8bbd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56504351aed0 0x56504351b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.656+0000 7fc0c37fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565043519fe0 0x56504351a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:46330/0 (socket says 172.21.15.77:46330) 2023-12-01T01:18:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.657+0000 7fc0c37fe700 1 -- 172.21.15.77:0/470171125 learned_addr learned my addr 172.21.15.77:0/470171125 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.657+0000 7fc0c3fff700 1 --2- 172.21.15.77:0/470171125 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565043519680 0x565043519aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.657+0000 7fc0c8bbd700 1 -- 172.21.15.77:0/470171125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565043519fe0 msgr2=0x56504351a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.657+0000 7fc0c8bbd700 1 --2- 172.21.15.77:0/470171125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565043519fe0 0x56504351a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.657+0000 7fc0c8bbd700 1 -- 172.21.15.77:0/470171125 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565043519680 msgr2=0x565043519aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.657+0000 7fc0c8bbd700 1 --2- 172.21.15.77:0/470171125 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565043519680 0x565043519aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.657+0000 7fc0c8bbd700 1 -- 172.21.15.77:0/470171125 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565043298d80 con 0x56504351aed0 2023-12-01T01:18:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.657+0000 7fc0c3fff700 1 --2- 172.21.15.77:0/470171125 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565043519680 0x565043519aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:18:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.657+0000 7fc0c37fe700 1 --2- 172.21.15.77:0/470171125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565043519fe0 0x56504351a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:18:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.657+0000 7fc0c8bbd700 1 --2- 172.21.15.77:0/470171125 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56504351aed0 0x56504351b350 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc011580 tx=0x7fc0bc009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b25de185198d0398 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.657+0000 7fc0aa7fc700 1 -- 172.21.15.77:0/470171125 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc00e040 con 0x56504351aed0 2023-12-01T01:18:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.657+0000 7fc0aa7fc700 1 -- 172.21.15.77:0/470171125 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0bc00f040 con 0x56504351aed0 2023-12-01T01:18:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.657+0000 7fc0aa7fc700 1 -- 172.21.15.77:0/470171125 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc0043f0 con 0x56504351aed0 2023-12-01T01:18:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.658+0000 7fc0d67b0b80 1 -- 172.21.15.77:0/470171125 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56504351aed0 msgr2=0x56504351b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.658+0000 7fc0d67b0b80 1 --2- 172.21.15.77:0/470171125 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56504351aed0 0x56504351b350 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc011580 tx=0x7fc0bc009920 comp rx=0 tx=0).stop 2023-12-01T01:18:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.658+0000 7fc0d67b0b80 1 -- 172.21.15.77:0/470171125 shutdown_connections 2023-12-01T01:18:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.658+0000 7fc0d67b0b80 1 --2- 172.21.15.77:0/470171125 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56504351aed0 0x56504351b350 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.658+0000 7fc0d67b0b80 1 --2- 172.21.15.77:0/470171125 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565043519680 0x565043519aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.658+0000 7fc0d67b0b80 1 --2- 172.21.15.77:0/470171125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565043519fe0 0x56504351a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.658+0000 7fc0d67b0b80 1 -- 172.21.15.77:0/470171125 >> 172.21.15.77:0/470171125 conn(0x565043479070 msgr2=0x565043516040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.658+0000 7fc0d67b0b80 1 -- 172.21.15.77:0/470171125 shutdown_connections 2023-12-01T01:18:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.658+0000 7fc0d67b0b80 1 -- 172.21.15.77:0/470171125 wait complete. 2023-12-01T01:18:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.658+0000 7fc0d67b0b80 1 -- start start 2023-12-01T01:18:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.658+0000 7fc0d67b0b80 1 Processor -- start 2023-12-01T01:18:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.659+0000 7fc0d67b0b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565043519680 0x565043523430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.659+0000 7fc0d67b0b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565043519fe0 0x565043523970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.659+0000 7fc0d67b0b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56504351aed0 0x56504352a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.659+0000 7fc0d67b0b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650434074b0 con 0x565043519fe0 2023-12-01T01:18:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.659+0000 7fc0d67b0b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x565043407150 con 0x565043519680 2023-12-01T01:18:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.659+0000 7fc0d67b0b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650432ee8c0 con 0x56504351aed0 2023-12-01T01:18:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.659+0000 7fc0d67b0b80 10 client.0 _ll_register_callbacks cb 0x565043354a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:18:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.659+0000 7fc0c37fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565043519fe0 0x565043523970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.659+0000 7fc0c37fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565043519fe0 0x565043523970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60148/0 (socket says 172.21.15.77:60148) 2023-12-01T01:18:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.659+0000 7fc0c37fe700 1 -- 172.21.15.77:0/1747948039 learned_addr learned my addr 172.21.15.77:0/1747948039 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.659+0000 7fc0c8bbd700 1 --2- 172.21.15.77:0/1747948039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56504351aed0 0x56504352a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.659+0000 7fc0c3fff700 1 --2- 172.21.15.77:0/1747948039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565043519680 0x565043523430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.659+0000 7fc0c37fe700 1 -- 172.21.15.77:0/1747948039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56504351aed0 msgr2=0x56504352a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.659+0000 7fc0c37fe700 1 --2- 172.21.15.77:0/1747948039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56504351aed0 0x56504352a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.659+0000 7fc0c37fe700 1 -- 172.21.15.77:0/1747948039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565043519680 msgr2=0x565043523430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.659+0000 7fc0c37fe700 1 --2- 172.21.15.77:0/1747948039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565043519680 0x565043523430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.659+0000 7fc0c37fe700 1 -- 172.21.15.77:0/1747948039 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0c4108a30 con 0x565043519fe0 2023-12-01T01:18:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.659+0000 7fc0c37fe700 1 --2- 172.21.15.77:0/1747948039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565043519fe0 0x565043523970 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fc0b80058e0 tx=0x7fc0b8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a3234c5a386aae3 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.659+0000 7fc0a37fe700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:18:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.659+0000 7fc0a37fe700 1 -- 172.21.15.77:0/1747948039 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b802a050 con 0x565043519fe0 2023-12-01T01:18:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.659+0000 7fc0a37fe700 1 -- 172.21.15.77:0/1747948039 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fc0b8030460 con 0x565043519fe0 2023-12-01T01:18:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.659+0000 7fc0d67b0b80 1 -- 172.21.15.77:0/1747948039 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565043495cb0 con 0x565043519fe0 2023-12-01T01:18:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.660+0000 7fc0a37fe700 1 -- 172.21.15.77:0/1747948039 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0b8030620 con 0x565043519fe0 2023-12-01T01:18:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.660+0000 7fc0d67b0b80 10 client.15828 fetch_fsmap learned FSMap version 29 2023-12-01T01:18:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.660+0000 7fc0d67b0b80 1 -- 172.21.15.77:0/1747948039 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fc0bc011230 con 0x565043519fe0 2023-12-01T01:18:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.660+0000 7fc0a37fe700 1 -- 172.21.15.77:0/1747948039 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0b803f020 con 0x565043519fe0 2023-12-01T01:18:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.660+0000 7fc0a37fe700 1 -- 172.21.15.77:0/1747948039 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc0b80246c0 con 0x565043519fe0 2023-12-01T01:18:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.660+0000 7fc0d67b0b80 10 client.15828 fetch_fsmap finished waiting for FSMap version 29 2023-12-01T01:18:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.660+0000 7fc0d67b0b80 10 client.15828 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-01T01:18:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.660+0000 7fc0d67b0b80 20 client.15828 populate_metadata read hostname 'smithi077' 2023-12-01T01:18:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.660+0000 7fc0d67b0b80 1 --2- 172.21.15.77:0/1747948039 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x56504352f330 0x565043531750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.660+0000 7fc0d67b0b80 4 client.15828 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:18:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.660+0000 7fc0d67b0b80 1 -- 172.21.15.77:0/1747948039 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5650434e8db0 con 0x56504352f330 2023-12-01T01:18:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.660+0000 7fc0c3fff700 1 --2- 172.21.15.77:0/1747948039 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x56504352f330 0x565043531750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.661+0000 7fc0a37fe700 10 client.15828 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.661+0000 7fc0c3fff700 1 --2- 172.21.15.77:0/1747948039 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x56504352f330 0x565043531750 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f188fff6ecab616 server_cookie=6cf3138e61474ab1 in_seq=0 out_seq=0 2023-12-01T01:18:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.661+0000 7fc0a37fe700 1 -- 172.21.15.77:0/1747948039 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5650434e8db0 con 0x56504352f330 2023-12-01T01:18:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.661+0000 7fc0a37fe700 10 client.15828 handle_command_reply: tid=0 2023-12-01T01:18:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.661+0000 7fc0d67b0b80 1 client.15828 shutdown 2023-12-01T01:18:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.661+0000 7fc0d67b0b80 1 -- 172.21.15.77:0/1747948039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565043519fe0 msgr2=0x565043523970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.661+0000 7fc0d67b0b80 1 --2- 172.21.15.77:0/1747948039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565043519fe0 0x565043523970 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fc0b80058e0 tx=0x7fc0b8009310 comp rx=0 tx=0).stop 2023-12-01T01:18:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.661+0000 7fc0d67b0b80 1 -- 172.21.15.77:0/1747948039 shutdown_connections 2023-12-01T01:18:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.661+0000 7fc0d67b0b80 1 --2- 172.21.15.77:0/1747948039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56504351aed0 0x56504352a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.661+0000 7fc0d67b0b80 1 --2- 172.21.15.77:0/1747948039 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x56504352f330 0x565043531750 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.661+0000 7fc0d67b0b80 1 --2- 172.21.15.77:0/1747948039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565043519680 0x565043523430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.661+0000 7fc0d67b0b80 1 --2- 172.21.15.77:0/1747948039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565043519fe0 0x565043523970 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.661+0000 7fc0a37fe700 0 client.15828 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.661+0000 7fc0d67b0b80 1 -- 172.21.15.77:0/1747948039 >> 172.21.15.77:0/1747948039 conn(0x565043479070 msgr2=0x5650435143c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.661+0000 7fc0d67b0b80 1 -- 172.21.15.77:0/1747948039 shutdown_connections 2023-12-01T01:18:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.662+0000 7fc0d67b0b80 1 -- 172.21.15.77:0/1747948039 wait complete. 2023-12-01T01:18:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.662+0000 7fc0d67b0b80 20 client.15828 trim_cache size 0 max 16384 2023-12-01T01:18:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.676+0000 7fc0d67b0b80 1 Processor -- start 2023-12-01T01:18:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.676+0000 7fc0d67b0b80 1 -- start start 2023-12-01T01:18:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.676+0000 7fc0d67b0b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565043519680 0x565043528960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.676+0000 7fc0d67b0b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565043519fe0 0x565043515c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.676+0000 7fc0d67b0b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56504351aed0 0x565043516160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.676+0000 7fc0d67b0b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x56504350c9f0 con 0x565043519680 2023-12-01T01:18:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.676+0000 7fc0d67b0b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56504350f500 con 0x565043519fe0 2023-12-01T01:18:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.676+0000 7fc0d67b0b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x565043522fa0 con 0x56504351aed0 2023-12-01T01:18:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.676+0000 7fc0c3fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565043519680 0x565043528960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.676+0000 7fc0c3fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565043519680 0x565043528960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60154/0 (socket says 172.21.15.77:60154) 2023-12-01T01:18:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.676+0000 7fc0c3fff700 1 -- 172.21.15.77:0/4079218753 learned_addr learned my addr 172.21.15.77:0/4079218753 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.676+0000 7fc0c8bbd700 1 --2- 172.21.15.77:0/4079218753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56504351aed0 0x565043516160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.676+0000 7fc0c37fe700 1 --2- 172.21.15.77:0/4079218753 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565043519fe0 0x565043515c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.676+0000 7fc0c3fff700 1 -- 172.21.15.77:0/4079218753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56504351aed0 msgr2=0x565043516160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.676+0000 7fc0c3fff700 1 --2- 172.21.15.77:0/4079218753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56504351aed0 0x565043516160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.676+0000 7fc0c3fff700 1 -- 172.21.15.77:0/4079218753 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565043519fe0 msgr2=0x565043515c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.676+0000 7fc0c3fff700 1 --2- 172.21.15.77:0/4079218753 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565043519fe0 0x565043515c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.676+0000 7fc0c3fff700 1 -- 172.21.15.77:0/4079218753 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565043281c00 con 0x565043519680 2023-12-01T01:18:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.676+0000 7fc0c3fff700 1 --2- 172.21.15.77:0/4079218753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565043519680 0x565043528960 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fc0b4095de0 tx=0x7fc0b4084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912c64002e034874 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.677+0000 7fc0aaffd700 1 -- 172.21.15.77:0/4079218753 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4099370 con 0x565043519680 2023-12-01T01:18:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.677+0000 7fc0aaffd700 1 -- 172.21.15.77:0/4079218753 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0b409d1a0 con 0x565043519680 2023-12-01T01:18:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.677+0000 7fc0aaffd700 1 -- 172.21.15.77:0/4079218753 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b40977a0 con 0x565043519680 2023-12-01T01:18:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.677+0000 7fc0d67b0b80 1 -- 172.21.15.77:0/4079218753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565043519680 msgr2=0x565043528960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.677+0000 7fc0d67b0b80 1 --2- 172.21.15.77:0/4079218753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565043519680 0x565043528960 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fc0b4095de0 tx=0x7fc0b4084040 comp rx=0 tx=0).stop 2023-12-01T01:18:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.677+0000 7fc0d67b0b80 1 -- 172.21.15.77:0/4079218753 shutdown_connections 2023-12-01T01:18:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.677+0000 7fc0d67b0b80 1 --2- 172.21.15.77:0/4079218753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56504351aed0 0x565043516160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.677+0000 7fc0d67b0b80 1 --2- 172.21.15.77:0/4079218753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565043519680 0x565043528960 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.677+0000 7fc0d67b0b80 1 --2- 172.21.15.77:0/4079218753 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565043519fe0 0x565043515c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.677+0000 7fc0d67b0b80 1 -- 172.21.15.77:0/4079218753 >> 172.21.15.77:0/4079218753 conn(0x565043479070 msgr2=0x565043521220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.677+0000 7fc0d67b0b80 1 -- 172.21.15.77:0/4079218753 shutdown_connections 2023-12-01T01:18:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.677+0000 7fc0d67b0b80 1 -- 172.21.15.77:0/4079218753 wait complete. 2023-12-01T01:18:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.677+0000 7fc0d67b0b80 1 -- start start 2023-12-01T01:18:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.678+0000 7fc0d67b0b80 1 Processor -- start 2023-12-01T01:18:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.678+0000 7fc0d67b0b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565043519680 0x565043579230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.678+0000 7fc0d67b0b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565043519fe0 0x565043579770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.678+0000 7fc0d67b0b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56504351aed0 0x5650435807b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.678+0000 7fc0d67b0b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x565043580cf0 con 0x565043519fe0 2023-12-01T01:18:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.678+0000 7fc0d67b0b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x565043580e30 con 0x565043519680 2023-12-01T01:18:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.678+0000 7fc0d67b0b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x565043580fc0 con 0x56504351aed0 2023-12-01T01:18:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.678+0000 7fc0d67b0b80 10 client.0 _ll_register_callbacks cb 0x565043354a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:18:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.678+0000 7fc0c37fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565043519fe0 0x565043579770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.678+0000 7fc0c37fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565043519fe0 0x565043579770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60162/0 (socket says 172.21.15.77:60162) 2023-12-01T01:18:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.678+0000 7fc0c37fe700 1 -- 172.21.15.77:0/1783499847 learned_addr learned my addr 172.21.15.77:0/1783499847 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.678+0000 7fc0c8bbd700 1 --2- 172.21.15.77:0/1783499847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56504351aed0 0x5650435807b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.678+0000 7fc0c3fff700 1 --2- 172.21.15.77:0/1783499847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565043519680 0x565043579230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.678+0000 7fc0c37fe700 1 -- 172.21.15.77:0/1783499847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56504351aed0 msgr2=0x5650435807b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.678+0000 7fc0c37fe700 1 --2- 172.21.15.77:0/1783499847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56504351aed0 0x5650435807b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.678+0000 7fc0c37fe700 1 -- 172.21.15.77:0/1783499847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565043519680 msgr2=0x565043579230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.678+0000 7fc0c37fe700 1 --2- 172.21.15.77:0/1783499847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565043519680 0x565043579230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.678+0000 7fc0c37fe700 1 -- 172.21.15.77:0/1783499847 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0c41a13f0 con 0x565043519fe0 2023-12-01T01:18:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.679+0000 7fc0c37fe700 1 --2- 172.21.15.77:0/1783499847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565043519fe0 0x565043579770 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8004820 tx=0x7fc0b800faf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9adcc67c3d657fe2 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.679+0000 7fc0a37fe700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:18:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.679+0000 7fc0a37fe700 1 -- 172.21.15.77:0/1783499847 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b802a040 con 0x565043519fe0 2023-12-01T01:18:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.679+0000 7fc0a37fe700 1 -- 172.21.15.77:0/1783499847 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fc0b8040080 con 0x565043519fe0 2023-12-01T01:18:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.679+0000 7fc0d67b0b80 1 -- 172.21.15.77:0/1783499847 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc0bc00f040 con 0x565043519fe0 2023-12-01T01:18:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.679+0000 7fc0a37fe700 1 -- 172.21.15.77:0/1783499847 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0b8044680 con 0x565043519fe0 2023-12-01T01:18:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.679+0000 7fc0d67b0b80 10 client.15834 fetch_fsmap learned FSMap version 29 2023-12-01T01:18:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.679+0000 7fc0d67b0b80 1 -- 172.21.15.77:0/1783499847 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5650432bf1f0 con 0x565043519fe0 2023-12-01T01:18:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.679+0000 7fc0a37fe700 1 -- 172.21.15.77:0/1783499847 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0bc00f040 con 0x565043519fe0 2023-12-01T01:18:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.679+0000 7fc0a37fe700 1 -- 172.21.15.77:0/1783499847 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc0b8030840 con 0x565043519fe0 2023-12-01T01:18:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.679+0000 7fc0d67b0b80 10 client.15834 fetch_fsmap finished waiting for FSMap version 29 2023-12-01T01:18:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.679+0000 7fc0d67b0b80 10 client.15834 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-01T01:18:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.679+0000 7fc0d67b0b80 20 client.15834 populate_metadata read hostname 'smithi077' 2023-12-01T01:18:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.679+0000 7fc0d67b0b80 1 --2- 172.21.15.77:0/1783499847 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x565043582630 0x565043584a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.679+0000 7fc0d67b0b80 4 client.15834 mds_command: new command op to 24500 tid=0[{"prefix": "flush journal"}] 2023-12-01T01:18:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.679+0000 7fc0d67b0b80 1 -- 172.21.15.77:0/1783499847 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x565043463ce0 con 0x565043582630 2023-12-01T01:18:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.679+0000 7fc0c3fff700 1 --2- 172.21.15.77:0/1783499847 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x565043582630 0x565043584a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.680+0000 7fc0a37fe700 10 client.15834 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.680+0000 7fc0c3fff700 1 --2- 172.21.15.77:0/1783499847 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x565043582630 0x565043584a50 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51b799d192ca057a server_cookie=9a2aa736e3e7e494 in_seq=0 out_seq=0 2023-12-01T01:18:57.803 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-12-01T01:18:57.803 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-12-01T01:18:57.878 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:57 smithi077 ceph-mon[98867]: pgmap v330: 97 pgs: 97 active+clean; 19 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 515 KiB/s wr, 136 op/s 2023-12-01T01:18:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.977+0000 7fc0a37fe700 1 -- 172.21.15.77:0/1783499847 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x565043463ce0 con 0x565043582630 2023-12-01T01:18:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.977+0000 7fc0a37fe700 10 client.15834 handle_command_reply: tid=0 2023-12-01T01:18:57.980 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.977+0000 7fc0d67b0b80 1 client.15834 shutdown 2023-12-01T01:18:57.980 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.978+0000 7fc0d67b0b80 1 -- 172.21.15.77:0/1783499847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565043519fe0 msgr2=0x565043579770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:57.980 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.978+0000 7fc0d67b0b80 1 --2- 172.21.15.77:0/1783499847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565043519fe0 0x565043579770 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8004820 tx=0x7fc0b800faf0 comp rx=0 tx=0).stop 2023-12-01T01:18:57.980 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.978+0000 7fc0d67b0b80 1 -- 172.21.15.77:0/1783499847 shutdown_connections 2023-12-01T01:18:57.980 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.978+0000 7fc0d67b0b80 1 --2- 172.21.15.77:0/1783499847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56504351aed0 0x5650435807b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:57.980 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.978+0000 7fc0d67b0b80 1 --2- 172.21.15.77:0/1783499847 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x565043582630 0x565043584a50 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:57.980 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.978+0000 7fc0d67b0b80 1 --2- 172.21.15.77:0/1783499847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565043519680 0x565043579230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:57.980 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.978+0000 7fc0a37fe700 0 client.15834 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:57.980 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.978+0000 7fc0d67b0b80 1 --2- 172.21.15.77:0/1783499847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x565043519fe0 0x565043579770 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:57.980 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.978+0000 7fc0d67b0b80 1 -- 172.21.15.77:0/1783499847 >> 172.21.15.77:0/1783499847 conn(0x565043479070 msgr2=0x565043515c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:57.980 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.978+0000 7fc0d67b0b80 1 -- 172.21.15.77:0/1783499847 shutdown_connections 2023-12-01T01:18:57.980 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.978+0000 7fc0d67b0b80 1 -- 172.21.15.77:0/1783499847 wait complete. 2023-12-01T01:18:57.980 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.978+0000 7fc0d67b0b80 20 client.15834 trim_cache size 0 max 16384 2023-12-01T01:18:57.981 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout:{ 2023-12-01T01:18:57.981 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout: "message": "", 2023-12-01T01:18:57.981 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout: "return_code": 0 2023-12-01T01:18:57.981 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout:} 2023-12-01T01:18:57.982 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.978+0000 7fc0ca620700 1 -- 172.21.15.77:0/3355842111 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc0ac0a05a0 msgr2=0x7fc0ac0a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:57.982 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.978+0000 7fc0ca620700 1 --2- 172.21.15.77:0/3355842111 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc0ac0a05a0 0x7fc0ac0a2a60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8009990 tx=0x7fc0b801a040 comp rx=0 tx=0).stop 2023-12-01T01:18:57.982 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.978+0000 7fc0ca620700 1 -- 172.21.15.77:0/3355842111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0c40ffca0 msgr2=0x7fc0c4102270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:57.982 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.978+0000 7fc0ca620700 1 --2- 172.21.15.77:0/3355842111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0c40ffca0 0x7fc0c4102270 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc0b400e950 tx=0x7fc0b400ec60 comp rx=0 tx=0).stop 2023-12-01T01:18:57.982 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.979+0000 7fc0ca620700 1 -- 172.21.15.77:0/3355842111 shutdown_connections 2023-12-01T01:18:57.982 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.979+0000 7fc0ca620700 1 --2- 172.21.15.77:0/3355842111 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc0ac0a05a0 0x7fc0ac0a2a60 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:57.982 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.979+0000 7fc0ca620700 1 --2- 172.21.15.77:0/3355842111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0c4101830 0x7fc0c4102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:57.982 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.979+0000 7fc0ca620700 1 --2- 172.21.15.77:0/3355842111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0c40ffca0 0x7fc0c4102270 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:57.982 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.979+0000 7fc0ca620700 1 --2- 172.21.15.77:0/3355842111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0c4100e90 0x7fc0c41027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:57.982 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.979+0000 7fc0ca620700 1 -- 172.21.15.77:0/3355842111 >> 172.21.15.77:0/3355842111 conn(0x7fc0c40fb230 msgr2=0x7fc0c4069090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:57.982 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.979+0000 7fc0ca620700 1 -- 172.21.15.77:0/3355842111 shutdown_connections 2023-12-01T01:18:57.982 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:18:57.979+0000 7fc0ca620700 1 -- 172.21.15.77:0/3355842111 wait complete. 2023-12-01T01:18:58.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:57 smithi203 ceph-mon[103581]: pgmap v330: 97 pgs: 97 active+clean; 19 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 515 KiB/s wr, 136 op/s 2023-12-01T01:18:58.022 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.019+0000 7fb1374d5700 1 -- 172.21.15.77:0/839086725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb130102040 msgr2=0x7fb1301024a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:58.022 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.019+0000 7fb1374d5700 1 --2- 172.21.15.77:0/839086725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb130102040 0x7fb1301024a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb124009990 tx=0x7fb124009ca0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.022 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.019+0000 7fb1374d5700 1 -- 172.21.15.77:0/839086725 shutdown_connections 2023-12-01T01:18:58.023 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.019+0000 7fb1374d5700 1 --2- 172.21.15.77:0/839086725 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1301029e0 0x7fb130109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.023 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.019+0000 7fb1374d5700 1 --2- 172.21.15.77:0/839086725 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1300ff070 0x7fb1300ff490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.023 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.019+0000 7fb1374d5700 1 --2- 172.21.15.77:0/839086725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb130102040 0x7fb1301024a0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.023 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.019+0000 7fb1374d5700 1 -- 172.21.15.77:0/839086725 >> 172.21.15.77:0/839086725 conn(0x7fb1300fa980 msgr2=0x7fb1300fcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:58.023 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.020+0000 7fb1374d5700 1 -- 172.21.15.77:0/839086725 shutdown_connections 2023-12-01T01:18:58.023 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.020+0000 7fb1374d5700 1 -- 172.21.15.77:0/839086725 wait complete. 2023-12-01T01:18:58.023 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.020+0000 7fb1374d5700 1 Processor -- start 2023-12-01T01:18:58.023 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.020+0000 7fb1374d5700 1 -- start start 2023-12-01T01:18:58.023 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.020+0000 7fb1374d5700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1300ff070 0x7fb130196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:58.023 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.020+0000 7fb1374d5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb130102040 0x7fb130196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:58.023 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.020+0000 7fb1374d5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1301029e0 0x7fb13019dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:58.023 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.020+0000 7fb1374d5700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13019e5c0 con 0x7fb1301029e0 2023-12-01T01:18:58.023 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.020+0000 7fb1374d5700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13019e700 con 0x7fb130102040 2023-12-01T01:18:58.023 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.020+0000 7fb1374d5700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13019e890 con 0x7fb1300ff070 2023-12-01T01:18:58.024 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.020+0000 7fb135a72700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1301029e0 0x7fb13019dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:58.024 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.020+0000 7fb135271700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1300ff070 0x7fb130196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:58.024 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.020+0000 7fb135271700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1300ff070 0x7fb130196a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:46362/0 (socket says 172.21.15.77:46362) 2023-12-01T01:18:58.024 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.020+0000 7fb135a72700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1301029e0 0x7fb13019dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60176/0 (socket says 172.21.15.77:60176) 2023-12-01T01:18:58.024 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.020+0000 7fb135271700 1 -- 172.21.15.77:0/1320113469 learned_addr learned my addr 172.21.15.77:0/1320113469 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:58.024 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.020+0000 7fb134a70700 1 --2- 172.21.15.77:0/1320113469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb130102040 0x7fb130196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:58.024 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.021+0000 7fb135a72700 1 -- 172.21.15.77:0/1320113469 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1300ff070 msgr2=0x7fb130196a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:58.024 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.021+0000 7fb135a72700 1 --2- 172.21.15.77:0/1320113469 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1300ff070 0x7fb130196a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.024 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.021+0000 7fb135a72700 1 -- 172.21.15.77:0/1320113469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb130102040 msgr2=0x7fb130196f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:58.024 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.021+0000 7fb135a72700 1 --2- 172.21.15.77:0/1320113469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb130102040 0x7fb130196f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.024 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.021+0000 7fb135a72700 1 -- 172.21.15.77:0/1320113469 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1240095f0 con 0x7fb1301029e0 2023-12-01T01:18:58.024 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.021+0000 7fb135a72700 1 --2- 172.21.15.77:0/1320113469 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1301029e0 0x7fb13019dff0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb12000d750 tx=0x7fb12000da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:58.024 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.021+0000 7fb12e7fc700 1 -- 172.21.15.77:0/1320113469 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb120004020 con 0x7fb1301029e0 2023-12-01T01:18:58.024 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.021+0000 7fb1374d5700 1 -- 172.21.15.77:0/1320113469 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb13019eb70 con 0x7fb1301029e0 2023-12-01T01:18:58.024 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.021+0000 7fb12e7fc700 1 -- 172.21.15.77:0/1320113469 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb120004830 con 0x7fb1301029e0 2023-12-01T01:18:58.025 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.021+0000 7fb12e7fc700 1 -- 172.21.15.77:0/1320113469 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb120005560 con 0x7fb1301029e0 2023-12-01T01:18:58.025 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.021+0000 7fb1374d5700 1 -- 172.21.15.77:0/1320113469 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb13019f030 con 0x7fb1301029e0 2023-12-01T01:18:58.025 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.023+0000 7fb12e7fc700 1 -- 172.21.15.77:0/1320113469 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fb1200049a0 con 0x7fb1301029e0 2023-12-01T01:18:58.025 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.024+0000 7fb12e7fc700 1 --2- 172.21.15.77:0/1320113469 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fb1180a0520 0x7fb1180a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:58.025 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.024+0000 7fb135271700 1 --2- 172.21.15.77:0/1320113469 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fb1180a0520 0x7fb1180a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:58.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.024+0000 7fb12e7fc700 1 -- 172.21.15.77:0/1320113469 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fb1200cb150 con 0x7fb1301029e0 2023-12-01T01:18:58.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.024+0000 7fb135271700 1 --2- 172.21.15.77:0/1320113469 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fb1180a0520 0x7fb1180a29e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb12800f5e0 tx=0x7fb12800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:58.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.025+0000 7fb143665b80 1 Processor -- start 2023-12-01T01:18:58.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.025+0000 7fb143665b80 1 -- start start 2023-12-01T01:18:58.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.025+0000 7fb143665b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597a3d52170 0x5597a3d525b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:58.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.025+0000 7fb143665b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597a3d52af0 0x5597a3d50ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:58.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.025+0000 7fb143665b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597a3d51470 0x5597a3d518f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:58.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.025+0000 7fb143665b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597a3ca8a90 con 0x5597a3d52170 2023-12-01T01:18:58.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.025+0000 7fb143665b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597a3ca6450 con 0x5597a3d51470 2023-12-01T01:18:58.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.025+0000 7fb143665b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597a3ca1d40 con 0x5597a3d52af0 2023-12-01T01:18:58.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.025+0000 7fb135271700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597a3d52170 0x5597a3d525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:58.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.025+0000 7fb134a70700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597a3d52af0 0x5597a3d50ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:58.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.025+0000 7fb135271700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597a3d52170 0x5597a3d525b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60182/0 (socket says 172.21.15.77:60182) 2023-12-01T01:18:58.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.025+0000 7fb135271700 1 -- 172.21.15.77:0/3884914839 learned_addr learned my addr 172.21.15.77:0/3884914839 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:58.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.025+0000 7fb135a72700 1 --2- 172.21.15.77:0/3884914839 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597a3d51470 0x5597a3d518f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:58.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.026+0000 7fb135271700 1 -- 172.21.15.77:0/3884914839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597a3d52af0 msgr2=0x5597a3d50ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:58.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.026+0000 7fb135271700 1 --2- 172.21.15.77:0/3884914839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597a3d52af0 0x5597a3d50ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.026+0000 7fb135271700 1 -- 172.21.15.77:0/3884914839 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597a3d51470 msgr2=0x5597a3d518f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:58.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.026+0000 7fb135271700 1 --2- 172.21.15.77:0/3884914839 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597a3d51470 0x5597a3d518f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.026+0000 7fb135271700 1 -- 172.21.15.77:0/3884914839 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597a3acfe60 con 0x5597a3d52170 2023-12-01T01:18:58.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.026+0000 7fb134a70700 1 --2- 172.21.15.77:0/3884914839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597a3d52af0 0x5597a3d50ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:18:58.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.026+0000 7fb135a72700 1 --2- 172.21.15.77:0/3884914839 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597a3d51470 0x5597a3d518f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:18:58.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.026+0000 7fb135271700 1 --2- 172.21.15.77:0/3884914839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597a3d52170 0x5597a3d525b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb1280177a0 tx=0x7fb128014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f02ad5955a0374 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:58.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.026+0000 7fb1177fe700 1 -- 172.21.15.77:0/3884914839 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12800e040 con 0x5597a3d52170 2023-12-01T01:18:58.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.026+0000 7fb1177fe700 1 -- 172.21.15.77:0/3884914839 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb12800f040 con 0x5597a3d52170 2023-12-01T01:18:58.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.026+0000 7fb1177fe700 1 -- 172.21.15.77:0/3884914839 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb128021c40 con 0x5597a3d52170 2023-12-01T01:18:58.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.027+0000 7fb143665b80 1 -- 172.21.15.77:0/3884914839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597a3d52170 msgr2=0x5597a3d525b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:58.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.027+0000 7fb143665b80 1 --2- 172.21.15.77:0/3884914839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597a3d52170 0x5597a3d525b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb1280177a0 tx=0x7fb128014000 comp rx=0 tx=0).stop 2023-12-01T01:18:58.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.027+0000 7fb143665b80 1 -- 172.21.15.77:0/3884914839 shutdown_connections 2023-12-01T01:18:58.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.027+0000 7fb143665b80 1 --2- 172.21.15.77:0/3884914839 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597a3d51470 0x5597a3d518f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.027+0000 7fb143665b80 1 --2- 172.21.15.77:0/3884914839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597a3d52170 0x5597a3d525b0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.027+0000 7fb143665b80 1 --2- 172.21.15.77:0/3884914839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597a3d52af0 0x5597a3d50ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.027+0000 7fb143665b80 1 -- 172.21.15.77:0/3884914839 >> 172.21.15.77:0/3884914839 conn(0x5597a3cb0150 msgr2=0x5597a3d4b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:58.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.027+0000 7fb143665b80 1 -- 172.21.15.77:0/3884914839 shutdown_connections 2023-12-01T01:18:58.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.027+0000 7fb143665b80 1 -- 172.21.15.77:0/3884914839 wait complete. 2023-12-01T01:18:58.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.027+0000 7fb143665b80 1 -- start start 2023-12-01T01:18:58.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.027+0000 7fb143665b80 1 Processor -- start 2023-12-01T01:18:58.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:57 smithi137 ceph-mon[103312]: pgmap v330: 97 pgs: 97 active+clean; 19 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 515 KiB/s wr, 136 op/s 2023-12-01T01:18:58.034 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.028+0000 7fb143665b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597a3d51470 0x5597a3d5aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:58.034 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.028+0000 7fb143665b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597a3d52170 0x5597a3d5afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:58.034 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.028+0000 7fb143665b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597a3d52af0 0x5597a3d62010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:58.034 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.028+0000 7fb143665b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597a3c3e590 con 0x5597a3d51470 2023-12-01T01:18:58.034 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.028+0000 7fb143665b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597a3c3e230 con 0x5597a3d52af0 2023-12-01T01:18:58.034 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.028+0000 7fb143665b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597a3b259a0 con 0x5597a3d52170 2023-12-01T01:18:58.034 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.028+0000 7fb143665b80 10 client.0 _ll_register_callbacks cb 0x5597a3b8bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:18:58.034 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.028+0000 7fb135271700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597a3d51470 0x5597a3d5aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:58.034 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.028+0000 7fb135271700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597a3d51470 0x5597a3d5aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60188/0 (socket says 172.21.15.77:60188) 2023-12-01T01:18:58.034 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.028+0000 7fb134a70700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597a3d52170 0x5597a3d5afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:58.035 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.028+0000 7fb135271700 1 -- 172.21.15.77:0/716450757 learned_addr learned my addr 172.21.15.77:0/716450757 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:58.035 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.028+0000 7fb135a72700 1 --2- 172.21.15.77:0/716450757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597a3d52af0 0x5597a3d62010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:58.035 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.028+0000 7fb134a70700 1 -- 172.21.15.77:0/716450757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597a3d52af0 msgr2=0x5597a3d62010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:58.035 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.028+0000 7fb134a70700 1 --2- 172.21.15.77:0/716450757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597a3d52af0 0x5597a3d62010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.035 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.028+0000 7fb134a70700 1 -- 172.21.15.77:0/716450757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597a3d51470 msgr2=0x5597a3d5aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:58.035 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.028+0000 7fb134a70700 1 --2- 172.21.15.77:0/716450757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597a3d51470 0x5597a3d5aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.035 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.028+0000 7fb134a70700 1 -- 172.21.15.77:0/716450757 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb130109b90 con 0x5597a3d52170 2023-12-01T01:18:58.035 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.028+0000 7fb135271700 1 --2- 172.21.15.77:0/716450757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597a3d51470 0x5597a3d5aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:18:58.035 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.028+0000 7fb135a72700 1 --2- 172.21.15.77:0/716450757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597a3d52af0 0x5597a3d62010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:18:58.035 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.028+0000 7fb134a70700 1 --2- 172.21.15.77:0/716450757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597a3d52170 0x5597a3d5afd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb124000c00 tx=0x7fb124005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31f382ee0c645955 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:58.035 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.028+0000 7fb114ff9700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2023-12-01T01:18:58.035 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.028+0000 7fb114ff9700 1 -- 172.21.15.77:0/716450757 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12401d070 con 0x5597a3d52170 2023-12-01T01:18:58.035 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.029+0000 7fb114ff9700 1 -- 172.21.15.77:0/716450757 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fb124004700 con 0x5597a3d52170 2023-12-01T01:18:58.035 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.029+0000 7fb143665b80 1 -- 172.21.15.77:0/716450757 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5597a3cccd90 con 0x5597a3d52170 2023-12-01T01:18:58.035 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.029+0000 7fb114ff9700 1 -- 172.21.15.77:0/716450757 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1240056f0 con 0x5597a3d52170 2023-12-01T01:18:58.035 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.029+0000 7fb143665b80 10 client.25441 fetch_fsmap learned FSMap version 29 2023-12-01T01:18:58.036 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.029+0000 7fb143665b80 1 -- 172.21.15.77:0/716450757 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fb128017480 con 0x5597a3d52170 2023-12-01T01:18:58.036 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.029+0000 7fb114ff9700 1 -- 172.21.15.77:0/716450757 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5597a3cccd90 con 0x5597a3d52170 2023-12-01T01:18:58.036 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.029+0000 7fb114ff9700 1 -- 172.21.15.77:0/716450757 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb12400fa00 con 0x5597a3d52170 2023-12-01T01:18:58.036 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.029+0000 7fb143665b80 10 client.25441 fetch_fsmap finished waiting for FSMap version 29 2023-12-01T01:18:58.036 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.029+0000 7fb143665b80 10 client.25441 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-01T01:18:58.036 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.029+0000 7fb143665b80 20 client.25441 populate_metadata read hostname 'smithi077' 2023-12-01T01:18:58.036 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.029+0000 7fb143665b80 1 --2- 172.21.15.77:0/716450757 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5597a3d668c0 0x5597a3d68ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:58.036 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.029+0000 7fb143665b80 4 client.25441 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:18:58.036 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.029+0000 7fb143665b80 1 -- 172.21.15.77:0/716450757 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5597a3d1fe90 con 0x5597a3d668c0 2023-12-01T01:18:58.036 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.029+0000 7fb135271700 1 --2- 172.21.15.77:0/716450757 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5597a3d668c0 0x5597a3d68ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:58.036 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.030+0000 7fb114ff9700 10 client.25441 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:58.036 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.030+0000 7fb135271700 1 --2- 172.21.15.77:0/716450757 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5597a3d668c0 0x5597a3d68ce0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5289d938663c455 server_cookie=45d1c52a86890c00 in_seq=0 out_seq=0 2023-12-01T01:18:58.036 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.030+0000 7fb114ff9700 1 -- 172.21.15.77:0/716450757 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5597a3d1fe90 con 0x5597a3d668c0 2023-12-01T01:18:58.036 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.030+0000 7fb114ff9700 10 client.25441 handle_command_reply: tid=0 2023-12-01T01:18:58.036 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.030+0000 7fb143665b80 1 client.25441 shutdown 2023-12-01T01:18:58.037 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.030+0000 7fb143665b80 1 -- 172.21.15.77:0/716450757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597a3d52170 msgr2=0x5597a3d5afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:58.037 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.030+0000 7fb143665b80 1 --2- 172.21.15.77:0/716450757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597a3d52170 0x5597a3d5afd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb124000c00 tx=0x7fb124005c00 comp rx=0 tx=0).stop 2023-12-01T01:18:58.037 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.030+0000 7fb143665b80 1 -- 172.21.15.77:0/716450757 shutdown_connections 2023-12-01T01:18:58.037 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.030+0000 7fb143665b80 1 --2- 172.21.15.77:0/716450757 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5597a3d668c0 0x5597a3d68ce0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.037 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.030+0000 7fb143665b80 1 --2- 172.21.15.77:0/716450757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597a3d52af0 0x5597a3d62010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.037 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.030+0000 7fb143665b80 1 --2- 172.21.15.77:0/716450757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597a3d51470 0x5597a3d5aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.037 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.030+0000 7fb143665b80 1 --2- 172.21.15.77:0/716450757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597a3d52170 0x5597a3d5afd0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.037 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.030+0000 7fb114ff9700 0 client.25441 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:58.037 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.030+0000 7fb143665b80 1 -- 172.21.15.77:0/716450757 >> 172.21.15.77:0/716450757 conn(0x5597a3cb0150 msgr2=0x5597a3d4b130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:58.037 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.031+0000 7fb143665b80 1 -- 172.21.15.77:0/716450757 shutdown_connections 2023-12-01T01:18:58.037 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.031+0000 7fb143665b80 1 -- 172.21.15.77:0/716450757 wait complete. 2023-12-01T01:18:58.037 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.031+0000 7fb143665b80 20 client.25441 trim_cache size 0 max 16384 2023-12-01T01:18:58.077 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.076+0000 7fb143665b80 1 Processor -- start 2023-12-01T01:18:58.092 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.090+0000 7fb143665b80 1 -- start start 2023-12-01T01:18:58.092 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.090+0000 7fb143665b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597a3d51470 0x5597a3d50ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:58.092 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.090+0000 7fb143665b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597a3d52170 0x5597a3d5ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:58.092 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.090+0000 7fb143665b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597a3d52af0 0x5597a3d5aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:58.092 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.090+0000 7fb143665b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597a3d59f40 con 0x5597a3d52af0 2023-12-01T01:18:58.092 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.090+0000 7fb143665b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597a3d5a600 con 0x5597a3d51470 2023-12-01T01:18:58.092 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.090+0000 7fb143665b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597a3d59580 con 0x5597a3d52170 2023-12-01T01:18:58.092 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.091+0000 7fb135a72700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597a3d52af0 0x5597a3d5aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:58.092 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.091+0000 7fb135a72700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597a3d52af0 0x5597a3d5aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60216/0 (socket says 172.21.15.77:60216) 2023-12-01T01:18:58.092 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.091+0000 7fb135a72700 1 -- 172.21.15.77:0/3368649835 learned_addr learned my addr 172.21.15.77:0/3368649835 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:58.093 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.091+0000 7fb135a72700 1 -- 172.21.15.77:0/3368649835 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597a3d52170 msgr2=0x5597a3d5ff20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-01T01:18:58.093 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.091+0000 7fb135a72700 1 --2- 172.21.15.77:0/3368649835 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597a3d52170 0x5597a3d5ff20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.093 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.091+0000 7fb135a72700 1 -- 172.21.15.77:0/3368649835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597a3d51470 msgr2=0x5597a3d50ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:58.093 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.091+0000 7fb135a72700 1 --2- 172.21.15.77:0/3368649835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597a3d51470 0x5597a3d50ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.093 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.091+0000 7fb135a72700 1 -- 172.21.15.77:0/3368649835 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597a3ab8ce0 con 0x5597a3d52af0 2023-12-01T01:18:58.094 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.091+0000 7fb135a72700 1 --2- 172.21.15.77:0/3368649835 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597a3d52af0 0x5597a3d5aa50 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb12000ddb0 tx=0x7fb120005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad86d5e2287666b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:58.094 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.092+0000 7fb117fff700 1 -- 172.21.15.77:0/3368649835 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12009c880 con 0x5597a3d52af0 2023-12-01T01:18:58.094 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.092+0000 7fb117fff700 1 -- 172.21.15.77:0/3368649835 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1200a0ac0 con 0x5597a3d52af0 2023-12-01T01:18:58.094 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.092+0000 7fb117fff700 1 -- 172.21.15.77:0/3368649835 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1200981a0 con 0x5597a3d52af0 2023-12-01T01:18:58.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.093+0000 7fb143665b80 1 -- 172.21.15.77:0/3368649835 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597a3d52af0 msgr2=0x5597a3d5aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:58.095 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.093+0000 7fb143665b80 1 --2- 172.21.15.77:0/3368649835 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597a3d52af0 0x5597a3d5aa50 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb12000ddb0 tx=0x7fb120005020 comp rx=0 tx=0).stop 2023-12-01T01:18:58.100 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.099+0000 7fb143665b80 1 -- 172.21.15.77:0/3368649835 shutdown_connections 2023-12-01T01:18:58.100 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.099+0000 7fb143665b80 1 --2- 172.21.15.77:0/3368649835 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597a3d52af0 0x5597a3d5aa50 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.100 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.099+0000 7fb143665b80 1 --2- 172.21.15.77:0/3368649835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597a3d51470 0x5597a3d50ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.100 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.099+0000 7fb143665b80 1 --2- 172.21.15.77:0/3368649835 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597a3d52170 0x5597a3d5ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.100 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.099+0000 7fb143665b80 1 -- 172.21.15.77:0/3368649835 >> 172.21.15.77:0/3368649835 conn(0x5597a3cb0150 msgr2=0x5597a3d4b860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:58.102 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.101+0000 7fb143665b80 1 -- 172.21.15.77:0/3368649835 shutdown_connections 2023-12-01T01:18:58.102 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.101+0000 7fb143665b80 1 -- 172.21.15.77:0/3368649835 wait complete. 2023-12-01T01:18:58.113 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.101+0000 7fb143665b80 1 -- start start 2023-12-01T01:18:58.113 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.101+0000 7fb143665b80 1 Processor -- start 2023-12-01T01:18:58.113 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.102+0000 7fb143665b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597a3d51470 0x5597a3db0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:58.113 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.102+0000 7fb143665b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597a3d52170 0x5597a3db1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:58.113 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.102+0000 7fb143665b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597a3d52af0 0x5597a3db8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:58.113 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.102+0000 7fb143665b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597a3db87a0 con 0x5597a3d51470 2023-12-01T01:18:58.113 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.102+0000 7fb143665b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597a3db8910 con 0x5597a3d52170 2023-12-01T01:18:58.113 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.102+0000 7fb143665b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597a3db8ad0 con 0x5597a3d52af0 2023-12-01T01:18:58.113 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.102+0000 7fb143665b80 10 client.0 _ll_register_callbacks cb 0x7fb12400bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:18:58.113 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.102+0000 7fb135271700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597a3d51470 0x5597a3db0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:58.114 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.102+0000 7fb135271700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597a3d51470 0x5597a3db0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60218/0 (socket says 172.21.15.77:60218) 2023-12-01T01:18:58.114 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.102+0000 7fb135271700 1 -- 172.21.15.77:0/2918314332 learned_addr learned my addr 172.21.15.77:0/2918314332 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:58.114 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.103+0000 7fb135271700 1 -- 172.21.15.77:0/2918314332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597a3d52af0 msgr2=0x5597a3db8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:58.114 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.103+0000 7fb135271700 1 --2- 172.21.15.77:0/2918314332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597a3d52af0 0x5597a3db8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.114 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.103+0000 7fb135271700 1 -- 172.21.15.77:0/2918314332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597a3d52170 msgr2=0x5597a3db1250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-01T01:18:58.114 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.103+0000 7fb135271700 1 --2- 172.21.15.77:0/2918314332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597a3d52170 0x5597a3db1250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.114 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.103+0000 7fb135271700 1 -- 172.21.15.77:0/2918314332 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb13019f030 con 0x5597a3d51470 2023-12-01T01:18:58.114 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.103+0000 7fb135271700 1 --2- 172.21.15.77:0/2918314332 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597a3d51470 0x5597a3db0d10 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb12800b010 tx=0x7fb1280214a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb29190b61f1d48 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:58.114 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.103+0000 7fb114ff9700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:18:58.114 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.103+0000 7fb114ff9700 1 -- 172.21.15.77:0/2918314332 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12802f030 con 0x5597a3d51470 2023-12-01T01:18:58.114 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.103+0000 7fb114ff9700 1 -- 172.21.15.77:0/2918314332 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fb128030070 con 0x5597a3d51470 2023-12-01T01:18:58.114 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.103+0000 7fb143665b80 1 -- 172.21.15.77:0/2918314332 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb12800f040 con 0x5597a3d51470 2023-12-01T01:18:58.114 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.103+0000 7fb114ff9700 1 -- 172.21.15.77:0/2918314332 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1280335b0 con 0x5597a3d51470 2023-12-01T01:18:58.114 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.103+0000 7fb143665b80 10 client.15852 fetch_fsmap learned FSMap version 29 2023-12-01T01:18:58.115 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.103+0000 7fb143665b80 1 -- 172.21.15.77:0/2918314332 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5597a3af62d0 con 0x5597a3d51470 2023-12-01T01:18:58.115 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.105+0000 7fb114ff9700 1 -- 172.21.15.77:0/2918314332 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb12800f040 con 0x5597a3d51470 2023-12-01T01:18:58.115 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.105+0000 7fb114ff9700 1 -- 172.21.15.77:0/2918314332 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb128015620 con 0x5597a3d51470 2023-12-01T01:18:58.115 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.105+0000 7fb143665b80 10 client.15852 fetch_fsmap finished waiting for FSMap version 29 2023-12-01T01:18:58.115 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.105+0000 7fb143665b80 10 client.15852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-01T01:18:58.115 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.105+0000 7fb143665b80 20 client.15852 populate_metadata read hostname 'smithi077' 2023-12-01T01:18:58.115 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.105+0000 7fb143665b80 1 --2- 172.21.15.77:0/2918314332 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5597a3dba3c0 0x5597a3dbc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:58.115 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.105+0000 7fb143665b80 4 client.15852 mds_command: new command op to 24500 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-12-01T01:18:58.115 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.105+0000 7fb143665b80 1 -- 172.21.15.77:0/2918314332 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5597a3c9adc0 con 0x5597a3dba3c0 2023-12-01T01:18:58.117 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.112+0000 7fb134a70700 1 --2- 172.21.15.77:0/2918314332 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5597a3dba3c0 0x5597a3dbc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:58.118 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.115+0000 7fb134a70700 1 --2- 172.21.15.77:0/2918314332 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5597a3dba3c0 0x5597a3dbc7e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91f9e05f100dd9bd server_cookie=b6db0cc4f2dd48b3 in_seq=0 out_seq=0 2023-12-01T01:18:58.118 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.115+0000 7fb114ff9700 10 client.15852 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:58.134 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.133+0000 7fb114ff9700 1 -- 172.21.15.77:0/2918314332 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5597a3c9adc0 con 0x5597a3dba3c0 2023-12-01T01:18:58.134 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.133+0000 7fb114ff9700 10 client.15852 handle_command_reply: tid=0 2023-12-01T01:18:58.136 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.135+0000 7fb143665b80 1 client.15852 shutdown 2023-12-01T01:18:58.145 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.141+0000 7fb143665b80 1 -- 172.21.15.77:0/2918314332 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597a3d51470 msgr2=0x5597a3db0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:58.145 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.141+0000 7fb143665b80 1 --2- 172.21.15.77:0/2918314332 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597a3d51470 0x5597a3db0d10 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb12800b010 tx=0x7fb1280214a0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.149 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.144+0000 7fb143665b80 1 -- 172.21.15.77:0/2918314332 shutdown_connections 2023-12-01T01:18:58.150 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.144+0000 7fb143665b80 1 --2- 172.21.15.77:0/2918314332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5597a3d52af0 0x5597a3db8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.150 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.144+0000 7fb143665b80 1 --2- 172.21.15.77:0/2918314332 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5597a3d51470 0x5597a3db0d10 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.150 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.144+0000 7fb143665b80 1 --2- 172.21.15.77:0/2918314332 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5597a3dba3c0 0x5597a3dbc7e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.150 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.144+0000 7fb143665b80 1 --2- 172.21.15.77:0/2918314332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597a3d52170 0x5597a3db1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.150 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.144+0000 7fb143665b80 1 -- 172.21.15.77:0/2918314332 >> 172.21.15.77:0/2918314332 conn(0x5597a3cb0150 msgr2=0x5597a3d4fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:58.150 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.144+0000 7fb114ff9700 0 client.15852 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:58.158 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.157+0000 7fb143665b80 1 -- 172.21.15.77:0/2918314332 shutdown_connections 2023-12-01T01:18:58.158 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.157+0000 7fb143665b80 1 -- 172.21.15.77:0/2918314332 wait complete. 2023-12-01T01:18:58.161 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.160+0000 7fb143665b80 20 client.15852 trim_cache size 0 max 16384 2023-12-01T01:18:58.161 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2023-12-01T01:18:58.161 INFO:teuthology.orchestra.run.smithi077.stdout: "return_code": 0, 2023-12-01T01:18:58.161 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_tag": "085290de-33b6-4527-ab7f-03f7dfd5dbba", 2023-12-01T01:18:58.161 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": "asynchronous" 2023-12-01T01:18:58.161 INFO:teuthology.orchestra.run.smithi077.stdout:} 2023-12-01T01:18:58.163 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.162+0000 7fb1374d5700 1 -- 172.21.15.77:0/1320113469 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fb1180a0520 msgr2=0x7fb1180a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:58.164 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.162+0000 7fb1374d5700 1 --2- 172.21.15.77:0/1320113469 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fb1180a0520 0x7fb1180a29e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb12800f5e0 tx=0x7fb12800b040 comp rx=0 tx=0).stop 2023-12-01T01:18:58.164 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.162+0000 7fb1374d5700 1 -- 172.21.15.77:0/1320113469 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1301029e0 msgr2=0x7fb13019dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:58.164 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.162+0000 7fb1374d5700 1 --2- 172.21.15.77:0/1320113469 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1301029e0 0x7fb13019dff0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb12000d750 tx=0x7fb12000da60 comp rx=0 tx=0).stop 2023-12-01T01:18:58.167 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.166+0000 7fb1374d5700 1 -- 172.21.15.77:0/1320113469 shutdown_connections 2023-12-01T01:18:58.167 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.166+0000 7fb1374d5700 1 --2- 172.21.15.77:0/1320113469 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fb1180a0520 0x7fb1180a29e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.167 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.166+0000 7fb1374d5700 1 --2- 172.21.15.77:0/1320113469 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1301029e0 0x7fb13019dff0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.167 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.166+0000 7fb1374d5700 1 --2- 172.21.15.77:0/1320113469 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1300ff070 0x7fb130196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.167 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.166+0000 7fb1374d5700 1 --2- 172.21.15.77:0/1320113469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb130102040 0x7fb130196f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.167 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.166+0000 7fb1374d5700 1 -- 172.21.15.77:0/1320113469 >> 172.21.15.77:0/1320113469 conn(0x7fb1300fa980 msgr2=0x7fb1300fcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:58.172 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.170+0000 7fb1374d5700 1 -- 172.21.15.77:0/1320113469 shutdown_connections 2023-12-01T01:18:58.172 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.170+0000 7fb1374d5700 1 -- 172.21.15.77:0/1320113469 wait complete. 2023-12-01T01:18:58.204 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-01T01:18:58.481 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.477+0000 7f14893cf700 1 -- 172.21.15.77:0/1339005418 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14841086f0 msgr2=0x7f148410f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:58.481 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.477+0000 7f14893cf700 1 --2- 172.21.15.77:0/1339005418 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14841086f0 0x7f148410f0d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1474009990 tx=0x7f1474009ca0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.481 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.478+0000 7f14893cf700 1 -- 172.21.15.77:0/1339005418 shutdown_connections 2023-12-01T01:18:58.481 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.478+0000 7f14893cf700 1 --2- 172.21.15.77:0/1339005418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f148410f610 0x7f1484111a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.481 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.478+0000 7f14893cf700 1 --2- 172.21.15.77:0/1339005418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1484107d90 0x7f14841081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.482 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.478+0000 7f14893cf700 1 --2- 172.21.15.77:0/1339005418 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14841086f0 0x7f148410f0d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.482 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.478+0000 7f14893cf700 1 -- 172.21.15.77:0/1339005418 >> 172.21.15.77:0/1339005418 conn(0x7f148406db40 msgr2=0x7f148406ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:58.483 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.481+0000 7f14893cf700 1 -- 172.21.15.77:0/1339005418 shutdown_connections 2023-12-01T01:18:58.483 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.482+0000 7f14893cf700 1 -- 172.21.15.77:0/1339005418 wait complete. 2023-12-01T01:18:58.486 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.482+0000 7f14893cf700 1 Processor -- start 2023-12-01T01:18:58.487 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.483+0000 7f14893cf700 1 -- start start 2023-12-01T01:18:58.487 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.483+0000 7f14893cf700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1484107d90 0x7f14841183e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:58.487 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.483+0000 7f14893cf700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14841086f0 0x7f1484118920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:58.487 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.483+0000 7f14893cf700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f148410f610 0x7f14841199a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:58.487 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.483+0000 7f14893cf700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1484119fa0 con 0x7f14841086f0 2023-12-01T01:18:58.487 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.483+0000 7f14893cf700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148411a110 con 0x7f148410f610 2023-12-01T01:18:58.487 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.483+0000 7f14893cf700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148411a280 con 0x7f1484107d90 2023-12-01T01:18:58.487 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.483+0000 7f14827fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14841086f0 0x7f1484118920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:58.487 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.483+0000 7f14827fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14841086f0 0x7f1484118920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60234/0 (socket says 172.21.15.77:60234) 2023-12-01T01:18:58.487 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.483+0000 7f1482ffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1484107d90 0x7f14841183e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:58.487 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.483+0000 7f14827fc700 1 -- 172.21.15.77:0/1144994954 learned_addr learned my addr 172.21.15.77:0/1144994954 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:58.487 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.484+0000 7f14837fe700 1 --2- 172.21.15.77:0/1144994954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f148410f610 0x7f14841199a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:58.487 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.484+0000 7f14827fc700 1 -- 172.21.15.77:0/1144994954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1484107d90 msgr2=0x7f14841183e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:58.487 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.484+0000 7f14827fc700 1 --2- 172.21.15.77:0/1144994954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1484107d90 0x7f14841183e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.488 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.484+0000 7f14827fc700 1 -- 172.21.15.77:0/1144994954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f148410f610 msgr2=0x7f14841199a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:58.488 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.484+0000 7f14827fc700 1 --2- 172.21.15.77:0/1144994954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f148410f610 0x7f14841199a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.488 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.484+0000 7f14827fc700 1 -- 172.21.15.77:0/1144994954 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14740095f0 con 0x7f14841086f0 2023-12-01T01:18:58.488 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.485+0000 7f14827fc700 1 --2- 172.21.15.77:0/1144994954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14841086f0 0x7f1484118920 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1474004800 tx=0x7f147400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:58.488 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.486+0000 7f146bfff700 1 -- 172.21.15.77:0/1144994954 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147401d070 con 0x7f14841086f0 2023-12-01T01:18:58.488 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.486+0000 7f146bfff700 1 -- 172.21.15.77:0/1144994954 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f147400be60 con 0x7f14841086f0 2023-12-01T01:18:58.488 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.486+0000 7f146bfff700 1 -- 172.21.15.77:0/1144994954 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1474017650 con 0x7f14841086f0 2023-12-01T01:18:58.488 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.486+0000 7f14893cf700 1 -- 172.21.15.77:0/1144994954 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f148411a500 con 0x7f14841086f0 2023-12-01T01:18:58.488 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.486+0000 7f14893cf700 1 -- 172.21.15.77:0/1144994954 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f148411a9e0 con 0x7f14841086f0 2023-12-01T01:18:58.492 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.491+0000 7f149555fb80 1 Processor -- start 2023-12-01T01:18:58.493 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.491+0000 7f149555fb80 1 -- start start 2023-12-01T01:18:58.493 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.491+0000 7f149555fb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d33b120f0 0x560d33b12530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:58.493 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.491+0000 7f149555fb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d33b12a70 0x560d33b10e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:58.493 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.491+0000 7f149555fb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d33b113f0 0x560d33b11870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:58.493 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.491+0000 7f149555fb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d33a68a10 con 0x560d33b113f0 2023-12-01T01:18:58.493 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.491+0000 7f149555fb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d33a663d0 con 0x560d33b120f0 2023-12-01T01:18:58.493 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.492+0000 7f149555fb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d33a61cc0 con 0x560d33b12a70 2023-12-01T01:18:58.493 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.492+0000 7f14837fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d33b120f0 0x560d33b12530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:58.494 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.492+0000 7f14837fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d33b120f0 0x560d33b12530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:39158/0 (socket says 172.21.15.77:39158) 2023-12-01T01:18:58.494 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.492+0000 7f14837fe700 1 -- 172.21.15.77:0/1378554912 learned_addr learned my addr 172.21.15.77:0/1378554912 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:58.494 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.492+0000 7f1482ffd700 1 --2- 172.21.15.77:0/1378554912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d33b12a70 0x560d33b10e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:58.503 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.493+0000 7f14837fe700 1 -- 172.21.15.77:0/1378554912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d33b12a70 msgr2=0x560d33b10e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:58.503 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.493+0000 7f14837fe700 1 --2- 172.21.15.77:0/1378554912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d33b12a70 0x560d33b10e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.503 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.493+0000 7f14837fe700 1 -- 172.21.15.77:0/1378554912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d33b113f0 msgr2=0x560d33b11870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-01T01:18:58.503 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.493+0000 7f14837fe700 1 --2- 172.21.15.77:0/1378554912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d33b113f0 0x560d33b11870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.503 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.493+0000 7f14837fe700 1 -- 172.21.15.77:0/1378554912 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d3388fde0 con 0x560d33b120f0 2023-12-01T01:18:58.503 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.494+0000 7f14837fe700 1 --2- 172.21.15.77:0/1378554912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d33b120f0 0x560d33b12530 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f147c0093e0 tx=0x7f147c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47c10dcc07deb12b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:58.503 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.495+0000 7f146bfff700 1 -- 172.21.15.77:0/1144994954 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f1474021410 con 0x7f14841086f0 2023-12-01T01:18:58.503 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.495+0000 7f146bfff700 1 --2- 172.21.15.77:0/1144994954 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f146c0a05d0 0x7f146c0a2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:58.503 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.496+0000 7f146bfff700 1 -- 172.21.15.77:0/1144994954 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f14740cce60 con 0x7f14841086f0 2023-12-01T01:18:58.503 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.497+0000 7f14697fa700 1 -- 172.21.15.77:0/1378554912 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147c00e070 con 0x560d33b120f0 2023-12-01T01:18:58.504 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.497+0000 7f14697fa700 1 -- 172.21.15.77:0/1378554912 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f147c00f040 con 0x560d33b120f0 2023-12-01T01:18:58.504 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.497+0000 7f14697fa700 1 -- 172.21.15.77:0/1378554912 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147c0147f0 con 0x560d33b120f0 2023-12-01T01:18:58.504 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.498+0000 7f149555fb80 1 -- 172.21.15.77:0/1378554912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d33b120f0 msgr2=0x560d33b12530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:58.504 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.498+0000 7f149555fb80 1 --2- 172.21.15.77:0/1378554912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d33b120f0 0x560d33b12530 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f147c0093e0 tx=0x7f147c009920 comp rx=0 tx=0).stop 2023-12-01T01:18:58.504 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.498+0000 7f149555fb80 1 -- 172.21.15.77:0/1378554912 shutdown_connections 2023-12-01T01:18:58.504 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.498+0000 7f149555fb80 1 --2- 172.21.15.77:0/1378554912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d33b113f0 0x560d33b11870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.504 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.498+0000 7f149555fb80 1 --2- 172.21.15.77:0/1378554912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d33b120f0 0x560d33b12530 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.504 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.498+0000 7f149555fb80 1 --2- 172.21.15.77:0/1378554912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d33b12a70 0x560d33b10e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.504 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.498+0000 7f149555fb80 1 -- 172.21.15.77:0/1378554912 >> 172.21.15.77:0/1378554912 conn(0x560d33a700d0 msgr2=0x560d33b0b940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:58.504 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.502+0000 7f149555fb80 1 -- 172.21.15.77:0/1378554912 shutdown_connections 2023-12-01T01:18:58.508 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.504+0000 7f1482ffd700 1 --2- 172.21.15.77:0/1144994954 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f146c0a05d0 0x7f146c0a2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:58.509 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.507+0000 7f1482ffd700 1 --2- 172.21.15.77:0/1144994954 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f146c0a05d0 0x7f146c0a2a90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x560d33b0da70 tx=0x7f1478008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:58.509 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.507+0000 7f149555fb80 1 -- 172.21.15.77:0/1378554912 wait complete. 2023-12-01T01:18:58.509 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.508+0000 7f149555fb80 1 -- start start 2023-12-01T01:18:58.510 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.508+0000 7f149555fb80 1 Processor -- start 2023-12-01T01:18:58.511 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.509+0000 7f149555fb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d33b113f0 0x560d33b1aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:58.511 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.509+0000 7f149555fb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d33b120f0 0x560d33b1b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:58.511 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.509+0000 7f149555fb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d33b20110 0x560d33b22500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:58.511 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.509+0000 7f149555fb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d339fe510 con 0x560d33b20110 2023-12-01T01:18:58.511 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.509+0000 7f149555fb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d339fe1b0 con 0x560d33b113f0 2023-12-01T01:18:58.511 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.509+0000 7f149555fb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d338e5920 con 0x560d33b120f0 2023-12-01T01:18:58.511 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.509+0000 7f149555fb80 10 client.0 _ll_register_callbacks cb 0x560d3394ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:18:58.511 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.509+0000 7f1482ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d33b113f0 0x560d33b1aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:58.512 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.509+0000 7f14837fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d33b20110 0x560d33b22500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:58.512 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.509+0000 7f1482ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d33b113f0 0x560d33b1aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:39164/0 (socket says 172.21.15.77:39164) 2023-12-01T01:18:58.512 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.509+0000 7f1482ffd700 1 -- 172.21.15.77:0/2434488696 learned_addr learned my addr 172.21.15.77:0/2434488696 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:58.513 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.509+0000 7f14837fe700 1 -- 172.21.15.77:0/2434488696 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d33b120f0 msgr2=0x560d33b1b0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-01T01:18:58.513 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.509+0000 7f14837fe700 1 --2- 172.21.15.77:0/2434488696 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d33b120f0 0x560d33b1b0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.513 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.509+0000 7f14837fe700 1 -- 172.21.15.77:0/2434488696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d33b113f0 msgr2=0x560d33b1aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:58.513 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.509+0000 7f14837fe700 1 --2- 172.21.15.77:0/2434488696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d33b113f0 0x560d33b1aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.513 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.509+0000 7f14837fe700 1 -- 172.21.15.77:0/2434488696 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f147c009090 con 0x560d33b20110 2023-12-01T01:18:58.513 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.510+0000 7f14837fe700 1 --2- 172.21.15.77:0/2434488696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d33b20110 0x560d33b22500 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f147c000f80 tx=0x7f147c005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca0786cc404b0a7 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:58.514 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.510+0000 7f1466ff5700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:18:58.516 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.511+0000 7f1466ff5700 1 -- 172.21.15.77:0/2434488696 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147c00e040 con 0x560d33b20110 2023-12-01T01:18:58.516 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.511+0000 7f1466ff5700 1 -- 172.21.15.77:0/2434488696 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f147c0147a0 con 0x560d33b20110 2023-12-01T01:18:58.516 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.511+0000 7f149555fb80 1 -- 172.21.15.77:0/2434488696 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560d33a8cd10 con 0x560d33b20110 2023-12-01T01:18:58.516 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.512+0000 7f1466ff5700 1 -- 172.21.15.77:0/2434488696 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f147c014960 con 0x560d33b20110 2023-12-01T01:18:58.516 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.512+0000 7f149555fb80 10 client.15861 fetch_fsmap learned FSMap version 29 2023-12-01T01:18:58.516 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.512+0000 7f149555fb80 1 -- 172.21.15.77:0/2434488696 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560d33878c60 con 0x560d33b20110 2023-12-01T01:18:58.516 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.513+0000 7f1466ff5700 1 -- 172.21.15.77:0/2434488696 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560d33a8cd10 con 0x560d33b20110 2023-12-01T01:18:58.516 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.513+0000 7f1466ff5700 1 -- 172.21.15.77:0/2434488696 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f147c01ece0 con 0x560d33b20110 2023-12-01T01:18:58.517 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.513+0000 7f149555fb80 10 client.15861 fetch_fsmap finished waiting for FSMap version 29 2023-12-01T01:18:58.517 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.513+0000 7f149555fb80 10 client.15861 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-01T01:18:58.517 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.513+0000 7f149555fb80 20 client.15861 populate_metadata read hostname 'smithi077' 2023-12-01T01:18:58.517 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.513+0000 7f149555fb80 1 --2- 172.21.15.77:0/2434488696 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x560d33b26cb0 0x560d33b290d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:58.517 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.513+0000 7f149555fb80 4 client.15861 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:18:58.517 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.513+0000 7f149555fb80 1 -- 172.21.15.77:0/2434488696 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560d33adfe10 con 0x560d33b26cb0 2023-12-01T01:18:58.517 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.514+0000 7f1482ffd700 1 --2- 172.21.15.77:0/2434488696 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x560d33b26cb0 0x560d33b290d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:58.518 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.516+0000 7f1466ff5700 10 client.15861 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:58.518 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.516+0000 7f1482ffd700 1 --2- 172.21.15.77:0/2434488696 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x560d33b26cb0 0x560d33b290d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4aa2cb06e0f4982e server_cookie=ba1e900bf08e9392 in_seq=0 out_seq=0 2023-12-01T01:18:58.520 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.519+0000 7f1466ff5700 1 -- 172.21.15.77:0/2434488696 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x560d33adfe10 con 0x560d33b26cb0 2023-12-01T01:18:58.521 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.519+0000 7f1466ff5700 10 client.15861 handle_command_reply: tid=0 2023-12-01T01:18:58.521 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.519+0000 7f149555fb80 1 client.15861 shutdown 2023-12-01T01:18:58.521 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.519+0000 7f149555fb80 1 -- 172.21.15.77:0/2434488696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d33b20110 msgr2=0x560d33b22500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:58.521 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.519+0000 7f149555fb80 1 --2- 172.21.15.77:0/2434488696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d33b20110 0x560d33b22500 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f147c000f80 tx=0x7f147c005ae0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.521 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.519+0000 7f149555fb80 1 -- 172.21.15.77:0/2434488696 shutdown_connections 2023-12-01T01:18:58.521 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.519+0000 7f149555fb80 1 --2- 172.21.15.77:0/2434488696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d33b20110 0x560d33b22500 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.521 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.519+0000 7f149555fb80 1 --2- 172.21.15.77:0/2434488696 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x560d33b26cb0 0x560d33b290d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.521 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.519+0000 7f149555fb80 1 --2- 172.21.15.77:0/2434488696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d33b113f0 0x560d33b1aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.521 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.519+0000 7f149555fb80 1 --2- 172.21.15.77:0/2434488696 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d33b120f0 0x560d33b1b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.521 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.519+0000 7f149555fb80 1 -- 172.21.15.77:0/2434488696 >> 172.21.15.77:0/2434488696 conn(0x560d33a700d0 msgr2=0x560d33b0fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:58.521 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.519+0000 7f1466ff5700 0 client.15861 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:58.524 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.523+0000 7f149555fb80 1 -- 172.21.15.77:0/2434488696 shutdown_connections 2023-12-01T01:18:58.524 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.523+0000 7f149555fb80 1 -- 172.21.15.77:0/2434488696 wait complete. 2023-12-01T01:18:58.524 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.523+0000 7f149555fb80 20 client.15861 trim_cache size 0 max 16384 2023-12-01T01:18:58.558 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.556+0000 7f149555fb80 1 Processor -- start 2023-12-01T01:18:58.558 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.556+0000 7f149555fb80 1 -- start start 2023-12-01T01:18:58.558 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.556+0000 7f149555fb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d33b113f0 0x560d33b10e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:58.558 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.556+0000 7f149555fb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d33b120f0 0x560d33b365b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:58.559 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.556+0000 7f149555fb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d33b20110 0x560d33b195d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:58.559 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.556+0000 7f149555fb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d33b267f0 con 0x560d33b120f0 2023-12-01T01:18:58.559 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.556+0000 7f149555fb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d33b1a050 con 0x560d33b20110 2023-12-01T01:18:58.559 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.556+0000 7f149555fb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d33b1a710 con 0x560d33b113f0 2023-12-01T01:18:58.559 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.556+0000 7f1482ffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d33b113f0 0x560d33b10e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:58.559 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.556+0000 7f1482ffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d33b113f0 0x560d33b10e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:46438/0 (socket says 172.21.15.77:46438) 2023-12-01T01:18:58.560 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.556+0000 7f1482ffd700 1 -- 172.21.15.77:0/926770508 learned_addr learned my addr 172.21.15.77:0/926770508 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:58.560 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.556+0000 7f14837fe700 1 --2- 172.21.15.77:0/926770508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d33b20110 0x560d33b195d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:58.560 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.556+0000 7f14827fc700 1 --2- 172.21.15.77:0/926770508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d33b120f0 0x560d33b365b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:58.560 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.557+0000 7f1482ffd700 1 -- 172.21.15.77:0/926770508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d33b20110 msgr2=0x560d33b195d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:58.560 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.557+0000 7f1482ffd700 1 --2- 172.21.15.77:0/926770508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d33b20110 0x560d33b195d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.560 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.557+0000 7f1482ffd700 1 -- 172.21.15.77:0/926770508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d33b120f0 msgr2=0x560d33b365b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:58.560 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.557+0000 7f1482ffd700 1 --2- 172.21.15.77:0/926770508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d33b120f0 0x560d33b365b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.560 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.557+0000 7f1482ffd700 1 -- 172.21.15.77:0/926770508 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d338b6250 con 0x560d33b113f0 2023-12-01T01:18:58.560 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.557+0000 7f14827fc700 1 --2- 172.21.15.77:0/926770508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d33b120f0 0x560d33b365b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:18:58.560 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.557+0000 7f14837fe700 1 --2- 172.21.15.77:0/926770508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d33b20110 0x560d33b195d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:18:58.560 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.557+0000 7f1482ffd700 1 --2- 172.21.15.77:0/926770508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d33b113f0 0x560d33b10e60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1478009850 tx=0x7f1478014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84aa3189708e01df server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:58.560 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.557+0000 7f1469ffb700 1 -- 172.21.15.77:0/926770508 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1478027070 con 0x560d33b113f0 2023-12-01T01:18:58.560 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.557+0000 7f1469ffb700 1 -- 172.21.15.77:0/926770508 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1478011dc0 con 0x560d33b113f0 2023-12-01T01:18:58.561 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.557+0000 7f1469ffb700 1 -- 172.21.15.77:0/926770508 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14780157f0 con 0x560d33b113f0 2023-12-01T01:18:58.564 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.558+0000 7f149555fb80 1 -- 172.21.15.77:0/926770508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d33b113f0 msgr2=0x560d33b10e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:58.564 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.558+0000 7f149555fb80 1 --2- 172.21.15.77:0/926770508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d33b113f0 0x560d33b10e60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1478009850 tx=0x7f1478014000 comp rx=0 tx=0).stop 2023-12-01T01:18:58.564 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.558+0000 7f149555fb80 1 -- 172.21.15.77:0/926770508 shutdown_connections 2023-12-01T01:18:58.564 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.558+0000 7f149555fb80 1 --2- 172.21.15.77:0/926770508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d33b20110 0x560d33b195d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.564 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.558+0000 7f149555fb80 1 --2- 172.21.15.77:0/926770508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d33b113f0 0x560d33b10e60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.564 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.558+0000 7f149555fb80 1 --2- 172.21.15.77:0/926770508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d33b120f0 0x560d33b365b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.564 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.558+0000 7f149555fb80 1 -- 172.21.15.77:0/926770508 >> 172.21.15.77:0/926770508 conn(0x560d33a700d0 msgr2=0x560d33b15620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:58.564 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.558+0000 7f149555fb80 1 -- 172.21.15.77:0/926770508 shutdown_connections 2023-12-01T01:18:58.564 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.558+0000 7f149555fb80 1 -- 172.21.15.77:0/926770508 wait complete. 2023-12-01T01:18:58.564 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.559+0000 7f149555fb80 1 -- start start 2023-12-01T01:18:58.564 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.559+0000 7f149555fb80 1 Processor -- start 2023-12-01T01:18:58.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.559+0000 7f149555fb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d33b113f0 0x560d33b365d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:58.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.559+0000 7f149555fb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d33b120f0 0x560d33b74d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:58.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.559+0000 7f149555fb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d33b20110 0x560d33b77360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:58.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.559+0000 7f149555fb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d33b77a70 con 0x560d33b20110 2023-12-01T01:18:58.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.559+0000 7f149555fb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d33b77be0 con 0x560d33b113f0 2023-12-01T01:18:58.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.559+0000 7f149555fb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d33b77da0 con 0x560d33b120f0 2023-12-01T01:18:58.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.559+0000 7f149555fb80 10 client.0 _ll_register_callbacks cb 0x7f147c0092b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:18:58.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.559+0000 7f1482ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d33b113f0 0x560d33b365d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:58.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.559+0000 7f1482ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d33b113f0 0x560d33b365d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:39178/0 (socket says 172.21.15.77:39178) 2023-12-01T01:18:58.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.559+0000 7f1482ffd700 1 -- 172.21.15.77:0/3216386598 learned_addr learned my addr 172.21.15.77:0/3216386598 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:18:58.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.559+0000 7f14837fe700 1 --2- 172.21.15.77:0/3216386598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d33b20110 0x560d33b77360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:58.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.560+0000 7f14827fc700 1 --2- 172.21.15.77:0/3216386598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d33b120f0 0x560d33b74d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:58.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.560+0000 7f1482ffd700 1 -- 172.21.15.77:0/3216386598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d33b120f0 msgr2=0x560d33b74d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:58.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.560+0000 7f1482ffd700 1 --2- 172.21.15.77:0/3216386598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d33b120f0 0x560d33b74d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.565 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.560+0000 7f1482ffd700 1 -- 172.21.15.77:0/3216386598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d33b20110 msgr2=0x560d33b77360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:18:58.566 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.560+0000 7f1482ffd700 1 --2- 172.21.15.77:0/3216386598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d33b20110 0x560d33b77360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.566 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.560+0000 7f1482ffd700 1 -- 172.21.15.77:0/3216386598 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f148411a9e0 con 0x560d33b113f0 2023-12-01T01:18:58.566 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.560+0000 7f14827fc700 1 --2- 172.21.15.77:0/3216386598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d33b120f0 0x560d33b74d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:18:58.566 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.560+0000 7f1482ffd700 1 --2- 172.21.15.77:0/3216386598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d33b113f0 0x560d33b365d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f14780285f0 tx=0x7f1478011f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7cbdb0452031476 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:18:58.566 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.560+0000 7f1466ff5700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2023-12-01T01:18:58.566 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.560+0000 7f1466ff5700 1 -- 172.21.15.77:0/3216386598 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1478027040 con 0x560d33b113f0 2023-12-01T01:18:58.566 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.560+0000 7f1466ff5700 1 -- 172.21.15.77:0/3216386598 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f147801b040 con 0x560d33b113f0 2023-12-01T01:18:58.566 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.560+0000 7f149555fb80 1 -- 172.21.15.77:0/3216386598 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f147c00f040 con 0x560d33b113f0 2023-12-01T01:18:58.566 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.560+0000 7f1466ff5700 1 -- 172.21.15.77:0/3216386598 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1478028a30 con 0x560d33b113f0 2023-12-01T01:18:58.566 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.560+0000 7f149555fb80 10 client.25439 fetch_fsmap learned FSMap version 29 2023-12-01T01:18:58.566 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.560+0000 7f149555fb80 1 -- 172.21.15.77:0/3216386598 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560d33878c60 con 0x560d33b113f0 2023-12-01T01:18:58.566 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.561+0000 7f1466ff5700 1 -- 172.21.15.77:0/3216386598 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f147c00f040 con 0x560d33b113f0 2023-12-01T01:18:58.566 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.561+0000 7f1466ff5700 1 -- 172.21.15.77:0/3216386598 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1478030070 con 0x560d33b113f0 2023-12-01T01:18:58.566 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.561+0000 7f149555fb80 10 client.25439 fetch_fsmap finished waiting for FSMap version 29 2023-12-01T01:18:58.566 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.561+0000 7f149555fb80 10 client.25439 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-01T01:18:58.567 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.561+0000 7f149555fb80 20 client.25439 populate_metadata read hostname 'smithi077' 2023-12-01T01:18:58.567 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.561+0000 7f149555fb80 1 --2- 172.21.15.77:0/3216386598 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x560d33b79960 0x560d33b7bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:18:58.567 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.561+0000 7f149555fb80 4 client.25439 mds_command: new command op to 24500 tid=0[{"prefix": "scrub status"}] 2023-12-01T01:18:58.567 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.561+0000 7f149555fb80 1 -- 172.21.15.77:0/3216386598 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560d33a5ad40 con 0x560d33b79960 2023-12-01T01:18:58.567 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.561+0000 7f14827fc700 1 --2- 172.21.15.77:0/3216386598 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x560d33b79960 0x560d33b7bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:18:58.567 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.562+0000 7f1466ff5700 10 client.25439 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:58.567 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.562+0000 7f14827fc700 1 --2- 172.21.15.77:0/3216386598 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x560d33b79960 0x560d33b7bd80 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75eae3abc86894eb server_cookie=d4c4dc9cbdcf871f in_seq=0 out_seq=0 2023-12-01T01:18:58.568 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.564+0000 7f1466ff5700 1 -- 172.21.15.77:0/3216386598 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x560d33a5ad40 con 0x560d33b79960 2023-12-01T01:18:58.568 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.564+0000 7f1466ff5700 10 client.25439 handle_command_reply: tid=0 2023-12-01T01:18:58.568 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.565+0000 7f149555fb80 1 client.25439 shutdown 2023-12-01T01:18:58.568 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.565+0000 7f149555fb80 1 -- 172.21.15.77:0/3216386598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d33b113f0 msgr2=0x560d33b365d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:58.568 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.565+0000 7f149555fb80 1 --2- 172.21.15.77:0/3216386598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d33b113f0 0x560d33b365d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f14780285f0 tx=0x7f1478011f30 comp rx=0 tx=0).stop 2023-12-01T01:18:58.568 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.565+0000 7f149555fb80 1 -- 172.21.15.77:0/3216386598 shutdown_connections 2023-12-01T01:18:58.569 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.565+0000 7f149555fb80 1 --2- 172.21.15.77:0/3216386598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d33b20110 0x560d33b77360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.569 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.565+0000 7f149555fb80 1 --2- 172.21.15.77:0/3216386598 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x560d33b79960 0x560d33b7bd80 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.569 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.565+0000 7f149555fb80 1 --2- 172.21.15.77:0/3216386598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d33b113f0 0x560d33b365d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.569 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.565+0000 7f1466ff5700 0 client.25439 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:18:58.569 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.565+0000 7f149555fb80 1 --2- 172.21.15.77:0/3216386598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560d33b120f0 0x560d33b74d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.569 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.565+0000 7f149555fb80 1 -- 172.21.15.77:0/3216386598 >> 172.21.15.77:0/3216386598 conn(0x560d33a700d0 msgr2=0x560d33b195d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:58.569 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.565+0000 7f149555fb80 1 -- 172.21.15.77:0/3216386598 shutdown_connections 2023-12-01T01:18:58.569 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.565+0000 7f149555fb80 1 -- 172.21.15.77:0/3216386598 wait complete. 2023-12-01T01:18:58.569 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.565+0000 7f149555fb80 20 client.25439 trim_cache size 0 max 16384 2023-12-01T01:18:58.570 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2023-12-01T01:18:58.570 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "scrub active (187 inodes in the stack)", 2023-12-01T01:18:58.570 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2023-12-01T01:18:58.570 INFO:teuthology.orchestra.run.smithi077.stdout: "085290de-33b6-4527-ab7f-03f7dfd5dbba": { 2023-12-01T01:18:58.570 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2023-12-01T01:18:58.570 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "085290de-33b6-4527-ab7f-03f7dfd5dbba", 2023-12-01T01:18:58.570 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2023-12-01T01:18:58.571 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:18:58.571 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:18:58.571 INFO:teuthology.orchestra.run.smithi077.stdout:} 2023-12-01T01:18:58.572 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.566+0000 7f14893cf700 1 -- 172.21.15.77:0/1144994954 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f146c0a05d0 msgr2=0x7f146c0a2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:58.572 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.566+0000 7f14893cf700 1 --2- 172.21.15.77:0/1144994954 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f146c0a05d0 0x7f146c0a2a90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x560d33b0da70 tx=0x7f1478008040 comp rx=0 tx=0).stop 2023-12-01T01:18:58.572 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.566+0000 7f14893cf700 1 -- 172.21.15.77:0/1144994954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14841086f0 msgr2=0x7f1484118920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:18:58.572 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.566+0000 7f14893cf700 1 --2- 172.21.15.77:0/1144994954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14841086f0 0x7f1484118920 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1474004800 tx=0x7f147400f690 comp rx=0 tx=0).stop 2023-12-01T01:18:58.572 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.566+0000 7f14893cf700 1 -- 172.21.15.77:0/1144994954 shutdown_connections 2023-12-01T01:18:58.572 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.566+0000 7f14893cf700 1 --2- 172.21.15.77:0/1144994954 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f146c0a05d0 0x7f146c0a2a90 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.572 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.566+0000 7f14893cf700 1 --2- 172.21.15.77:0/1144994954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f148410f610 0x7f14841199a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.572 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.566+0000 7f14893cf700 1 --2- 172.21.15.77:0/1144994954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1484107d90 0x7f14841183e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.572 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.566+0000 7f14893cf700 1 --2- 172.21.15.77:0/1144994954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14841086f0 0x7f1484118920 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:18:58.572 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.566+0000 7f14893cf700 1 -- 172.21.15.77:0/1144994954 >> 172.21.15.77:0/1144994954 conn(0x7f148406db40 msgr2=0x7f148406e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:18:58.572 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.566+0000 7f14893cf700 1 -- 172.21.15.77:0/1144994954 shutdown_connections 2023-12-01T01:18:58.573 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:18:58.566+0000 7f14893cf700 1 -- 172.21.15.77:0/1144994954 wait complete. 2023-12-01T01:18:58.581 INFO:tasks.cephfs.filesystem:scrub status for tag:085290de-33b6-4527-ab7f-03f7dfd5dbba - {'path': '/', 'tag': '085290de-33b6-4527-ab7f-03f7dfd5dbba', 'options': 'recursive,force'} 2023-12-01T01:18:59.014 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:58.665+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000020 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/HEAD auth v233 ap=1 s=23 n(v0 rc2023-12-01T01:18:39.586223+0000 b23 1=1+0) (ifile excl) (iversion lock) cr={24992=0-4194304@1} caps={24992=pAsLsXsFsxcrwb/pFscr@3},l=24992 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563635f60b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000020:[<0x100000005e5/HEAD v146>,<0x100000005e4/.git v138>,<0x100000005e3/xfstests-dev v74>,<0x100000005e2/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/_nogroup v33>,<0x1/volumes v31>]//[]","memoryvalue":"(3)0x10000000020:[<0x100000005e5/HEAD v233>,<0x100000005e4/.git v886>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.014 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:58.665+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005e7 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/description auth v199 ap=1 s=73 n(v0 rc2023-12-01T01:18:37.630268+0000 b73 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@1},l=24992(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563635f61080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000005e7:[<0x100000005e5/description v4>,<0x100000005e4/.git v138>,<0x100000005e3/xfstests-dev v74>,<0x100000005e2/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/_nogroup v33>,<0x1/volumes v31>]//[]","memoryvalue":"(3)0x100000005e7:[<0x100000005e5/description v199>,<0x100000005e4/.git v886>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.014 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:58.665+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000000a [...2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/objects/ auth{0=1,2=1,3=1,4=1} v179 ap=1 f(v0 m2023-12-01T01:18:37.689267+0000 2=0+2) n(v1 rc2023-12-01T01:18:39.622222+0000 b12985822 5=2+3) (inest mix) (ifile excl) (iversion lock) caps={24992=pAsLsXsFsx/pAsLsXsFsx@2},l=24992 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x563635d75080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x1000000000a:[<0x100000005e5/objects v118>,<0x100000005e4/.git v138>,<0x100000005e3/xfstests-dev v74>,<0x100000005e2/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/_nogroup v33>,<0x1/volumes v31>]//[]","memoryvalue":"(2)0x1000000000a:[<0x100000005e5/objects v179>,<0x100000005e4/.git v886>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-12-01T01:18:37.689267+0000 2=0+2)","ondisk_value.rstat":"n(v0 rc2023-12-01T01:18:39.622222+0000 b12985822 5=2+3)","memory_value.dirstat":"f(v0 m2023-12-01T01:18:37.689267+0000 2=0+2)","memory_value.rstat":"n(v1 rc2023-12-01T01:18:39.622222+0000 b12985822 5=2+3)","error_str":""},"return_code":0} 2023-12-01T01:18:59.015 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:58.665+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007de [...2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/ auth{0=1} v231 ap=1 f(v0 m2023-12-01T01:18:37.643268+0000 13=13+0) n(v0 rc2023-12-01T01:18:37.644268+0000 b23442 14=13+1) (inest mix) (ifile excl) (iversion lock) caps={24992=pAsLsXsFsx/pAsLsXsFsx@2},l=24992 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x563635f61600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x100000007de:[<0x100000005e5/hooks v36>,<0x100000005e4/.git v138>,<0x100000005e3/xfstests-dev v74>,<0x100000005e2/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/_nogroup v33>,<0x1/volumes v31>]//[]","memoryvalue":"(2)0x100000007de:[<0x100000005e5/hooks v231>,<0x100000005e4/.git v886>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-12-01T01:18:37.643268+0000 13=13+0)","ondisk_value.rstat":"n(v0 rc2023-12-01T01:18:37.644268+0000 b23442 14=13+1)","memory_value.dirstat":"f(v0 m2023-12-01T01:18:37.643268+0000 13=13+0)","memory_value.rstat":"n(v0 rc2023-12-01T01:18:37.644268+0000 b23442 14=13+1)","error_str":""},"return_code":0} 2023-12-01T01:18:59.015 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:58.666+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000001b [...2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/ auth{0=1} v225 ap=1 f(v0 m2023-12-01T01:18:39.605222+0000 2=1+1) n(v2 rc2023-12-01T01:18:39.607222+0000 b392 8=3+5) (inest mix) (ifile excl) (iversion lock) caps={24992=pAsLsXsFsx/pAsLsXsFsx@1},l=24992 | lock=0 importingcaps=0 dirfrag=1 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x563635f61b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x1000000001b:[<0x100000005e5/logs v153>,<0x100000005e4/.git v138>,<0x100000005e3/xfstests-dev v74>,<0x100000005e2/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/_nogroup v33>,<0x1/volumes v31>]//[]","memoryvalue":"(2)0x1000000001b:[<0x100000005e5/logs v225>,<0x100000005e4/.git v886>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-12-01T01:18:39.605222+0000 2=1+1)","ondisk_value.rstat":"n(v0 rc2023-12-01T01:18:39.607222+0000 b392 8=3+5)","memory_value.dirstat":"f(v0 m2023-12-01T01:18:39.605222+0000 2=1+1)","memory_value.rstat":"n(v2 rc2023-12-01T01:18:39.607222+0000 b392 8=3+5)","error_str":""},"return_code":0} 2023-12-01T01:18:59.015 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:58.666+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000017 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/packed-refs auth v235 ap=1 s=5324 n(v0 rc2023-12-01T01:18:39.575223+0000 b5324 1=1+0)/n(v0 1=1+0) (ifile excl) (iversion lock) cr={24992=0-4194304@1} caps={24992=pAsLsXsFsxcrwb/pFscr@3},l=24992 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 dirtyrstat=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563635f62100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000017:[<0x100000005e5/packed-refs v132>,<0x100000005e4/.git v138>,<0x100000005e3/xfstests-dev v74>,<0x100000005e2/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/_nogroup v33>,<0x1/volumes v31>]//[]","memoryvalue":"(3)0x10000000017:[<0x100000005e5/packed-refs v235>,<0x100000005e4/.git v886>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.015 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:58.666+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000027 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/config auth v207 ap=1 s=262 n(v0 rc2023-12-01T01:18:39.620222+0000 b262 1=1+0) (ifile excl) (iversion lock) cr={24992=0-4194304@1} caps={24992=pAsLsXsFsxcrwb/pFscr@4},l=24992 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563635f62680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000027:[<0x100000005e5/config v166>,<0x100000005e4/.git v138>,<0x100000005e3/xfstests-dev v74>,<0x100000005e2/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/_nogroup v33>,<0x1/volumes v31>]//[]","memoryvalue":"(3)0x10000000027:[<0x100000005e5/config v207>,<0x100000005e4/.git v886>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.015 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:58.666+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007dc [...2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/info/ auth{0=1} v223 ap=1 f(v0 m2023-12-01T01:18:37.631268+0000 1=1+0) n(v0 rc2023-12-01T01:18:37.632268+0000 b240 2=1+1) (inest mix) (ifile excl) (iversion lock) caps={24992=pAsLsXsFsx/pAsLsXsFsx@3},l=24992 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x563635f62c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x100000007dc:[<0x100000005e5/info v8>,<0x100000005e4/.git v138>,<0x100000005e3/xfstests-dev v74>,<0x100000005e2/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/_nogroup v33>,<0x1/volumes v31>]//[]","memoryvalue":"(2)0x100000007dc:[<0x100000005e5/info v223>,<0x100000005e4/.git v886>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-12-01T01:18:37.631268+0000 1=1+0)","ondisk_value.rstat":"n(v0 rc2023-12-01T01:18:37.632268+0000 b240 2=1+1)","memory_value.dirstat":"f(v0 m2023-12-01T01:18:37.631268+0000 1=1+0)","memory_value.rstat":"n(v0 rc2023-12-01T01:18:37.632268+0000 b240 2=1+1)","error_str":""},"return_code":0} 2023-12-01T01:18:59.015 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:58.667+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000015 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/shallow auth v193 ap=1 s=41 n(v0 rc2023-12-01T01:18:39.535224+0000 b41 1=1+0) (ifile excl) (iversion lock) cr={24992=0-4194304@1} caps={24992=pAsLsXsFsxcrwb/pFscr@3},l=24992 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563635f63180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000015:[<0x100000005e5/shallow v122>,<0x100000005e4/.git v138>,<0x100000005e3/xfstests-dev v74>,<0x100000005e2/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/_nogroup v33>,<0x1/volumes v31>]//[]","memoryvalue":"(3)0x10000000015:[<0x100000005e5/shallow v193>,<0x100000005e4/.git v886>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.015 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:58.667+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005e6 [...2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/branches/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) (iversion lock) caps={24992=pAsLsXsFsx/AsLsXsFsx@2},l=24992 | importingcaps=0 dirfrag=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x563635f63700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x100000005e6:[<0x100000005e5/branches v2>,<0x100000005e4/.git v138>,<0x100000005e3/xfstests-dev v74>,<0x100000005e2/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/_nogroup v33>,<0x1/volumes v31>]//[]","memoryvalue":"(2)0x100000005e6:[<0x100000005e5/branches v2>,<0x100000005e4/.git v886>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":0} 2023-12-01T01:18:59.015 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:58.667+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000000c [...2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/objects/info/ auth v4 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) (iversion lock) caps={24992=pAsLsXsFsx/pAsLsXsFs@2},l=24992 | importingcaps=0 dirfrag=1 caps=1 replicated=0 authpin=1 scrubqueue=0 0x563635f9e000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x1000000000c:[<0x1000000000a/info v4>,<0x100000005e5/objects v118>,<0x100000005e4/.git v138>,<0x100000005e3/xfstests-dev v74>,<0x100000005e2/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/_nogroup v33>,<0x1/volumes v31>]//[]","memoryvalue":"(2)0x1000000000c:[<0x1000000000a/info v4>,<0x100000005e5/objects v179>,<0x100000005e4/.git v886>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":0} 2023-12-01T01:18:59.015 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:58.668+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007ec [...2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/refs/ auth{0=1,2=1,3=1,4=1} v237 ap=1 f(v0 m2023-12-01T01:18:39.577223+0000 3=0+3) n(v1 rc2023-12-01T01:18:39.607222+0000 b73 7=2+5)/n(v1 rc2023-12-01T01:18:39.607222+0000 b41 7=2+5) (inest mix dirty) (ifile excl) (iversion lock) caps={24992=pAsLsXsFsx/pAsLsXsFsx@5},l=24992 | dirtyscattered=1 request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=1 dirwaiter=0 replicated=1 dirty=1 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 0x563635e27700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x100000007ec:[<0x100000005e5/refs v138>,<0x100000005e4/.git v138>,<0x100000005e3/xfstests-dev v74>,<0x100000005e2/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/_nogroup v33>,<0x1/volumes v31>]//[]","memoryvalue":"(2)0x100000007ec:[<0x100000005e5/refs v237>,<0x100000005e4/.git v886>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-12-01T01:18:39.577223+0000 3=0+3)","ondisk_value.rstat":"n(v0 rc2023-12-01T01:18:39.607222+0000 b73 7=2+5)","memory_value.dirstat":"f(v0 m2023-12-01T01:18:39.577223+0000 3=0+3)","memory_value.rstat":"n(v1 rc2023-12-01T01:18:39.607222+0000 b73 7=2+5)","error_str":""},"return_code":0} 2023-12-01T01:18:59.015 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:58.668+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000000b [...2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/objects/pack/ auth{0=1,2=1,3=1,4=1} v42 ap=1 f(v0 m2023-12-01T01:18:39.622222+0000 2=2+0) n(v0 rc2023-12-01T01:18:39.622222+0000 b12985822 3=2+1) (inest mix) (ifile excl) (iversion lock) caps={24992=pAsLsXsFsx/pAsLsXsFsx@4},l=24992 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirwaiter=0 replicated=1 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 0x563635d75600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x1000000000b:[<0x1000000000a/pack v42>,<0x100000005e5/objects v118>,<0x100000005e4/.git v138>,<0x100000005e3/xfstests-dev v74>,<0x100000005e2/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/_nogroup v33>,<0x1/volumes v31>]//[]","memoryvalue":"(2)0x1000000000b:[<0x1000000000a/pack v42>,<0x100000005e5/objects v179>,<0x100000005e4/.git v886>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-12-01T01:18:39.622222+0000 2=2+0)","ondisk_value.rstat":"n(v0 rc2023-12-01T01:18:39.622222+0000 b12985822 3=2+1)","memory_value.dirstat":"f(v0 m2023-12-01T01:18:39.622222+0000 2=2+0)","memory_value.rstat":"n(v0 rc2023-12-01T01:18:39.622222+0000 b12985822 3=2+1)","error_str":""},"return_code":0} 2023-12-01T01:18:59.015 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:58.668+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007e6 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/pre-merge-commit.sample auth v46 ap=1 s=416 n(v0 b416 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@1},l=24992(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563635f9f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000007e6:[<0x100000007de/pre-merge-commit.sample v16>,<0x100000005e5/hooks v36>,<0x100000005e4/.git v138>,<0x100000005e3/xfstests-dev v74>,<0x100000005e2/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/_nogroup v33>,<0x1/volumes v31>]//[]","memoryvalue":"(3)0x100000007e6:[<0x100000007de/pre-merge-commit.sample v46>,<0x100000005e5/hooks v231>,<0x100000005e4/.git v886>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.015 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:58.668+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007e3 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/commit-msg.sample auth v38 ap=1 s=896 n(v0 b896 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@1},l=24992(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563635fa0100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000007e3:[<0x100000007de/commit-msg.sample v10>,<0x100000005e5/hooks v36>,<0x100000005e4/.git v138>,<0x100000005e3/xfstests-dev v74>,<0x100000005e2/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/_nogroup v33>,<0x1/volumes v31>]//[]","memoryvalue":"(3)0x100000007e3:[<0x100000007de/commit-msg.sample v38>,<0x100000005e5/hooks v231>,<0x100000005e4/.git v886>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.016 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:58.668+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007e1 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/pre-commit.sample auth v50 ap=1 s=1643 n(v0 rc2023-12-01T01:18:37.636268+0000 b1643 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@1},l=24992(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563635f9f600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000007e1:[<0x100000007de/pre-commit.sample v6>,<0x100000005e5/hooks v36>,<0x100000005e4/.git v138>,<0x100000005e3/xfstests-dev v74>,<0x100000005e2/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/_nogroup v33>,<0x1/volumes v31>]//[]","memoryvalue":"(3)0x100000007e1:[<0x100000007de/pre-commit.sample v50>,<0x100000005e5/hooks v231>,<0x100000005e4/.git v886>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.016 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:58.668+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007e4 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/pre-rebase.sample auth v32 ap=1 s=4898 n(v0 rc2023-12-01T01:18:37.638268+0000 b4898 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@1},l=24992(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563635fa0680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000007e4:[<0x100000007de/pre-rebase.sample v12>,<0x100000005e5/hooks v36>,<0x100000005e4/.git v138>,<0x100000005e3/xfstests-dev v74>,<0x100000005e2/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/_nogroup v33>,<0x1/volumes v31>]//[]","memoryvalue":"(3)0x100000007e4:[<0x100000007de/pre-rebase.sample v32>,<0x100000005e5/hooks v231>,<0x100000005e4/.git v886>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.016 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:58.669+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007e7 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/fsmonitor-watchman.sample auth v40 ap=1 s=4655 n(v0 rc2023-12-01T01:18:37.640268+0000 b4655 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@1},l=24992(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563635f9fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000007e7:[<0x100000007de/fsmonitor-watchman.sample v18>,<0x100000005e5/hooks v36>,<0x100000005e4/.git v138>,<0x100000005e3/xfstests-dev v74>,<0x100000005e2/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/_nogroup v33>,<0x1/volumes v31>]//[]","memoryvalue":"(3)0x100000007e7:[<0x100000007de/fsmonitor-watchman.sample v40>,<0x100000005e5/hooks v231>,<0x100000005e4/.git v886>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.016 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:58.669+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007ea [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/pre-receive.sample auth v42 ap=1 s=544 n(v0 rc2023-12-01T01:18:37.643268+0000 b544 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@1},l=24992(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563635fa0c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000007ea:[<0x100000007de/pre-receive.sample v24>,<0x100000005e5/hooks v36>,<0x100000005e4/.git v138>,<0x100000005e3/xfstests-dev v74>,<0x100000005e2/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/_nogroup v33>,<0x1/volumes v31>]//[]","memoryvalue":"(3)0x100000007ea:[<0x100000007de/pre-receive.sample v42>,<0x100000005e5/hooks v231>,<0x100000005e4/.git v886>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.016 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:58.669+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007df [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/prepare-commit-msg.sample auth v34 ap=1 s=1492 n(v0 rc2023-12-01T01:18:37.634268+0000 b1492 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@1},l=24992(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563635fa1180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000007df:[<0x100000007de/prepare-commit-msg.sample v2>,<0x100000005e5/hooks v36>,<0x100000005e4/.git v138>,<0x100000005e3/xfstests-dev v74>,<0x100000005e2/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/_nogroup v33>,<0x1/volumes v31>]//[]","memoryvalue":"(3)0x100000007df:[<0x100000007de/prepare-commit-msg.sample v34>,<0x100000005e5/hooks v231>,<0x100000005e4/.git v886>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.016 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:58.669+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007e2 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/update.sample auth v44 ap=1 s=3650 n(v0 b3650 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@1},l=24992(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563635fa1700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000007e2:[<0x100000007de/update.sample v8>,<0x100000005e5/hooks v36>,<0x100000005e4/.git v138>,<0x100000005e3/xfstests-dev v74>,<0x100000005e2/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/_nogroup v33>,<0x1/volumes v31>]//[]","memoryvalue":"(3)0x100000007e2:[<0x100000007de/update.sample v44>,<0x100000005e5/hooks v231>,<0x100000005e4/.git v886>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.016 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:58.669+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007e0 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/pre-push.sample auth v48 ap=1 s=1374 n(v0 rc2023-12-01T01:18:37.635268+0000 b1374 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@1},l=24992(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563636072000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000007e0:[<0x100000007de/pre-push.sample v4>,<0x100000005e5/hooks v36>,<0x100000005e4/.git v138>,<0x100000005e3/xfstests-dev v74>,<0x100000005e2/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/_nogroup v33>,<0x1/volumes v31>]//[]","memoryvalue":"(3)0x100000007e0:[<0x100000007de/pre-push.sample v48>,<0x100000005e5/hooks v231>,<0x100000005e4/.git v886>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.016 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:58.669+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007e8 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/pre-applypatch.sample auth v52 ap=1 s=424 n(v0 rc2023-12-01T01:18:37.641268+0000 b424 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@1},l=24992(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563636072580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000007e8:[<0x100000007de/pre-applypatch.sample v20>,<0x100000005e5/hooks v36>,<0x100000005e4/.git v138>,<0x100000005e3/xfstests-dev v74>,<0x100000005e2/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/_nogroup v33>,<0x1/volumes v31>]//[]","memoryvalue":"(3)0x100000007e8:[<0x100000007de/pre-applypatch.sample v52>,<0x100000005e5/hooks v231>,<0x100000005e4/.git v886>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.016 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:58.669+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007eb [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/push-to-checkout.sample auth v36 ap=1 s=2783 n(v0 rc2023-12-01T01:18:37.644268+0000 b2783 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@1},l=24992(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563636072b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000007eb:[<0x100000007de/push-to-checkout.sample v26>,<0x100000005e5/hooks v36>,<0x100000005e4/.git v138>,<0x100000005e3/xfstests-dev v74>,<0x100000005e2/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/_nogroup v33>,<0x1/volumes v31>]//[]","memoryvalue":"(3)0x100000007eb:[<0x100000007de/push-to-checkout.sample v36>,<0x100000005e5/hooks v231>,<0x100000005e4/.git v886>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.016 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:58.670+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007e5 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/post-update.sample auth v28 ap=1 s=189 n(v0 rc2023-12-01T01:18:37.639268+0000 b189 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@1},l=24992(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563636073080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000007e5:[<0x100000007de/post-update.sample v14>,<0x100000005e5/hooks v36>,<0x100000005e4/.git v138>,<0x100000005e3/xfstests-dev v74>,<0x100000005e2/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/_nogroup v33>,<0x1/volumes v31>]//[]","memoryvalue":"(3)0x100000007e5:[<0x100000007de/post-update.sample v28>,<0x100000005e5/hooks v231>,<0x100000005e4/.git v886>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.016 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:58.670+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007e9 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/applypatch-msg.sample auth v30 ap=1 s=478 n(v0 rc2023-12-01T01:18:37.642268+0000 b478 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@1},l=24992(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563636073600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000007e9:[<0x100000007de/applypatch-msg.sample v22>,<0x100000005e5/hooks v36>,<0x100000005e4/.git v138>,<0x100000005e3/xfstests-dev v74>,<0x100000005e2/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/_nogroup v33>,<0x1/volumes v31>]//[]","memoryvalue":"(3)0x100000007e9:[<0x100000007de/applypatch-msg.sample v30>,<0x100000005e5/hooks v231>,<0x100000005e4/.git v886>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.016 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:58.670+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000023 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/HEAD auth v19 ap=1 s=196 n(v0 rc2023-12-01T01:18:39.606222+0000 b196 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@1},l=24992(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563636073b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000023:[<0x1000000001b/HEAD v6>,<0x100000005e5/logs v153>,<0x100000005e4/.git v138>,<0x100000005e3/xfstests-dev v74>,<0x100000005e2/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/_nogroup v33>,<0x1/volumes v31>]//[]","memoryvalue":"(3)0x10000000023:[<0x1000000001b/HEAD v19>,<0x100000005e5/logs v225>,<0x100000005e4/.git v886>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.016 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:58.670+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000001c [...2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/refs/ auth{0=1} v17 ap=1 f(v0 m2023-12-01T01:18:39.606222+0000 2=0+2) n(v1 rc2023-12-01T01:18:39.607222+0000 b196 6=2+4) (inest mix dirty) (ifile excl) (iversion lock) caps={24992=pAsLsXsFsx/pAsLsXsFsx@1},l=24992 | dirtyscattered=1 lock=0 importingcaps=0 dirfrag=1 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x563636074100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x1000000001c:[<0x1000000001b/refs v8>,<0x100000005e5/logs v153>,<0x100000005e4/.git v138>,<0x100000005e3/xfstests-dev v74>,<0x100000005e2/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/_nogroup v33>,<0x1/volumes v31>]//[]","memoryvalue":"(2)0x1000000001c:[<0x1000000001b/refs v17>,<0x100000005e5/logs v225>,<0x100000005e4/.git v886>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-12-01T01:18:39.606222+0000 2=0+2)","ondisk_value.rstat":"n(v0 rc2023-12-01T01:18:39.607222+0000 b196 6=2+4)","memory_value.dirstat":"f(v0 m2023-12-01T01:18:39.606222+0000 2=0+2)","memory_value.rstat":"n(v1 rc2023-12-01T01:18:39.607222+0000 b196 6=2+4)","error_str":""},"return_code":0} 2023-12-01T01:18:59.017 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:58.670+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007dd [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/info/exclude auth v4 ap=1 s=240 n(v0 rc2023-12-01T01:18:37.632268+0000 b240 1=1+0) (ifile excl) (iversion lock) cr={24992=0-4194304@1} caps={24992=pAsLsXsFsxcrwb/pFscr@2},l=24992 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563636084580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000007dd:[<0x100000007dc/exclude v2>,<0x100000005e5/info v8>,<0x100000005e4/.git v138>,<0x100000005e3/xfstests-dev v74>,<0x100000005e2/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/_nogroup v33>,<0x1/volumes v31>]//[]","memoryvalue":"(3)0x100000007dd:[<0x100000007dc/exclude v4>,<0x100000005e5/info v223>,<0x100000005e4/.git v886>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.017 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:58.671+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007ee [...2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/refs/tags/ auth v4 ap=1 f() n(v0 1=0+1) (inest lock) (ifile excl) (iversion lock) caps={24992=pAsLsXsFsx/pAsLsXsFsx@3},l=24992 | importingcaps=0 dirfrag=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x563636084b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x100000007ee:[<0x100000007ec/tags v4>,<0x100000005e5/refs v138>,<0x100000005e4/.git v138>,<0x100000005e3/xfstests-dev v74>,<0x100000005e2/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/_nogroup v33>,<0x1/volumes v31>]//[]","memoryvalue":"(2)0x100000007ee:[<0x100000007ec/tags v4>,<0x100000005e5/refs v237>,<0x100000005e4/.git v886>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":0} 2023-12-01T01:18:59.017 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:58.671+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007ed [...2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/refs/heads/ auth{0=1} v23 ap=1 f(v0 m2023-12-01T01:18:39.607222+0000 1=1+0) n(v1 rc2023-12-01T01:18:39.607222+0000 b41 2=1+1) (inest mix) (ifile excl) (iversion lock) caps={24992=pAsLsXsFsx/pAsLsXsFsx@2},l=24992 | dirtyscattered=0 lock=0 importingcaps=0 dirfrag=1 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x563636085080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x100000007ed:[<0x100000007ec/heads v14>,<0x100000005e5/refs v138>,<0x100000005e4/.git v138>,<0x100000005e3/xfstests-dev v74>,<0x100000005e2/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/_nogroup v33>,<0x1/volumes v31>]//[]","memoryvalue":"(2)0x100000007ed:[<0x100000007ec/heads v23>,<0x100000005e5/refs v237>,<0x100000005e4/.git v886>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-12-01T01:18:39.607222+0000 1=1+0)","ondisk_value.rstat":"n(v0 rc2023-12-01T01:18:39.607222+0000 b41 2=1+1)","memory_value.dirstat":"f(v0 m2023-12-01T01:18:39.607222+0000 1=1+0)","memory_value.rstat":"n(v1 rc2023-12-01T01:18:39.607222+0000 b41 2=1+1)","error_str":""},"return_code":0} 2023-12-01T01:18:59.017 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:58.671+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000018 [...2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/refs/remotes/ auth{0=1} v25 ap=1 f(v0 m2023-12-01T01:18:39.578223+0000 1=0+1) n(v1 rc2023-12-01T01:18:39.585223+0000 b32 3=1+2) (inest mix) (ifile excl) (iversion lock) caps={24992=pAsLsXsFsx/pAsLsXsFsx@2},l=24992 | lock=0 importingcaps=0 dirfrag=1 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x563636085600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000018:[<0x100000007ec/remotes v8>,<0x100000005e5/refs v138>,<0x100000005e4/.git v138>,<0x100000005e3/xfstests-dev v74>,<0x100000005e2/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/_nogroup v33>,<0x1/volumes v31>]//[]","memoryvalue":"(2)0x10000000018:[<0x100000007ec/remotes v25>,<0x100000005e5/refs v237>,<0x100000005e4/.git v886>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-12-01T01:18:39.578223+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-12-01T01:18:39.585223+0000 b32 3=1+2)","memory_value.dirstat":"f(v0 m2023-12-01T01:18:39.578223+0000 1=0+1)","memory_value.rstat":"n(v1 rc2023-12-01T01:18:39.585223+0000 b32 3=1+2)","error_str":""},"return_code":0} 2023-12-01T01:18:59.017 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:58.671+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000013 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/objects/pack/pack-fe9e56eb4d11faa0800e85314d6db62e368e37e2.idx auth v113 ap=1 s=1283192 n(v0 rc2023-12-01T01:18:39.508224+0000 b1283192 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={24992=0-4194304@1} caps={24992=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=24992 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x563635f9e580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000013:[<0x1000000000b/pack-fe9e56eb4d11faa0800e85314d6db62e368e37e2.idx v113>,<0x1000000000a/pack v42>,<0x100000005e5/objects v118>,<0x100000005e4/.git v138>,<0x100000005e3/xfstests-dev v74>,<0x100000005e2/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/_nogroup v33>,<0x1/volumes v31>]//[]","memoryvalue":"(3)0x10000000013:[<0x1000000000b/pack-fe9e56eb4d11faa0800e85314d6db62e368e37e2.idx v113>,<0x1000000000a/pack v42>,<0x100000005e5/objects v179>,<0x100000005e4/.git v886>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.017 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:58.671+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000012 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/objects/pack/pack-fe9e56eb4d11faa0800e85314d6db62e368e37e2.pack auth v117 ap=1 s=11702630 n(v0 rc2023-12-01T01:18:39.504224+0000 b11702630 1=1+0) (iversion lock) caps={24992=pAsLsXsFscr/pFscr@2},l=24992 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x563635f9eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000012:[<0x1000000000b/pack-fe9e56eb4d11faa0800e85314d6db62e368e37e2.pack v117>,<0x1000000000a/pack v42>,<0x100000005e5/objects v118>,<0x100000005e4/.git v138>,<0x100000005e3/xfstests-dev v74>,<0x100000005e2/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/_nogroup v33>,<0x1/volumes v31>]//[]","memoryvalue":"(3)0x10000000012:[<0x1000000000b/pack-fe9e56eb4d11faa0800e85314d6db62e368e37e2.pack v117>,<0x1000000000a/pack v42>,<0x100000005e5/objects v179>,<0x100000005e4/.git v886>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.017 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:58.672+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000024 [...2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/refs/heads/ auth{0=1} v15 ap=1 f(v0 m2023-12-01T01:18:39.607222+0000 1=1+0) n(v0 rc2023-12-01T01:18:39.607222+0000 b196 2=1+1) (inest mix) (ifile excl) (iversion lock) caps={24992=pAsLsXsFsx/pAsLsXsFsx@1},l=24992 | lock=0 importingcaps=0 dirfrag=1 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x563636074680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000024:[<0x1000000001c/heads v8>,<0x1000000001b/refs v8>,<0x100000005e5/logs v153>,<0x100000005e4/.git v138>,<0x100000005e3/xfstests-dev v74>,<0x100000005e2/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/_nogroup v33>,<0x1/volumes v31>]//[]","memoryvalue":"(2)0x10000000024:[<0x1000000001c/heads v15>,<0x1000000001b/refs v17>,<0x100000005e5/logs v225>,<0x100000005e4/.git v886>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-12-01T01:18:39.607222+0000 1=1+0)","ondisk_value.rstat":"n(v0 rc2023-12-01T01:18:39.607222+0000 b196 2=1+1)","memory_value.dirstat":"f(v0 m2023-12-01T01:18:39.607222+0000 1=1+0)","memory_value.rstat":"n(v0 rc2023-12-01T01:18:39.607222+0000 b196 2=1+1)","error_str":""},"return_code":0} 2023-12-01T01:18:59.017 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:58.672+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000001d [...2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/refs/remotes/ auth{0=1} v13 ap=1 f(v0 m2023-12-01T01:18:39.582223+0000 1=0+1) n(v1 rc2023-12-01T01:18:39.585223+0000 b196 3=1+2) (inest mix) (ifile excl) (iversion lock) caps={24992=pAsLsXsFsx/pAsLsXsFsx@1},l=24992 | lock=0 importingcaps=0 dirfrag=1 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x563636074c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x1000000001d:[<0x1000000001c/remotes v4>,<0x1000000001b/refs v8>,<0x100000005e5/logs v153>,<0x100000005e4/.git v138>,<0x100000005e3/xfstests-dev v74>,<0x100000005e2/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/_nogroup v33>,<0x1/volumes v31>]//[]","memoryvalue":"(2)0x1000000001d:[<0x1000000001c/remotes v13>,<0x1000000001b/refs v17>,<0x100000005e5/logs v225>,<0x100000005e4/.git v886>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-12-01T01:18:39.582223+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2023-12-01T01:18:39.585223+0000 b196 3=1+2)","memory_value.dirstat":"f(v0 m2023-12-01T01:18:39.582223+0000 1=0+1)","memory_value.rstat":"n(v1 rc2023-12-01T01:18:39.585223+0000 b196 3=1+2)","error_str":""},"return_code":0} 2023-12-01T01:18:59.017 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:58.672+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000022 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/refs/heads/master auth v8 ap=1 s=41 n(v0 rc2023-12-01T01:18:39.607222+0000 b41 1=1+0) (ifile excl) (iversion lock) cr={24992=0-4194304@1} caps={24992=pAsLsXsFsxcrwb/pFscr@2},l=24992 | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563636085b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000022:[<0x100000007ed/master v4>,<0x100000007ec/heads v14>,<0x100000005e5/refs v138>,<0x100000005e4/.git v138>,<0x100000005e3/xfstests-dev v74>,<0x100000005e2/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/_nogroup v33>,<0x1/volumes v31>]//[]","memoryvalue":"(3)0x10000000022:[<0x100000007ed/master v8>,<0x100000007ec/heads v23>,<0x100000005e5/refs v237>,<0x100000005e4/.git v886>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.017 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:58.672+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000019 [...2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/refs/remotes/origin/ auth{0=1} v13 ap=1 f(v0 m2023-12-01T01:18:39.585223+0000 1=1+0) n(v1 rc2023-12-01T01:18:39.585223+0000 b32 2=1+1) (inest mix) (ifile excl) (iversion lock) caps={24992=pAsLsXsFsx/pAsLsXsFsx@1},l=24992 | dirtyscattered=0 lock=0 importingcaps=0 dirfrag=1 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x563636086100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000019:[<0x10000000018/origin v8>,<0x100000007ec/remotes v8>,<0x100000005e5/refs v138>,<0x100000005e4/.git v138>,<0x100000005e3/xfstests-dev v74>,<0x100000005e2/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/_nogroup v33>,<0x1/volumes v31>]//[]","memoryvalue":"(2)0x10000000019:[<0x10000000018/origin v13>,<0x100000007ec/remotes v25>,<0x100000005e5/refs v237>,<0x100000005e4/.git v886>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-12-01T01:18:39.585223+0000 1=1+0)","ondisk_value.rstat":"n(v0 rc2023-12-01T01:18:39.585223+0000 b32 2=1+1)","memory_value.dirstat":"f(v0 m2023-12-01T01:18:39.585223+0000 1=1+0)","memory_value.rstat":"n(v1 rc2023-12-01T01:18:39.585223+0000 b32 2=1+1)","error_str":""},"return_code":0} 2023-12-01T01:18:59.017 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:58.672+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000025 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/refs/heads/master auth v4 ap=1 s=196 n(v0 b196 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@1},l=24992(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563636075180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000025:[<0x10000000024/master v2>,<0x1000000001c/heads v8>,<0x1000000001b/refs v8>,<0x100000005e5/logs v153>,<0x100000005e4/.git v138>,<0x100000005e3/xfstests-dev v74>,<0x100000005e2/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/_nogroup v33>,<0x1/volumes v31>]//[]","memoryvalue":"(3)0x10000000025:[<0x10000000024/master v4>,<0x1000000001c/heads v15>,<0x1000000001b/refs v17>,<0x100000005e5/logs v225>,<0x100000005e4/.git v886>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.017 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:58.673+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000001e [...2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/refs/remotes/origin/ auth{0=1} v7 ap=1 f(v0 m2023-12-01T01:18:39.583223+0000 1=1+0) n(v0 rc2023-12-01T01:18:39.585223+0000 b196 2=1+1) (inest mix) (ifile excl) (iversion lock) caps={24992=pAsLsXsFsx/pAsLsXsFsx@1},l=24992 | lock=0 importingcaps=0 dirfrag=1 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x563636075700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x1000000001e:[<0x1000000001d/origin v4>,<0x1000000001c/remotes v4>,<0x1000000001b/refs v8>,<0x100000005e5/logs v153>,<0x100000005e4/.git v138>,<0x100000005e3/xfstests-dev v74>,<0x100000005e2/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/_nogroup v33>,<0x1/volumes v31>]//[]","memoryvalue":"(2)0x1000000001e:[<0x1000000001d/origin v7>,<0x1000000001c/remotes v13>,<0x1000000001b/refs v17>,<0x100000005e5/logs v225>,<0x100000005e4/.git v886>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-12-01T01:18:39.583223+0000 1=1+0)","ondisk_value.rstat":"n(v0 rc2023-12-01T01:18:39.585223+0000 b196 2=1+1)","memory_value.dirstat":"f(v0 m2023-12-01T01:18:39.583223+0000 1=1+0)","memory_value.rstat":"n(v0 rc2023-12-01T01:18:39.585223+0000 b196 2=1+1)","error_str":""},"return_code":0} 2023-12-01T01:18:59.017 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:58.673+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000001a [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/refs/remotes/origin/HEAD auth v8 ap=1 s=32 n(v0 rc2023-12-01T01:18:39.585223+0000 b32 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@1},l=24992(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 dirtyrstat=0 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563636086680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000001a:[<0x10000000019/HEAD v4>,<0x10000000018/origin v8>,<0x100000007ec/remotes v8>,<0x100000005e5/refs v138>,<0x100000005e4/.git v138>,<0x100000005e3/xfstests-dev v74>,<0x100000005e2/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/_nogroup v33>,<0x1/volumes v31>]//[]","memoryvalue":"(3)0x1000000001a:[<0x10000000019/HEAD v8>,<0x10000000018/origin v13>,<0x100000007ec/remotes v25>,<0x100000005e5/refs v237>,<0x100000005e4/.git v886>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.018 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:58.673+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000001f [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/refs/remotes/origin/HEAD auth v4 ap=1 s=196 n(v0 rc2023-12-01T01:18:39.585223+0000 b196 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@1},l=24992(-1) | ptrwaiter=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563636084000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000001f:[<0x1000000001e/HEAD v2>,<0x1000000001d/origin v4>,<0x1000000001c/remotes v4>,<0x1000000001b/refs v8>,<0x100000005e5/logs v153>,<0x100000005e4/.git v138>,<0x100000005e3/xfstests-dev v74>,<0x100000005e2/tmp v6>,<0x100000001fb/client.0 v8>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v24>,<0x10000000001/sv_1 v47>,<0x10000000000/_nogroup v33>,<0x1/volumes v31>]//[]","memoryvalue":"(3)0x1000000001f:[<0x1000000001e/HEAD v4>,<0x1000000001d/origin v7>,<0x1000000001c/remotes v13>,<0x1000000001b/refs v17>,<0x100000005e5/logs v225>,<0x100000005e4/.git v886>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.762 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: 2023-12-01T01:18:59.411+0000 7fbefb03d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000193 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/004 auth v539 ap=1 s=6448 n(v0 b6448 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ea5f103700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000193:[<0x1000000018c/004 v14>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x10000000193:[<0x1000000018c/004 v539>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.762 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: 2023-12-01T01:18:59.411+0000 7fbefb03d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f0 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/050.out auth v485 ap=1 s=25 n(v0 b25 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ea5f12a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000001f0:[<0x1000000018c/050.out v200>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000001f0:[<0x1000000018c/050.out v485>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.763 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:59.411+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001c2 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/027.out auth v471 ap=1 s=43 n(v0 rc2023-12-01T01:18:40.076211+0000 b43 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563636222b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000001c2:[<0x1000000018c/027.out v108>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000001c2:[<0x1000000018c/027.out v471>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.763 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:59.411+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000201 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/056.out auth v485 ap=1 s=2419 n(v0 rc2023-12-01T01:18:40.120210+0000 b2419 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563636223b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000201:[<0x1000000018c/056.out v224>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x10000000201:[<0x1000000018c/056.out v485>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: scrub queued for path: / 2023-12-01T01:18:59.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: scrub summary: idle+waiting paths [/] 2023-12-01T01:18:59.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: scrub summary: active paths [/] 2023-12-01T01:18:59.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: pgmap v331: 97 pgs: 97 active+clean; 19 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 525 KiB/s wr, 106 op/s 2023-12-01T01:18:59.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: scrub summary: active paths [/] 2023-12-01T01:18:59.764 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: Scrub error on inode 0x10000000020 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/HEAD) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.764 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000005e7 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/description) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.764 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: Scrub error on inode 0x1000000000a (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/objects) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.764 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000007de (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.764 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: Scrub error on inode 0x1000000001b (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.764 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: Scrub error on inode 0x10000000017 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/packed-refs) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.764 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: Scrub error on inode 0x10000000027 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/config) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.764 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000007dc (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/info) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.764 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: Scrub error on inode 0x10000000015 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/shallow) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.764 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000005e6 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/branches) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.764 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: Scrub error on inode 0x1000000000c (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/objects/info) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.764 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000007ec (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/refs) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.764 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: Scrub error on inode 0x1000000000b (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/objects/pack) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.764 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000007e6 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/pre-merge-commit.sample) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.764 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000007e3 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/commit-msg.sample) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.765 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:59.411+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007fc [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/063.out auth v508 ap=1 s=43 n(v0 rc2023-12-01T01:18:40.128210+0000 b43 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563636224100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000007fc:[<0x1000000018c/063.out v252>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000007fc:[<0x1000000018c/063.out v508>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.765 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:59.412+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001b1 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/019 auth v497 ap=1 s=2496 n(v0 rc2023-12-01T01:18:40.063211+0000 b2496 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56363624ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000001b1:[<0x1000000018c/019 v74>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000001b1:[<0x1000000018c/019 v497>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.765 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:59.412+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001d7 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/038 auth v451 ap=1 s=2917 n(v0 rc2023-12-01T01:18:40.093211+0000 b2917 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563636225180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000001d7:[<0x1000000018c/038 v150>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000001d7:[<0x1000000018c/038 v451>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.765 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: 2023-12-01T01:18:59.411+0000 7fbefb03d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000018e [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/001.out auth v465 ap=1 s=695 n(v0 b695 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ea5f12b700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000018e:[<0x1000000018c/001.out v4>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x1000000018e:[<0x1000000018c/001.out v465>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.765 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: 2023-12-01T01:18:59.411+0000 7fbefb03d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001cc [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/032.out auth v541 ap=1 s=126 n(v0 b126 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ea5f135080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000001cc:[<0x1000000018c/032.out v128>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000001cc:[<0x1000000018c/032.out v541>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.765 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: 2023-12-01T01:18:59.412+0000 7fbefb03d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001c1 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/027 auth v479 ap=1 s=2430 n(v0 rc2023-12-01T01:18:40.075211+0000 b2430 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ea5f13c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000001c1:[<0x1000000018c/027 v106>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000001c1:[<0x1000000018c/027 v479>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.766 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:59.413+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001b7 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/022 auth v467 ap=1 s=4063 n(v0 b4063 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56363624d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000001b7:[<0x1000000018c/022 v86>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000001b7:[<0x1000000018c/022 v467>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.766 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:59.413+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000811 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/074 auth v500 ap=1 s=2248 n(v0 rc2023-12-01T01:18:40.140210+0000 b2248 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56363624d700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000811:[<0x1000000018c/074 v294>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x10000000811:[<0x1000000018c/074 v500>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.766 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: 2023-12-01T01:18:59.412+0000 7fbefb03d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001ce [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/033.out auth v515 ap=1 s=43 n(v0 rc2023-12-01T01:18:40.085211+0000 b43 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ea5f13e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000001ce:[<0x1000000018c/033.out v132>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000001ce:[<0x1000000018c/033.out v515>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.766 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: 2023-12-01T01:18:59.412+0000 7fbefb03d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001e1 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/043 auth v497 ap=1 s=3518 n(v0 rc2023-12-01T01:18:40.101211+0000 b3518 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ea5f13e100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000001e1:[<0x1000000018c/043 v170>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000001e1:[<0x1000000018c/043 v497>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.767 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:59.413+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001eb [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/048 auth v455 ap=1 s=8577 n(v0 rc2023-12-01T01:18:40.108210+0000 b8577 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563636257080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000001eb:[<0x1000000018c/048 v190>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000001eb:[<0x1000000018c/048 v455>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.767 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:59.413+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001a6 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/013.out auth v483 ap=1 s=43 n(v0 b43 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563636257b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000001a6:[<0x1000000018c/013.out v52>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000001a6:[<0x1000000018c/013.out v483>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.767 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:59.414+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001d3 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/036 auth v469 ap=1 s=1543 n(v0 rc2023-12-01T01:18:40.089211+0000 b1543 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563636259180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000001d3:[<0x1000000018c/036 v142>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000001d3:[<0x1000000018c/036 v469>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.767 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000007e1 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/pre-commit.sample) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.767 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000007e4 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/pre-rebase.sample) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.767 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000007e7 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/fsmonitor-watchman.sample) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.767 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000007ea (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/pre-receive.sample) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.767 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000007df (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/prepare-commit-msg.sample) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.767 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000007e2 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/update.sample) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.767 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000007e0 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/pre-push.sample) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.767 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000007e8 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/pre-applypatch.sample) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.768 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000007eb (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/push-to-checkout.sample) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.768 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000007e5 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/post-update.sample) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.768 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000007e9 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/applypatch-msg.sample) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.768 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: Scrub error on inode 0x10000000023 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/HEAD) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.768 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: Scrub error on inode 0x1000000001c (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/refs) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.768 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000007dd (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/info/exclude) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.768 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000007ee (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/refs/tags) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.768 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000007ed (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/refs/heads) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.768 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: Scrub error on inode 0x10000000018 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/refs/remotes) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.768 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:59.414+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000080e [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/072.out auth v449 ap=1 s=43 n(v0 b43 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563636259700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000080e:[<0x1000000018c/072.out v288>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x1000000080e:[<0x1000000018c/072.out v449>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.768 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:59.415+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001ff [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/055.out auth v502 ap=1 s=9130 n(v0 rc2023-12-01T01:18:40.118210+0000 b9130 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563636265180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000001ff:[<0x1000000018c/055.out v220>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000001ff:[<0x1000000018c/055.out v502>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.769 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: 2023-12-01T01:18:59.413+0000 7fbefb03d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000813 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/075 auth v551 ap=1 s=1136 n(v0 b1136 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ea5f13ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000813:[<0x1000000018c/075 v298>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x10000000813:[<0x1000000018c/075 v551>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.769 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: 2023-12-01T01:18:59.414+0000 7fbefb03d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001d9 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/039 auth v499 ap=1 s=3022 n(v0 b3022 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ea5f155080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000001d9:[<0x1000000018c/039 v154>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000001d9:[<0x1000000018c/039 v499>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.769 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: 2023-12-01T01:18:59.414+0000 7fbefb03d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000814 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/075.out auth v489 ap=1 s=43 n(v0 rc2023-12-01T01:18:40.142210+0000 b43 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ea5f0df080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000814:[<0x1000000018c/075.out v300>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x10000000814:[<0x1000000018c/075.out v489>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.769 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: Scrub error on inode 0x10000000013 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/objects/pack/pack-fe9e56eb4d11faa0800e85314d6db62e368e37e2.idx) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.769 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: Scrub error on inode 0x10000000012 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/objects/pack/pack-fe9e56eb4d11faa0800e85314d6db62e368e37e2.pack) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.769 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: Scrub error on inode 0x10000000024 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/refs/heads) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.769 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: Scrub error on inode 0x1000000001d (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/refs/remotes) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.770 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: Scrub error on inode 0x10000000022 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/refs/heads/master) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.770 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: Scrub error on inode 0x10000000019 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/refs/remotes/origin) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.770 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: Scrub error on inode 0x10000000025 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/refs/heads/master) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.770 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: Scrub error on inode 0x1000000001e (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/refs/remotes/origin) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.770 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: Scrub error on inode 0x1000000001a (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/refs/remotes/origin/HEAD) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.770 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: Scrub error on inode 0x1000000001f (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/refs/remotes/origin/HEAD) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.770 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: scrub summary: idle+waiting paths [/] 2023-12-01T01:18:59.770 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: scrub summary: active paths [/] 2023-12-01T01:18:59.770 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: scrub summary: active paths [/] 2023-12-01T01:18:59.770 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: scrub summary: idle+waiting paths [/] 2023-12-01T01:18:59.770 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-mon[103581]: scrub summary: idle+waiting paths [/] 2023-12-01T01:18:59.771 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: 2023-12-01T01:18:59.414+0000 7fbefb03d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000080f [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/073 auth v507 ap=1 s=2117 n(v0 rc2023-12-01T01:18:40.139210+0000 b2117 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ea5f155600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000080f:[<0x1000000018c/073 v290>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x1000000080f:[<0x1000000018c/073 v507>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.771 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: 2023-12-01T01:18:59.414+0000 7fbefb03d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001c4 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/028.out auth v535 ap=1 s=43 n(v0 rc2023-12-01T01:18:40.077211+0000 b43 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ea5f155b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000001c4:[<0x1000000018c/028.out v112>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000001c4:[<0x1000000018c/028.out v535>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.771 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: 2023-12-01T01:18:59.415+0000 7fbefb03d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001d5 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/037 auth v509 ap=1 s=3087 n(v0 rc2023-12-01T01:18:40.091211+0000 b3087 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ea5f157700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000001d5:[<0x1000000018c/037 v146>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000001d5:[<0x1000000018c/037 v509>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.771 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: 2023-12-01T01:18:59.415+0000 7fbefb03d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007f4 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/059.out auth v473 ap=1 s=340 n(v0 b340 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ea5f15f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000007f4:[<0x1000000018c/059.out v236>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000007f4:[<0x1000000018c/059.out v473>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.771 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: 2023-12-01T01:18:59.416+0000 7fbefb03d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000801 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/066 auth v543 ap=1 s=2290 n(v0 rc2023-12-01T01:18:40.131210+0000 b2290 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ea5f161180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000801:[<0x1000000018c/066 v262>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x10000000801:[<0x1000000018c/066 v543>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.771 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: 2023-12-01T01:18:59.416+0000 7fbefb03d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001e7 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/046 auth v531 ap=1 s=11673 n(v0 rc2023-12-01T01:18:40.105210+0000 b11673 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ea5f09e000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000001e7:[<0x1000000018c/046 v182>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000001e7:[<0x1000000018c/046 v531>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.771 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: 2023-12-01T01:18:59.417+0000 7fbefb03d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000019d [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/009 auth v603 ap=1 s=886 n(v0 rc2023-12-01T01:18:40.048212+0000 b886 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ea5f09eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000019d:[<0x1000000018c/009 v34>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x1000000019d:[<0x1000000018c/009 v603>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.771 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: 2023-12-01T01:18:59.417+0000 7fbefb03d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000080c [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/071.out auth v557 ap=1 s=43 n(v0 b43 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ea5f0a0680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000080c:[<0x1000000018c/071.out v284>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x1000000080c:[<0x1000000018c/071.out v557>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.771 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: 2023-12-01T01:18:59.417+0000 7fbefb03d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001ec [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/048.out auth v495 ap=1 s=1835 n(v0 rc2023-12-01T01:18:40.109210+0000 b1835 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ea5f0a1700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000001ec:[<0x1000000018c/048.out v192>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000001ec:[<0x1000000018c/048.out v495>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.771 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: 2023-12-01T01:18:59.418+0000 7fbefb03d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001e6 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/045.out auth v569 ap=1 s=25 n(v0 b25 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ea5f0a1180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000001e6:[<0x1000000018c/045.out v180>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000001e6:[<0x1000000018c/045.out v569>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.772 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:59.415+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007f7 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/061 auth v461 ap=1 s=2069 n(v0 rc2023-12-01T01:18:40.125210+0000 b2069 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563636264100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000007f7:[<0x1000000018c/061 v242>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000007f7:[<0x1000000018c/061 v461>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.772 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:59.415+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000803 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/067 auth v463 ap=1 s=2449 n(v0 rc2023-12-01T01:18:40.132210+0000 b2449 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563636264680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000803:[<0x1000000018c/067 v266>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x10000000803:[<0x1000000018c/067 v463>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.772 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:59.416+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000202 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/057 auth v487 ap=1 s=989 n(v0 b989 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56363626e580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000202:[<0x1000000018c/057 v226>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x10000000202:[<0x1000000018c/057 v487>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.772 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:59.417+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000203 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/057.out auth v518 ap=1 s=43 n(v0 rc2023-12-01T01:18:40.121210+0000 b43 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56363627c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000203:[<0x1000000018c/057.out v228>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x10000000203:[<0x1000000018c/057.out v518>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.772 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:59.417+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000019b [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/008 auth v499 ap=1 s=1991 n(v0 b1991 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5636360e0000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000019b:[<0x1000000018c/008 v30>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x1000000019b:[<0x1000000018c/008 v499>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.772 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:59.418+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000019a [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/007.out auth v521 ap=1 s=58 n(v0 b58 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5636360e0580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000019a:[<0x1000000018c/007.out v28>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x1000000019a:[<0x1000000018c/007.out v521>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.772 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:59.418+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001bb [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/024 auth v525 ap=1 s=986 n(v0 rc2023-12-01T01:18:40.070211+0000 b986 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5636360e2680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000001bb:[<0x1000000018c/024 v94>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000001bb:[<0x1000000018c/024 v525>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.772 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:59.419+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007fb [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/063 auth v586 ap=1 s=2136 n(v0 b2136 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5636360e2c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000007fb:[<0x1000000018c/063 v250>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000007fb:[<0x1000000018c/063 v586>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.772 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:59.419+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000806 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/068.out auth v483 ap=1 s=43 n(v0 rc2023-12-01T01:18:40.134210+0000 b43 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5636360e2100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000806:[<0x1000000018c/068.out v272>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x10000000806:[<0x1000000018c/068.out v483>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.772 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:59.419+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007fa [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/062.out auth v541 ap=1 s=43 n(v0 rc2023-12-01T01:18:40.127210+0000 b43 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5636360e3180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000007fa:[<0x1000000018c/062.out v248>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000007fa:[<0x1000000018c/062.out v541>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.773 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:59.419+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fe [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/055 auth v507 ap=1 s=5906 n(v0 b5906 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5636360e3700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000001fe:[<0x1000000018c/055 v218>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000001fe:[<0x1000000018c/055 v507>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.773 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: 2023-12-01T01:18:59.418+0000 7fbefb03d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001a0 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/010.out auth v575 ap=1 s=43 n(v0 rc2023-12-01T01:18:40.050212+0000 b43 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ea5f0acb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000001a0:[<0x1000000018c/010.out v40>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000001a0:[<0x1000000018c/010.out v575>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.773 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: 2023-12-01T01:18:59.418+0000 7fbefb03d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000815 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/076 auth v541 ap=1 s=1668 n(v0 b1668 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ea5f0ad600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000815:[<0x1000000018c/076 v302>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x10000000815:[<0x1000000018c/076 v541>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.773 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: 2023-12-01T01:18:59.419+0000 7fbefb03d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001c0 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/026.out auth v539 ap=1 s=742 n(v0 rc2023-12-01T01:18:40.074211+0000 b742 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ea5f0aec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000001c0:[<0x1000000018c/026.out v104>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000001c0:[<0x1000000018c/026.out v539>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.773 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: 2023-12-01T01:18:59.419+0000 7fbefb03d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001be [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/025.out auth v527 ap=1 s=307 n(v0 rc2023-12-01T01:18:40.073211+0000 b307 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ea5f0c8580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000001be:[<0x1000000018c/025.out v100>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000001be:[<0x1000000018c/025.out v527>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.773 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: 2023-12-01T01:18:59.419+0000 7fbefb03d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001a5 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/013 auth v581 ap=1 s=1583 n(v0 rc2023-12-01T01:18:40.054212+0000 b1583 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ea5f0c9600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000001a5:[<0x1000000018c/013 v50>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000001a5:[<0x1000000018c/013 v581>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.773 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: 2023-12-01T01:18:59.420+0000 7fbefb03d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007f6 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/060.out auth v529 ap=1 s=43 n(v0 b43 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ea5f0cb700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000007f6:[<0x1000000018c/060.out v240>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000007f6:[<0x1000000018c/060.out v529>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.773 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: 2023-12-01T01:18:59.420+0000 7fbefb03d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001a9 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/015 auth v525 ap=1 s=658 n(v0 rc2023-12-01T01:18:40.057212+0000 b658 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ea5f0d4580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000001a9:[<0x1000000018c/015 v58>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000001a9:[<0x1000000018c/015 v525>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.774 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: 2023-12-01T01:18:59.421+0000 7fbefb03d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007fd [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/064 auth v501 ap=1 s=2854 n(v0 b2854 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ea5f0de580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000007fd:[<0x1000000018c/064 v254>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000007fd:[<0x1000000018c/064 v501>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.774 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: 2023-12-01T01:18:59.421+0000 7fbefb03d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001bf [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/026 auth v537 ap=1 s=3125 n(v0 b3125 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ea5f0deb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000001bf:[<0x1000000018c/026 v102>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000001bf:[<0x1000000018c/026 v537>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.774 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: 2023-12-01T01:18:59.422+0000 7fbefb03d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000200 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/056 auth v547 ap=1 s=4098 n(v0 rc2023-12-01T01:18:40.119210+0000 b4098 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ea5f0e0680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000200:[<0x1000000018c/056 v222>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x10000000200:[<0x1000000018c/056 v547>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.774 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:59.420+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000802 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/066.out auth v487 ap=1 s=43 n(v0 b43 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563636104c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000802:[<0x1000000018c/066.out v264>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x10000000802:[<0x1000000018c/066.out v487>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.774 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:59.420+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001ee [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/049.out auth v485 ap=1 s=43 n(v0 rc2023-12-01T01:18:40.110210+0000 b43 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563636105700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000001ee:[<0x1000000018c/049.out v196>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000001ee:[<0x1000000018c/049.out v485>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.774 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:59.421+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f8 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/054.out auth v539 ap=1 s=25 n(v0 rc2023-12-01T01:18:40.117210+0000 b25 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563636110c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000001f8:[<0x1000000018c/054.out v216>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000001f8:[<0x1000000018c/054.out v539>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.774 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:59.422+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f6 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/053.out auth v594 ap=1 s=25 n(v0 rc2023-12-01T01:18:40.116210+0000 b25 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563636116580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000001f6:[<0x1000000018c/053.out v212>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000001f6:[<0x1000000018c/053.out v594>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.774 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:59.422+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001e5 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/045 auth v511 ap=1 s=12842 n(v0 rc2023-12-01T01:18:40.104210+0000 b12842 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563636118c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000001e5:[<0x1000000018c/045 v178>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000001e5:[<0x1000000018c/045 v511>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.774 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:59.422+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007f8 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/061.out auth v561 ap=1 s=43 n(v0 rc2023-12-01T01:18:40.126210+0000 b43 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563636120000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000007f8:[<0x1000000018c/061.out v244>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000007f8:[<0x1000000018c/061.out v561>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.775 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:59.423+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000019f [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/010 auth v545 ap=1 s=1324 n(v0 rc2023-12-01T01:18:40.049212+0000 b1324 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563636122c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000019f:[<0x1000000018c/010 v38>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x1000000019f:[<0x1000000018c/010 v545>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.775 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:59.423+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001a1 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/011 auth v583 ap=1 s=8198 n(v0 rc2023-12-01T01:18:40.051212+0000 b8198 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563636123180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000001a1:[<0x1000000018c/011 v42>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000001a1:[<0x1000000018c/011 v583>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.775 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:59.423+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001bd [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/025 auth v598 ap=1 s=2158 n(v0 rc2023-12-01T01:18:40.072211+0000 b2158 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56363612c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000001bd:[<0x1000000018c/025 v98>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000001bd:[<0x1000000018c/025 v598>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.775 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:59.423+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007f5 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/060 auth v559 ap=1 s=2320 n(v0 rc2023-12-01T01:18:40.124210+0000 b2320 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56363612c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000007f5:[<0x1000000018c/060 v238>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000007f5:[<0x1000000018c/060 v559>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.775 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: 2023-12-01T01:18:59.422+0000 7fbefb03d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f4 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/052.out auth v573 ap=1 s=15846 n(v0 rc2023-12-01T01:18:40.114210+0000 b15846 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ea5f0e8000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000001f4:[<0x1000000018c/052.out v208>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000001f4:[<0x1000000018c/052.out v573>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.775 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: 2023-12-01T01:18:59.423+0000 7fbefb03d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001e0 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/042.out auth v599 ap=1 s=43 n(v0 rc2023-12-01T01:18:40.100211+0000 b43 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ea5f0e9600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000001e0:[<0x1000000018c/042.out v168>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000001e0:[<0x1000000018c/042.out v599>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.775 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: 2023-12-01T01:18:59.424+0000 7fbefb03d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001bc [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/024.out auth v535 ap=1 s=280 n(v0 rc2023-12-01T01:18:40.071211+0000 b280 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ea5f0f6680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000001bc:[<0x1000000018c/024.out v96>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000001bc:[<0x1000000018c/024.out v535>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.775 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: 2023-12-01T01:18:59.424+0000 7fbefb03d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001a8 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/014.out auth v553 ap=1 s=68 n(v0 rc2023-12-01T01:18:40.056212+0000 b68 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ea5f0f6100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000001a8:[<0x1000000018c/014.out v56>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000001a8:[<0x1000000018c/014.out v553>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.775 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: 2023-12-01T01:18:59.425+0000 7fbefb03d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001ba [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/023.out auth v505 ap=1 s=43 n(v0 b43 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ea5f0f7700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000001ba:[<0x1000000018c/023.out v92>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000001ba:[<0x1000000018c/023.out v505>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.776 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: 2023-12-01T01:18:59.425+0000 7fbefb03d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f1 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/051 auth v549 ap=1 s=1716 n(v0 rc2023-12-01T01:18:40.112210+0000 b1716 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ea5f100580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000001f1:[<0x1000000018c/051 v202>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000001f1:[<0x1000000018c/051 v549>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.776 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: 2023-12-01T01:18:59.425+0000 7fbefb03d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000805 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/068 auth v565 ap=1 s=2382 n(v0 b2382 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ea5f101600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000805:[<0x1000000018c/068 v270>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x10000000805:[<0x1000000018c/068 v565>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.776 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: 2023-12-01T01:18:59.426+0000 7fbefb03d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000080a [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/070.out auth v503 ap=1 s=43 n(v0 rc2023-12-01T01:18:40.136210+0000 b43 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ea5f102100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000080a:[<0x1000000018c/070.out v280>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x1000000080a:[<0x1000000018c/070.out v503>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.776 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: 2023-12-01T01:18:59.426+0000 7fbefb03d700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000018d [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/001 auth v497 ap=1 s=2165 n(v0 rc2023-12-01T01:18:40.037212+0000 b2165 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ea5f102c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000018d:[<0x1000000018c/001 v2>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x1000000018d:[<0x1000000018c/001 v497>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.776 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:59.424+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000080d [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/072 auth v563 ap=1 s=2174 n(v0 rc2023-12-01T01:18:40.138210+0000 b2174 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x56363612f700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x1000000080d:[<0x1000000018c/072 v286>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x1000000080d:[<0x1000000018c/072 v563>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.776 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:59.425+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000190 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/002.out auth v489 ap=1 s=43 n(v0 b43 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563636136000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x10000000190:[<0x1000000018c/002.out v8>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x10000000190:[<0x1000000018c/002.out v489>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.776 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:59.426+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001d4 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/036.out auth v493 ap=1 s=54 n(v0 rc2023-12-01T01:18:40.090211+0000 b54 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563636144580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000001d4:[<0x1000000018c/036.out v144>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000001d4:[<0x1000000018c/036.out v493>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.776 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:59.426+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f7 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/054 auth v495 ap=1 s=2543 n(v0 b2543 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563636145600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000001f7:[<0x1000000018c/054 v214>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000001f7:[<0x1000000018c/054 v495>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.776 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:18:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:18:59.426+0000 7ff2eabb2700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007f1 [2,head] /volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/058 auth v529 ap=1 s=1556 n(v0 b1556 1=1+0) (iversion lock) caps={24992=pAsLsXsFsc/-@2},l=24992(-1) | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x563636145b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(3)0x100000007f1:[<0x1000000018c/058 v230>,<0x1000000018a/btrfs v311>,<0x100000005e4/tests v777>,<0x100000005e3/xfstests-dev v79>,<0x100000005e2/tmp v12>,<0x100000001fb/client.0 v10>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v26>,<0x10000000001/sv_1 v49>,<0x10000000000/_nogroup v35>,<0x1/volumes v33>]//[]","memoryvalue":"(3)0x100000007f1:[<0x1000000018c/058 v529>,<0x1000000018a/btrfs v7825>,<0x100000005e4/tests v982>,<0x100000005e3/xfstests-dev v108>,<0x100000005e2/tmp v27>,<0x100000001fb/client.0 v22>,<0x100000001fa/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39 v36>,<0x10000000001/sv_1 v36>,<0x10000000000/_nogroup v27>,<0x1/volumes v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-12-01T01:18:59.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: scrub queued for path: / 2023-12-01T01:18:59.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: scrub summary: idle+waiting paths [/] 2023-12-01T01:18:59.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: scrub summary: active paths [/] 2023-12-01T01:18:59.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: pgmap v331: 97 pgs: 97 active+clean; 19 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 525 KiB/s wr, 106 op/s 2023-12-01T01:18:59.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: scrub summary: active paths [/] 2023-12-01T01:18:59.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: Scrub error on inode 0x10000000020 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/HEAD) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000005e7 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/description) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: Scrub error on inode 0x1000000000a (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/objects) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000007de (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: Scrub error on inode 0x1000000001b (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: Scrub error on inode 0x10000000017 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/packed-refs) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: Scrub error on inode 0x10000000027 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/config) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000007dc (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/info) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: Scrub error on inode 0x10000000015 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/shallow) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000005e6 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/branches) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: Scrub error on inode 0x1000000000c (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/objects/info) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000007ec (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/refs) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: Scrub error on inode 0x1000000000b (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/objects/pack) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000007e6 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/pre-merge-commit.sample) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000007e3 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/commit-msg.sample) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000007e1 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/pre-commit.sample) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000007e4 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/pre-rebase.sample) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000007e7 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/fsmonitor-watchman.sample) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000007ea (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/pre-receive.sample) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000007df (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/prepare-commit-msg.sample) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000007e2 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/update.sample) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000007e0 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/pre-push.sample) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000007e8 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/pre-applypatch.sample) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000007eb (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/push-to-checkout.sample) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000007e5 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/post-update.sample) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000007e9 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/applypatch-msg.sample) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: Scrub error on inode 0x10000000023 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/HEAD) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: Scrub error on inode 0x1000000001c (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/refs) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000007dd (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/info/exclude) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.907 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000007ee (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/refs/tags) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.907 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000007ed (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/refs/heads) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.907 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: Scrub error on inode 0x10000000018 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/refs/remotes) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.907 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: Scrub error on inode 0x10000000013 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/objects/pack/pack-fe9e56eb4d11faa0800e85314d6db62e368e37e2.idx) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.907 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: Scrub error on inode 0x10000000012 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/objects/pack/pack-fe9e56eb4d11faa0800e85314d6db62e368e37e2.pack) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.907 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: Scrub error on inode 0x10000000024 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/refs/heads) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.907 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: Scrub error on inode 0x1000000001d (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/refs/remotes) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.907 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: Scrub error on inode 0x10000000022 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/refs/heads/master) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.907 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: Scrub error on inode 0x10000000019 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/refs/remotes/origin) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.907 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: Scrub error on inode 0x10000000025 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/refs/heads/master) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.907 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: Scrub error on inode 0x1000000001e (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/refs/remotes/origin) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.907 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: Scrub error on inode 0x1000000001a (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/refs/remotes/origin/HEAD) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.907 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: Scrub error on inode 0x1000000001f (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/refs/remotes/origin/HEAD) see mds.i log and `damage ls` output for details 2023-12-01T01:18:59.907 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: scrub summary: idle+waiting paths [/] 2023-12-01T01:18:59.907 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: scrub summary: active paths [/] 2023-12-01T01:18:59.908 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: scrub summary: active paths [/] 2023-12-01T01:18:59.908 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: scrub summary: idle+waiting paths [/] 2023-12-01T01:18:59.908 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:18:59 smithi077 ceph-mon[98867]: scrub summary: idle+waiting paths [/] 2023-12-01T01:19:00.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: scrub queued for path: / 2023-12-01T01:19:00.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: scrub summary: idle+waiting paths [/] 2023-12-01T01:19:00.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: scrub summary: active paths [/] 2023-12-01T01:19:00.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: pgmap v331: 97 pgs: 97 active+clean; 19 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 525 KiB/s wr, 106 op/s 2023-12-01T01:19:00.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: scrub summary: active paths [/] 2023-12-01T01:19:00.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: Scrub error on inode 0x10000000020 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/HEAD) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000005e7 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/description) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: Scrub error on inode 0x1000000000a (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/objects) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000007de (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: Scrub error on inode 0x1000000001b (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: Scrub error on inode 0x10000000017 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/packed-refs) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: Scrub error on inode 0x10000000027 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/config) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000007dc (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/info) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: Scrub error on inode 0x10000000015 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/shallow) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000005e6 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/branches) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: Scrub error on inode 0x1000000000c (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/objects/info) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000007ec (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/refs) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: Scrub error on inode 0x1000000000b (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/objects/pack) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000007e6 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/pre-merge-commit.sample) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000007e3 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/commit-msg.sample) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000007e1 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/pre-commit.sample) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000007e4 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/pre-rebase.sample) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000007e7 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/fsmonitor-watchman.sample) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000007ea (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/pre-receive.sample) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000007df (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/prepare-commit-msg.sample) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000007e2 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/update.sample) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000007e0 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/pre-push.sample) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000007e8 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/pre-applypatch.sample) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000007eb (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/push-to-checkout.sample) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000007e5 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/post-update.sample) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000007e9 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/applypatch-msg.sample) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: Scrub error on inode 0x10000000023 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/HEAD) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: Scrub error on inode 0x1000000001c (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/refs) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000007dd (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/info/exclude) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000007ee (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/refs/tags) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000007ed (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/refs/heads) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: Scrub error on inode 0x10000000018 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/refs/remotes) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.032 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: Scrub error on inode 0x10000000013 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/objects/pack/pack-fe9e56eb4d11faa0800e85314d6db62e368e37e2.idx) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.032 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: Scrub error on inode 0x10000000012 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/objects/pack/pack-fe9e56eb4d11faa0800e85314d6db62e368e37e2.pack) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.032 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: Scrub error on inode 0x10000000024 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/refs/heads) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.032 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: Scrub error on inode 0x1000000001d (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/refs/remotes) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.032 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: Scrub error on inode 0x10000000022 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/refs/heads/master) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.032 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: Scrub error on inode 0x10000000019 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/refs/remotes/origin) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.032 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: Scrub error on inode 0x10000000025 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/refs/heads/master) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.032 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: Scrub error on inode 0x1000000001e (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/refs/remotes/origin) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.032 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: Scrub error on inode 0x1000000001a (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/refs/remotes/origin/HEAD) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.032 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: Scrub error on inode 0x1000000001f (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/refs/remotes/origin/HEAD) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.032 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: scrub summary: idle+waiting paths [/] 2023-12-01T01:19:00.033 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: scrub summary: active paths [/] 2023-12-01T01:19:00.033 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: scrub summary: active paths [/] 2023-12-01T01:19:00.033 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: scrub summary: idle+waiting paths [/] 2023-12-01T01:19:00.033 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:18:59 smithi137 ceph-mon[103312]: scrub summary: idle+waiting paths [/] 2023-12-01T01:19:00.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: scrub summary: active paths [/] 2023-12-01T01:19:00.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: scrub summary: active paths [/] 2023-12-01T01:19:00.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: scrub summary: idle+waiting paths [/] 2023-12-01T01:19:00.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: scrub summary: idle+waiting paths [/] 2023-12-01T01:19:00.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: scrub summary: active paths [/] 2023-12-01T01:19:00.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: scrub summary: active paths [/] 2023-12-01T01:19:00.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000001c2 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/027.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x10000000193 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/004) see mds.c log and `damage ls` output for details 2023-12-01T01:19:00.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x10000000201 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/056.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000001f0 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/050.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:00.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x1000000018e (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/001.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:00.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000007fc (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/063.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000001cc (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/032.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:00.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000001b1 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/019) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000001c1 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/027) see mds.c log and `damage ls` output for details 2023-12-01T01:19:00.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000001d7 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/038) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000001ce (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/033.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:00.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000001e1 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/043) see mds.c log and `damage ls` output for details 2023-12-01T01:19:00.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000001b7 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/022) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x10000000813 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/075) see mds.c log and `damage ls` output for details 2023-12-01T01:19:00.906 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x10000000811 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/074) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.907 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000001eb (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/048) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.907 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000001a6 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/013.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.907 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000001d3 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/036) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.907 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x1000000080e (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/072.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.907 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000001d9 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/039) see mds.c log and `damage ls` output for details 2023-12-01T01:19:00.907 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x10000000814 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/075.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:00.907 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x1000000080f (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/073) see mds.c log and `damage ls` output for details 2023-12-01T01:19:00.907 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000001c4 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/028.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:00.907 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000001d5 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/037) see mds.c log and `damage ls` output for details 2023-12-01T01:19:00.907 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000001ff (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/055.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.907 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000007f7 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/061) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.907 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000007f4 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/059.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:00.908 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x10000000803 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/067) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.908 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x10000000202 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/057) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.908 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x10000000801 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/066) see mds.c log and `damage ls` output for details 2023-12-01T01:19:00.908 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000001e7 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/046) see mds.c log and `damage ls` output for details 2023-12-01T01:19:00.908 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x1000000019d (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/009) see mds.c log and `damage ls` output for details 2023-12-01T01:19:00.908 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x10000000203 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/057.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.908 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x1000000080c (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/071.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:00.908 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000001ec (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/048.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:00.908 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x1000000019b (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/008) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.908 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x1000000019a (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/007.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.908 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000001e6 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/045.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:00.908 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000001a0 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/010.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:00.908 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000001bb (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/024) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.909 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x10000000815 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/076) see mds.c log and `damage ls` output for details 2023-12-01T01:19:00.909 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000007fb (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/063) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.909 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x10000000806 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/068.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.909 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000001c0 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/026.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:00.909 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000007fa (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/062.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.909 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000001fe (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/055) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.909 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000001be (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/025.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:00.909 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000001a5 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/013) see mds.c log and `damage ls` output for details 2023-12-01T01:19:00.909 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000007f6 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/060.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:00.909 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x10000000802 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/066.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.909 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000001a9 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/015) see mds.c log and `damage ls` output for details 2023-12-01T01:19:00.910 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000001ee (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/049.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.910 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000007fd (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/064) see mds.c log and `damage ls` output for details 2023-12-01T01:19:00.910 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000001f8 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/054.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.910 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000001bf (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/026) see mds.c log and `damage ls` output for details 2023-12-01T01:19:00.910 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000001f6 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/053.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.910 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x10000000200 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/056) see mds.c log and `damage ls` output for details 2023-12-01T01:19:00.910 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000001e5 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/045) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.910 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000001f4 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/052.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:00.910 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000007f8 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/061.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.910 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x1000000019f (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/010) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.910 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000001a1 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/011) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.911 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000001e0 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/042.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:00.911 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000001bd (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/025) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.911 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000007f5 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/060) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.911 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000001bc (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/024.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:00.911 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000001a8 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/014.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:00.911 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x1000000080d (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/072) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.911 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x10000000190 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/002.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.911 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000001ba (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/023.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:00.911 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000001f1 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/051) see mds.c log and `damage ls` output for details 2023-12-01T01:19:00.911 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x10000000805 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/068) see mds.c log and `damage ls` output for details 2023-12-01T01:19:00.912 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x1000000080a (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/070.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:00.912 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000001d4 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/036.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.912 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x1000000018d (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/001) see mds.c log and `damage ls` output for details 2023-12-01T01:19:00.912 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: scrub summary: idle+waiting paths [/] 2023-12-01T01:19:00.912 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000001f7 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/054) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.912 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: Scrub error on inode 0x100000007f1 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/058) see mds.i log and `damage ls` output for details 2023-12-01T01:19:00.912 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:00 smithi077 ceph-mon[98867]: scrub summary: idle+waiting paths [/] 2023-12-01T01:19:01.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: scrub summary: active paths [/] 2023-12-01T01:19:01.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: scrub summary: active paths [/] 2023-12-01T01:19:01.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: scrub summary: idle+waiting paths [/] 2023-12-01T01:19:01.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: scrub summary: idle+waiting paths [/] 2023-12-01T01:19:01.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: scrub summary: active paths [/] 2023-12-01T01:19:01.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: scrub summary: active paths [/] 2023-12-01T01:19:01.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000001c2 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/027.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x10000000193 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/004) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x10000000201 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/056.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000001f0 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/050.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x1000000018e (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/001.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000007fc (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/063.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000001cc (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/032.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000001b1 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/019) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000001c1 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/027) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000001d7 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/038) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000001ce (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/033.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000001e1 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/043) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000001b7 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/022) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x10000000813 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/075) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x10000000811 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/074) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000001eb (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/048) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000001a6 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/013.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.015 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000001d3 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/036) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.015 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x1000000080e (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/072.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.015 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000001d9 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/039) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.015 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x10000000814 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/075.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.015 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x1000000080f (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/073) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.015 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000001c4 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/028.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.015 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000001d5 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/037) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.015 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000001ff (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/055.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.015 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000007f7 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/061) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.015 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000007f4 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/059.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.015 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x10000000803 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/067) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.015 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x10000000202 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/057) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.015 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x10000000801 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/066) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.015 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000001e7 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/046) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.016 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x1000000019d (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/009) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.016 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x10000000203 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/057.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.016 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x1000000080c (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/071.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.016 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000001ec (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/048.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.016 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x1000000019b (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/008) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.016 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x1000000019a (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/007.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.016 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000001e6 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/045.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.016 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000001a0 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/010.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.016 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000001bb (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/024) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.016 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x10000000815 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/076) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.016 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000007fb (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/063) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.016 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x10000000806 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/068.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.016 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000001c0 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/026.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.016 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000007fa (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/062.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.017 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000001fe (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/055) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.017 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000001be (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/025.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.017 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000001a5 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/013) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.017 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000007f6 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/060.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.017 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x10000000802 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/066.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.017 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000001a9 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/015) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.017 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000001ee (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/049.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.017 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000007fd (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/064) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.017 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000001f8 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/054.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.017 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000001bf (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/026) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.017 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000001f6 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/053.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.017 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x10000000200 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/056) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.017 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000001e5 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/045) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.017 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000001f4 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/052.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.017 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000007f8 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/061.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.018 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x1000000019f (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/010) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.018 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000001a1 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/011) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.018 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000001e0 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/042.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.018 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000001bd (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/025) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.018 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000007f5 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/060) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.018 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000001bc (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/024.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.018 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000001a8 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/014.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.018 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x1000000080d (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/072) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.018 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x10000000190 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/002.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.018 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000001ba (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/023.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.018 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000001f1 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/051) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.018 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x10000000805 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/068) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.018 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x1000000080a (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/070.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.018 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000001d4 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/036.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.019 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x1000000018d (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/001) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.019 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: scrub summary: idle+waiting paths [/] 2023-12-01T01:19:01.019 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000001f7 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/054) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.019 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: Scrub error on inode 0x100000007f1 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/058) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.019 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:00 smithi203 ceph-mon[103581]: scrub summary: idle+waiting paths [/] 2023-12-01T01:19:01.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: scrub summary: active paths [/] 2023-12-01T01:19:01.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: scrub summary: active paths [/] 2023-12-01T01:19:01.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: scrub summary: idle+waiting paths [/] 2023-12-01T01:19:01.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: scrub summary: idle+waiting paths [/] 2023-12-01T01:19:01.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: scrub summary: active paths [/] 2023-12-01T01:19:01.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: scrub summary: active paths [/] 2023-12-01T01:19:01.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000001c2 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/027.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x10000000193 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/004) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x10000000201 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/056.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000001f0 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/050.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x1000000018e (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/001.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000007fc (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/063.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000001cc (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/032.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000001b1 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/019) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000001c1 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/027) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000001d7 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/038) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000001ce (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/033.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000001e1 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/043) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000001b7 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/022) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x10000000813 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/075) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x10000000811 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/074) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000001eb (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/048) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000001a6 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/013.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000001d3 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/036) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x1000000080e (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/072.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000001d9 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/039) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x10000000814 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/075.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x1000000080f (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/073) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000001c4 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/028.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000001d5 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/037) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.032 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000001ff (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/055.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.032 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000007f7 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/061) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.032 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000007f4 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/059.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.032 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x10000000803 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/067) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.032 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x10000000202 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/057) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.032 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x10000000801 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/066) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.032 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000001e7 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/046) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.032 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x1000000019d (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/009) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.032 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x10000000203 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/057.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.032 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x1000000080c (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/071.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.032 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000001ec (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/048.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.032 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x1000000019b (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/008) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.032 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x1000000019a (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/007.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.033 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000001e6 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/045.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.033 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000001a0 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/010.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.033 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000001bb (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/024) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.033 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x10000000815 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/076) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.033 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000007fb (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/063) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.033 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x10000000806 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/068.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.033 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000001c0 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/026.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.033 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000007fa (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/062.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.033 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000001fe (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/055) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.033 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000001be (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/025.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.033 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000001a5 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/013) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.033 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000007f6 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/060.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.033 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x10000000802 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/066.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.033 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000001a9 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/015) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.034 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000001ee (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/049.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.034 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000007fd (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/064) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.034 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000001f8 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/054.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.034 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000001bf (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/026) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.034 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000001f6 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/053.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.034 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x10000000200 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/056) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.034 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000001e5 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/045) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.034 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000001f4 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/052.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.034 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000007f8 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/061.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.034 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x1000000019f (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/010) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.034 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000001a1 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/011) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.034 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000001e0 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/042.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.034 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000001bd (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/025) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.034 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000007f5 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/060) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.034 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000001bc (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/024.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.035 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000001a8 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/014.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.035 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x1000000080d (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/072) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.035 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x10000000190 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/002.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.035 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000001ba (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/023.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.035 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000001f1 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/051) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.035 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x10000000805 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/068) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.035 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x1000000080a (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/070.out) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.035 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000001d4 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/036.out) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.035 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x1000000018d (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/001) see mds.c log and `damage ls` output for details 2023-12-01T01:19:01.035 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: scrub summary: idle+waiting paths [/] 2023-12-01T01:19:01.035 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000001f7 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/054) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.035 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: Scrub error on inode 0x100000007f1 (/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/058) see mds.i log and `damage ls` output for details 2023-12-01T01:19:01.035 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:00 smithi137 ceph-mon[103312]: scrub summary: idle+waiting paths [/] 2023-12-01T01:19:01.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:01 smithi077 ceph-mon[98867]: pgmap v332: 97 pgs: 97 active+clean; 24 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 374 op/s 2023-12-01T01:19:02.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:01 smithi203 ceph-mon[103581]: pgmap v332: 97 pgs: 97 active+clean; 24 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 374 op/s 2023-12-01T01:19:02.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:01 smithi137 ceph-mon[103312]: pgmap v332: 97 pgs: 97 active+clean; 24 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 374 op/s 2023-12-01T01:19:03.903 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:03 smithi077 ceph-mon[98867]: pgmap v333: 97 pgs: 97 active+clean; 35 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 721 op/s 2023-12-01T01:19:04.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:03 smithi203 ceph-mon[103581]: pgmap v333: 97 pgs: 97 active+clean; 35 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 721 op/s 2023-12-01T01:19:04.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:03 smithi137 ceph-mon[103312]: pgmap v333: 97 pgs: 97 active+clean; 35 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 721 op/s 2023-12-01T01:19:05.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:05 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mon-a[98843]: 2023-12-01T01:19:05.637+0000 7f988a60c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-12-01T01:19:05.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:05 smithi077 ceph-mon[98867]: pgmap v334: 97 pgs: 97 active+clean; 35 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 1.08k op/s 2023-12-01T01:19:06.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:05 smithi203 ceph-mon[103581]: pgmap v334: 97 pgs: 97 active+clean; 35 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 1.08k op/s 2023-12-01T01:19:06.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:05 smithi137 ceph-mon[103312]: pgmap v334: 97 pgs: 97 active+clean; 35 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 1.08k op/s 2023-12-01T01:19:06.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:06 smithi077 ceph-mon[98867]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-12-01T01:19:06.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:06 smithi077 ceph-mon[98867]: mds.? [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] up:active 2023-12-01T01:19:06.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:06 smithi077 ceph-mon[98867]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:19:07.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:06 smithi203 ceph-mon[103581]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-12-01T01:19:07.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:06 smithi203 ceph-mon[103581]: mds.? [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] up:active 2023-12-01T01:19:07.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:06 smithi203 ceph-mon[103581]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:19:07.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:06 smithi137 ceph-mon[103312]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-12-01T01:19:07.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:06 smithi137 ceph-mon[103312]: mds.? [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] up:active 2023-12-01T01:19:07.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:06 smithi137 ceph-mon[103312]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:19:07.504 INFO:tasks.workunit.client.0.smithi077.stderr:Note: switching to '12973fc04fd10d4af086901e10ffa8e48866b735'. 2023-12-01T01:19:07.504 INFO:tasks.workunit.client.0.smithi077.stderr: 2023-12-01T01:19:07.504 INFO:tasks.workunit.client.0.smithi077.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-12-01T01:19:07.504 INFO:tasks.workunit.client.0.smithi077.stderr:changes and commit them, and you can discard any commits you make in this 2023-12-01T01:19:07.504 INFO:tasks.workunit.client.0.smithi077.stderr:state without impacting any branches by switching back to a branch. 2023-12-01T01:19:07.504 INFO:tasks.workunit.client.0.smithi077.stderr: 2023-12-01T01:19:07.504 INFO:tasks.workunit.client.0.smithi077.stderr:If you want to create a new branch to retain commits you create, you may 2023-12-01T01:19:07.505 INFO:tasks.workunit.client.0.smithi077.stderr:do so (now or later) by using -c with the switch command. Example: 2023-12-01T01:19:07.505 INFO:tasks.workunit.client.0.smithi077.stderr: 2023-12-01T01:19:07.505 INFO:tasks.workunit.client.0.smithi077.stderr: git switch -c 2023-12-01T01:19:07.505 INFO:tasks.workunit.client.0.smithi077.stderr: 2023-12-01T01:19:07.505 INFO:tasks.workunit.client.0.smithi077.stderr:Or undo this operation with: 2023-12-01T01:19:07.505 INFO:tasks.workunit.client.0.smithi077.stderr: 2023-12-01T01:19:07.505 INFO:tasks.workunit.client.0.smithi077.stderr: git switch - 2023-12-01T01:19:07.505 INFO:tasks.workunit.client.0.smithi077.stderr: 2023-12-01T01:19:07.505 INFO:tasks.workunit.client.0.smithi077.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-12-01T01:19:07.505 INFO:tasks.workunit.client.0.smithi077.stderr: 2023-12-01T01:19:07.505 INFO:tasks.workunit.client.0.smithi077.stderr:HEAD is now at 12973fc0 btrfs/022: Correct seq number in golden output 2023-12-01T01:19:07.510 INFO:tasks.workunit.client.0.smithi077.stderr:+ make -j4 2023-12-01T01:19:07.519 INFO:tasks.workunit.client.0.smithi077.stdout:libtoolize -cfi 2023-12-01T01:19:07.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:07 smithi077 ceph-mon[98867]: pgmap v335: 97 pgs: 97 active+clean; 35 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 1.11k op/s 2023-12-01T01:19:07.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:07 smithi077 ceph-mon[98867]: mds.? [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] up:active 2023-12-01T01:19:07.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:07 smithi077 ceph-mon[98867]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:19:08.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:07 smithi203 ceph-mon[103581]: pgmap v335: 97 pgs: 97 active+clean; 35 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 1.11k op/s 2023-12-01T01:19:08.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:07 smithi203 ceph-mon[103581]: mds.? [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] up:active 2023-12-01T01:19:08.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:07 smithi203 ceph-mon[103581]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:19:08.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:07 smithi137 ceph-mon[103312]: pgmap v335: 97 pgs: 97 active+clean; 35 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 1.11k op/s 2023-12-01T01:19:08.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:07 smithi137 ceph-mon[103312]: mds.? [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] up:active 2023-12-01T01:19:08.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:07 smithi137 ceph-mon[103312]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:19:08.142 INFO:tasks.workunit.client.0.smithi077.stdout:libtoolize: putting auxiliary files in '.'. 2023-12-01T01:19:08.142 INFO:tasks.workunit.client.0.smithi077.stdout:libtoolize: copying file './config.guess' 2023-12-01T01:19:08.146 INFO:tasks.workunit.client.0.smithi077.stdout:libtoolize: copying file './config.sub' 2023-12-01T01:19:08.150 INFO:tasks.workunit.client.0.smithi077.stdout:libtoolize: copying file './install-sh' 2023-12-01T01:19:08.155 INFO:tasks.workunit.client.0.smithi077.stdout:libtoolize: copying file './ltmain.sh' 2023-12-01T01:19:08.176 INFO:tasks.workunit.client.0.smithi077.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2023-12-01T01:19:08.176 INFO:tasks.workunit.client.0.smithi077.stdout:libtoolize: copying file 'm4/libtool.m4' 2023-12-01T01:19:08.181 INFO:tasks.workunit.client.0.smithi077.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2023-12-01T01:19:08.187 INFO:tasks.workunit.client.0.smithi077.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2023-12-01T01:19:08.192 INFO:tasks.workunit.client.0.smithi077.stdout:libtoolize: copying file 'm4/ltversion.m4' 2023-12-01T01:19:08.197 INFO:tasks.workunit.client.0.smithi077.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2023-12-01T01:19:08.203 INFO:tasks.workunit.client.0.smithi077.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2023-12-01T01:19:08.204 INFO:tasks.workunit.client.0.smithi077.stdout:cp include/install-sh . 2023-12-01T01:19:08.242 INFO:tasks.workunit.client.0.smithi077.stdout:aclocal -I m4 2023-12-01T01:19:09.984 INFO:tasks.workunit.client.0.smithi077.stdout:autoheader 2023-12-01T01:19:10.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:09 smithi203 ceph-mon[103581]: pgmap v336: 97 pgs: 97 active+clean; 37 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 1.13k op/s 2023-12-01T01:19:10.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:09 smithi137 ceph-mon[103312]: pgmap v336: 97 pgs: 97 active+clean; 37 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 1.13k op/s 2023-12-01T01:19:10.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:09 smithi077 ceph-mon[98867]: pgmap v336: 97 pgs: 97 active+clean; 37 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 1.13k op/s 2023-12-01T01:19:10.388 INFO:tasks.workunit.client.0.smithi077.stdout:autoconf 2023-12-01T01:19:10.583 INFO:tasks.workunit.client.0.smithi077.stdout:./configure \ 2023-12-01T01:19:10.584 INFO:tasks.workunit.client.0.smithi077.stdout: --libexecdir=/usr/lib \ 2023-12-01T01:19:10.584 INFO:tasks.workunit.client.0.smithi077.stdout: --exec_prefix=/var/lib 2023-12-01T01:19:10.734 INFO:tasks.workunit.client.0.smithi077.stdout:checking build system type... x86_64-pc-linux-gnu 2023-12-01T01:19:10.735 INFO:tasks.workunit.client.0.smithi077.stdout:checking host system type... x86_64-pc-linux-gnu 2023-12-01T01:19:10.736 INFO:tasks.workunit.client.0.smithi077.stdout:checking how to print strings... printf 2023-12-01T01:19:10.736 INFO:tasks.workunit.client.0.smithi077.stdout:checking for gcc... gcc 2023-12-01T01:19:10.818 INFO:tasks.workunit.client.0.smithi077.stdout:checking whether the C compiler works... yes 2023-12-01T01:19:10.818 INFO:tasks.workunit.client.0.smithi077.stdout:checking for C compiler default output file name... a.out 2023-12-01T01:19:10.854 INFO:tasks.workunit.client.0.smithi077.stdout:checking for suffix of executables... 2023-12-01T01:19:10.895 INFO:tasks.workunit.client.0.smithi077.stdout:checking whether we are cross compiling... no 2023-12-01T01:19:10.933 INFO:tasks.workunit.client.0.smithi077.stdout:checking for suffix of object files... o 2023-12-01T01:19:10.956 INFO:tasks.workunit.client.0.smithi077.stdout:checking whether we are using the GNU C compiler... yes 2023-12-01T01:19:10.978 INFO:tasks.workunit.client.0.smithi077.stdout:checking whether gcc accepts -g... yes 2023-12-01T01:19:11.015 INFO:tasks.workunit.client.0.smithi077.stdout:checking for gcc option to accept ISO C89... none needed 2023-12-01T01:19:11.019 INFO:tasks.workunit.client.0.smithi077.stdout:checking for a sed that does not truncate output... /usr/bin/sed 2023-12-01T01:19:11.023 INFO:tasks.workunit.client.0.smithi077.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2023-12-01T01:19:11.024 INFO:tasks.workunit.client.0.smithi077.stdout:checking for egrep... /usr/bin/grep -E 2023-12-01T01:19:11.025 INFO:tasks.workunit.client.0.smithi077.stdout:checking for fgrep... /usr/bin/grep -F 2023-12-01T01:19:11.028 INFO:tasks.workunit.client.0.smithi077.stdout:checking for ld used by gcc... /usr/bin/ld 2023-12-01T01:19:11.030 INFO:tasks.workunit.client.0.smithi077.stdout:checking if the linker (/usr/bin/ld) is GNU ld... yes 2023-12-01T01:19:11.038 INFO:tasks.workunit.client.0.smithi077.stdout:checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B 2023-12-01T01:19:11.077 INFO:tasks.workunit.client.0.smithi077.stdout:checking the name lister (/usr/bin/nm -B) interface... BSD nm 2023-12-01T01:19:11.077 INFO:tasks.workunit.client.0.smithi077.stdout:checking whether ln -s works... yes 2023-12-01T01:19:11.091 INFO:tasks.workunit.client.0.smithi077.stdout:checking the maximum length of command line arguments... 1572864 2023-12-01T01:19:11.093 INFO:tasks.workunit.client.0.smithi077.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2023-12-01T01:19:11.093 INFO:tasks.workunit.client.0.smithi077.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2023-12-01T01:19:11.093 INFO:tasks.workunit.client.0.smithi077.stdout:checking for /usr/bin/ld option to reload object files... -r 2023-12-01T01:19:11.094 INFO:tasks.workunit.client.0.smithi077.stdout:checking for objdump... objdump 2023-12-01T01:19:11.094 INFO:tasks.workunit.client.0.smithi077.stdout:checking how to recognize dependent libraries... pass_all 2023-12-01T01:19:11.094 INFO:tasks.workunit.client.0.smithi077.stdout:checking for dlltool... no 2023-12-01T01:19:11.094 INFO:tasks.workunit.client.0.smithi077.stdout:checking how to associate runtime and link libraries... printf %s\n 2023-12-01T01:19:11.095 INFO:tasks.workunit.client.0.smithi077.stdout:checking for ar... ar 2023-12-01T01:19:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.100+0000 7f5597342700 1 -- 172.21.15.77:0/832294463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5590106ac0 msgr2=0x7f5590108ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.100+0000 7f5597342700 1 --2- 172.21.15.77:0/832294463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5590106ac0 0x7f5590108ef0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f558800b0a0 tx=0x7f558800b3b0 comp rx=0 tx=0).stop 2023-12-01T01:19:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.100+0000 7f5597342700 1 -- 172.21.15.77:0/832294463 shutdown_connections 2023-12-01T01:19:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.100+0000 7f5597342700 1 --2- 172.21.15.77:0/832294463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5590106ac0 0x7f5590108ef0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.100+0000 7f5597342700 1 --2- 172.21.15.77:0/832294463 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5590101860 0x7f5590103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.100+0000 7f5597342700 1 --2- 172.21.15.77:0/832294463 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5590104190 0x7f5590106580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.100+0000 7f5597342700 1 -- 172.21.15.77:0/832294463 >> 172.21.15.77:0/832294463 conn(0x7f55900fb230 msgr2=0x7f55900fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:19:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.101+0000 7f5597342700 1 -- 172.21.15.77:0/832294463 shutdown_connections 2023-12-01T01:19:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.101+0000 7f5597342700 1 -- 172.21.15.77:0/832294463 wait complete. 2023-12-01T01:19:11.104 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.101+0000 7f5597342700 1 Processor -- start 2023-12-01T01:19:11.104 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.101+0000 7f5597342700 1 -- start start 2023-12-01T01:19:11.104 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.101+0000 7f5597342700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5590101860 0x7f5590196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:11.104 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.101+0000 7f5597342700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5590104190 0x7f5590197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:11.104 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.101+0000 7f5597342700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5590106ac0 0x7f5590197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:11.104 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.101+0000 7f5597342700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5590198650 con 0x7f5590101860 2023-12-01T01:19:11.104 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.101+0000 7f5597342700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55901987c0 con 0x7f5590106ac0 2023-12-01T01:19:11.105 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.101+0000 7f5597342700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5590198980 con 0x7f5590104190 2023-12-01T01:19:11.105 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.102+0000 7f55950de700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5590101860 0x7f5590196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:11.105 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.102+0000 7f55948dd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5590104190 0x7f5590197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:11.105 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.102+0000 7f55950de700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5590101860 0x7f5590196c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60276/0 (socket says 172.21.15.77:60276) 2023-12-01T01:19:11.105 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.102+0000 7f55948dd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5590104190 0x7f5590197170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:46466/0 (socket says 172.21.15.77:46466) 2023-12-01T01:19:11.105 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.102+0000 7f55950de700 1 -- 172.21.15.77:0/2835116384 learned_addr learned my addr 172.21.15.77:0/2835116384 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:19:11.105 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.102+0000 7f55958df700 1 --2- 172.21.15.77:0/2835116384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5590106ac0 0x7f5590197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:11.105 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.102+0000 7f55948dd700 1 -- 172.21.15.77:0/2835116384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5590106ac0 msgr2=0x7f5590197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:11.105 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.102+0000 7f55948dd700 1 --2- 172.21.15.77:0/2835116384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5590106ac0 0x7f5590197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:11.105 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.102+0000 7f55948dd700 1 -- 172.21.15.77:0/2835116384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5590101860 msgr2=0x7f5590196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:11.105 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.102+0000 7f55948dd700 1 --2- 172.21.15.77:0/2835116384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5590101860 0x7f5590196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:11.105 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.102+0000 7f55948dd700 1 -- 172.21.15.77:0/2835116384 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5588009d00 con 0x7f5590104190 2023-12-01T01:19:11.106 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.102+0000 7f55958df700 1 --2- 172.21.15.77:0/2835116384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5590106ac0 0x7f5590197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:19:11.106 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.102+0000 7f55948dd700 1 --2- 172.21.15.77:0/2835116384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5590104190 0x7f5590197170 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f558000e9f0 tx=0x7f558000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:19:11.106 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.102+0000 7f55867fc700 1 -- 172.21.15.77:0/2835116384 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558000c9c0 con 0x7f5590104190 2023-12-01T01:19:11.106 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.102+0000 7f5597342700 1 -- 172.21.15.77:0/2835116384 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5590198c60 con 0x7f5590104190 2023-12-01T01:19:11.106 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.102+0000 7f55867fc700 1 -- 172.21.15.77:0/2835116384 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5580010430 con 0x7f5590104190 2023-12-01T01:19:11.106 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.102+0000 7f5597342700 1 -- 172.21.15.77:0/2835116384 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55900ff830 con 0x7f5590104190 2023-12-01T01:19:11.106 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.102+0000 7f55867fc700 1 -- 172.21.15.77:0/2835116384 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5580018680 con 0x7f5590104190 2023-12-01T01:19:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.105+0000 7f55867fc700 1 -- 172.21.15.77:0/2835116384 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f5580010860 con 0x7f5590104190 2023-12-01T01:19:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.105+0000 7f55867fc700 1 --2- 172.21.15.77:0/2835116384 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f557c0a0520 0x7f557c0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.105+0000 7f55867fc700 1 -- 172.21.15.77:0/2835116384 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f5580014070 con 0x7f5590104190 2023-12-01T01:19:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.105+0000 7f55950de700 1 --2- 172.21.15.77:0/2835116384 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f557c0a0520 0x7f557c0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.106+0000 7f55950de700 1 --2- 172.21.15.77:0/2835116384 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f557c0a0520 0x7f557c0a29e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f558c009d20 tx=0x7f558c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:19:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.106+0000 7f55a34d2b80 1 Processor -- start 2023-12-01T01:19:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.107+0000 7f55a34d2b80 1 -- start start 2023-12-01T01:19:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.107+0000 7f55a34d2b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d6ba7680 0x5582d6ba7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.107+0000 7f55a34d2b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582d6ba7fe0 0x5582d6ba88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.107+0000 7f55a34d2b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582d6ba8ed0 0x5582d6ba9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.107+0000 7f55a34d2b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582d6aff9b0 con 0x5582d6ba7fe0 2023-12-01T01:19:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.107+0000 7f55a34d2b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582d6afd370 con 0x5582d6ba8ed0 2023-12-01T01:19:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.107+0000 7f55a34d2b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582d6af8c60 con 0x5582d6ba7680 2023-12-01T01:19:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.107+0000 7f55948dd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582d6ba7fe0 0x5582d6ba88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.107+0000 7f55950de700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d6ba7680 0x5582d6ba7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.107+0000 7f55948dd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582d6ba7fe0 0x5582d6ba88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60286/0 (socket says 172.21.15.77:60286) 2023-12-01T01:19:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.107+0000 7f55958df700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582d6ba8ed0 0x5582d6ba9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.107+0000 7f55948dd700 1 -- 172.21.15.77:0/1332578664 learned_addr learned my addr 172.21.15.77:0/1332578664 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:19:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.107+0000 7f55950de700 1 -- 172.21.15.77:0/1332578664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582d6ba8ed0 msgr2=0x5582d6ba9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.107+0000 7f55950de700 1 --2- 172.21.15.77:0/1332578664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582d6ba8ed0 0x5582d6ba9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.107+0000 7f55950de700 1 -- 172.21.15.77:0/1332578664 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582d6ba7fe0 msgr2=0x5582d6ba88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:11.116 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.107+0000 7f55950de700 1 --2- 172.21.15.77:0/1332578664 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582d6ba7fe0 0x5582d6ba88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:11.116 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.107+0000 7f55950de700 1 -- 172.21.15.77:0/1332578664 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582d6926d80 con 0x5582d6ba7680 2023-12-01T01:19:11.116 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.107+0000 7f55958df700 1 --2- 172.21.15.77:0/1332578664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582d6ba8ed0 0x5582d6ba9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:19:11.116 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.107+0000 7f55948dd700 1 --2- 172.21.15.77:0/1332578664 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582d6ba7fe0 0x5582d6ba88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:19:11.116 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.107+0000 7f55950de700 1 --2- 172.21.15.77:0/1332578664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d6ba7680 0x5582d6ba7aa0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f558c011ab0 tx=0x7f558c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cb230a37a682609 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:19:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.107+0000 7f557b7fe700 1 -- 172.21.15.77:0/1332578664 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558c00edf0 con 0x5582d6ba7680 2023-12-01T01:19:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.108+0000 7f557b7fe700 1 -- 172.21.15.77:0/1332578664 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f558c021070 con 0x5582d6ba7680 2023-12-01T01:19:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.108+0000 7f557b7fe700 1 -- 172.21.15.77:0/1332578664 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558c01d830 con 0x5582d6ba7680 2023-12-01T01:19:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.108+0000 7f55a34d2b80 1 -- 172.21.15.77:0/1332578664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d6ba7680 msgr2=0x5582d6ba7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:11.117 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.108+0000 7f55a34d2b80 1 --2- 172.21.15.77:0/1332578664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d6ba7680 0x5582d6ba7aa0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f558c011ab0 tx=0x7f558c014000 comp rx=0 tx=0).stop 2023-12-01T01:19:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.108+0000 7f55a34d2b80 1 -- 172.21.15.77:0/1332578664 shutdown_connections 2023-12-01T01:19:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.108+0000 7f55a34d2b80 1 --2- 172.21.15.77:0/1332578664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582d6ba8ed0 0x5582d6ba9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.108+0000 7f55a34d2b80 1 --2- 172.21.15.77:0/1332578664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d6ba7680 0x5582d6ba7aa0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.108+0000 7f55a34d2b80 1 --2- 172.21.15.77:0/1332578664 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582d6ba7fe0 0x5582d6ba88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.108+0000 7f55a34d2b80 1 -- 172.21.15.77:0/1332578664 >> 172.21.15.77:0/1332578664 conn(0x5582d6b07070 msgr2=0x5582d6ba4040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:19:11.118 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.108+0000 7f55a34d2b80 1 -- 172.21.15.77:0/1332578664 shutdown_connections 2023-12-01T01:19:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.108+0000 7f55a34d2b80 1 -- 172.21.15.77:0/1332578664 wait complete. 2023-12-01T01:19:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.109+0000 7f55a34d2b80 1 -- start start 2023-12-01T01:19:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.109+0000 7f55a34d2b80 1 Processor -- start 2023-12-01T01:19:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.109+0000 7f55a34d2b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582d6ba7680 0x5582d6bb14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.109+0000 7f55a34d2b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582d6ba7fe0 0x5582d6bb1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.109+0000 7f55a34d2b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d6ba8ed0 0x5582d6bb8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.109+0000 7f55a34d2b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582d6a954b0 con 0x5582d6ba7fe0 2023-12-01T01:19:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.109+0000 7f55a34d2b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582d6a95150 con 0x5582d6ba7680 2023-12-01T01:19:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.109+0000 7f55a34d2b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582d697c8c0 con 0x5582d6ba8ed0 2023-12-01T01:19:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.109+0000 7f55a34d2b80 10 client.0 _ll_register_callbacks cb 0x5582d69e2a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:19:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.109+0000 7f55950de700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582d6ba7680 0x5582d6bb14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:11.119 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.109+0000 7f55950de700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582d6ba7680 0x5582d6bb14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:39212/0 (socket says 172.21.15.77:39212) 2023-12-01T01:19:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.109+0000 7f55950de700 1 -- 172.21.15.77:0/2589637015 learned_addr learned my addr 172.21.15.77:0/2589637015 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:19:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.109+0000 7f55948dd700 1 --2- 172.21.15.77:0/2589637015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582d6ba7fe0 0x5582d6bb1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.109+0000 7f55958df700 1 --2- 172.21.15.77:0/2589637015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d6ba8ed0 0x5582d6bb8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.110+0000 7f55958df700 1 -- 172.21.15.77:0/2589637015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582d6ba7680 msgr2=0x5582d6bb14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.110+0000 7f55958df700 1 --2- 172.21.15.77:0/2589637015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582d6ba7680 0x5582d6bb14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.110+0000 7f55958df700 1 -- 172.21.15.77:0/2589637015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582d6ba7fe0 msgr2=0x5582d6bb1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.110+0000 7f55958df700 1 --2- 172.21.15.77:0/2589637015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582d6ba7fe0 0x5582d6bb1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:11.120 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.110+0000 7f55958df700 1 -- 172.21.15.77:0/2589637015 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55900ff830 con 0x5582d6ba8ed0 2023-12-01T01:19:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.110+0000 7f55950de700 1 --2- 172.21.15.77:0/2589637015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582d6ba7680 0x5582d6bb14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:19:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.110+0000 7f55948dd700 1 --2- 172.21.15.77:0/2589637015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582d6ba7fe0 0x5582d6bb1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:19:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.110+0000 7f55958df700 1 --2- 172.21.15.77:0/2589637015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d6ba8ed0 0x5582d6bb8a70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5588009fd0 tx=0x7f558800bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86103e1a107b9693 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:19:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.110+0000 7f5578ff9700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2023-12-01T01:19:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.110+0000 7f5578ff9700 1 -- 172.21.15.77:0/2589637015 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558800e040 con 0x5582d6ba8ed0 2023-12-01T01:19:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.110+0000 7f5578ff9700 1 -- 172.21.15.77:0/2589637015 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f5588003a00 con 0x5582d6ba8ed0 2023-12-01T01:19:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.110+0000 7f55a34d2b80 1 -- 172.21.15.77:0/2589637015 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5582d6b23cb0 con 0x5582d6ba8ed0 2023-12-01T01:19:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.110+0000 7f5578ff9700 1 -- 172.21.15.77:0/2589637015 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5588003bc0 con 0x5582d6ba8ed0 2023-12-01T01:19:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.110+0000 7f55a34d2b80 10 client.25468 fetch_fsmap learned FSMap version 31 2023-12-01T01:19:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.110+0000 7f55a34d2b80 1 -- 172.21.15.77:0/2589637015 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f5590109ae0 con 0x5582d6ba8ed0 2023-12-01T01:19:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.111+0000 7f5578ff9700 1 -- 172.21.15.77:0/2589637015 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5582d6b23cb0 con 0x5582d6ba8ed0 2023-12-01T01:19:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.111+0000 7f5578ff9700 1 -- 172.21.15.77:0/2589637015 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f558800b0d0 con 0x5582d6ba8ed0 2023-12-01T01:19:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.111+0000 7f55a34d2b80 10 client.25468 fetch_fsmap finished waiting for FSMap version 31 2023-12-01T01:19:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.111+0000 7f55a34d2b80 10 client.25468 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-01T01:19:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.111+0000 7f55a34d2b80 20 client.25468 populate_metadata read hostname 'smithi077' 2023-12-01T01:19:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.111+0000 7f55a34d2b80 1 --2- 172.21.15.77:0/2589637015 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5582d6bbd370 0x5582d6bbf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.111+0000 7f55a34d2b80 4 client.25468 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:19:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.111+0000 7f55a34d2b80 1 -- 172.21.15.77:0/2589637015 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5582d6b76db0 con 0x5582d6bbd370 2023-12-01T01:19:11.122 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.111+0000 7f55950de700 1 --2- 172.21.15.77:0/2589637015 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5582d6bbd370 0x5582d6bbf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.111+0000 7f5578ff9700 10 client.25468 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:19:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.111+0000 7f55950de700 1 --2- 172.21.15.77:0/2589637015 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5582d6bbd370 0x5582d6bbf790 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2dd9744187950e9 server_cookie=77aed38e0ecc940e in_seq=0 out_seq=0 2023-12-01T01:19:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.112+0000 7f5578ff9700 1 -- 172.21.15.77:0/2589637015 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5582d6b76db0 con 0x5582d6bbd370 2023-12-01T01:19:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.112+0000 7f5578ff9700 10 client.25468 handle_command_reply: tid=0 2023-12-01T01:19:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.112+0000 7f55a34d2b80 1 client.25468 shutdown 2023-12-01T01:19:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.112+0000 7f55a34d2b80 1 -- 172.21.15.77:0/2589637015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d6ba8ed0 msgr2=0x5582d6bb8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.112+0000 7f55a34d2b80 1 --2- 172.21.15.77:0/2589637015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d6ba8ed0 0x5582d6bb8a70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5588009fd0 tx=0x7f558800bab0 comp rx=0 tx=0).stop 2023-12-01T01:19:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.112+0000 7f55a34d2b80 1 -- 172.21.15.77:0/2589637015 shutdown_connections 2023-12-01T01:19:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.112+0000 7f55a34d2b80 1 --2- 172.21.15.77:0/2589637015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d6ba8ed0 0x5582d6bb8a70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.112+0000 7f55a34d2b80 1 --2- 172.21.15.77:0/2589637015 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5582d6bbd370 0x5582d6bbf790 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:11.123 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.112+0000 7f55a34d2b80 1 --2- 172.21.15.77:0/2589637015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582d6ba7680 0x5582d6bb14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.112+0000 7f5578ff9700 0 client.25468 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:19:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.112+0000 7f55a34d2b80 1 --2- 172.21.15.77:0/2589637015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582d6ba7fe0 0x5582d6bb1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.112+0000 7f55a34d2b80 1 -- 172.21.15.77:0/2589637015 >> 172.21.15.77:0/2589637015 conn(0x5582d6b07070 msgr2=0x5582d6ba23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:19:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.112+0000 7f55a34d2b80 1 -- 172.21.15.77:0/2589637015 shutdown_connections 2023-12-01T01:19:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.113+0000 7f55a34d2b80 1 -- 172.21.15.77:0/2589637015 wait complete. 2023-12-01T01:19:11.124 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.113+0000 7f55a34d2b80 20 client.25468 trim_cache size 0 max 16384 2023-12-01T01:19:11.131 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.128+0000 7f55a34d2b80 1 Processor -- start 2023-12-01T01:19:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.128+0000 7f55a34d2b80 1 -- start start 2023-12-01T01:19:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.128+0000 7f55a34d2b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582d6ba7680 0x5582d6ba92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.128+0000 7f55a34d2b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d6ba7fe0 0x5582d6bb8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.129+0000 7f55a34d2b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582d6ba8ed0 0x5582d6ba3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.129+0000 7f55a34d2b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582d6b9d500 con 0x5582d6ba7680 2023-12-01T01:19:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.129+0000 7f55a34d2b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582d6b9add0 con 0x5582d6ba8ed0 2023-12-01T01:19:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.129+0000 7f55a34d2b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582d6bb1060 con 0x5582d6ba7fe0 2023-12-01T01:19:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.129+0000 7f55950de700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582d6ba7680 0x5582d6ba92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.129+0000 7f55948dd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d6ba7fe0 0x5582d6bb8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.129+0000 7f55950de700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582d6ba7680 0x5582d6ba92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60298/0 (socket says 172.21.15.77:60298) 2023-12-01T01:19:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.129+0000 7f55950de700 1 -- 172.21.15.77:0/1713738864 learned_addr learned my addr 172.21.15.77:0/1713738864 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:19:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.129+0000 7f55958df700 1 --2- 172.21.15.77:0/1713738864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582d6ba8ed0 0x5582d6ba3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:11.132 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.129+0000 7f55950de700 1 -- 172.21.15.77:0/1713738864 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d6ba7fe0 msgr2=0x5582d6bb8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.129+0000 7f55950de700 1 --2- 172.21.15.77:0/1713738864 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d6ba7fe0 0x5582d6bb8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.129+0000 7f55950de700 1 -- 172.21.15.77:0/1713738864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582d6ba8ed0 msgr2=0x5582d6ba3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.129+0000 7f55950de700 1 --2- 172.21.15.77:0/1713738864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582d6ba8ed0 0x5582d6ba3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.129+0000 7f55950de700 1 -- 172.21.15.77:0/1713738864 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f558c017070 con 0x5582d6ba7680 2023-12-01T01:19:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.129+0000 7f55950de700 1 --2- 172.21.15.77:0/1713738864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582d6ba7680 0x5582d6ba92b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f558c011ab0 tx=0x7f558c00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ccca576abd52f99 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:19:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.129+0000 7f557bfff700 1 -- 172.21.15.77:0/1713738864 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558c00ec60 con 0x5582d6ba7680 2023-12-01T01:19:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.129+0000 7f557bfff700 1 -- 172.21.15.77:0/1713738864 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f558c033dd0 con 0x5582d6ba7680 2023-12-01T01:19:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.129+0000 7f557bfff700 1 -- 172.21.15.77:0/1713738864 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558c005630 con 0x5582d6ba7680 2023-12-01T01:19:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.130+0000 7f55a34d2b80 1 -- 172.21.15.77:0/1713738864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582d6ba7680 msgr2=0x5582d6ba92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.130+0000 7f55a34d2b80 1 --2- 172.21.15.77:0/1713738864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582d6ba7680 0x5582d6ba92b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f558c011ab0 tx=0x7f558c00ebf0 comp rx=0 tx=0).stop 2023-12-01T01:19:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.130+0000 7f55a34d2b80 1 -- 172.21.15.77:0/1713738864 shutdown_connections 2023-12-01T01:19:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.130+0000 7f55a34d2b80 1 --2- 172.21.15.77:0/1713738864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582d6ba8ed0 0x5582d6ba3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.130+0000 7f55a34d2b80 1 --2- 172.21.15.77:0/1713738864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582d6ba7680 0x5582d6ba92b0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.130+0000 7f55a34d2b80 1 --2- 172.21.15.77:0/1713738864 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d6ba7fe0 0x5582d6bb8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.130+0000 7f55a34d2b80 1 -- 172.21.15.77:0/1713738864 >> 172.21.15.77:0/1713738864 conn(0x5582d6b07070 msgr2=0x5582d6bb01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:19:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.130+0000 7f55a34d2b80 1 -- 172.21.15.77:0/1713738864 shutdown_connections 2023-12-01T01:19:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.130+0000 7f55a34d2b80 1 -- 172.21.15.77:0/1713738864 wait complete. 2023-12-01T01:19:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.130+0000 7f55a34d2b80 1 -- start start 2023-12-01T01:19:11.134 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.131+0000 7f55a34d2b80 1 Processor -- start 2023-12-01T01:19:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.131+0000 7f55a34d2b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582d6ba7680 0x5582d6bb6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.131+0000 7f55a34d2b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d6ba7fe0 0x5582d6bb6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.131+0000 7f55a34d2b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582d6ba8ed0 0x5582d6bb7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.131+0000 7f55a34d2b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582d6bb80a0 con 0x5582d6ba7680 2023-12-01T01:19:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.131+0000 7f55a34d2b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582d6bb8210 con 0x5582d6ba8ed0 2023-12-01T01:19:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.131+0000 7f55a34d2b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582d6bb83d0 con 0x5582d6ba7fe0 2023-12-01T01:19:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.131+0000 7f55a34d2b80 10 client.0 _ll_register_callbacks cb 0x7f55880074e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:19:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.131+0000 7f55950de700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582d6ba7680 0x5582d6bb6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.131+0000 7f55950de700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582d6ba7680 0x5582d6bb6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60304/0 (socket says 172.21.15.77:60304) 2023-12-01T01:19:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.131+0000 7f55950de700 1 -- 172.21.15.77:0/2484188470 learned_addr learned my addr 172.21.15.77:0/2484188470 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:19:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.131+0000 7f55958df700 1 --2- 172.21.15.77:0/2484188470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582d6ba8ed0 0x5582d6bb7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.131+0000 7f55948dd700 1 --2- 172.21.15.77:0/2484188470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d6ba7fe0 0x5582d6bb6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.131+0000 7f55950de700 1 -- 172.21.15.77:0/2484188470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d6ba7fe0 msgr2=0x5582d6bb6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.131+0000 7f55950de700 1 --2- 172.21.15.77:0/2484188470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d6ba7fe0 0x5582d6bb6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.131+0000 7f55950de700 1 -- 172.21.15.77:0/2484188470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582d6ba8ed0 msgr2=0x5582d6bb7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.131+0000 7f55950de700 1 --2- 172.21.15.77:0/2484188470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582d6ba8ed0 0x5582d6bb7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.131+0000 7f55950de700 1 -- 172.21.15.77:0/2484188470 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f558c02a040 con 0x5582d6ba7680 2023-12-01T01:19:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.131+0000 7f55950de700 1 --2- 172.21.15.77:0/2484188470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582d6ba7680 0x5582d6bb6610 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f558c00e480 tx=0x7f558c033170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9f69cd3338cffb4 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:19:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.131+0000 7f5578ff9700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:19:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.132+0000 7f5578ff9700 1 -- 172.21.15.77:0/2484188470 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558c033450 con 0x5582d6ba7680 2023-12-01T01:19:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.132+0000 7f55a34d2b80 1 -- 172.21.15.77:0/2484188470 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f558c021070 con 0x5582d6ba7680 2023-12-01T01:19:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.132+0000 7f5578ff9700 1 -- 172.21.15.77:0/2484188470 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f558c033910 con 0x5582d6ba7680 2023-12-01T01:19:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.132+0000 7f5578ff9700 1 -- 172.21.15.77:0/2484188470 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f558c033ad0 con 0x5582d6ba7680 2023-12-01T01:19:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.132+0000 7f55a34d2b80 10 client.15885 fetch_fsmap learned FSMap version 31 2023-12-01T01:19:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.132+0000 7f55a34d2b80 1 -- 172.21.15.77:0/2484188470 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5582d690fc00 con 0x5582d6ba7680 2023-12-01T01:19:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.132+0000 7f5578ff9700 1 -- 172.21.15.77:0/2484188470 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f558c021070 con 0x5582d6ba7680 2023-12-01T01:19:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.132+0000 7f5578ff9700 1 -- 172.21.15.77:0/2484188470 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f558c03a070 con 0x5582d6ba7680 2023-12-01T01:19:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.132+0000 7f55a34d2b80 10 client.15885 fetch_fsmap finished waiting for FSMap version 31 2023-12-01T01:19:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.132+0000 7f55a34d2b80 10 client.15885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-01T01:19:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.132+0000 7f55a34d2b80 20 client.15885 populate_metadata read hostname 'smithi077' 2023-12-01T01:19:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.132+0000 7f55a34d2b80 1 --2- 172.21.15.77:0/2484188470 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5582d6c0f820 0x5582d6c11c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.132+0000 7f55a34d2b80 4 client.15885 mds_command: new command op to 24500 tid=0[{"prefix": "flush journal"}] 2023-12-01T01:19:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.132+0000 7f55a34d2b80 1 -- 172.21.15.77:0/2484188470 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5582d6af1ce0 con 0x5582d6c0f820 2023-12-01T01:19:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.132+0000 7f55948dd700 1 --2- 172.21.15.77:0/2484188470 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5582d6c0f820 0x5582d6c11c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.133+0000 7f5578ff9700 10 client.15885 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:19:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.133+0000 7f55948dd700 1 --2- 172.21.15.77:0/2484188470 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5582d6c0f820 0x5582d6c11c40 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1fd669dbec59980 server_cookie=a06f97c3faed9ac4 in_seq=0 out_seq=0 2023-12-01T01:19:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.276+0000 7f5578ff9700 1 -- 172.21.15.77:0/2484188470 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5582d6af1ce0 con 0x5582d6c0f820 2023-12-01T01:19:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.276+0000 7f5578ff9700 10 client.15885 handle_command_reply: tid=0 2023-12-01T01:19:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.276+0000 7f55a34d2b80 1 client.15885 shutdown 2023-12-01T01:19:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.277+0000 7f55a34d2b80 1 -- 172.21.15.77:0/2484188470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582d6ba7680 msgr2=0x5582d6bb6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.277+0000 7f55a34d2b80 1 --2- 172.21.15.77:0/2484188470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582d6ba7680 0x5582d6bb6610 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f558c00e480 tx=0x7f558c033170 comp rx=0 tx=0).stop 2023-12-01T01:19:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.277+0000 7f55a34d2b80 1 -- 172.21.15.77:0/2484188470 shutdown_connections 2023-12-01T01:19:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.277+0000 7f55a34d2b80 1 --2- 172.21.15.77:0/2484188470 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5582d6c0f820 0x5582d6c11c40 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.277+0000 7f55a34d2b80 1 --2- 172.21.15.77:0/2484188470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582d6ba8ed0 0x5582d6bb7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.277+0000 7f55a34d2b80 1 --2- 172.21.15.77:0/2484188470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582d6ba7680 0x5582d6bb6610 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.278+0000 7f55a34d2b80 1 --2- 172.21.15.77:0/2484188470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582d6ba7fe0 0x5582d6bb6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.278+0000 7f55a34d2b80 1 -- 172.21.15.77:0/2484188470 >> 172.21.15.77:0/2484188470 conn(0x5582d6b07070 msgr2=0x5582d6ba3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:19:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.278+0000 7f5578ff9700 0 client.15885 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:19:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.278+0000 7f55a34d2b80 1 -- 172.21.15.77:0/2484188470 shutdown_connections 2023-12-01T01:19:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.278+0000 7f55a34d2b80 1 -- 172.21.15.77:0/2484188470 wait complete. 2023-12-01T01:19:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.278+0000 7f55a34d2b80 20 client.15885 trim_cache size 0 max 16384 2023-12-01T01:19:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout:{ 2023-12-01T01:19:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout: "message": "", 2023-12-01T01:19:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout: "return_code": 0 2023-12-01T01:19:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout:} 2023-12-01T01:19:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.278+0000 7f5597342700 1 -- 172.21.15.77:0/2835116384 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f557c0a0520 msgr2=0x7f557c0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.278+0000 7f5597342700 1 --2- 172.21.15.77:0/2835116384 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f557c0a0520 0x7f557c0a29e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f558c009d20 tx=0x7f558c009260 comp rx=0 tx=0).stop 2023-12-01T01:19:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.278+0000 7f5597342700 1 -- 172.21.15.77:0/2835116384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5590104190 msgr2=0x7f5590197170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.278+0000 7f5597342700 1 --2- 172.21.15.77:0/2835116384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5590104190 0x7f5590197170 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f558000e9f0 tx=0x7f558000ed00 comp rx=0 tx=0).stop 2023-12-01T01:19:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.278+0000 7f5597342700 1 -- 172.21.15.77:0/2835116384 shutdown_connections 2023-12-01T01:19:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.278+0000 7f5597342700 1 --2- 172.21.15.77:0/2835116384 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f557c0a0520 0x7f557c0a29e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.278+0000 7f5597342700 1 --2- 172.21.15.77:0/2835116384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5590106ac0 0x7f5590197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.278+0000 7f5597342700 1 --2- 172.21.15.77:0/2835116384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5590101860 0x7f5590196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.278+0000 7f5597342700 1 --2- 172.21.15.77:0/2835116384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5590104190 0x7f5590197170 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.278+0000 7f5597342700 1 -- 172.21.15.77:0/2835116384 >> 172.21.15.77:0/2835116384 conn(0x7f55900fb230 msgr2=0x7f55901024f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:19:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.279+0000 7f5597342700 1 -- 172.21.15.77:0/2835116384 shutdown_connections 2023-12-01T01:19:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:11.279+0000 7f5597342700 1 -- 172.21.15.77:0/2835116384 wait complete. 2023-12-01T01:19:11.291 INFO:tasks.workunit.client.0.smithi077.stdout:checking for archiver @FILE support... @ 2023-12-01T01:19:11.291 INFO:tasks.workunit.client.0.smithi077.stdout:checking for strip... strip 2023-12-01T01:19:11.292 INFO:tasks.workunit.client.0.smithi077.stdout:checking for ranlib... ranlib 2023-12-01T01:19:11.292 INFO:tasks.workunit.client.0.smithi077.stdout:checking for gawk... gawk 2023-12-01T01:19:11.386 INFO:tasks.workunit.client.0.smithi077.stdout:checking command to parse /usr/bin/nm -B output from gcc object... ok 2023-12-01T01:19:11.387 INFO:tasks.workunit.client.0.smithi077.stdout:checking for sysroot... no 2023-12-01T01:19:11.399 INFO:tasks.workunit.client.0.smithi077.stdout:checking for a working dd... /usr/bin/dd 2023-12-01T01:19:11.410 INFO:tasks.workunit.client.0.smithi077.stdout:checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 2023-12-01T01:19:11.444 INFO:tasks.workunit.client.0.smithi077.stdout:checking for mt... no 2023-12-01T01:19:11.458 INFO:tasks.workunit.client.0.smithi077.stdout:checking if : is a manifest tool... no 2023-12-01T01:19:11.530 INFO:tasks.workunit.client.0.smithi077.stdout:checking how to run the C preprocessor... gcc -E 2023-12-01T01:19:11.710 INFO:tasks.workunit.client.0.smithi077.stdout:checking for ANSI C header files... yes 2023-12-01T01:19:11.740 INFO:tasks.workunit.client.0.smithi077.stdout:checking for sys/types.h... yes 2023-12-01T01:19:11.776 INFO:tasks.workunit.client.0.smithi077.stdout:checking for sys/stat.h... yes 2023-12-01T01:19:11.812 INFO:tasks.workunit.client.0.smithi077.stdout:checking for stdlib.h... yes 2023-12-01T01:19:11.849 INFO:tasks.workunit.client.0.smithi077.stdout:checking for string.h... yes 2023-12-01T01:19:11.889 INFO:tasks.workunit.client.0.smithi077.stdout:checking for memory.h... yes 2023-12-01T01:19:11.903 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:11 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mon-a[98843]: 2023-12-01T01:19:11.625+0000 7f988a60c700 -1 log_channel(cluster) log [ERR] : Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2023-12-01T01:19:11.903 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:11 smithi077 ceph-mon[98867]: pgmap v337: 97 pgs: 97 active+clean; 37 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 1.18k op/s 2023-12-01T01:19:11.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:11 smithi077 ceph-mon[98867]: Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2023-12-01T01:19:11.930 INFO:tasks.workunit.client.0.smithi077.stdout:checking for strings.h... yes 2023-12-01T01:19:11.972 INFO:tasks.workunit.client.0.smithi077.stdout:checking for inttypes.h... yes 2023-12-01T01:19:12.012 INFO:tasks.workunit.client.0.smithi077.stdout:checking for stdint.h... yes 2023-12-01T01:19:12.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:11 smithi203 ceph-mon[103581]: pgmap v337: 97 pgs: 97 active+clean; 37 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 1.18k op/s 2023-12-01T01:19:12.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:11 smithi203 ceph-mon[103581]: Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2023-12-01T01:19:12.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:11 smithi137 ceph-mon[103312]: pgmap v337: 97 pgs: 97 active+clean; 37 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 1.18k op/s 2023-12-01T01:19:12.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:11 smithi137 ceph-mon[103312]: Health check update: 2 MDSs report damaged metadata (MDS_DAMAGE) 2023-12-01T01:19:12.051 INFO:tasks.workunit.client.0.smithi077.stdout:checking for unistd.h... yes 2023-12-01T01:19:12.099 INFO:tasks.workunit.client.0.smithi077.stdout:checking for dlfcn.h... yes 2023-12-01T01:19:12.104 INFO:tasks.workunit.client.0.smithi077.stdout:checking for objdir... .libs 2023-12-01T01:19:12.228 INFO:tasks.workunit.client.0.smithi077.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2023-12-01T01:19:12.228 INFO:tasks.workunit.client.0.smithi077.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2023-12-01T01:19:12.265 INFO:tasks.workunit.client.0.smithi077.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2023-12-01T01:19:12.314 INFO:tasks.workunit.client.0.smithi077.stdout:checking if gcc static flag -static works... no 2023-12-01T01:19:12.380 INFO:tasks.workunit.client.0.smithi077.stdout:checking if gcc supports -c -o file.o... yes 2023-12-01T01:19:12.380 INFO:tasks.workunit.client.0.smithi077.stdout:checking if gcc supports -c -o file.o... (cached) yes 2023-12-01T01:19:12.385 INFO:tasks.workunit.client.0.smithi077.stdout:checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes 2023-12-01T01:19:12.427 INFO:tasks.workunit.client.0.smithi077.stdout:checking whether -lc should be explicitly linked in... no 2023-12-01T01:19:13.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:12 smithi203 ceph-mon[103581]: pgmap v338: 97 pgs: 97 active+clean; 33 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 929 op/s 2023-12-01T01:19:13.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:12 smithi137 ceph-mon[103312]: pgmap v338: 97 pgs: 97 active+clean; 33 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 929 op/s 2023-12-01T01:19:13.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:12 smithi077 ceph-mon[98867]: pgmap v338: 97 pgs: 97 active+clean; 33 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 929 op/s 2023-12-01T01:19:13.460 INFO:tasks.workunit.client.0.smithi077.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2023-12-01T01:19:13.460 INFO:tasks.workunit.client.0.smithi077.stdout:checking how to hardcode library paths into programs... immediate 2023-12-01T01:19:13.470 INFO:tasks.workunit.client.0.smithi077.stdout:checking whether stripping libraries is possible... yes 2023-12-01T01:19:13.470 INFO:tasks.workunit.client.0.smithi077.stdout:checking if libtool supports shared libraries... yes 2023-12-01T01:19:13.470 INFO:tasks.workunit.client.0.smithi077.stdout:checking whether to build shared libraries... yes 2023-12-01T01:19:13.470 INFO:tasks.workunit.client.0.smithi077.stdout:checking whether to build static libraries... yes 2023-12-01T01:19:13.475 INFO:tasks.workunit.client.0.smithi077.stdout:checking for gcc... (cached) gcc 2023-12-01T01:19:13.509 INFO:tasks.workunit.client.0.smithi077.stdout:checking whether we are using the GNU C compiler... (cached) yes 2023-12-01T01:19:13.509 INFO:tasks.workunit.client.0.smithi077.stdout:checking whether gcc accepts -g... (cached) yes 2023-12-01T01:19:13.509 INFO:tasks.workunit.client.0.smithi077.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2023-12-01T01:19:13.509 INFO:tasks.workunit.client.0.smithi077.stdout:checking for gmake... /usr/bin/gmake 2023-12-01T01:19:13.510 INFO:tasks.workunit.client.0.smithi077.stderr:./configure: line 11956: test: $(SHELL): binary operator expected 2023-12-01T01:19:13.510 INFO:tasks.workunit.client.0.smithi077.stdout:checking for tar... /bin/tar 2023-12-01T01:19:13.511 INFO:tasks.workunit.client.0.smithi077.stdout:checking for gzip... /bin/gzip 2023-12-01T01:19:13.520 INFO:tasks.workunit.client.0.smithi077.stdout:checking whether gcc -MM is supported... yes 2023-12-01T01:19:13.520 INFO:tasks.workunit.client.0.smithi077.stdout:checking for sort... /bin/sort 2023-12-01T01:19:13.521 INFO:tasks.workunit.client.0.smithi077.stdout:checking whether ln -s works... yes 2023-12-01T01:19:13.521 INFO:tasks.workunit.client.0.smithi077.stdout:checking for rpm... /bin/rpm 2023-12-01T01:19:13.525 INFO:tasks.workunit.client.0.smithi077.stdout:checking for rpmbuild... no 2023-12-01T01:19:13.525 INFO:tasks.workunit.client.0.smithi077.stdout:checking for ANSI C header files... (cached) yes 2023-12-01T01:19:13.617 INFO:tasks.workunit.client.0.smithi077.stdout:checking assert.h usability... yes 2023-12-01T01:19:13.629 INFO:tasks.workunit.client.0.smithi077.stdout:checking assert.h presence... yes 2023-12-01T01:19:13.629 INFO:tasks.workunit.client.0.smithi077.stdout:checking for assert.h... yes 2023-12-01T01:19:13.676 INFO:tasks.workunit.client.0.smithi077.stdout:checking bstring.h usability... no 2023-12-01T01:19:13.706 INFO:tasks.workunit.client.0.smithi077.stdout:checking bstring.h presence... no 2023-12-01T01:19:13.706 INFO:tasks.workunit.client.0.smithi077.stdout:checking for bstring.h... no 2023-12-01T01:19:13.762 INFO:tasks.workunit.client.0.smithi077.stdout:checking libgen.h usability... yes 2023-12-01T01:19:13.775 INFO:tasks.workunit.client.0.smithi077.stdout:checking libgen.h presence... yes 2023-12-01T01:19:13.775 INFO:tasks.workunit.client.0.smithi077.stdout:checking for libgen.h... yes 2023-12-01T01:19:13.824 INFO:tasks.workunit.client.0.smithi077.stdout:checking dirent.h usability... yes 2023-12-01T01:19:13.840 INFO:tasks.workunit.client.0.smithi077.stdout:checking dirent.h presence... yes 2023-12-01T01:19:13.840 INFO:tasks.workunit.client.0.smithi077.stdout:checking for dirent.h... yes 2023-12-01T01:19:13.886 INFO:tasks.workunit.client.0.smithi077.stdout:checking errno.h usability... yes 2023-12-01T01:19:13.898 INFO:tasks.workunit.client.0.smithi077.stdout:checking errno.h presence... yes 2023-12-01T01:19:13.899 INFO:tasks.workunit.client.0.smithi077.stdout:checking for errno.h... yes 2023-12-01T01:19:13.944 INFO:tasks.workunit.client.0.smithi077.stdout:checking malloc.h usability... yes 2023-12-01T01:19:13.961 INFO:tasks.workunit.client.0.smithi077.stdout:checking malloc.h presence... yes 2023-12-01T01:19:13.961 INFO:tasks.workunit.client.0.smithi077.stdout:checking for malloc.h... yes 2023-12-01T01:19:14.012 INFO:tasks.workunit.client.0.smithi077.stdout:checking uuid.h usability... no 2023-12-01T01:19:14.035 INFO:tasks.workunit.client.0.smithi077.stdout:checking uuid.h presence... no 2023-12-01T01:19:14.035 INFO:tasks.workunit.client.0.smithi077.stdout:checking for uuid.h... no 2023-12-01T01:19:14.082 INFO:tasks.workunit.client.0.smithi077.stdout:checking uuid/uuid.h usability... yes 2023-12-01T01:19:14.099 INFO:tasks.workunit.client.0.smithi077.stdout:checking uuid/uuid.h presence... yes 2023-12-01T01:19:14.099 INFO:tasks.workunit.client.0.smithi077.stdout:checking for uuid/uuid.h... yes 2023-12-01T01:19:14.147 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/uuid.h usability... no 2023-12-01T01:19:14.170 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/uuid.h presence... no 2023-12-01T01:19:14.170 INFO:tasks.workunit.client.0.smithi077.stdout:checking for sys/uuid.h... no 2023-12-01T01:19:14.207 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/file.h usability... yes 2023-12-01T01:19:14.222 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/file.h presence... yes 2023-12-01T01:19:14.223 INFO:tasks.workunit.client.0.smithi077.stdout:checking for sys/file.h... yes 2023-12-01T01:19:14.266 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/fcntl.h usability... yes 2023-12-01T01:19:14.279 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/fcntl.h presence... yes 2023-12-01T01:19:14.279 INFO:tasks.workunit.client.0.smithi077.stdout:checking for sys/fcntl.h... yes 2023-12-01T01:19:14.326 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/syssgi.h usability... no 2023-12-01T01:19:14.349 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/syssgi.h presence... no 2023-12-01T01:19:14.350 INFO:tasks.workunit.client.0.smithi077.stdout:checking for sys/syssgi.h... no 2023-12-01T01:19:14.440 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/param.h usability... yes 2023-12-01T01:19:14.459 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/param.h presence... yes 2023-12-01T01:19:14.459 INFO:tasks.workunit.client.0.smithi077.stdout:checking for sys/param.h... yes 2023-12-01T01:19:14.474 INFO:tasks.workunit.client.0.smithi077.stdout:checking for sys/stat.h... (cached) yes 2023-12-01T01:19:14.513 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/statvfs.h usability... yes 2023-12-01T01:19:14.526 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/statvfs.h presence... yes 2023-12-01T01:19:14.526 INFO:tasks.workunit.client.0.smithi077.stdout:checking for sys/statvfs.h... yes 2023-12-01T01:19:14.572 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/time.h usability... yes 2023-12-01T01:19:14.585 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/time.h presence... yes 2023-12-01T01:19:14.585 INFO:tasks.workunit.client.0.smithi077.stdout:checking for sys/time.h... yes 2023-12-01T01:19:14.628 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/ioctl.h usability... yes 2023-12-01T01:19:14.642 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/ioctl.h presence... yes 2023-12-01T01:19:14.642 INFO:tasks.workunit.client.0.smithi077.stdout:checking for sys/ioctl.h... yes 2023-12-01T01:19:14.689 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/wait.h usability... yes 2023-12-01T01:19:14.703 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/wait.h presence... yes 2023-12-01T01:19:14.703 INFO:tasks.workunit.client.0.smithi077.stdout:checking for sys/wait.h... yes 2023-12-01T01:19:14.718 INFO:tasks.workunit.client.0.smithi077.stdout:checking for sys/types.h... (cached) yes 2023-12-01T01:19:14.722 INFO:tasks.workunit.client.0.smithi077.stdout:checking for strings.h... (cached) yes 2023-12-01T01:19:14.755 INFO:tasks.workunit.client.0.smithi077.stdout:checking err.h usability... yes 2023-12-01T01:19:14.767 INFO:tasks.workunit.client.0.smithi077.stdout:checking err.h presence... yes 2023-12-01T01:19:14.767 INFO:tasks.workunit.client.0.smithi077.stdout:checking for err.h... yes 2023-12-01T01:19:14.811 INFO:tasks.workunit.client.0.smithi077.stdout:checking linux/falloc.h usability... yes 2023-12-01T01:19:14.823 INFO:tasks.workunit.client.0.smithi077.stdout:checking linux/falloc.h presence... yes 2023-12-01T01:19:14.824 INFO:tasks.workunit.client.0.smithi077.stdout:checking for linux/falloc.h... yes 2023-12-01T01:19:14.874 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/fs/xfs_fsops.h usability... no 2023-12-01T01:19:14.895 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/fs/xfs_fsops.h presence... no 2023-12-01T01:19:14.896 INFO:tasks.workunit.client.0.smithi077.stdout:checking for sys/fs/xfs_fsops.h... no 2023-12-01T01:19:14.935 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/fs/xfs_itable.h usability... no 2023-12-01T01:19:14.959 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/fs/xfs_itable.h presence... no 2023-12-01T01:19:14.959 INFO:tasks.workunit.client.0.smithi077.stdout:checking for sys/fs/xfs_itable.h... no 2023-12-01T01:19:15.000 INFO:tasks.workunit.client.0.smithi077.stdout:checking xfs/platform_defs.h usability... no 2023-12-01T01:19:15.023 INFO:tasks.workunit.client.0.smithi077.stdout:checking xfs/platform_defs.h presence... no 2023-12-01T01:19:15.023 INFO:tasks.workunit.client.0.smithi077.stdout:checking for xfs/platform_defs.h... no 2023-12-01T01:19:15.063 INFO:tasks.workunit.client.0.smithi077.stdout:checking btrfs/ioctl.h usability... no 2023-12-01T01:19:15.086 INFO:tasks.workunit.client.0.smithi077.stdout:checking btrfs/ioctl.h presence... no 2023-12-01T01:19:15.086 INFO:tasks.workunit.client.0.smithi077.stdout:checking for btrfs/ioctl.h... no 2023-12-01T01:19:15.127 INFO:tasks.workunit.client.0.smithi077.stdout:checking cifs/ioctl.h usability... no 2023-12-01T01:19:15.152 INFO:tasks.workunit.client.0.smithi077.stdout:checking cifs/ioctl.h presence... no 2023-12-01T01:19:15.153 INFO:tasks.workunit.client.0.smithi077.stdout:checking for cifs/ioctl.h... no 2023-12-01T01:19:15.196 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/mman.h usability... yes 2023-12-01T01:19:15.209 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/mman.h presence... yes 2023-12-01T01:19:15.209 INFO:tasks.workunit.client.0.smithi077.stdout:checking for sys/mman.h... yes 2023-12-01T01:19:15.248 INFO:tasks.workunit.client.0.smithi077.stdout:checking for xfs/xfs_log_format.h... no 2023-12-01T01:19:15.301 INFO:tasks.workunit.client.0.smithi077.stdout:checking for uuid_compare... no 2023-12-01T01:19:15.344 INFO:tasks.workunit.client.0.smithi077.stdout:checking for uuid_compare in -luuid... yes 2023-12-01T01:19:15.385 INFO:tasks.workunit.client.0.smithi077.stdout:checking for xfs/xfs.h... yes 2023-12-01T01:19:15.417 INFO:tasks.workunit.client.0.smithi077.stdout:checking for xfs/libxfs.h... no 2023-12-01T01:19:15.447 INFO:tasks.workunit.client.0.smithi077.stdout:checking whether xlog_assign_lsn is declared... no 2023-12-01T01:19:15.483 INFO:tasks.workunit.client.0.smithi077.stdout:checking for xfs/xqm.h... yes 2023-12-01T01:19:15.531 INFO:tasks.workunit.client.0.smithi077.stdout:checking xfsctl from xfs/xfs.h... ok 2023-12-01T01:19:15.563 INFO:tasks.workunit.client.0.smithi077.stdout:checking xfs/handle.h usability... yes 2023-12-01T01:19:15.575 INFO:tasks.workunit.client.0.smithi077.stdout:checking xfs/handle.h presence... yes 2023-12-01T01:19:15.575 INFO:tasks.workunit.client.0.smithi077.stdout:checking for xfs/handle.h... yes 2023-12-01T01:19:15.623 INFO:tasks.workunit.client.0.smithi077.stdout:checking for attr_list_by_handle in -lhandle... yes 2023-12-01T01:19:15.653 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:15 smithi077 ceph-mon[98867]: pgmap v339: 97 pgs: 97 active+clean; 30 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 664 op/s 2023-12-01T01:19:15.688 INFO:tasks.workunit.client.0.smithi077.stdout:checking attr/attributes.h usability... yes 2023-12-01T01:19:15.702 INFO:tasks.workunit.client.0.smithi077.stdout:checking attr/attributes.h presence... yes 2023-12-01T01:19:15.702 INFO:tasks.workunit.client.0.smithi077.stdout:checking for attr/attributes.h... yes 2023-12-01T01:19:15.751 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/attributes.h usability... no 2023-12-01T01:19:15.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:15 smithi203 ceph-mon[103581]: pgmap v339: 97 pgs: 97 active+clean; 30 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 664 op/s 2023-12-01T01:19:15.775 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/attributes.h presence... no 2023-12-01T01:19:15.775 INFO:tasks.workunit.client.0.smithi077.stdout:checking for sys/attributes.h... no 2023-12-01T01:19:15.778 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:15 smithi137 ceph-mon[103312]: pgmap v339: 97 pgs: 97 active+clean; 30 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 664 op/s 2023-12-01T01:19:15.812 INFO:tasks.workunit.client.0.smithi077.stdout:checking for attr_list in -lattr... yes 2023-12-01T01:19:15.850 INFO:tasks.workunit.client.0.smithi077.stdout:checking for attr_set in -lattr... yes 2023-12-01T01:19:15.892 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/acl.h usability... yes 2023-12-01T01:19:15.906 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/acl.h presence... yes 2023-12-01T01:19:15.906 INFO:tasks.workunit.client.0.smithi077.stdout:checking for sys/acl.h... yes 2023-12-01T01:19:15.946 INFO:tasks.workunit.client.0.smithi077.stdout:checking acl/libacl.h usability... yes 2023-12-01T01:19:15.960 INFO:tasks.workunit.client.0.smithi077.stdout:checking acl/libacl.h presence... yes 2023-12-01T01:19:15.960 INFO:tasks.workunit.client.0.smithi077.stdout:checking for acl/libacl.h... yes 2023-12-01T01:19:15.999 INFO:tasks.workunit.client.0.smithi077.stdout:checking for acl_init in -lacl... yes 2023-12-01T01:19:16.055 INFO:tasks.workunit.client.0.smithi077.stdout:checking gdbm-ndbm.h usability... no 2023-12-01T01:19:16.080 INFO:tasks.workunit.client.0.smithi077.stdout:checking gdbm-ndbm.h presence... no 2023-12-01T01:19:16.080 INFO:tasks.workunit.client.0.smithi077.stdout:checking for gdbm-ndbm.h... no 2023-12-01T01:19:16.116 INFO:tasks.workunit.client.0.smithi077.stdout:checking gdbm/ndbm.h usability... no 2023-12-01T01:19:16.139 INFO:tasks.workunit.client.0.smithi077.stdout:checking gdbm/ndbm.h presence... no 2023-12-01T01:19:16.140 INFO:tasks.workunit.client.0.smithi077.stdout:checking for gdbm/ndbm.h... no 2023-12-01T01:19:16.176 INFO:tasks.workunit.client.0.smithi077.stdout:checking ndbm.h usability... no 2023-12-01T01:19:16.202 INFO:tasks.workunit.client.0.smithi077.stdout:checking ndbm.h presence... no 2023-12-01T01:19:16.202 INFO:tasks.workunit.client.0.smithi077.stdout:checking for ndbm.h... no 2023-12-01T01:19:16.235 INFO:tasks.workunit.client.0.smithi077.stdout:checking libaio.h usability... yes 2023-12-01T01:19:16.252 INFO:tasks.workunit.client.0.smithi077.stdout:checking libaio.h presence... yes 2023-12-01T01:19:16.252 INFO:tasks.workunit.client.0.smithi077.stdout:checking for libaio.h... yes 2023-12-01T01:19:16.302 INFO:tasks.workunit.client.0.smithi077.stdout:checking for dm_make_handle in -ldm... no 2023-12-01T01:19:16.302 INFO:tasks.workunit.client.0.smithi077.stdout: 2023-12-01T01:19:16.303 INFO:tasks.workunit.client.0.smithi077.stdout:WARNING: could not find a valid DMAPI base library. 2023-12-01T01:19:16.303 INFO:tasks.workunit.client.0.smithi077.stdout:If you want DMAPI support please install the data migration 2023-12-01T01:19:16.303 INFO:tasks.workunit.client.0.smithi077.stdout:API (dmapi) library package. Alternatively, run "make install" 2023-12-01T01:19:16.303 INFO:tasks.workunit.client.0.smithi077.stdout:from the dmapi source. 2023-12-01T01:19:16.303 INFO:tasks.workunit.client.0.smithi077.stdout: 2023-12-01T01:19:16.335 INFO:tasks.workunit.client.0.smithi077.stdout:checking linux/fiemap.h usability... yes 2023-12-01T01:19:16.348 INFO:tasks.workunit.client.0.smithi077.stdout:checking linux/fiemap.h presence... yes 2023-12-01T01:19:16.348 INFO:tasks.workunit.client.0.smithi077.stdout:checking for linux/fiemap.h... yes 2023-12-01T01:19:16.386 INFO:tasks.workunit.client.0.smithi077.stdout:checking for fallocate... yes 2023-12-01T01:19:16.428 INFO:tasks.workunit.client.0.smithi077.stdout:checking for open_by_handle_at... yes 2023-12-01T01:19:16.465 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/prctl.h usability... yes 2023-12-01T01:19:16.478 INFO:tasks.workunit.client.0.smithi077.stdout:checking sys/prctl.h presence... yes 2023-12-01T01:19:16.478 INFO:tasks.workunit.client.0.smithi077.stdout:checking for sys/prctl.h... yes 2023-12-01T01:19:16.510 INFO:tasks.workunit.client.0.smithi077.stdout:checking linux/fs.h usability... yes 2023-12-01T01:19:16.524 INFO:tasks.workunit.client.0.smithi077.stdout:checking linux/fs.h presence... yes 2023-12-01T01:19:16.524 INFO:tasks.workunit.client.0.smithi077.stdout:checking for linux/fs.h... yes 2023-12-01T01:19:16.584 INFO:tasks.workunit.client.0.smithi077.stdout:checking btrfsutil.h usability... no 2023-12-01T01:19:16.698 INFO:tasks.workunit.client.0.smithi077.stdout:checking btrfsutil.h presence... no 2023-12-01T01:19:16.698 INFO:tasks.workunit.client.0.smithi077.stdout:checking for btrfsutil.h... no 2023-12-01T01:19:16.749 INFO:tasks.workunit.client.0.smithi077.stdout:checking for copy_file_range... yes 2023-12-01T01:19:16.791 INFO:tasks.workunit.client.0.smithi077.stdout:checking for renameat2... yes 2023-12-01T01:19:16.810 INFO:tasks.workunit.client.0.smithi077.stdout:configure: creating ./config.status 2023-12-01T01:19:17.141 INFO:tasks.workunit.client.0.smithi077.stdout:config.status: creating include/builddefs 2023-12-01T01:19:17.903 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:17 smithi077 ceph-mon[98867]: pgmap v340: 97 pgs: 97 active+clean; 29 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 360 op/s 2023-12-01T01:19:18.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:17 smithi203 ceph-mon[103581]: pgmap v340: 97 pgs: 97 active+clean; 29 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 360 op/s 2023-12-01T01:19:18.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:17 smithi137 ceph-mon[103312]: pgmap v340: 97 pgs: 97 active+clean; 29 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 360 op/s 2023-12-01T01:19:18.691 INFO:tasks.workunit.client.0.smithi077.stdout:config.status: creating include/config.h 2023-12-01T01:19:19.903 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:19 smithi077 ceph-mon[98867]: pgmap v341: 97 pgs: 97 active+clean; 29 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 333 op/s 2023-12-01T01:19:20.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:19 smithi203 ceph-mon[103581]: pgmap v341: 97 pgs: 97 active+clean; 29 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 333 op/s 2023-12-01T01:19:20.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:19 smithi137 ceph-mon[103312]: pgmap v341: 97 pgs: 97 active+clean; 29 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 333 op/s 2023-12-01T01:19:21.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:21 smithi077 ceph-mon[98867]: pgmap v342: 97 pgs: 97 active+clean; 29 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 1.0 MiB/s wr, 305 op/s 2023-12-01T01:19:22.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:21 smithi203 ceph-mon[103581]: pgmap v342: 97 pgs: 97 active+clean; 29 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 1.0 MiB/s wr, 305 op/s 2023-12-01T01:19:22.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:21 smithi137 ceph-mon[103312]: pgmap v342: 97 pgs: 97 active+clean; 29 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 1.0 MiB/s wr, 305 op/s 2023-12-01T01:19:23.490 INFO:tasks.workunit.client.0.smithi077.stdout:config.status: executing libtool commands 2023-12-01T01:19:23.658 INFO:tasks.workunit.client.0.smithi077.stdout:Building include 2023-12-01T01:19:23.659 INFO:tasks.workunit.client.0.smithi077.stdout:Building lib 2023-12-01T01:19:23.659 INFO:tasks.workunit.client.0.smithi077.stdout:Building tests 2023-12-01T01:19:23.673 INFO:tasks.workunit.client.0.smithi077.stdout: [LTDEP] 2023-12-01T01:19:23.824 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] databin.lo 2023-12-01T01:19:23.824 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] dataascii.lo 2023-12-01T01:19:23.824 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] datapid.lo 2023-12-01T01:19:23.824 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] file_lock.lo 2023-12-01T01:19:24.010 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] pattern.lo 2023-12-01T01:19:24.010 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] forker.lo 2023-12-01T01:19:24.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:23 smithi203 ceph-mon[103581]: pgmap v343: 97 pgs: 97 active+clean; 30 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 994 KiB/s wr, 260 op/s 2023-12-01T01:19:24.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:23 smithi137 ceph-mon[103312]: pgmap v343: 97 pgs: 97 active+clean; 30 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 994 KiB/s wr, 260 op/s 2023-12-01T01:19:24.031 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] random_range.lo 2023-12-01T01:19:24.031 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] open_flags.lo 2023-12-01T01:19:24.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:23 smithi077 ceph-mon[98867]: pgmap v343: 97 pgs: 97 active+clean; 30 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 994 KiB/s wr, 260 op/s 2023-12-01T01:19:24.187 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] string_to_tokens.lo 2023-12-01T01:19:24.217 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] str_to_bytes.lo 2023-12-01T01:19:24.262 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] tlibio.lo 2023-12-01T01:19:24.356 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] random.lo 2023-12-01T01:19:24.356 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] write_log.lo 2023-12-01T01:19:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.438+0000 7fcc7256a700 1 -- 172.21.15.77:0/3335297811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc6c0ffc90 msgr2=0x7fcc6c1000b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.438+0000 7fcc7256a700 1 --2- 172.21.15.77:0/3335297811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc6c0ffc90 0x7fcc6c1000b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c0099c0 tx=0x7fcc5c009cd0 comp rx=0 tx=0).stop 2023-12-01T01:19:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.438+0000 7fcc7256a700 1 -- 172.21.15.77:0/3335297811 shutdown_connections 2023-12-01T01:19:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.438+0000 7fcc7256a700 1 --2- 172.21.15.77:0/3335297811 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc6c101820 0x7fcc6c1080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.438+0000 7fcc7256a700 1 --2- 172.21.15.77:0/3335297811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc6c0ffc90 0x7fcc6c1000b0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.438+0000 7fcc7256a700 1 --2- 172.21.15.77:0/3335297811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc6c100e80 0x7fcc6c1012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.438+0000 7fcc7256a700 1 -- 172.21.15.77:0/3335297811 >> 172.21.15.77:0/3335297811 conn(0x7fcc6c0fb1e0 msgr2=0x7fcc6c0fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:19:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.438+0000 7fcc7256a700 1 -- 172.21.15.77:0/3335297811 shutdown_connections 2023-12-01T01:19:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.438+0000 7fcc7256a700 1 -- 172.21.15.77:0/3335297811 wait complete. 2023-12-01T01:19:24.441 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.440+0000 7fcc7256a700 1 Processor -- start 2023-12-01T01:19:24.441 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.440+0000 7fcc7256a700 1 -- start start 2023-12-01T01:19:24.443 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.440+0000 7fcc7256a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc6c0ffc90 0x7fcc6c195ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:24.443 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.440+0000 7fcc7256a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc6c100e80 0x7fcc6c196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:24.443 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.440+0000 7fcc7256a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc6c101820 0x7fcc6c196e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:24.443 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.440+0000 7fcc7256a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6c1974e0 con 0x7fcc6c101820 2023-12-01T01:19:24.443 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.440+0000 7fcc7256a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6c197650 con 0x7fcc6c0ffc90 2023-12-01T01:19:24.444 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.440+0000 7fcc7256a700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6c197810 con 0x7fcc6c100e80 2023-12-01T01:19:24.444 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.440+0000 7fcc6bfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc6c0ffc90 0x7fcc6c195ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:24.444 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.440+0000 7fcc70b07700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc6c101820 0x7fcc6c196e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:24.444 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.440+0000 7fcc6bfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc6c0ffc90 0x7fcc6c195ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:39238/0 (socket says 172.21.15.77:39238) 2023-12-01T01:19:24.444 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.441+0000 7fcc6bfff700 1 -- 172.21.15.77:0/3930384867 learned_addr learned my addr 172.21.15.77:0/3930384867 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:19:24.444 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.441+0000 7fcc6b7fe700 1 --2- 172.21.15.77:0/3930384867 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc6c100e80 0x7fcc6c196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:24.444 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.441+0000 7fcc6bfff700 1 -- 172.21.15.77:0/3930384867 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc6c100e80 msgr2=0x7fcc6c196000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:24.444 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.441+0000 7fcc6bfff700 1 --2- 172.21.15.77:0/3930384867 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc6c100e80 0x7fcc6c196000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:24.444 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.441+0000 7fcc6bfff700 1 -- 172.21.15.77:0/3930384867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc6c101820 msgr2=0x7fcc6c196e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:24.444 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.441+0000 7fcc6bfff700 1 --2- 172.21.15.77:0/3930384867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc6c101820 0x7fcc6c196e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:24.444 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.441+0000 7fcc6bfff700 1 -- 172.21.15.77:0/3930384867 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc60009520 con 0x7fcc6c0ffc90 2023-12-01T01:19:24.444 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.441+0000 7fcc6b7fe700 1 --2- 172.21.15.77:0/3930384867 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc6c100e80 0x7fcc6c196000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:19:24.444 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.441+0000 7fcc6bfff700 1 --2- 172.21.15.77:0/3930384867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc6c0ffc90 0x7fcc6c195ac0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c0098e0 tx=0x7fcc5c00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:19:24.444 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.441+0000 7fcc697fa700 1 -- 172.21.15.77:0/3930384867 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c01d070 con 0x7fcc6c0ffc90 2023-12-01T01:19:24.445 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.441+0000 7fcc697fa700 1 -- 172.21.15.77:0/3930384867 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc5c005050 con 0x7fcc6c0ffc90 2023-12-01T01:19:24.445 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.441+0000 7fcc697fa700 1 -- 172.21.15.77:0/3930384867 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c004b40 con 0x7fcc6c0ffc90 2023-12-01T01:19:24.445 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.441+0000 7fcc7256a700 1 -- 172.21.15.77:0/3930384867 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc5c0095f0 con 0x7fcc6c0ffc90 2023-12-01T01:19:24.445 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.441+0000 7fcc7256a700 1 -- 172.21.15.77:0/3930384867 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc6c197cd0 con 0x7fcc6c0ffc90 2023-12-01T01:19:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.445+0000 7fcc697fa700 1 -- 172.21.15.77:0/3930384867 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fcc5c004ce0 con 0x7fcc6c0ffc90 2023-12-01T01:19:24.447 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.445+0000 7fcc7e6fab80 1 Processor -- start 2023-12-01T01:19:24.447 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.446+0000 7fcc7e6fab80 1 -- start start 2023-12-01T01:19:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.446+0000 7fcc7e6fab80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56405dab2680 0x56405dab2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.446+0000 7fcc7e6fab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56405dab2fe0 0x56405dab38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.446+0000 7fcc7e6fab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56405dab3ed0 0x56405dab4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.446+0000 7fcc7e6fab80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x56405da0a9b0 con 0x56405dab2680 2023-12-01T01:19:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.446+0000 7fcc7e6fab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56405da08370 con 0x56405dab2fe0 2023-12-01T01:19:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.446+0000 7fcc7e6fab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56405da03c60 con 0x56405dab3ed0 2023-12-01T01:19:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.446+0000 7fcc6bfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56405dab2fe0 0x56405dab38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.446+0000 7fcc70b07700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56405dab2680 0x56405dab2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.446+0000 7fcc6bfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56405dab2fe0 0x56405dab38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:39246/0 (socket says 172.21.15.77:39246) 2023-12-01T01:19:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.446+0000 7fcc6bfff700 1 -- 172.21.15.77:0/1453104698 learned_addr learned my addr 172.21.15.77:0/1453104698 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:19:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.446+0000 7fcc697fa700 1 --2- 172.21.15.77:0/3930384867 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fcc540a0520 0x7fcc540a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:24.448 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.446+0000 7fcc6b7fe700 1 --2- 172.21.15.77:0/1453104698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56405dab3ed0 0x56405dab4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:24.449 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.447+0000 7fcc697fa700 1 -- 172.21.15.77:0/3930384867 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fcc5c0cc580 con 0x7fcc6c0ffc90 2023-12-01T01:19:24.449 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.447+0000 7fcc70b07700 1 --2- 172.21.15.77:0/3930384867 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fcc540a0520 0x7fcc540a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:24.449 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.447+0000 7fcc6bfff700 1 -- 172.21.15.77:0/1453104698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56405dab3ed0 msgr2=0x56405dab4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:24.449 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.447+0000 7fcc6bfff700 1 --2- 172.21.15.77:0/1453104698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56405dab3ed0 0x56405dab4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:24.449 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.447+0000 7fcc6bfff700 1 -- 172.21.15.77:0/1453104698 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56405dab2680 msgr2=0x56405dab2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:24.449 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.447+0000 7fcc6bfff700 1 --2- 172.21.15.77:0/1453104698 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56405dab2680 0x56405dab2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:24.449 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.447+0000 7fcc6bfff700 1 -- 172.21.15.77:0/1453104698 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56405d831d80 con 0x56405dab2fe0 2023-12-01T01:19:24.449 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.447+0000 7fcc70b07700 1 --2- 172.21.15.77:0/1453104698 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56405dab2680 0x56405dab2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:19:24.452 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.447+0000 7fcc6bfff700 1 --2- 172.21.15.77:0/1453104698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56405dab2fe0 0x56405dab38f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c00bf40 tx=0x7fcc5c004f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a0600d2782946e0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:19:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.447+0000 7fcc70b07700 1 --2- 172.21.15.77:0/3930384867 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fcc540a0520 0x7fcc540a29e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcc64009ca0 tx=0x7fcc64004b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:19:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.447+0000 7fcc527fc700 1 -- 172.21.15.77:0/1453104698 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c099870 con 0x56405dab2fe0 2023-12-01T01:19:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.447+0000 7fcc527fc700 1 -- 172.21.15.77:0/1453104698 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc5c09a610 con 0x56405dab2fe0 2023-12-01T01:19:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.447+0000 7fcc527fc700 1 -- 172.21.15.77:0/1453104698 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c0982b0 con 0x56405dab2fe0 2023-12-01T01:19:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.448+0000 7fcc7e6fab80 1 -- 172.21.15.77:0/1453104698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56405dab2fe0 msgr2=0x56405dab38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.448+0000 7fcc7e6fab80 1 --2- 172.21.15.77:0/1453104698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56405dab2fe0 0x56405dab38f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c00bf40 tx=0x7fcc5c004f90 comp rx=0 tx=0).stop 2023-12-01T01:19:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.448+0000 7fcc7e6fab80 1 -- 172.21.15.77:0/1453104698 shutdown_connections 2023-12-01T01:19:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.448+0000 7fcc7e6fab80 1 --2- 172.21.15.77:0/1453104698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56405dab3ed0 0x56405dab4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.448+0000 7fcc7e6fab80 1 --2- 172.21.15.77:0/1453104698 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56405dab2680 0x56405dab2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.448+0000 7fcc7e6fab80 1 --2- 172.21.15.77:0/1453104698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56405dab2fe0 0x56405dab38f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.448+0000 7fcc7e6fab80 1 -- 172.21.15.77:0/1453104698 >> 172.21.15.77:0/1453104698 conn(0x56405da12070 msgr2=0x56405daaf040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:19:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.448+0000 7fcc7e6fab80 1 -- 172.21.15.77:0/1453104698 shutdown_connections 2023-12-01T01:19:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.448+0000 7fcc7e6fab80 1 -- 172.21.15.77:0/1453104698 wait complete. 2023-12-01T01:19:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.449+0000 7fcc7e6fab80 1 -- start start 2023-12-01T01:19:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.449+0000 7fcc7e6fab80 1 Processor -- start 2023-12-01T01:19:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.449+0000 7fcc7e6fab80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56405dab2680 0x56405dac4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.449+0000 7fcc7e6fab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56405dab2fe0 0x56405dac4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.449+0000 7fcc7e6fab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56405dab3ed0 0x56405dacbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.449+0000 7fcc7e6fab80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x56405d9a04b0 con 0x56405dab2680 2023-12-01T01:19:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.449+0000 7fcc7e6fab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56405d9a0150 con 0x56405dab3ed0 2023-12-01T01:19:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.449+0000 7fcc7e6fab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56405d8878c0 con 0x56405dab2fe0 2023-12-01T01:19:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.449+0000 7fcc7e6fab80 10 client.0 _ll_register_callbacks cb 0x56405d8eda20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:19:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.449+0000 7fcc70b07700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56405dab3ed0 0x56405dacbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.450+0000 7fcc70b07700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56405dab3ed0 0x56405dacbde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:39254/0 (socket says 172.21.15.77:39254) 2023-12-01T01:19:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.450+0000 7fcc70b07700 1 -- 172.21.15.77:0/1535361808 learned_addr learned my addr 172.21.15.77:0/1535361808 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:19:24.454 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.450+0000 7fcc6b7fe700 1 --2- 172.21.15.77:0/1535361808 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56405dab2fe0 0x56405dac4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.450+0000 7fcc6bfff700 1 --2- 172.21.15.77:0/1535361808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56405dab2680 0x56405dac4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.450+0000 7fcc70b07700 1 -- 172.21.15.77:0/1535361808 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56405dab2fe0 msgr2=0x56405dac4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.450+0000 7fcc70b07700 1 --2- 172.21.15.77:0/1535361808 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56405dab2fe0 0x56405dac4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.450+0000 7fcc70b07700 1 -- 172.21.15.77:0/1535361808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56405dab2680 msgr2=0x56405dac4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.450+0000 7fcc70b07700 1 --2- 172.21.15.77:0/1535361808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56405dab2680 0x56405dac4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.450+0000 7fcc70b07700 1 -- 172.21.15.77:0/1535361808 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc6c197cd0 con 0x56405dab3ed0 2023-12-01T01:19:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.450+0000 7fcc6b7fe700 1 --2- 172.21.15.77:0/1535361808 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56405dab2fe0 0x56405dac4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:19:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.450+0000 7fcc70b07700 1 --2- 172.21.15.77:0/1535361808 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56405dab3ed0 0x56405dacbde0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcc640049a0 tx=0x7fcc64004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9114b5e98915725b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:19:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.450+0000 7fcc4b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2023-12-01T01:19:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.450+0000 7fcc4b7fe700 1 -- 172.21.15.77:0/1535361808 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc64018620 con 0x56405dab3ed0 2023-12-01T01:19:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.450+0000 7fcc4b7fe700 1 -- 172.21.15.77:0/1535361808 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fcc64007e80 con 0x56405dab3ed0 2023-12-01T01:19:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.450+0000 7fcc7e6fab80 1 -- 172.21.15.77:0/1535361808 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56405da2ecb0 con 0x56405dab3ed0 2023-12-01T01:19:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.451+0000 7fcc4b7fe700 1 -- 172.21.15.77:0/1535361808 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc64018780 con 0x56405dab3ed0 2023-12-01T01:19:24.455 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.451+0000 7fcc7e6fab80 10 client.25469 fetch_fsmap learned FSMap version 31 2023-12-01T01:19:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.451+0000 7fcc7e6fab80 1 -- 172.21.15.77:0/1535361808 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56405d81ac00 con 0x56405dab3ed0 2023-12-01T01:19:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.451+0000 7fcc4b7fe700 1 -- 172.21.15.77:0/1535361808 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56405da2ecb0 con 0x56405dab3ed0 2023-12-01T01:19:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.451+0000 7fcc4b7fe700 1 -- 172.21.15.77:0/1535361808 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcc64024980 con 0x56405dab3ed0 2023-12-01T01:19:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.451+0000 7fcc7e6fab80 10 client.25469 fetch_fsmap finished waiting for FSMap version 31 2023-12-01T01:19:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.451+0000 7fcc7e6fab80 10 client.25469 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-01T01:19:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.451+0000 7fcc7e6fab80 20 client.25469 populate_metadata read hostname 'smithi077' 2023-12-01T01:19:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.451+0000 7fcc7e6fab80 1 --2- 172.21.15.77:0/1535361808 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x56405dad0740 0x56405dad2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.451+0000 7fcc7e6fab80 4 client.25469 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:19:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.451+0000 7fcc7e6fab80 1 -- 172.21.15.77:0/1535361808 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56405da81db0 con 0x56405dad0740 2023-12-01T01:19:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.451+0000 7fcc6bfff700 1 --2- 172.21.15.77:0/1535361808 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x56405dad0740 0x56405dad2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.452+0000 7fcc4b7fe700 10 client.25469 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:19:24.456 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.452+0000 7fcc6bfff700 1 --2- 172.21.15.77:0/1535361808 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x56405dad0740 0x56405dad2b60 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8495e1fc05f99a63 server_cookie=9ed3c9274929ca61 in_seq=0 out_seq=0 2023-12-01T01:19:24.461 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.459+0000 7fcc4b7fe700 1 -- 172.21.15.77:0/1535361808 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56405da81db0 con 0x56405dad0740 2023-12-01T01:19:24.461 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.459+0000 7fcc4b7fe700 10 client.25469 handle_command_reply: tid=0 2023-12-01T01:19:24.461 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.459+0000 7fcc7e6fab80 1 client.25469 shutdown 2023-12-01T01:19:24.461 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.459+0000 7fcc7e6fab80 1 -- 172.21.15.77:0/1535361808 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56405dab3ed0 msgr2=0x56405dacbde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:24.461 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.459+0000 7fcc7e6fab80 1 --2- 172.21.15.77:0/1535361808 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56405dab3ed0 0x56405dacbde0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcc640049a0 tx=0x7fcc64004ad0 comp rx=0 tx=0).stop 2023-12-01T01:19:24.461 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.459+0000 7fcc7e6fab80 1 -- 172.21.15.77:0/1535361808 shutdown_connections 2023-12-01T01:19:24.461 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.459+0000 7fcc7e6fab80 1 --2- 172.21.15.77:0/1535361808 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x56405dad0740 0x56405dad2b60 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:24.461 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.459+0000 7fcc7e6fab80 1 --2- 172.21.15.77:0/1535361808 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56405dab3ed0 0x56405dacbde0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:24.461 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.459+0000 7fcc4b7fe700 0 client.25469 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:19:24.461 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.459+0000 7fcc7e6fab80 1 --2- 172.21.15.77:0/1535361808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56405dab2680 0x56405dac4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:24.461 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.459+0000 7fcc7e6fab80 1 --2- 172.21.15.77:0/1535361808 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56405dab2fe0 0x56405dac4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:24.461 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.459+0000 7fcc7e6fab80 1 -- 172.21.15.77:0/1535361808 >> 172.21.15.77:0/1535361808 conn(0x56405da12070 msgr2=0x56405daad3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:19:24.461 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.459+0000 7fcc7e6fab80 1 -- 172.21.15.77:0/1535361808 shutdown_connections 2023-12-01T01:19:24.461 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.459+0000 7fcc7e6fab80 1 -- 172.21.15.77:0/1535361808 wait complete. 2023-12-01T01:19:24.462 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.459+0000 7fcc7e6fab80 20 client.25469 trim_cache size 0 max 16384 2023-12-01T01:19:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.477+0000 7fcc7e6fab80 1 Processor -- start 2023-12-01T01:19:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.477+0000 7fcc7e6fab80 1 -- start start 2023-12-01T01:19:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.478+0000 7fcc7e6fab80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56405dab2680 0x56405dac9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.478+0000 7fcc7e6fab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56405dab2fe0 0x56405dab42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.478+0000 7fcc7e6fab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56405dab3ed0 0x56405dab70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.478+0000 7fcc7e6fab80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x56405daa57a0 con 0x56405dab2680 2023-12-01T01:19:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.478+0000 7fcc7e6fab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56405daa8500 con 0x56405dab3ed0 2023-12-01T01:19:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.478+0000 7fcc7e6fab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56405daae880 con 0x56405dab2fe0 2023-12-01T01:19:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.478+0000 7fcc6b7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56405dab2fe0 0x56405dab42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.478+0000 7fcc70b07700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56405dab3ed0 0x56405dab70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.478+0000 7fcc6b7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56405dab2fe0 0x56405dab42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:46530/0 (socket says 172.21.15.77:46530) 2023-12-01T01:19:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.478+0000 7fcc70b07700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56405dab3ed0 0x56405dab70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:39262/0 (socket says 172.21.15.77:39262) 2023-12-01T01:19:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.478+0000 7fcc6b7fe700 1 -- 172.21.15.77:0/758003282 learned_addr learned my addr 172.21.15.77:0/758003282 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:19:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.478+0000 7fcc6bfff700 1 --2- 172.21.15.77:0/758003282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56405dab2680 0x56405dac9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.478+0000 7fcc6b7fe700 1 -- 172.21.15.77:0/758003282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56405dab3ed0 msgr2=0x56405dab70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.478+0000 7fcc6b7fe700 1 --2- 172.21.15.77:0/758003282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56405dab3ed0 0x56405dab70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.478+0000 7fcc6b7fe700 1 -- 172.21.15.77:0/758003282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56405dab2680 msgr2=0x56405dac9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.478+0000 7fcc6b7fe700 1 --2- 172.21.15.77:0/758003282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56405dab2680 0x56405dac9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.478+0000 7fcc6b7fe700 1 -- 172.21.15.77:0/758003282 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56405d8581f0 con 0x56405dab2fe0 2023-12-01T01:19:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.478+0000 7fcc6b7fe700 1 --2- 172.21.15.77:0/758003282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56405dab2fe0 0x56405dab42b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcc60009ac0 tx=0x7fcc60009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1962e63f7269688c server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:19:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.479+0000 7fcc52ffd700 1 -- 172.21.15.77:0/758003282 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc60017040 con 0x56405dab2fe0 2023-12-01T01:19:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.479+0000 7fcc52ffd700 1 -- 172.21.15.77:0/758003282 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc60010d60 con 0x56405dab2fe0 2023-12-01T01:19:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.479+0000 7fcc52ffd700 1 -- 172.21.15.77:0/758003282 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6000ed70 con 0x56405dab2fe0 2023-12-01T01:19:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.480+0000 7fcc7e6fab80 1 -- 172.21.15.77:0/758003282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56405dab2fe0 msgr2=0x56405dab42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.480+0000 7fcc7e6fab80 1 --2- 172.21.15.77:0/758003282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56405dab2fe0 0x56405dab42b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcc60009ac0 tx=0x7fcc60009260 comp rx=0 tx=0).stop 2023-12-01T01:19:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.480+0000 7fcc7e6fab80 1 -- 172.21.15.77:0/758003282 shutdown_connections 2023-12-01T01:19:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.480+0000 7fcc7e6fab80 1 --2- 172.21.15.77:0/758003282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56405dab3ed0 0x56405dab70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.480+0000 7fcc7e6fab80 1 --2- 172.21.15.77:0/758003282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56405dab2680 0x56405dac9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.480+0000 7fcc7e6fab80 1 --2- 172.21.15.77:0/758003282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56405dab2fe0 0x56405dab42b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.480+0000 7fcc7e6fab80 1 -- 172.21.15.77:0/758003282 >> 172.21.15.77:0/758003282 conn(0x56405da12070 msgr2=0x56405daaec20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:19:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.480+0000 7fcc7e6fab80 1 -- 172.21.15.77:0/758003282 shutdown_connections 2023-12-01T01:19:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.480+0000 7fcc7e6fab80 1 -- 172.21.15.77:0/758003282 wait complete. 2023-12-01T01:19:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.480+0000 7fcc7e6fab80 1 -- start start 2023-12-01T01:19:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.480+0000 7fcc7e6fab80 1 Processor -- start 2023-12-01T01:19:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.480+0000 7fcc7e6fab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56405dab2680 0x56405db1c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.481+0000 7fcc7e6fab80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56405dab2fe0 0x56405db1e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.481+0000 7fcc7e6fab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56405dab3ed0 0x56405db20cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.481+0000 7fcc7e6fab80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x56405db21290 con 0x56405dab2fe0 2023-12-01T01:19:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.481+0000 7fcc7e6fab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56405db213d0 con 0x56405dab2680 2023-12-01T01:19:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.481+0000 7fcc7e6fab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56405db21590 con 0x56405dab3ed0 2023-12-01T01:19:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.481+0000 7fcc7e6fab80 10 client.0 _ll_register_callbacks cb 0x7fcc64004580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:19:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.481+0000 7fcc6bfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56405dab2680 0x56405db1c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.481+0000 7fcc6bfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56405dab2680 0x56405db1c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:39266/0 (socket says 172.21.15.77:39266) 2023-12-01T01:19:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.481+0000 7fcc6bfff700 1 -- 172.21.15.77:0/3311480180 learned_addr learned my addr 172.21.15.77:0/3311480180 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:19:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.481+0000 7fcc70b07700 1 --2- 172.21.15.77:0/3311480180 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56405dab3ed0 0x56405db20cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.481+0000 7fcc6bfff700 1 -- 172.21.15.77:0/3311480180 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56405dab3ed0 msgr2=0x56405db20cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.481+0000 7fcc6bfff700 1 --2- 172.21.15.77:0/3311480180 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56405dab3ed0 0x56405db20cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.481+0000 7fcc6bfff700 1 -- 172.21.15.77:0/3311480180 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56405dab2fe0 msgr2=0x56405db1e770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-01T01:19:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.481+0000 7fcc6bfff700 1 --2- 172.21.15.77:0/3311480180 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56405dab2fe0 0x56405db1e770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.481+0000 7fcc6bfff700 1 -- 172.21.15.77:0/3311480180 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc60009800 con 0x56405dab2680 2023-12-01T01:19:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.481+0000 7fcc6bfff700 1 --2- 172.21.15.77:0/3311480180 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56405dab2680 0x56405db1c220 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c0cfc00 tx=0x7fcc5c0cff90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d7bac088ec917f8 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:19:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.481+0000 7fcc4b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2023-12-01T01:19:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.481+0000 7fcc4b7fe700 1 -- 172.21.15.77:0/3311480180 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c099870 con 0x56405dab2680 2023-12-01T01:19:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.481+0000 7fcc4b7fe700 1 -- 172.21.15.77:0/3311480180 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fcc5c099f10 con 0x56405dab2680 2023-12-01T01:19:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.481+0000 7fcc7e6fab80 1 -- 172.21.15.77:0/3311480180 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcc5c09a610 con 0x56405dab2680 2023-12-01T01:19:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.482+0000 7fcc4b7fe700 1 -- 172.21.15.77:0/3311480180 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc5c0051c0 con 0x56405dab2680 2023-12-01T01:19:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.482+0000 7fcc7e6fab80 10 client.25475 fetch_fsmap learned FSMap version 31 2023-12-01T01:19:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.482+0000 7fcc7e6fab80 1 -- 172.21.15.77:0/3311480180 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56405d830250 con 0x56405dab2680 2023-12-01T01:19:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.482+0000 7fcc4b7fe700 1 -- 172.21.15.77:0/3311480180 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc5c09a610 con 0x56405dab2680 2023-12-01T01:19:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.482+0000 7fcc4b7fe700 1 -- 172.21.15.77:0/3311480180 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcc5c0b2290 con 0x56405dab2680 2023-12-01T01:19:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.482+0000 7fcc7e6fab80 10 client.25475 fetch_fsmap finished waiting for FSMap version 31 2023-12-01T01:19:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.482+0000 7fcc7e6fab80 10 client.25475 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-01T01:19:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.482+0000 7fcc7e6fab80 20 client.25475 populate_metadata read hostname 'smithi077' 2023-12-01T01:19:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.482+0000 7fcc7e6fab80 1 --2- 172.21.15.77:0/3311480180 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x56405dad0740 0x56405db24cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.482+0000 7fcc7e6fab80 4 client.25475 mds_command: new command op to 24500 tid=0[{"prefix": "flush journal"}] 2023-12-01T01:19:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.482+0000 7fcc7e6fab80 1 -- 172.21.15.77:0/3311480180 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56405d9fcce0 con 0x56405dad0740 2023-12-01T01:19:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.482+0000 7fcc6b7fe700 1 --2- 172.21.15.77:0/3311480180 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x56405dad0740 0x56405db24cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.483+0000 7fcc4b7fe700 10 client.25475 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:19:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.483+0000 7fcc6b7fe700 1 --2- 172.21.15.77:0/3311480180 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x56405dad0740 0x56405db24cc0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2276c69c3df7e3be server_cookie=d26e299133209f40 in_seq=0 out_seq=0 2023-12-01T01:19:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.535+0000 7fcc4b7fe700 1 -- 172.21.15.77:0/3311480180 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56405d9fcce0 con 0x56405dad0740 2023-12-01T01:19:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.535+0000 7fcc4b7fe700 10 client.25475 handle_command_reply: tid=0 2023-12-01T01:19:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.535+0000 7fcc7e6fab80 1 client.25475 shutdown 2023-12-01T01:19:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.536+0000 7fcc7e6fab80 1 -- 172.21.15.77:0/3311480180 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56405dab2680 msgr2=0x56405db1c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.536+0000 7fcc7e6fab80 1 --2- 172.21.15.77:0/3311480180 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56405dab2680 0x56405db1c220 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c0cfc00 tx=0x7fcc5c0cff90 comp rx=0 tx=0).stop 2023-12-01T01:19:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.536+0000 7fcc7e6fab80 1 -- 172.21.15.77:0/3311480180 shutdown_connections 2023-12-01T01:19:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.536+0000 7fcc7e6fab80 1 --2- 172.21.15.77:0/3311480180 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56405dab3ed0 0x56405db20cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.536+0000 7fcc7e6fab80 1 --2- 172.21.15.77:0/3311480180 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x56405dad0740 0x56405db24cc0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.536+0000 7fcc7e6fab80 1 --2- 172.21.15.77:0/3311480180 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56405dab2680 0x56405db1c220 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.536+0000 7fcc7e6fab80 1 --2- 172.21.15.77:0/3311480180 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56405dab2fe0 0x56405db1e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.536+0000 7fcc7e6fab80 1 -- 172.21.15.77:0/3311480180 >> 172.21.15.77:0/3311480180 conn(0x56405da12070 msgr2=0x56405dab70a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:19:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.536+0000 7fcc4b7fe700 0 client.25475 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:19:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.536+0000 7fcc7e6fab80 1 -- 172.21.15.77:0/3311480180 shutdown_connections 2023-12-01T01:19:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.536+0000 7fcc7e6fab80 1 -- 172.21.15.77:0/3311480180 wait complete. 2023-12-01T01:19:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.536+0000 7fcc7e6fab80 20 client.25475 trim_cache size 0 max 16384 2023-12-01T01:19:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout:{ 2023-12-01T01:19:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout: "message": "", 2023-12-01T01:19:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout: "return_code": 0 2023-12-01T01:19:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout:} 2023-12-01T01:19:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.537+0000 7fcc7256a700 1 -- 172.21.15.77:0/3930384867 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fcc540a0520 msgr2=0x7fcc540a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.537+0000 7fcc7256a700 1 --2- 172.21.15.77:0/3930384867 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fcc540a0520 0x7fcc540a29e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcc64009ca0 tx=0x7fcc64004b40 comp rx=0 tx=0).stop 2023-12-01T01:19:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.537+0000 7fcc7256a700 1 -- 172.21.15.77:0/3930384867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc6c0ffc90 msgr2=0x7fcc6c195ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.537+0000 7fcc7256a700 1 --2- 172.21.15.77:0/3930384867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc6c0ffc90 0x7fcc6c195ac0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c0098e0 tx=0x7fcc5c00f690 comp rx=0 tx=0).stop 2023-12-01T01:19:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.537+0000 7fcc7256a700 1 -- 172.21.15.77:0/3930384867 shutdown_connections 2023-12-01T01:19:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.537+0000 7fcc7256a700 1 --2- 172.21.15.77:0/3930384867 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fcc540a0520 0x7fcc540a29e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.537+0000 7fcc7256a700 1 --2- 172.21.15.77:0/3930384867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc6c101820 0x7fcc6c196e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.537+0000 7fcc7256a700 1 --2- 172.21.15.77:0/3930384867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc6c0ffc90 0x7fcc6c195ac0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.537+0000 7fcc7256a700 1 --2- 172.21.15.77:0/3930384867 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc6c100e80 0x7fcc6c196000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.537+0000 7fcc7256a700 1 -- 172.21.15.77:0/3930384867 >> 172.21.15.77:0/3930384867 conn(0x7fcc6c0fb1e0 msgr2=0x7fcc6c0fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:19:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.537+0000 7fcc7256a700 1 -- 172.21.15.77:0/3930384867 shutdown_connections 2023-12-01T01:19:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:24.537+0000 7fcc7256a700 1 -- 172.21.15.77:0/3930384867 wait complete. 2023-12-01T01:19:24.670 INFO:tasks.workunit.client.0.smithi077.stdout: [LD] 2023-12-01T01:19:24.774 INFO:tasks.workunit.client.0.smithi077.stdout:Building ltp 2023-12-01T01:19:24.774 INFO:tasks.workunit.client.0.smithi077.stdout:Building m4 2023-12-01T01:19:24.774 INFO:tasks.workunit.client.0.smithi077.stdout:Building src 2023-12-01T01:19:24.774 INFO:tasks.workunit.client.0.smithi077.stdout:Building common 2023-12-01T01:19:24.786 INFO:tasks.workunit.client.0.smithi077.stdout: [DEP] 2023-12-01T01:19:24.802 INFO:tasks.workunit.client.0.smithi077.stdout: [DEP] 2023-12-01T01:19:25.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:24 smithi203 ceph-mon[103581]: pgmap v344: 97 pgs: 97 active+clean; 33 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 799 KiB/s wr, 239 op/s 2023-12-01T01:19:25.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:24 smithi137 ceph-mon[103312]: pgmap v344: 97 pgs: 97 active+clean; 33 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 799 KiB/s wr, 239 op/s 2023-12-01T01:19:25.068 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] doio 2023-12-01T01:19:25.068 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] fsstress 2023-12-01T01:19:25.068 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] fsx 2023-12-01T01:19:25.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:24 smithi077 ceph-mon[98867]: pgmap v344: 97 pgs: 97 active+clean; 33 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 799 KiB/s wr, 239 op/s 2023-12-01T01:19:25.256 INFO:tasks.workunit.client.0.smithi077.stderr:fsstress.c: In function 'attr_list_path': 2023-12-01T01:19:25.256 INFO:tasks.workunit.client.0.smithi077.stderr:fsstress.c:859:2: warning: 'attr_list' is deprecated: Use listxattr or llistxattr instead [-Wdeprecated-declarations] 2023-12-01T01:19:25.256 INFO:tasks.workunit.client.0.smithi077.stderr: rval = attr_list(name->path, buffer, buffersize, flags, cursor); 2023-12-01T01:19:25.256 INFO:tasks.workunit.client.0.smithi077.stderr: ^~~~ 2023-12-01T01:19:25.256 INFO:tasks.workunit.client.0.smithi077.stderr:In file included from ../src/global.h:25, 2023-12-01T01:19:25.256 INFO:tasks.workunit.client.0.smithi077.stderr: from fsstress.c:12: 2023-12-01T01:19:25.256 INFO:tasks.workunit.client.0.smithi077.stderr:/usr/include/attr/attributes.h:165:12: note: declared here 2023-12-01T01:19:25.256 INFO:tasks.workunit.client.0.smithi077.stderr: extern int attr_list(const char *__path, char *__buffer, const int __buffersize, 2023-12-01T01:19:25.256 INFO:tasks.workunit.client.0.smithi077.stderr: ^~~~~~~~~ 2023-12-01T01:19:25.256 INFO:tasks.workunit.client.0.smithi077.stderr:fsstress.c: In function 'attr_remove_path': 2023-12-01T01:19:25.256 INFO:tasks.workunit.client.0.smithi077.stderr:fsstress.c:878:2: warning: 'attr_remove' is deprecated: Use removexattr or lremovexattr instead [-Wdeprecated-declarations] 2023-12-01T01:19:25.256 INFO:tasks.workunit.client.0.smithi077.stderr: rval = attr_remove(name->path, attrname, flags); 2023-12-01T01:19:25.257 INFO:tasks.workunit.client.0.smithi077.stderr: ^~~~ 2023-12-01T01:19:25.257 INFO:tasks.workunit.client.0.smithi077.stderr:In file included from ../src/global.h:25, 2023-12-01T01:19:25.257 INFO:tasks.workunit.client.0.smithi077.stderr: from fsstress.c:12: 2023-12-01T01:19:25.257 INFO:tasks.workunit.client.0.smithi077.stderr:/usr/include/attr/attributes.h:152:12: note: declared here 2023-12-01T01:19:25.257 INFO:tasks.workunit.client.0.smithi077.stderr: extern int attr_remove (const char *__path, const char *__attrname, 2023-12-01T01:19:25.257 INFO:tasks.workunit.client.0.smithi077.stderr: ^~~~~~~~~~~ 2023-12-01T01:19:25.257 INFO:tasks.workunit.client.0.smithi077.stderr:fsstress.c: In function 'attr_set_path': 2023-12-01T01:19:25.257 INFO:tasks.workunit.client.0.smithi077.stderr:fsstress.c:898:2: warning: 'attr_set' is deprecated: Use setxattr or lsetxattr instead [-Wdeprecated-declarations] 2023-12-01T01:19:25.257 INFO:tasks.workunit.client.0.smithi077.stderr: rval = attr_set(name->path, attrname, attrvalue, valuelength, flags); 2023-12-01T01:19:25.257 INFO:tasks.workunit.client.0.smithi077.stderr: ^~~~ 2023-12-01T01:19:25.257 INFO:tasks.workunit.client.0.smithi077.stderr:In file included from ../src/global.h:25, 2023-12-01T01:19:25.257 INFO:tasks.workunit.client.0.smithi077.stderr: from fsstress.c:12: 2023-12-01T01:19:25.257 INFO:tasks.workunit.client.0.smithi077.stderr:/usr/include/attr/attributes.h:139:12: note: declared here 2023-12-01T01:19:25.257 INFO:tasks.workunit.client.0.smithi077.stderr: extern int attr_set (const char *__path, const char *__attrname, 2023-12-01T01:19:25.258 INFO:tasks.workunit.client.0.smithi077.stderr: ^~~~~~~~ 2023-12-01T01:19:25.834 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] growfiles 2023-12-01T01:19:25.895 INFO:tasks.workunit.client.0.smithi077.stderr:fsx.c: In function 'main': 2023-12-01T01:19:25.896 INFO:tasks.workunit.client.0.smithi077.stderr:fsx.c:2662:5: warning: 'strncpy' specified bound 4096 equals destination size [-Wstringop-truncation] 2023-12-01T01:19:25.896 INFO:tasks.workunit.client.0.smithi077.stderr: strncpy(opsfile, optarg, sizeof(opsfile)); 2023-12-01T01:19:25.896 INFO:tasks.workunit.client.0.smithi077.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-12-01T01:19:25.896 INFO:tasks.workunit.client.0.smithi077.stderr:fsx.c:2632:4: warning: 'strncpy' specified bound 1024 equals destination size [-Wstringop-truncation] 2023-12-01T01:19:25.896 INFO:tasks.workunit.client.0.smithi077.stderr: strncpy(dname, optarg, sizeof(dname)); 2023-12-01T01:19:25.896 INFO:tasks.workunit.client.0.smithi077.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-12-01T01:19:26.051 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] iogen 2023-12-01T01:19:26.233 INFO:tasks.workunit.client.0.smithi077.stdout: [DEP] 2023-12-01T01:19:26.273 INFO:tasks.workunit.client.0.smithi077.stdout: [DEP] 2023-12-01T01:19:26.289 INFO:tasks.workunit.client.0.smithi077.stdout: [DEP] 2023-12-01T01:19:26.469 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] aio-stress 2023-12-01T01:19:26.545 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] dirstress 2023-12-01T01:19:26.548 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] fill 2023-12-01T01:19:26.579 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] fill2 2023-12-01T01:19:26.770 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] getpagesize 2023-12-01T01:19:26.876 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] holes 2023-12-01T01:19:26.897 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] lstat64 2023-12-01T01:19:26.966 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] nametest 2023-12-01T01:19:27.018 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] permname 2023-12-01T01:19:27.128 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] randholes 2023-12-01T01:19:27.149 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] runas 2023-12-01T01:19:27.273 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] truncfile 2023-12-01T01:19:27.283 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] usemem 2023-12-01T01:19:27.370 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] mmapcat 2023-12-01T01:19:27.469 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] append_reader 2023-12-01T01:19:27.469 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] append_writer 2023-12-01T01:19:27.507 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] dirperf 2023-12-01T01:19:27.569 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] metaperf 2023-12-01T01:19:27.676 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] devzero 2023-12-01T01:19:27.712 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] feature 2023-12-01T01:19:27.823 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] alloc 2023-12-01T01:19:27.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:27 smithi077 ceph-mon[98867]: pgmap v345: 97 pgs: 97 active+clean; 35 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 1.3 MiB/s wr, 221 op/s 2023-12-01T01:19:27.920 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] fault 2023-12-01T01:19:27.949 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] fstest 2023-12-01T01:19:27.968 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_access_root 2023-12-01T01:19:28.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:27 smithi203 ceph-mon[103581]: pgmap v345: 97 pgs: 97 active+clean; 35 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 1.3 MiB/s wr, 221 op/s 2023-12-01T01:19:28.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:27 smithi137 ceph-mon[103312]: pgmap v345: 97 pgs: 97 active+clean; 35 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 1.3 MiB/s wr, 221 op/s 2023-12-01T01:19:28.106 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] godown 2023-12-01T01:19:28.130 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] resvtest 2023-12-01T01:19:28.161 INFO:tasks.workunit.client.0.smithi077.stderr:fstest.c: In function 'run_child': 2023-12-01T01:19:28.161 INFO:tasks.workunit.client.0.smithi077.stderr:fstest.c:95:20: warning: '/file' directive writing 5 bytes into a region of size between 1 and 1024 [-Wformat-overflow=] 2023-12-01T01:19:28.161 INFO:tasks.workunit.client.0.smithi077.stderr: sprintf(fname, "%s/file%d", dir, fnum); 2023-12-01T01:19:28.161 INFO:tasks.workunit.client.0.smithi077.stderr: ^~~~~ 2023-12-01T01:19:28.161 INFO:tasks.workunit.client.0.smithi077.stderr:fstest.c:95:17: note: directive argument in the range [0, 2147483647] 2023-12-01T01:19:28.161 INFO:tasks.workunit.client.0.smithi077.stderr: sprintf(fname, "%s/file%d", dir, fnum); 2023-12-01T01:19:28.161 INFO:tasks.workunit.client.0.smithi077.stderr: ^~~~~~~~~~~ 2023-12-01T01:19:28.161 INFO:tasks.workunit.client.0.smithi077.stderr:fstest.c:95:2: note: 'sprintf' output between 7 and 1039 bytes into a destination of size 1024 2023-12-01T01:19:28.162 INFO:tasks.workunit.client.0.smithi077.stderr: sprintf(fname, "%s/file%d", dir, fnum); 2023-12-01T01:19:28.162 INFO:tasks.workunit.client.0.smithi077.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-12-01T01:19:28.162 INFO:tasks.workunit.client.0.smithi077.stderr:fstest.c:166:20: warning: '/file' directive writing 5 bytes into a region of size between 1 and 1024 [-Wformat-overflow=] 2023-12-01T01:19:28.162 INFO:tasks.workunit.client.0.smithi077.stderr: sprintf(fname, "%s/file%d", dir, fnum); 2023-12-01T01:19:28.162 INFO:tasks.workunit.client.0.smithi077.stderr: ^~~~~ 2023-12-01T01:19:28.162 INFO:tasks.workunit.client.0.smithi077.stderr:fstest.c:166:17: note: directive argument in the range [0, 2147483647] 2023-12-01T01:19:28.162 INFO:tasks.workunit.client.0.smithi077.stderr: sprintf(fname, "%s/file%d", dir, fnum); 2023-12-01T01:19:28.162 INFO:tasks.workunit.client.0.smithi077.stderr: ^~~~~~~~~~~ 2023-12-01T01:19:28.163 INFO:tasks.workunit.client.0.smithi077.stderr:fstest.c:166:2: note: 'sprintf' output between 7 and 1039 bytes into a destination of size 1024 2023-12-01T01:19:28.163 INFO:tasks.workunit.client.0.smithi077.stderr: sprintf(fname, "%s/file%d", dir, fnum); 2023-12-01T01:19:28.163 INFO:tasks.workunit.client.0.smithi077.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-12-01T01:19:28.181 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] writemod 2023-12-01T01:19:28.248 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] writev_on_pagefault 2023-12-01T01:19:28.350 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] makeextents 2023-12-01T01:19:28.374 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] itrash 2023-12-01T01:19:28.377 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] rename 2023-12-01T01:19:28.469 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] multi_open_unlink 2023-12-01T01:19:28.561 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] dmiperf 2023-12-01T01:19:28.563 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] unwritten_sync 2023-12-01T01:19:28.582 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-01T01:19:28.631 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] genhashnames 2023-12-01T01:19:28.648 INFO:tasks.workunit.client.0.smithi077.stderr:multi_open_unlink.c: In function 'main': 2023-12-01T01:19:28.648 INFO:tasks.workunit.client.0.smithi077.stderr:multi_open_unlink.c:101:4: warning: 'attr_set' is deprecated: Use setxattr or lsetxattr instead [-Wdeprecated-declarations] 2023-12-01T01:19:28.648 INFO:tasks.workunit.client.0.smithi077.stderr: sts = attr_set(path, attrname, attrvalue, value_size, flags); 2023-12-01T01:19:28.648 INFO:tasks.workunit.client.0.smithi077.stderr: ^~~ 2023-12-01T01:19:28.648 INFO:tasks.workunit.client.0.smithi077.stderr:In file included from multi_open_unlink.c:15: 2023-12-01T01:19:28.648 INFO:tasks.workunit.client.0.smithi077.stderr:/usr/include/attr/attributes.h:139:12: note: declared here 2023-12-01T01:19:28.648 INFO:tasks.workunit.client.0.smithi077.stderr: extern int attr_set (const char *__path, const char *__attrname, 2023-12-01T01:19:28.648 INFO:tasks.workunit.client.0.smithi077.stderr: ^~~~~~~~ 2023-12-01T01:19:28.708 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_holes 2023-12-01T01:19:28.753 INFO:tasks.workunit.client.0.smithi077.stderr:dmiperf.c: In function 'mkfile': 2023-12-01T01:19:28.753 INFO:tasks.workunit.client.0.smithi077.stderr:dmiperf.c:192:2: warning: 'attr_setf' is deprecated: Use fsetxattr instead [-Wdeprecated-declarations] 2023-12-01T01:19:28.753 INFO:tasks.workunit.client.0.smithi077.stderr: if (attr_setf(fd, DMFATTRNAME, attr, DMFATTRLEN, ATTR_ROOT) < 0) { 2023-12-01T01:19:28.753 INFO:tasks.workunit.client.0.smithi077.stderr: ^~ 2023-12-01T01:19:28.754 INFO:tasks.workunit.client.0.smithi077.stderr:In file included from dmiperf.c:20: 2023-12-01T01:19:28.754 INFO:tasks.workunit.client.0.smithi077.stderr:/usr/include/attr/attributes.h:143:12: note: declared here 2023-12-01T01:19:28.754 INFO:tasks.workunit.client.0.smithi077.stderr: extern int attr_setf (int __fd, const char *__attrname, 2023-12-01T01:19:28.754 INFO:tasks.workunit.client.0.smithi077.stderr: ^~~~~~~~~ 2023-12-01T01:19:28.760 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.759+0000 7f2f2f00e700 1 -- 172.21.15.77:0/648878917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f28100ee0 msgr2=0x7f2f28101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:28.760 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.759+0000 7f2f2f00e700 1 --2- 172.21.15.77:0/648878917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f28100ee0 0x7f2f28101340 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c0099c0 tx=0x7f2f1c009cd0 comp rx=0 tx=0).stop 2023-12-01T01:19:28.761 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.760+0000 7f2f2f00e700 1 -- 172.21.15.77:0/648878917 shutdown_connections 2023-12-01T01:19:28.761 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.760+0000 7f2f2f00e700 1 --2- 172.21.15.77:0/648878917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f28101880 0x7f2f28108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:28.762 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.760+0000 7f2f2f00e700 1 --2- 172.21.15.77:0/648878917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f280ffcf0 0x7f2f28100110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:28.762 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.760+0000 7f2f2f00e700 1 --2- 172.21.15.77:0/648878917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f28100ee0 0x7f2f28101340 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:28.762 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.760+0000 7f2f2f00e700 1 -- 172.21.15.77:0/648878917 >> 172.21.15.77:0/648878917 conn(0x7f2f280fb260 msgr2=0x7f2f280fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:19:28.763 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.762+0000 7f2f2f00e700 1 -- 172.21.15.77:0/648878917 shutdown_connections 2023-12-01T01:19:28.763 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.762+0000 7f2f2f00e700 1 -- 172.21.15.77:0/648878917 wait complete. 2023-12-01T01:19:28.764 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.762+0000 7f2f2f00e700 1 Processor -- start 2023-12-01T01:19:28.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.762+0000 7f2f2f00e700 1 -- start start 2023-12-01T01:19:28.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.763+0000 7f2f2f00e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f280ffcf0 0x7f2f28069c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:28.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.763+0000 7f2f2f00e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f28100ee0 0x7f2f2819b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:28.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.763+0000 7f2f2f00e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f28101880 0x7f2f2819de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:28.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.763+0000 7f2f2f00e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2819e5f0 con 0x7f2f28101880 2023-12-01T01:19:28.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.763+0000 7f2f2f00e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2819e760 con 0x7f2f280ffcf0 2023-12-01T01:19:28.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.763+0000 7f2f2f00e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2819e920 con 0x7f2f28100ee0 2023-12-01T01:19:28.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.763+0000 7f2f2cdaa700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f280ffcf0 0x7f2f28069c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:28.766 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.763+0000 7f2f2cdaa700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f280ffcf0 0x7f2f28069c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:39278/0 (socket says 172.21.15.77:39278) 2023-12-01T01:19:28.766 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.763+0000 7f2f2cdaa700 1 -- 172.21.15.77:0/2923270843 learned_addr learned my addr 172.21.15.77:0/2923270843 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:19:28.766 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.763+0000 7f2f27fff700 1 --2- 172.21.15.77:0/2923270843 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f28100ee0 0x7f2f2819b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:28.766 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.763+0000 7f2f2cdaa700 1 -- 172.21.15.77:0/2923270843 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f28100ee0 msgr2=0x7f2f2819b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:28.766 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.763+0000 7f2f2cdaa700 1 --2- 172.21.15.77:0/2923270843 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f28100ee0 0x7f2f2819b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:28.766 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.763+0000 7f2f2cdaa700 1 -- 172.21.15.77:0/2923270843 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f28101880 msgr2=0x7f2f2819de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:28.766 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.763+0000 7f2f2cdaa700 1 --2- 172.21.15.77:0/2923270843 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f28101880 0x7f2f2819de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:28.766 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.763+0000 7f2f2cdaa700 1 -- 172.21.15.77:0/2923270843 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f14009520 con 0x7f2f280ffcf0 2023-12-01T01:19:28.767 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.763+0000 7f2f27fff700 1 --2- 172.21.15.77:0/2923270843 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f28100ee0 0x7f2f2819b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:19:28.767 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.763+0000 7f2f2cdaa700 1 --2- 172.21.15.77:0/2923270843 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f280ffcf0 0x7f2f28069c30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2f1400ea50 tx=0x7f2f1400ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:19:28.767 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.764+0000 7f2f25ffb700 1 -- 172.21.15.77:0/2923270843 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1400cbd0 con 0x7f2f280ffcf0 2023-12-01T01:19:28.767 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.764+0000 7f2f25ffb700 1 -- 172.21.15.77:0/2923270843 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f1400cd30 con 0x7f2f280ffcf0 2023-12-01T01:19:28.767 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.764+0000 7f2f25ffb700 1 -- 172.21.15.77:0/2923270843 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f140186c0 con 0x7f2f280ffcf0 2023-12-01T01:19:28.767 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.765+0000 7f2f2f00e700 1 -- 172.21.15.77:0/2923270843 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f1c0095f0 con 0x7f2f280ffcf0 2023-12-01T01:19:28.767 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.765+0000 7f2f2f00e700 1 -- 172.21.15.77:0/2923270843 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f2819ef00 con 0x7f2f280ffcf0 2023-12-01T01:19:28.768 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.767+0000 7f2f25ffb700 1 -- 172.21.15.77:0/2923270843 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f2f14018820 con 0x7f2f280ffcf0 2023-12-01T01:19:28.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.768+0000 7f2f25ffb700 1 --2- 172.21.15.77:0/2923270843 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f2f180a05a0 0x7f2f180a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:28.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.768+0000 7f2f27fff700 1 --2- 172.21.15.77:0/2923270843 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f2f180a05a0 0x7f2f180a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:28.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.768+0000 7f2f25ffb700 1 -- 172.21.15.77:0/2923270843 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f2f14014070 con 0x7f2f280ffcf0 2023-12-01T01:19:28.769 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.769+0000 7f2f27fff700 1 --2- 172.21.15.77:0/2923270843 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f2f180a05a0 0x7f2f180a2a60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c00b580 tx=0x7f2f1c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:19:28.771 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.770+0000 7f2f3b19eb80 1 Processor -- start 2023-12-01T01:19:28.771 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.770+0000 7f2f3b19eb80 1 -- start start 2023-12-01T01:19:28.772 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.770+0000 7f2f3b19eb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b3a949500 0x564b3a949920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:28.772 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.770+0000 7f2f3b19eb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b3a949e60 0x564b3a948270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:28.772 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.770+0000 7f2f3b19eb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a948800 0x564b3a948c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:28.772 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.770+0000 7f2f3b19eb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b3a89ba10 con 0x564b3a948800 2023-12-01T01:19:28.772 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.770+0000 7f2f3b19eb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b3a8993d0 con 0x564b3a949e60 2023-12-01T01:19:28.772 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.770+0000 7f2f3b19eb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b3a894cc0 con 0x564b3a949500 2023-12-01T01:19:28.772 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.771+0000 7f2f2cdaa700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b3a949500 0x564b3a949920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:28.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.771+0000 7f2f2cdaa700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b3a949500 0x564b3a949920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:46552/0 (socket says 172.21.15.77:46552) 2023-12-01T01:19:28.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.771+0000 7f2f2cdaa700 1 -- 172.21.15.77:0/69855609 learned_addr learned my addr 172.21.15.77:0/69855609 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:19:28.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.771+0000 7f2f27fff700 1 --2- 172.21.15.77:0/69855609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b3a949e60 0x564b3a948270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:28.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.771+0000 7f2f2cdaa700 1 -- 172.21.15.77:0/69855609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b3a949e60 msgr2=0x564b3a948270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:28.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.771+0000 7f2f2cdaa700 1 --2- 172.21.15.77:0/69855609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b3a949e60 0x564b3a948270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:28.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.771+0000 7f2f2cdaa700 1 -- 172.21.15.77:0/69855609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a948800 msgr2=0x564b3a948c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-01T01:19:28.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.771+0000 7f2f2cdaa700 1 --2- 172.21.15.77:0/69855609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a948800 0x564b3a948c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:28.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.771+0000 7f2f2cdaa700 1 -- 172.21.15.77:0/69855609 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b3a6c2de0 con 0x564b3a949500 2023-12-01T01:19:28.773 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.771+0000 7f2f2cdaa700 1 --2- 172.21.15.77:0/69855609 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b3a949500 0x564b3a949920 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2f14096280 tx=0x7f2f14099420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61d82b91e6c9b92a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:19:28.774 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.772+0000 7f2f12ffd700 1 -- 172.21.15.77:0/69855609 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f14099f10 con 0x564b3a949500 2023-12-01T01:19:28.774 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.772+0000 7f2f12ffd700 1 -- 172.21.15.77:0/69855609 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f14099810 con 0x564b3a949500 2023-12-01T01:19:28.774 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.772+0000 7f2f12ffd700 1 -- 172.21.15.77:0/69855609 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f140a13f0 con 0x564b3a949500 2023-12-01T01:19:28.776 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.774+0000 7f2f3b19eb80 1 -- 172.21.15.77:0/69855609 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b3a949500 msgr2=0x564b3a949920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:28.776 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.774+0000 7f2f3b19eb80 1 --2- 172.21.15.77:0/69855609 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b3a949500 0x564b3a949920 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2f14096280 tx=0x7f2f14099420 comp rx=0 tx=0).stop 2023-12-01T01:19:28.776 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.775+0000 7f2f3b19eb80 1 -- 172.21.15.77:0/69855609 shutdown_connections 2023-12-01T01:19:28.776 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.775+0000 7f2f3b19eb80 1 --2- 172.21.15.77:0/69855609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a948800 0x564b3a948c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:28.776 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.775+0000 7f2f3b19eb80 1 --2- 172.21.15.77:0/69855609 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b3a949500 0x564b3a949920 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:28.776 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.775+0000 7f2f3b19eb80 1 --2- 172.21.15.77:0/69855609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b3a949e60 0x564b3a948270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:28.776 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.775+0000 7f2f3b19eb80 1 -- 172.21.15.77:0/69855609 >> 172.21.15.77:0/69855609 conn(0x564b3a8a30d0 msgr2=0x564b3a93e950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:19:28.776 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.775+0000 7f2f3b19eb80 1 -- 172.21.15.77:0/69855609 shutdown_connections 2023-12-01T01:19:28.776 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.775+0000 7f2f3b19eb80 1 -- 172.21.15.77:0/69855609 wait complete. 2023-12-01T01:19:28.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.775+0000 7f2f3b19eb80 1 -- start start 2023-12-01T01:19:28.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.775+0000 7f2f3b19eb80 1 Processor -- start 2023-12-01T01:19:28.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.776+0000 7f2f3b19eb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b3a948800 0x564b3a945be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:28.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.776+0000 7f2f3b19eb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a949500 0x564b3a946120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:28.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.776+0000 7f2f3b19eb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b3a949e60 0x564b3a947130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:28.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.776+0000 7f2f3b19eb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b3a831510 con 0x564b3a949500 2023-12-01T01:19:28.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.776+0000 7f2f3b19eb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b3a8311b0 con 0x564b3a949e60 2023-12-01T01:19:28.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.776+0000 7f2f3b19eb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b3a718920 con 0x564b3a948800 2023-12-01T01:19:28.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.776+0000 7f2f3b19eb80 10 client.0 _ll_register_callbacks cb 0x564b3a77ea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:19:28.779 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.776+0000 7f2f27fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a949500 0x564b3a946120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:28.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.776+0000 7f2f27fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a949500 0x564b3a946120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60370/0 (socket says 172.21.15.77:60370) 2023-12-01T01:19:28.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.776+0000 7f2f27fff700 1 -- 172.21.15.77:0/407957350 learned_addr learned my addr 172.21.15.77:0/407957350 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:19:28.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.776+0000 7f2f2cdaa700 1 --2- 172.21.15.77:0/407957350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b3a948800 0x564b3a945be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:28.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.777+0000 7f2f2cdaa700 1 -- 172.21.15.77:0/407957350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b3a949e60 msgr2=0x564b3a947130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:28.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.777+0000 7f2f2cdaa700 1 --2- 172.21.15.77:0/407957350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b3a949e60 0x564b3a947130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:28.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.777+0000 7f2f2cdaa700 1 -- 172.21.15.77:0/407957350 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a949500 msgr2=0x564b3a946120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:28.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.777+0000 7f2f2cdaa700 1 --2- 172.21.15.77:0/407957350 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a949500 0x564b3a946120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:28.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.777+0000 7f2f2cdaa700 1 -- 172.21.15.77:0/407957350 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f2819ef00 con 0x564b3a948800 2023-12-01T01:19:28.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.777+0000 7f2f27fff700 1 --2- 172.21.15.77:0/407957350 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a949500 0x564b3a946120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:19:28.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.777+0000 7f2f2cdaa700 1 --2- 172.21.15.77:0/407957350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b3a948800 0x564b3a945be0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2f14005950 tx=0x7f2f14095430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50e29a84d6941303 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:19:28.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.777+0000 7f2f07fff700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2023-12-01T01:19:28.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.777+0000 7f2f07fff700 1 -- 172.21.15.77:0/407957350 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f140a1880 con 0x564b3a948800 2023-12-01T01:19:28.780 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.777+0000 7f2f3b19eb80 1 -- 172.21.15.77:0/407957350 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564b3a8bfd10 con 0x564b3a948800 2023-12-01T01:19:28.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.777+0000 7f2f07fff700 1 -- 172.21.15.77:0/407957350 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f2f1409d170 con 0x564b3a948800 2023-12-01T01:19:28.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.777+0000 7f2f07fff700 1 -- 172.21.15.77:0/407957350 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f1409d330 con 0x564b3a948800 2023-12-01T01:19:28.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.777+0000 7f2f3b19eb80 10 client.25504 fetch_fsmap learned FSMap version 31 2023-12-01T01:19:28.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.777+0000 7f2f3b19eb80 1 -- 172.21.15.77:0/407957350 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x564b3a6abc60 con 0x564b3a948800 2023-12-01T01:19:28.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.778+0000 7f2f07fff700 1 -- 172.21.15.77:0/407957350 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564b3a8bfd10 con 0x564b3a948800 2023-12-01T01:19:28.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.778+0000 7f2f07fff700 1 -- 172.21.15.77:0/407957350 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2f140987e0 con 0x564b3a948800 2023-12-01T01:19:28.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.778+0000 7f2f3b19eb80 10 client.25504 fetch_fsmap finished waiting for FSMap version 31 2023-12-01T01:19:28.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.778+0000 7f2f3b19eb80 10 client.25504 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-01T01:19:28.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.778+0000 7f2f3b19eb80 20 client.25504 populate_metadata read hostname 'smithi077' 2023-12-01T01:19:28.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.778+0000 7f2f3b19eb80 1 --2- 172.21.15.77:0/407957350 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x564b3a972d00 0x564b3a975120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:28.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.778+0000 7f2f3b19eb80 4 client.25504 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:19:28.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.778+0000 7f2f3b19eb80 1 -- 172.21.15.77:0/407957350 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564b3a912e10 con 0x564b3a972d00 2023-12-01T01:19:28.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.778+0000 7f2f27fff700 1 --2- 172.21.15.77:0/407957350 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x564b3a972d00 0x564b3a975120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:28.782 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.778+0000 7f2f07fff700 10 client.25504 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:19:28.782 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.778+0000 7f2f27fff700 1 --2- 172.21.15.77:0/407957350 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x564b3a972d00 0x564b3a975120 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4b1167659b1e84a server_cookie=7c8c9cfc74fc5e7d in_seq=0 out_seq=0 2023-12-01T01:19:28.783 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.779+0000 7f2f07fff700 1 -- 172.21.15.77:0/407957350 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x564b3a912e10 con 0x564b3a972d00 2023-12-01T01:19:28.783 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.779+0000 7f2f07fff700 10 client.25504 handle_command_reply: tid=0 2023-12-01T01:19:28.783 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.779+0000 7f2f3b19eb80 1 client.25504 shutdown 2023-12-01T01:19:28.783 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.781+0000 7f2f3b19eb80 1 -- 172.21.15.77:0/407957350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b3a948800 msgr2=0x564b3a945be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:28.783 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.781+0000 7f2f3b19eb80 1 --2- 172.21.15.77:0/407957350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b3a948800 0x564b3a945be0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2f14005950 tx=0x7f2f14095430 comp rx=0 tx=0).stop 2023-12-01T01:19:28.783 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.781+0000 7f2f3b19eb80 1 -- 172.21.15.77:0/407957350 shutdown_connections 2023-12-01T01:19:28.783 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.781+0000 7f2f3b19eb80 1 --2- 172.21.15.77:0/407957350 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x564b3a972d00 0x564b3a975120 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:28.783 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.781+0000 7f2f3b19eb80 1 --2- 172.21.15.77:0/407957350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b3a949e60 0x564b3a947130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:28.783 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.781+0000 7f2f3b19eb80 1 --2- 172.21.15.77:0/407957350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b3a948800 0x564b3a945be0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:28.783 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.781+0000 7f2f07fff700 0 client.25504 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:19:28.783 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.781+0000 7f2f3b19eb80 1 --2- 172.21.15.77:0/407957350 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a949500 0x564b3a946120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:28.783 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.781+0000 7f2f3b19eb80 1 -- 172.21.15.77:0/407957350 >> 172.21.15.77:0/407957350 conn(0x564b3a8a30d0 msgr2=0x564b3a93e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:19:28.783 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.781+0000 7f2f3b19eb80 1 -- 172.21.15.77:0/407957350 shutdown_connections 2023-12-01T01:19:28.784 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.782+0000 7f2f3b19eb80 1 -- 172.21.15.77:0/407957350 wait complete. 2023-12-01T01:19:28.785 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.782+0000 7f2f3b19eb80 20 client.25504 trim_cache size 0 max 16384 2023-12-01T01:19:28.807 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.804+0000 7f2f3b19eb80 1 Processor -- start 2023-12-01T01:19:28.807 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.804+0000 7f2f3b19eb80 1 -- start start 2023-12-01T01:19:28.807 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.804+0000 7f2f3b19eb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b3a948800 0x564b3a946220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:28.807 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.804+0000 7f2f3b19eb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b3a949500 0x564b3a945290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:28.807 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.804+0000 7f2f3b19eb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a949e60 0x564b3a94fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:28.807 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.804+0000 7f2f3b19eb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b3a93f3c0 con 0x564b3a949e60 2023-12-01T01:19:28.807 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.804+0000 7f2f3b19eb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b3a912e40 con 0x564b3a949500 2023-12-01T01:19:28.807 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.804+0000 7f2f3b19eb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b3a943d50 con 0x564b3a948800 2023-12-01T01:19:28.807 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.805+0000 7f2f27fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b3a949500 0x564b3a945290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:28.807 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.805+0000 7f2f27fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b3a949500 0x564b3a945290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:39298/0 (socket says 172.21.15.77:39298) 2023-12-01T01:19:28.807 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.805+0000 7f2f27fff700 1 -- 172.21.15.77:0/189266421 learned_addr learned my addr 172.21.15.77:0/189266421 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:19:28.808 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.805+0000 7f2f2d5ab700 1 --2- 172.21.15.77:0/189266421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a949e60 0x564b3a94fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:28.808 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.805+0000 7f2f27fff700 1 -- 172.21.15.77:0/189266421 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b3a948800 msgr2=0x564b3a946220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-01T01:19:28.808 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.805+0000 7f2f27fff700 1 --2- 172.21.15.77:0/189266421 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b3a948800 0x564b3a946220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:28.808 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.805+0000 7f2f27fff700 1 -- 172.21.15.77:0/189266421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a949e60 msgr2=0x564b3a94fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:28.808 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.805+0000 7f2f27fff700 1 --2- 172.21.15.77:0/189266421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a949e60 0x564b3a94fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:28.808 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.805+0000 7f2f27fff700 1 -- 172.21.15.77:0/189266421 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b3a6e9250 con 0x564b3a949500 2023-12-01T01:19:28.808 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.805+0000 7f2f27fff700 1 --2- 172.21.15.77:0/189266421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b3a949500 0x564b3a945290 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c009990 tx=0x7f2f1c020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bed4ac15fd9f4ea server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:19:28.808 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.805+0000 7f2f137fe700 1 -- 172.21.15.77:0/189266421 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c004d30 con 0x564b3a949500 2023-12-01T01:19:28.808 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.805+0000 7f2f137fe700 1 -- 172.21.15.77:0/189266421 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f1c01e040 con 0x564b3a949500 2023-12-01T01:19:28.808 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.805+0000 7f2f137fe700 1 -- 172.21.15.77:0/189266421 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c00fb80 con 0x564b3a949500 2023-12-01T01:19:28.812 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.806+0000 7f2f3b19eb80 1 -- 172.21.15.77:0/189266421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b3a949500 msgr2=0x564b3a945290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:28.812 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.806+0000 7f2f3b19eb80 1 --2- 172.21.15.77:0/189266421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b3a949500 0x564b3a945290 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c009990 tx=0x7f2f1c020000 comp rx=0 tx=0).stop 2023-12-01T01:19:28.812 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.806+0000 7f2f3b19eb80 1 -- 172.21.15.77:0/189266421 shutdown_connections 2023-12-01T01:19:28.813 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.806+0000 7f2f3b19eb80 1 --2- 172.21.15.77:0/189266421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a949e60 0x564b3a94fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:28.813 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.806+0000 7f2f3b19eb80 1 --2- 172.21.15.77:0/189266421 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b3a948800 0x564b3a946220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:28.813 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.806+0000 7f2f3b19eb80 1 --2- 172.21.15.77:0/189266421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b3a949500 0x564b3a945290 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:28.813 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.806+0000 7f2f3b19eb80 1 -- 172.21.15.77:0/189266421 >> 172.21.15.77:0/189266421 conn(0x564b3a8a30d0 msgr2=0x564b3a948be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:19:28.813 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.807+0000 7f2f3b19eb80 1 -- 172.21.15.77:0/189266421 shutdown_connections 2023-12-01T01:19:28.813 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.807+0000 7f2f3b19eb80 1 -- 172.21.15.77:0/189266421 wait complete. 2023-12-01T01:19:28.813 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.807+0000 7f2f3b19eb80 1 -- start start 2023-12-01T01:19:28.813 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.807+0000 7f2f3b19eb80 1 Processor -- start 2023-12-01T01:19:28.813 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.807+0000 7f2f3b19eb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b3a949500 0x564b3a950620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:28.813 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.807+0000 7f2f3b19eb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a949e60 0x564b3a950b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:28.814 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.807+0000 7f2f3b19eb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b3a955530 0x564b3a9579c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:28.814 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.807+0000 7f2f3b19eb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b3a957f00 con 0x564b3a949e60 2023-12-01T01:19:28.814 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.807+0000 7f2f3b19eb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b3a958070 con 0x564b3a955530 2023-12-01T01:19:28.814 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.807+0000 7f2f3b19eb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b3a958230 con 0x564b3a949500 2023-12-01T01:19:28.814 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.807+0000 7f2f3b19eb80 10 client.0 _ll_register_callbacks cb 0x7f2f1c00bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:19:28.814 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.808+0000 7f2f2cdaa700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b3a949500 0x564b3a950620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:28.814 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.808+0000 7f2f27fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a949e60 0x564b3a950b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:28.814 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.808+0000 7f2f2cdaa700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b3a949500 0x564b3a950620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:46568/0 (socket says 172.21.15.77:46568) 2023-12-01T01:19:28.814 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.808+0000 7f2f27fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a949e60 0x564b3a950b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60382/0 (socket says 172.21.15.77:60382) 2023-12-01T01:19:28.814 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.808+0000 7f2f2cdaa700 1 -- 172.21.15.77:0/949040233 learned_addr learned my addr 172.21.15.77:0/949040233 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:19:28.814 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.808+0000 7f2f2d5ab700 1 --2- 172.21.15.77:0/949040233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b3a955530 0x564b3a9579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:28.814 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.808+0000 7f2f2cdaa700 1 -- 172.21.15.77:0/949040233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b3a955530 msgr2=0x564b3a9579c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:28.814 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.808+0000 7f2f2cdaa700 1 --2- 172.21.15.77:0/949040233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b3a955530 0x564b3a9579c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:28.814 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.808+0000 7f2f2cdaa700 1 -- 172.21.15.77:0/949040233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a949e60 msgr2=0x564b3a950b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:28.814 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.808+0000 7f2f2cdaa700 1 --2- 172.21.15.77:0/949040233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a949e60 0x564b3a950b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:28.815 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.808+0000 7f2f2cdaa700 1 -- 172.21.15.77:0/949040233 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f28108a80 con 0x564b3a949500 2023-12-01T01:19:28.815 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.808+0000 7f2f27fff700 1 --2- 172.21.15.77:0/949040233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a949e60 0x564b3a950b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:19:28.815 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.808+0000 7f2f2d5ab700 1 --2- 172.21.15.77:0/949040233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b3a955530 0x564b3a9579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:19:28.815 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.808+0000 7f2f2cdaa700 1 --2- 172.21.15.77:0/949040233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b3a949500 0x564b3a950620 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2f1400c5b0 tx=0x7f2f14097c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd7f961e50b67402 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:19:28.815 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.808+0000 7f2f07fff700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2023-12-01T01:19:28.815 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.809+0000 7f2f07fff700 1 -- 172.21.15.77:0/949040233 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1407ae40 con 0x564b3a949500 2023-12-01T01:19:28.815 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.809+0000 7f2f3b19eb80 1 -- 172.21.15.77:0/949040233 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2f14099810 con 0x564b3a949500 2023-12-01T01:19:28.815 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.809+0000 7f2f07fff700 1 -- 172.21.15.77:0/949040233 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f2f140a1880 con 0x564b3a949500 2023-12-01T01:19:28.815 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.809+0000 7f2f07fff700 1 -- 172.21.15.77:0/949040233 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f1409d640 con 0x564b3a949500 2023-12-01T01:19:28.815 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.809+0000 7f2f3b19eb80 10 client.25507 fetch_fsmap learned FSMap version 31 2023-12-01T01:19:28.815 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.809+0000 7f2f3b19eb80 1 -- 172.21.15.77:0/949040233 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x564b3a6c12b0 con 0x564b3a949500 2023-12-01T01:19:28.815 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.809+0000 7f2f07fff700 1 -- 172.21.15.77:0/949040233 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f14099810 con 0x564b3a949500 2023-12-01T01:19:28.815 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.809+0000 7f2f07fff700 1 -- 172.21.15.77:0/949040233 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2f1409e580 con 0x564b3a949500 2023-12-01T01:19:28.815 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.809+0000 7f2f3b19eb80 10 client.25507 fetch_fsmap finished waiting for FSMap version 31 2023-12-01T01:19:28.815 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.809+0000 7f2f3b19eb80 10 client.25507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-01T01:19:28.816 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.809+0000 7f2f3b19eb80 20 client.25507 populate_metadata read hostname 'smithi077' 2023-12-01T01:19:28.816 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.809+0000 7f2f3b19eb80 1 --2- 172.21.15.77:0/949040233 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x564b3a9bd070 0x564b3a9bf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:28.816 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.809+0000 7f2f3b19eb80 4 client.25507 mds_command: new command op to 24500 tid=0[{"prefix": "scrub status"}] 2023-12-01T01:19:28.816 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.809+0000 7f2f3b19eb80 1 -- 172.21.15.77:0/949040233 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564b3a88dd40 con 0x564b3a9bd070 2023-12-01T01:19:28.816 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.809+0000 7f2f27fff700 1 --2- 172.21.15.77:0/949040233 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x564b3a9bd070 0x564b3a9bf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:28.816 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.810+0000 7f2f07fff700 10 client.25507 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:19:28.816 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.810+0000 7f2f27fff700 1 --2- 172.21.15.77:0/949040233 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x564b3a9bd070 0x564b3a9bf490 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=184070e3fc65a86f server_cookie=4511971cf76ad2d4 in_seq=0 out_seq=0 2023-12-01T01:19:28.816 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.810+0000 7f2f07fff700 1 -- 172.21.15.77:0/949040233 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x564b3a88dd40 con 0x564b3a9bd070 2023-12-01T01:19:28.816 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.810+0000 7f2f07fff700 10 client.25507 handle_command_reply: tid=0 2023-12-01T01:19:28.816 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.810+0000 7f2f3b19eb80 1 client.25507 shutdown 2023-12-01T01:19:28.816 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.810+0000 7f2f3b19eb80 1 -- 172.21.15.77:0/949040233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b3a949500 msgr2=0x564b3a950620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:28.816 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.810+0000 7f2f3b19eb80 1 --2- 172.21.15.77:0/949040233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b3a949500 0x564b3a950620 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2f1400c5b0 tx=0x7f2f14097c70 comp rx=0 tx=0).stop 2023-12-01T01:19:28.816 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.811+0000 7f2f3b19eb80 1 -- 172.21.15.77:0/949040233 shutdown_connections 2023-12-01T01:19:28.816 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.811+0000 7f2f3b19eb80 1 --2- 172.21.15.77:0/949040233 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x564b3a9bd070 0x564b3a9bf490 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:28.817 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.811+0000 7f2f3b19eb80 1 --2- 172.21.15.77:0/949040233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b3a955530 0x564b3a9579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:28.817 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.811+0000 7f2f07fff700 0 client.25507 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:19:28.817 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.811+0000 7f2f3b19eb80 1 --2- 172.21.15.77:0/949040233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564b3a949500 0x564b3a950620 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:28.817 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.811+0000 7f2f3b19eb80 1 --2- 172.21.15.77:0/949040233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564b3a949e60 0x564b3a950b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:28.817 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.811+0000 7f2f3b19eb80 1 -- 172.21.15.77:0/949040233 >> 172.21.15.77:0/949040233 conn(0x564b3a8a30d0 msgr2=0x564b3a943fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:19:28.817 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.811+0000 7f2f3b19eb80 1 -- 172.21.15.77:0/949040233 shutdown_connections 2023-12-01T01:19:28.817 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.811+0000 7f2f3b19eb80 1 -- 172.21.15.77:0/949040233 wait complete. 2023-12-01T01:19:28.817 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.811+0000 7f2f3b19eb80 20 client.25507 trim_cache size 0 max 16384 2023-12-01T01:19:28.817 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2023-12-01T01:19:28.817 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "scrub active (2 inodes in the stack)", 2023-12-01T01:19:28.817 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2023-12-01T01:19:28.818 INFO:teuthology.orchestra.run.smithi077.stdout: "085290de-33b6-4527-ab7f-03f7dfd5dbba": { 2023-12-01T01:19:28.818 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2023-12-01T01:19:28.818 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "085290de-33b6-4527-ab7f-03f7dfd5dbba", 2023-12-01T01:19:28.818 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2023-12-01T01:19:28.818 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:19:28.818 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:19:28.818 INFO:teuthology.orchestra.run.smithi077.stdout:} 2023-12-01T01:19:28.819 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.811+0000 7f2f2f00e700 1 -- 172.21.15.77:0/2923270843 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f2f180a05a0 msgr2=0x7f2f180a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:28.819 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.812+0000 7f2f2f00e700 1 --2- 172.21.15.77:0/2923270843 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f2f180a05a0 0x7f2f180a2a60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c00b580 tx=0x7f2f1c005fb0 comp rx=0 tx=0).stop 2023-12-01T01:19:28.819 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.812+0000 7f2f2f00e700 1 -- 172.21.15.77:0/2923270843 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f280ffcf0 msgr2=0x7f2f28069c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:28.819 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.812+0000 7f2f2f00e700 1 --2- 172.21.15.77:0/2923270843 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f280ffcf0 0x7f2f28069c30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2f1400ea50 tx=0x7f2f1400ed60 comp rx=0 tx=0).stop 2023-12-01T01:19:28.819 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.813+0000 7f2f2f00e700 1 -- 172.21.15.77:0/2923270843 shutdown_connections 2023-12-01T01:19:28.819 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.813+0000 7f2f2f00e700 1 --2- 172.21.15.77:0/2923270843 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f2f180a05a0 0x7f2f180a2a60 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:28.819 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.813+0000 7f2f2f00e700 1 --2- 172.21.15.77:0/2923270843 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f28101880 0x7f2f2819de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:28.820 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.813+0000 7f2f2f00e700 1 --2- 172.21.15.77:0/2923270843 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f280ffcf0 0x7f2f28069c30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:28.820 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.813+0000 7f2f2f00e700 1 --2- 172.21.15.77:0/2923270843 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f28100ee0 0x7f2f2819b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:28.820 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.813+0000 7f2f2f00e700 1 -- 172.21.15.77:0/2923270843 >> 172.21.15.77:0/2923270843 conn(0x7f2f280fb260 msgr2=0x7f2f281043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:19:28.820 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.813+0000 7f2f2f00e700 1 -- 172.21.15.77:0/2923270843 shutdown_connections 2023-12-01T01:19:28.820 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:28.813+0000 7f2f2f00e700 1 -- 172.21.15.77:0/2923270843 wait complete. 2023-12-01T01:19:28.834 INFO:tasks.cephfs.filesystem:scrub status for tag:085290de-33b6-4527-ab7f-03f7dfd5dbba - {'path': '/', 'tag': '085290de-33b6-4527-ab7f-03f7dfd5dbba', 'options': 'recursive,force'} 2023-12-01T01:19:28.845 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_mmap_writev 2023-12-01T01:19:28.941 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_truncate_cmtime 2023-12-01T01:19:28.992 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] dirhash_collide 2023-12-01T01:19:28.992 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_rename_overwrite 2023-12-01T01:19:29.128 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] holetest 2023-12-01T01:19:29.245 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_truncate_self 2023-12-01T01:19:29.247 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_mmap_dio 2023-12-01T01:19:29.317 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] af_unix 2023-12-01T01:19:29.501 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_mmap_stale_pmd 2023-12-01T01:19:29.506 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_mmap_cow_race 2023-12-01T01:19:29.508 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_mmap_fallocate 2023-12-01T01:19:29.553 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] fsync-err 2023-12-01T01:19:29.780 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_mmap_write_ro 2023-12-01T01:19:29.781 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_ext4_dax_journal_corruption 2023-12-01T01:19:29.781 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_ext4_dax_inline_corruption 2023-12-01T01:19:29.858 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_ofd_locks 2023-12-01T01:19:29.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:29 smithi077 ceph-mon[98867]: pgmap v346: 97 pgs: 97 active+clean; 39 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 1.4 MiB/s wr, 263 op/s 2023-12-01T01:19:30.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:29 smithi203 ceph-mon[103581]: pgmap v346: 97 pgs: 97 active+clean; 39 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 1.4 MiB/s wr, 263 op/s 2023-12-01T01:19:30.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:29 smithi137 ceph-mon[103312]: pgmap v346: 97 pgs: 97 active+clean; 39 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 1.4 MiB/s wr, 263 op/s 2023-12-01T01:19:30.047 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_locks_execve 2023-12-01T01:19:30.047 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_mmap_collision 2023-12-01T01:19:30.048 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] mmap-write-concurrent 2023-12-01T01:19:30.131 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] xfsctl 2023-12-01T01:19:30.373 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] bstat 2023-12-01T01:19:30.374 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_mtab 2023-12-01T01:19:30.383 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] getdevicesize 2023-12-01T01:19:30.391 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] preallo_rw_pattern_reader 2023-12-01T01:19:30.620 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] preallo_rw_pattern_writer 2023-12-01T01:19:30.668 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] ftrunc 2023-12-01T01:19:30.668 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] trunc 2023-12-01T01:19:30.669 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] fs_perms 2023-12-01T01:19:30.821 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] testx 2023-12-01T01:19:30.873 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] looptest 2023-12-01T01:19:30.876 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] locktest 2023-12-01T01:19:30.892 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] unwritten_mmap 2023-12-01T01:19:30.979 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] bulkstat_unlink_test 2023-12-01T01:19:31.099 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] bulkstat_unlink_test_modified 2023-12-01T01:19:31.110 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_dir_offset 2023-12-01T01:19:31.259 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_futimens 2023-12-01T01:19:31.262 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_immutable 2023-12-01T01:19:31.318 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] stale_handle 2023-12-01T01:19:31.352 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] pwrite_mmap_blocked 2023-12-01T01:19:31.448 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_dir_offset2 2023-12-01T01:19:31.546 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] seek_sanity_test 2023-12-01T01:19:31.558 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] seek_copy_test 2023-12-01T01:19:31.692 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_readdir_1 2023-12-01T01:19:31.825 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_readdir_2 2023-12-01T01:19:31.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:31 smithi077 ceph-mon[98867]: pgmap v347: 97 pgs: 97 active+clean; 40 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 1.5 MiB/s wr, 274 op/s 2023-12-01T01:19:31.906 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] fsync-tester 2023-12-01T01:19:32.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:31 smithi203 ceph-mon[103581]: pgmap v347: 97 pgs: 97 active+clean; 40 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 1.5 MiB/s wr, 274 op/s 2023-12-01T01:19:32.016 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] nsexec 2023-12-01T01:19:32.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:31 smithi137 ceph-mon[103312]: pgmap v347: 97 pgs: 97 active+clean; 40 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 1.5 MiB/s wr, 274 op/s 2023-12-01T01:19:32.058 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] cloner 2023-12-01T01:19:32.109 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] renameat2 2023-12-01T01:19:32.248 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_getcwd 2023-12-01T01:19:32.260 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] e4compact 2023-12-01T01:19:32.295 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] test-nextquota 2023-12-01T01:19:32.351 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] punch-alternating 2023-12-01T01:19:32.427 INFO:tasks.workunit.client.0.smithi077.stderr:t_getcwd.c: In function 'do_rename': 2023-12-01T01:19:32.428 INFO:tasks.workunit.client.0.smithi077.stderr:t_getcwd.c:40:2: warning: 'strncpy' specified bound 256 equals destination size [-Wstringop-truncation] 2023-12-01T01:19:32.428 INFO:tasks.workunit.client.0.smithi077.stderr: strncpy(c_name, prefix, BUF_SIZE); 2023-12-01T01:19:32.428 INFO:tasks.workunit.client.0.smithi077.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-12-01T01:19:32.469 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] attr-list-by-handle-cursor-test 2023-12-01T01:19:32.525 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] listxattr 2023-12-01T01:19:32.526 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] dio-interleaved 2023-12-01T01:19:32.564 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_dir_type 2023-12-01T01:19:32.685 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] dio-invalidate-cache 2023-12-01T01:19:32.708 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] stat_test 2023-12-01T01:19:32.729 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_encrypted_d_revalidate 2023-12-01T01:19:32.792 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] attr_replace_test 2023-12-01T01:19:32.956 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] swapon 2023-12-01T01:19:32.961 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] mkswap 2023-12-01T01:19:32.982 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_attr_corruption 2023-12-01T01:19:33.119 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_open_tmpfiles 2023-12-01T01:19:33.129 INFO:tasks.workunit.client.0.smithi077.stderr:swapon.c: In function 'main': 2023-12-01T01:19:33.129 INFO:tasks.workunit.client.0.smithi077.stderr:swapon.c:135:3: warning: 'p' may be used uninitialized in this function [-Wmaybe-uninitialized] 2023-12-01T01:19:33.129 INFO:tasks.workunit.client.0.smithi077.stderr: memcpy(p, buf, BUF_SIZE); 2023-12-01T01:19:33.129 INFO:tasks.workunit.client.0.smithi077.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~ 2023-12-01T01:19:33.192 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] bulkstat_null_ocount 2023-12-01T01:19:33.192 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] fscrypt-crypt-util 2023-12-01T01:19:33.206 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] splice-test 2023-12-01T01:19:33.339 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] fiemap-tester 2023-12-01T01:19:33.405 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] t_stripealign 2023-12-01T01:19:33.439 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] open_by_handle 2023-12-01T01:19:33.602 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] fssum 2023-12-01T01:19:33.696 INFO:tasks.workunit.client.0.smithi077.stdout:Building log-writes 2023-12-01T01:19:33.705 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] replay-log 2023-12-01T01:19:33.757 INFO:tasks.workunit.client.0.smithi077.stdout:Building perf 2023-12-01T01:19:33.761 INFO:tasks.workunit.client.0.smithi077.stdout:Building aio-dio-regress 2023-12-01T01:19:33.768 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] aio-dio-append-write-fallocate-race 2023-12-01T01:19:33.857 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] aio-dio-fcntl-race 2023-12-01T01:19:33.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:33 smithi077 ceph-mon[98867]: pgmap v348: 97 pgs: 97 active+clean; 40 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 1.5 MiB/s wr, 277 op/s 2023-12-01T01:19:34.008 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] aio-dio-subblock-eof-read 2023-12-01T01:19:34.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:33 smithi203 ceph-mon[103581]: pgmap v348: 97 pgs: 97 active+clean; 40 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 1.5 MiB/s wr, 277 op/s 2023-12-01T01:19:34.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:33 smithi137 ceph-mon[103312]: pgmap v348: 97 pgs: 97 active+clean; 40 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 1.5 MiB/s wr, 277 op/s 2023-12-01T01:19:34.061 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] aio-dio-append-write-read-race 2023-12-01T01:19:34.074 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] aio-dio-invalidate-readahead 2023-12-01T01:19:34.171 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] aio-dio-hole-filling-race 2023-12-01T01:19:34.187 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2023-12-01T01:19:34.287 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] aio-last-ref-held-by-io 2023-12-01T01:19:34.300 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] aio-dio-cycle-write 2023-12-01T01:19:34.397 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] aio-dio-extend-stat 2023-12-01T01:19:34.406 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2023-12-01T01:19:34.536 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] aiocp 2023-12-01T01:19:34.537 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] aio-dio-invalidate-failure 2023-12-01T01:19:34.581 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] aiodio_sparse2 2023-12-01T01:19:34.636 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] aio-dio-cow-race 2023-12-01T01:19:34.777 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] aio-dio-write-verify 2023-12-01T01:19:34.835 INFO:tasks.workunit.client.0.smithi077.stderr:aiodio_sparse2.c: In function 'main': 2023-12-01T01:19:34.835 INFO:tasks.workunit.client.0.smithi077.stderr:aiodio_sparse2.c:404:2: warning: 'strncpy' specified bound 4096 equals destination size [-Wstringop-truncation] 2023-12-01T01:19:34.835 INFO:tasks.workunit.client.0.smithi077.stderr: strncpy(filename, argv[argc-1], PATH_MAX); 2023-12-01T01:19:34.835 INFO:tasks.workunit.client.0.smithi077.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2023-12-01T01:19:34.855 INFO:tasks.workunit.client.0.smithi077.stdout: [CC] aio-dio-eof-race 2023-12-01T01:19:34.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:34 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:19:35.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:34 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:19:35.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:34 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:19:35.059 INFO:tasks.workunit.client.0.smithi077.stderr:+ cd .. 2023-12-01T01:19:35.059 INFO:tasks.workunit.client.0.smithi077.stderr:+ cp xfstests-dev/ltp/fsx . 2023-12-01T01:19:35.061 INFO:tasks.workunit.client.0.smithi077.stderr:+ OPTIONS=-z 2023-12-01T01:19:35.061 INFO:tasks.workunit.client.0.smithi077.stderr:+ ./fsx -z 1MB -N 50000 -p 10000 -l 1048576 2023-12-01T01:19:35.061 INFO:tasks.workunit.client.0.smithi077.stdout:Seed set to 1 2023-12-01T01:19:35.081 INFO:tasks.workunit.client.0.smithi077.stderr:main: filesystem does not support fallocate mode 0, disabling! 2023-12-01T01:19:35.081 INFO:tasks.workunit.client.0.smithi077.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2023-12-01T01:19:35.083 INFO:tasks.workunit.client.0.smithi077.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2023-12-01T01:19:35.083 INFO:tasks.workunit.client.0.smithi077.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2023-12-01T01:19:35.083 INFO:tasks.workunit.client.0.smithi077.stderr:main: filesystem does not support clone range, disabling! 2023-12-01T01:19:35.086 INFO:tasks.workunit.client.0.smithi077.stderr:main: filesystem does not support dedupe range, disabling! 2023-12-01T01:19:35.093 INFO:tasks.workunit.client.0.smithi077.stdout:truncating to largest ever: 0x38fe9 2023-12-01T01:19:35.118 INFO:tasks.workunit.client.0.smithi077.stdout:copying to largest ever: 0xead53 2023-12-01T01:19:35.179 INFO:tasks.workunit.client.0.smithi077.stdout:truncating to largest ever: 0xf2cdd 2023-12-01T01:19:35.185 INFO:tasks.workunit.client.0.smithi077.stdout:truncating to largest ever: 0xfca47 2023-12-01T01:19:35.662 INFO:tasks.workunit.client.0.smithi077.stdout:copying to largest ever: 0xfcdd6 2023-12-01T01:19:35.681 INFO:tasks.workunit.client.0.smithi077.stdout:copying to largest ever: 0xfe854 2023-12-01T01:19:35.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:35 smithi077 ceph-mon[98867]: pgmap v349: 97 pgs: 97 active+clean; 40 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 1.4 MiB/s wr, 277 op/s 2023-12-01T01:19:35.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:35 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:19:35.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:35 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:19:35.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:35 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:19:36.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:35 smithi203 ceph-mon[103581]: pgmap v349: 97 pgs: 97 active+clean; 40 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 1.4 MiB/s wr, 277 op/s 2023-12-01T01:19:36.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:35 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:19:36.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:35 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:19:36.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:35 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:19:36.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:35 smithi137 ceph-mon[103312]: pgmap v349: 97 pgs: 97 active+clean; 40 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 1.4 MiB/s wr, 277 op/s 2023-12-01T01:19:36.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:35 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:19:36.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:35 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:19:36.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:35 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:19:36.964 INFO:tasks.workunit.client.0.smithi077.stdout:copying to largest ever: 0xffce4 2023-12-01T01:19:37.670 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.669+0000 7f8b404f8700 1 -- 172.21.15.77:0/4045217498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b38100e80 msgr2=0x7f8b381012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:37.670 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.669+0000 7f8b404f8700 1 --2- 172.21.15.77:0/4045217498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b38100e80 0x7f8b381012e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f8b280099c0 tx=0x7f8b28009cd0 comp rx=0 tx=0).stop 2023-12-01T01:19:37.670 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.669+0000 7f8b404f8700 1 -- 172.21.15.77:0/4045217498 shutdown_connections 2023-12-01T01:19:37.671 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.669+0000 7f8b404f8700 1 --2- 172.21.15.77:0/4045217498 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b38101820 0x7f8b381080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:37.671 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.669+0000 7f8b404f8700 1 --2- 172.21.15.77:0/4045217498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b380ffc90 0x7f8b381000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:37.671 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.669+0000 7f8b404f8700 1 --2- 172.21.15.77:0/4045217498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b38100e80 0x7f8b381012e0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:37.671 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.669+0000 7f8b404f8700 1 -- 172.21.15.77:0/4045217498 >> 172.21.15.77:0/4045217498 conn(0x7f8b380fb1e0 msgr2=0x7f8b380fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:19:37.671 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.669+0000 7f8b404f8700 1 -- 172.21.15.77:0/4045217498 shutdown_connections 2023-12-01T01:19:37.671 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.669+0000 7f8b404f8700 1 -- 172.21.15.77:0/4045217498 wait complete. 2023-12-01T01:19:37.671 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.669+0000 7f8b404f8700 1 Processor -- start 2023-12-01T01:19:37.671 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.669+0000 7f8b404f8700 1 -- start start 2023-12-01T01:19:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.670+0000 7f8b404f8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b380ffc90 0x7f8b38198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.670+0000 7f8b404f8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b38100e80 0x7f8b381992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.670+0000 7f8b404f8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b38101820 0x7f8b3819a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.670+0000 7f8b404f8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3819a790 con 0x7f8b38100e80 2023-12-01T01:19:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.670+0000 7f8b404f8700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3819a900 con 0x7f8b38101820 2023-12-01T01:19:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.670+0000 7f8b404f8700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3819aac0 con 0x7f8b380ffc90 2023-12-01T01:19:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.670+0000 7f8b3da93700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b38100e80 0x7f8b381992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.670+0000 7f8b3da93700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b38100e80 0x7f8b381992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60396/0 (socket says 172.21.15.77:60396) 2023-12-01T01:19:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.670+0000 7f8b3da93700 1 -- 172.21.15.77:0/1794418062 learned_addr learned my addr 172.21.15.77:0/1794418062 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:19:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.670+0000 7f8b3e294700 1 --2- 172.21.15.77:0/1794418062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b380ffc90 0x7f8b38198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.670+0000 7f8b3ea95700 1 --2- 172.21.15.77:0/1794418062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b38101820 0x7f8b3819a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.670+0000 7f8b3da93700 1 -- 172.21.15.77:0/1794418062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b380ffc90 msgr2=0x7f8b38198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.670+0000 7f8b3da93700 1 --2- 172.21.15.77:0/1794418062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b380ffc90 0x7f8b38198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.670+0000 7f8b3da93700 1 -- 172.21.15.77:0/1794418062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b38101820 msgr2=0x7f8b3819a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.670+0000 7f8b3da93700 1 --2- 172.21.15.77:0/1794418062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b38101820 0x7f8b3819a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.670+0000 7f8b3da93700 1 -- 172.21.15.77:0/1794418062 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b280095f0 con 0x7f8b38100e80 2023-12-01T01:19:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.670+0000 7f8b3da93700 1 --2- 172.21.15.77:0/1794418062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b38100e80 0x7f8b381992e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f8b2800c010 tx=0x7f8b2800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:19:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.670+0000 7f8b2f7fe700 1 -- 172.21.15.77:0/1794418062 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2801d070 con 0x7f8b38100e80 2023-12-01T01:19:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.670+0000 7f8b2f7fe700 1 -- 172.21.15.77:0/1794418062 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b28005050 con 0x7f8b38100e80 2023-12-01T01:19:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.670+0000 7f8b404f8700 1 -- 172.21.15.77:0/1794418062 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b3819ad40 con 0x7f8b38100e80 2023-12-01T01:19:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.670+0000 7f8b2f7fe700 1 -- 172.21.15.77:0/1794418062 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b28004ad0 con 0x7f8b38100e80 2023-12-01T01:19:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.670+0000 7f8b404f8700 1 -- 172.21.15.77:0/1794418062 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b381a14e0 con 0x7f8b38100e80 2023-12-01T01:19:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.673+0000 7f8b2f7fe700 1 -- 172.21.15.77:0/1794418062 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f8b280051c0 con 0x7f8b38100e80 2023-12-01T01:19:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.674+0000 7f8b2f7fe700 1 --2- 172.21.15.77:0/1794418062 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8b240a05a0 0x7f8b240a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.674+0000 7f8b2f7fe700 1 -- 172.21.15.77:0/1794418062 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f8b280cc830 con 0x7f8b38100e80 2023-12-01T01:19:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.674+0000 7f8b3e294700 1 --2- 172.21.15.77:0/1794418062 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8b240a05a0 0x7f8b240a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.674+0000 7f8b3e294700 1 --2- 172.21.15.77:0/1794418062 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8b240a05a0 0x7f8b240a2a60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8b38199b40 tx=0x7f8b34009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:19:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.674+0000 7f8b4c688b80 1 Processor -- start 2023-12-01T01:19:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.675+0000 7f8b4c688b80 1 -- start start 2023-12-01T01:19:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.675+0000 7f8b4c688b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d994ec6680 0x55d994ec6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.675+0000 7f8b4c688b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d994ec6fe0 0x55d994ec78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.675+0000 7f8b4c688b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d994ec7ed0 0x55d994ec8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.675+0000 7f8b4c688b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d994e1e9b0 con 0x55d994ec6680 2023-12-01T01:19:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.675+0000 7f8b4c688b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d994e1c370 con 0x55d994ec7ed0 2023-12-01T01:19:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.675+0000 7f8b4c688b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d994e17c60 con 0x55d994ec6fe0 2023-12-01T01:19:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.675+0000 7f8b3e294700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d994ec6680 0x55d994ec6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.675+0000 7f8b3e294700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d994ec6680 0x55d994ec6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60402/0 (socket says 172.21.15.77:60402) 2023-12-01T01:19:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.675+0000 7f8b3e294700 1 -- 172.21.15.77:0/3246295198 learned_addr learned my addr 172.21.15.77:0/3246295198 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:19:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.675+0000 7f8b3da93700 1 --2- 172.21.15.77:0/3246295198 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d994ec6fe0 0x55d994ec78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.675+0000 7f8b3ea95700 1 --2- 172.21.15.77:0/3246295198 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d994ec7ed0 0x55d994ec8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.676+0000 7f8b3da93700 1 -- 172.21.15.77:0/3246295198 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d994ec7ed0 msgr2=0x55d994ec8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.676+0000 7f8b3da93700 1 --2- 172.21.15.77:0/3246295198 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d994ec7ed0 0x55d994ec8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.676+0000 7f8b3da93700 1 -- 172.21.15.77:0/3246295198 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d994ec6680 msgr2=0x55d994ec6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.676+0000 7f8b3da93700 1 --2- 172.21.15.77:0/3246295198 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d994ec6680 0x55d994ec6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.676+0000 7f8b3da93700 1 -- 172.21.15.77:0/3246295198 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d994c45d80 con 0x55d994ec6fe0 2023-12-01T01:19:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.676+0000 7f8b3da93700 1 --2- 172.21.15.77:0/3246295198 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d994ec6fe0 0x55d994ec78f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8b28096620 tx=0x7f8b2809a820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc9c0f9e811e01af server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:19:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.676+0000 7f8b2cff9700 1 -- 172.21.15.77:0/3246295198 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2809c070 con 0x55d994ec6fe0 2023-12-01T01:19:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.676+0000 7f8b2cff9700 1 -- 172.21.15.77:0/3246295198 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b280a0070 con 0x55d994ec6fe0 2023-12-01T01:19:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.676+0000 7f8b2cff9700 1 -- 172.21.15.77:0/3246295198 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2809a380 con 0x55d994ec6fe0 2023-12-01T01:19:37.682 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.677+0000 7f8b4c688b80 1 -- 172.21.15.77:0/3246295198 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d994ec6fe0 msgr2=0x55d994ec78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:37.682 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.677+0000 7f8b4c688b80 1 --2- 172.21.15.77:0/3246295198 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d994ec6fe0 0x55d994ec78f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8b28096620 tx=0x7f8b2809a820 comp rx=0 tx=0).stop 2023-12-01T01:19:37.682 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.677+0000 7f8b4c688b80 1 -- 172.21.15.77:0/3246295198 shutdown_connections 2023-12-01T01:19:37.682 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.677+0000 7f8b4c688b80 1 --2- 172.21.15.77:0/3246295198 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d994ec7ed0 0x55d994ec8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:37.682 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.677+0000 7f8b4c688b80 1 --2- 172.21.15.77:0/3246295198 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d994ec6680 0x55d994ec6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:37.682 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.677+0000 7f8b4c688b80 1 --2- 172.21.15.77:0/3246295198 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d994ec6fe0 0x55d994ec78f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:37.682 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.677+0000 7f8b4c688b80 1 -- 172.21.15.77:0/3246295198 >> 172.21.15.77:0/3246295198 conn(0x55d994e26070 msgr2=0x55d994ec3040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:19:37.682 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.677+0000 7f8b4c688b80 1 -- 172.21.15.77:0/3246295198 shutdown_connections 2023-12-01T01:19:37.682 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.677+0000 7f8b4c688b80 1 -- 172.21.15.77:0/3246295198 wait complete. 2023-12-01T01:19:37.682 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.678+0000 7f8b4c688b80 1 -- start start 2023-12-01T01:19:37.682 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.678+0000 7f8b4c688b80 1 Processor -- start 2023-12-01T01:19:37.682 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.678+0000 7f8b4c688b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d994ec6680 0x55d994ed0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:37.682 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.678+0000 7f8b4c688b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d994ec6fe0 0x55d994ed0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.678+0000 7f8b4c688b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d994ec7ed0 0x55d994edfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.678+0000 7f8b4c688b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d994db44b0 con 0x55d994ec6fe0 2023-12-01T01:19:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.678+0000 7f8b4c688b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d994db4150 con 0x55d994ec6680 2023-12-01T01:19:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.678+0000 7f8b4c688b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d994c9b8c0 con 0x55d994ec7ed0 2023-12-01T01:19:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.678+0000 7f8b4c688b80 10 client.0 _ll_register_callbacks cb 0x55d994d01a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:19:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.678+0000 7f8b3da93700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d994ec6fe0 0x55d994ed0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.678+0000 7f8b3e294700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d994ec6680 0x55d994ed0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.678+0000 7f8b3da93700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d994ec6fe0 0x55d994ed0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60410/0 (socket says 172.21.15.77:60410) 2023-12-01T01:19:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.678+0000 7f8b3da93700 1 -- 172.21.15.77:0/1055160202 learned_addr learned my addr 172.21.15.77:0/1055160202 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:19:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.679+0000 7f8b3ea95700 1 --2- 172.21.15.77:0/1055160202 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d994ec7ed0 0x55d994edfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.679+0000 7f8b3e294700 1 -- 172.21.15.77:0/1055160202 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d994ec7ed0 msgr2=0x55d994edfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.679+0000 7f8b3e294700 1 --2- 172.21.15.77:0/1055160202 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d994ec7ed0 0x55d994edfed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.679+0000 7f8b3e294700 1 -- 172.21.15.77:0/1055160202 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d994ec6fe0 msgr2=0x55d994ed0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.679+0000 7f8b3e294700 1 --2- 172.21.15.77:0/1055160202 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d994ec6fe0 0x55d994ed0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.679+0000 7f8b3e294700 1 -- 172.21.15.77:0/1055160202 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b381a14e0 con 0x55d994ec6680 2023-12-01T01:19:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.679+0000 7f8b3e294700 1 --2- 172.21.15.77:0/1055160202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d994ec6680 0x55d994ed0140 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8b34009860 tx=0x7f8b340047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2666ec9744d25356 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:19:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.679+0000 7f8b21ffb700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2023-12-01T01:19:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.679+0000 7f8b21ffb700 1 -- 172.21.15.77:0/1055160202 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b34022070 con 0x55d994ec6680 2023-12-01T01:19:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.679+0000 7f8b4c688b80 1 -- 172.21.15.77:0/1055160202 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d994e42cb0 con 0x55d994ec6680 2023-12-01T01:19:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.679+0000 7f8b21ffb700 1 -- 172.21.15.77:0/1055160202 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f8b3401d410 con 0x55d994ec6680 2023-12-01T01:19:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.679+0000 7f8b21ffb700 1 -- 172.21.15.77:0/1055160202 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b3401d5d0 con 0x55d994ec6680 2023-12-01T01:19:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.679+0000 7f8b4c688b80 10 client.25502 fetch_fsmap learned FSMap version 31 2023-12-01T01:19:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.679+0000 7f8b4c688b80 1 -- 172.21.15.77:0/1055160202 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55d994c2ec00 con 0x55d994ec6680 2023-12-01T01:19:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.680+0000 7f8b21ffb700 1 -- 172.21.15.77:0/1055160202 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d994e42cb0 con 0x55d994ec6680 2023-12-01T01:19:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.680+0000 7f8b21ffb700 1 -- 172.21.15.77:0/1055160202 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8b340276c0 con 0x55d994ec6680 2023-12-01T01:19:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.680+0000 7f8b4c688b80 10 client.25502 fetch_fsmap finished waiting for FSMap version 31 2023-12-01T01:19:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.680+0000 7f8b4c688b80 10 client.25502 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-01T01:19:37.684 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.680+0000 7f8b4c688b80 20 client.25502 populate_metadata read hostname 'smithi077' 2023-12-01T01:19:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.680+0000 7f8b4c688b80 1 --2- 172.21.15.77:0/1055160202 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55d994ed5a40 0x55d994ed7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.680+0000 7f8b4c688b80 4 client.25502 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:19:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.680+0000 7f8b4c688b80 1 -- 172.21.15.77:0/1055160202 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d994e95db0 con 0x55d994ed5a40 2023-12-01T01:19:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.680+0000 7f8b3da93700 1 --2- 172.21.15.77:0/1055160202 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55d994ed5a40 0x55d994ed7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.680+0000 7f8b3da93700 1 --2- 172.21.15.77:0/1055160202 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55d994ed5a40 0x55d994ed7e60 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bdf8b2f12d8ff7b server_cookie=78cb3b05e9f4e5e2 in_seq=0 out_seq=0 2023-12-01T01:19:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.680+0000 7f8b21ffb700 10 client.25502 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:19:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.681+0000 7f8b21ffb700 1 -- 172.21.15.77:0/1055160202 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55d994e95db0 con 0x55d994ed5a40 2023-12-01T01:19:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.681+0000 7f8b21ffb700 10 client.25502 handle_command_reply: tid=0 2023-12-01T01:19:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.681+0000 7f8b4c688b80 1 client.25502 shutdown 2023-12-01T01:19:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.681+0000 7f8b4c688b80 1 -- 172.21.15.77:0/1055160202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d994ec6680 msgr2=0x55d994ed0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.681+0000 7f8b4c688b80 1 --2- 172.21.15.77:0/1055160202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d994ec6680 0x55d994ed0140 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8b34009860 tx=0x7f8b340047c0 comp rx=0 tx=0).stop 2023-12-01T01:19:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.681+0000 7f8b4c688b80 1 -- 172.21.15.77:0/1055160202 shutdown_connections 2023-12-01T01:19:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.681+0000 7f8b4c688b80 1 --2- 172.21.15.77:0/1055160202 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d994ec7ed0 0x55d994edfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.681+0000 7f8b4c688b80 1 --2- 172.21.15.77:0/1055160202 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55d994ed5a40 0x55d994ed7e60 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.681+0000 7f8b4c688b80 1 --2- 172.21.15.77:0/1055160202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d994ec6680 0x55d994ed0140 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.681+0000 7f8b21ffb700 0 client.25502 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:19:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.681+0000 7f8b4c688b80 1 --2- 172.21.15.77:0/1055160202 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d994ec6fe0 0x55d994ed0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.681+0000 7f8b4c688b80 1 -- 172.21.15.77:0/1055160202 >> 172.21.15.77:0/1055160202 conn(0x55d994e26070 msgr2=0x55d994ec13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:19:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.681+0000 7f8b4c688b80 1 -- 172.21.15.77:0/1055160202 shutdown_connections 2023-12-01T01:19:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.681+0000 7f8b4c688b80 1 -- 172.21.15.77:0/1055160202 wait complete. 2023-12-01T01:19:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.682+0000 7f8b4c688b80 20 client.25502 trim_cache size 0 max 16384 2023-12-01T01:19:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.698+0000 7f8b4c688b80 1 Processor -- start 2023-12-01T01:19:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.698+0000 7f8b4c688b80 1 -- start start 2023-12-01T01:19:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.698+0000 7f8b4c688b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d994ec6680 0x55d994edc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.698+0000 7f8b4c688b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d994ec6fe0 0x55d994ecf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.698+0000 7f8b4c688b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d994ec7ed0 0x55d994ec2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.698+0000 7f8b4c688b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d994eb97a0 con 0x55d994ec7ed0 2023-12-01T01:19:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.698+0000 7f8b4c688b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d994ecf5f0 con 0x55d994ec6fe0 2023-12-01T01:19:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.698+0000 7f8b4c688b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d994ebc500 con 0x55d994ec6680 2023-12-01T01:19:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.698+0000 7f8b3ea95700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d994ec7ed0 0x55d994ec2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.698+0000 7f8b3da93700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d994ec6fe0 0x55d994ecf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.698+0000 7f8b3ea95700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d994ec7ed0 0x55d994ec2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60420/0 (socket says 172.21.15.77:60420) 2023-12-01T01:19:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.698+0000 7f8b3ea95700 1 -- 172.21.15.77:0/2980847137 learned_addr learned my addr 172.21.15.77:0/2980847137 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:19:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.698+0000 7f8b3e294700 1 --2- 172.21.15.77:0/2980847137 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d994ec6680 0x55d994edc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.699+0000 7f8b3ea95700 1 -- 172.21.15.77:0/2980847137 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d994ec6680 msgr2=0x55d994edc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.699+0000 7f8b3ea95700 1 --2- 172.21.15.77:0/2980847137 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d994ec6680 0x55d994edc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.699+0000 7f8b3ea95700 1 -- 172.21.15.77:0/2980847137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d994ec6fe0 msgr2=0x55d994ecf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.699+0000 7f8b3ea95700 1 --2- 172.21.15.77:0/2980847137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d994ec6fe0 0x55d994ecf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.699+0000 7f8b3ea95700 1 -- 172.21.15.77:0/2980847137 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d994c6c1f0 con 0x55d994ec7ed0 2023-12-01T01:19:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.699+0000 7f8b3ea95700 1 --2- 172.21.15.77:0/2980847137 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d994ec7ed0 0x55d994ec2c20 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8b30012890 tx=0x7f8b30009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=417398649266c449 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:19:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.699+0000 7f8b2d7fa700 1 -- 172.21.15.77:0/2980847137 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3000e040 con 0x55d994ec7ed0 2023-12-01T01:19:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.699+0000 7f8b2d7fa700 1 -- 172.21.15.77:0/2980847137 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b3000f040 con 0x55d994ec7ed0 2023-12-01T01:19:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.699+0000 7f8b2d7fa700 1 -- 172.21.15.77:0/2980847137 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b30004730 con 0x55d994ec7ed0 2023-12-01T01:19:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.700+0000 7f8b4c688b80 1 -- 172.21.15.77:0/2980847137 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d994ec7ed0 msgr2=0x55d994ec2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.700+0000 7f8b4c688b80 1 --2- 172.21.15.77:0/2980847137 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d994ec7ed0 0x55d994ec2c20 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8b30012890 tx=0x7f8b30009920 comp rx=0 tx=0).stop 2023-12-01T01:19:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.700+0000 7f8b4c688b80 1 -- 172.21.15.77:0/2980847137 shutdown_connections 2023-12-01T01:19:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.700+0000 7f8b4c688b80 1 --2- 172.21.15.77:0/2980847137 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d994ec7ed0 0x55d994ec2c20 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.700+0000 7f8b4c688b80 1 --2- 172.21.15.77:0/2980847137 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d994ec6680 0x55d994edc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.700+0000 7f8b4c688b80 1 --2- 172.21.15.77:0/2980847137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d994ec6fe0 0x55d994ecf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.700+0000 7f8b4c688b80 1 -- 172.21.15.77:0/2980847137 >> 172.21.15.77:0/2980847137 conn(0x55d994e26070 msgr2=0x55d994ecee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:19:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.700+0000 7f8b4c688b80 1 -- 172.21.15.77:0/2980847137 shutdown_connections 2023-12-01T01:19:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.700+0000 7f8b4c688b80 1 -- 172.21.15.77:0/2980847137 wait complete. 2023-12-01T01:19:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.700+0000 7f8b4c688b80 1 -- start start 2023-12-01T01:19:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.700+0000 7f8b4c688b80 1 Processor -- start 2023-12-01T01:19:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.701+0000 7f8b4c688b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d994ec6680 0x55d994edae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.701+0000 7f8b4c688b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d994ec6fe0 0x55d994edb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.701+0000 7f8b4c688b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d994ec7ed0 0x55d994edb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.701+0000 7f8b4c688b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d994edcb10 con 0x55d994ec6680 2023-12-01T01:19:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.701+0000 7f8b4c688b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d994f3e210 con 0x55d994ec7ed0 2023-12-01T01:19:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.701+0000 7f8b4c688b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d994f3e3a0 con 0x55d994ec6fe0 2023-12-01T01:19:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.701+0000 7f8b4c688b80 10 client.0 _ll_register_callbacks cb 0x7f8b340040d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:19:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.701+0000 7f8b3e294700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d994ec6680 0x55d994edae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.701+0000 7f8b3e294700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d994ec6680 0x55d994edae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60422/0 (socket says 172.21.15.77:60422) 2023-12-01T01:19:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.701+0000 7f8b3e294700 1 -- 172.21.15.77:0/595587836 learned_addr learned my addr 172.21.15.77:0/595587836 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:19:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.701+0000 7f8b3ea95700 1 --2- 172.21.15.77:0/595587836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d994ec7ed0 0x55d994edb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.701+0000 7f8b3da93700 1 --2- 172.21.15.77:0/595587836 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d994ec6fe0 0x55d994edb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.701+0000 7f8b3e294700 1 -- 172.21.15.77:0/595587836 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d994ec6fe0 msgr2=0x55d994edb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.701+0000 7f8b3e294700 1 --2- 172.21.15.77:0/595587836 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d994ec6fe0 0x55d994edb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.701+0000 7f8b3e294700 1 -- 172.21.15.77:0/595587836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d994ec7ed0 msgr2=0x55d994edb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.701+0000 7f8b3e294700 1 --2- 172.21.15.77:0/595587836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d994ec7ed0 0x55d994edb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.701+0000 7f8b3e294700 1 -- 172.21.15.77:0/595587836 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b30012540 con 0x55d994ec6680 2023-12-01T01:19:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.701+0000 7f8b3ea95700 1 --2- 172.21.15.77:0/595587836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d994ec7ed0 0x55d994edb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:19:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.701+0000 7f8b3da93700 1 --2- 172.21.15.77:0/595587836 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d994ec6fe0 0x55d994edb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:19:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.701+0000 7f8b3e294700 1 --2- 172.21.15.77:0/595587836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d994ec6680 0x55d994edae90 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8b34004010 tx=0x7f8b340274a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30bdf04fc8e11c6c server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:19:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.701+0000 7f8b21ffb700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:19:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.701+0000 7f8b21ffb700 1 -- 172.21.15.77:0/595587836 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b34037070 con 0x55d994ec6680 2023-12-01T01:19:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.701+0000 7f8b21ffb700 1 -- 172.21.15.77:0/595587836 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f8b3402d030 con 0x55d994ec6680 2023-12-01T01:19:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.701+0000 7f8b4c688b80 1 -- 172.21.15.77:0/595587836 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b280a0070 con 0x55d994ec6680 2023-12-01T01:19:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.702+0000 7f8b21ffb700 1 -- 172.21.15.77:0/595587836 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b34022040 con 0x55d994ec6680 2023-12-01T01:19:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.702+0000 7f8b4c688b80 10 client.15927 fetch_fsmap learned FSMap version 31 2023-12-01T01:19:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.702+0000 7f8b4c688b80 1 -- 172.21.15.77:0/595587836 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55d994c44250 con 0x55d994ec6680 2023-12-01T01:19:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.702+0000 7f8b21ffb700 1 -- 172.21.15.77:0/595587836 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b280a0070 con 0x55d994ec6680 2023-12-01T01:19:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.702+0000 7f8b21ffb700 1 -- 172.21.15.77:0/595587836 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8b34005500 con 0x55d994ec6680 2023-12-01T01:19:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.702+0000 7f8b4c688b80 10 client.15927 fetch_fsmap finished waiting for FSMap version 31 2023-12-01T01:19:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.702+0000 7f8b4c688b80 10 client.15927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-01T01:19:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.702+0000 7f8b4c688b80 20 client.15927 populate_metadata read hostname 'smithi077' 2023-12-01T01:19:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.702+0000 7f8b4c688b80 1 --2- 172.21.15.77:0/595587836 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55d994f3fd10 0x55d994f42130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.702+0000 7f8b4c688b80 4 client.15927 mds_command: new command op to 24500 tid=0[{"prefix": "flush journal"}] 2023-12-01T01:19:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.702+0000 7f8b4c688b80 1 -- 172.21.15.77:0/595587836 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d994e10ce0 con 0x55d994f3fd10 2023-12-01T01:19:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.702+0000 7f8b3da93700 1 --2- 172.21.15.77:0/595587836 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55d994f3fd10 0x55d994f42130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.703+0000 7f8b21ffb700 10 client.15927 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:19:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.703+0000 7f8b3da93700 1 --2- 172.21.15.77:0/595587836 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55d994f3fd10 0x55d994f42130 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fa8a95f756f436b server_cookie=8611cba9dc5ef08f in_seq=0 out_seq=0 2023-12-01T01:19:37.724 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.722+0000 7f8b21ffb700 1 -- 172.21.15.77:0/595587836 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d994e10ce0 con 0x55d994f3fd10 2023-12-01T01:19:37.724 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.722+0000 7f8b21ffb700 10 client.15927 handle_command_reply: tid=0 2023-12-01T01:19:37.725 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.722+0000 7f8b4c688b80 1 client.15927 shutdown 2023-12-01T01:19:37.725 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.723+0000 7f8b4c688b80 1 -- 172.21.15.77:0/595587836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d994ec6680 msgr2=0x55d994edae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:37.725 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.723+0000 7f8b4c688b80 1 --2- 172.21.15.77:0/595587836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d994ec6680 0x55d994edae90 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8b34004010 tx=0x7f8b340274a0 comp rx=0 tx=0).stop 2023-12-01T01:19:37.725 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.723+0000 7f8b4c688b80 1 -- 172.21.15.77:0/595587836 shutdown_connections 2023-12-01T01:19:37.725 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.723+0000 7f8b4c688b80 1 --2- 172.21.15.77:0/595587836 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55d994f3fd10 0x55d994f42130 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:37.725 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.723+0000 7f8b4c688b80 1 --2- 172.21.15.77:0/595587836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d994ec7ed0 0x55d994edb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:37.725 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.723+0000 7f8b4c688b80 1 --2- 172.21.15.77:0/595587836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d994ec6680 0x55d994edae90 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:37.725 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.723+0000 7f8b21ffb700 0 client.15927 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:19:37.725 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.723+0000 7f8b4c688b80 1 --2- 172.21.15.77:0/595587836 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d994ec6fe0 0x55d994edb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:37.725 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.723+0000 7f8b4c688b80 1 -- 172.21.15.77:0/595587836 >> 172.21.15.77:0/595587836 conn(0x55d994e26070 msgr2=0x55d994ec2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:19:37.725 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.723+0000 7f8b4c688b80 1 -- 172.21.15.77:0/595587836 shutdown_connections 2023-12-01T01:19:37.725 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.723+0000 7f8b4c688b80 1 -- 172.21.15.77:0/595587836 wait complete. 2023-12-01T01:19:37.725 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.723+0000 7f8b4c688b80 20 client.15927 trim_cache size 0 max 16384 2023-12-01T01:19:37.726 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout:{ 2023-12-01T01:19:37.726 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout: "message": "", 2023-12-01T01:19:37.726 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout: "return_code": 0 2023-12-01T01:19:37.726 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout:} 2023-12-01T01:19:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.723+0000 7f8b404f8700 1 -- 172.21.15.77:0/1794418062 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8b240a05a0 msgr2=0x7f8b240a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.723+0000 7f8b404f8700 1 --2- 172.21.15.77:0/1794418062 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8b240a05a0 0x7f8b240a2a60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8b38199b40 tx=0x7f8b34009260 comp rx=0 tx=0).stop 2023-12-01T01:19:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.723+0000 7f8b404f8700 1 -- 172.21.15.77:0/1794418062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b38100e80 msgr2=0x7f8b381992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.723+0000 7f8b404f8700 1 --2- 172.21.15.77:0/1794418062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b38100e80 0x7f8b381992e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f8b2800c010 tx=0x7f8b2800f690 comp rx=0 tx=0).stop 2023-12-01T01:19:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.723+0000 7f8b404f8700 1 -- 172.21.15.77:0/1794418062 shutdown_connections 2023-12-01T01:19:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.723+0000 7f8b404f8700 1 --2- 172.21.15.77:0/1794418062 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8b240a05a0 0x7f8b240a2a60 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.723+0000 7f8b404f8700 1 --2- 172.21.15.77:0/1794418062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b38101820 0x7f8b3819a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.723+0000 7f8b404f8700 1 --2- 172.21.15.77:0/1794418062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b380ffc90 0x7f8b38198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.723+0000 7f8b404f8700 1 --2- 172.21.15.77:0/1794418062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8b38100e80 0x7f8b381992e0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.723+0000 7f8b404f8700 1 -- 172.21.15.77:0/1794418062 >> 172.21.15.77:0/1794418062 conn(0x7f8b380fb1e0 msgr2=0x7f8b380681c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:19:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.724+0000 7f8b404f8700 1 -- 172.21.15.77:0/1794418062 shutdown_connections 2023-12-01T01:19:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:37.724+0000 7f8b404f8700 1 -- 172.21.15.77:0/1794418062 wait complete. 2023-12-01T01:19:37.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:37 smithi077 ceph-mon[98867]: pgmap v350: 97 pgs: 97 active+clean; 41 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 1.3 MiB/s wr, 261 op/s 2023-12-01T01:19:38.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:37 smithi203 ceph-mon[103581]: pgmap v350: 97 pgs: 97 active+clean; 41 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 1.3 MiB/s wr, 261 op/s 2023-12-01T01:19:38.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:37 smithi137 ceph-mon[103312]: pgmap v350: 97 pgs: 97 active+clean; 41 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 1.3 MiB/s wr, 261 op/s 2023-12-01T01:19:40.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:39 smithi203 ceph-mon[103581]: pgmap v351: 97 pgs: 97 active+clean; 41 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 1.5 MiB/s wr, 290 op/s 2023-12-01T01:19:40.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:39 smithi137 ceph-mon[103312]: pgmap v351: 97 pgs: 97 active+clean; 41 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 1.5 MiB/s wr, 290 op/s 2023-12-01T01:19:40.153 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:39 smithi077 ceph-mon[98867]: pgmap v351: 97 pgs: 97 active+clean; 41 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 1.5 MiB/s wr, 290 op/s 2023-12-01T01:19:40.520 INFO:tasks.workunit.client.0.smithi077.stdout:truncating to largest ever: 0xfffc5 2023-12-01T01:19:41.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:40 smithi203 ceph-mon[103581]: scrub summary: active paths [/] 2023-12-01T01:19:41.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:40 smithi137 ceph-mon[103312]: scrub summary: active paths [/] 2023-12-01T01:19:41.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:40 smithi077 ceph-mon[98867]: scrub summary: active paths [/] 2023-12-01T01:19:41.425 INFO:tasks.workunit.client.0.smithi077.stdout:10000 mapread 0x32cdc thru 0x3e88f (0xbbb4 bytes) 2023-12-01T01:19:42.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:41 smithi203 ceph-mon[103581]: scrub summary: idle+waiting paths [/] 2023-12-01T01:19:42.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:41 smithi203 ceph-mon[103581]: pgmap v352: 97 pgs: 97 active+clean; 43 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 1.3 MiB/s wr, 245 op/s 2023-12-01T01:19:42.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:41 smithi137 ceph-mon[103312]: scrub summary: idle+waiting paths [/] 2023-12-01T01:19:42.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:41 smithi137 ceph-mon[103312]: pgmap v352: 97 pgs: 97 active+clean; 43 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 1.3 MiB/s wr, 245 op/s 2023-12-01T01:19:42.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:41 smithi077 ceph-mon[98867]: scrub summary: idle+waiting paths [/] 2023-12-01T01:19:42.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:41 smithi077 ceph-mon[98867]: pgmap v352: 97 pgs: 97 active+clean; 43 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 1.3 MiB/s wr, 245 op/s 2023-12-01T01:19:44.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:43 smithi203 ceph-mon[103581]: pgmap v353: 97 pgs: 97 active+clean; 39 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 1.2 MiB/s wr, 230 op/s 2023-12-01T01:19:44.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:43 smithi137 ceph-mon[103312]: pgmap v353: 97 pgs: 97 active+clean; 39 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 1.2 MiB/s wr, 230 op/s 2023-12-01T01:19:44.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:43 smithi077 ceph-mon[98867]: pgmap v353: 97 pgs: 97 active+clean; 39 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 1.2 MiB/s wr, 230 op/s 2023-12-01T01:19:45.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:44 smithi203 ceph-mon[103581]: pgmap v354: 97 pgs: 97 active+clean; 42 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 557 op/s 2023-12-01T01:19:45.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:44 smithi137 ceph-mon[103312]: pgmap v354: 97 pgs: 97 active+clean; 42 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 557 op/s 2023-12-01T01:19:45.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:44 smithi077 ceph-mon[98867]: pgmap v354: 97 pgs: 97 active+clean; 42 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 557 op/s 2023-12-01T01:19:47.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:47 smithi077 ceph-mon[98867]: pgmap v355: 97 pgs: 97 active+clean; 43 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 617 op/s 2023-12-01T01:19:48.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:47 smithi203 ceph-mon[103581]: pgmap v355: 97 pgs: 97 active+clean; 43 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 617 op/s 2023-12-01T01:19:48.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:47 smithi137 ceph-mon[103312]: pgmap v355: 97 pgs: 97 active+clean; 43 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 617 op/s 2023-12-01T01:19:49.905 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:49 smithi077 ceph-mon[98867]: pgmap v356: 97 pgs: 97 active+clean; 43 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.7 MiB/s wr, 792 op/s 2023-12-01T01:19:50.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:49 smithi203 ceph-mon[103581]: pgmap v356: 97 pgs: 97 active+clean; 43 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.7 MiB/s wr, 792 op/s 2023-12-01T01:19:50.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:49 smithi137 ceph-mon[103312]: pgmap v356: 97 pgs: 97 active+clean; 43 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.7 MiB/s wr, 792 op/s 2023-12-01T01:19:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.858+0000 7f84d3b9b700 1 -- 172.21.15.77:0/3360714292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84cc101820 msgr2=0x7f84cc1080d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.858+0000 7f84d3b9b700 1 --2- 172.21.15.77:0/3360714292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84cc101820 0x7f84cc1080d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f84c4009fc0 tx=0x7f84c400b330 comp rx=0 tx=0).stop 2023-12-01T01:19:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.858+0000 7f84d3b9b700 1 -- 172.21.15.77:0/3360714292 shutdown_connections 2023-12-01T01:19:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.858+0000 7f84d3b9b700 1 --2- 172.21.15.77:0/3360714292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84cc101820 0x7f84cc1080d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.858+0000 7f84d3b9b700 1 --2- 172.21.15.77:0/3360714292 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84cc0ffc90 0x7f84cc1000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.858+0000 7f84d3b9b700 1 --2- 172.21.15.77:0/3360714292 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84cc100e80 0x7f84cc1012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.858+0000 7f84d3b9b700 1 -- 172.21.15.77:0/3360714292 >> 172.21.15.77:0/3360714292 conn(0x7f84cc0fb1e0 msgr2=0x7f84cc0fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:19:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.858+0000 7f84d3b9b700 1 -- 172.21.15.77:0/3360714292 shutdown_connections 2023-12-01T01:19:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.859+0000 7f84d3b9b700 1 -- 172.21.15.77:0/3360714292 wait complete. 2023-12-01T01:19:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.859+0000 7f84d3b9b700 1 Processor -- start 2023-12-01T01:19:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.859+0000 7f84d3b9b700 1 -- start start 2023-12-01T01:19:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.859+0000 7f84d3b9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84cc0ffc90 0x7f84cc102260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.859+0000 7f84d3b9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84cc100e80 0x7f84cc1027a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.859+0000 7f84d3b9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84cc101820 0x7f84cc102ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.859+0000 7f84d3b9b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84cc1a0c10 con 0x7f84cc100e80 2023-12-01T01:19:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.859+0000 7f84d3b9b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84cc1a0d50 con 0x7f84cc101820 2023-12-01T01:19:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.859+0000 7f84d3b9b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84cc1a0e90 con 0x7f84cc0ffc90 2023-12-01T01:19:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.859+0000 7f84d1136700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84cc100e80 0x7f84cc1027a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.859+0000 7f84d1937700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84cc0ffc90 0x7f84cc102260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.859+0000 7f84d2138700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84cc101820 0x7f84cc102ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.859+0000 7f84d1937700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84cc0ffc90 0x7f84cc102260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:46624/0 (socket says 172.21.15.77:46624) 2023-12-01T01:19:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.859+0000 7f84d1136700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84cc100e80 0x7f84cc1027a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60438/0 (socket says 172.21.15.77:60438) 2023-12-01T01:19:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.859+0000 7f84d1937700 1 -- 172.21.15.77:0/2315058553 learned_addr learned my addr 172.21.15.77:0/2315058553 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:19:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.860+0000 7f84d2138700 1 -- 172.21.15.77:0/2315058553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84cc0ffc90 msgr2=0x7f84cc102260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.860+0000 7f84d2138700 1 --2- 172.21.15.77:0/2315058553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84cc0ffc90 0x7f84cc102260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.860+0000 7f84d2138700 1 -- 172.21.15.77:0/2315058553 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84cc100e80 msgr2=0x7f84cc1027a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.860+0000 7f84d2138700 1 --2- 172.21.15.77:0/2315058553 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84cc100e80 0x7f84cc1027a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.860+0000 7f84d2138700 1 -- 172.21.15.77:0/2315058553 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c4009c40 con 0x7f84cc101820 2023-12-01T01:19:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.860+0000 7f84d2138700 1 --2- 172.21.15.77:0/2315058553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84cc101820 0x7f84cc102ce0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f84c400b6c0 tx=0x7f84c40077a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:19:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.860+0000 7f84c2ffd700 1 -- 172.21.15.77:0/2315058553 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c400e040 con 0x7f84cc101820 2023-12-01T01:19:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.860+0000 7f84c2ffd700 1 -- 172.21.15.77:0/2315058553 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84c4004e90 con 0x7f84cc101820 2023-12-01T01:19:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.860+0000 7f84d3b9b700 1 -- 172.21.15.77:0/2315058553 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84cc1a10c0 con 0x7f84cc101820 2023-12-01T01:19:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.860+0000 7f84c2ffd700 1 -- 172.21.15.77:0/2315058553 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c4005040 con 0x7f84cc101820 2023-12-01T01:19:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.860+0000 7f84d3b9b700 1 -- 172.21.15.77:0/2315058553 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84cc1a1540 con 0x7f84cc101820 2023-12-01T01:19:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.862+0000 7f84c2ffd700 1 -- 172.21.15.77:0/2315058553 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f84c401d470 con 0x7f84cc101820 2023-12-01T01:19:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.863+0000 7f84c2ffd700 1 --2- 172.21.15.77:0/2315058553 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f84b80a0520 0x7f84b80a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.863+0000 7f84d1937700 1 --2- 172.21.15.77:0/2315058553 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f84b80a0520 0x7f84b80a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.863+0000 7f84c2ffd700 1 -- 172.21.15.77:0/2315058553 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f84c40ccce0 con 0x7f84cc101820 2023-12-01T01:19:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.864+0000 7f84dfd2bb80 1 Processor -- start 2023-12-01T01:19:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.864+0000 7f84d1937700 1 --2- 172.21.15.77:0/2315058553 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f84b80a0520 0x7f84b80a29e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f84cc103b70 tx=0x7f84c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:19:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.864+0000 7f84dfd2bb80 1 -- start start 2023-12-01T01:19:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.864+0000 7f84dfd2bb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df085c3680 0x55df085c3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.864+0000 7f84dfd2bb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df085c3fe0 0x55df085c48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.864+0000 7f84dfd2bb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df085c4ed0 0x55df085c5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.864+0000 7f84dfd2bb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df0851b9b0 con 0x55df085c3fe0 2023-12-01T01:19:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.864+0000 7f84dfd2bb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df08519370 con 0x55df085c3680 2023-12-01T01:19:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.864+0000 7f84dfd2bb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df08514c60 con 0x55df085c4ed0 2023-12-01T01:19:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.864+0000 7f84d1136700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df085c3fe0 0x55df085c48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.864+0000 7f84d2138700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df085c4ed0 0x55df085c5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.864+0000 7f84d1136700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df085c3fe0 0x55df085c48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60446/0 (socket says 172.21.15.77:60446) 2023-12-01T01:19:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.864+0000 7f84d1937700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df085c3680 0x55df085c3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.864+0000 7f84d1136700 1 -- 172.21.15.77:0/380230740 learned_addr learned my addr 172.21.15.77:0/380230740 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:19:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.865+0000 7f84d2138700 1 -- 172.21.15.77:0/380230740 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df085c3680 msgr2=0x55df085c3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.865+0000 7f84d2138700 1 --2- 172.21.15.77:0/380230740 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df085c3680 0x55df085c3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.865+0000 7f84d2138700 1 -- 172.21.15.77:0/380230740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df085c3fe0 msgr2=0x55df085c48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.865+0000 7f84d2138700 1 --2- 172.21.15.77:0/380230740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df085c3fe0 0x55df085c48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.865+0000 7f84d2138700 1 -- 172.21.15.77:0/380230740 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df08342d80 con 0x55df085c4ed0 2023-12-01T01:19:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.865+0000 7f84d1937700 1 --2- 172.21.15.77:0/380230740 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df085c3680 0x55df085c3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:19:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.865+0000 7f84d1136700 1 --2- 172.21.15.77:0/380230740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df085c3fe0 0x55df085c48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:19:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.865+0000 7f84d2138700 1 --2- 172.21.15.77:0/380230740 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df085c4ed0 0x55df085c5350 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f84c409a2d0 tx=0x7f84c400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1466bf2879bb2c0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:19:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.865+0000 7f84b7fff700 1 -- 172.21.15.77:0/380230740 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c409a150 con 0x55df085c4ed0 2023-12-01T01:19:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.865+0000 7f84b7fff700 1 -- 172.21.15.77:0/380230740 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84c40a0040 con 0x55df085c4ed0 2023-12-01T01:19:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.865+0000 7f84b7fff700 1 -- 172.21.15.77:0/380230740 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c400bd80 con 0x55df085c4ed0 2023-12-01T01:19:50.875 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.866+0000 7f84dfd2bb80 1 -- 172.21.15.77:0/380230740 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df085c4ed0 msgr2=0x55df085c5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:50.875 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.866+0000 7f84dfd2bb80 1 --2- 172.21.15.77:0/380230740 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df085c4ed0 0x55df085c5350 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f84c409a2d0 tx=0x7f84c400b330 comp rx=0 tx=0).stop 2023-12-01T01:19:50.875 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.866+0000 7f84dfd2bb80 1 -- 172.21.15.77:0/380230740 shutdown_connections 2023-12-01T01:19:50.875 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.866+0000 7f84dfd2bb80 1 --2- 172.21.15.77:0/380230740 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df085c4ed0 0x55df085c5350 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:50.875 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.866+0000 7f84dfd2bb80 1 --2- 172.21.15.77:0/380230740 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df085c3680 0x55df085c3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:50.875 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.866+0000 7f84dfd2bb80 1 --2- 172.21.15.77:0/380230740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df085c3fe0 0x55df085c48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:50.875 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.866+0000 7f84dfd2bb80 1 -- 172.21.15.77:0/380230740 >> 172.21.15.77:0/380230740 conn(0x55df08523070 msgr2=0x55df085c0040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:19:50.875 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.866+0000 7f84dfd2bb80 1 -- 172.21.15.77:0/380230740 shutdown_connections 2023-12-01T01:19:50.875 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.866+0000 7f84dfd2bb80 1 -- 172.21.15.77:0/380230740 wait complete. 2023-12-01T01:19:50.875 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.866+0000 7f84dfd2bb80 1 -- start start 2023-12-01T01:19:50.875 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.866+0000 7f84dfd2bb80 1 Processor -- start 2023-12-01T01:19:50.876 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.867+0000 7f84dfd2bb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df085c3680 0x55df085cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:50.876 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.867+0000 7f84dfd2bb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df085c3fe0 0x55df085cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:50.876 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.867+0000 7f84dfd2bb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df085c4ed0 0x55df085d48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:50.876 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.867+0000 7f84dfd2bb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df084b14b0 con 0x55df085c3fe0 2023-12-01T01:19:50.876 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.867+0000 7f84dfd2bb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df084b1150 con 0x55df085c3680 2023-12-01T01:19:50.876 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.867+0000 7f84dfd2bb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df083988c0 con 0x55df085c4ed0 2023-12-01T01:19:50.876 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.867+0000 7f84dfd2bb80 10 client.0 _ll_register_callbacks cb 0x55df083fea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:19:50.876 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.867+0000 7f84d1937700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df085c3680 0x55df085cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:50.876 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.867+0000 7f84d1937700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df085c3680 0x55df085cd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:39372/0 (socket says 172.21.15.77:39372) 2023-12-01T01:19:50.876 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.867+0000 7f84d1937700 1 -- 172.21.15.77:0/2684575083 learned_addr learned my addr 172.21.15.77:0/2684575083 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:19:50.876 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.867+0000 7f84d1136700 1 --2- 172.21.15.77:0/2684575083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df085c3fe0 0x55df085cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:50.876 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.867+0000 7f84d2138700 1 --2- 172.21.15.77:0/2684575083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df085c4ed0 0x55df085d48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:50.876 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.867+0000 7f84d1937700 1 -- 172.21.15.77:0/2684575083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df085c4ed0 msgr2=0x55df085d48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:50.876 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.867+0000 7f84d1937700 1 --2- 172.21.15.77:0/2684575083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df085c4ed0 0x55df085d48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:50.877 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.867+0000 7f84d1937700 1 -- 172.21.15.77:0/2684575083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df085c3fe0 msgr2=0x55df085cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:50.877 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.867+0000 7f84d1937700 1 --2- 172.21.15.77:0/2684575083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df085c3fe0 0x55df085cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:50.877 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.867+0000 7f84d1937700 1 -- 172.21.15.77:0/2684575083 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c8017070 con 0x55df085c3680 2023-12-01T01:19:50.877 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.868+0000 7f84d1937700 1 --2- 172.21.15.77:0/2684575083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df085c3680 0x55df085cd320 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f84c8003920 tx=0x7f84c80047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9904510870b92e75 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:19:50.877 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.868+0000 7f84b57fa700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2023-12-01T01:19:50.877 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.868+0000 7f84b57fa700 1 -- 172.21.15.77:0/2684575083 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c801f940 con 0x55df085c3680 2023-12-01T01:19:50.877 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.868+0000 7f84b57fa700 1 -- 172.21.15.77:0/2684575083 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f84c801faa0 con 0x55df085c3680 2023-12-01T01:19:50.877 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.868+0000 7f84dfd2bb80 1 -- 172.21.15.77:0/2684575083 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df0853fcb0 con 0x55df085c3680 2023-12-01T01:19:50.877 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.868+0000 7f84b57fa700 1 -- 172.21.15.77:0/2684575083 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84c801fc60 con 0x55df085c3680 2023-12-01T01:19:50.877 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.868+0000 7f84dfd2bb80 10 client.25520 fetch_fsmap learned FSMap version 31 2023-12-01T01:19:50.877 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.868+0000 7f84dfd2bb80 1 -- 172.21.15.77:0/2684575083 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f84bc009960 con 0x55df085c3680 2023-12-01T01:19:50.877 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.868+0000 7f84b57fa700 1 -- 172.21.15.77:0/2684575083 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55df0853fcb0 con 0x55df085c3680 2023-12-01T01:19:50.877 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.868+0000 7f84b57fa700 1 -- 172.21.15.77:0/2684575083 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f84c8005470 con 0x55df085c3680 2023-12-01T01:19:50.877 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.869+0000 7f84dfd2bb80 10 client.25520 fetch_fsmap finished waiting for FSMap version 31 2023-12-01T01:19:50.877 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.869+0000 7f84dfd2bb80 10 client.25520 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-01T01:19:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.869+0000 7f84dfd2bb80 20 client.25520 populate_metadata read hostname 'smithi077' 2023-12-01T01:19:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.869+0000 7f84dfd2bb80 1 --2- 172.21.15.77:0/2684575083 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55df085d9260 0x55df085db680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.869+0000 7f84dfd2bb80 4 client.25520 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:19:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.869+0000 7f84dfd2bb80 1 -- 172.21.15.77:0/2684575083 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df08592db0 con 0x55df085d9260 2023-12-01T01:19:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.869+0000 7f84d1136700 1 --2- 172.21.15.77:0/2684575083 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55df085d9260 0x55df085db680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.869+0000 7f84b57fa700 10 client.25520 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:19:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.869+0000 7f84d1136700 1 --2- 172.21.15.77:0/2684575083 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55df085d9260 0x55df085db680 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c600ac381455cbc7 server_cookie=b7e69cde2af1fda5 in_seq=0 out_seq=0 2023-12-01T01:19:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.870+0000 7f84b57fa700 1 -- 172.21.15.77:0/2684575083 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55df08592db0 con 0x55df085d9260 2023-12-01T01:19:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.870+0000 7f84b57fa700 10 client.25520 handle_command_reply: tid=0 2023-12-01T01:19:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.870+0000 7f84dfd2bb80 1 client.25520 shutdown 2023-12-01T01:19:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.870+0000 7f84dfd2bb80 1 -- 172.21.15.77:0/2684575083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df085c3680 msgr2=0x55df085cd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.870+0000 7f84dfd2bb80 1 --2- 172.21.15.77:0/2684575083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df085c3680 0x55df085cd320 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f84c8003920 tx=0x7f84c80047c0 comp rx=0 tx=0).stop 2023-12-01T01:19:50.878 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.870+0000 7f84dfd2bb80 1 -- 172.21.15.77:0/2684575083 shutdown_connections 2023-12-01T01:19:50.879 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.870+0000 7f84dfd2bb80 1 --2- 172.21.15.77:0/2684575083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df085c4ed0 0x55df085d48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:50.879 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.870+0000 7f84dfd2bb80 1 --2- 172.21.15.77:0/2684575083 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55df085d9260 0x55df085db680 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:50.879 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.870+0000 7f84dfd2bb80 1 --2- 172.21.15.77:0/2684575083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df085c3680 0x55df085cd320 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f84c8003920 tx=0x7f84c80047c0 comp rx=0 tx=0).stop 2023-12-01T01:19:50.879 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.870+0000 7f84dfd2bb80 1 --2- 172.21.15.77:0/2684575083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df085c3fe0 0x55df085cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:50.879 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.870+0000 7f84b57fa700 0 client.25520 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:19:50.879 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.870+0000 7f84dfd2bb80 1 -- 172.21.15.77:0/2684575083 >> 172.21.15.77:0/2684575083 conn(0x55df08523070 msgr2=0x55df085be3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:19:50.879 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.871+0000 7f84dfd2bb80 1 -- 172.21.15.77:0/2684575083 shutdown_connections 2023-12-01T01:19:50.879 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.871+0000 7f84dfd2bb80 1 -- 172.21.15.77:0/2684575083 wait complete. 2023-12-01T01:19:50.880 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.871+0000 7f84dfd2bb80 20 client.25520 trim_cache size 0 max 16384 2023-12-01T01:19:50.891 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.889+0000 7f84dfd2bb80 1 Processor -- start 2023-12-01T01:19:50.891 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.889+0000 7f84dfd2bb80 1 -- start start 2023-12-01T01:19:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.889+0000 7f84dfd2bb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df085c3fe0 0x55df085c3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.889+0000 7f84dfd2bb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df085c4ed0 0x55df085d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.889+0000 7f84dfd2bb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df085cc9d0 0x55df085cb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.889+0000 7f84dfd2bb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df085b9500 con 0x55df085cc9d0 2023-12-01T01:19:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.889+0000 7f84dfd2bb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df085bf6e0 con 0x55df085c3fe0 2023-12-01T01:19:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.889+0000 7f84dfd2bb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df085cce50 con 0x55df085c4ed0 2023-12-01T01:19:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.889+0000 7f84d2138700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df085cc9d0 0x55df085cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.889+0000 7f84d1937700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df085c3fe0 0x55df085c3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.889+0000 7f84d1136700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df085c4ed0 0x55df085d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:50.892 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.889+0000 7f84d2138700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df085cc9d0 0x55df085cb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60462/0 (socket says 172.21.15.77:60462) 2023-12-01T01:19:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.889+0000 7f84d2138700 1 -- 172.21.15.77:0/4007385368 learned_addr learned my addr 172.21.15.77:0/4007385368 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:19:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.889+0000 7f84d2138700 1 -- 172.21.15.77:0/4007385368 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df085c4ed0 msgr2=0x55df085d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.889+0000 7f84d2138700 1 --2- 172.21.15.77:0/4007385368 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df085c4ed0 0x55df085d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.890+0000 7f84d2138700 1 -- 172.21.15.77:0/4007385368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df085c3fe0 msgr2=0x55df085c3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.890+0000 7f84d2138700 1 --2- 172.21.15.77:0/4007385368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df085c3fe0 0x55df085c3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.890+0000 7f84d2138700 1 -- 172.21.15.77:0/4007385368 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84cc1a1540 con 0x55df085cc9d0 2023-12-01T01:19:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.890+0000 7f84d1937700 1 --2- 172.21.15.77:0/4007385368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df085c3fe0 0x55df085c3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:19:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.890+0000 7f84d2138700 1 --2- 172.21.15.77:0/4007385368 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df085cc9d0 0x55df085cb320 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f84c401db10 tx=0x7f84c401d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f1f5df66c2bbef server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:19:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.890+0000 7f84c0da9700 1 -- 172.21.15.77:0/4007385368 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c4099f30 con 0x55df085cc9d0 2023-12-01T01:19:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.890+0000 7f84c0da9700 1 -- 172.21.15.77:0/4007385368 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84c409e400 con 0x55df085cc9d0 2023-12-01T01:19:50.893 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.890+0000 7f84c0da9700 1 -- 172.21.15.77:0/4007385368 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c4098970 con 0x55df085cc9d0 2023-12-01T01:19:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.891+0000 7f84dfd2bb80 1 -- 172.21.15.77:0/4007385368 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df085cc9d0 msgr2=0x55df085cb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.891+0000 7f84dfd2bb80 1 --2- 172.21.15.77:0/4007385368 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df085cc9d0 0x55df085cb320 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f84c401db10 tx=0x7f84c401d9d0 comp rx=0 tx=0).stop 2023-12-01T01:19:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.891+0000 7f84dfd2bb80 1 -- 172.21.15.77:0/4007385368 shutdown_connections 2023-12-01T01:19:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.891+0000 7f84dfd2bb80 1 --2- 172.21.15.77:0/4007385368 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df085cc9d0 0x55df085cb320 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.891+0000 7f84dfd2bb80 1 --2- 172.21.15.77:0/4007385368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df085c3fe0 0x55df085c3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.891+0000 7f84dfd2bb80 1 --2- 172.21.15.77:0/4007385368 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df085c4ed0 0x55df085d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.891+0000 7f84dfd2bb80 1 -- 172.21.15.77:0/4007385368 >> 172.21.15.77:0/4007385368 conn(0x55df08523070 msgr2=0x55df085cbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:19:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.891+0000 7f84dfd2bb80 1 -- 172.21.15.77:0/4007385368 shutdown_connections 2023-12-01T01:19:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.891+0000 7f84dfd2bb80 1 -- 172.21.15.77:0/4007385368 wait complete. 2023-12-01T01:19:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.891+0000 7f84dfd2bb80 1 -- start start 2023-12-01T01:19:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.891+0000 7f84dfd2bb80 1 Processor -- start 2023-12-01T01:19:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.892+0000 7f84dfd2bb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df085c3fe0 0x55df08624d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.892+0000 7f84dfd2bb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df085c4ed0 0x55df08627250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.892+0000 7f84dfd2bb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df085cc9d0 0x55df08629850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.892+0000 7f84dfd2bb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df08629f60 con 0x55df085c3fe0 2023-12-01T01:19:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.892+0000 7f84dfd2bb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df0862a0d0 con 0x55df085cc9d0 2023-12-01T01:19:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.892+0000 7f84dfd2bb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df0862a290 con 0x55df085c4ed0 2023-12-01T01:19:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.892+0000 7f84dfd2bb80 10 client.0 _ll_register_callbacks cb 0x55df083fea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:19:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.892+0000 7f84d1937700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df085c3fe0 0x55df08624d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.892+0000 7f84d1136700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df085c4ed0 0x55df08627250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.892+0000 7f84d1937700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df085c3fe0 0x55df08624d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60464/0 (socket says 172.21.15.77:60464) 2023-12-01T01:19:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.892+0000 7f84d1937700 1 -- 172.21.15.77:0/4038312580 learned_addr learned my addr 172.21.15.77:0/4038312580 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:19:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.892+0000 7f84d2138700 1 --2- 172.21.15.77:0/4038312580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df085cc9d0 0x55df08629850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.892+0000 7f84d1937700 1 -- 172.21.15.77:0/4038312580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df085c4ed0 msgr2=0x55df08627250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.892+0000 7f84d1937700 1 --2- 172.21.15.77:0/4038312580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df085c4ed0 0x55df08627250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.892+0000 7f84d1937700 1 -- 172.21.15.77:0/4038312580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df085cc9d0 msgr2=0x55df08629850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.892+0000 7f84d1937700 1 --2- 172.21.15.77:0/4038312580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df085cc9d0 0x55df08629850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.892+0000 7f84d1937700 1 -- 172.21.15.77:0/4038312580 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84bc009960 con 0x55df085c3fe0 2023-12-01T01:19:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.892+0000 7f84d1136700 1 --2- 172.21.15.77:0/4038312580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df085c4ed0 0x55df08627250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:19:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.892+0000 7f84d2138700 1 --2- 172.21.15.77:0/4038312580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df085cc9d0 0x55df08629850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:19:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.892+0000 7f84d1937700 1 --2- 172.21.15.77:0/4038312580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df085c3fe0 0x55df08624d00 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f84c80040b0 tx=0x7f84c8028b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5580742f72d302a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:19:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.892+0000 7f84b57fa700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:19:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.893+0000 7f84b57fa700 1 -- 172.21.15.77:0/4038312580 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c801f3d0 con 0x55df085c3fe0 2023-12-01T01:19:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.893+0000 7f84b57fa700 1 -- 172.21.15.77:0/4038312580 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f84c802e070 con 0x55df085c3fe0 2023-12-01T01:19:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.893+0000 7f84dfd2bb80 1 -- 172.21.15.77:0/4038312580 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f84c40a0040 con 0x55df085c3fe0 2023-12-01T01:19:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.893+0000 7f84b57fa700 1 -- 172.21.15.77:0/4038312580 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84c801f940 con 0x55df085c3fe0 2023-12-01T01:19:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.893+0000 7f84dfd2bb80 10 client.15945 fetch_fsmap learned FSMap version 31 2023-12-01T01:19:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.893+0000 7f84dfd2bb80 1 -- 172.21.15.77:0/4038312580 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55df08342d80 con 0x55df085c3fe0 2023-12-01T01:19:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.893+0000 7f84b57fa700 1 -- 172.21.15.77:0/4038312580 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84c40a0040 con 0x55df085c3fe0 2023-12-01T01:19:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.894+0000 7f84b57fa700 1 -- 172.21.15.77:0/4038312580 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f84c803bb40 con 0x55df085c3fe0 2023-12-01T01:19:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.894+0000 7f84dfd2bb80 10 client.15945 fetch_fsmap finished waiting for FSMap version 31 2023-12-01T01:19:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.894+0000 7f84dfd2bb80 10 client.15945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-01T01:19:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.894+0000 7f84dfd2bb80 20 client.15945 populate_metadata read hostname 'smithi077' 2023-12-01T01:19:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.894+0000 7f84dfd2bb80 1 --2- 172.21.15.77:0/4038312580 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55df0862bc00 0x55df0862e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.894+0000 7f84dfd2bb80 4 client.15945 mds_command: new command op to 24500 tid=0[{"prefix": "flush journal"}] 2023-12-01T01:19:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.894+0000 7f84dfd2bb80 1 -- 172.21.15.77:0/4038312580 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55df0850dce0 con 0x55df0862bc00 2023-12-01T01:19:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.894+0000 7f84d1136700 1 --2- 172.21.15.77:0/4038312580 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55df0862bc00 0x55df0862e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:50.900 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.894+0000 7f84b57fa700 10 client.15945 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:19:50.901 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.894+0000 7f84d1136700 1 --2- 172.21.15.77:0/4038312580 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55df0862bc00 0x55df0862e020 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7433e910a7468fb5 server_cookie=34c183b1c654e0de in_seq=0 out_seq=0 2023-12-01T01:19:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.912+0000 7f84b57fa700 1 -- 172.21.15.77:0/4038312580 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55df0850dce0 con 0x55df0862bc00 2023-12-01T01:19:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.912+0000 7f84b57fa700 10 client.15945 handle_command_reply: tid=0 2023-12-01T01:19:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.912+0000 7f84dfd2bb80 1 client.15945 shutdown 2023-12-01T01:19:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.912+0000 7f84dfd2bb80 1 -- 172.21.15.77:0/4038312580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df085c3fe0 msgr2=0x55df08624d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.912+0000 7f84dfd2bb80 1 --2- 172.21.15.77:0/4038312580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df085c3fe0 0x55df08624d00 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f84c80040b0 tx=0x7f84c8028b70 comp rx=0 tx=0).stop 2023-12-01T01:19:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.913+0000 7f84dfd2bb80 1 -- 172.21.15.77:0/4038312580 shutdown_connections 2023-12-01T01:19:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.913+0000 7f84dfd2bb80 1 --2- 172.21.15.77:0/4038312580 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55df0862bc00 0x55df0862e020 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.913+0000 7f84dfd2bb80 1 --2- 172.21.15.77:0/4038312580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df085cc9d0 0x55df08629850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.913+0000 7f84b57fa700 0 client.15945 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:19:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.913+0000 7f84dfd2bb80 1 --2- 172.21.15.77:0/4038312580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df085c3fe0 0x55df08624d00 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.913+0000 7f84dfd2bb80 1 --2- 172.21.15.77:0/4038312580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55df085c4ed0 0x55df08627250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.913+0000 7f84dfd2bb80 1 -- 172.21.15.77:0/4038312580 >> 172.21.15.77:0/4038312580 conn(0x55df08523070 msgr2=0x55df085cb320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:19:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.913+0000 7f84dfd2bb80 1 -- 172.21.15.77:0/4038312580 shutdown_connections 2023-12-01T01:19:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.913+0000 7f84dfd2bb80 1 -- 172.21.15.77:0/4038312580 wait complete. 2023-12-01T01:19:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.913+0000 7f84dfd2bb80 20 client.15945 trim_cache size 0 max 16384 2023-12-01T01:19:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout:{ 2023-12-01T01:19:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout: "message": "", 2023-12-01T01:19:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout: "return_code": 0 2023-12-01T01:19:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout:} 2023-12-01T01:19:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.913+0000 7f84d3b9b700 1 -- 172.21.15.77:0/2315058553 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f84b80a0520 msgr2=0x7f84b80a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.913+0000 7f84d3b9b700 1 --2- 172.21.15.77:0/2315058553 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f84b80a0520 0x7f84b80a29e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f84cc103b70 tx=0x7f84c8009260 comp rx=0 tx=0).stop 2023-12-01T01:19:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.913+0000 7f84d3b9b700 1 -- 172.21.15.77:0/2315058553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84cc101820 msgr2=0x7f84cc102ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.913+0000 7f84d3b9b700 1 --2- 172.21.15.77:0/2315058553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84cc101820 0x7f84cc102ce0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f84c400b6c0 tx=0x7f84c40077a0 comp rx=0 tx=0).stop 2023-12-01T01:19:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.913+0000 7f84d3b9b700 1 -- 172.21.15.77:0/2315058553 shutdown_connections 2023-12-01T01:19:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.913+0000 7f84d3b9b700 1 --2- 172.21.15.77:0/2315058553 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f84b80a0520 0x7f84b80a29e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.913+0000 7f84d3b9b700 1 --2- 172.21.15.77:0/2315058553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84cc101820 0x7f84cc102ce0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.913+0000 7f84d3b9b700 1 --2- 172.21.15.77:0/2315058553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84cc0ffc90 0x7f84cc102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.913+0000 7f84d3b9b700 1 --2- 172.21.15.77:0/2315058553 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84cc100e80 0x7f84cc1027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.914+0000 7f84d3b9b700 1 -- 172.21.15.77:0/2315058553 >> 172.21.15.77:0/2315058553 conn(0x7f84cc0fb1e0 msgr2=0x7f84cc068260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:19:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.914+0000 7f84d3b9b700 1 -- 172.21.15.77:0/2315058553 shutdown_connections 2023-12-01T01:19:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:19:50.914+0000 7f84d3b9b700 1 -- 172.21.15.77:0/2315058553 wait complete. 2023-12-01T01:19:51.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:51 smithi077 ceph-mon[98867]: pgmap v357: 97 pgs: 97 active+clean; 43 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 674 op/s 2023-12-01T01:19:52.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:51 smithi203 ceph-mon[103581]: pgmap v357: 97 pgs: 97 active+clean; 43 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 674 op/s 2023-12-01T01:19:52.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:51 smithi137 ceph-mon[103312]: pgmap v357: 97 pgs: 97 active+clean; 43 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 674 op/s 2023-12-01T01:19:52.492 INFO:tasks.workunit.client.0.smithi077.stdout:truncating to largest ever: 0xfffe8 2023-12-01T01:19:53.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:53 smithi077 ceph-mon[98867]: pgmap v358: 97 pgs: 97 active+clean; 45 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 711 op/s 2023-12-01T01:19:54.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:53 smithi203 ceph-mon[103581]: pgmap v358: 97 pgs: 97 active+clean; 45 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 711 op/s 2023-12-01T01:19:54.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:53 smithi137 ceph-mon[103312]: pgmap v358: 97 pgs: 97 active+clean; 45 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 711 op/s 2023-12-01T01:19:56.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:55 smithi203 ceph-mon[103581]: pgmap v359: 97 pgs: 97 active+clean; 43 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.3 MiB/s wr, 930 op/s 2023-12-01T01:19:56.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:55 smithi137 ceph-mon[103312]: pgmap v359: 97 pgs: 97 active+clean; 43 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.3 MiB/s wr, 930 op/s 2023-12-01T01:19:56.155 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:55 smithi077 ceph-mon[98867]: pgmap v359: 97 pgs: 97 active+clean; 43 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.3 MiB/s wr, 930 op/s 2023-12-01T01:19:57.272 INFO:tasks.workunit.client.0.smithi077.stdout:30000 trunc from 0xf3838 to 0x7d16c 2023-12-01T01:19:58.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:57 smithi203 ceph-mon[103581]: pgmap v360: 97 pgs: 97 active+clean; 43 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 611 op/s 2023-12-01T01:19:58.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:57 smithi137 ceph-mon[103312]: pgmap v360: 97 pgs: 97 active+clean; 43 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 611 op/s 2023-12-01T01:19:58.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:57 smithi077 ceph-mon[98867]: pgmap v360: 97 pgs: 97 active+clean; 43 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 611 op/s 2023-12-01T01:19:58.834 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-01T01:19:58.985 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.983+0000 7f29ca104700 1 -- 172.21.15.77:0/1838875133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29c4106b70 msgr2=0x7f29c4108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:58.985 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.983+0000 7f29ca104700 1 --2- 172.21.15.77:0/1838875133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29c4106b70 0x7f29c4108f60 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f29bc00b0a0 tx=0x7f29bc00b3b0 comp rx=0 tx=0).stop 2023-12-01T01:19:58.985 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.984+0000 7f29ca104700 1 -- 172.21.15.77:0/1838875133 shutdown_connections 2023-12-01T01:19:58.985 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.984+0000 7f29ca104700 1 --2- 172.21.15.77:0/1838875133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29c4106b70 0x7f29c4108f60 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:58.986 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.984+0000 7f29ca104700 1 --2- 172.21.15.77:0/1838875133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29c4101910 0x7f29c4103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:58.986 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.984+0000 7f29ca104700 1 --2- 172.21.15.77:0/1838875133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29c4104240 0x7f29c4106630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:58.986 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.984+0000 7f29ca104700 1 -- 172.21.15.77:0/1838875133 >> 172.21.15.77:0/1838875133 conn(0x7f29c40fb260 msgr2=0x7f29c40fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:19:58.986 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.984+0000 7f29ca104700 1 -- 172.21.15.77:0/1838875133 shutdown_connections 2023-12-01T01:19:58.986 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.984+0000 7f29ca104700 1 -- 172.21.15.77:0/1838875133 wait complete. 2023-12-01T01:19:58.986 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.984+0000 7f29ca104700 1 Processor -- start 2023-12-01T01:19:58.988 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.985+0000 7f29ca104700 1 -- start start 2023-12-01T01:19:58.988 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.985+0000 7f29ca104700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29c4101910 0x7f29c4196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:58.988 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.985+0000 7f29ca104700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29c4104240 0x7f29c41970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:58.989 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.985+0000 7f29ca104700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29c4106b70 0x7f29c419e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:58.989 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.985+0000 7f29ca104700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c419e6f0 con 0x7f29c4101910 2023-12-01T01:19:58.989 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.985+0000 7f29ca104700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c419e830 con 0x7f29c4104240 2023-12-01T01:19:58.989 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.985+0000 7f29ca104700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c419e9c0 con 0x7f29c4106b70 2023-12-01T01:19:58.989 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.985+0000 7f29c37fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29c4101910 0x7f29c4196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:58.989 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.985+0000 7f29c3fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29c4106b70 0x7f29c419e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:58.989 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.985+0000 7f29c3fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29c4106b70 0x7f29c419e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:46668/0 (socket says 172.21.15.77:46668) 2023-12-01T01:19:58.989 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.985+0000 7f29c37fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29c4101910 0x7f29c4196b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60478/0 (socket says 172.21.15.77:60478) 2023-12-01T01:19:58.989 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.985+0000 7f29c3fff700 1 -- 172.21.15.77:0/3893962629 learned_addr learned my addr 172.21.15.77:0/3893962629 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:19:58.989 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.985+0000 7f29c2ffd700 1 --2- 172.21.15.77:0/3893962629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29c4104240 0x7f29c41970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:58.989 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.985+0000 7f29c3fff700 1 -- 172.21.15.77:0/3893962629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29c4104240 msgr2=0x7f29c41970b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:58.989 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.985+0000 7f29c3fff700 1 --2- 172.21.15.77:0/3893962629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29c4104240 0x7f29c41970b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:58.990 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.985+0000 7f29c3fff700 1 -- 172.21.15.77:0/3893962629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29c4101910 msgr2=0x7f29c4196b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:58.990 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.985+0000 7f29c3fff700 1 --2- 172.21.15.77:0/3893962629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29c4101910 0x7f29c4196b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:58.990 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.985+0000 7f29c3fff700 1 -- 172.21.15.77:0/3893962629 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29bc009d00 con 0x7f29c4106b70 2023-12-01T01:19:58.990 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.985+0000 7f29c2ffd700 1 --2- 172.21.15.77:0/3893962629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29c4104240 0x7f29c41970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:19:58.990 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.986+0000 7f29c3fff700 1 --2- 172.21.15.77:0/3893962629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29c4106b70 0x7f29c419e120 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f29bc009fd0 tx=0x7f29bc00bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:19:58.990 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.986+0000 7f29c0ff9700 1 -- 172.21.15.77:0/3893962629 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc00e040 con 0x7f29c4106b70 2023-12-01T01:19:58.990 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.986+0000 7f29ca104700 1 -- 172.21.15.77:0/3893962629 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c419ec40 con 0x7f29c4106b70 2023-12-01T01:19:58.990 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.986+0000 7f29c0ff9700 1 -- 172.21.15.77:0/3893962629 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29bc0094a0 con 0x7f29c4106b70 2023-12-01T01:19:58.990 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.986+0000 7f29c0ff9700 1 -- 172.21.15.77:0/3893962629 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc01b6a0 con 0x7f29c4106b70 2023-12-01T01:19:58.990 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.986+0000 7f29ca104700 1 -- 172.21.15.77:0/3893962629 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c419f0c0 con 0x7f29c4106b70 2023-12-01T01:19:58.993 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.988+0000 7f29c0ff9700 1 -- 172.21.15.77:0/3893962629 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f29bc01d470 con 0x7f29c4106b70 2023-12-01T01:19:58.993 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.989+0000 7f29c0ff9700 1 --2- 172.21.15.77:0/3893962629 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f29ac0a0520 0x7f29ac0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:58.994 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.989+0000 7f29c37fe700 1 --2- 172.21.15.77:0/3893962629 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f29ac0a0520 0x7f29ac0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:58.994 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.989+0000 7f29c0ff9700 1 -- 172.21.15.77:0/3893962629 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f29bc0cca20 con 0x7f29c4106b70 2023-12-01T01:19:58.994 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.990+0000 7f29d6294b80 1 Processor -- start 2023-12-01T01:19:58.994 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.990+0000 7f29c37fe700 1 --2- 172.21.15.77:0/3893962629 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f29ac0a0520 0x7f29ac0a29e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f29b4009c60 tx=0x7f29b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:19:58.994 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.990+0000 7f29d6294b80 1 -- start start 2023-12-01T01:19:58.994 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.990+0000 7f29d6294b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8602690f0 0x55f860269530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:58.994 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.990+0000 7f29d6294b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f860269a70 0x55f860267e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:58.994 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.990+0000 7f29d6294b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8602683f0 0x55f860268870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:58.994 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.990+0000 7f29d6294b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8601bfa10 con 0x55f8602690f0 2023-12-01T01:19:58.994 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.990+0000 7f29d6294b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8601bd3d0 con 0x55f860269a70 2023-12-01T01:19:58.994 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.990+0000 7f29d6294b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8601b8cc0 con 0x55f8602683f0 2023-12-01T01:19:58.994 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.990+0000 7f29c37fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8602690f0 0x55f860269530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:58.995 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.990+0000 7f29c37fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8602690f0 0x55f860269530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60484/0 (socket says 172.21.15.77:60484) 2023-12-01T01:19:58.995 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.990+0000 7f29c37fe700 1 -- 172.21.15.77:0/1843525413 learned_addr learned my addr 172.21.15.77:0/1843525413 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:19:58.995 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.990+0000 7f29c2ffd700 1 --2- 172.21.15.77:0/1843525413 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f860269a70 0x55f860267e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:58.995 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.990+0000 7f29c3fff700 1 --2- 172.21.15.77:0/1843525413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8602683f0 0x55f860268870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:58.995 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.990+0000 7f29c37fe700 1 -- 172.21.15.77:0/1843525413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8602683f0 msgr2=0x55f860268870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:58.995 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.990+0000 7f29c37fe700 1 --2- 172.21.15.77:0/1843525413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8602683f0 0x55f860268870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:58.995 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.990+0000 7f29c37fe700 1 -- 172.21.15.77:0/1843525413 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f860269a70 msgr2=0x55f860267e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:58.996 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.990+0000 7f29c37fe700 1 --2- 172.21.15.77:0/1843525413 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f860269a70 0x55f860267e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:58.996 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.990+0000 7f29c37fe700 1 -- 172.21.15.77:0/1843525413 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f85ffe6de0 con 0x55f8602690f0 2023-12-01T01:19:58.996 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.990+0000 7f29c2ffd700 1 --2- 172.21.15.77:0/1843525413 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f860269a70 0x55f860267e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:19:58.996 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.991+0000 7f29c37fe700 1 --2- 172.21.15.77:0/1843525413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8602690f0 0x55f860269530 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f29b400e4e0 tx=0x7f29b4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf89ae4351fbbbac server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:19:58.996 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.991+0000 7f29a9ffb700 1 -- 172.21.15.77:0/1843525413 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b400ee00 con 0x55f8602690f0 2023-12-01T01:19:58.996 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.991+0000 7f29a9ffb700 1 -- 172.21.15.77:0/1843525413 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29b4021070 con 0x55f8602690f0 2023-12-01T01:19:58.996 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.991+0000 7f29a9ffb700 1 -- 172.21.15.77:0/1843525413 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b401d7b0 con 0x55f8602690f0 2023-12-01T01:19:58.996 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.992+0000 7f29d6294b80 1 -- 172.21.15.77:0/1843525413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8602690f0 msgr2=0x55f860269530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:58.996 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.992+0000 7f29d6294b80 1 --2- 172.21.15.77:0/1843525413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8602690f0 0x55f860269530 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f29b400e4e0 tx=0x7f29b4014000 comp rx=0 tx=0).stop 2023-12-01T01:19:58.996 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.992+0000 7f29d6294b80 1 -- 172.21.15.77:0/1843525413 shutdown_connections 2023-12-01T01:19:58.996 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.992+0000 7f29d6294b80 1 --2- 172.21.15.77:0/1843525413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8602683f0 0x55f860268870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:58.996 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.992+0000 7f29d6294b80 1 --2- 172.21.15.77:0/1843525413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8602690f0 0x55f860269530 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:58.997 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.992+0000 7f29d6294b80 1 --2- 172.21.15.77:0/1843525413 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f860269a70 0x55f860267e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:58.997 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.992+0000 7f29d6294b80 1 -- 172.21.15.77:0/1843525413 >> 172.21.15.77:0/1843525413 conn(0x55f8601c70d0 msgr2=0x55f860262940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:19:58.997 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.992+0000 7f29d6294b80 1 -- 172.21.15.77:0/1843525413 shutdown_connections 2023-12-01T01:19:58.997 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.992+0000 7f29d6294b80 1 -- 172.21.15.77:0/1843525413 wait complete. 2023-12-01T01:19:58.997 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.992+0000 7f29d6294b80 1 -- start start 2023-12-01T01:19:58.997 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.993+0000 7f29d6294b80 1 Processor -- start 2023-12-01T01:19:59.001 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.993+0000 7f29d6294b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8602683f0 0x55f8602719e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:59.001 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.993+0000 7f29d6294b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8602690f0 0x55f860271f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:59.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.993+0000 7f29d6294b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f860269a70 0x55f860278f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:59.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.993+0000 7f29d6294b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f860155510 con 0x55f8602690f0 2023-12-01T01:19:59.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.993+0000 7f29d6294b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8601551b0 con 0x55f860269a70 2023-12-01T01:19:59.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.993+0000 7f29d6294b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f86003c920 con 0x55f8602683f0 2023-12-01T01:19:59.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.993+0000 7f29d6294b80 10 client.0 _ll_register_callbacks cb 0x55f8600a2a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:19:59.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.993+0000 7f29c2ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8602690f0 0x55f860271f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:59.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.993+0000 7f29c2ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8602690f0 0x55f860271f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60492/0 (socket says 172.21.15.77:60492) 2023-12-01T01:19:59.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.993+0000 7f29c2ffd700 1 -- 172.21.15.77:0/3001711271 learned_addr learned my addr 172.21.15.77:0/3001711271 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:19:59.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.993+0000 7f29c37fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8602683f0 0x55f8602719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:59.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.993+0000 7f29c3fff700 1 --2- 172.21.15.77:0/3001711271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f860269a70 0x55f860278f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:59.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.993+0000 7f29c2ffd700 1 -- 172.21.15.77:0/3001711271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8602683f0 msgr2=0x55f8602719e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:59.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.993+0000 7f29c2ffd700 1 --2- 172.21.15.77:0/3001711271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8602683f0 0x55f8602719e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:59.002 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.993+0000 7f29c2ffd700 1 -- 172.21.15.77:0/3001711271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f860269a70 msgr2=0x55f860278f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:59.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.993+0000 7f29c2ffd700 1 --2- 172.21.15.77:0/3001711271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f860269a70 0x55f860278f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:59.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.993+0000 7f29c2ffd700 1 -- 172.21.15.77:0/3001711271 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29b80098f0 con 0x55f8602690f0 2023-12-01T01:19:59.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.993+0000 7f29c37fe700 1 --2- 172.21.15.77:0/3001711271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8602683f0 0x55f8602719e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:19:59.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.994+0000 7f29c2ffd700 1 --2- 172.21.15.77:0/3001711271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8602690f0 0x55f860271f20 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f29b8012a10 tx=0x7f29b8012dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fae5ddba26ec5ae8 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:19:59.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.994+0000 7f29a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:19:59.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.994+0000 7f29a2ffd700 1 -- 172.21.15.77:0/3001711271 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b8017070 con 0x55f8602690f0 2023-12-01T01:19:59.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.994+0000 7f29a2ffd700 1 -- 172.21.15.77:0/3001711271 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f29b8011b00 con 0x55f8602690f0 2023-12-01T01:19:59.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.994+0000 7f29d6294b80 1 -- 172.21.15.77:0/3001711271 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f8601e3d10 con 0x55f8602690f0 2023-12-01T01:19:59.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.994+0000 7f29a2ffd700 1 -- 172.21.15.77:0/3001711271 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29b8011d00 con 0x55f8602690f0 2023-12-01T01:19:59.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.994+0000 7f29d6294b80 10 client.15957 fetch_fsmap learned FSMap version 31 2023-12-01T01:19:59.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.994+0000 7f29d6294b80 1 -- 172.21.15.77:0/3001711271 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f29b4017070 con 0x55f8602690f0 2023-12-01T01:19:59.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.994+0000 7f29a2ffd700 1 -- 172.21.15.77:0/3001711271 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f8601e3d10 con 0x55f8602690f0 2023-12-01T01:19:59.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.994+0000 7f29a2ffd700 1 -- 172.21.15.77:0/3001711271 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f29b8003c60 con 0x55f8602690f0 2023-12-01T01:19:59.003 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.994+0000 7f29d6294b80 10 client.15957 fetch_fsmap finished waiting for FSMap version 31 2023-12-01T01:19:59.004 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.995+0000 7f29d6294b80 10 client.15957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-01T01:19:59.004 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.995+0000 7f29d6294b80 20 client.15957 populate_metadata read hostname 'smithi077' 2023-12-01T01:19:59.004 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.995+0000 7f29d6294b80 1 --2- 172.21.15.77:0/3001711271 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55f86027d820 0x55f86027fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:59.004 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.995+0000 7f29d6294b80 4 client.15957 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:19:59.004 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.995+0000 7f29d6294b80 1 -- 172.21.15.77:0/3001711271 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f860236e10 con 0x55f86027d820 2023-12-01T01:19:59.004 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.995+0000 7f29c37fe700 1 --2- 172.21.15.77:0/3001711271 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55f86027d820 0x55f86027fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:59.004 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.995+0000 7f29a2ffd700 10 client.15957 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:19:59.004 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.995+0000 7f29c37fe700 1 --2- 172.21.15.77:0/3001711271 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55f86027d820 0x55f86027fc40 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12a360ce7dc3fbf6 server_cookie=6956bd6655ffa5c7 in_seq=0 out_seq=0 2023-12-01T01:19:59.004 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.996+0000 7f29a2ffd700 1 -- 172.21.15.77:0/3001711271 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55f860236e10 con 0x55f86027d820 2023-12-01T01:19:59.004 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.996+0000 7f29a2ffd700 10 client.15957 handle_command_reply: tid=0 2023-12-01T01:19:59.004 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.996+0000 7f29d6294b80 1 client.15957 shutdown 2023-12-01T01:19:59.004 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.996+0000 7f29d6294b80 1 -- 172.21.15.77:0/3001711271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8602690f0 msgr2=0x55f860271f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:59.005 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.996+0000 7f29d6294b80 1 --2- 172.21.15.77:0/3001711271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8602690f0 0x55f860271f20 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f29b8012a10 tx=0x7f29b8012dd0 comp rx=0 tx=0).stop 2023-12-01T01:19:59.005 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.996+0000 7f29d6294b80 1 -- 172.21.15.77:0/3001711271 shutdown_connections 2023-12-01T01:19:59.005 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.996+0000 7f29d6294b80 1 --2- 172.21.15.77:0/3001711271 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55f86027d820 0x55f86027fc40 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:59.005 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.996+0000 7f29d6294b80 1 --2- 172.21.15.77:0/3001711271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f860269a70 0x55f860278f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:59.005 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.996+0000 7f29a2ffd700 0 client.15957 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:19:59.005 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.996+0000 7f29d6294b80 1 --2- 172.21.15.77:0/3001711271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8602683f0 0x55f8602719e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:59.005 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.996+0000 7f29d6294b80 1 --2- 172.21.15.77:0/3001711271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8602690f0 0x55f860271f20 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:59.005 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.996+0000 7f29d6294b80 1 -- 172.21.15.77:0/3001711271 >> 172.21.15.77:0/3001711271 conn(0x55f8601c70d0 msgr2=0x55f8602620b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:19:59.005 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.996+0000 7f29d6294b80 1 -- 172.21.15.77:0/3001711271 shutdown_connections 2023-12-01T01:19:59.005 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.996+0000 7f29d6294b80 1 -- 172.21.15.77:0/3001711271 wait complete. 2023-12-01T01:19:59.005 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:58.996+0000 7f29d6294b80 20 client.15957 trim_cache size 0 max 16384 2023-12-01T01:19:59.019 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.016+0000 7f29d6294b80 1 Processor -- start 2023-12-01T01:19:59.019 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.016+0000 7f29d6294b80 1 -- start start 2023-12-01T01:19:59.019 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.016+0000 7f29d6294b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8602683f0 0x55f860267e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:59.019 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.016+0000 7f29d6294b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8602690f0 0x55f860276f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:59.019 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.017+0000 7f29d6294b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f860269a70 0x55f8602719a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:59.019 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.017+0000 7f29d6294b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f860271550 con 0x55f860269a70 2023-12-01T01:19:59.019 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.017+0000 7f29d6294b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f86027abd0 con 0x55f8602683f0 2023-12-01T01:19:59.019 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.017+0000 7f29d6294b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f860270050 con 0x55f8602690f0 2023-12-01T01:19:59.020 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.017+0000 7f29c3fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f860269a70 0x55f8602719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:59.020 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.017+0000 7f29c37fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8602683f0 0x55f860267e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:59.020 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.017+0000 7f29c2ffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8602690f0 0x55f860276f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:59.020 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.017+0000 7f29c37fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8602683f0 0x55f860267e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:39420/0 (socket says 172.21.15.77:39420) 2023-12-01T01:19:59.020 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.017+0000 7f29c37fe700 1 -- 172.21.15.77:0/1607948042 learned_addr learned my addr 172.21.15.77:0/1607948042 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:19:59.020 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.017+0000 7f29c3fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f860269a70 0x55f8602719a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60502/0 (socket says 172.21.15.77:60502) 2023-12-01T01:19:59.020 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.017+0000 7f29c2ffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8602690f0 0x55f860276f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:46686/0 (socket says 172.21.15.77:46686) 2023-12-01T01:19:59.020 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.017+0000 7f29c37fe700 1 -- 172.21.15.77:0/1607948042 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8602690f0 msgr2=0x55f860276f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:59.020 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.017+0000 7f29c37fe700 1 --2- 172.21.15.77:0/1607948042 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8602690f0 0x55f860276f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:59.020 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.017+0000 7f29c37fe700 1 -- 172.21.15.77:0/1607948042 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f860269a70 msgr2=0x55f8602719a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:59.020 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.017+0000 7f29c37fe700 1 --2- 172.21.15.77:0/1607948042 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f860269a70 0x55f8602719a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:59.020 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.017+0000 7f29c37fe700 1 -- 172.21.15.77:0/1607948042 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f85ffcfc60 con 0x55f8602683f0 2023-12-01T01:19:59.020 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.017+0000 7f29c3fff700 1 --2- 172.21.15.77:0/1607948042 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f860269a70 0x55f8602719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:19:59.021 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.017+0000 7f29c37fe700 1 --2- 172.21.15.77:0/1607948042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8602683f0 0x55f860267e60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x55f860268e90 tx=0x7f29b400ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d02a4663ce55f22 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:19:59.021 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.017+0000 7f29aa7fc700 1 -- 172.21.15.77:0/1607948042 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b400edb0 con 0x55f8602683f0 2023-12-01T01:19:59.021 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.018+0000 7f29aa7fc700 1 -- 172.21.15.77:0/1607948042 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29b4005260 con 0x55f8602683f0 2023-12-01T01:19:59.021 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.018+0000 7f29aa7fc700 1 -- 172.21.15.77:0/1607948042 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b4032c70 con 0x55f8602683f0 2023-12-01T01:19:59.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.018+0000 7f29d6294b80 1 -- 172.21.15.77:0/1607948042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8602683f0 msgr2=0x55f860267e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:59.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.018+0000 7f29d6294b80 1 --2- 172.21.15.77:0/1607948042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8602683f0 0x55f860267e60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x55f860268e90 tx=0x7f29b400ecc0 comp rx=0 tx=0).stop 2023-12-01T01:19:59.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.018+0000 7f29d6294b80 1 -- 172.21.15.77:0/1607948042 shutdown_connections 2023-12-01T01:19:59.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.018+0000 7f29d6294b80 1 --2- 172.21.15.77:0/1607948042 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f860269a70 0x55f8602719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:59.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.018+0000 7f29d6294b80 1 --2- 172.21.15.77:0/1607948042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f8602683f0 0x55f860267e60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:59.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.018+0000 7f29d6294b80 1 --2- 172.21.15.77:0/1607948042 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8602690f0 0x55f860276f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:59.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.018+0000 7f29d6294b80 1 -- 172.21.15.77:0/1607948042 >> 172.21.15.77:0/1607948042 conn(0x55f8601c70d0 msgr2=0x55f86026c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:19:59.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.018+0000 7f29d6294b80 1 -- 172.21.15.77:0/1607948042 shutdown_connections 2023-12-01T01:19:59.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.018+0000 7f29d6294b80 1 -- 172.21.15.77:0/1607948042 wait complete. 2023-12-01T01:19:59.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.019+0000 7f29d6294b80 1 -- start start 2023-12-01T01:19:59.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.019+0000 7f29d6294b80 1 Processor -- start 2023-12-01T01:19:59.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.019+0000 7f29d6294b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8602683f0 0x55f860276b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:59.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.019+0000 7f29d6294b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8602690f0 0x55f860275130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:59.027 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.019+0000 7f29d6294b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f860269a70 0x55f860275670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:59.027 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.019+0000 7f29d6294b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f860275bb0 con 0x55f8602690f0 2023-12-01T01:19:59.027 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.019+0000 7f29d6294b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f860275d20 con 0x55f860269a70 2023-12-01T01:19:59.027 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.019+0000 7f29d6294b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f860275ee0 con 0x55f8602683f0 2023-12-01T01:19:59.027 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.019+0000 7f29d6294b80 10 client.0 _ll_register_callbacks cb 0x7f29b80124e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:19:59.027 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.020+0000 7f29c37fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8602683f0 0x55f860276b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:59.027 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.020+0000 7f29c37fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8602683f0 0x55f860276b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:46692/0 (socket says 172.21.15.77:46692) 2023-12-01T01:19:59.027 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.020+0000 7f29c37fe700 1 -- 172.21.15.77:0/174948742 learned_addr learned my addr 172.21.15.77:0/174948742 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:19:59.027 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.020+0000 7f29c2ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8602690f0 0x55f860275130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:59.027 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.020+0000 7f29c3fff700 1 --2- 172.21.15.77:0/174948742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f860269a70 0x55f860275670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:59.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.020+0000 7f29c2ffd700 1 -- 172.21.15.77:0/174948742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8602683f0 msgr2=0x55f860276b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:59.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.020+0000 7f29c2ffd700 1 --2- 172.21.15.77:0/174948742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8602683f0 0x55f860276b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:59.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.020+0000 7f29c2ffd700 1 -- 172.21.15.77:0/174948742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f860269a70 msgr2=0x55f860275670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:19:59.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.020+0000 7f29c2ffd700 1 --2- 172.21.15.77:0/174948742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f860269a70 0x55f860275670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:59.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.020+0000 7f29c2ffd700 1 -- 172.21.15.77:0/174948742 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c419f0c0 con 0x55f8602690f0 2023-12-01T01:19:59.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.020+0000 7f29c37fe700 1 --2- 172.21.15.77:0/174948742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8602683f0 0x55f860276b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:19:59.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.020+0000 7f29c3fff700 1 --2- 172.21.15.77:0/174948742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f860269a70 0x55f860275670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:19:59.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.020+0000 7f29c2ffd700 1 --2- 172.21.15.77:0/174948742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8602690f0 0x55f860275130 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f29b8000c00 tx=0x7f29b8011d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58e67447a022ea7 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:19:59.028 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.020+0000 7f29a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:19:59.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.020+0000 7f29a2ffd700 1 -- 172.21.15.77:0/174948742 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b8017040 con 0x55f8602690f0 2023-12-01T01:19:59.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.020+0000 7f29a2ffd700 1 -- 172.21.15.77:0/174948742 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f29b8009710 con 0x55f8602690f0 2023-12-01T01:19:59.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.020+0000 7f29d6294b80 1 -- 172.21.15.77:0/174948742 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29b4021070 con 0x55f8602690f0 2023-12-01T01:19:59.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.021+0000 7f29a2ffd700 1 -- 172.21.15.77:0/174948742 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29b8011ea0 con 0x55f8602690f0 2023-12-01T01:19:59.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.021+0000 7f29d6294b80 10 client.15963 fetch_fsmap learned FSMap version 31 2023-12-01T01:19:59.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.021+0000 7f29d6294b80 1 -- 172.21.15.77:0/174948742 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f29b402a040 con 0x55f8602690f0 2023-12-01T01:19:59.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.021+0000 7f29a2ffd700 1 -- 172.21.15.77:0/174948742 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29b4021070 con 0x55f8602690f0 2023-12-01T01:19:59.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.021+0000 7f29a2ffd700 1 -- 172.21.15.77:0/174948742 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f29b8005070 con 0x55f8602690f0 2023-12-01T01:19:59.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.021+0000 7f29d6294b80 10 client.15963 fetch_fsmap finished waiting for FSMap version 31 2023-12-01T01:19:59.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.021+0000 7f29d6294b80 10 client.15963 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-01T01:19:59.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.021+0000 7f29d6294b80 20 client.15963 populate_metadata read hostname 'smithi077' 2023-12-01T01:19:59.029 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.021+0000 7f29d6294b80 1 --2- 172.21.15.77:0/174948742 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55f86027d820 0x55f8602d2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:19:59.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.021+0000 7f29d6294b80 4 client.15963 mds_command: new command op to 24500 tid=0[{"prefix": "scrub status"}] 2023-12-01T01:19:59.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.021+0000 7f29d6294b80 1 -- 172.21.15.77:0/174948742 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f8601b1d40 con 0x55f86027d820 2023-12-01T01:19:59.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.021+0000 7f29c37fe700 1 --2- 172.21.15.77:0/174948742 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55f86027d820 0x55f8602d2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:19:59.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.022+0000 7f29c37fe700 1 --2- 172.21.15.77:0/174948742 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55f86027d820 0x55f8602d2a90 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e1fdb42226babe server_cookie=ba916164d2dfa2c2 in_seq=0 out_seq=0 2023-12-01T01:19:59.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.022+0000 7f29a2ffd700 10 client.15963 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:19:59.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.022+0000 7f29a2ffd700 1 -- 172.21.15.77:0/174948742 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55f8601b1d40 con 0x55f86027d820 2023-12-01T01:19:59.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.022+0000 7f29a2ffd700 10 client.15963 handle_command_reply: tid=0 2023-12-01T01:19:59.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.022+0000 7f29d6294b80 1 client.15963 shutdown 2023-12-01T01:19:59.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.022+0000 7f29d6294b80 1 -- 172.21.15.77:0/174948742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8602690f0 msgr2=0x55f860275130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:59.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.022+0000 7f29d6294b80 1 --2- 172.21.15.77:0/174948742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8602690f0 0x55f860275130 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f29b8000c00 tx=0x7f29b8011d30 comp rx=0 tx=0).stop 2023-12-01T01:19:59.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.022+0000 7f29d6294b80 1 -- 172.21.15.77:0/174948742 shutdown_connections 2023-12-01T01:19:59.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.022+0000 7f29d6294b80 1 --2- 172.21.15.77:0/174948742 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55f86027d820 0x55f8602d2a90 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:59.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.022+0000 7f29d6294b80 1 --2- 172.21.15.77:0/174948742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f860269a70 0x55f860275670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:59.030 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.022+0000 7f29a2ffd700 0 client.15963 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:19:59.031 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.022+0000 7f29d6294b80 1 --2- 172.21.15.77:0/174948742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f8602683f0 0x55f860276b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:59.031 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.022+0000 7f29d6294b80 1 --2- 172.21.15.77:0/174948742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8602690f0 0x55f860275130 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:59.031 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.022+0000 7f29d6294b80 1 -- 172.21.15.77:0/174948742 >> 172.21.15.77:0/174948742 conn(0x55f8601c70d0 msgr2=0x55f860271980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:19:59.031 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.023+0000 7f29d6294b80 1 -- 172.21.15.77:0/174948742 shutdown_connections 2023-12-01T01:19:59.031 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.023+0000 7f29d6294b80 1 -- 172.21.15.77:0/174948742 wait complete. 2023-12-01T01:19:59.031 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.023+0000 7f29d6294b80 20 client.15963 trim_cache size 0 max 16384 2023-12-01T01:19:59.032 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2023-12-01T01:19:59.032 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2023-12-01T01:19:59.032 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2023-12-01T01:19:59.032 INFO:teuthology.orchestra.run.smithi077.stdout: "085290de-33b6-4527-ab7f-03f7dfd5dbba": { 2023-12-01T01:19:59.032 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2023-12-01T01:19:59.032 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "085290de-33b6-4527-ab7f-03f7dfd5dbba", 2023-12-01T01:19:59.032 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2023-12-01T01:19:59.032 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:19:59.032 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:19:59.032 INFO:teuthology.orchestra.run.smithi077.stdout:} 2023-12-01T01:19:59.033 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.023+0000 7f29ca104700 1 -- 172.21.15.77:0/3893962629 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f29ac0a0520 msgr2=0x7f29ac0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:59.033 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.023+0000 7f29ca104700 1 --2- 172.21.15.77:0/3893962629 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f29ac0a0520 0x7f29ac0a29e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f29b4009c60 tx=0x7f29b4009260 comp rx=0 tx=0).stop 2023-12-01T01:19:59.033 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.023+0000 7f29ca104700 1 -- 172.21.15.77:0/3893962629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29c4106b70 msgr2=0x7f29c419e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:19:59.033 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.023+0000 7f29ca104700 1 --2- 172.21.15.77:0/3893962629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29c4106b70 0x7f29c419e120 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f29bc009fd0 tx=0x7f29bc00bce0 comp rx=0 tx=0).stop 2023-12-01T01:19:59.033 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.023+0000 7f29ca104700 1 -- 172.21.15.77:0/3893962629 shutdown_connections 2023-12-01T01:19:59.033 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.023+0000 7f29ca104700 1 --2- 172.21.15.77:0/3893962629 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f29ac0a0520 0x7f29ac0a29e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:59.033 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.023+0000 7f29ca104700 1 --2- 172.21.15.77:0/3893962629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29c4106b70 0x7f29c419e120 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:59.034 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.023+0000 7f29ca104700 1 --2- 172.21.15.77:0/3893962629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29c4101910 0x7f29c4196b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:59.034 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.023+0000 7f29ca104700 1 --2- 172.21.15.77:0/3893962629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29c4104240 0x7f29c41970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:19:59.034 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.023+0000 7f29ca104700 1 -- 172.21.15.77:0/3893962629 >> 172.21.15.77:0/3893962629 conn(0x7f29c40fb260 msgr2=0x7f29c4102d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:19:59.034 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.023+0000 7f29ca104700 1 -- 172.21.15.77:0/3893962629 shutdown_connections 2023-12-01T01:19:59.034 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:19:59.023+0000 7f29ca104700 1 -- 172.21.15.77:0/3893962629 wait complete. 2023-12-01T01:19:59.039 INFO:tasks.cephfs.filesystem:scrub status for tag:085290de-33b6-4527-ab7f-03f7dfd5dbba - {'path': '/', 'tag': '085290de-33b6-4527-ab7f-03f7dfd5dbba', 'options': 'recursive,force'} 2023-12-01T01:20:00.001 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:19:59 smithi077 ceph-mon[98867]: pgmap v361: 97 pgs: 97 active+clean; 43 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.4 MiB/s wr, 723 op/s 2023-12-01T01:20:00.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:19:59 smithi203 ceph-mon[103581]: pgmap v361: 97 pgs: 97 active+clean; 43 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.4 MiB/s wr, 723 op/s 2023-12-01T01:20:00.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:19:59 smithi137 ceph-mon[103312]: pgmap v361: 97 pgs: 97 active+clean; 43 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.4 MiB/s wr, 723 op/s 2023-12-01T01:20:00.403 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:00 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mon-a[98843]: 2023-12-01T01:19:59.999+0000 7f988a60c700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 2 MDSs report damaged metadata 2023-12-01T01:20:00.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:00 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mon-a[98843]: 2023-12-01T01:19:59.999+0000 7f988a60c700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_DAMAGE: 2 MDSs report damaged metadata 2023-12-01T01:20:00.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:00 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mon-a[98843]: 2023-12-01T01:19:59.999+0000 7f988a60c700 -1 log_channel(cluster) log [ERR] : mds.c(mds.2): Metadata damage detected 2023-12-01T01:20:00.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:00 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mon-a[98843]: 2023-12-01T01:19:59.999+0000 7f988a60c700 -1 log_channel(cluster) log [ERR] : mds.i(mds.1): Metadata damage detected 2023-12-01T01:20:01.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:20:00 smithi203 ceph-mon[103581]: Health detail: HEALTH_ERR 2 MDSs report damaged metadata 2023-12-01T01:20:01.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:20:00 smithi203 ceph-mon[103581]: [ERR] MDS_DAMAGE: 2 MDSs report damaged metadata 2023-12-01T01:20:01.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:20:00 smithi203 ceph-mon[103581]: mds.c(mds.2): Metadata damage detected 2023-12-01T01:20:01.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:20:00 smithi203 ceph-mon[103581]: mds.i(mds.1): Metadata damage detected 2023-12-01T01:20:01.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:20:00 smithi137 ceph-mon[103312]: Health detail: HEALTH_ERR 2 MDSs report damaged metadata 2023-12-01T01:20:01.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:20:00 smithi137 ceph-mon[103312]: [ERR] MDS_DAMAGE: 2 MDSs report damaged metadata 2023-12-01T01:20:01.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:20:00 smithi137 ceph-mon[103312]: mds.c(mds.2): Metadata damage detected 2023-12-01T01:20:01.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:20:00 smithi137 ceph-mon[103312]: mds.i(mds.1): Metadata damage detected 2023-12-01T01:20:01.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:00 smithi077 ceph-mon[98867]: Health detail: HEALTH_ERR 2 MDSs report damaged metadata 2023-12-01T01:20:01.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:00 smithi077 ceph-mon[98867]: [ERR] MDS_DAMAGE: 2 MDSs report damaged metadata 2023-12-01T01:20:01.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:00 smithi077 ceph-mon[98867]: mds.c(mds.2): Metadata damage detected 2023-12-01T01:20:01.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:00 smithi077 ceph-mon[98867]: mds.i(mds.1): Metadata damage detected 2023-12-01T01:20:02.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:20:01 smithi203 ceph-mon[103581]: pgmap v362: 97 pgs: 97 active+clean; 46 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.4 MiB/s wr, 598 op/s 2023-12-01T01:20:02.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:20:01 smithi137 ceph-mon[103312]: pgmap v362: 97 pgs: 97 active+clean; 46 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.4 MiB/s wr, 598 op/s 2023-12-01T01:20:02.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:01 smithi077 ceph-mon[98867]: pgmap v362: 97 pgs: 97 active+clean; 46 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.4 MiB/s wr, 598 op/s 2023-12-01T01:20:03.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:20:02 smithi203 ceph-mon[103581]: pgmap v363: 97 pgs: 97 active+clean; 47 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.4 MiB/s wr, 598 op/s 2023-12-01T01:20:03.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:20:02 smithi137 ceph-mon[103312]: pgmap v363: 97 pgs: 97 active+clean; 47 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.4 MiB/s wr, 598 op/s 2023-12-01T01:20:03.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:02 smithi077 ceph-mon[98867]: pgmap v363: 97 pgs: 97 active+clean; 47 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.4 MiB/s wr, 598 op/s 2023-12-01T01:20:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.046+0000 7f3e3cc7b700 1 -- 172.21.15.77:0/3532234567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e38101fc0 msgr2=0x7f3e38102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.046+0000 7f3e3cc7b700 1 --2- 172.21.15.77:0/3532234567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e38101fc0 0x7f3e38102420 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c0099c0 tx=0x7f3e2c009cd0 comp rx=0 tx=0).stop 2023-12-01T01:20:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.046+0000 7f3e3cc7b700 1 -- 172.21.15.77:0/3532234567 shutdown_connections 2023-12-01T01:20:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.046+0000 7f3e3cc7b700 1 --2- 172.21.15.77:0/3532234567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e38102960 0x7f3e38109210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.046+0000 7f3e3cc7b700 1 --2- 172.21.15.77:0/3532234567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e38100dd0 0x7f3e381011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.046+0000 7f3e3cc7b700 1 --2- 172.21.15.77:0/3532234567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e38101fc0 0x7f3e38102420 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.046+0000 7f3e3cc7b700 1 -- 172.21.15.77:0/3532234567 >> 172.21.15.77:0/3532234567 conn(0x7f3e380fc360 msgr2=0x7f3e380fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:20:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.046+0000 7f3e3cc7b700 1 -- 172.21.15.77:0/3532234567 shutdown_connections 2023-12-01T01:20:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.046+0000 7f3e3cc7b700 1 -- 172.21.15.77:0/3532234567 wait complete. 2023-12-01T01:20:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.047+0000 7f3e3cc7b700 1 Processor -- start 2023-12-01T01:20:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.047+0000 7f3e3cc7b700 1 -- start start 2023-12-01T01:20:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.047+0000 7f3e3cc7b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e38100dd0 0x7f3e38196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.047+0000 7f3e3cc7b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e38101fc0 0x7f3e38197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.047+0000 7f3e3cc7b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e38102960 0x7f3e38197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.047+0000 7f3e3cc7b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e38198560 con 0x7f3e38102960 2023-12-01T01:20:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.047+0000 7f3e3cc7b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e381986d0 con 0x7f3e38101fc0 2023-12-01T01:20:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.047+0000 7f3e3cc7b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e38198890 con 0x7f3e38100dd0 2023-12-01T01:20:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.047+0000 7f3e3659c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e38100dd0 0x7f3e38196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.047+0000 7f3e3659c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e38100dd0 0x7f3e38196b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:46706/0 (socket says 172.21.15.77:46706) 2023-12-01T01:20:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.047+0000 7f3e3659c700 1 -- 172.21.15.77:0/4293259662 learned_addr learned my addr 172.21.15.77:0/4293259662 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:20:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.047+0000 7f3e36d9d700 1 --2- 172.21.15.77:0/4293259662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e38102960 0x7f3e38197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.047+0000 7f3e35d9b700 1 --2- 172.21.15.77:0/4293259662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e38101fc0 0x7f3e38197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.047+0000 7f3e3659c700 1 -- 172.21.15.77:0/4293259662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e38101fc0 msgr2=0x7f3e38197080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.047+0000 7f3e3659c700 1 --2- 172.21.15.77:0/4293259662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e38101fc0 0x7f3e38197080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.047+0000 7f3e3659c700 1 -- 172.21.15.77:0/4293259662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e38102960 msgr2=0x7f3e38197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.047+0000 7f3e3659c700 1 --2- 172.21.15.77:0/4293259662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e38102960 0x7f3e38197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.048+0000 7f3e3659c700 1 -- 172.21.15.77:0/4293259662 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e2c0095f0 con 0x7f3e38100dd0 2023-12-01T01:20:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.048+0000 7f3e36d9d700 1 --2- 172.21.15.77:0/4293259662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e38102960 0x7f3e38197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:20:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.048+0000 7f3e3659c700 1 --2- 172.21.15.77:0/4293259662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e38100dd0 0x7f3e38196b40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800d980 tx=0x7f3e2800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:20:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.048+0000 7f3e277fe700 1 -- 172.21.15.77:0/4293259662 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e28004020 con 0x7f3e38100dd0 2023-12-01T01:20:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.048+0000 7f3e3cc7b700 1 -- 172.21.15.77:0/4293259662 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e38198b70 con 0x7f3e38100dd0 2023-12-01T01:20:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.048+0000 7f3e277fe700 1 -- 172.21.15.77:0/4293259662 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e2800f690 con 0x7f3e38100dd0 2023-12-01T01:20:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.048+0000 7f3e277fe700 1 -- 172.21.15.77:0/4293259662 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e28003a60 con 0x7f3e38100dd0 2023-12-01T01:20:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.049+0000 7f3e3cc7b700 1 -- 172.21.15.77:0/4293259662 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e3819f330 con 0x7f3e38100dd0 2023-12-01T01:20:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.051+0000 7f3e277fe700 1 -- 172.21.15.77:0/4293259662 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f3e2800fb00 con 0x7f3e38100dd0 2023-12-01T01:20:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.052+0000 7f3e277fe700 1 --2- 172.21.15.77:0/4293259662 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f3e200a0520 0x7f3e200a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.052+0000 7f3e277fe700 1 -- 172.21.15.77:0/4293259662 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f3e280cacd0 con 0x7f3e38100dd0 2023-12-01T01:20:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.052+0000 7f3e35d9b700 1 --2- 172.21.15.77:0/4293259662 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f3e200a0520 0x7f3e200a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.052+0000 7f3e35d9b700 1 --2- 172.21.15.77:0/4293259662 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f3e200a0520 0x7f3e200a29e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00b580 tx=0x7f3e2c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:20:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.053+0000 7f3e48e0bb80 1 Processor -- start 2023-12-01T01:20:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.053+0000 7f3e48e0bb80 1 -- start start 2023-12-01T01:20:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.053+0000 7f3e48e0bb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e37e1b8680 0x55e37e1b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.053+0000 7f3e48e0bb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e37e1b8fe0 0x55e37e1b98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.054+0000 7f3e48e0bb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e37e1b9ed0 0x55e37e1ba350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.054+0000 7f3e48e0bb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e37e1109b0 con 0x55e37e1b8680 2023-12-01T01:20:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.054+0000 7f3e48e0bb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e37e10e370 con 0x55e37e1b8fe0 2023-12-01T01:20:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.054+0000 7f3e48e0bb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e37e109c60 con 0x55e37e1b9ed0 2023-12-01T01:20:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.054+0000 7f3e3659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e37e1b8680 0x55e37e1b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.054+0000 7f3e3659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e37e1b8680 0x55e37e1b8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60528/0 (socket says 172.21.15.77:60528) 2023-12-01T01:20:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.054+0000 7f3e3659c700 1 -- 172.21.15.77:0/3271837243 learned_addr learned my addr 172.21.15.77:0/3271837243 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:20:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.054+0000 7f3e36d9d700 1 --2- 172.21.15.77:0/3271837243 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e37e1b9ed0 0x55e37e1ba350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.054+0000 7f3e35d9b700 1 --2- 172.21.15.77:0/3271837243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e37e1b8fe0 0x55e37e1b98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.054+0000 7f3e3659c700 1 -- 172.21.15.77:0/3271837243 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e37e1b9ed0 msgr2=0x55e37e1ba350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.054+0000 7f3e3659c700 1 --2- 172.21.15.77:0/3271837243 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e37e1b9ed0 0x55e37e1ba350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.054+0000 7f3e3659c700 1 -- 172.21.15.77:0/3271837243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e37e1b8fe0 msgr2=0x55e37e1b98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.054+0000 7f3e3659c700 1 --2- 172.21.15.77:0/3271837243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e37e1b8fe0 0x55e37e1b98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.054+0000 7f3e3659c700 1 -- 172.21.15.77:0/3271837243 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e37df37d80 con 0x55e37e1b8680 2023-12-01T01:20:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.054+0000 7f3e35d9b700 1 --2- 172.21.15.77:0/3271837243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e37e1b8fe0 0x55e37e1b98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:20:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.054+0000 7f3e3659c700 1 --2- 172.21.15.77:0/3271837243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e37e1b8680 0x55e37e1b8aa0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f3e28094ad0 tx=0x7f3e280981e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=558347306561c3d5 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:20:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.054+0000 7f3e24ff9700 1 -- 172.21.15.77:0/3271837243 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2809c400 con 0x55e37e1b8680 2023-12-01T01:20:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.054+0000 7f3e24ff9700 1 -- 172.21.15.77:0/3271837243 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e280af070 con 0x55e37e1b8680 2023-12-01T01:20:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.054+0000 7f3e24ff9700 1 -- 172.21.15.77:0/3271837243 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e280184d0 con 0x55e37e1b8680 2023-12-01T01:20:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.055+0000 7f3e48e0bb80 1 -- 172.21.15.77:0/3271837243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e37e1b8680 msgr2=0x55e37e1b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.055+0000 7f3e48e0bb80 1 --2- 172.21.15.77:0/3271837243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e37e1b8680 0x55e37e1b8aa0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f3e28094ad0 tx=0x7f3e280981e0 comp rx=0 tx=0).stop 2023-12-01T01:20:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.055+0000 7f3e48e0bb80 1 -- 172.21.15.77:0/3271837243 shutdown_connections 2023-12-01T01:20:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.055+0000 7f3e48e0bb80 1 --2- 172.21.15.77:0/3271837243 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e37e1b9ed0 0x55e37e1ba350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.055+0000 7f3e48e0bb80 1 --2- 172.21.15.77:0/3271837243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e37e1b8680 0x55e37e1b8aa0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.055+0000 7f3e48e0bb80 1 --2- 172.21.15.77:0/3271837243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e37e1b8fe0 0x55e37e1b98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.055+0000 7f3e48e0bb80 1 -- 172.21.15.77:0/3271837243 >> 172.21.15.77:0/3271837243 conn(0x55e37e118070 msgr2=0x55e37e1b5040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:20:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.055+0000 7f3e48e0bb80 1 -- 172.21.15.77:0/3271837243 shutdown_connections 2023-12-01T01:20:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.055+0000 7f3e48e0bb80 1 -- 172.21.15.77:0/3271837243 wait complete. 2023-12-01T01:20:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.056+0000 7f3e48e0bb80 1 -- start start 2023-12-01T01:20:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.056+0000 7f3e48e0bb80 1 Processor -- start 2023-12-01T01:20:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.056+0000 7f3e48e0bb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e37e1b8680 0x55e37e1bdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:04.065 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.056+0000 7f3e48e0bb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e37e1b8fe0 0x55e37e1be450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:04.065 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.056+0000 7f3e48e0bb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e37e1b9ed0 0x55e37e1bf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:04.065 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.056+0000 7f3e48e0bb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e37e0a64b0 con 0x55e37e1b8680 2023-12-01T01:20:04.065 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.056+0000 7f3e48e0bb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e37e0a6150 con 0x55e37e1b9ed0 2023-12-01T01:20:04.065 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.056+0000 7f3e48e0bb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e37df8d8c0 con 0x55e37e1b8fe0 2023-12-01T01:20:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.056+0000 7f3e48e0bb80 10 client.0 _ll_register_callbacks cb 0x55e37dff3a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:20:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.056+0000 7f3e3659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e37e1b8680 0x55e37e1bdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.056+0000 7f3e3659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e37e1b8680 0x55e37e1bdf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60532/0 (socket says 172.21.15.77:60532) 2023-12-01T01:20:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.056+0000 7f3e3659c700 1 -- 172.21.15.77:0/4194508665 learned_addr learned my addr 172.21.15.77:0/4194508665 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:20:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.056+0000 7f3e36d9d700 1 --2- 172.21.15.77:0/4194508665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e37e1b9ed0 0x55e37e1bf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.057+0000 7f3e3659c700 1 -- 172.21.15.77:0/4194508665 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e37e1b8fe0 msgr2=0x55e37e1be450 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-01T01:20:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.057+0000 7f3e3659c700 1 --2- 172.21.15.77:0/4194508665 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e37e1b8fe0 0x55e37e1be450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.057+0000 7f3e3659c700 1 -- 172.21.15.77:0/4194508665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e37e1b9ed0 msgr2=0x55e37e1bf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.057+0000 7f3e3659c700 1 --2- 172.21.15.77:0/4194508665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e37e1b9ed0 0x55e37e1bf460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.057+0000 7f3e3659c700 1 -- 172.21.15.77:0/4194508665 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e3819f330 con 0x55e37e1b8680 2023-12-01T01:20:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.057+0000 7f3e3659c700 1 --2- 172.21.15.77:0/4194508665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e37e1b8680 0x55e37e1bdf10 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3e280a0110 tx=0x7f3e28094a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56fb21f8a6fb9ccc server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:20:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.057+0000 7f3e1dffb700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:20:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.057+0000 7f3e1dffb700 1 -- 172.21.15.77:0/4194508665 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e280832d0 con 0x55e37e1b8680 2023-12-01T01:20:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.057+0000 7f3e1dffb700 1 -- 172.21.15.77:0/4194508665 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f3e28096710 con 0x55e37e1b8680 2023-12-01T01:20:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.057+0000 7f3e48e0bb80 1 -- 172.21.15.77:0/4194508665 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e37e134cb0 con 0x55e37e1b8680 2023-12-01T01:20:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.057+0000 7f3e1dffb700 1 -- 172.21.15.77:0/4194508665 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e2809c400 con 0x55e37e1b8680 2023-12-01T01:20:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.057+0000 7f3e48e0bb80 10 client.15975 fetch_fsmap learned FSMap version 31 2023-12-01T01:20:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.057+0000 7f3e48e0bb80 1 -- 172.21.15.77:0/4194508665 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55e37df20c00 con 0x55e37e1b8680 2023-12-01T01:20:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.057+0000 7f3e1dffb700 1 -- 172.21.15.77:0/4194508665 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e37e134cb0 con 0x55e37e1b8680 2023-12-01T01:20:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.058+0000 7f3e1dffb700 1 -- 172.21.15.77:0/4194508665 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3e28097ac0 con 0x55e37e1b8680 2023-12-01T01:20:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.058+0000 7f3e48e0bb80 10 client.15975 fetch_fsmap finished waiting for FSMap version 31 2023-12-01T01:20:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.058+0000 7f3e48e0bb80 10 client.15975 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-01T01:20:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.058+0000 7f3e48e0bb80 20 client.15975 populate_metadata read hostname 'smithi077' 2023-12-01T01:20:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.058+0000 7f3e48e0bb80 1 --2- 172.21.15.77:0/4194508665 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55e37e1d68b0 0x55e37e1d8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.058+0000 7f3e48e0bb80 4 client.15975 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:20:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.058+0000 7f3e48e0bb80 1 -- 172.21.15.77:0/4194508665 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e37e187db0 con 0x55e37e1d68b0 2023-12-01T01:20:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.058+0000 7f3e35d9b700 1 --2- 172.21.15.77:0/4194508665 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55e37e1d68b0 0x55e37e1d8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.058+0000 7f3e1dffb700 10 client.15975 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:20:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.058+0000 7f3e35d9b700 1 --2- 172.21.15.77:0/4194508665 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55e37e1d68b0 0x55e37e1d8cd0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c5e84f9f160b9cb server_cookie=e1b5ded2cf9bda01 in_seq=0 out_seq=0 2023-12-01T01:20:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.059+0000 7f3e1dffb700 1 -- 172.21.15.77:0/4194508665 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55e37e187db0 con 0x55e37e1d68b0 2023-12-01T01:20:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.059+0000 7f3e1dffb700 10 client.15975 handle_command_reply: tid=0 2023-12-01T01:20:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.059+0000 7f3e48e0bb80 1 client.15975 shutdown 2023-12-01T01:20:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.059+0000 7f3e48e0bb80 1 -- 172.21.15.77:0/4194508665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e37e1b8680 msgr2=0x55e37e1bdf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.059+0000 7f3e48e0bb80 1 --2- 172.21.15.77:0/4194508665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e37e1b8680 0x55e37e1bdf10 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3e280a0110 tx=0x7f3e28094a40 comp rx=0 tx=0).stop 2023-12-01T01:20:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.059+0000 7f3e48e0bb80 1 -- 172.21.15.77:0/4194508665 shutdown_connections 2023-12-01T01:20:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.059+0000 7f3e48e0bb80 1 --2- 172.21.15.77:0/4194508665 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55e37e1d68b0 0x55e37e1d8cd0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.059+0000 7f3e48e0bb80 1 --2- 172.21.15.77:0/4194508665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e37e1b9ed0 0x55e37e1bf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.059+0000 7f3e1dffb700 0 client.15975 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:20:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.059+0000 7f3e48e0bb80 1 --2- 172.21.15.77:0/4194508665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e37e1b8680 0x55e37e1bdf10 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.059+0000 7f3e48e0bb80 1 --2- 172.21.15.77:0/4194508665 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e37e1b8fe0 0x55e37e1be450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.059+0000 7f3e48e0bb80 1 -- 172.21.15.77:0/4194508665 >> 172.21.15.77:0/4194508665 conn(0x55e37e118070 msgr2=0x55e37e1b33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:20:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.059+0000 7f3e48e0bb80 1 -- 172.21.15.77:0/4194508665 shutdown_connections 2023-12-01T01:20:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.059+0000 7f3e48e0bb80 1 -- 172.21.15.77:0/4194508665 wait complete. 2023-12-01T01:20:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.059+0000 7f3e48e0bb80 20 client.15975 trim_cache size 0 max 16384 2023-12-01T01:20:04.079 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.076+0000 7f3e48e0bb80 1 Processor -- start 2023-12-01T01:20:04.079 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.076+0000 7f3e48e0bb80 1 -- start start 2023-12-01T01:20:04.079 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.077+0000 7f3e48e0bb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e37e1b8680 0x55e37e1bf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:04.079 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.077+0000 7f3e48e0bb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e37e1b8fe0 0x55e37e1bd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:04.079 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.077+0000 7f3e48e0bb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e37e1b9ed0 0x55e37e1bdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:04.079 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.077+0000 7f3e48e0bb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e37e134ce0 con 0x55e37e1b9ed0 2023-12-01T01:20:04.079 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.077+0000 7f3e48e0bb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e37e1bf130 con 0x55e37e1b8fe0 2023-12-01T01:20:04.080 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.077+0000 7f3e48e0bb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e37e1b9d50 con 0x55e37e1b8680 2023-12-01T01:20:04.080 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.077+0000 7f3e36d9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e37e1b9ed0 0x55e37e1bdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:04.080 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.077+0000 7f3e3659c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e37e1b8680 0x55e37e1bf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:04.080 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.077+0000 7f3e36d9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e37e1b9ed0 0x55e37e1bdcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60540/0 (socket says 172.21.15.77:60540) 2023-12-01T01:20:04.080 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.077+0000 7f3e36d9d700 1 -- 172.21.15.77:0/2639644332 learned_addr learned my addr 172.21.15.77:0/2639644332 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:20:04.080 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.077+0000 7f3e35d9b700 1 --2- 172.21.15.77:0/2639644332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e37e1b8fe0 0x55e37e1bd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:04.080 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.077+0000 7f3e35d9b700 1 -- 172.21.15.77:0/2639644332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e37e1b8680 msgr2=0x55e37e1bf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:04.080 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.077+0000 7f3e35d9b700 1 --2- 172.21.15.77:0/2639644332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e37e1b8680 0x55e37e1bf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:04.080 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.077+0000 7f3e35d9b700 1 -- 172.21.15.77:0/2639644332 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e37e1b9ed0 msgr2=0x55e37e1bdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:04.080 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.077+0000 7f3e35d9b700 1 --2- 172.21.15.77:0/2639644332 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e37e1b9ed0 0x55e37e1bdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:04.080 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.077+0000 7f3e35d9b700 1 -- 172.21.15.77:0/2639644332 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e37df5e1f0 con 0x55e37e1b8fe0 2023-12-01T01:20:04.081 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.077+0000 7f3e3659c700 1 --2- 172.21.15.77:0/2639644332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e37e1b8680 0x55e37e1bf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:20:04.081 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.077+0000 7f3e35d9b700 1 --2- 172.21.15.77:0/2639644332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e37e1b8fe0 0x55e37e1bd5c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c009fd0 tx=0x7f3e2c021000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddc4f6776511d903 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:20:04.081 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.077+0000 7f3e257fa700 1 -- 172.21.15.77:0/2639644332 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c031250 con 0x55e37e1b8fe0 2023-12-01T01:20:04.081 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.078+0000 7f3e257fa700 1 -- 172.21.15.77:0/2639644332 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e2c02a040 con 0x55e37e1b8fe0 2023-12-01T01:20:04.081 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.078+0000 7f3e257fa700 1 -- 172.21.15.77:0/2639644332 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c003bd0 con 0x55e37e1b8fe0 2023-12-01T01:20:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.078+0000 7f3e48e0bb80 1 -- 172.21.15.77:0/2639644332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e37e1b8fe0 msgr2=0x55e37e1bd5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.078+0000 7f3e48e0bb80 1 --2- 172.21.15.77:0/2639644332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e37e1b8fe0 0x55e37e1bd5c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c009fd0 tx=0x7f3e2c021000 comp rx=0 tx=0).stop 2023-12-01T01:20:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.078+0000 7f3e48e0bb80 1 -- 172.21.15.77:0/2639644332 shutdown_connections 2023-12-01T01:20:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.078+0000 7f3e48e0bb80 1 --2- 172.21.15.77:0/2639644332 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e37e1b9ed0 0x55e37e1bdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.078+0000 7f3e48e0bb80 1 --2- 172.21.15.77:0/2639644332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e37e1b8680 0x55e37e1bf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.078+0000 7f3e48e0bb80 1 --2- 172.21.15.77:0/2639644332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e37e1b8fe0 0x55e37e1bd5c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.078+0000 7f3e48e0bb80 1 -- 172.21.15.77:0/2639644332 >> 172.21.15.77:0/2639644332 conn(0x55e37e118070 msgr2=0x55e37e1b4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:20:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.078+0000 7f3e48e0bb80 1 -- 172.21.15.77:0/2639644332 shutdown_connections 2023-12-01T01:20:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.078+0000 7f3e48e0bb80 1 -- 172.21.15.77:0/2639644332 wait complete. 2023-12-01T01:20:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.079+0000 7f3e48e0bb80 1 -- start start 2023-12-01T01:20:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.079+0000 7f3e48e0bb80 1 Processor -- start 2023-12-01T01:20:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.079+0000 7f3e48e0bb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e37e1b8680 0x55e37e1c6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.079+0000 7f3e48e0bb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e37e1b8fe0 0x55e37e1c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.079+0000 7f3e48e0bb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e37e1b9ed0 0x55e37e1ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:04.088 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.079+0000 7f3e48e0bb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e37e1ce560 con 0x55e37e1b8680 2023-12-01T01:20:04.088 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.079+0000 7f3e48e0bb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e37e1ce6d0 con 0x55e37e1b8fe0 2023-12-01T01:20:04.088 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.079+0000 7f3e48e0bb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e37e1ce890 con 0x55e37e1b9ed0 2023-12-01T01:20:04.088 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.079+0000 7f3e48e0bb80 10 client.0 _ll_register_callbacks cb 0x55e37dff3a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:20:04.088 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.079+0000 7f3e3659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e37e1b8680 0x55e37e1c6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:04.088 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.079+0000 7f3e3659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e37e1b8680 0x55e37e1c6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60544/0 (socket says 172.21.15.77:60544) 2023-12-01T01:20:04.088 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.079+0000 7f3e3659c700 1 -- 172.21.15.77:0/869945710 learned_addr learned my addr 172.21.15.77:0/869945710 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:20:04.088 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.079+0000 7f3e36d9d700 1 --2- 172.21.15.77:0/869945710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e37e1b9ed0 0x55e37e1ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:04.088 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.080+0000 7f3e35d9b700 1 --2- 172.21.15.77:0/869945710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e37e1b8fe0 0x55e37e1c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:04.088 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.080+0000 7f3e3659c700 1 -- 172.21.15.77:0/869945710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e37e1b9ed0 msgr2=0x55e37e1ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:04.088 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.080+0000 7f3e3659c700 1 --2- 172.21.15.77:0/869945710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e37e1b9ed0 0x55e37e1ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:04.089 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.080+0000 7f3e3659c700 1 -- 172.21.15.77:0/869945710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e37e1b8fe0 msgr2=0x55e37e1c6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:04.089 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.080+0000 7f3e3659c700 1 --2- 172.21.15.77:0/869945710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e37e1b8fe0 0x55e37e1c6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:04.089 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.080+0000 7f3e3659c700 1 -- 172.21.15.77:0/869945710 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e37df20c00 con 0x55e37e1b8680 2023-12-01T01:20:04.089 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.080+0000 7f3e36d9d700 1 --2- 172.21.15.77:0/869945710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e37e1b9ed0 0x55e37e1ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:20:04.089 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.080+0000 7f3e3659c700 1 --2- 172.21.15.77:0/869945710 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e37e1b8680 0x55e37e1c6aa0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800be80 tx=0x7f3e2800ffa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d4df562fd1c4ffb server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:20:04.089 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.080+0000 7f3e1dffb700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:20:04.089 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.080+0000 7f3e1dffb700 1 -- 172.21.15.77:0/869945710 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e28098110 con 0x55e37e1b8680 2023-12-01T01:20:04.089 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.080+0000 7f3e1dffb700 1 -- 172.21.15.77:0/869945710 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f3e280a03f0 con 0x55e37e1b8680 2023-12-01T01:20:04.089 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.080+0000 7f3e48e0bb80 1 -- 172.21.15.77:0/869945710 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3e280af070 con 0x55e37e1b8680 2023-12-01T01:20:04.089 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.080+0000 7f3e1dffb700 1 -- 172.21.15.77:0/869945710 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e280832d0 con 0x55e37e1b8680 2023-12-01T01:20:04.089 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.080+0000 7f3e48e0bb80 10 client.15981 fetch_fsmap learned FSMap version 31 2023-12-01T01:20:04.089 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.080+0000 7f3e48e0bb80 1 -- 172.21.15.77:0/869945710 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f3e38109b60 con 0x55e37e1b8680 2023-12-01T01:20:04.089 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.081+0000 7f3e1dffb700 1 -- 172.21.15.77:0/869945710 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e280af070 con 0x55e37e1b8680 2023-12-01T01:20:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.081+0000 7f3e1dffb700 1 -- 172.21.15.77:0/869945710 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3e28018270 con 0x55e37e1b8680 2023-12-01T01:20:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.081+0000 7f3e48e0bb80 10 client.15981 fetch_fsmap finished waiting for FSMap version 31 2023-12-01T01:20:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.081+0000 7f3e48e0bb80 10 client.15981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-01T01:20:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.081+0000 7f3e48e0bb80 20 client.15981 populate_metadata read hostname 'smithi077' 2023-12-01T01:20:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.081+0000 7f3e48e0bb80 1 --2- 172.21.15.77:0/869945710 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55e37e1cff90 0x55e37e1d23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.081+0000 7f3e48e0bb80 4 client.15981 mds_command: new command op to 24500 tid=0[{"prefix": "flush journal"}] 2023-12-01T01:20:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.081+0000 7f3e48e0bb80 1 -- 172.21.15.77:0/869945710 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e37e102ce0 con 0x55e37e1cff90 2023-12-01T01:20:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.081+0000 7f3e35d9b700 1 --2- 172.21.15.77:0/869945710 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55e37e1cff90 0x55e37e1d23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.081+0000 7f3e1dffb700 10 client.15981 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:20:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.081+0000 7f3e35d9b700 1 --2- 172.21.15.77:0/869945710 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55e37e1cff90 0x55e37e1d23b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91408c74848a97b6 server_cookie=7bd74f0779af75f1 in_seq=0 out_seq=0 2023-12-01T01:20:04.133 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.131+0000 7f3e1dffb700 1 -- 172.21.15.77:0/869945710 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e37e102ce0 con 0x55e37e1cff90 2023-12-01T01:20:04.133 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.131+0000 7f3e1dffb700 10 client.15981 handle_command_reply: tid=0 2023-12-01T01:20:04.133 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.131+0000 7f3e48e0bb80 1 client.15981 shutdown 2023-12-01T01:20:04.133 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.131+0000 7f3e48e0bb80 1 -- 172.21.15.77:0/869945710 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e37e1b8680 msgr2=0x55e37e1c6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:04.134 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.131+0000 7f3e48e0bb80 1 --2- 172.21.15.77:0/869945710 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e37e1b8680 0x55e37e1c6aa0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800be80 tx=0x7f3e2800ffa0 comp rx=0 tx=0).stop 2023-12-01T01:20:04.134 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.132+0000 7f3e48e0bb80 1 -- 172.21.15.77:0/869945710 shutdown_connections 2023-12-01T01:20:04.134 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.132+0000 7f3e48e0bb80 1 --2- 172.21.15.77:0/869945710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e37e1b9ed0 0x55e37e1ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:04.134 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.132+0000 7f3e48e0bb80 1 --2- 172.21.15.77:0/869945710 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e37e1b8680 0x55e37e1c6aa0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:04.134 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.132+0000 7f3e48e0bb80 1 --2- 172.21.15.77:0/869945710 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55e37e1cff90 0x55e37e1d23b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:04.134 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.132+0000 7f3e48e0bb80 1 --2- 172.21.15.77:0/869945710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e37e1b8fe0 0x55e37e1c6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:04.134 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.132+0000 7f3e1dffb700 0 client.15981 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:20:04.134 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.132+0000 7f3e48e0bb80 1 -- 172.21.15.77:0/869945710 >> 172.21.15.77:0/869945710 conn(0x55e37e118070 msgr2=0x55e37e1c3060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:20:04.134 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.132+0000 7f3e48e0bb80 1 -- 172.21.15.77:0/869945710 shutdown_connections 2023-12-01T01:20:04.134 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.132+0000 7f3e48e0bb80 1 -- 172.21.15.77:0/869945710 wait complete. 2023-12-01T01:20:04.134 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.132+0000 7f3e48e0bb80 20 client.15981 trim_cache size 0 max 16384 2023-12-01T01:20:04.135 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout:{ 2023-12-01T01:20:04.135 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout: "message": "", 2023-12-01T01:20:04.135 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout: "return_code": 0 2023-12-01T01:20:04.135 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout:} 2023-12-01T01:20:04.136 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.132+0000 7f3e3cc7b700 1 -- 172.21.15.77:0/4293259662 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f3e200a0520 msgr2=0x7f3e200a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:04.136 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.132+0000 7f3e3cc7b700 1 --2- 172.21.15.77:0/4293259662 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f3e200a0520 0x7f3e200a29e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00b580 tx=0x7f3e2c005fb0 comp rx=0 tx=0).stop 2023-12-01T01:20:04.136 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.132+0000 7f3e3cc7b700 1 -- 172.21.15.77:0/4293259662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e38100dd0 msgr2=0x7f3e38196b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:04.136 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.132+0000 7f3e3cc7b700 1 --2- 172.21.15.77:0/4293259662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e38100dd0 0x7f3e38196b40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800d980 tx=0x7f3e2800dc90 comp rx=0 tx=0).stop 2023-12-01T01:20:04.136 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.132+0000 7f3e3cc7b700 1 -- 172.21.15.77:0/4293259662 shutdown_connections 2023-12-01T01:20:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.132+0000 7f3e3cc7b700 1 --2- 172.21.15.77:0/4293259662 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f3e200a0520 0x7f3e200a29e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.132+0000 7f3e3cc7b700 1 --2- 172.21.15.77:0/4293259662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e38102960 0x7f3e38197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.132+0000 7f3e3cc7b700 1 --2- 172.21.15.77:0/4293259662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e38100dd0 0x7f3e38196b40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.132+0000 7f3e3cc7b700 1 --2- 172.21.15.77:0/4293259662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e38101fc0 0x7f3e38197080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.132+0000 7f3e3cc7b700 1 -- 172.21.15.77:0/4293259662 >> 172.21.15.77:0/4293259662 conn(0x7f3e380fc360 msgr2=0x7f3e381054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:20:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.133+0000 7f3e3cc7b700 1 -- 172.21.15.77:0/4293259662 shutdown_connections 2023-12-01T01:20:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:04.133+0000 7f3e3cc7b700 1 -- 172.21.15.77:0/4293259662 wait complete. 2023-12-01T01:20:05.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:05 smithi077 ceph-mon[98867]: pgmap v364: 97 pgs: 97 active+clean; 51 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.5 MiB/s wr, 722 op/s 2023-12-01T01:20:05.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:20:05 smithi203 ceph-mon[103581]: pgmap v364: 97 pgs: 97 active+clean; 51 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.5 MiB/s wr, 722 op/s 2023-12-01T01:20:05.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:20:05 smithi137 ceph-mon[103312]: pgmap v364: 97 pgs: 97 active+clean; 51 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.5 MiB/s wr, 722 op/s 2023-12-01T01:20:07.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:07 smithi077 ceph-mon[98867]: pgmap v365: 97 pgs: 97 active+clean; 48 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.1 MiB/s wr, 503 op/s 2023-12-01T01:20:08.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:20:07 smithi203 ceph-mon[103581]: pgmap v365: 97 pgs: 97 active+clean; 48 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.1 MiB/s wr, 503 op/s 2023-12-01T01:20:08.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:20:07 smithi137 ceph-mon[103312]: pgmap v365: 97 pgs: 97 active+clean; 48 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.1 MiB/s wr, 503 op/s 2023-12-01T01:20:08.143 INFO:tasks.workunit.client.0.smithi077.stdout:40000 copy from 0x5dfcb to 0x607bc, (0x27f1 bytes) at 0xf5933 2023-12-01T01:20:09.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:09 smithi077 ceph-mon[98867]: pgmap v366: 97 pgs: 97 active+clean; 49 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.1 MiB/s wr, 627 op/s 2023-12-01T01:20:10.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:20:09 smithi203 ceph-mon[103581]: pgmap v366: 97 pgs: 97 active+clean; 49 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.1 MiB/s wr, 627 op/s 2023-12-01T01:20:10.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:20:09 smithi137 ceph-mon[103312]: pgmap v366: 97 pgs: 97 active+clean; 49 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.1 MiB/s wr, 627 op/s 2023-12-01T01:20:11.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:11 smithi077 ceph-mon[98867]: pgmap v367: 97 pgs: 97 active+clean; 47 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 522 op/s 2023-12-01T01:20:12.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:20:11 smithi203 ceph-mon[103581]: pgmap v367: 97 pgs: 97 active+clean; 47 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 522 op/s 2023-12-01T01:20:12.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:20:11 smithi137 ceph-mon[103312]: pgmap v367: 97 pgs: 97 active+clean; 47 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 522 op/s 2023-12-01T01:20:13.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:13 smithi077 ceph-mon[98867]: pgmap v368: 97 pgs: 97 active+clean; 47 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 439 op/s 2023-12-01T01:20:14.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:20:13 smithi203 ceph-mon[103581]: pgmap v368: 97 pgs: 97 active+clean; 47 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 439 op/s 2023-12-01T01:20:14.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:20:13 smithi137 ceph-mon[103312]: pgmap v368: 97 pgs: 97 active+clean; 47 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 439 op/s 2023-12-01T01:20:16.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:20:15 smithi203 ceph-mon[103581]: pgmap v369: 97 pgs: 97 active+clean; 49 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.8 MiB/s wr, 627 op/s 2023-12-01T01:20:16.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:20:15 smithi137 ceph-mon[103312]: pgmap v369: 97 pgs: 97 active+clean; 49 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.8 MiB/s wr, 627 op/s 2023-12-01T01:20:16.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:15 smithi077 ceph-mon[98867]: pgmap v369: 97 pgs: 97 active+clean; 49 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.8 MiB/s wr, 627 op/s 2023-12-01T01:20:17.267 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.265+0000 7f4bf022d700 1 -- 172.21.15.77:0/1496132354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4be80ffc90 msgr2=0x7f4be81000b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:17.267 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.265+0000 7f4bf022d700 1 --2- 172.21.15.77:0/1496132354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4be80ffc90 0x7f4be81000b0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4be40099c0 tx=0x7f4be4009cd0 comp rx=0 tx=0).stop 2023-12-01T01:20:17.267 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.265+0000 7f4bf022d700 1 -- 172.21.15.77:0/1496132354 shutdown_connections 2023-12-01T01:20:17.268 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.265+0000 7f4bf022d700 1 --2- 172.21.15.77:0/1496132354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4be8101820 0x7f4be81080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:17.268 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.265+0000 7f4bf022d700 1 --2- 172.21.15.77:0/1496132354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4be80ffc90 0x7f4be81000b0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:17.268 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.265+0000 7f4bf022d700 1 --2- 172.21.15.77:0/1496132354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4be8100e80 0x7f4be81012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:17.268 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.265+0000 7f4bf022d700 1 -- 172.21.15.77:0/1496132354 >> 172.21.15.77:0/1496132354 conn(0x7f4be80fb1e0 msgr2=0x7f4be80fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:20:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.265+0000 7f4bf022d700 1 -- 172.21.15.77:0/1496132354 shutdown_connections 2023-12-01T01:20:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.265+0000 7f4bf022d700 1 -- 172.21.15.77:0/1496132354 wait complete. 2023-12-01T01:20:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.266+0000 7f4bf022d700 1 Processor -- start 2023-12-01T01:20:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.266+0000 7f4bf022d700 1 -- start start 2023-12-01T01:20:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.266+0000 7f4bf022d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4be80ffc90 0x7f4be8199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.266+0000 7f4bf022d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4be8100e80 0x7f4be819bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.266+0000 7f4bf022d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4be8101820 0x7f4be819e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.266+0000 7f4bf022d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be819e800 con 0x7f4be8101820 2023-12-01T01:20:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.266+0000 7f4bf022d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be819e970 con 0x7f4be8100e80 2023-12-01T01:20:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.266+0000 7f4bf022d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be819eb30 con 0x7f4be80ffc90 2023-12-01T01:20:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.266+0000 7f4bedfc9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4be80ffc90 0x7f4be8199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:17.272 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.267+0000 7f4bedfc9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4be80ffc90 0x7f4be8199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:46744/0 (socket says 172.21.15.77:46744) 2023-12-01T01:20:17.272 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.267+0000 7f4bedfc9700 1 -- 172.21.15.77:0/1267306586 learned_addr learned my addr 172.21.15.77:0/1267306586 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:20:17.272 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.267+0000 7f4bed7c8700 1 --2- 172.21.15.77:0/1267306586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4be8100e80 0x7f4be819bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:17.272 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.267+0000 7f4bee7ca700 1 --2- 172.21.15.77:0/1267306586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4be8101820 0x7f4be819e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:17.272 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.267+0000 7f4bedfc9700 1 -- 172.21.15.77:0/1267306586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4be8100e80 msgr2=0x7f4be819bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:17.272 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.267+0000 7f4bedfc9700 1 --2- 172.21.15.77:0/1267306586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4be8100e80 0x7f4be819bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:17.273 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.267+0000 7f4bedfc9700 1 -- 172.21.15.77:0/1267306586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4be8101820 msgr2=0x7f4be819e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:17.273 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.267+0000 7f4bedfc9700 1 --2- 172.21.15.77:0/1267306586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4be8101820 0x7f4be819e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:17.273 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.267+0000 7f4bedfc9700 1 -- 172.21.15.77:0/1267306586 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be40095f0 con 0x7f4be80ffc90 2023-12-01T01:20:17.273 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.267+0000 7f4bedfc9700 1 --2- 172.21.15.77:0/1267306586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4be80ffc90 0x7f4be8199760 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4be4009fd0 tx=0x7f4be4009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:20:17.273 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.267+0000 7f4bdeffd700 1 -- 172.21.15.77:0/1267306586 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be401d070 con 0x7f4be80ffc90 2023-12-01T01:20:17.273 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.267+0000 7f4bf022d700 1 -- 172.21.15.77:0/1267306586 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4be819edb0 con 0x7f4be80ffc90 2023-12-01T01:20:17.274 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.267+0000 7f4bdeffd700 1 -- 172.21.15.77:0/1267306586 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4be4005030 con 0x7f4be80ffc90 2023-12-01T01:20:17.274 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.267+0000 7f4bdeffd700 1 -- 172.21.15.77:0/1267306586 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be4004a90 con 0x7f4be80ffc90 2023-12-01T01:20:17.274 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.267+0000 7f4bf022d700 1 -- 172.21.15.77:0/1267306586 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be819f180 con 0x7f4be80ffc90 2023-12-01T01:20:17.287 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.270+0000 7f4bdeffd700 1 -- 172.21.15.77:0/1267306586 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f4be40051a0 con 0x7f4be80ffc90 2023-12-01T01:20:17.287 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.271+0000 7f4bdeffd700 1 --2- 172.21.15.77:0/1267306586 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f4bd40a0520 0x7f4bd40a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:17.287 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.271+0000 7f4bdeffd700 1 -- 172.21.15.77:0/1267306586 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f4be40cc600 con 0x7f4be80ffc90 2023-12-01T01:20:17.287 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.272+0000 7f4bfc3bdb80 1 Processor -- start 2023-12-01T01:20:17.287 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.272+0000 7f4bed7c8700 1 --2- 172.21.15.77:0/1267306586 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f4bd40a0520 0x7f4bd40a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:17.287 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.272+0000 7f4bed7c8700 1 --2- 172.21.15.77:0/1267306586 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f4bd40a0520 0x7f4bd40a29e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4be8068690 tx=0x7f4bd8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:20:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.272+0000 7f4bfc3bdb80 1 -- start start 2023-12-01T01:20:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.272+0000 7f4bfc3bdb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6350b4680 0x55f6350b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.272+0000 7f4bfc3bdb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6350b4fe0 0x55f6350b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.272+0000 7f4bfc3bdb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6350b5ed0 0x55f6350b6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.272+0000 7f4bfc3bdb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f63500c9b0 con 0x55f6350b5ed0 2023-12-01T01:20:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.272+0000 7f4bfc3bdb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f63500a370 con 0x55f6350b4fe0 2023-12-01T01:20:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.272+0000 7f4bfc3bdb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f635005c60 con 0x55f6350b4680 2023-12-01T01:20:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.273+0000 7f4bee7ca700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6350b5ed0 0x55f6350b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:17.288 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.273+0000 7f4bedfc9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6350b4680 0x55f6350b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:17.289 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.273+0000 7f4bee7ca700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6350b5ed0 0x55f6350b6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60568/0 (socket says 172.21.15.77:60568) 2023-12-01T01:20:17.289 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.273+0000 7f4bee7ca700 1 -- 172.21.15.77:0/2754182537 learned_addr learned my addr 172.21.15.77:0/2754182537 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:20:17.289 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.273+0000 7f4bed7c8700 1 --2- 172.21.15.77:0/2754182537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6350b4fe0 0x55f6350b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:17.289 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.273+0000 7f4bedfc9700 1 -- 172.21.15.77:0/2754182537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6350b4fe0 msgr2=0x55f6350b58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:17.289 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.273+0000 7f4bedfc9700 1 --2- 172.21.15.77:0/2754182537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6350b4fe0 0x55f6350b58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:17.289 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.273+0000 7f4bedfc9700 1 -- 172.21.15.77:0/2754182537 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6350b5ed0 msgr2=0x55f6350b6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:17.289 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.273+0000 7f4bedfc9700 1 --2- 172.21.15.77:0/2754182537 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6350b5ed0 0x55f6350b6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:17.289 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.273+0000 7f4bedfc9700 1 -- 172.21.15.77:0/2754182537 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f634e33d80 con 0x55f6350b4680 2023-12-01T01:20:17.289 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.273+0000 7f4bee7ca700 1 --2- 172.21.15.77:0/2754182537 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6350b5ed0 0x55f6350b6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:20:17.289 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.273+0000 7f4bedfc9700 1 --2- 172.21.15.77:0/2754182537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6350b4680 0x55f6350b4aa0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4be40ca1f0 tx=0x7f4be409a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b99c188007ab6094 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:20:17.289 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.273+0000 7f4bd3fff700 1 -- 172.21.15.77:0/2754182537 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be40a1f70 con 0x55f6350b4680 2023-12-01T01:20:17.289 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.273+0000 7f4bd3fff700 1 -- 172.21.15.77:0/2754182537 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4be4098330 con 0x55f6350b4680 2023-12-01T01:20:17.289 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.273+0000 7f4bd3fff700 1 -- 172.21.15.77:0/2754182537 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be40a1ae0 con 0x55f6350b4680 2023-12-01T01:20:17.290 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.274+0000 7f4bfc3bdb80 1 -- 172.21.15.77:0/2754182537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6350b4680 msgr2=0x55f6350b4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:17.290 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.274+0000 7f4bfc3bdb80 1 --2- 172.21.15.77:0/2754182537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6350b4680 0x55f6350b4aa0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4be40ca1f0 tx=0x7f4be409a710 comp rx=0 tx=0).stop 2023-12-01T01:20:17.290 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.274+0000 7f4bfc3bdb80 1 -- 172.21.15.77:0/2754182537 shutdown_connections 2023-12-01T01:20:17.290 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.274+0000 7f4bfc3bdb80 1 --2- 172.21.15.77:0/2754182537 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6350b5ed0 0x55f6350b6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:17.291 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.274+0000 7f4bfc3bdb80 1 --2- 172.21.15.77:0/2754182537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6350b4680 0x55f6350b4aa0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:17.291 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.274+0000 7f4bfc3bdb80 1 --2- 172.21.15.77:0/2754182537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6350b4fe0 0x55f6350b58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:17.291 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.274+0000 7f4bfc3bdb80 1 -- 172.21.15.77:0/2754182537 >> 172.21.15.77:0/2754182537 conn(0x55f635014070 msgr2=0x55f6350b1040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:20:17.291 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.274+0000 7f4bfc3bdb80 1 -- 172.21.15.77:0/2754182537 shutdown_connections 2023-12-01T01:20:17.291 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.274+0000 7f4bfc3bdb80 1 -- 172.21.15.77:0/2754182537 wait complete. 2023-12-01T01:20:17.292 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.275+0000 7f4bfc3bdb80 1 -- start start 2023-12-01T01:20:17.292 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.275+0000 7f4bfc3bdb80 1 Processor -- start 2023-12-01T01:20:17.292 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.275+0000 7f4bfc3bdb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6350b4680 0x55f6350be0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:17.292 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.275+0000 7f4bfc3bdb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6350b4fe0 0x55f6350be5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:17.292 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.275+0000 7f4bfc3bdb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6350b5ed0 0x55f6350c5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:17.292 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.275+0000 7f4bfc3bdb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f634fa24b0 con 0x55f6350b4680 2023-12-01T01:20:17.293 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.275+0000 7f4bfc3bdb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f634fa2150 con 0x55f6350b4fe0 2023-12-01T01:20:17.293 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.275+0000 7f4bfc3bdb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f634e898c0 con 0x55f6350b5ed0 2023-12-01T01:20:17.293 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.275+0000 7f4bfc3bdb80 10 client.0 _ll_register_callbacks cb 0x55f634eefa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:20:17.293 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.276+0000 7f4bed7c8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6350b4fe0 0x55f6350be5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:17.293 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.276+0000 7f4bed7c8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6350b4fe0 0x55f6350be5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:39494/0 (socket says 172.21.15.77:39494) 2023-12-01T01:20:17.293 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.276+0000 7f4bed7c8700 1 -- 172.21.15.77:0/2071423586 learned_addr learned my addr 172.21.15.77:0/2071423586 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:20:17.294 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.275+0000 7f4bedfc9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6350b4680 0x55f6350be0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:17.294 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.276+0000 7f4bee7ca700 1 --2- 172.21.15.77:0/2071423586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6350b5ed0 0x55f6350c5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:17.294 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.276+0000 7f4bed7c8700 1 -- 172.21.15.77:0/2071423586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6350b5ed0 msgr2=0x55f6350c5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:17.294 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.276+0000 7f4bed7c8700 1 --2- 172.21.15.77:0/2071423586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6350b5ed0 0x55f6350c5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:17.294 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.276+0000 7f4bed7c8700 1 -- 172.21.15.77:0/2071423586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6350b4680 msgr2=0x55f6350be0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:17.295 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.276+0000 7f4bed7c8700 1 --2- 172.21.15.77:0/2071423586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6350b4680 0x55f6350be0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:17.295 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.276+0000 7f4bed7c8700 1 -- 172.21.15.77:0/2071423586 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be0011330 con 0x55f6350b4fe0 2023-12-01T01:20:17.295 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.276+0000 7f4bee7ca700 1 --2- 172.21.15.77:0/2071423586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6350b5ed0 0x55f6350c5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:20:17.295 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.276+0000 7f4bed7c8700 1 --2- 172.21.15.77:0/2071423586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6350b4fe0 0x55f6350be5f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4bd80045b0 tx=0x7f4bd80047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=277717435954ae25 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:20:17.295 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.276+0000 7f4bd17fa700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2023-12-01T01:20:17.296 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.276+0000 7f4bd17fa700 1 -- 172.21.15.77:0/2071423586 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd8022070 con 0x55f6350b4fe0 2023-12-01T01:20:17.296 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.276+0000 7f4bd17fa700 1 -- 172.21.15.77:0/2071423586 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f4bd8003e20 con 0x55f6350b4fe0 2023-12-01T01:20:17.296 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.276+0000 7f4bfc3bdb80 1 -- 172.21.15.77:0/2071423586 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f635030cb0 con 0x55f6350b4fe0 2023-12-01T01:20:17.296 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.277+0000 7f4bd17fa700 1 -- 172.21.15.77:0/2071423586 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4bd8004830 con 0x55f6350b4fe0 2023-12-01T01:20:17.296 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.277+0000 7f4bfc3bdb80 10 client.25574 fetch_fsmap learned FSMap version 31 2023-12-01T01:20:17.296 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.277+0000 7f4bfc3bdb80 1 -- 172.21.15.77:0/2071423586 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f4be819f180 con 0x55f6350b4fe0 2023-12-01T01:20:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.277+0000 7f4bd17fa700 1 -- 172.21.15.77:0/2071423586 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f635030cb0 con 0x55f6350b4fe0 2023-12-01T01:20:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.277+0000 7f4bd17fa700 1 -- 172.21.15.77:0/2071423586 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4bd801d620 con 0x55f6350b4fe0 2023-12-01T01:20:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.277+0000 7f4bfc3bdb80 10 client.25574 fetch_fsmap finished waiting for FSMap version 31 2023-12-01T01:20:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.277+0000 7f4bfc3bdb80 10 client.25574 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-01T01:20:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.277+0000 7f4bfc3bdb80 20 client.25574 populate_metadata read hostname 'smithi077' 2023-12-01T01:20:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.277+0000 7f4bfc3bdb80 1 --2- 172.21.15.77:0/2071423586 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55f6350ca070 0x55f6350cc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.277+0000 7f4bfc3bdb80 4 client.25574 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:20:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.277+0000 7f4bfc3bdb80 1 -- 172.21.15.77:0/2071423586 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f635083db0 con 0x55f6350ca070 2023-12-01T01:20:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.277+0000 7f4bedfc9700 1 --2- 172.21.15.77:0/2071423586 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55f6350ca070 0x55f6350cc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.278+0000 7f4bd17fa700 10 client.25574 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:20:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.278+0000 7f4bedfc9700 1 --2- 172.21.15.77:0/2071423586 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55f6350ca070 0x55f6350cc490 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29d1f73188307701 server_cookie=8055e7cc0e74b427 in_seq=0 out_seq=0 2023-12-01T01:20:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.278+0000 7f4bd17fa700 1 -- 172.21.15.77:0/2071423586 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55f635083db0 con 0x55f6350ca070 2023-12-01T01:20:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.278+0000 7f4bd17fa700 10 client.25574 handle_command_reply: tid=0 2023-12-01T01:20:17.300 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.278+0000 7f4bfc3bdb80 1 client.25574 shutdown 2023-12-01T01:20:17.300 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.278+0000 7f4bfc3bdb80 1 -- 172.21.15.77:0/2071423586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6350b4fe0 msgr2=0x55f6350be5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:17.300 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.278+0000 7f4bfc3bdb80 1 --2- 172.21.15.77:0/2071423586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6350b4fe0 0x55f6350be5f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4bd80045b0 tx=0x7f4bd80047c0 comp rx=0 tx=0).stop 2023-12-01T01:20:17.300 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.279+0000 7f4bfc3bdb80 1 -- 172.21.15.77:0/2071423586 shutdown_connections 2023-12-01T01:20:17.301 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.279+0000 7f4bfc3bdb80 1 --2- 172.21.15.77:0/2071423586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6350b5ed0 0x55f6350c5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:17.301 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.279+0000 7f4bfc3bdb80 1 --2- 172.21.15.77:0/2071423586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6350b4680 0x55f6350be0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:17.301 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.279+0000 7f4bfc3bdb80 1 --2- 172.21.15.77:0/2071423586 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55f6350ca070 0x55f6350cc490 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:17.301 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.279+0000 7f4bfc3bdb80 1 --2- 172.21.15.77:0/2071423586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6350b4fe0 0x55f6350be5f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.279+0000 7f4bd17fa700 0 client.25574 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:20:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.279+0000 7f4bfc3bdb80 1 -- 172.21.15.77:0/2071423586 >> 172.21.15.77:0/2071423586 conn(0x55f635014070 msgr2=0x55f6350af3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:20:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.279+0000 7f4bfc3bdb80 1 -- 172.21.15.77:0/2071423586 shutdown_connections 2023-12-01T01:20:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.279+0000 7f4bfc3bdb80 1 -- 172.21.15.77:0/2071423586 wait complete. 2023-12-01T01:20:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.279+0000 7f4bfc3bdb80 20 client.25574 trim_cache size 0 max 16384 2023-12-01T01:20:17.309 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.296+0000 7f4bfc3bdb80 1 Processor -- start 2023-12-01T01:20:17.309 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.296+0000 7f4bfc3bdb80 1 -- start start 2023-12-01T01:20:17.309 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.296+0000 7f4bfc3bdb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6350b4680 0x55f6350b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:17.309 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.296+0000 7f4bfc3bdb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6350b4fe0 0x55f6350b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:17.309 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.296+0000 7f4bfc3bdb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6350b5ed0 0x55f6350bc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:17.309 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.296+0000 7f4bfc3bdb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6350a77a0 con 0x55f6350b4680 2023-12-01T01:20:17.309 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.296+0000 7f4bfc3bdb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6350befb0 con 0x55f6350b4fe0 2023-12-01T01:20:17.309 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.296+0000 7f4bfc3bdb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6350aa500 con 0x55f6350b5ed0 2023-12-01T01:20:17.309 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.297+0000 7f4bedfc9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6350b4680 0x55f6350b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:17.309 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.297+0000 7f4bedfc9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6350b4680 0x55f6350b62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60578/0 (socket says 172.21.15.77:60578) 2023-12-01T01:20:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.297+0000 7f4bedfc9700 1 -- 172.21.15.77:0/3796410254 learned_addr learned my addr 172.21.15.77:0/3796410254 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:20:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.297+0000 7f4bed7c8700 1 --2- 172.21.15.77:0/3796410254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6350b4fe0 0x55f6350b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.297+0000 7f4bee7ca700 1 --2- 172.21.15.77:0/3796410254 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6350b5ed0 0x55f6350bc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.297+0000 7f4bedfc9700 1 -- 172.21.15.77:0/3796410254 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6350b5ed0 msgr2=0x55f6350bc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.297+0000 7f4bedfc9700 1 --2- 172.21.15.77:0/3796410254 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6350b5ed0 0x55f6350bc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.297+0000 7f4bedfc9700 1 -- 172.21.15.77:0/3796410254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6350b4fe0 msgr2=0x55f6350b0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.297+0000 7f4bedfc9700 1 --2- 172.21.15.77:0/3796410254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6350b4fe0 0x55f6350b0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.297+0000 7f4bedfc9700 1 -- 172.21.15.77:0/3796410254 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f634e1cc00 con 0x55f6350b4680 2023-12-01T01:20:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.297+0000 7f4bed7c8700 1 --2- 172.21.15.77:0/3796410254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6350b4fe0 0x55f6350b0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:20:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.297+0000 7f4bedfc9700 1 --2- 172.21.15.77:0/3796410254 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6350b4680 0x55f6350b62b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4be40ca1f0 tx=0x7f4be409a800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e14bd4bef04cc7 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:20:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.297+0000 7f4bdcff9700 1 -- 172.21.15.77:0/3796410254 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be4099d60 con 0x55f6350b4680 2023-12-01T01:20:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.297+0000 7f4bdcff9700 1 -- 172.21.15.77:0/3796410254 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4be4099560 con 0x55f6350b4680 2023-12-01T01:20:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.297+0000 7f4bdcff9700 1 -- 172.21.15.77:0/3796410254 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be4004bf0 con 0x55f6350b4680 2023-12-01T01:20:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.298+0000 7f4bfc3bdb80 1 -- 172.21.15.77:0/3796410254 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6350b4680 msgr2=0x55f6350b62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.298+0000 7f4bfc3bdb80 1 --2- 172.21.15.77:0/3796410254 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6350b4680 0x55f6350b62b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4be40ca1f0 tx=0x7f4be409a800 comp rx=0 tx=0).stop 2023-12-01T01:20:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.298+0000 7f4bfc3bdb80 1 -- 172.21.15.77:0/3796410254 shutdown_connections 2023-12-01T01:20:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.298+0000 7f4bfc3bdb80 1 --2- 172.21.15.77:0/3796410254 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6350b5ed0 0x55f6350bc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.298+0000 7f4bfc3bdb80 1 --2- 172.21.15.77:0/3796410254 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6350b4680 0x55f6350b62b0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.298+0000 7f4bfc3bdb80 1 --2- 172.21.15.77:0/3796410254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6350b4fe0 0x55f6350b0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.298+0000 7f4bfc3bdb80 1 -- 172.21.15.77:0/3796410254 >> 172.21.15.77:0/3796410254 conn(0x55f635014070 msgr2=0x55f6350ae660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:20:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.298+0000 7f4bfc3bdb80 1 -- 172.21.15.77:0/3796410254 shutdown_connections 2023-12-01T01:20:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.298+0000 7f4bfc3bdb80 1 -- 172.21.15.77:0/3796410254 wait complete. 2023-12-01T01:20:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.298+0000 7f4bfc3bdb80 1 -- start start 2023-12-01T01:20:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.299+0000 7f4bfc3bdb80 1 Processor -- start 2023-12-01T01:20:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.299+0000 7f4bfc3bdb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6350b4680 0x55f635117290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.299+0000 7f4bfc3bdb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6350b4fe0 0x55f6351177d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.299+0000 7f4bfc3bdb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6350b5ed0 0x55f63511e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.299+0000 7f4bfc3bdb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f63511ed50 con 0x55f6350b5ed0 2023-12-01T01:20:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.299+0000 7f4bfc3bdb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f63511ee90 con 0x55f6350b4680 2023-12-01T01:20:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.299+0000 7f4bfc3bdb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f63511f020 con 0x55f6350b4fe0 2023-12-01T01:20:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.299+0000 7f4bfc3bdb80 10 client.0 _ll_register_callbacks cb 0x7f4bd80043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:20:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.299+0000 7f4bee7ca700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6350b5ed0 0x55f63511e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.299+0000 7f4bed7c8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6350b4fe0 0x55f6351177d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.299+0000 7f4bee7ca700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6350b5ed0 0x55f63511e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60588/0 (socket says 172.21.15.77:60588) 2023-12-01T01:20:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.299+0000 7f4bee7ca700 1 -- 172.21.15.77:0/2236125724 learned_addr learned my addr 172.21.15.77:0/2236125724 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:20:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.299+0000 7f4bed7c8700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6350b4fe0 0x55f6351177d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:46774/0 (socket says 172.21.15.77:46774) 2023-12-01T01:20:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.299+0000 7f4bedfc9700 1 --2- 172.21.15.77:0/2236125724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6350b4680 0x55f635117290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.299+0000 7f4bee7ca700 1 -- 172.21.15.77:0/2236125724 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6350b4fe0 msgr2=0x55f6351177d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.299+0000 7f4bee7ca700 1 --2- 172.21.15.77:0/2236125724 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6350b4fe0 0x55f6351177d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.299+0000 7f4bee7ca700 1 -- 172.21.15.77:0/2236125724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6350b4680 msgr2=0x55f635117290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.299+0000 7f4bee7ca700 1 --2- 172.21.15.77:0/2236125724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6350b4680 0x55f635117290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.299+0000 7f4bee7ca700 1 -- 172.21.15.77:0/2236125724 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f634e33d80 con 0x55f6350b5ed0 2023-12-01T01:20:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.299+0000 7f4bed7c8700 1 --2- 172.21.15.77:0/2236125724 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6350b4fe0 0x55f6351177d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:20:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.299+0000 7f4bedfc9700 1 --2- 172.21.15.77:0/2236125724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6350b4680 0x55f635117290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:20:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.300+0000 7f4bee7ca700 1 --2- 172.21.15.77:0/2236125724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6350b5ed0 0x55f63511e810 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f4be0007f00 tx=0x7f4be0007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8bf7f43057d70c6 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:20:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.300+0000 7f4bd17fa700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:20:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.300+0000 7f4bd17fa700 1 -- 172.21.15.77:0/2236125724 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be000e040 con 0x55f6350b5ed0 2023-12-01T01:20:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.300+0000 7f4bd17fa700 1 -- 172.21.15.77:0/2236125724 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f4be00119f0 con 0x55f6350b5ed0 2023-12-01T01:20:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.300+0000 7f4bfc3bdb80 1 -- 172.21.15.77:0/2236125724 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4be4098330 con 0x55f6350b5ed0 2023-12-01T01:20:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.300+0000 7f4bd17fa700 1 -- 172.21.15.77:0/2236125724 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4be0011bb0 con 0x55f6350b5ed0 2023-12-01T01:20:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.300+0000 7f4bfc3bdb80 10 client.15999 fetch_fsmap learned FSMap version 31 2023-12-01T01:20:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.300+0000 7f4bfc3bdb80 1 -- 172.21.15.77:0/2236125724 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55f634e5a1f0 con 0x55f6350b5ed0 2023-12-01T01:20:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.300+0000 7f4bd17fa700 1 -- 172.21.15.77:0/2236125724 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4be4098330 con 0x55f6350b5ed0 2023-12-01T01:20:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.300+0000 7f4bd17fa700 1 -- 172.21.15.77:0/2236125724 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4be0017410 con 0x55f6350b5ed0 2023-12-01T01:20:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.300+0000 7f4bfc3bdb80 10 client.15999 fetch_fsmap finished waiting for FSMap version 31 2023-12-01T01:20:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.300+0000 7f4bfc3bdb80 10 client.15999 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-01T01:20:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.300+0000 7f4bfc3bdb80 20 client.15999 populate_metadata read hostname 'smithi077' 2023-12-01T01:20:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.300+0000 7f4bfc3bdb80 1 --2- 172.21.15.77:0/2236125724 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55f635120c20 0x55f635123040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.300+0000 7f4bfc3bdb80 4 client.15999 mds_command: new command op to 24500 tid=0[{"prefix": "flush journal"}] 2023-12-01T01:20:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.300+0000 7f4bfc3bdb80 1 -- 172.21.15.77:0/2236125724 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f634ffece0 con 0x55f635120c20 2023-12-01T01:20:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.301+0000 7f4bedfc9700 1 --2- 172.21.15.77:0/2236125724 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55f635120c20 0x55f635123040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.301+0000 7f4bd17fa700 10 client.15999 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:20:17.314 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.301+0000 7f4bedfc9700 1 --2- 172.21.15.77:0/2236125724 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55f635120c20 0x55f635123040 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f66edc33f6be11f2 server_cookie=66ddfedb55976e10 in_seq=0 out_seq=0 2023-12-01T01:20:17.321 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.319+0000 7f4bd17fa700 1 -- 172.21.15.77:0/2236125724 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f634ffece0 con 0x55f635120c20 2023-12-01T01:20:17.321 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.319+0000 7f4bd17fa700 10 client.15999 handle_command_reply: tid=0 2023-12-01T01:20:17.321 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.319+0000 7f4bfc3bdb80 1 client.15999 shutdown 2023-12-01T01:20:17.321 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.319+0000 7f4bfc3bdb80 1 -- 172.21.15.77:0/2236125724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6350b5ed0 msgr2=0x55f63511e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:17.321 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.319+0000 7f4bfc3bdb80 1 --2- 172.21.15.77:0/2236125724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6350b5ed0 0x55f63511e810 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f4be0007f00 tx=0x7f4be0007f30 comp rx=0 tx=0).stop 2023-12-01T01:20:17.321 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.319+0000 7f4bfc3bdb80 1 -- 172.21.15.77:0/2236125724 shutdown_connections 2023-12-01T01:20:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.319+0000 7f4bfc3bdb80 1 --2- 172.21.15.77:0/2236125724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6350b5ed0 0x55f63511e810 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.319+0000 7f4bfc3bdb80 1 --2- 172.21.15.77:0/2236125724 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55f635120c20 0x55f635123040 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.319+0000 7f4bfc3bdb80 1 --2- 172.21.15.77:0/2236125724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6350b4680 0x55f635117290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.319+0000 7f4bfc3bdb80 1 --2- 172.21.15.77:0/2236125724 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6350b4fe0 0x55f6351177d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.319+0000 7f4bd17fa700 0 client.15999 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:20:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.319+0000 7f4bfc3bdb80 1 -- 172.21.15.77:0/2236125724 >> 172.21.15.77:0/2236125724 conn(0x55f635014070 msgr2=0x55f6350b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:20:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.319+0000 7f4bfc3bdb80 1 -- 172.21.15.77:0/2236125724 shutdown_connections 2023-12-01T01:20:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.319+0000 7f4bfc3bdb80 1 -- 172.21.15.77:0/2236125724 wait complete. 2023-12-01T01:20:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.320+0000 7f4bfc3bdb80 20 client.15999 trim_cache size 0 max 16384 2023-12-01T01:20:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout:{ 2023-12-01T01:20:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout: "message": "", 2023-12-01T01:20:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout: "return_code": 0 2023-12-01T01:20:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout:} 2023-12-01T01:20:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.320+0000 7f4bf022d700 1 -- 172.21.15.77:0/1267306586 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f4bd40a0520 msgr2=0x7f4bd40a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.320+0000 7f4bf022d700 1 --2- 172.21.15.77:0/1267306586 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f4bd40a0520 0x7f4bd40a29e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4be8068690 tx=0x7f4bd8009260 comp rx=0 tx=0).stop 2023-12-01T01:20:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.320+0000 7f4bf022d700 1 -- 172.21.15.77:0/1267306586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4be80ffc90 msgr2=0x7f4be8199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.320+0000 7f4bf022d700 1 --2- 172.21.15.77:0/1267306586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4be80ffc90 0x7f4be8199760 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4be4009fd0 tx=0x7f4be4009580 comp rx=0 tx=0).stop 2023-12-01T01:20:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.321+0000 7f4bf022d700 1 -- 172.21.15.77:0/1267306586 shutdown_connections 2023-12-01T01:20:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.321+0000 7f4bf022d700 1 --2- 172.21.15.77:0/1267306586 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f4bd40a0520 0x7f4bd40a29e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.321+0000 7f4bf022d700 1 --2- 172.21.15.77:0/1267306586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4be8101820 0x7f4be819e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.321+0000 7f4bf022d700 1 --2- 172.21.15.77:0/1267306586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4be80ffc90 0x7f4be8199760 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.321+0000 7f4bf022d700 1 --2- 172.21.15.77:0/1267306586 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4be8100e80 0x7f4be819bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.321+0000 7f4bf022d700 1 -- 172.21.15.77:0/1267306586 >> 172.21.15.77:0/1267306586 conn(0x7f4be80fb1e0 msgr2=0x7f4be80fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:20:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.321+0000 7f4bf022d700 1 -- 172.21.15.77:0/1267306586 shutdown_connections 2023-12-01T01:20:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:17.321+0000 7f4bf022d700 1 -- 172.21.15.77:0/1267306586 wait complete. 2023-12-01T01:20:18.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:20:17 smithi203 ceph-mon[103581]: pgmap v370: 97 pgs: 97 active+clean; 49 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 419 op/s 2023-12-01T01:20:18.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:20:17 smithi137 ceph-mon[103312]: pgmap v370: 97 pgs: 97 active+clean; 49 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 419 op/s 2023-12-01T01:20:18.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:17 smithi077 ceph-mon[98867]: pgmap v370: 97 pgs: 97 active+clean; 49 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 419 op/s 2023-12-01T01:20:20.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:20:19 smithi203 ceph-mon[103581]: pgmap v371: 97 pgs: 97 active+clean; 49 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 532 op/s 2023-12-01T01:20:20.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:20:19 smithi137 ceph-mon[103312]: pgmap v371: 97 pgs: 97 active+clean; 49 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 532 op/s 2023-12-01T01:20:20.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:19 smithi077 ceph-mon[98867]: pgmap v371: 97 pgs: 97 active+clean; 49 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 532 op/s 2023-12-01T01:20:20.586 INFO:tasks.workunit.client.0.smithi077.stdout:50000 mapwrite 0xd9535 thru 0xdf124 (0x5bf0 bytes) 2023-12-01T01:20:20.590 INFO:tasks.workunit.client.0.smithi077.stdout:All 50000 operations completed A-OK! 2023-12-01T01:20:20.591 INFO:tasks.workunit.client.0.smithi077.stderr:+ ./fsx -z 10MB -N 50000 -p 10000 -l 10485760 2023-12-01T01:20:20.591 INFO:tasks.workunit.client.0.smithi077.stdout:Seed set to 1 2023-12-01T01:20:20.720 INFO:tasks.workunit.client.0.smithi077.stderr:main: filesystem does not support fallocate mode 0, disabling! 2023-12-01T01:20:20.720 INFO:tasks.workunit.client.0.smithi077.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2023-12-01T01:20:20.722 INFO:tasks.workunit.client.0.smithi077.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2023-12-01T01:20:20.722 INFO:tasks.workunit.client.0.smithi077.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2023-12-01T01:20:20.722 INFO:tasks.workunit.client.0.smithi077.stderr:main: filesystem does not support clone range, disabling! 2023-12-01T01:20:20.726 INFO:tasks.workunit.client.0.smithi077.stderr:main: filesystem does not support dedupe range, disabling! 2023-12-01T01:20:20.728 INFO:tasks.workunit.client.0.smithi077.stdout:truncating to largest ever: 0x5a0dfb 2023-12-01T01:20:20.740 INFO:tasks.workunit.client.0.smithi077.stdout:copying to largest ever: 0x5c415c 2023-12-01T01:20:20.741 INFO:tasks.workunit.client.0.smithi077.stdout:truncating to largest ever: 0x9c27b4 2023-12-01T01:20:20.800 INFO:tasks.workunit.client.0.smithi077.stdout:copying to largest ever: 0x9d03f4 2023-12-01T01:20:20.875 INFO:tasks.workunit.client.0.smithi077.stdout:copying to largest ever: 0x9f6462 2023-12-01T01:20:20.956 INFO:tasks.workunit.client.0.smithi077.stdout:copying to largest ever: 0x9fa7e5 2023-12-01T01:20:22.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:20:21 smithi203 ceph-mon[103581]: pgmap v372: 97 pgs: 97 active+clean; 46 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 413 op/s 2023-12-01T01:20:22.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:20:21 smithi137 ceph-mon[103312]: pgmap v372: 97 pgs: 97 active+clean; 46 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 413 op/s 2023-12-01T01:20:22.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:21 smithi077 ceph-mon[98867]: pgmap v372: 97 pgs: 97 active+clean; 46 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 413 op/s 2023-12-01T01:20:23.036 INFO:tasks.workunit.client.0.smithi077.stdout:truncating to largest ever: 0x9fd4ba 2023-12-01T01:20:23.429 INFO:tasks.workunit.client.0.smithi077.stdout:truncating to largest ever: 0x9fe323 2023-12-01T01:20:24.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:20:23 smithi203 ceph-mon[103581]: pgmap v373: 97 pgs: 97 active+clean; 47 MiB data, 825 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 399 op/s 2023-12-01T01:20:24.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:20:23 smithi137 ceph-mon[103312]: pgmap v373: 97 pgs: 97 active+clean; 47 MiB data, 825 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 399 op/s 2023-12-01T01:20:24.153 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:23 smithi077 ceph-mon[98867]: pgmap v373: 97 pgs: 97 active+clean; 47 MiB data, 825 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 399 op/s 2023-12-01T01:20:25.108 INFO:tasks.workunit.client.0.smithi077.stdout:truncating to largest ever: 0x9fec09 2023-12-01T01:20:26.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:20:25 smithi203 ceph-mon[103581]: pgmap v374: 97 pgs: 97 active+clean; 47 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 552 op/s 2023-12-01T01:20:26.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:20:25 smithi137 ceph-mon[103312]: pgmap v374: 97 pgs: 97 active+clean; 47 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 552 op/s 2023-12-01T01:20:26.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:25 smithi077 ceph-mon[98867]: pgmap v374: 97 pgs: 97 active+clean; 47 MiB data, 811 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 552 op/s 2023-12-01T01:20:26.415 INFO:tasks.workunit.client.0.smithi077.stdout:copying to largest ever: 0x9ff6e7 2023-12-01T01:20:27.704 INFO:tasks.workunit.client.0.smithi077.stdout:10000 copy from 0x2ab252 to 0x2b6f08, (0xbcb6 bytes) at 0x1cd955 2023-12-01T01:20:27.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:27 smithi077 ceph-mon[98867]: pgmap v375: 97 pgs: 97 active+clean; 54 MiB data, 831 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.9 MiB/s wr, 562 op/s 2023-12-01T01:20:28.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:20:27 smithi203 ceph-mon[103581]: pgmap v375: 97 pgs: 97 active+clean; 54 MiB data, 831 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.9 MiB/s wr, 562 op/s 2023-12-01T01:20:28.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:20:27 smithi137 ceph-mon[103312]: pgmap v375: 97 pgs: 97 active+clean; 54 MiB data, 831 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.9 MiB/s wr, 562 op/s 2023-12-01T01:20:29.040 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-01T01:20:29.191 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.189+0000 7f138d49c700 1 -- 172.21.15.77:0/1704399124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13881001b0 msgr2=0x7f13881005d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:29.192 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.189+0000 7f138d49c700 1 --2- 172.21.15.77:0/1704399124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13881001b0 0x7f13881005d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f1378009990 tx=0x7f1378009ca0 comp rx=0 tx=0).stop 2023-12-01T01:20:29.192 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.189+0000 7f138d49c700 1 -- 172.21.15.77:0/1704399124 shutdown_connections 2023-12-01T01:20:29.192 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.189+0000 7f138d49c700 1 --2- 172.21.15.77:0/1704399124 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1388101880 0x7f1388108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:29.192 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.189+0000 7f138d49c700 1 --2- 172.21.15.77:0/1704399124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13881001b0 0x7f13881005d0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:29.192 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.189+0000 7f138d49c700 1 --2- 172.21.15.77:0/1704399124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1388100ec0 0x7f1388101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:29.192 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.189+0000 7f138d49c700 1 -- 172.21.15.77:0/1704399124 >> 172.21.15.77:0/1704399124 conn(0x7f13880fa960 msgr2=0x7f13880fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:20:29.192 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.190+0000 7f138d49c700 1 -- 172.21.15.77:0/1704399124 shutdown_connections 2023-12-01T01:20:29.192 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.190+0000 7f138d49c700 1 -- 172.21.15.77:0/1704399124 wait complete. 2023-12-01T01:20:29.192 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.190+0000 7f138d49c700 1 Processor -- start 2023-12-01T01:20:29.192 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.190+0000 7f138d49c700 1 -- start start 2023-12-01T01:20:29.194 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.191+0000 7f138d49c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13881001b0 0x7f13880ff890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:29.194 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.191+0000 7f138d49c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1388100ec0 0x7f138819b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:29.194 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.191+0000 7f138d49c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1388101880 0x7f138819df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:29.194 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.191+0000 7f138d49c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138819e6d0 con 0x7f1388100ec0 2023-12-01T01:20:29.194 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.191+0000 7f138d49c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138819e840 con 0x7f1388101880 2023-12-01T01:20:29.194 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.191+0000 7f138d49c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138819ea00 con 0x7f13881001b0 2023-12-01T01:20:29.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.191+0000 7f1386ffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13881001b0 0x7f13880ff890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:29.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.191+0000 7f1386ffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13881001b0 0x7f13880ff890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:46784/0 (socket says 172.21.15.77:46784) 2023-12-01T01:20:29.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.191+0000 7f1386ffd700 1 -- 172.21.15.77:0/3565849094 learned_addr learned my addr 172.21.15.77:0/3565849094 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:20:29.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.191+0000 7f13867fc700 1 --2- 172.21.15.77:0/3565849094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1388100ec0 0x7f138819b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:29.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.191+0000 7f13877fe700 1 --2- 172.21.15.77:0/3565849094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1388101880 0x7f138819df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:29.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.191+0000 7f1386ffd700 1 -- 172.21.15.77:0/3565849094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1388101880 msgr2=0x7f138819df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:29.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.191+0000 7f1386ffd700 1 --2- 172.21.15.77:0/3565849094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1388101880 0x7f138819df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:29.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.191+0000 7f1386ffd700 1 -- 172.21.15.77:0/3565849094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1388100ec0 msgr2=0x7f138819b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:29.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.191+0000 7f1386ffd700 1 --2- 172.21.15.77:0/3565849094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1388100ec0 0x7f138819b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:29.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.191+0000 7f1386ffd700 1 -- 172.21.15.77:0/3565849094 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13780095f0 con 0x7f13881001b0 2023-12-01T01:20:29.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.191+0000 7f1386ffd700 1 --2- 172.21.15.77:0/3565849094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13881001b0 0x7f13880ff890 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1378004730 tx=0x7f1378004760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:20:29.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.191+0000 7f136ffff700 1 -- 172.21.15.77:0/3565849094 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137801d070 con 0x7f13881001b0 2023-12-01T01:20:29.195 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.192+0000 7f138d49c700 1 -- 172.21.15.77:0/3565849094 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f138819ec80 con 0x7f13881001b0 2023-12-01T01:20:29.196 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.192+0000 7f136ffff700 1 -- 172.21.15.77:0/3565849094 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1378021410 con 0x7f13881001b0 2023-12-01T01:20:29.196 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.192+0000 7f136ffff700 1 -- 172.21.15.77:0/3565849094 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1378017650 con 0x7f13881001b0 2023-12-01T01:20:29.196 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.192+0000 7f138d49c700 1 -- 172.21.15.77:0/3565849094 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f138819f100 con 0x7f13881001b0 2023-12-01T01:20:29.198 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.194+0000 7f136ffff700 1 -- 172.21.15.77:0/3565849094 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f1378021580 con 0x7f13881001b0 2023-12-01T01:20:29.198 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.195+0000 7f136ffff700 1 --2- 172.21.15.77:0/3565849094 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f13700a0520 0x7f13700a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:29.198 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.195+0000 7f136ffff700 1 -- 172.21.15.77:0/3565849094 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f13780ccd20 con 0x7f13881001b0 2023-12-01T01:20:29.198 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.195+0000 7f13867fc700 1 --2- 172.21.15.77:0/3565849094 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f13700a0520 0x7f13700a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:29.198 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.195+0000 7f139962cb80 1 Processor -- start 2023-12-01T01:20:29.198 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.195+0000 7f139962cb80 1 -- start start 2023-12-01T01:20:29.198 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.196+0000 7f13867fc700 1 --2- 172.21.15.77:0/3565849094 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f13700a0520 0x7f13700a29e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f13880ff600 tx=0x7f137c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:20:29.198 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.196+0000 7f139962cb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55825b7c60f0 0x55825b7c6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:29.199 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.196+0000 7f139962cb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55825b7c6a70 0x55825b7c4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:29.199 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.196+0000 7f139962cb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55825b7c53f0 0x55825b7c5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:29.199 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.196+0000 7f139962cb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55825b71ca10 con 0x55825b7c53f0 2023-12-01T01:20:29.199 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.196+0000 7f139962cb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55825b71a3d0 con 0x55825b7c60f0 2023-12-01T01:20:29.199 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.196+0000 7f139962cb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55825b715cc0 con 0x55825b7c6a70 2023-12-01T01:20:29.199 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.196+0000 7f1386ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55825b7c60f0 0x55825b7c6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:29.199 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.196+0000 7f13877fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55825b7c53f0 0x55825b7c5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:29.199 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.196+0000 7f1386ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55825b7c60f0 0x55825b7c6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:39526/0 (socket says 172.21.15.77:39526) 2023-12-01T01:20:29.199 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.196+0000 7f1386ffd700 1 -- 172.21.15.77:0/1798409806 learned_addr learned my addr 172.21.15.77:0/1798409806 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:20:29.199 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.196+0000 7f13867fc700 1 --2- 172.21.15.77:0/1798409806 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55825b7c6a70 0x55825b7c4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:29.199 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.196+0000 7f1386ffd700 1 -- 172.21.15.77:0/1798409806 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55825b7c6a70 msgr2=0x55825b7c4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:29.199 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.196+0000 7f1386ffd700 1 --2- 172.21.15.77:0/1798409806 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55825b7c6a70 0x55825b7c4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:29.199 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.196+0000 7f1386ffd700 1 -- 172.21.15.77:0/1798409806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55825b7c53f0 msgr2=0x55825b7c5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:29.199 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.196+0000 7f1386ffd700 1 --2- 172.21.15.77:0/1798409806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55825b7c53f0 0x55825b7c5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:29.199 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.196+0000 7f1386ffd700 1 -- 172.21.15.77:0/1798409806 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55825b543de0 con 0x55825b7c60f0 2023-12-01T01:20:29.200 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.196+0000 7f13877fe700 1 --2- 172.21.15.77:0/1798409806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55825b7c53f0 0x55825b7c5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:20:29.200 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.196+0000 7f1386ffd700 1 --2- 172.21.15.77:0/1798409806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55825b7c60f0 0x55825b7c6530 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f137809a640 tx=0x7f137809a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6288c9e97ade8750 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:20:29.200 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.196+0000 7f136d7fa700 1 -- 172.21.15.77:0/1798409806 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13780b0070 con 0x55825b7c60f0 2023-12-01T01:20:29.200 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.197+0000 7f136d7fa700 1 -- 172.21.15.77:0/1798409806 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f137809a6b0 con 0x55825b7c60f0 2023-12-01T01:20:29.200 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.197+0000 7f136d7fa700 1 -- 172.21.15.77:0/1798409806 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137809a910 con 0x55825b7c60f0 2023-12-01T01:20:29.204 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.197+0000 7f139962cb80 1 -- 172.21.15.77:0/1798409806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55825b7c60f0 msgr2=0x55825b7c6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:29.204 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.197+0000 7f139962cb80 1 --2- 172.21.15.77:0/1798409806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55825b7c60f0 0x55825b7c6530 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f137809a640 tx=0x7f137809a270 comp rx=0 tx=0).stop 2023-12-01T01:20:29.204 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.197+0000 7f139962cb80 1 -- 172.21.15.77:0/1798409806 shutdown_connections 2023-12-01T01:20:29.204 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.197+0000 7f139962cb80 1 --2- 172.21.15.77:0/1798409806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55825b7c53f0 0x55825b7c5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:29.204 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.197+0000 7f139962cb80 1 --2- 172.21.15.77:0/1798409806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55825b7c60f0 0x55825b7c6530 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:29.204 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.197+0000 7f139962cb80 1 --2- 172.21.15.77:0/1798409806 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55825b7c6a70 0x55825b7c4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:29.204 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.197+0000 7f139962cb80 1 -- 172.21.15.77:0/1798409806 >> 172.21.15.77:0/1798409806 conn(0x55825b7240d0 msgr2=0x55825b7bf940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:20:29.205 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.197+0000 7f139962cb80 1 -- 172.21.15.77:0/1798409806 shutdown_connections 2023-12-01T01:20:29.205 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.197+0000 7f139962cb80 1 -- 172.21.15.77:0/1798409806 wait complete. 2023-12-01T01:20:29.205 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.198+0000 7f139962cb80 1 -- start start 2023-12-01T01:20:29.205 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.198+0000 7f139962cb80 1 Processor -- start 2023-12-01T01:20:29.205 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.198+0000 7f139962cb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55825b7c53f0 0x55825b7ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:29.205 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.198+0000 7f139962cb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55825b7c60f0 0x55825b7cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:29.205 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.198+0000 7f139962cb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55825b7c6a70 0x55825b7d5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:29.205 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.198+0000 7f139962cb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55825b6b2510 con 0x55825b7c53f0 2023-12-01T01:20:29.205 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.198+0000 7f139962cb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55825b6b21b0 con 0x55825b7c60f0 2023-12-01T01:20:29.205 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.198+0000 7f139962cb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55825b599920 con 0x55825b7c6a70 2023-12-01T01:20:29.205 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.198+0000 7f139962cb80 10 client.0 _ll_register_callbacks cb 0x55825b5ffa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:20:29.205 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.198+0000 7f1386ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55825b7c53f0 0x55825b7ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:29.205 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.198+0000 7f1386ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55825b7c53f0 0x55825b7ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60610/0 (socket says 172.21.15.77:60610) 2023-12-01T01:20:29.205 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.198+0000 7f1386ffd700 1 -- 172.21.15.77:0/3365659570 learned_addr learned my addr 172.21.15.77:0/3365659570 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:20:29.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.198+0000 7f13867fc700 1 --2- 172.21.15.77:0/3365659570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55825b7c60f0 0x55825b7cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:29.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.198+0000 7f13877fe700 1 --2- 172.21.15.77:0/3365659570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55825b7c6a70 0x55825b7d5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:29.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.199+0000 7f1386ffd700 1 -- 172.21.15.77:0/3365659570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55825b7c6a70 msgr2=0x55825b7d5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:29.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.199+0000 7f1386ffd700 1 --2- 172.21.15.77:0/3365659570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55825b7c6a70 0x55825b7d5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:29.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.199+0000 7f1386ffd700 1 -- 172.21.15.77:0/3365659570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55825b7c60f0 msgr2=0x55825b7cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:29.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.199+0000 7f1386ffd700 1 --2- 172.21.15.77:0/3365659570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55825b7c60f0 0x55825b7cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:29.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.199+0000 7f1386ffd700 1 -- 172.21.15.77:0/3365659570 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1380011230 con 0x55825b7c53f0 2023-12-01T01:20:29.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.199+0000 7f1386ffd700 1 --2- 172.21.15.77:0/3365659570 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55825b7c53f0 0x55825b7ce670 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f1378097120 tx=0x7f1378096a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97ebeb5b23a44254 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:20:29.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.199+0000 7f136aff5700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:20:29.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.199+0000 7f136aff5700 1 -- 172.21.15.77:0/3365659570 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13780b6070 con 0x55825b7c53f0 2023-12-01T01:20:29.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.199+0000 7f136aff5700 1 -- 172.21.15.77:0/3365659570 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f13780b1070 con 0x55825b7c53f0 2023-12-01T01:20:29.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.199+0000 7f139962cb80 1 -- 172.21.15.77:0/3365659570 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55825b740d10 con 0x55825b7c53f0 2023-12-01T01:20:29.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.199+0000 7f136aff5700 1 -- 172.21.15.77:0/3365659570 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13780b0030 con 0x55825b7c53f0 2023-12-01T01:20:29.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.199+0000 7f139962cb80 10 client.16011 fetch_fsmap learned FSMap version 31 2023-12-01T01:20:29.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.199+0000 7f139962cb80 1 -- 172.21.15.77:0/3365659570 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f138819f100 con 0x55825b7c53f0 2023-12-01T01:20:29.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.199+0000 7f136aff5700 1 -- 172.21.15.77:0/3365659570 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55825b740d10 con 0x55825b7c53f0 2023-12-01T01:20:29.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.200+0000 7f136aff5700 1 -- 172.21.15.77:0/3365659570 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f13780a2200 con 0x55825b7c53f0 2023-12-01T01:20:29.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.200+0000 7f139962cb80 10 client.16011 fetch_fsmap finished waiting for FSMap version 31 2023-12-01T01:20:29.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.200+0000 7f139962cb80 10 client.16011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-01T01:20:29.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.200+0000 7f139962cb80 20 client.16011 populate_metadata read hostname 'smithi077' 2023-12-01T01:20:29.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.200+0000 7f139962cb80 1 --2- 172.21.15.77:0/3365659570 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55825b7d9ef0 0x55825b7dc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:29.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.200+0000 7f139962cb80 4 client.16011 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:20:29.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.200+0000 7f139962cb80 1 -- 172.21.15.77:0/3365659570 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55825b793e10 con 0x55825b7d9ef0 2023-12-01T01:20:29.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.200+0000 7f13867fc700 1 --2- 172.21.15.77:0/3365659570 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55825b7d9ef0 0x55825b7dc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:29.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.200+0000 7f136aff5700 10 client.16011 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:20:29.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.200+0000 7f13867fc700 1 --2- 172.21.15.77:0/3365659570 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55825b7d9ef0 0x55825b7dc310 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6ca4fa4dd3f3d62 server_cookie=677a77301465b643 in_seq=0 out_seq=0 2023-12-01T01:20:29.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.201+0000 7f136aff5700 1 -- 172.21.15.77:0/3365659570 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55825b793e10 con 0x55825b7d9ef0 2023-12-01T01:20:29.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.201+0000 7f136aff5700 10 client.16011 handle_command_reply: tid=0 2023-12-01T01:20:29.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.201+0000 7f139962cb80 1 client.16011 shutdown 2023-12-01T01:20:29.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.201+0000 7f139962cb80 1 -- 172.21.15.77:0/3365659570 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55825b7c53f0 msgr2=0x55825b7ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:29.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.201+0000 7f139962cb80 1 --2- 172.21.15.77:0/3365659570 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55825b7c53f0 0x55825b7ce670 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f1378097120 tx=0x7f1378096a80 comp rx=0 tx=0).stop 2023-12-01T01:20:29.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.201+0000 7f139962cb80 1 -- 172.21.15.77:0/3365659570 shutdown_connections 2023-12-01T01:20:29.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.201+0000 7f139962cb80 1 --2- 172.21.15.77:0/3365659570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55825b7c6a70 0x55825b7d5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:29.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.201+0000 7f139962cb80 1 --2- 172.21.15.77:0/3365659570 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55825b7c53f0 0x55825b7ce670 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:29.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.201+0000 7f139962cb80 1 --2- 172.21.15.77:0/3365659570 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55825b7d9ef0 0x55825b7dc310 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:29.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.201+0000 7f139962cb80 1 --2- 172.21.15.77:0/3365659570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55825b7c60f0 0x55825b7cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:29.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.201+0000 7f136aff5700 0 client.16011 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:20:29.210 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.201+0000 7f139962cb80 1 -- 172.21.15.77:0/3365659570 >> 172.21.15.77:0/3365659570 conn(0x55825b7240d0 msgr2=0x55825b7bf0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:20:29.210 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.201+0000 7f139962cb80 1 -- 172.21.15.77:0/3365659570 shutdown_connections 2023-12-01T01:20:29.210 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.201+0000 7f139962cb80 1 -- 172.21.15.77:0/3365659570 wait complete. 2023-12-01T01:20:29.210 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.201+0000 7f139962cb80 20 client.16011 trim_cache size 0 max 16384 2023-12-01T01:20:29.229 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.220+0000 7f139962cb80 1 Processor -- start 2023-12-01T01:20:29.229 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.220+0000 7f139962cb80 1 -- start start 2023-12-01T01:20:29.229 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.220+0000 7f139962cb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55825b7c53f0 0x55825b7ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:29.229 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.220+0000 7f139962cb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55825b7c60f0 0x55825b7d5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:29.229 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.220+0000 7f139962cb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55825b7c6a70 0x55825b7ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:29.229 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.220+0000 7f139962cb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55825b7cdb20 con 0x55825b7c53f0 2023-12-01T01:20:29.229 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.220+0000 7f139962cb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55825b7c0240 con 0x55825b7c6a70 2023-12-01T01:20:29.230 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.220+0000 7f139962cb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55825b7dc970 con 0x55825b7c60f0 2023-12-01T01:20:29.230 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.220+0000 7f1386ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55825b7c53f0 0x55825b7ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:29.230 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.221+0000 7f13867fc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55825b7c60f0 0x55825b7d5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:29.230 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.221+0000 7f13877fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55825b7c6a70 0x55825b7ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:29.230 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.221+0000 7f1386ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55825b7c53f0 0x55825b7ccc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60620/0 (socket says 172.21.15.77:60620) 2023-12-01T01:20:29.230 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.221+0000 7f13867fc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55825b7c60f0 0x55825b7d5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:46806/0 (socket says 172.21.15.77:46806) 2023-12-01T01:20:29.230 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.221+0000 7f13867fc700 1 -- 172.21.15.77:0/3268107239 learned_addr learned my addr 172.21.15.77:0/3268107239 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:20:29.230 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.221+0000 7f1386ffd700 1 -- 172.21.15.77:0/3268107239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55825b7c60f0 msgr2=0x55825b7d5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:29.230 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.221+0000 7f1386ffd700 1 --2- 172.21.15.77:0/3268107239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55825b7c60f0 0x55825b7d5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:29.230 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.221+0000 7f1386ffd700 1 -- 172.21.15.77:0/3268107239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55825b7c6a70 msgr2=0x55825b7ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:29.230 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.221+0000 7f1386ffd700 1 --2- 172.21.15.77:0/3268107239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55825b7c6a70 0x55825b7ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:29.230 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.221+0000 7f1386ffd700 1 -- 172.21.15.77:0/3268107239 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55825b52cc60 con 0x55825b7c53f0 2023-12-01T01:20:29.230 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.221+0000 7f1386ffd700 1 --2- 172.21.15.77:0/3268107239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55825b7c53f0 0x55825b7ccc90 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f13780098e0 tx=0x7f137809a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a47739508ccd78a9 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:20:29.231 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.221+0000 7f136dffb700 1 -- 172.21.15.77:0/3268107239 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13780b6070 con 0x55825b7c53f0 2023-12-01T01:20:29.231 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.221+0000 7f136dffb700 1 -- 172.21.15.77:0/3268107239 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13780a2690 con 0x55825b7c53f0 2023-12-01T01:20:29.231 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.221+0000 7f136dffb700 1 -- 172.21.15.77:0/3268107239 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13780853b0 con 0x55825b7c53f0 2023-12-01T01:20:29.231 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.222+0000 7f139962cb80 1 -- 172.21.15.77:0/3268107239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55825b7c53f0 msgr2=0x55825b7ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:29.231 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.222+0000 7f139962cb80 1 --2- 172.21.15.77:0/3268107239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55825b7c53f0 0x55825b7ccc90 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f13780098e0 tx=0x7f137809a270 comp rx=0 tx=0).stop 2023-12-01T01:20:29.231 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.222+0000 7f139962cb80 1 -- 172.21.15.77:0/3268107239 shutdown_connections 2023-12-01T01:20:29.231 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.222+0000 7f139962cb80 1 --2- 172.21.15.77:0/3268107239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55825b7c6a70 0x55825b7ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:29.231 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.222+0000 7f139962cb80 1 --2- 172.21.15.77:0/3268107239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55825b7c53f0 0x55825b7ccc90 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:29.231 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.222+0000 7f139962cb80 1 --2- 172.21.15.77:0/3268107239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55825b7c60f0 0x55825b7d5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:29.231 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.222+0000 7f139962cb80 1 -- 172.21.15.77:0/3268107239 >> 172.21.15.77:0/3268107239 conn(0x55825b7240d0 msgr2=0x55825b7cc580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:20:29.231 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.222+0000 7f139962cb80 1 -- 172.21.15.77:0/3268107239 shutdown_connections 2023-12-01T01:20:29.231 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.222+0000 7f139962cb80 1 -- 172.21.15.77:0/3268107239 wait complete. 2023-12-01T01:20:29.231 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.222+0000 7f139962cb80 1 -- start start 2023-12-01T01:20:29.231 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.222+0000 7f139962cb80 1 Processor -- start 2023-12-01T01:20:29.231 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.223+0000 7f139962cb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55825b7c53f0 0x55825b7d4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:29.232 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.223+0000 7f139962cb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55825b7c60f0 0x55825b7d4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:29.232 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.223+0000 7f139962cb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55825b7c6a70 0x55825b7d4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:29.232 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.223+0000 7f139962cb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55825b7d5f90 con 0x55825b7c53f0 2023-12-01T01:20:29.232 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.223+0000 7f139962cb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55825b82b9f0 con 0x55825b7c6a70 2023-12-01T01:20:29.232 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.223+0000 7f139962cb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55825b82bbb0 con 0x55825b7c60f0 2023-12-01T01:20:29.232 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.223+0000 7f139962cb80 10 client.0 _ll_register_callbacks cb 0x7f1380009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:20:29.232 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.223+0000 7f1386ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55825b7c53f0 0x55825b7d4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:29.232 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.223+0000 7f1386ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55825b7c53f0 0x55825b7d4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60624/0 (socket says 172.21.15.77:60624) 2023-12-01T01:20:29.232 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.223+0000 7f1386ffd700 1 -- 172.21.15.77:0/2204615224 learned_addr learned my addr 172.21.15.77:0/2204615224 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:20:29.232 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.223+0000 7f13867fc700 1 --2- 172.21.15.77:0/2204615224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55825b7c60f0 0x55825b7d4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:29.232 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.223+0000 7f13877fe700 1 --2- 172.21.15.77:0/2204615224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55825b7c6a70 0x55825b7d4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:29.232 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.223+0000 7f1386ffd700 1 -- 172.21.15.77:0/2204615224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55825b7c60f0 msgr2=0x55825b7d4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:29.232 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.223+0000 7f1386ffd700 1 --2- 172.21.15.77:0/2204615224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55825b7c60f0 0x55825b7d4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:29.232 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.223+0000 7f1386ffd700 1 -- 172.21.15.77:0/2204615224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55825b7c6a70 msgr2=0x55825b7d4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:29.233 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.223+0000 7f1386ffd700 1 --2- 172.21.15.77:0/2204615224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55825b7c6a70 0x55825b7d4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:29.233 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.223+0000 7f1386ffd700 1 -- 172.21.15.77:0/2204615224 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f138819f100 con 0x55825b7c53f0 2023-12-01T01:20:29.233 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.223+0000 7f1386ffd700 1 --2- 172.21.15.77:0/2204615224 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55825b7c53f0 0x55825b7d4310 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f1378005b00 tx=0x7f1378021c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df4a23f97be695a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:20:29.233 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.223+0000 7f136aff5700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:20:29.233 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.223+0000 7f136aff5700 1 -- 172.21.15.77:0/2204615224 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13780b0060 con 0x55825b7c53f0 2023-12-01T01:20:29.233 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.224+0000 7f136aff5700 1 -- 172.21.15.77:0/2204615224 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f137809e4e0 con 0x55825b7c53f0 2023-12-01T01:20:29.233 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.224+0000 7f139962cb80 1 -- 172.21.15.77:0/2204615224 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f137809a6b0 con 0x55825b7c53f0 2023-12-01T01:20:29.233 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.224+0000 7f136aff5700 1 -- 172.21.15.77:0/2204615224 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13780b6070 con 0x55825b7c53f0 2023-12-01T01:20:29.233 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.224+0000 7f139962cb80 10 client.16017 fetch_fsmap learned FSMap version 31 2023-12-01T01:20:29.233 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.224+0000 7f139962cb80 1 -- 172.21.15.77:0/2204615224 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55825b56a250 con 0x55825b7c53f0 2023-12-01T01:20:29.233 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.224+0000 7f136aff5700 1 -- 172.21.15.77:0/2204615224 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f137809a6b0 con 0x55825b7c53f0 2023-12-01T01:20:29.233 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.224+0000 7f136aff5700 1 -- 172.21.15.77:0/2204615224 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f13780661a0 con 0x55825b7c53f0 2023-12-01T01:20:29.233 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.224+0000 7f139962cb80 10 client.16017 fetch_fsmap finished waiting for FSMap version 31 2023-12-01T01:20:29.233 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.224+0000 7f139962cb80 10 client.16017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-01T01:20:29.233 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.224+0000 7f139962cb80 20 client.16017 populate_metadata read hostname 'smithi077' 2023-12-01T01:20:29.234 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.224+0000 7f139962cb80 1 --2- 172.21.15.77:0/2204615224 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55825b82cf20 0x55825b82f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:29.234 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.224+0000 7f139962cb80 4 client.16017 mds_command: new command op to 24500 tid=0[{"prefix": "scrub status"}] 2023-12-01T01:20:29.234 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.224+0000 7f139962cb80 1 -- 172.21.15.77:0/2204615224 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55825b70ed40 con 0x55825b82cf20 2023-12-01T01:20:29.234 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.224+0000 7f13867fc700 1 --2- 172.21.15.77:0/2204615224 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55825b82cf20 0x55825b82f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:29.234 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.225+0000 7f136aff5700 10 client.16017 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:20:29.234 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.225+0000 7f13867fc700 1 --2- 172.21.15.77:0/2204615224 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55825b82cf20 0x55825b82f310 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b286e5da229fc354 server_cookie=322ece05f4f55682 in_seq=0 out_seq=0 2023-12-01T01:20:29.234 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.225+0000 7f136aff5700 1 -- 172.21.15.77:0/2204615224 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55825b70ed40 con 0x55825b82cf20 2023-12-01T01:20:29.234 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.225+0000 7f136aff5700 10 client.16017 handle_command_reply: tid=0 2023-12-01T01:20:29.234 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.225+0000 7f139962cb80 1 client.16017 shutdown 2023-12-01T01:20:29.234 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.225+0000 7f139962cb80 1 -- 172.21.15.77:0/2204615224 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55825b7c53f0 msgr2=0x55825b7d4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:29.234 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.225+0000 7f139962cb80 1 --2- 172.21.15.77:0/2204615224 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55825b7c53f0 0x55825b7d4310 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f1378005b00 tx=0x7f1378021c00 comp rx=0 tx=0).stop 2023-12-01T01:20:29.234 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.225+0000 7f139962cb80 1 -- 172.21.15.77:0/2204615224 shutdown_connections 2023-12-01T01:20:29.234 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.225+0000 7f139962cb80 1 --2- 172.21.15.77:0/2204615224 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55825b82cf20 0x55825b82f310 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:29.234 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.225+0000 7f139962cb80 1 --2- 172.21.15.77:0/2204615224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55825b7c6a70 0x55825b7d4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:29.235 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.225+0000 7f139962cb80 1 --2- 172.21.15.77:0/2204615224 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55825b7c53f0 0x55825b7d4310 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:29.235 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.225+0000 7f139962cb80 1 --2- 172.21.15.77:0/2204615224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55825b7c60f0 0x55825b7d4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:29.235 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.225+0000 7f136aff5700 0 client.16017 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:20:29.235 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.225+0000 7f139962cb80 1 -- 172.21.15.77:0/2204615224 >> 172.21.15.77:0/2204615224 conn(0x55825b7240d0 msgr2=0x55825b7ce420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:20:29.235 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.225+0000 7f139962cb80 1 -- 172.21.15.77:0/2204615224 shutdown_connections 2023-12-01T01:20:29.235 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.225+0000 7f139962cb80 1 -- 172.21.15.77:0/2204615224 wait complete. 2023-12-01T01:20:29.235 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.226+0000 7f139962cb80 20 client.16017 trim_cache size 0 max 16384 2023-12-01T01:20:29.235 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2023-12-01T01:20:29.235 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2023-12-01T01:20:29.235 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2023-12-01T01:20:29.236 INFO:teuthology.orchestra.run.smithi077.stdout: "085290de-33b6-4527-ab7f-03f7dfd5dbba": { 2023-12-01T01:20:29.236 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2023-12-01T01:20:29.236 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "085290de-33b6-4527-ab7f-03f7dfd5dbba", 2023-12-01T01:20:29.236 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2023-12-01T01:20:29.236 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:20:29.236 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:20:29.236 INFO:teuthology.orchestra.run.smithi077.stdout:} 2023-12-01T01:20:29.237 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.226+0000 7f138d49c700 1 -- 172.21.15.77:0/3565849094 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f13700a0520 msgr2=0x7f13700a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:29.237 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.226+0000 7f138d49c700 1 --2- 172.21.15.77:0/3565849094 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f13700a0520 0x7f13700a29e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f13880ff600 tx=0x7f137c009260 comp rx=0 tx=0).stop 2023-12-01T01:20:29.237 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.226+0000 7f138d49c700 1 -- 172.21.15.77:0/3565849094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13881001b0 msgr2=0x7f13880ff890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:29.237 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.226+0000 7f138d49c700 1 --2- 172.21.15.77:0/3565849094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13881001b0 0x7f13880ff890 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1378004730 tx=0x7f1378004760 comp rx=0 tx=0).stop 2023-12-01T01:20:29.237 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.226+0000 7f138d49c700 1 -- 172.21.15.77:0/3565849094 shutdown_connections 2023-12-01T01:20:29.237 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.226+0000 7f138d49c700 1 --2- 172.21.15.77:0/3565849094 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f13700a0520 0x7f13700a29e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:29.237 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.226+0000 7f138d49c700 1 --2- 172.21.15.77:0/3565849094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1388101880 0x7f138819df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:29.237 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.226+0000 7f138d49c700 1 --2- 172.21.15.77:0/3565849094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13881001b0 0x7f13880ff890 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:29.237 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.226+0000 7f138d49c700 1 --2- 172.21.15.77:0/3565849094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1388100ec0 0x7f138819b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:29.237 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.226+0000 7f138d49c700 1 -- 172.21.15.77:0/3565849094 >> 172.21.15.77:0/3565849094 conn(0x7f13880fa960 msgr2=0x7f13881044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:20:29.237 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.227+0000 7f138d49c700 1 -- 172.21.15.77:0/3565849094 shutdown_connections 2023-12-01T01:20:29.237 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:29.227+0000 7f138d49c700 1 -- 172.21.15.77:0/3565849094 wait complete. 2023-12-01T01:20:29.241 INFO:tasks.cephfs.filesystem:scrub status for tag:085290de-33b6-4527-ab7f-03f7dfd5dbba - {'path': '/', 'tag': '085290de-33b6-4527-ab7f-03f7dfd5dbba', 'options': 'recursive,force'} 2023-12-01T01:20:29.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:29 smithi077 ceph-mon[98867]: pgmap v376: 97 pgs: 97 active+clean; 54 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.7 MiB/s wr, 669 op/s 2023-12-01T01:20:30.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:20:29 smithi203 ceph-mon[103581]: pgmap v376: 97 pgs: 97 active+clean; 54 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.7 MiB/s wr, 669 op/s 2023-12-01T01:20:30.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:20:29 smithi137 ceph-mon[103312]: pgmap v376: 97 pgs: 97 active+clean; 54 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.7 MiB/s wr, 669 op/s 2023-12-01T01:20:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.446+0000 7fdcb5fac700 1 -- 172.21.15.77:0/1679099000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdcb0101f90 msgr2=0x7fdcb01023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.446+0000 7fdcb5fac700 1 --2- 172.21.15.77:0/1679099000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdcb0101f90 0x7fdcb01023f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fdca40099c0 tx=0x7fdca4009cd0 comp rx=0 tx=0).stop 2023-12-01T01:20:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.446+0000 7fdcb5fac700 1 -- 172.21.15.77:0/1679099000 shutdown_connections 2023-12-01T01:20:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.446+0000 7fdcb5fac700 1 --2- 172.21.15.77:0/1679099000 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdcb0102930 0x7fdcb01091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.446+0000 7fdcb5fac700 1 --2- 172.21.15.77:0/1679099000 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdcb0100da0 0x7fdcb01011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.446+0000 7fdcb5fac700 1 --2- 172.21.15.77:0/1679099000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdcb0101f90 0x7fdcb01023f0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.446+0000 7fdcb5fac700 1 -- 172.21.15.77:0/1679099000 >> 172.21.15.77:0/1679099000 conn(0x7fdcb00fc2d0 msgr2=0x7fdcb00fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:20:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.446+0000 7fdcb5fac700 1 -- 172.21.15.77:0/1679099000 shutdown_connections 2023-12-01T01:20:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.446+0000 7fdcb5fac700 1 -- 172.21.15.77:0/1679099000 wait complete. 2023-12-01T01:20:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.447+0000 7fdcb5fac700 1 Processor -- start 2023-12-01T01:20:30.449 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.447+0000 7fdcb5fac700 1 -- start start 2023-12-01T01:20:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.447+0000 7fdcb5fac700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdcb0100da0 0x7fdcb0073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.447+0000 7fdcb5fac700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdcb0101f90 0x7fdcb00736c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.447+0000 7fdcb5fac700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdcb0102930 0x7fdcb0073c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.447+0000 7fdcb5fac700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb01a0b50 con 0x7fdcb0101f90 2023-12-01T01:20:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.447+0000 7fdcb5fac700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb01a0c90 con 0x7fdcb0100da0 2023-12-01T01:20:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.447+0000 7fdcb5fac700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb01a0dd0 con 0x7fdcb0102930 2023-12-01T01:20:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.448+0000 7fdcaeffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdcb0101f90 0x7fdcb00736c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.448+0000 7fdcaeffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdcb0101f90 0x7fdcb00736c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60640/0 (socket says 172.21.15.77:60640) 2023-12-01T01:20:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.448+0000 7fdcaf7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdcb0100da0 0x7fdcb0073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.448+0000 7fdcaeffd700 1 -- 172.21.15.77:0/818507923 learned_addr learned my addr 172.21.15.77:0/818507923 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:20:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.448+0000 7fdcaffff700 1 --2- 172.21.15.77:0/818507923 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdcb0102930 0x7fdcb0073c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.448+0000 7fdcaeffd700 1 -- 172.21.15.77:0/818507923 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdcb0102930 msgr2=0x7fdcb0073c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.448+0000 7fdcaeffd700 1 --2- 172.21.15.77:0/818507923 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdcb0102930 0x7fdcb0073c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.448+0000 7fdcaeffd700 1 -- 172.21.15.77:0/818507923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdcb0100da0 msgr2=0x7fdcb0073180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.448+0000 7fdcaeffd700 1 --2- 172.21.15.77:0/818507923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdcb0100da0 0x7fdcb0073180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.448+0000 7fdcaeffd700 1 -- 172.21.15.77:0/818507923 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca40095f0 con 0x7fdcb0101f90 2023-12-01T01:20:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.448+0000 7fdcaeffd700 1 --2- 172.21.15.77:0/818507923 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdcb0101f90 0x7fdcb00736c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fdca4009990 tx=0x7fdca400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:20:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.448+0000 7fdcacff9700 1 -- 172.21.15.77:0/818507923 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca401d070 con 0x7fdcb0101f90 2023-12-01T01:20:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.448+0000 7fdcacff9700 1 -- 172.21.15.77:0/818507923 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdca4005030 con 0x7fdcb0101f90 2023-12-01T01:20:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.448+0000 7fdcb5fac700 1 -- 172.21.15.77:0/818507923 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcb01a1000 con 0x7fdcb0101f90 2023-12-01T01:20:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.448+0000 7fdcacff9700 1 -- 172.21.15.77:0/818507923 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca4004a70 con 0x7fdcb0101f90 2023-12-01T01:20:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.448+0000 7fdcb5fac700 1 -- 172.21.15.77:0/818507923 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcb01a1420 con 0x7fdcb0101f90 2023-12-01T01:20:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.451+0000 7fdcacff9700 1 -- 172.21.15.77:0/818507923 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fdca4005410 con 0x7fdcb0101f90 2023-12-01T01:20:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.452+0000 7fdcacff9700 1 --2- 172.21.15.77:0/818507923 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fdc980a0520 0x7fdc980a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.452+0000 7fdcaf7fe700 1 --2- 172.21.15.77:0/818507923 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fdc980a0520 0x7fdc980a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.452+0000 7fdcacff9700 1 -- 172.21.15.77:0/818507923 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fdca40cc690 con 0x7fdcb0101f90 2023-12-01T01:20:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.452+0000 7fdcaf7fe700 1 --2- 172.21.15.77:0/818507923 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fdc980a0520 0x7fdc980a29e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0074a90 tx=0x7fdca0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:20:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.452+0000 7fdcc213cb80 1 Processor -- start 2023-12-01T01:20:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.452+0000 7fdcc213cb80 1 -- start start 2023-12-01T01:20:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.453+0000 7fdcc213cb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560881c1c680 0x560881c1caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.453+0000 7fdcc213cb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560881c1cfe0 0x560881c1d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.453+0000 7fdcc213cb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560881c1ded0 0x560881c1e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.453+0000 7fdcc213cb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x560881b749b0 con 0x560881c1cfe0 2023-12-01T01:20:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.453+0000 7fdcc213cb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x560881b72370 con 0x560881c1c680 2023-12-01T01:20:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.453+0000 7fdcc213cb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x560881b6dc60 con 0x560881c1ded0 2023-12-01T01:20:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.453+0000 7fdcaf7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560881c1c680 0x560881c1caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.453+0000 7fdcaeffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560881c1cfe0 0x560881c1d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.453+0000 7fdcaf7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560881c1c680 0x560881c1caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:39568/0 (socket says 172.21.15.77:39568) 2023-12-01T01:20:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.453+0000 7fdcaf7fe700 1 -- 172.21.15.77:0/4078097325 learned_addr learned my addr 172.21.15.77:0/4078097325 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:20:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.453+0000 7fdcaffff700 1 --2- 172.21.15.77:0/4078097325 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560881c1ded0 0x560881c1e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.453+0000 7fdcaeffd700 1 -- 172.21.15.77:0/4078097325 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560881c1ded0 msgr2=0x560881c1e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.453+0000 7fdcaeffd700 1 --2- 172.21.15.77:0/4078097325 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560881c1ded0 0x560881c1e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.453+0000 7fdcaeffd700 1 -- 172.21.15.77:0/4078097325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560881c1c680 msgr2=0x560881c1caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.453+0000 7fdcaeffd700 1 --2- 172.21.15.77:0/4078097325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560881c1c680 0x560881c1caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.453+0000 7fdcaeffd700 1 -- 172.21.15.77:0/4078097325 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56088199bd80 con 0x560881c1cfe0 2023-12-01T01:20:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.453+0000 7fdcaf7fe700 1 --2- 172.21.15.77:0/4078097325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560881c1c680 0x560881c1caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:20:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.453+0000 7fdcaeffd700 1 --2- 172.21.15.77:0/4078097325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560881c1cfe0 0x560881c1d8f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fdca40ca280 tx=0x7fdca4099a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e9db35eac8b803 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:20:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.453+0000 7fdc95ffb700 1 -- 172.21.15.77:0/4078097325 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca409de50 con 0x560881c1cfe0 2023-12-01T01:20:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.453+0000 7fdc95ffb700 1 -- 172.21.15.77:0/4078097325 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdca40a2000 con 0x560881c1cfe0 2023-12-01T01:20:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.453+0000 7fdc95ffb700 1 -- 172.21.15.77:0/4078097325 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca4099510 con 0x560881c1cfe0 2023-12-01T01:20:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.454+0000 7fdcc213cb80 1 -- 172.21.15.77:0/4078097325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560881c1cfe0 msgr2=0x560881c1d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.454+0000 7fdcc213cb80 1 --2- 172.21.15.77:0/4078097325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560881c1cfe0 0x560881c1d8f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fdca40ca280 tx=0x7fdca4099a70 comp rx=0 tx=0).stop 2023-12-01T01:20:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.454+0000 7fdcc213cb80 1 -- 172.21.15.77:0/4078097325 shutdown_connections 2023-12-01T01:20:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.454+0000 7fdcc213cb80 1 --2- 172.21.15.77:0/4078097325 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560881c1ded0 0x560881c1e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.454+0000 7fdcc213cb80 1 --2- 172.21.15.77:0/4078097325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560881c1c680 0x560881c1caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.454+0000 7fdcc213cb80 1 --2- 172.21.15.77:0/4078097325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560881c1cfe0 0x560881c1d8f0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.454+0000 7fdcc213cb80 1 -- 172.21.15.77:0/4078097325 >> 172.21.15.77:0/4078097325 conn(0x560881b7c070 msgr2=0x560881c19040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:20:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.454+0000 7fdcc213cb80 1 -- 172.21.15.77:0/4078097325 shutdown_connections 2023-12-01T01:20:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.454+0000 7fdcc213cb80 1 -- 172.21.15.77:0/4078097325 wait complete. 2023-12-01T01:20:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.455+0000 7fdcc213cb80 1 -- start start 2023-12-01T01:20:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.455+0000 7fdcc213cb80 1 Processor -- start 2023-12-01T01:20:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.455+0000 7fdcc213cb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560881c1c680 0x560881c26110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.455+0000 7fdcc213cb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560881c1cfe0 0x560881c26650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.455+0000 7fdcc213cb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560881c1ded0 0x560881c2d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.455+0000 7fdcc213cb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x560881b0a4b0 con 0x560881c1cfe0 2023-12-01T01:20:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.455+0000 7fdcc213cb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x560881b0a150 con 0x560881c1ded0 2023-12-01T01:20:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.455+0000 7fdcc213cb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608819f18c0 con 0x560881c1c680 2023-12-01T01:20:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.455+0000 7fdcc213cb80 10 client.0 _ll_register_callbacks cb 0x560881a57a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:20:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.456+0000 7fdcaf7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560881c1c680 0x560881c26110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.456+0000 7fdcaf7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560881c1c680 0x560881c26110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:46840/0 (socket says 172.21.15.77:46840) 2023-12-01T01:20:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.456+0000 7fdcaf7fe700 1 -- 172.21.15.77:0/4150038289 learned_addr learned my addr 172.21.15.77:0/4150038289 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:20:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.456+0000 7fdcaeffd700 1 --2- 172.21.15.77:0/4150038289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560881c1cfe0 0x560881c26650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.456+0000 7fdcaffff700 1 --2- 172.21.15.77:0/4150038289 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560881c1ded0 0x560881c2d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.456+0000 7fdcaf7fe700 1 -- 172.21.15.77:0/4150038289 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560881c1ded0 msgr2=0x560881c2d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:30.463 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.456+0000 7fdcaf7fe700 1 --2- 172.21.15.77:0/4150038289 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560881c1ded0 0x560881c2d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:30.463 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.456+0000 7fdcaf7fe700 1 -- 172.21.15.77:0/4150038289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560881c1cfe0 msgr2=0x560881c26650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:30.463 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.456+0000 7fdcaf7fe700 1 --2- 172.21.15.77:0/4150038289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560881c1cfe0 0x560881c26650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:30.463 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.456+0000 7fdcaf7fe700 1 -- 172.21.15.77:0/4150038289 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdca0017070 con 0x560881c1c680 2023-12-01T01:20:30.463 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.456+0000 7fdcaeffd700 1 --2- 172.21.15.77:0/4150038289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560881c1cfe0 0x560881c26650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:20:30.463 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.456+0000 7fdcaf7fe700 1 --2- 172.21.15.77:0/4150038289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560881c1c680 0x560881c26110 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdca001d7f0 tx=0x7fdca00047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1be8bcfca9fc766a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:20:30.463 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.456+0000 7fdc8effd700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2023-12-01T01:20:30.463 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.456+0000 7fdc8effd700 1 -- 172.21.15.77:0/4150038289 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca0022070 con 0x560881c1c680 2023-12-01T01:20:30.463 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.456+0000 7fdc8effd700 1 -- 172.21.15.77:0/4150038289 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fdca0004240 con 0x560881c1c680 2023-12-01T01:20:30.463 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.456+0000 7fdcc213cb80 1 -- 172.21.15.77:0/4150038289 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560881b98cb0 con 0x560881c1c680 2023-12-01T01:20:30.463 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.457+0000 7fdc8effd700 1 -- 172.21.15.77:0/4150038289 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdca0004400 con 0x560881c1c680 2023-12-01T01:20:30.463 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.457+0000 7fdcc213cb80 10 client.25624 fetch_fsmap learned FSMap version 31 2023-12-01T01:20:30.463 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.457+0000 7fdcc213cb80 1 -- 172.21.15.77:0/4150038289 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fdcb01a1420 con 0x560881c1c680 2023-12-01T01:20:30.463 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.457+0000 7fdc8effd700 1 -- 172.21.15.77:0/4150038289 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560881b98cb0 con 0x560881c1c680 2023-12-01T01:20:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.457+0000 7fdc8effd700 1 -- 172.21.15.77:0/4150038289 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdca001c7a0 con 0x560881c1c680 2023-12-01T01:20:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.457+0000 7fdcc213cb80 10 client.25624 fetch_fsmap finished waiting for FSMap version 31 2023-12-01T01:20:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.457+0000 7fdcc213cb80 10 client.25624 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-01T01:20:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.457+0000 7fdcc213cb80 20 client.25624 populate_metadata read hostname 'smithi077' 2023-12-01T01:20:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.457+0000 7fdcc213cb80 1 --2- 172.21.15.77:0/4150038289 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x560881c32050 0x560881c34470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.457+0000 7fdcc213cb80 4 client.25624 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:20:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.457+0000 7fdcc213cb80 1 -- 172.21.15.77:0/4150038289 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560881bebdb0 con 0x560881c32050 2023-12-01T01:20:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.457+0000 7fdcaeffd700 1 --2- 172.21.15.77:0/4150038289 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x560881c32050 0x560881c34470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.458+0000 7fdcaeffd700 1 --2- 172.21.15.77:0/4150038289 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x560881c32050 0x560881c34470 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab4ac18144a3af2e server_cookie=90575226684ce61 in_seq=0 out_seq=0 2023-12-01T01:20:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.458+0000 7fdc8effd700 10 client.25624 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:20:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.458+0000 7fdc8effd700 1 -- 172.21.15.77:0/4150038289 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x560881bebdb0 con 0x560881c32050 2023-12-01T01:20:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.458+0000 7fdc8effd700 10 client.25624 handle_command_reply: tid=0 2023-12-01T01:20:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.458+0000 7fdcc213cb80 1 client.25624 shutdown 2023-12-01T01:20:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.459+0000 7fdcc213cb80 1 -- 172.21.15.77:0/4150038289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560881c1c680 msgr2=0x560881c26110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.459+0000 7fdcc213cb80 1 --2- 172.21.15.77:0/4150038289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560881c1c680 0x560881c26110 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdca001d7f0 tx=0x7fdca00047c0 comp rx=0 tx=0).stop 2023-12-01T01:20:30.465 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.459+0000 7fdcc213cb80 1 -- 172.21.15.77:0/4150038289 shutdown_connections 2023-12-01T01:20:30.465 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.459+0000 7fdcc213cb80 1 --2- 172.21.15.77:0/4150038289 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x560881c32050 0x560881c34470 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:30.465 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.459+0000 7fdcc213cb80 1 --2- 172.21.15.77:0/4150038289 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560881c1ded0 0x560881c2d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:30.465 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.459+0000 7fdcc213cb80 1 --2- 172.21.15.77:0/4150038289 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560881c1c680 0x560881c26110 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:30.465 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.459+0000 7fdcc213cb80 1 --2- 172.21.15.77:0/4150038289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560881c1cfe0 0x560881c26650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:30.465 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.459+0000 7fdcc213cb80 1 -- 172.21.15.77:0/4150038289 >> 172.21.15.77:0/4150038289 conn(0x560881b7c070 msgr2=0x560881c173c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:20:30.465 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.459+0000 7fdc8effd700 0 client.25624 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:20:30.465 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.459+0000 7fdcc213cb80 1 -- 172.21.15.77:0/4150038289 shutdown_connections 2023-12-01T01:20:30.465 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.459+0000 7fdcc213cb80 1 -- 172.21.15.77:0/4150038289 wait complete. 2023-12-01T01:20:30.465 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.459+0000 7fdcc213cb80 20 client.25624 trim_cache size 0 max 16384 2023-12-01T01:20:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.478+0000 7fdcc213cb80 1 Processor -- start 2023-12-01T01:20:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.478+0000 7fdcc213cb80 1 -- start start 2023-12-01T01:20:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.478+0000 7fdcc213cb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560881c1c680 0x560881c34060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.478+0000 7fdcc213cb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560881c1cfe0 0x560881c2d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.479+0000 7fdcc213cb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560881c1ded0 0x560881c24210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.479+0000 7fdcc213cb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x560881c0f7e0 con 0x560881c1cfe0 2023-12-01T01:20:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.479+0000 7fdcc213cb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x560881c31c10 con 0x560881c1ded0 2023-12-01T01:20:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.479+0000 7fdcc213cb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x560881c2f300 con 0x560881c1c680 2023-12-01T01:20:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.479+0000 7fdcaf7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560881c1c680 0x560881c34060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.479+0000 7fdcaeffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560881c1cfe0 0x560881c2d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.479+0000 7fdcaf7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560881c1c680 0x560881c34060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:46848/0 (socket says 172.21.15.77:46848) 2023-12-01T01:20:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.479+0000 7fdcaf7fe700 1 -- 172.21.15.77:0/1848939925 learned_addr learned my addr 172.21.15.77:0/1848939925 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:20:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.479+0000 7fdcaeffd700 1 --2- 172.21.15.77:0/1848939925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560881c1cfe0 0x560881c2d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60662/0 (socket says 172.21.15.77:60662) 2023-12-01T01:20:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.479+0000 7fdcaffff700 1 --2- 172.21.15.77:0/1848939925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560881c1ded0 0x560881c24210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.479+0000 7fdcaeffd700 1 -- 172.21.15.77:0/1848939925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560881c1c680 msgr2=0x560881c34060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.479+0000 7fdcaeffd700 1 --2- 172.21.15.77:0/1848939925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560881c1c680 0x560881c34060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.479+0000 7fdcaeffd700 1 -- 172.21.15.77:0/1848939925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560881c1ded0 msgr2=0x560881c24210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.479+0000 7fdcaeffd700 1 --2- 172.21.15.77:0/1848939925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560881c1ded0 0x560881c24210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.479+0000 7fdcaeffd700 1 -- 172.21.15.77:0/1848939925 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560881984c00 con 0x560881c1cfe0 2023-12-01T01:20:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.479+0000 7fdcaf7fe700 1 --2- 172.21.15.77:0/1848939925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560881c1c680 0x560881c34060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:20:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.479+0000 7fdcaffff700 1 --2- 172.21.15.77:0/1848939925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560881c1ded0 0x560881c24210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:20:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.479+0000 7fdcaeffd700 1 --2- 172.21.15.77:0/1848939925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560881c1cfe0 0x560881c2d650 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fdca4009d70 tx=0x7fdca4005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5d1f88000de1cb1 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:20:30.482 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.479+0000 7fdc967fc700 1 -- 172.21.15.77:0/1848939925 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca4099980 con 0x560881c1cfe0 2023-12-01T01:20:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.480+0000 7fdc967fc700 1 -- 172.21.15.77:0/1848939925 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdca40051a0 con 0x560881c1cfe0 2023-12-01T01:20:30.483 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.480+0000 7fdc967fc700 1 -- 172.21.15.77:0/1848939925 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca4004bd0 con 0x560881c1cfe0 2023-12-01T01:20:30.485 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.480+0000 7fdcc213cb80 1 -- 172.21.15.77:0/1848939925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560881c1cfe0 msgr2=0x560881c2d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.480+0000 7fdcc213cb80 1 --2- 172.21.15.77:0/1848939925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560881c1cfe0 0x560881c2d650 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fdca4009d70 tx=0x7fdca4005720 comp rx=0 tx=0).stop 2023-12-01T01:20:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.480+0000 7fdcc213cb80 1 -- 172.21.15.77:0/1848939925 shutdown_connections 2023-12-01T01:20:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.480+0000 7fdcc213cb80 1 --2- 172.21.15.77:0/1848939925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560881c1ded0 0x560881c24210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.480+0000 7fdcc213cb80 1 --2- 172.21.15.77:0/1848939925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560881c1c680 0x560881c34060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.480+0000 7fdcc213cb80 1 --2- 172.21.15.77:0/1848939925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560881c1cfe0 0x560881c2d650 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.480+0000 7fdcc213cb80 1 -- 172.21.15.77:0/1848939925 >> 172.21.15.77:0/1848939925 conn(0x560881b7c070 msgr2=0x560881c18c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:20:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.480+0000 7fdcc213cb80 1 -- 172.21.15.77:0/1848939925 shutdown_connections 2023-12-01T01:20:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.480+0000 7fdcc213cb80 1 -- 172.21.15.77:0/1848939925 wait complete. 2023-12-01T01:20:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.481+0000 7fdcc213cb80 1 -- start start 2023-12-01T01:20:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.481+0000 7fdcc213cb80 1 Processor -- start 2023-12-01T01:20:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.481+0000 7fdcc213cb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560881c1c680 0x560881c2aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.481+0000 7fdcc213cb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560881c1cfe0 0x560881c2b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.481+0000 7fdcc213cb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560881c1ded0 0x560881c2c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:30.486 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.481+0000 7fdcc213cb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x560881c2c940 con 0x560881c1c680 2023-12-01T01:20:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.481+0000 7fdcc213cb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x560881c2cab0 con 0x560881c1cfe0 2023-12-01T01:20:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.481+0000 7fdcc213cb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x560881c2cc70 con 0x560881c1ded0 2023-12-01T01:20:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.481+0000 7fdcc213cb80 10 client.0 _ll_register_callbacks cb 0x7fdca001d410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:20:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.481+0000 7fdcaf7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560881c1c680 0x560881c2aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.481+0000 7fdcaf7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560881c1c680 0x560881c2aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60666/0 (socket says 172.21.15.77:60666) 2023-12-01T01:20:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.481+0000 7fdcaf7fe700 1 -- 172.21.15.77:0/414104321 learned_addr learned my addr 172.21.15.77:0/414104321 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:20:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.482+0000 7fdcaffff700 1 --2- 172.21.15.77:0/414104321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560881c1ded0 0x560881c2c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.482+0000 7fdcaeffd700 1 --2- 172.21.15.77:0/414104321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560881c1cfe0 0x560881c2b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.482+0000 7fdcaf7fe700 1 -- 172.21.15.77:0/414104321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560881c1ded0 msgr2=0x560881c2c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.482+0000 7fdcaf7fe700 1 --2- 172.21.15.77:0/414104321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560881c1ded0 0x560881c2c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.482+0000 7fdcaf7fe700 1 -- 172.21.15.77:0/414104321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560881c1cfe0 msgr2=0x560881c2b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.482+0000 7fdcaf7fe700 1 --2- 172.21.15.77:0/414104321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560881c1cfe0 0x560881c2b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.482+0000 7fdcaf7fe700 1 -- 172.21.15.77:0/414104321 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcb01a1420 con 0x560881c1c680 2023-12-01T01:20:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.482+0000 7fdcaf7fe700 1 --2- 172.21.15.77:0/414104321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560881c1c680 0x560881c2aeb0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fdca001d7f0 tx=0x7fdca0009760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e931e5f0711a4a5b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:20:30.487 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.482+0000 7fdc8effd700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:20:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.482+0000 7fdc8effd700 1 -- 172.21.15.77:0/414104321 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca0022040 con 0x560881c1c680 2023-12-01T01:20:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.482+0000 7fdcc213cb80 1 -- 172.21.15.77:0/414104321 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdca40a2000 con 0x560881c1c680 2023-12-01T01:20:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.482+0000 7fdc8effd700 1 -- 172.21.15.77:0/414104321 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fdca001c620 con 0x560881c1c680 2023-12-01T01:20:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.482+0000 7fdc8effd700 1 -- 172.21.15.77:0/414104321 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdca001c7e0 con 0x560881c1c680 2023-12-01T01:20:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.482+0000 7fdcc213cb80 10 client.16035 fetch_fsmap learned FSMap version 31 2023-12-01T01:20:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.482+0000 7fdcc213cb80 1 -- 172.21.15.77:0/414104321 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fdca80115b0 con 0x560881c1c680 2023-12-01T01:20:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.482+0000 7fdc8effd700 1 -- 172.21.15.77:0/414104321 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdca40a2000 con 0x560881c1c680 2023-12-01T01:20:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.483+0000 7fdc8effd700 1 -- 172.21.15.77:0/414104321 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdca003b6e0 con 0x560881c1c680 2023-12-01T01:20:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.483+0000 7fdcc213cb80 10 client.16035 fetch_fsmap finished waiting for FSMap version 31 2023-12-01T01:20:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.483+0000 7fdcc213cb80 10 client.16035 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-01T01:20:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.483+0000 7fdcc213cb80 20 client.16035 populate_metadata read hostname 'smithi077' 2023-12-01T01:20:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.483+0000 7fdcc213cb80 1 --2- 172.21.15.77:0/414104321 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x560881c845a0 0x560881c869c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.483+0000 7fdcc213cb80 4 client.16035 mds_command: new command op to 24500 tid=0[{"prefix": "flush journal"}] 2023-12-01T01:20:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.483+0000 7fdcc213cb80 1 -- 172.21.15.77:0/414104321 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560881b66ce0 con 0x560881c845a0 2023-12-01T01:20:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.483+0000 7fdcaeffd700 1 --2- 172.21.15.77:0/414104321 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x560881c845a0 0x560881c869c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:30.489 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.483+0000 7fdc8effd700 10 client.16035 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:20:30.489 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.483+0000 7fdcaeffd700 1 --2- 172.21.15.77:0/414104321 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x560881c845a0 0x560881c869c0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4923a1426005fe7 server_cookie=7329d87db3e2cccc in_seq=0 out_seq=0 2023-12-01T01:20:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.500+0000 7fdc8effd700 1 -- 172.21.15.77:0/414104321 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560881b66ce0 con 0x560881c845a0 2023-12-01T01:20:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.500+0000 7fdc8effd700 10 client.16035 handle_command_reply: tid=0 2023-12-01T01:20:30.502 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.500+0000 7fdcc213cb80 1 client.16035 shutdown 2023-12-01T01:20:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.502+0000 7fdcc213cb80 1 -- 172.21.15.77:0/414104321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560881c1c680 msgr2=0x560881c2aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.502+0000 7fdcc213cb80 1 --2- 172.21.15.77:0/414104321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560881c1c680 0x560881c2aeb0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fdca001d7f0 tx=0x7fdca0009760 comp rx=0 tx=0).stop 2023-12-01T01:20:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.502+0000 7fdcc213cb80 1 -- 172.21.15.77:0/414104321 shutdown_connections 2023-12-01T01:20:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.502+0000 7fdcc213cb80 1 --2- 172.21.15.77:0/414104321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560881c1ded0 0x560881c2c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.502+0000 7fdcc213cb80 1 --2- 172.21.15.77:0/414104321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560881c1c680 0x560881c2aeb0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.502+0000 7fdcc213cb80 1 --2- 172.21.15.77:0/414104321 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x560881c845a0 0x560881c869c0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.503+0000 7fdcc213cb80 1 --2- 172.21.15.77:0/414104321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560881c1cfe0 0x560881c2b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.503+0000 7fdcc213cb80 1 -- 172.21.15.77:0/414104321 >> 172.21.15.77:0/414104321 conn(0x560881b7c070 msgr2=0x560881c2dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:20:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.503+0000 7fdc8effd700 0 client.16035 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:20:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.503+0000 7fdcc213cb80 1 -- 172.21.15.77:0/414104321 shutdown_connections 2023-12-01T01:20:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.503+0000 7fdcc213cb80 1 -- 172.21.15.77:0/414104321 wait complete. 2023-12-01T01:20:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.503+0000 7fdcc213cb80 20 client.16035 trim_cache size 0 max 16384 2023-12-01T01:20:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout:{ 2023-12-01T01:20:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout: "message": "", 2023-12-01T01:20:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout: "return_code": 0 2023-12-01T01:20:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout:} 2023-12-01T01:20:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.503+0000 7fdcb5fac700 1 -- 172.21.15.77:0/818507923 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fdc980a0520 msgr2=0x7fdc980a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.503+0000 7fdcb5fac700 1 --2- 172.21.15.77:0/818507923 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fdc980a0520 0x7fdc980a29e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0074a90 tx=0x7fdca0009260 comp rx=0 tx=0).stop 2023-12-01T01:20:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.503+0000 7fdcb5fac700 1 -- 172.21.15.77:0/818507923 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdcb0101f90 msgr2=0x7fdcb00736c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.503+0000 7fdcb5fac700 1 --2- 172.21.15.77:0/818507923 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdcb0101f90 0x7fdcb00736c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fdca4009990 tx=0x7fdca400f690 comp rx=0 tx=0).stop 2023-12-01T01:20:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.504+0000 7fdcb5fac700 1 -- 172.21.15.77:0/818507923 shutdown_connections 2023-12-01T01:20:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.504+0000 7fdcb5fac700 1 --2- 172.21.15.77:0/818507923 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fdc980a0520 0x7fdc980a29e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.504+0000 7fdcb5fac700 1 --2- 172.21.15.77:0/818507923 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdcb0102930 0x7fdcb0073c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.504+0000 7fdcb5fac700 1 --2- 172.21.15.77:0/818507923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdcb0100da0 0x7fdcb0073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.504+0000 7fdcb5fac700 1 --2- 172.21.15.77:0/818507923 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdcb0101f90 0x7fdcb00736c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.504+0000 7fdcb5fac700 1 -- 172.21.15.77:0/818507923 >> 172.21.15.77:0/818507923 conn(0x7fdcb00fc2d0 msgr2=0x7fdcb01054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:20:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.504+0000 7fdcb5fac700 1 -- 172.21.15.77:0/818507923 shutdown_connections 2023-12-01T01:20:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:30.504+0000 7fdcb5fac700 1 -- 172.21.15.77:0/818507923 wait complete. 2023-12-01T01:20:31.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:31 smithi077 ceph-mon[98867]: pgmap v377: 97 pgs: 97 active+clean; 55 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.9 MiB/s wr, 803 op/s 2023-12-01T01:20:32.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:20:31 smithi203 ceph-mon[103581]: pgmap v377: 97 pgs: 97 active+clean; 55 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.9 MiB/s wr, 803 op/s 2023-12-01T01:20:32.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:20:31 smithi137 ceph-mon[103312]: pgmap v377: 97 pgs: 97 active+clean; 55 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.9 MiB/s wr, 803 op/s 2023-12-01T01:20:34.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:20:33 smithi203 ceph-mon[103581]: pgmap v378: 97 pgs: 97 active+clean; 52 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.9 MiB/s wr, 788 op/s 2023-12-01T01:20:34.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:20:33 smithi137 ceph-mon[103312]: pgmap v378: 97 pgs: 97 active+clean; 52 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.9 MiB/s wr, 788 op/s 2023-12-01T01:20:34.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:33 smithi077 ceph-mon[98867]: pgmap v378: 97 pgs: 97 active+clean; 52 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.9 MiB/s wr, 788 op/s 2023-12-01T01:20:36.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:20:35 smithi203 ceph-mon[103581]: pgmap v379: 97 pgs: 97 active+clean; 54 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 4.6 MiB/s wr, 901 op/s 2023-12-01T01:20:36.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:20:35 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:20:36.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:20:35 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:20:36.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:20:35 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:20:36.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:20:35 smithi137 ceph-mon[103312]: pgmap v379: 97 pgs: 97 active+clean; 54 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 4.6 MiB/s wr, 901 op/s 2023-12-01T01:20:36.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:20:35 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:20:36.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:20:35 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:20:36.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:20:35 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:20:36.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:35 smithi077 ceph-mon[98867]: pgmap v379: 97 pgs: 97 active+clean; 54 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 4.6 MiB/s wr, 901 op/s 2023-12-01T01:20:36.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:35 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:20:36.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:35 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:20:36.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:35 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:20:37.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:20:36 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:20:37.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:20:36 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:20:37.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:20:36 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:20:37.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:20:36 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:20:37.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:20:36 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:20:37.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:20:36 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:20:37.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:36 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:20:37.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:36 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:20:37.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:36 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:20:38.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:20:37 smithi203 ceph-mon[103581]: pgmap v380: 97 pgs: 97 active+clean; 56 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 4.5 MiB/s wr, 933 op/s 2023-12-01T01:20:38.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:20:37 smithi137 ceph-mon[103312]: pgmap v380: 97 pgs: 97 active+clean; 56 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 4.5 MiB/s wr, 933 op/s 2023-12-01T01:20:38.153 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:37 smithi077 ceph-mon[98867]: pgmap v380: 97 pgs: 97 active+clean; 56 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 4.5 MiB/s wr, 933 op/s 2023-12-01T01:20:40.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:20:39 smithi203 ceph-mon[103581]: pgmap v381: 97 pgs: 97 active+clean; 55 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 4.3 MiB/s wr, 843 op/s 2023-12-01T01:20:40.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:20:39 smithi137 ceph-mon[103312]: pgmap v381: 97 pgs: 97 active+clean; 55 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 4.3 MiB/s wr, 843 op/s 2023-12-01T01:20:40.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:39 smithi077 ceph-mon[98867]: pgmap v381: 97 pgs: 97 active+clean; 55 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 4.3 MiB/s wr, 843 op/s 2023-12-01T01:20:42.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:20:41 smithi203 ceph-mon[103581]: pgmap v382: 97 pgs: 97 active+clean; 54 MiB data, 967 MiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 4.7 MiB/s wr, 974 op/s 2023-12-01T01:20:42.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:20:41 smithi137 ceph-mon[103312]: pgmap v382: 97 pgs: 97 active+clean; 54 MiB data, 967 MiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 4.7 MiB/s wr, 974 op/s 2023-12-01T01:20:42.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:41 smithi077 ceph-mon[98867]: pgmap v382: 97 pgs: 97 active+clean; 54 MiB data, 967 MiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 4.7 MiB/s wr, 974 op/s 2023-12-01T01:20:43.639 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.635+0000 7f6b2cfb2700 1 -- 172.21.15.77:0/435922605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b28101f90 msgr2=0x7f6b281023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:43.639 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.635+0000 7f6b2cfb2700 1 --2- 172.21.15.77:0/435922605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b28101f90 0x7f6b281023f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c0099c0 tx=0x7f6b1c009cd0 comp rx=0 tx=0).stop 2023-12-01T01:20:43.639 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.635+0000 7f6b2cfb2700 1 -- 172.21.15.77:0/435922605 shutdown_connections 2023-12-01T01:20:43.639 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.635+0000 7f6b2cfb2700 1 --2- 172.21.15.77:0/435922605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b28102930 0x7f6b281091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:43.639 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.635+0000 7f6b2cfb2700 1 --2- 172.21.15.77:0/435922605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b28100da0 0x7f6b281011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:43.639 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.635+0000 7f6b2cfb2700 1 --2- 172.21.15.77:0/435922605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b28101f90 0x7f6b281023f0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:43.639 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.635+0000 7f6b2cfb2700 1 -- 172.21.15.77:0/435922605 >> 172.21.15.77:0/435922605 conn(0x7f6b280fc310 msgr2=0x7f6b280fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:20:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.636+0000 7f6b2cfb2700 1 -- 172.21.15.77:0/435922605 shutdown_connections 2023-12-01T01:20:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.636+0000 7f6b2cfb2700 1 -- 172.21.15.77:0/435922605 wait complete. 2023-12-01T01:20:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.636+0000 7f6b2cfb2700 1 Processor -- start 2023-12-01T01:20:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.636+0000 7f6b2cfb2700 1 -- start start 2023-12-01T01:20:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.636+0000 7f6b2cfb2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b28100da0 0x7f6b28073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.636+0000 7f6b2cfb2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b28101f90 0x7f6b280736c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.636+0000 7f6b2cfb2700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b28102930 0x7f6b28073c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.636+0000 7f6b2cfb2700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b281a0b00 con 0x7f6b28100da0 2023-12-01T01:20:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.636+0000 7f6b2cfb2700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b281a0c40 con 0x7f6b28101f90 2023-12-01T01:20:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.636+0000 7f6b2cfb2700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b281a0d80 con 0x7f6b28102930 2023-12-01T01:20:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.637+0000 7f6b2659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b28100da0 0x7f6b28073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.637+0000 7f6b25d9b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b28101f90 0x7f6b280736c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:43.640 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.637+0000 7f6b2659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b28100da0 0x7f6b28073180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60680/0 (socket says 172.21.15.77:60680) 2023-12-01T01:20:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.637+0000 7f6b2659c700 1 -- 172.21.15.77:0/2440201211 learned_addr learned my addr 172.21.15.77:0/2440201211 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:20:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.637+0000 7f6b26d9d700 1 --2- 172.21.15.77:0/2440201211 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b28102930 0x7f6b28073c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.637+0000 7f6b2659c700 1 -- 172.21.15.77:0/2440201211 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b28102930 msgr2=0x7f6b28073c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.637+0000 7f6b2659c700 1 --2- 172.21.15.77:0/2440201211 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b28102930 0x7f6b28073c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.637+0000 7f6b2659c700 1 -- 172.21.15.77:0/2440201211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b28101f90 msgr2=0x7f6b280736c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.637+0000 7f6b2659c700 1 --2- 172.21.15.77:0/2440201211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b28101f90 0x7f6b280736c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.637+0000 7f6b2659c700 1 -- 172.21.15.77:0/2440201211 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b1c0095f0 con 0x7f6b28100da0 2023-12-01T01:20:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.637+0000 7f6b25d9b700 1 --2- 172.21.15.77:0/2440201211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b28101f90 0x7f6b280736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:20:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.637+0000 7f6b2659c700 1 --2- 172.21.15.77:0/2440201211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b28100da0 0x7f6b28073180 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800e970 tx=0x7f6b1800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:20:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.637+0000 7f6b177fe700 1 -- 172.21.15.77:0/2440201211 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b18009790 con 0x7f6b28100da0 2023-12-01T01:20:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.637+0000 7f6b177fe700 1 -- 172.21.15.77:0/2440201211 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b1800c7e0 con 0x7f6b28100da0 2023-12-01T01:20:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.637+0000 7f6b177fe700 1 -- 172.21.15.77:0/2440201211 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b18003c70 con 0x7f6b28100da0 2023-12-01T01:20:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.637+0000 7f6b2cfb2700 1 -- 172.21.15.77:0/2440201211 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b281a0fb0 con 0x7f6b28100da0 2023-12-01T01:20:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.637+0000 7f6b2cfb2700 1 -- 172.21.15.77:0/2440201211 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b281a13e0 con 0x7f6b28100da0 2023-12-01T01:20:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.640+0000 7f6b177fe700 1 -- 172.21.15.77:0/2440201211 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f6b1800cc50 con 0x7f6b28100da0 2023-12-01T01:20:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.640+0000 7f6b177fe700 1 --2- 172.21.15.77:0/2440201211 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f6b100a05a0 0x7f6b100a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.640+0000 7f6b177fe700 1 -- 172.21.15.77:0/2440201211 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f6b18014070 con 0x7f6b28100da0 2023-12-01T01:20:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.640+0000 7f6b25d9b700 1 --2- 172.21.15.77:0/2440201211 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f6b100a05a0 0x7f6b100a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.641+0000 7f6b39142b80 1 Processor -- start 2023-12-01T01:20:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.641+0000 7f6b25d9b700 1 --2- 172.21.15.77:0/2440201211 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f6b100a05a0 0x7f6b100a2a60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c009990 tx=0x7f6b1c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:20:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.642+0000 7f6b39142b80 1 -- start start 2023-12-01T01:20:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.642+0000 7f6b39142b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f8a337680 0x564f8a337aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.642+0000 7f6b39142b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f8a337fe0 0x564f8a3388f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:43.644 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.642+0000 7f6b39142b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f8a338ed0 0x564f8a339350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.642+0000 7f6b39142b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f8a28f9b0 con 0x564f8a338ed0 2023-12-01T01:20:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.642+0000 7f6b39142b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f8a28d370 con 0x564f8a337680 2023-12-01T01:20:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.642+0000 7f6b39142b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f8a288c60 con 0x564f8a337fe0 2023-12-01T01:20:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.642+0000 7f6b25d9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f8a337fe0 0x564f8a3388f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.642+0000 7f6b2659c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f8a337680 0x564f8a337aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.642+0000 7f6b25d9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f8a337fe0 0x564f8a3388f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:46876/0 (socket says 172.21.15.77:46876) 2023-12-01T01:20:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.642+0000 7f6b26d9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f8a338ed0 0x564f8a339350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.642+0000 7f6b2659c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f8a337680 0x564f8a337aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:39610/0 (socket says 172.21.15.77:39610) 2023-12-01T01:20:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.642+0000 7f6b25d9b700 1 -- 172.21.15.77:0/2039574005 learned_addr learned my addr 172.21.15.77:0/2039574005 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:20:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.642+0000 7f6b25d9b700 1 -- 172.21.15.77:0/2039574005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f8a337680 msgr2=0x564f8a337aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.642+0000 7f6b25d9b700 1 --2- 172.21.15.77:0/2039574005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f8a337680 0x564f8a337aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.642+0000 7f6b25d9b700 1 -- 172.21.15.77:0/2039574005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f8a338ed0 msgr2=0x564f8a339350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.642+0000 7f6b25d9b700 1 --2- 172.21.15.77:0/2039574005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f8a338ed0 0x564f8a339350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:43.645 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.642+0000 7f6b25d9b700 1 -- 172.21.15.77:0/2039574005 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f8a0b6d80 con 0x564f8a337fe0 2023-12-01T01:20:43.646 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.642+0000 7f6b2659c700 1 --2- 172.21.15.77:0/2039574005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f8a337680 0x564f8a337aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:20:43.646 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.642+0000 7f6b26d9d700 1 --2- 172.21.15.77:0/2039574005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f8a338ed0 0x564f8a339350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:20:43.646 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.643+0000 7f6b25d9b700 1 --2- 172.21.15.77:0/2039574005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f8a337fe0 0x564f8a3388f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c00b580 tx=0x7f6b1c01f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d52f4946522dad0b server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:20:43.646 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.643+0000 7f6b14ff9700 1 -- 172.21.15.77:0/2039574005 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c005250 con 0x564f8a337fe0 2023-12-01T01:20:43.646 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.643+0000 7f6b14ff9700 1 -- 172.21.15.77:0/2039574005 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b1c01e040 con 0x564f8a337fe0 2023-12-01T01:20:43.646 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.643+0000 7f6b14ff9700 1 -- 172.21.15.77:0/2039574005 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c024420 con 0x564f8a337fe0 2023-12-01T01:20:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.644+0000 7f6b39142b80 1 -- 172.21.15.77:0/2039574005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f8a337fe0 msgr2=0x564f8a3388f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.644+0000 7f6b39142b80 1 --2- 172.21.15.77:0/2039574005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f8a337fe0 0x564f8a3388f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c00b580 tx=0x7f6b1c01f000 comp rx=0 tx=0).stop 2023-12-01T01:20:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.644+0000 7f6b39142b80 1 -- 172.21.15.77:0/2039574005 shutdown_connections 2023-12-01T01:20:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.644+0000 7f6b39142b80 1 --2- 172.21.15.77:0/2039574005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f8a338ed0 0x564f8a339350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.644+0000 7f6b39142b80 1 --2- 172.21.15.77:0/2039574005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f8a337680 0x564f8a337aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.644+0000 7f6b39142b80 1 --2- 172.21.15.77:0/2039574005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f8a337fe0 0x564f8a3388f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.644+0000 7f6b39142b80 1 -- 172.21.15.77:0/2039574005 >> 172.21.15.77:0/2039574005 conn(0x564f8a297070 msgr2=0x564f8a334040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:20:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.644+0000 7f6b39142b80 1 -- 172.21.15.77:0/2039574005 shutdown_connections 2023-12-01T01:20:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.644+0000 7f6b39142b80 1 -- 172.21.15.77:0/2039574005 wait complete. 2023-12-01T01:20:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.644+0000 7f6b39142b80 1 -- start start 2023-12-01T01:20:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.644+0000 7f6b39142b80 1 Processor -- start 2023-12-01T01:20:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.644+0000 7f6b39142b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f8a337680 0x564f8a341400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.644+0000 7f6b39142b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f8a337fe0 0x564f8a341940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.644+0000 7f6b39142b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f8a338ed0 0x564f8a348980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.645+0000 7f6b39142b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f8a2254b0 con 0x564f8a338ed0 2023-12-01T01:20:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.645+0000 7f6b39142b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f8a225150 con 0x564f8a337680 2023-12-01T01:20:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.645+0000 7f6b39142b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f8a10c8c0 con 0x564f8a337fe0 2023-12-01T01:20:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.645+0000 7f6b39142b80 10 client.0 _ll_register_callbacks cb 0x564f8a172a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:20:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.645+0000 7f6b2659c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f8a337680 0x564f8a341400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.645+0000 7f6b25d9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f8a337fe0 0x564f8a341940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.645+0000 7f6b2659c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f8a337680 0x564f8a341400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:39616/0 (socket says 172.21.15.77:39616) 2023-12-01T01:20:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.645+0000 7f6b25d9b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f8a337fe0 0x564f8a341940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:46884/0 (socket says 172.21.15.77:46884) 2023-12-01T01:20:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.645+0000 7f6b2659c700 1 -- 172.21.15.77:0/4078934863 learned_addr learned my addr 172.21.15.77:0/4078934863 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:20:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.645+0000 7f6b26d9d700 1 --2- 172.21.15.77:0/4078934863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f8a338ed0 0x564f8a348980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.645+0000 7f6b2659c700 1 -- 172.21.15.77:0/4078934863 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f8a337fe0 msgr2=0x564f8a341940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.645+0000 7f6b2659c700 1 --2- 172.21.15.77:0/4078934863 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f8a337fe0 0x564f8a341940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.645+0000 7f6b2659c700 1 -- 172.21.15.77:0/4078934863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f8a338ed0 msgr2=0x564f8a348980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.645+0000 7f6b2659c700 1 --2- 172.21.15.77:0/4078934863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f8a338ed0 0x564f8a348980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.645+0000 7f6b2659c700 1 -- 172.21.15.77:0/4078934863 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b281a13e0 con 0x564f8a337680 2023-12-01T01:20:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.645+0000 7f6b2659c700 1 --2- 172.21.15.77:0/4078934863 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f8a337680 0x564f8a341400 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800efd0 tx=0x7f6b18098cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b51dd02a9c484ac server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:20:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.645+0000 7f6b0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2023-12-01T01:20:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.645+0000 7f6b0dffb700 1 -- 172.21.15.77:0/4078934863 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b180997c0 con 0x564f8a337680 2023-12-01T01:20:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.645+0000 7f6b0dffb700 1 -- 172.21.15.77:0/4078934863 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f6b18099360 con 0x564f8a337680 2023-12-01T01:20:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.645+0000 7f6b39142b80 1 -- 172.21.15.77:0/4078934863 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564f8a2b3cb0 con 0x564f8a337680 2023-12-01T01:20:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.646+0000 7f6b0dffb700 1 -- 172.21.15.77:0/4078934863 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b1809d190 con 0x564f8a337680 2023-12-01T01:20:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.646+0000 7f6b39142b80 10 client.25628 fetch_fsmap learned FSMap version 31 2023-12-01T01:20:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.646+0000 7f6b39142b80 1 -- 172.21.15.77:0/4078934863 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f6b20011230 con 0x564f8a337680 2023-12-01T01:20:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.646+0000 7f6b0dffb700 1 -- 172.21.15.77:0/4078934863 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564f8a2b3cb0 con 0x564f8a337680 2023-12-01T01:20:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.646+0000 7f6b0dffb700 1 -- 172.21.15.77:0/4078934863 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6b18098950 con 0x564f8a337680 2023-12-01T01:20:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.646+0000 7f6b39142b80 10 client.25628 fetch_fsmap finished waiting for FSMap version 31 2023-12-01T01:20:43.652 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.646+0000 7f6b39142b80 10 client.25628 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-01T01:20:43.655 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.646+0000 7f6b39142b80 20 client.25628 populate_metadata read hostname 'smithi077' 2023-12-01T01:20:43.655 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.646+0000 7f6b39142b80 1 --2- 172.21.15.77:0/4078934863 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x564f8a34d190 0x564f8a34f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:43.655 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.646+0000 7f6b39142b80 4 client.25628 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:20:43.655 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.646+0000 7f6b39142b80 1 -- 172.21.15.77:0/4078934863 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564f8a306db0 con 0x564f8a34d190 2023-12-01T01:20:43.655 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.646+0000 7f6b25d9b700 1 --2- 172.21.15.77:0/4078934863 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x564f8a34d190 0x564f8a34f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:43.655 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.647+0000 7f6b0dffb700 10 client.25628 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:20:43.655 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.647+0000 7f6b25d9b700 1 --2- 172.21.15.77:0/4078934863 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x564f8a34d190 0x564f8a34f5b0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79ce509a48e42159 server_cookie=645369b62f365fc3 in_seq=0 out_seq=0 2023-12-01T01:20:43.655 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.647+0000 7f6b0dffb700 1 -- 172.21.15.77:0/4078934863 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x564f8a306db0 con 0x564f8a34d190 2023-12-01T01:20:43.655 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.647+0000 7f6b0dffb700 10 client.25628 handle_command_reply: tid=0 2023-12-01T01:20:43.655 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.647+0000 7f6b39142b80 1 client.25628 shutdown 2023-12-01T01:20:43.655 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.647+0000 7f6b39142b80 1 -- 172.21.15.77:0/4078934863 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f8a337680 msgr2=0x564f8a341400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:43.655 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.647+0000 7f6b39142b80 1 --2- 172.21.15.77:0/4078934863 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f8a337680 0x564f8a341400 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800efd0 tx=0x7f6b18098cc0 comp rx=0 tx=0).stop 2023-12-01T01:20:43.656 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.647+0000 7f6b39142b80 1 -- 172.21.15.77:0/4078934863 shutdown_connections 2023-12-01T01:20:43.656 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.647+0000 7f6b39142b80 1 --2- 172.21.15.77:0/4078934863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f8a338ed0 0x564f8a348980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:43.656 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.647+0000 7f6b39142b80 1 --2- 172.21.15.77:0/4078934863 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x564f8a34d190 0x564f8a34f5b0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:43.656 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.647+0000 7f6b39142b80 1 --2- 172.21.15.77:0/4078934863 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f8a337680 0x564f8a341400 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:43.656 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.647+0000 7f6b0dffb700 0 client.25628 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:20:43.656 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.647+0000 7f6b39142b80 1 --2- 172.21.15.77:0/4078934863 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f8a337fe0 0x564f8a341940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:43.656 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.647+0000 7f6b39142b80 1 -- 172.21.15.77:0/4078934863 >> 172.21.15.77:0/4078934863 conn(0x564f8a297070 msgr2=0x564f8a3323c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:20:43.656 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.648+0000 7f6b39142b80 1 -- 172.21.15.77:0/4078934863 shutdown_connections 2023-12-01T01:20:43.656 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.648+0000 7f6b39142b80 1 -- 172.21.15.77:0/4078934863 wait complete. 2023-12-01T01:20:43.656 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.648+0000 7f6b39142b80 20 client.25628 trim_cache size 0 max 16384 2023-12-01T01:20:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.664+0000 7f6b39142b80 1 Processor -- start 2023-12-01T01:20:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.664+0000 7f6b39142b80 1 -- start start 2023-12-01T01:20:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.664+0000 7f6b39142b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f8a337680 0x564f8a333c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.664+0000 7f6b39142b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f8a337fe0 0x564f8a348910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.664+0000 7f6b39142b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f8a338ed0 0x564f8a334160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.664+0000 7f6b39142b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f8a340fd0 con 0x564f8a338ed0 2023-12-01T01:20:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.664+0000 7f6b39142b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f8a3425f0 con 0x564f8a337680 2023-12-01T01:20:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.664+0000 7f6b39142b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f8a34a610 con 0x564f8a337fe0 2023-12-01T01:20:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.664+0000 7f6b2659c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f8a337680 0x564f8a333c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.664+0000 7f6b2659c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f8a337680 0x564f8a333c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:39624/0 (socket says 172.21.15.77:39624) 2023-12-01T01:20:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.664+0000 7f6b2659c700 1 -- 172.21.15.77:0/3616910775 learned_addr learned my addr 172.21.15.77:0/3616910775 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:20:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.664+0000 7f6b25d9b700 1 --2- 172.21.15.77:0/3616910775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f8a337fe0 0x564f8a348910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.664+0000 7f6b26d9d700 1 --2- 172.21.15.77:0/3616910775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f8a338ed0 0x564f8a334160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.664+0000 7f6b2659c700 1 -- 172.21.15.77:0/3616910775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f8a337fe0 msgr2=0x564f8a348910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.664+0000 7f6b2659c700 1 --2- 172.21.15.77:0/3616910775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f8a337fe0 0x564f8a348910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.664+0000 7f6b2659c700 1 -- 172.21.15.77:0/3616910775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f8a338ed0 msgr2=0x564f8a334160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.664+0000 7f6b2659c700 1 --2- 172.21.15.77:0/3616910775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f8a338ed0 0x564f8a334160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.664+0000 7f6b2659c700 1 -- 172.21.15.77:0/3616910775 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b28109b30 con 0x564f8a337680 2023-12-01T01:20:43.667 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.665+0000 7f6b2659c700 1 --2- 172.21.15.77:0/3616910775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f8a337680 0x564f8a333c20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6b18095dd0 tx=0x7f6b1800c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4f0cdc306ea10c0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:20:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.668+0000 7f6b157fa700 1 -- 172.21.15.77:0/3616910775 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b180a1100 con 0x564f8a337680 2023-12-01T01:20:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.669+0000 7f6b157fa700 1 -- 172.21.15.77:0/3616910775 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b18099360 con 0x564f8a337680 2023-12-01T01:20:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.669+0000 7f6b157fa700 1 -- 172.21.15.77:0/3616910775 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b180995c0 con 0x564f8a337680 2023-12-01T01:20:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.669+0000 7f6b39142b80 1 -- 172.21.15.77:0/3616910775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f8a337680 msgr2=0x564f8a333c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.669+0000 7f6b39142b80 1 --2- 172.21.15.77:0/3616910775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f8a337680 0x564f8a333c20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6b18095dd0 tx=0x7f6b1800c450 comp rx=0 tx=0).stop 2023-12-01T01:20:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.669+0000 7f6b39142b80 1 -- 172.21.15.77:0/3616910775 shutdown_connections 2023-12-01T01:20:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.669+0000 7f6b39142b80 1 --2- 172.21.15.77:0/3616910775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f8a338ed0 0x564f8a334160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.669+0000 7f6b39142b80 1 --2- 172.21.15.77:0/3616910775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f8a337680 0x564f8a333c20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.669+0000 7f6b39142b80 1 --2- 172.21.15.77:0/3616910775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f8a337fe0 0x564f8a348910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.669+0000 7f6b39142b80 1 -- 172.21.15.77:0/3616910775 >> 172.21.15.77:0/3616910775 conn(0x564f8a297070 msgr2=0x564f8a331660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:20:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.669+0000 7f6b39142b80 1 -- 172.21.15.77:0/3616910775 shutdown_connections 2023-12-01T01:20:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.669+0000 7f6b39142b80 1 -- 172.21.15.77:0/3616910775 wait complete. 2023-12-01T01:20:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.670+0000 7f6b39142b80 1 -- start start 2023-12-01T01:20:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.670+0000 7f6b39142b80 1 Processor -- start 2023-12-01T01:20:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.670+0000 7f6b39142b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f8a337680 0x564f8a3481a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.670+0000 7f6b39142b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f8a337fe0 0x564f8a3486e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.670+0000 7f6b39142b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f8a338ed0 0x564f8a3467b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.670+0000 7f6b39142b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f8a346dd0 con 0x564f8a338ed0 2023-12-01T01:20:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.670+0000 7f6b39142b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f8a346f10 con 0x564f8a337680 2023-12-01T01:20:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.670+0000 7f6b39142b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f8a3470d0 con 0x564f8a337fe0 2023-12-01T01:20:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.670+0000 7f6b39142b80 10 client.0 _ll_register_callbacks cb 0x7f6b180840b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:20:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.670+0000 7f6b26d9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f8a338ed0 0x564f8a3467b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.670+0000 7f6b26d9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f8a338ed0 0x564f8a3467b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60710/0 (socket says 172.21.15.77:60710) 2023-12-01T01:20:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.670+0000 7f6b26d9d700 1 -- 172.21.15.77:0/3065100123 learned_addr learned my addr 172.21.15.77:0/3065100123 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:20:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.670+0000 7f6b2659c700 1 --2- 172.21.15.77:0/3065100123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f8a337680 0x564f8a3481a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.670+0000 7f6b25d9b700 1 --2- 172.21.15.77:0/3065100123 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f8a337fe0 0x564f8a3486e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.670+0000 7f6b2659c700 1 -- 172.21.15.77:0/3065100123 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f8a337fe0 msgr2=0x564f8a3486e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.670+0000 7f6b2659c700 1 --2- 172.21.15.77:0/3065100123 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f8a337fe0 0x564f8a3486e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.670+0000 7f6b2659c700 1 -- 172.21.15.77:0/3065100123 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f8a338ed0 msgr2=0x564f8a3467b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.670+0000 7f6b2659c700 1 --2- 172.21.15.77:0/3065100123 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f8a338ed0 0x564f8a3467b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.671+0000 7f6b2659c700 1 -- 172.21.15.77:0/3065100123 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b20011230 con 0x564f8a337680 2023-12-01T01:20:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.671+0000 7f6b26d9d700 1 --2- 172.21.15.77:0/3065100123 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f8a338ed0 0x564f8a3467b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:20:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.671+0000 7f6b2659c700 1 --2- 172.21.15.77:0/3065100123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f8a337680 0x564f8a3481a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800efd0 tx=0x7f6b18099240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1b24c937abd56c8 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:20:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.671+0000 7f6b0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2023-12-01T01:20:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.671+0000 7f6b0dffb700 1 -- 172.21.15.77:0/3065100123 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b180a1100 con 0x564f8a337680 2023-12-01T01:20:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.671+0000 7f6b0dffb700 1 -- 172.21.15.77:0/3065100123 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f6b1809d190 con 0x564f8a337680 2023-12-01T01:20:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.671+0000 7f6b39142b80 1 -- 172.21.15.77:0/3065100123 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6b1c01e040 con 0x564f8a337680 2023-12-01T01:20:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.671+0000 7f6b0dffb700 1 -- 172.21.15.77:0/3065100123 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b18098cc0 con 0x564f8a337680 2023-12-01T01:20:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.671+0000 7f6b39142b80 10 client.25634 fetch_fsmap learned FSMap version 31 2023-12-01T01:20:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.671+0000 7f6b39142b80 1 -- 172.21.15.77:0/3065100123 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x564f8a09fc00 con 0x564f8a337680 2023-12-01T01:20:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.671+0000 7f6b0dffb700 1 -- 172.21.15.77:0/3065100123 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b1c01e040 con 0x564f8a337680 2023-12-01T01:20:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.671+0000 7f6b0dffb700 1 -- 172.21.15.77:0/3065100123 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6b180ab030 con 0x564f8a337680 2023-12-01T01:20:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.672+0000 7f6b39142b80 10 client.25634 fetch_fsmap finished waiting for FSMap version 31 2023-12-01T01:20:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.672+0000 7f6b39142b80 10 client.25634 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-01T01:20:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.672+0000 7f6b39142b80 20 client.25634 populate_metadata read hostname 'smithi077' 2023-12-01T01:20:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.672+0000 7f6b39142b80 1 --2- 172.21.15.77:0/3065100123 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x564f8a39fa50 0x564f8a3a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.672+0000 7f6b39142b80 4 client.25634 mds_command: new command op to 24500 tid=0[{"prefix": "flush journal"}] 2023-12-01T01:20:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.672+0000 7f6b39142b80 1 -- 172.21.15.77:0/3065100123 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564f8a281ce0 con 0x564f8a39fa50 2023-12-01T01:20:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.672+0000 7f6b25d9b700 1 --2- 172.21.15.77:0/3065100123 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x564f8a39fa50 0x564f8a3a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.672+0000 7f6b25d9b700 1 --2- 172.21.15.77:0/3065100123 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x564f8a39fa50 0x564f8a3a1e70 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ebf8c28c818d63a server_cookie=646e9967a079fdae in_seq=0 out_seq=0 2023-12-01T01:20:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.672+0000 7f6b0dffb700 10 client.25634 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:20:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.689+0000 7f6b0dffb700 1 -- 172.21.15.77:0/3065100123 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564f8a281ce0 con 0x564f8a39fa50 2023-12-01T01:20:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.689+0000 7f6b0dffb700 10 client.25634 handle_command_reply: tid=0 2023-12-01T01:20:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.689+0000 7f6b39142b80 1 client.25634 shutdown 2023-12-01T01:20:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.690+0000 7f6b39142b80 1 -- 172.21.15.77:0/3065100123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f8a337680 msgr2=0x564f8a3481a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.690+0000 7f6b39142b80 1 --2- 172.21.15.77:0/3065100123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f8a337680 0x564f8a3481a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800efd0 tx=0x7f6b18099240 comp rx=0 tx=0).stop 2023-12-01T01:20:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.690+0000 7f6b39142b80 1 -- 172.21.15.77:0/3065100123 shutdown_connections 2023-12-01T01:20:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.690+0000 7f6b39142b80 1 --2- 172.21.15.77:0/3065100123 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564f8a338ed0 0x564f8a3467b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.690+0000 7f6b39142b80 1 --2- 172.21.15.77:0/3065100123 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x564f8a39fa50 0x564f8a3a1e70 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.690+0000 7f6b39142b80 1 --2- 172.21.15.77:0/3065100123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f8a337680 0x564f8a3481a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.690+0000 7f6b39142b80 1 --2- 172.21.15.77:0/3065100123 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564f8a337fe0 0x564f8a3486e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.690+0000 7f6b0dffb700 0 client.25634 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:20:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.690+0000 7f6b39142b80 1 -- 172.21.15.77:0/3065100123 >> 172.21.15.77:0/3065100123 conn(0x564f8a297070 msgr2=0x564f8a333c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:20:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.690+0000 7f6b39142b80 1 -- 172.21.15.77:0/3065100123 shutdown_connections 2023-12-01T01:20:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.690+0000 7f6b39142b80 1 -- 172.21.15.77:0/3065100123 wait complete. 2023-12-01T01:20:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.690+0000 7f6b39142b80 20 client.25634 trim_cache size 0 max 16384 2023-12-01T01:20:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout:{ 2023-12-01T01:20:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout: "message": "", 2023-12-01T01:20:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout: "return_code": 0 2023-12-01T01:20:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout:} 2023-12-01T01:20:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.690+0000 7f6b2cfb2700 1 -- 172.21.15.77:0/2440201211 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f6b100a05a0 msgr2=0x7f6b100a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.690+0000 7f6b2cfb2700 1 --2- 172.21.15.77:0/2440201211 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f6b100a05a0 0x7f6b100a2a60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c009990 tx=0x7f6b1c01a040 comp rx=0 tx=0).stop 2023-12-01T01:20:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.690+0000 7f6b2cfb2700 1 -- 172.21.15.77:0/2440201211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b28100da0 msgr2=0x7f6b28073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.690+0000 7f6b2cfb2700 1 --2- 172.21.15.77:0/2440201211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b28100da0 0x7f6b28073180 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800e970 tx=0x7f6b1800ec80 comp rx=0 tx=0).stop 2023-12-01T01:20:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.690+0000 7f6b2cfb2700 1 -- 172.21.15.77:0/2440201211 shutdown_connections 2023-12-01T01:20:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.690+0000 7f6b2cfb2700 1 --2- 172.21.15.77:0/2440201211 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f6b100a05a0 0x7f6b100a2a60 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.690+0000 7f6b2cfb2700 1 --2- 172.21.15.77:0/2440201211 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b28102930 0x7f6b28073c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.690+0000 7f6b2cfb2700 1 --2- 172.21.15.77:0/2440201211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b28100da0 0x7f6b28073180 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.690+0000 7f6b2cfb2700 1 --2- 172.21.15.77:0/2440201211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b28101f90 0x7f6b280736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.690+0000 7f6b2cfb2700 1 -- 172.21.15.77:0/2440201211 >> 172.21.15.77:0/2440201211 conn(0x7f6b280fc310 msgr2=0x7f6b281054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:20:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.692+0000 7f6b2cfb2700 1 -- 172.21.15.77:0/2440201211 shutdown_connections 2023-12-01T01:20:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:43.692+0000 7f6b2cfb2700 1 -- 172.21.15.77:0/2440201211 wait complete. 2023-12-01T01:20:44.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:20:43 smithi203 ceph-mon[103581]: pgmap v383: 97 pgs: 97 active+clean; 54 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 3.5 MiB/s wr, 722 op/s 2023-12-01T01:20:44.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:20:43 smithi137 ceph-mon[103312]: pgmap v383: 97 pgs: 97 active+clean; 54 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 3.5 MiB/s wr, 722 op/s 2023-12-01T01:20:44.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:43 smithi077 ceph-mon[98867]: pgmap v383: 97 pgs: 97 active+clean; 54 MiB data, 951 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 3.5 MiB/s wr, 722 op/s 2023-12-01T01:20:46.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:20:45 smithi203 ceph-mon[103581]: pgmap v384: 97 pgs: 97 active+clean; 53 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 4.5 MiB/s wr, 942 op/s 2023-12-01T01:20:46.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:20:45 smithi137 ceph-mon[103312]: pgmap v384: 97 pgs: 97 active+clean; 53 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 4.5 MiB/s wr, 942 op/s 2023-12-01T01:20:46.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:45 smithi077 ceph-mon[98867]: pgmap v384: 97 pgs: 97 active+clean; 53 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 4.5 MiB/s wr, 942 op/s 2023-12-01T01:20:48.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:20:47 smithi203 ceph-mon[103581]: pgmap v385: 97 pgs: 97 active+clean; 53 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 4.2 MiB/s wr, 898 op/s 2023-12-01T01:20:48.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:20:47 smithi137 ceph-mon[103312]: pgmap v385: 97 pgs: 97 active+clean; 53 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 4.2 MiB/s wr, 898 op/s 2023-12-01T01:20:48.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:47 smithi077 ceph-mon[98867]: pgmap v385: 97 pgs: 97 active+clean; 53 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 4.2 MiB/s wr, 898 op/s 2023-12-01T01:20:50.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:20:49 smithi203 ceph-mon[103581]: pgmap v386: 97 pgs: 97 active+clean; 50 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 4.0 MiB/s wr, 814 op/s 2023-12-01T01:20:50.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:20:49 smithi137 ceph-mon[103312]: pgmap v386: 97 pgs: 97 active+clean; 50 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 4.0 MiB/s wr, 814 op/s 2023-12-01T01:20:50.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:49 smithi077 ceph-mon[98867]: pgmap v386: 97 pgs: 97 active+clean; 50 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 4.0 MiB/s wr, 814 op/s 2023-12-01T01:20:51.742 INFO:tasks.workunit.client.0.smithi077.stdout:40000 copy from 0x19d87f to 0x1a73ec, (0x9b6d bytes) at 0x9241fd 2023-12-01T01:20:52.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:20:51 smithi203 ceph-mon[103581]: pgmap v387: 97 pgs: 97 active+clean; 50 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 4.4 MiB/s wr, 929 op/s 2023-12-01T01:20:52.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:20:51 smithi137 ceph-mon[103312]: pgmap v387: 97 pgs: 97 active+clean; 50 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 4.4 MiB/s wr, 929 op/s 2023-12-01T01:20:52.153 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:51 smithi077 ceph-mon[98867]: pgmap v387: 97 pgs: 97 active+clean; 50 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 4.4 MiB/s wr, 929 op/s 2023-12-01T01:20:54.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:20:53 smithi203 ceph-mon[103581]: pgmap v388: 97 pgs: 97 active+clean; 50 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.3 MiB/s wr, 681 op/s 2023-12-01T01:20:54.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:20:53 smithi137 ceph-mon[103312]: pgmap v388: 97 pgs: 97 active+clean; 50 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.3 MiB/s wr, 681 op/s 2023-12-01T01:20:54.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:53 smithi077 ceph-mon[98867]: pgmap v388: 97 pgs: 97 active+clean; 50 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.3 MiB/s wr, 681 op/s 2023-12-01T01:20:56.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:20:55 smithi203 ceph-mon[103581]: pgmap v389: 97 pgs: 97 active+clean; 52 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 4.1 MiB/s wr, 822 op/s 2023-12-01T01:20:56.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:20:55 smithi137 ceph-mon[103312]: pgmap v389: 97 pgs: 97 active+clean; 52 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 4.1 MiB/s wr, 822 op/s 2023-12-01T01:20:56.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:55 smithi077 ceph-mon[98867]: pgmap v389: 97 pgs: 97 active+clean; 52 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 4.1 MiB/s wr, 822 op/s 2023-12-01T01:20:56.827 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.824+0000 7f00740fa700 1 -- 172.21.15.77:0/2458673253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f006c106b10 msgr2=0x7f006c108f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:56.827 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.824+0000 7f00740fa700 1 --2- 172.21.15.77:0/2458673253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f006c106b10 0x7f006c108f00 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0064009f90 tx=0x7f006400b330 comp rx=0 tx=0).stop 2023-12-01T01:20:56.827 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.824+0000 7f00740fa700 1 -- 172.21.15.77:0/2458673253 shutdown_connections 2023-12-01T01:20:56.828 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.824+0000 7f00740fa700 1 --2- 172.21.15.77:0/2458673253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f006c106b10 0x7f006c108f00 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:56.828 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.824+0000 7f00740fa700 1 --2- 172.21.15.77:0/2458673253 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f006c1018b0 0x7f006c103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:56.828 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.824+0000 7f00740fa700 1 --2- 172.21.15.77:0/2458673253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f006c1041e0 0x7f006c1065d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:56.828 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.824+0000 7f00740fa700 1 -- 172.21.15.77:0/2458673253 >> 172.21.15.77:0/2458673253 conn(0x7f006c0fb1e0 msgr2=0x7f006c0fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:20:56.828 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.824+0000 7f00740fa700 1 -- 172.21.15.77:0/2458673253 shutdown_connections 2023-12-01T01:20:56.828 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.824+0000 7f00740fa700 1 -- 172.21.15.77:0/2458673253 wait complete. 2023-12-01T01:20:56.828 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.824+0000 7f00740fa700 1 Processor -- start 2023-12-01T01:20:56.828 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.824+0000 7f00740fa700 1 -- start start 2023-12-01T01:20:56.828 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.825+0000 7f00740fa700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f006c1018b0 0x7f006c198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:56.828 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.825+0000 7f00740fa700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f006c1041e0 0x7f006c199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:56.828 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.825+0000 7f00740fa700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f006c106b10 0x7f006c19a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:56.828 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.825+0000 7f00740fa700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006c19a8e0 con 0x7f006c1041e0 2023-12-01T01:20:56.828 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.825+0000 7f00740fa700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006c19aa50 con 0x7f006c1018b0 2023-12-01T01:20:56.829 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.825+0000 7f00740fa700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006c19ac10 con 0x7f006c106b10 2023-12-01T01:20:56.829 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.825+0000 7f0071695700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f006c1041e0 0x7f006c199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:56.829 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.825+0000 7f0071e96700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f006c1018b0 0x7f006c198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:56.829 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.825+0000 7f0071e96700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f006c1018b0 0x7f006c198ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:39644/0 (socket says 172.21.15.77:39644) 2023-12-01T01:20:56.829 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.825+0000 7f0071e96700 1 -- 172.21.15.77:0/464153230 learned_addr learned my addr 172.21.15.77:0/464153230 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:20:56.829 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.825+0000 7f0071695700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f006c1041e0 0x7f006c199430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60724/0 (socket says 172.21.15.77:60724) 2023-12-01T01:20:56.829 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.825+0000 7f0072697700 1 --2- 172.21.15.77:0/464153230 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f006c106b10 0x7f006c19a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:56.829 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.825+0000 7f0071e96700 1 -- 172.21.15.77:0/464153230 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f006c106b10 msgr2=0x7f006c19a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:56.829 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.825+0000 7f0071e96700 1 --2- 172.21.15.77:0/464153230 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f006c106b10 0x7f006c19a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:56.829 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.825+0000 7f0071e96700 1 -- 172.21.15.77:0/464153230 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f006c1041e0 msgr2=0x7f006c199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:56.829 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.825+0000 7f0071e96700 1 --2- 172.21.15.77:0/464153230 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f006c1041e0 0x7f006c199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:56.829 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.825+0000 7f0071e96700 1 -- 172.21.15.77:0/464153230 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0064009c40 con 0x7f006c1018b0 2023-12-01T01:20:56.830 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.825+0000 7f0071695700 1 --2- 172.21.15.77:0/464153230 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f006c1041e0 0x7f006c199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:20:56.830 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.825+0000 7f0071e96700 1 --2- 172.21.15.77:0/464153230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f006c1018b0 0x7f006c198ef0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f006800e9c0 tx=0x7f006800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:20:56.830 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.825+0000 7f0062ffd700 1 -- 172.21.15.77:0/464153230 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0068004a30 con 0x7f006c1018b0 2023-12-01T01:20:56.830 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.826+0000 7f00740fa700 1 -- 172.21.15.77:0/464153230 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f006c19aef0 con 0x7f006c1018b0 2023-12-01T01:20:56.830 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.826+0000 7f0062ffd700 1 -- 172.21.15.77:0/464153230 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f006800cc80 con 0x7f006c1018b0 2023-12-01T01:20:56.830 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.826+0000 7f0062ffd700 1 -- 172.21.15.77:0/464153230 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00680054e0 con 0x7f006c1018b0 2023-12-01T01:20:56.830 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.826+0000 7f00740fa700 1 -- 172.21.15.77:0/464153230 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f006c1a16b0 con 0x7f006c1018b0 2023-12-01T01:20:56.833 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.828+0000 7f0062ffd700 1 -- 172.21.15.77:0/464153230 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f0068004b90 con 0x7f006c1018b0 2023-12-01T01:20:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.829+0000 7f0062ffd700 1 --2- 172.21.15.77:0/464153230 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f00580a0520 0x7f00580a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.829+0000 7f0062ffd700 1 -- 172.21.15.77:0/464153230 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f0068014070 con 0x7f006c1018b0 2023-12-01T01:20:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.829+0000 7f0071695700 1 --2- 172.21.15.77:0/464153230 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f00580a0520 0x7f00580a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.830+0000 7f0071695700 1 --2- 172.21.15.77:0/464153230 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f00580a0520 0x7f00580a29e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f005c011490 tx=0x7f005c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:20:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.830+0000 7f008028ab80 1 Processor -- start 2023-12-01T01:20:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.830+0000 7f008028ab80 1 -- start start 2023-12-01T01:20:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.830+0000 7f008028ab80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f019756680 0x55f019756aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.830+0000 7f008028ab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f019756fe0 0x55f0197578f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:56.835 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.830+0000 7f008028ab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f019757ed0 0x55f019758350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:56.835 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.830+0000 7f008028ab80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0196ae9b0 con 0x55f019756680 2023-12-01T01:20:56.835 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.830+0000 7f008028ab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0196ac370 con 0x55f019757ed0 2023-12-01T01:20:56.835 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.830+0000 7f008028ab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0196a7c60 con 0x55f019756fe0 2023-12-01T01:20:56.835 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.831+0000 7f0071e96700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f019756680 0x55f019756aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:56.835 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.831+0000 7f0071e96700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f019756680 0x55f019756aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60730/0 (socket says 172.21.15.77:60730) 2023-12-01T01:20:56.835 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.831+0000 7f0071e96700 1 -- 172.21.15.77:0/3140342574 learned_addr learned my addr 172.21.15.77:0/3140342574 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:20:56.835 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.831+0000 7f0071695700 1 --2- 172.21.15.77:0/3140342574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f019756fe0 0x55f0197578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:56.835 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.831+0000 7f0072697700 1 --2- 172.21.15.77:0/3140342574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f019757ed0 0x55f019758350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:56.835 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.831+0000 7f0071e96700 1 -- 172.21.15.77:0/3140342574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f019756fe0 msgr2=0x55f0197578f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:56.835 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.831+0000 7f0071e96700 1 --2- 172.21.15.77:0/3140342574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f019756fe0 0x55f0197578f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.831+0000 7f0071e96700 1 -- 172.21.15.77:0/3140342574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f019757ed0 msgr2=0x55f019758350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.831+0000 7f0071e96700 1 --2- 172.21.15.77:0/3140342574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f019757ed0 0x55f019758350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.831+0000 7f0071e96700 1 -- 172.21.15.77:0/3140342574 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0194d5d80 con 0x55f019756680 2023-12-01T01:20:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.831+0000 7f0071e96700 1 --2- 172.21.15.77:0/3140342574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f019756680 0x55f019756aa0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0068095d30 tx=0x7f0068098ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1716c50d536b37bd server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:20:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.831+0000 7f0057fff700 1 -- 172.21.15.77:0/3140342574 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00680b0070 con 0x55f019756680 2023-12-01T01:20:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.831+0000 7f0057fff700 1 -- 172.21.15.77:0/3140342574 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00680992c0 con 0x55f019756680 2023-12-01T01:20:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.831+0000 7f0057fff700 1 -- 172.21.15.77:0/3140342574 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0068099520 con 0x55f019756680 2023-12-01T01:20:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.832+0000 7f008028ab80 1 -- 172.21.15.77:0/3140342574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f019756680 msgr2=0x55f019756aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.832+0000 7f008028ab80 1 --2- 172.21.15.77:0/3140342574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f019756680 0x55f019756aa0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0068095d30 tx=0x7f0068098ca0 comp rx=0 tx=0).stop 2023-12-01T01:20:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.832+0000 7f008028ab80 1 -- 172.21.15.77:0/3140342574 shutdown_connections 2023-12-01T01:20:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.832+0000 7f008028ab80 1 --2- 172.21.15.77:0/3140342574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f019757ed0 0x55f019758350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.832+0000 7f008028ab80 1 --2- 172.21.15.77:0/3140342574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f019756680 0x55f019756aa0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.832+0000 7f008028ab80 1 --2- 172.21.15.77:0/3140342574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f019756fe0 0x55f0197578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:56.837 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.832+0000 7f008028ab80 1 -- 172.21.15.77:0/3140342574 >> 172.21.15.77:0/3140342574 conn(0x55f0196b6070 msgr2=0x55f019753040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:20:56.837 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.832+0000 7f008028ab80 1 -- 172.21.15.77:0/3140342574 shutdown_connections 2023-12-01T01:20:56.837 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.832+0000 7f008028ab80 1 -- 172.21.15.77:0/3140342574 wait complete. 2023-12-01T01:20:56.837 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.832+0000 7f008028ab80 1 -- start start 2023-12-01T01:20:56.837 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.833+0000 7f008028ab80 1 Processor -- start 2023-12-01T01:20:56.840 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.833+0000 7f008028ab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f019756680 0x55f019760560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:56.840 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.833+0000 7f008028ab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f019756fe0 0x55f019760aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.833+0000 7f008028ab80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f019757ed0 0x55f019767ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.833+0000 7f008028ab80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0196444b0 con 0x55f019757ed0 2023-12-01T01:20:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.833+0000 7f008028ab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f019644150 con 0x55f019756fe0 2023-12-01T01:20:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.833+0000 7f008028ab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f01952b8c0 con 0x55f019756680 2023-12-01T01:20:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.833+0000 7f008028ab80 10 client.0 _ll_register_callbacks cb 0x55f019591a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:20:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.833+0000 7f0072697700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f019757ed0 0x55f019767ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.833+0000 7f0071e96700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f019756680 0x55f019760560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.833+0000 7f0071e96700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f019756680 0x55f019760560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:46924/0 (socket says 172.21.15.77:46924) 2023-12-01T01:20:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.833+0000 7f0071695700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f019756fe0 0x55f019760aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.833+0000 7f0072697700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f019757ed0 0x55f019767ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60740/0 (socket says 172.21.15.77:60740) 2023-12-01T01:20:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.833+0000 7f0071e96700 1 -- 172.21.15.77:0/2353577967 learned_addr learned my addr 172.21.15.77:0/2353577967 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:20:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.833+0000 7f0071e96700 1 -- 172.21.15.77:0/2353577967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f019756fe0 msgr2=0x55f019760aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:56.842 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.833+0000 7f0071e96700 1 --2- 172.21.15.77:0/2353577967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f019756fe0 0x55f019760aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:56.842 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.833+0000 7f0071e96700 1 -- 172.21.15.77:0/2353577967 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f019757ed0 msgr2=0x55f019767ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:56.842 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.833+0000 7f0071e96700 1 --2- 172.21.15.77:0/2353577967 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f019757ed0 0x55f019767ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:56.842 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.833+0000 7f0071e96700 1 -- 172.21.15.77:0/2353577967 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f006c1a16b0 con 0x55f019756680 2023-12-01T01:20:56.842 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.834+0000 7f0072697700 1 --2- 172.21.15.77:0/2353577967 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f019757ed0 0x55f019767ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:20:56.842 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.834+0000 7f0071e96700 1 --2- 172.21.15.77:0/2353577967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f019756680 0x55f019760560 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f006800cfd0 tx=0x7f0068098d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3eb79ed1975a2256 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:20:56.842 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.834+0000 7f00557fa700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2023-12-01T01:20:56.842 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.834+0000 7f00557fa700 1 -- 172.21.15.77:0/2353577967 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00680b7070 con 0x55f019756680 2023-12-01T01:20:56.842 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.834+0000 7f00557fa700 1 -- 172.21.15.77:0/2353577967 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f00680b1070 con 0x55f019756680 2023-12-01T01:20:56.842 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.834+0000 7f008028ab80 1 -- 172.21.15.77:0/2353577967 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f0196d2cb0 con 0x55f019756680 2023-12-01T01:20:56.842 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.834+0000 7f00557fa700 1 -- 172.21.15.77:0/2353577967 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00680b0030 con 0x55f019756680 2023-12-01T01:20:56.842 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.834+0000 7f008028ab80 10 client.25660 fetch_fsmap learned FSMap version 31 2023-12-01T01:20:56.842 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.834+0000 7f008028ab80 1 -- 172.21.15.77:0/2353577967 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55f0194bec00 con 0x55f019756680 2023-12-01T01:20:56.843 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.834+0000 7f00557fa700 1 -- 172.21.15.77:0/2353577967 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f006807c020 con 0x55f019756680 2023-12-01T01:20:56.843 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.834+0000 7f00557fa700 1 -- 172.21.15.77:0/2353577967 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f00680a0ea0 con 0x55f019756680 2023-12-01T01:20:56.843 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.835+0000 7f008028ab80 10 client.25660 fetch_fsmap finished waiting for FSMap version 31 2023-12-01T01:20:56.843 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.835+0000 7f008028ab80 10 client.25660 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-01T01:20:56.843 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.835+0000 7f008028ab80 20 client.25660 populate_metadata read hostname 'smithi077' 2023-12-01T01:20:56.843 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.835+0000 7f008028ab80 1 --2- 172.21.15.77:0/2353577967 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55f01976c050 0x55f01976e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:56.843 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.835+0000 7f008028ab80 4 client.25660 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:20:56.843 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.835+0000 7f008028ab80 1 -- 172.21.15.77:0/2353577967 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f019725db0 con 0x55f01976c050 2023-12-01T01:20:56.843 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.835+0000 7f0071695700 1 --2- 172.21.15.77:0/2353577967 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55f01976c050 0x55f01976e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:56.843 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.835+0000 7f00557fa700 10 client.25660 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:20:56.843 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.835+0000 7f0071695700 1 --2- 172.21.15.77:0/2353577967 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55f01976c050 0x55f01976e470 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92ae7860ae0a8f65 server_cookie=277c53a708bf4f13 in_seq=0 out_seq=0 2023-12-01T01:20:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.836+0000 7f00557fa700 1 -- 172.21.15.77:0/2353577967 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55f019725db0 con 0x55f01976c050 2023-12-01T01:20:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.836+0000 7f00557fa700 10 client.25660 handle_command_reply: tid=0 2023-12-01T01:20:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.836+0000 7f008028ab80 1 client.25660 shutdown 2023-12-01T01:20:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.836+0000 7f008028ab80 1 -- 172.21.15.77:0/2353577967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f019756680 msgr2=0x55f019760560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.836+0000 7f008028ab80 1 --2- 172.21.15.77:0/2353577967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f019756680 0x55f019760560 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f006800cfd0 tx=0x7f0068098d10 comp rx=0 tx=0).stop 2023-12-01T01:20:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.836+0000 7f008028ab80 1 -- 172.21.15.77:0/2353577967 shutdown_connections 2023-12-01T01:20:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.836+0000 7f008028ab80 1 --2- 172.21.15.77:0/2353577967 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f019757ed0 0x55f019767ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.836+0000 7f008028ab80 1 --2- 172.21.15.77:0/2353577967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f019756680 0x55f019760560 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.836+0000 7f008028ab80 1 --2- 172.21.15.77:0/2353577967 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55f01976c050 0x55f01976e470 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.836+0000 7f008028ab80 1 --2- 172.21.15.77:0/2353577967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f019756fe0 0x55f019760aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.836+0000 7f00557fa700 0 client.25660 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:20:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.836+0000 7f008028ab80 1 -- 172.21.15.77:0/2353577967 >> 172.21.15.77:0/2353577967 conn(0x55f0196b6070 msgr2=0x55f0197513c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:20:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.836+0000 7f008028ab80 1 -- 172.21.15.77:0/2353577967 shutdown_connections 2023-12-01T01:20:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.836+0000 7f008028ab80 1 -- 172.21.15.77:0/2353577967 wait complete. 2023-12-01T01:20:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.836+0000 7f008028ab80 20 client.25660 trim_cache size 0 max 16384 2023-12-01T01:20:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.853+0000 7f008028ab80 1 Processor -- start 2023-12-01T01:20:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.853+0000 7f008028ab80 1 -- start start 2023-12-01T01:20:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.853+0000 7f008028ab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f019756680 0x55f019752c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.853+0000 7f008028ab80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f019756fe0 0x55f019767a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.853+0000 7f008028ab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f019757ed0 0x55f019753160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.853+0000 7f008028ab80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f019761750 con 0x55f019756fe0 2023-12-01T01:20:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.853+0000 7f008028ab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0197497a0 con 0x55f019757ed0 2023-12-01T01:20:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.853+0000 7f008028ab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0197499f0 con 0x55f019756680 2023-12-01T01:20:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.853+0000 7f0071695700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f019756fe0 0x55f019767a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.853+0000 7f0071695700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f019756fe0 0x55f019767a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60748/0 (socket says 172.21.15.77:60748) 2023-12-01T01:20:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.853+0000 7f0071695700 1 -- 172.21.15.77:0/3242943104 learned_addr learned my addr 172.21.15.77:0/3242943104 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:20:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.853+0000 7f0071e96700 1 --2- 172.21.15.77:0/3242943104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f019756680 0x55f019752c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.853+0000 7f0072697700 1 --2- 172.21.15.77:0/3242943104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f019757ed0 0x55f019753160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.853+0000 7f0071695700 1 -- 172.21.15.77:0/3242943104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f019756680 msgr2=0x55f019752c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.853+0000 7f0071695700 1 --2- 172.21.15.77:0/3242943104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f019756680 0x55f019752c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.853+0000 7f0071695700 1 -- 172.21.15.77:0/3242943104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f019757ed0 msgr2=0x55f019753160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.853+0000 7f0071695700 1 --2- 172.21.15.77:0/3242943104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f019757ed0 0x55f019753160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.853+0000 7f0071695700 1 -- 172.21.15.77:0/3242943104 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0194fc1f0 con 0x55f019756fe0 2023-12-01T01:20:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.854+0000 7f0071695700 1 --2- 172.21.15.77:0/3242943104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f019756fe0 0x55f019767a70 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x55f01976ed10 tx=0x7f005c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d493b55508a43af server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:20:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.854+0000 7f0060ff9700 1 -- 172.21.15.77:0/3242943104 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c024040 con 0x55f019756fe0 2023-12-01T01:20:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.854+0000 7f0060ff9700 1 -- 172.21.15.77:0/3242943104 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f005c003c40 con 0x55f019756fe0 2023-12-01T01:20:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.854+0000 7f0060ff9700 1 -- 172.21.15.77:0/3242943104 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c01e7f0 con 0x55f019756fe0 2023-12-01T01:20:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.854+0000 7f008028ab80 1 -- 172.21.15.77:0/3242943104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f019756fe0 msgr2=0x55f019767a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.854+0000 7f008028ab80 1 --2- 172.21.15.77:0/3242943104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f019756fe0 0x55f019767a70 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x55f01976ed10 tx=0x7f005c014000 comp rx=0 tx=0).stop 2023-12-01T01:20:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.855+0000 7f008028ab80 1 -- 172.21.15.77:0/3242943104 shutdown_connections 2023-12-01T01:20:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.855+0000 7f008028ab80 1 --2- 172.21.15.77:0/3242943104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f019757ed0 0x55f019753160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.855+0000 7f008028ab80 1 --2- 172.21.15.77:0/3242943104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f019756680 0x55f019752c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.855+0000 7f008028ab80 1 --2- 172.21.15.77:0/3242943104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f019756fe0 0x55f019767a70 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.855+0000 7f008028ab80 1 -- 172.21.15.77:0/3242943104 >> 172.21.15.77:0/3242943104 conn(0x55f0196b6070 msgr2=0x55f01975e360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:20:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.855+0000 7f008028ab80 1 -- 172.21.15.77:0/3242943104 shutdown_connections 2023-12-01T01:20:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.855+0000 7f008028ab80 1 -- 172.21.15.77:0/3242943104 wait complete. 2023-12-01T01:20:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.855+0000 7f008028ab80 1 -- start start 2023-12-01T01:20:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.855+0000 7f008028ab80 1 Processor -- start 2023-12-01T01:20:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.856+0000 7f008028ab80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f019756680 0x55f0197672f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.856+0000 7f008028ab80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f019756fe0 0x55f019767830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.856+0000 7f008028ab80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f019757ed0 0x55f019765900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.856+0000 7f008028ab80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f019765ee0 con 0x55f019757ed0 2023-12-01T01:20:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.856+0000 7f008028ab80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f019766050 con 0x55f019756680 2023-12-01T01:20:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.856+0000 7f008028ab80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f019766210 con 0x55f019756fe0 2023-12-01T01:20:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.856+0000 7f008028ab80 10 client.0 _ll_register_callbacks cb 0x7f006c199ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:20:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.856+0000 7f0072697700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f019757ed0 0x55f019765900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.856+0000 7f0071695700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f019756fe0 0x55f019767830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.856+0000 7f0071e96700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f019756680 0x55f0197672f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.856+0000 7f0071695700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f019756fe0 0x55f019767830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:46940/0 (socket says 172.21.15.77:46940) 2023-12-01T01:20:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.856+0000 7f0072697700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f019757ed0 0x55f019765900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60754/0 (socket says 172.21.15.77:60754) 2023-12-01T01:20:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.856+0000 7f0071695700 1 -- 172.21.15.77:0/911497748 learned_addr learned my addr 172.21.15.77:0/911497748 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:20:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.856+0000 7f0071e96700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f019756680 0x55f0197672f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:39672/0 (socket says 172.21.15.77:39672) 2023-12-01T01:20:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.856+0000 7f0072697700 1 -- 172.21.15.77:0/911497748 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f019756fe0 msgr2=0x55f019767830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.856+0000 7f0072697700 1 --2- 172.21.15.77:0/911497748 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f019756fe0 0x55f019767830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.856+0000 7f0072697700 1 -- 172.21.15.77:0/911497748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f019756680 msgr2=0x55f0197672f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.856+0000 7f0072697700 1 --2- 172.21.15.77:0/911497748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f019756680 0x55f0197672f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.856+0000 7f0072697700 1 -- 172.21.15.77:0/911497748 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f005c017070 con 0x55f019757ed0 2023-12-01T01:20:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.856+0000 7f0071e96700 1 --2- 172.21.15.77:0/911497748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f019756680 0x55f0197672f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:20:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.856+0000 7f0071695700 1 --2- 172.21.15.77:0/911497748 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f019756fe0 0x55f019767830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:20:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.856+0000 7f0072697700 1 --2- 172.21.15.77:0/911497748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f019757ed0 0x55f019765900 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0064007900 tx=0x7f0064009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af4172e3a6b86138 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:20:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.856+0000 7f00557fa700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:20:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.856+0000 7f00557fa700 1 -- 172.21.15.77:0/911497748 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006400e040 con 0x55f019757ed0 2023-12-01T01:20:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.856+0000 7f00557fa700 1 -- 172.21.15.77:0/911497748 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f006400bbf0 con 0x55f019757ed0 2023-12-01T01:20:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.856+0000 7f008028ab80 1 -- 172.21.15.77:0/911497748 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f00680992c0 con 0x55f019757ed0 2023-12-01T01:20:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.857+0000 7f00557fa700 1 -- 172.21.15.77:0/911497748 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f006400bdb0 con 0x55f019757ed0 2023-12-01T01:20:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.857+0000 7f008028ab80 10 client.16071 fetch_fsmap learned FSMap version 31 2023-12-01T01:20:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.857+0000 7f008028ab80 1 -- 172.21.15.77:0/911497748 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55f0194d4250 con 0x55f019757ed0 2023-12-01T01:20:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.857+0000 7f00557fa700 1 -- 172.21.15.77:0/911497748 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00680992c0 con 0x55f019757ed0 2023-12-01T01:20:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.857+0000 7f00557fa700 1 -- 172.21.15.77:0/911497748 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f006400b050 con 0x55f019757ed0 2023-12-01T01:20:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.857+0000 7f008028ab80 10 client.16071 fetch_fsmap finished waiting for FSMap version 31 2023-12-01T01:20:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.857+0000 7f008028ab80 10 client.16071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-01T01:20:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.857+0000 7f008028ab80 20 client.16071 populate_metadata read hostname 'smithi077' 2023-12-01T01:20:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.857+0000 7f008028ab80 1 --2- 172.21.15.77:0/911497748 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55f0197beef0 0x55f0197c1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.857+0000 7f008028ab80 4 client.16071 mds_command: new command op to 24500 tid=0[{"prefix": "flush journal"}] 2023-12-01T01:20:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.857+0000 7f008028ab80 1 -- 172.21.15.77:0/911497748 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f0196a0ce0 con 0x55f0197beef0 2023-12-01T01:20:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.857+0000 7f0071e96700 1 --2- 172.21.15.77:0/911497748 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55f0197beef0 0x55f0197c1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.858+0000 7f00557fa700 10 client.16071 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:20:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.858+0000 7f0071e96700 1 --2- 172.21.15.77:0/911497748 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55f0197beef0 0x55f0197c1310 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f1ad4ab5896f649 server_cookie=2b85b1367e54cd32 in_seq=0 out_seq=0 2023-12-01T01:20:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.875+0000 7f00557fa700 1 -- 172.21.15.77:0/911497748 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f0196a0ce0 con 0x55f0197beef0 2023-12-01T01:20:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.875+0000 7f00557fa700 10 client.16071 handle_command_reply: tid=0 2023-12-01T01:20:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.875+0000 7f008028ab80 1 client.16071 shutdown 2023-12-01T01:20:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.875+0000 7f008028ab80 1 -- 172.21.15.77:0/911497748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f019757ed0 msgr2=0x55f019765900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.875+0000 7f008028ab80 1 --2- 172.21.15.77:0/911497748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f019757ed0 0x55f019765900 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0064007900 tx=0x7f0064009bd0 comp rx=0 tx=0).stop 2023-12-01T01:20:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.875+0000 7f008028ab80 1 -- 172.21.15.77:0/911497748 shutdown_connections 2023-12-01T01:20:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.875+0000 7f008028ab80 1 --2- 172.21.15.77:0/911497748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f019757ed0 0x55f019765900 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.875+0000 7f008028ab80 1 --2- 172.21.15.77:0/911497748 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55f0197beef0 0x55f0197c1310 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.875+0000 7f008028ab80 1 --2- 172.21.15.77:0/911497748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f019756680 0x55f0197672f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.875+0000 7f00557fa700 0 client.16071 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:20:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.875+0000 7f008028ab80 1 --2- 172.21.15.77:0/911497748 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f019756fe0 0x55f019767830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.875+0000 7f008028ab80 1 -- 172.21.15.77:0/911497748 >> 172.21.15.77:0/911497748 conn(0x55f0196b6070 msgr2=0x55f019752c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:20:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.875+0000 7f008028ab80 1 -- 172.21.15.77:0/911497748 shutdown_connections 2023-12-01T01:20:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.875+0000 7f008028ab80 1 -- 172.21.15.77:0/911497748 wait complete. 2023-12-01T01:20:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.875+0000 7f008028ab80 20 client.16071 trim_cache size 0 max 16384 2023-12-01T01:20:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout:{ 2023-12-01T01:20:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout: "message": "", 2023-12-01T01:20:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout: "return_code": 0 2023-12-01T01:20:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout:} 2023-12-01T01:20:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.876+0000 7f00740fa700 1 -- 172.21.15.77:0/464153230 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f00580a0520 msgr2=0x7f00580a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.876+0000 7f00740fa700 1 --2- 172.21.15.77:0/464153230 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f00580a0520 0x7f00580a29e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f005c011490 tx=0x7f005c009260 comp rx=0 tx=0).stop 2023-12-01T01:20:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.876+0000 7f00740fa700 1 -- 172.21.15.77:0/464153230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f006c1018b0 msgr2=0x7f006c198ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.876+0000 7f00740fa700 1 --2- 172.21.15.77:0/464153230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f006c1018b0 0x7f006c198ef0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f006800e9c0 tx=0x7f006800ecd0 comp rx=0 tx=0).stop 2023-12-01T01:20:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.876+0000 7f00740fa700 1 -- 172.21.15.77:0/464153230 shutdown_connections 2023-12-01T01:20:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.876+0000 7f00740fa700 1 --2- 172.21.15.77:0/464153230 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f00580a0520 0x7f00580a29e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.876+0000 7f00740fa700 1 --2- 172.21.15.77:0/464153230 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f006c106b10 0x7f006c19a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.876+0000 7f00740fa700 1 --2- 172.21.15.77:0/464153230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f006c1018b0 0x7f006c198ef0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.876+0000 7f00740fa700 1 --2- 172.21.15.77:0/464153230 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f006c1041e0 0x7f006c199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.876+0000 7f00740fa700 1 -- 172.21.15.77:0/464153230 >> 172.21.15.77:0/464153230 conn(0x7f006c0fb1e0 msgr2=0x7f006c102520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:20:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.876+0000 7f00740fa700 1 -- 172.21.15.77:0/464153230 shutdown_connections 2023-12-01T01:20:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:20:56.876+0000 7f00740fa700 1 -- 172.21.15.77:0/464153230 wait complete. 2023-12-01T01:20:58.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:20:57 smithi203 ceph-mon[103581]: pgmap v390: 97 pgs: 97 active+clean; 54 MiB data, 1006 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 3.7 MiB/s wr, 734 op/s 2023-12-01T01:20:58.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:20:57 smithi137 ceph-mon[103312]: pgmap v390: 97 pgs: 97 active+clean; 54 MiB data, 1006 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 3.7 MiB/s wr, 734 op/s 2023-12-01T01:20:58.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:57 smithi077 ceph-mon[98867]: pgmap v390: 97 pgs: 97 active+clean; 54 MiB data, 1006 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 3.7 MiB/s wr, 734 op/s 2023-12-01T01:20:59.241 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-01T01:20:59.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.392+0000 7f542e196700 1 -- 172.21.15.77:0/1293776501 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5428101810 msgr2=0x7f54281080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:59.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.392+0000 7f542e196700 1 --2- 172.21.15.77:0/1293776501 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5428101810 0x7f54281080c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f542000b0d0 tx=0x7f542000b3e0 comp rx=0 tx=0).stop 2023-12-01T01:20:59.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.392+0000 7f542e196700 1 -- 172.21.15.77:0/1293776501 shutdown_connections 2023-12-01T01:20:59.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.392+0000 7f542e196700 1 --2- 172.21.15.77:0/1293776501 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5428101810 0x7f54281080c0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:59.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.392+0000 7f542e196700 1 --2- 172.21.15.77:0/1293776501 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54280ffc80 0x7f54281000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:59.395 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.392+0000 7f542e196700 1 --2- 172.21.15.77:0/1293776501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5428100e70 0x7f54281012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:59.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.392+0000 7f542e196700 1 -- 172.21.15.77:0/1293776501 >> 172.21.15.77:0/1293776501 conn(0x7f54280fb1f0 msgr2=0x7f54280fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:20:59.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.392+0000 7f542e196700 1 -- 172.21.15.77:0/1293776501 shutdown_connections 2023-12-01T01:20:59.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.392+0000 7f542e196700 1 -- 172.21.15.77:0/1293776501 wait complete. 2023-12-01T01:20:59.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.392+0000 7f542e196700 1 Processor -- start 2023-12-01T01:20:59.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.393+0000 7f542e196700 1 -- start start 2023-12-01T01:20:59.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.393+0000 7f542e196700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54280ffc80 0x7f5428103c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:59.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.393+0000 7f542e196700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5428100e70 0x7f5428102250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:59.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.393+0000 7f542e196700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5428101810 0x7f5428102790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:59.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.393+0000 7f542e196700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5428102f30 con 0x7f5428100e70 2023-12-01T01:20:59.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.393+0000 7f542e196700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54281030a0 con 0x7f5428101810 2023-12-01T01:20:59.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.393+0000 7f542e196700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54281a0bb0 con 0x7f54280ffc80 2023-12-01T01:20:59.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.393+0000 7f5426ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5428100e70 0x7f5428102250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:59.396 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.393+0000 7f54277fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54280ffc80 0x7f5428103c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:59.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.393+0000 7f5426ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5428100e70 0x7f5428102250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60768/0 (socket says 172.21.15.77:60768) 2023-12-01T01:20:59.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.393+0000 7f54277fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54280ffc80 0x7f5428103c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:46952/0 (socket says 172.21.15.77:46952) 2023-12-01T01:20:59.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.393+0000 7f5426ffd700 1 -- 172.21.15.77:0/805858460 learned_addr learned my addr 172.21.15.77:0/805858460 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:20:59.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.393+0000 7f5427fff700 1 --2- 172.21.15.77:0/805858460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5428101810 0x7f5428102790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:59.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.393+0000 7f5426ffd700 1 -- 172.21.15.77:0/805858460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54280ffc80 msgr2=0x7f5428103c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:59.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.393+0000 7f5426ffd700 1 --2- 172.21.15.77:0/805858460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54280ffc80 0x7f5428103c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:59.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.393+0000 7f5426ffd700 1 -- 172.21.15.77:0/805858460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5428101810 msgr2=0x7f5428102790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:59.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.393+0000 7f5426ffd700 1 --2- 172.21.15.77:0/805858460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5428101810 0x7f5428102790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:59.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.393+0000 7f5426ffd700 1 -- 172.21.15.77:0/805858460 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5420009d00 con 0x7f5428100e70 2023-12-01T01:20:59.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.393+0000 7f5427fff700 1 --2- 172.21.15.77:0/805858460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5428101810 0x7f5428102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:20:59.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.394+0000 7f5426ffd700 1 --2- 172.21.15.77:0/805858460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5428100e70 0x7f5428102250 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f541c00e9a0 tx=0x7f541c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:20:59.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.394+0000 7f5424ff9700 1 -- 172.21.15.77:0/805858460 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541c009790 con 0x7f5428100e70 2023-12-01T01:20:59.397 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.394+0000 7f542e196700 1 -- 172.21.15.77:0/805858460 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54281a0e90 con 0x7f5428100e70 2023-12-01T01:20:59.398 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.394+0000 7f5424ff9700 1 -- 172.21.15.77:0/805858460 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f541c010430 con 0x7f5428100e70 2023-12-01T01:20:59.398 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.394+0000 7f5424ff9700 1 -- 172.21.15.77:0/805858460 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541c018680 con 0x7f5428100e70 2023-12-01T01:20:59.398 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.394+0000 7f542e196700 1 -- 172.21.15.77:0/805858460 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54281a1360 con 0x7f5428100e70 2023-12-01T01:20:59.399 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.396+0000 7f5424ff9700 1 -- 172.21.15.77:0/805858460 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f541c0105a0 con 0x7f5428100e70 2023-12-01T01:20:59.399 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.397+0000 7f5424ff9700 1 --2- 172.21.15.77:0/805858460 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f54100a05a0 0x7f54100a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:59.399 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.397+0000 7f5424ff9700 1 -- 172.21.15.77:0/805858460 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f541c014070 con 0x7f5428100e70 2023-12-01T01:20:59.399 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.397+0000 7f54277fe700 1 --2- 172.21.15.77:0/805858460 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f54100a05a0 0x7f54100a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:59.400 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.398+0000 7f54277fe700 1 --2- 172.21.15.77:0/805858460 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f54100a05a0 0x7f54100a2a60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5428100cd0 tx=0x7f5418009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:20:59.401 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.398+0000 7f543a326b80 1 Processor -- start 2023-12-01T01:20:59.401 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.398+0000 7f543a326b80 1 -- start start 2023-12-01T01:20:59.401 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.398+0000 7f543a326b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642113360f0 0x564211336530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:59.401 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.398+0000 7f543a326b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564211336a70 0x564211334e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:59.401 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.398+0000 7f543a326b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642113353f0 0x564211335870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:59.401 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.398+0000 7f543a326b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x56421128ca10 con 0x564211336a70 2023-12-01T01:20:59.401 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.398+0000 7f543a326b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56421128a3d0 con 0x5642113353f0 2023-12-01T01:20:59.401 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.398+0000 7f543a326b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x564211285cc0 con 0x5642113360f0 2023-12-01T01:20:59.401 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.399+0000 7f54277fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642113360f0 0x564211336530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:59.401 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.399+0000 7f5426ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564211336a70 0x564211334e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:59.401 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.399+0000 7f54277fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642113360f0 0x564211336530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:46960/0 (socket says 172.21.15.77:46960) 2023-12-01T01:20:59.401 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.399+0000 7f54277fe700 1 -- 172.21.15.77:0/1359133034 learned_addr learned my addr 172.21.15.77:0/1359133034 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:20:59.401 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.399+0000 7f5427fff700 1 --2- 172.21.15.77:0/1359133034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642113353f0 0x564211335870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:59.402 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.399+0000 7f54277fe700 1 -- 172.21.15.77:0/1359133034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642113353f0 msgr2=0x564211335870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:59.402 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.399+0000 7f54277fe700 1 --2- 172.21.15.77:0/1359133034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642113353f0 0x564211335870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:59.402 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.399+0000 7f54277fe700 1 -- 172.21.15.77:0/1359133034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564211336a70 msgr2=0x564211334e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:59.402 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.399+0000 7f54277fe700 1 --2- 172.21.15.77:0/1359133034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564211336a70 0x564211334e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:59.402 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.399+0000 7f54277fe700 1 -- 172.21.15.77:0/1359133034 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642110b3de0 con 0x5642113360f0 2023-12-01T01:20:59.402 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.399+0000 7f54277fe700 1 --2- 172.21.15.77:0/1359133034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642113360f0 0x564211336530 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5418011d50 tx=0x7f5418014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6c6ce6cf11d315d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:20:59.402 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.399+0000 7f540dffb700 1 -- 172.21.15.77:0/1359133034 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541800ed20 con 0x5642113360f0 2023-12-01T01:20:59.402 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.399+0000 7f540dffb700 1 -- 172.21.15.77:0/1359133034 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5418021070 con 0x5642113360f0 2023-12-01T01:20:59.402 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.399+0000 7f540dffb700 1 -- 172.21.15.77:0/1359133034 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541801d410 con 0x5642113360f0 2023-12-01T01:20:59.403 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.400+0000 7f543a326b80 1 -- 172.21.15.77:0/1359133034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642113360f0 msgr2=0x564211336530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:59.403 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.400+0000 7f543a326b80 1 --2- 172.21.15.77:0/1359133034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642113360f0 0x564211336530 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5418011d50 tx=0x7f5418014000 comp rx=0 tx=0).stop 2023-12-01T01:20:59.403 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.400+0000 7f543a326b80 1 -- 172.21.15.77:0/1359133034 shutdown_connections 2023-12-01T01:20:59.403 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.400+0000 7f543a326b80 1 --2- 172.21.15.77:0/1359133034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642113353f0 0x564211335870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:59.404 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.400+0000 7f543a326b80 1 --2- 172.21.15.77:0/1359133034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642113360f0 0x564211336530 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5418011d50 tx=0x7f5418014000 comp rx=0 tx=0).stop 2023-12-01T01:20:59.404 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.400+0000 7f543a326b80 1 --2- 172.21.15.77:0/1359133034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564211336a70 0x564211334e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:59.404 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.400+0000 7f543a326b80 1 -- 172.21.15.77:0/1359133034 >> 172.21.15.77:0/1359133034 conn(0x5642112940d0 msgr2=0x56421132f940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:20:59.404 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.402+0000 7f543a326b80 1 -- 172.21.15.77:0/1359133034 shutdown_connections 2023-12-01T01:20:59.404 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.402+0000 7f543a326b80 1 -- 172.21.15.77:0/1359133034 wait complete. 2023-12-01T01:20:59.407 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.403+0000 7f543a326b80 1 -- start start 2023-12-01T01:20:59.407 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.403+0000 7f543a326b80 1 Processor -- start 2023-12-01T01:20:59.407 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.403+0000 7f543a326b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642113353f0 0x56421133ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:59.407 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.403+0000 7f543a326b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564211336a70 0x56421133f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:59.407 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.403+0000 7f543a326b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564211344120 0x564211346510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:59.407 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.403+0000 7f543a326b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x564211222510 con 0x564211336a70 2023-12-01T01:20:59.407 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.403+0000 7f543a326b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642112221b0 con 0x564211344120 2023-12-01T01:20:59.407 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.403+0000 7f543a326b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x564211109920 con 0x5642113353f0 2023-12-01T01:20:59.407 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.403+0000 7f543a326b80 10 client.0 _ll_register_callbacks cb 0x56421116fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:20:59.407 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.404+0000 7f54277fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642113353f0 0x56421133ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:59.407 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.404+0000 7f5426ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564211336a70 0x56421133f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:59.408 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.404+0000 7f54277fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642113353f0 0x56421133ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:46966/0 (socket says 172.21.15.77:46966) 2023-12-01T01:20:59.408 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.404+0000 7f54277fe700 1 -- 172.21.15.77:0/1706534572 learned_addr learned my addr 172.21.15.77:0/1706534572 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:20:59.408 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.404+0000 7f5427fff700 1 --2- 172.21.15.77:0/1706534572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564211344120 0x564211346510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:59.408 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.404+0000 7f54277fe700 1 -- 172.21.15.77:0/1706534572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564211344120 msgr2=0x564211346510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:59.408 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.404+0000 7f54277fe700 1 --2- 172.21.15.77:0/1706534572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564211344120 0x564211346510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:59.408 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.404+0000 7f54277fe700 1 -- 172.21.15.77:0/1706534572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564211336a70 msgr2=0x56421133f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:59.408 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.404+0000 7f54277fe700 1 --2- 172.21.15.77:0/1706534572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564211336a70 0x56421133f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:59.408 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.404+0000 7f54277fe700 1 -- 172.21.15.77:0/1706534572 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5418017070 con 0x5642113353f0 2023-12-01T01:20:59.408 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.404+0000 7f5426ffd700 1 --2- 172.21.15.77:0/1706534572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564211336a70 0x56421133f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:20:59.408 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.404+0000 7f54277fe700 1 --2- 172.21.15.77:0/1706534572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642113353f0 0x56421133ebb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f541800e970 tx=0x7f541801cda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3699e081065da398 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:20:59.408 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.404+0000 7f5406ffd700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2023-12-01T01:20:59.408 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.404+0000 7f5406ffd700 1 -- 172.21.15.77:0/1706534572 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541800ed30 con 0x5642113353f0 2023-12-01T01:20:59.408 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.404+0000 7f5406ffd700 1 -- 172.21.15.77:0/1706534572 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f541801c410 con 0x5642113353f0 2023-12-01T01:20:59.408 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.404+0000 7f543a326b80 1 -- 172.21.15.77:0/1706534572 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5642112b0d10 con 0x5642113353f0 2023-12-01T01:20:59.409 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.405+0000 7f5406ffd700 1 -- 172.21.15.77:0/1706534572 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f541801c5d0 con 0x5642113353f0 2023-12-01T01:20:59.409 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.405+0000 7f543a326b80 10 client.25678 fetch_fsmap learned FSMap version 31 2023-12-01T01:20:59.409 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.405+0000 7f543a326b80 1 -- 172.21.15.77:0/1706534572 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56421109cc60 con 0x5642113353f0 2023-12-01T01:20:59.409 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.405+0000 7f5406ffd700 1 -- 172.21.15.77:0/1706534572 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5642112b0d10 con 0x5642113353f0 2023-12-01T01:20:59.409 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.405+0000 7f5406ffd700 1 -- 172.21.15.77:0/1706534572 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f54180048b0 con 0x5642113353f0 2023-12-01T01:20:59.409 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.405+0000 7f543a326b80 10 client.25678 fetch_fsmap finished waiting for FSMap version 31 2023-12-01T01:20:59.409 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.405+0000 7f543a326b80 10 client.25678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-01T01:20:59.409 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.405+0000 7f543a326b80 20 client.25678 populate_metadata read hostname 'smithi077' 2023-12-01T01:20:59.409 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.405+0000 7f543a326b80 1 --2- 172.21.15.77:0/1706534572 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x56421134ad40 0x56421134d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:59.409 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.405+0000 7f543a326b80 4 client.25678 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:20:59.409 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.405+0000 7f543a326b80 1 -- 172.21.15.77:0/1706534572 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564211303e10 con 0x56421134ad40 2023-12-01T01:20:59.409 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.405+0000 7f5426ffd700 1 --2- 172.21.15.77:0/1706534572 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x56421134ad40 0x56421134d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:59.409 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.406+0000 7f5406ffd700 10 client.25678 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:20:59.409 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.406+0000 7f5426ffd700 1 --2- 172.21.15.77:0/1706534572 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x56421134ad40 0x56421134d160 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d367f494ce385b69 server_cookie=96f3bad95b13b5c1 in_seq=0 out_seq=0 2023-12-01T01:20:59.410 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.406+0000 7f5406ffd700 1 -- 172.21.15.77:0/1706534572 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x564211303e10 con 0x56421134ad40 2023-12-01T01:20:59.411 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.406+0000 7f5406ffd700 10 client.25678 handle_command_reply: tid=0 2023-12-01T01:20:59.411 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.406+0000 7f543a326b80 1 client.25678 shutdown 2023-12-01T01:20:59.411 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.406+0000 7f543a326b80 1 -- 172.21.15.77:0/1706534572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642113353f0 msgr2=0x56421133ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:59.411 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.406+0000 7f543a326b80 1 --2- 172.21.15.77:0/1706534572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642113353f0 0x56421133ebb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f541800e970 tx=0x7f541801cda0 comp rx=0 tx=0).stop 2023-12-01T01:20:59.411 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.407+0000 7f543a326b80 1 -- 172.21.15.77:0/1706534572 shutdown_connections 2023-12-01T01:20:59.411 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.407+0000 7f543a326b80 1 --2- 172.21.15.77:0/1706534572 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x56421134ad40 0x56421134d160 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:59.411 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.407+0000 7f543a326b80 1 --2- 172.21.15.77:0/1706534572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564211344120 0x564211346510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:59.411 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.407+0000 7f5406ffd700 0 client.25678 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:20:59.411 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.407+0000 7f543a326b80 1 --2- 172.21.15.77:0/1706534572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642113353f0 0x56421133ebb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:59.411 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.407+0000 7f543a326b80 1 --2- 172.21.15.77:0/1706534572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564211336a70 0x56421133f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:59.411 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.407+0000 7f543a326b80 1 -- 172.21.15.77:0/1706534572 >> 172.21.15.77:0/1706534572 conn(0x5642112940d0 msgr2=0x56421132f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:20:59.411 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.407+0000 7f543a326b80 1 -- 172.21.15.77:0/1706534572 shutdown_connections 2023-12-01T01:20:59.411 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.407+0000 7f543a326b80 1 -- 172.21.15.77:0/1706534572 wait complete. 2023-12-01T01:20:59.411 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.407+0000 7f543a326b80 20 client.25678 trim_cache size 0 max 16384 2023-12-01T01:20:59.428 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.426+0000 7f543a326b80 1 Processor -- start 2023-12-01T01:20:59.428 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.426+0000 7f543a326b80 1 -- start start 2023-12-01T01:20:59.428 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.426+0000 7f543a326b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5642113353f0 0x564211334e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:59.428 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.426+0000 7f543a326b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564211336a70 0x56421135a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:59.429 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.426+0000 7f543a326b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564211344120 0x5642113394f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:59.429 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.426+0000 7f543a326b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x564211346a00 con 0x5642113353f0 2023-12-01T01:20:59.429 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.426+0000 7f543a326b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56421134a860 con 0x564211336a70 2023-12-01T01:20:59.429 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.426+0000 7f543a326b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x564211327b50 con 0x564211344120 2023-12-01T01:20:59.429 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.426+0000 7f54277fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5642113353f0 0x564211334e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:59.429 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.426+0000 7f54277fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5642113353f0 0x564211334e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60786/0 (socket says 172.21.15.77:60786) 2023-12-01T01:20:59.429 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.426+0000 7f54277fe700 1 -- 172.21.15.77:0/3391716776 learned_addr learned my addr 172.21.15.77:0/3391716776 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:20:59.429 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.426+0000 7f5427fff700 1 --2- 172.21.15.77:0/3391716776 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564211344120 0x5642113394f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:59.429 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.426+0000 7f5426ffd700 1 --2- 172.21.15.77:0/3391716776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564211336a70 0x56421135a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:59.429 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.426+0000 7f54277fe700 1 -- 172.21.15.77:0/3391716776 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564211344120 msgr2=0x5642113394f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:59.429 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.426+0000 7f54277fe700 1 --2- 172.21.15.77:0/3391716776 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564211344120 0x5642113394f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:59.429 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.427+0000 7f54277fe700 1 -- 172.21.15.77:0/3391716776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564211336a70 msgr2=0x56421135a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:59.429 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.427+0000 7f54277fe700 1 --2- 172.21.15.77:0/3391716776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564211336a70 0x56421135a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:59.430 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.427+0000 7f54277fe700 1 -- 172.21.15.77:0/3391716776 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642110da250 con 0x5642113353f0 2023-12-01T01:20:59.430 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.427+0000 7f5426ffd700 1 --2- 172.21.15.77:0/3391716776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564211336a70 0x56421135a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:20:59.430 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.427+0000 7f54277fe700 1 --2- 172.21.15.77:0/3391716776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5642113353f0 0x564211334e60 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5418016040 tx=0x7f541801cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7866af32d957817d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:20:59.430 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.427+0000 7f540e7fc700 1 -- 172.21.15.77:0/3391716776 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5418004ea0 con 0x5642113353f0 2023-12-01T01:20:59.430 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.427+0000 7f540e7fc700 1 -- 172.21.15.77:0/3391716776 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f541801ce10 con 0x5642113353f0 2023-12-01T01:20:59.430 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.427+0000 7f540e7fc700 1 -- 172.21.15.77:0/3391716776 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5418004020 con 0x5642113353f0 2023-12-01T01:20:59.434 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.428+0000 7f543a326b80 1 -- 172.21.15.77:0/3391716776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5642113353f0 msgr2=0x564211334e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:59.434 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.428+0000 7f543a326b80 1 --2- 172.21.15.77:0/3391716776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5642113353f0 0x564211334e60 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5418016040 tx=0x7f541801cb50 comp rx=0 tx=0).stop 2023-12-01T01:20:59.434 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.428+0000 7f543a326b80 1 -- 172.21.15.77:0/3391716776 shutdown_connections 2023-12-01T01:20:59.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.428+0000 7f543a326b80 1 --2- 172.21.15.77:0/3391716776 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564211344120 0x5642113394f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:59.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.428+0000 7f543a326b80 1 --2- 172.21.15.77:0/3391716776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5642113353f0 0x564211334e60 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:59.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.428+0000 7f543a326b80 1 --2- 172.21.15.77:0/3391716776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564211336a70 0x56421135a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:59.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.428+0000 7f543a326b80 1 -- 172.21.15.77:0/3391716776 >> 172.21.15.77:0/3391716776 conn(0x5642112940d0 msgr2=0x56421133d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:20:59.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.428+0000 7f543a326b80 1 -- 172.21.15.77:0/3391716776 shutdown_connections 2023-12-01T01:20:59.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.428+0000 7f543a326b80 1 -- 172.21.15.77:0/3391716776 wait complete. 2023-12-01T01:20:59.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.428+0000 7f543a326b80 1 -- start start 2023-12-01T01:20:59.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.429+0000 7f543a326b80 1 Processor -- start 2023-12-01T01:20:59.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.429+0000 7f543a326b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642113353f0 0x564211396da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:59.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.429+0000 7f543a326b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564211336a70 0x5642113992f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:59.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.429+0000 7f543a326b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564211344120 0x56421139b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:59.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.429+0000 7f543a326b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x56421139c0e0 con 0x564211344120 2023-12-01T01:20:59.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.429+0000 7f543a326b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56421139c250 con 0x5642113353f0 2023-12-01T01:20:59.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.429+0000 7f543a326b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642113ad420 con 0x564211336a70 2023-12-01T01:20:59.435 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.429+0000 7f543a326b80 10 client.0 _ll_register_callbacks cb 0x7f541c005760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:20:59.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.429+0000 7f5427fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564211344120 0x56421139b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:59.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.429+0000 7f5426ffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564211336a70 0x5642113992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:59.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.429+0000 7f54277fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642113353f0 0x564211396da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:59.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.429+0000 7f5426ffd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564211336a70 0x5642113992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:46980/0 (socket says 172.21.15.77:46980) 2023-12-01T01:20:59.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.429+0000 7f54277fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642113353f0 0x564211396da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:39716/0 (socket says 172.21.15.77:39716) 2023-12-01T01:20:59.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.429+0000 7f5427fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564211344120 0x56421139b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60796/0 (socket says 172.21.15.77:60796) 2023-12-01T01:20:59.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.429+0000 7f5426ffd700 1 -- 172.21.15.77:0/26029218 learned_addr learned my addr 172.21.15.77:0/26029218 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:20:59.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.429+0000 7f5426ffd700 1 -- 172.21.15.77:0/26029218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642113353f0 msgr2=0x564211396da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:59.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.429+0000 7f5426ffd700 1 --2- 172.21.15.77:0/26029218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642113353f0 0x564211396da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:59.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.429+0000 7f5426ffd700 1 -- 172.21.15.77:0/26029218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564211344120 msgr2=0x56421139b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:20:59.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.429+0000 7f5426ffd700 1 --2- 172.21.15.77:0/26029218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564211344120 0x56421139b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:59.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.429+0000 7f5426ffd700 1 -- 172.21.15.77:0/26029218 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56421109cc60 con 0x564211336a70 2023-12-01T01:20:59.436 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.429+0000 7f5427fff700 1 --2- 172.21.15.77:0/26029218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564211344120 0x56421139b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:20:59.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.430+0000 7f5426ffd700 1 --2- 172.21.15.77:0/26029218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564211336a70 0x5642113992f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f541c095d50 tx=0x7f541c084040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71684c05a70e86a9 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:20:59.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.430+0000 7f5406ffd700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2023-12-01T01:20:59.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.430+0000 7f5406ffd700 1 -- 172.21.15.77:0/26029218 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541c07c070 con 0x564211336a70 2023-12-01T01:20:59.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.430+0000 7f5406ffd700 1 -- 172.21.15.77:0/26029218 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f541c0b1040 con 0x564211336a70 2023-12-01T01:20:59.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.430+0000 7f543a326b80 1 -- 172.21.15.77:0/26029218 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5418021070 con 0x564211336a70 2023-12-01T01:20:59.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.430+0000 7f5406ffd700 1 -- 172.21.15.77:0/26029218 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f541c09d160 con 0x564211336a70 2023-12-01T01:20:59.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.430+0000 7f543a326b80 10 client.25684 fetch_fsmap learned FSMap version 31 2023-12-01T01:20:59.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.430+0000 7f543a326b80 1 -- 172.21.15.77:0/26029218 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5642110b22b0 con 0x564211336a70 2023-12-01T01:20:59.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.430+0000 7f5406ffd700 1 -- 172.21.15.77:0/26029218 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f541c0a5020 con 0x564211336a70 2023-12-01T01:20:59.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.430+0000 7f5406ffd700 1 -- 172.21.15.77:0/26029218 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f541c07f030 con 0x564211336a70 2023-12-01T01:20:59.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.430+0000 7f543a326b80 10 client.25684 fetch_fsmap finished waiting for FSMap version 31 2023-12-01T01:20:59.437 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.430+0000 7f543a326b80 10 client.25684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-01T01:20:59.438 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.430+0000 7f543a326b80 20 client.25684 populate_metadata read hostname 'smithi077' 2023-12-01T01:20:59.438 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.430+0000 7f543a326b80 1 --2- 172.21.15.77:0/26029218 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5642113aef50 0x5642113b1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:20:59.438 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.430+0000 7f543a326b80 4 client.25684 mds_command: new command op to 24500 tid=0[{"prefix": "scrub status"}] 2023-12-01T01:20:59.438 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.430+0000 7f543a326b80 1 -- 172.21.15.77:0/26029218 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56421127ed40 con 0x5642113aef50 2023-12-01T01:20:59.438 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.431+0000 7f54277fe700 1 --2- 172.21.15.77:0/26029218 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5642113aef50 0x5642113b1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:20:59.438 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.431+0000 7f5406ffd700 10 client.25684 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:20:59.438 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.431+0000 7f54277fe700 1 --2- 172.21.15.77:0/26029218 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5642113aef50 0x5642113b1370 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f29c854db9793d8b server_cookie=dc3049c15bb6c8c8 in_seq=0 out_seq=0 2023-12-01T01:20:59.438 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.431+0000 7f5406ffd700 1 -- 172.21.15.77:0/26029218 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56421127ed40 con 0x5642113aef50 2023-12-01T01:20:59.438 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.431+0000 7f5406ffd700 10 client.25684 handle_command_reply: tid=0 2023-12-01T01:20:59.438 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.431+0000 7f543a326b80 1 client.25684 shutdown 2023-12-01T01:20:59.438 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.432+0000 7f543a326b80 1 -- 172.21.15.77:0/26029218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564211336a70 msgr2=0x5642113992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:59.438 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.432+0000 7f543a326b80 1 --2- 172.21.15.77:0/26029218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564211336a70 0x5642113992f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f541c095d50 tx=0x7f541c084040 comp rx=0 tx=0).stop 2023-12-01T01:20:59.438 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.432+0000 7f543a326b80 1 -- 172.21.15.77:0/26029218 shutdown_connections 2023-12-01T01:20:59.438 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.432+0000 7f543a326b80 1 --2- 172.21.15.77:0/26029218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564211344120 0x56421139b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:59.439 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.432+0000 7f543a326b80 1 --2- 172.21.15.77:0/26029218 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5642113aef50 0x5642113b1370 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:59.439 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.432+0000 7f543a326b80 1 --2- 172.21.15.77:0/26029218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642113353f0 0x564211396da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:59.439 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.432+0000 7f543a326b80 1 --2- 172.21.15.77:0/26029218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564211336a70 0x5642113992f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:59.439 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.432+0000 7f5406ffd700 0 client.25684 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:20:59.439 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.432+0000 7f543a326b80 1 -- 172.21.15.77:0/26029218 >> 172.21.15.77:0/26029218 conn(0x5642112940d0 msgr2=0x5642113394f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:20:59.439 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.432+0000 7f543a326b80 1 -- 172.21.15.77:0/26029218 shutdown_connections 2023-12-01T01:20:59.439 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.432+0000 7f543a326b80 1 -- 172.21.15.77:0/26029218 wait complete. 2023-12-01T01:20:59.439 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.432+0000 7f543a326b80 20 client.25684 trim_cache size 0 max 16384 2023-12-01T01:20:59.439 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2023-12-01T01:20:59.439 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2023-12-01T01:20:59.439 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2023-12-01T01:20:59.440 INFO:teuthology.orchestra.run.smithi077.stdout: "085290de-33b6-4527-ab7f-03f7dfd5dbba": { 2023-12-01T01:20:59.440 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2023-12-01T01:20:59.440 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "085290de-33b6-4527-ab7f-03f7dfd5dbba", 2023-12-01T01:20:59.440 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2023-12-01T01:20:59.440 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:20:59.440 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:20:59.440 INFO:teuthology.orchestra.run.smithi077.stdout:} 2023-12-01T01:20:59.441 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.432+0000 7f542e196700 1 -- 172.21.15.77:0/805858460 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f54100a05a0 msgr2=0x7f54100a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:59.441 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.432+0000 7f542e196700 1 --2- 172.21.15.77:0/805858460 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f54100a05a0 0x7f54100a2a60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5428100cd0 tx=0x7f5418009260 comp rx=0 tx=0).stop 2023-12-01T01:20:59.441 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.432+0000 7f542e196700 1 -- 172.21.15.77:0/805858460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5428100e70 msgr2=0x7f5428102250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:20:59.441 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.432+0000 7f542e196700 1 --2- 172.21.15.77:0/805858460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5428100e70 0x7f5428102250 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f541c00e9a0 tx=0x7f541c00ecb0 comp rx=0 tx=0).stop 2023-12-01T01:20:59.441 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.433+0000 7f542e196700 1 -- 172.21.15.77:0/805858460 shutdown_connections 2023-12-01T01:20:59.441 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.433+0000 7f542e196700 1 --2- 172.21.15.77:0/805858460 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f54100a05a0 0x7f54100a2a60 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:59.441 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.433+0000 7f542e196700 1 --2- 172.21.15.77:0/805858460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5428101810 0x7f5428102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:59.441 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.433+0000 7f542e196700 1 --2- 172.21.15.77:0/805858460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f54280ffc80 0x7f5428103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:59.441 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.433+0000 7f542e196700 1 --2- 172.21.15.77:0/805858460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5428100e70 0x7f5428102250 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:20:59.441 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.433+0000 7f542e196700 1 -- 172.21.15.77:0/805858460 >> 172.21.15.77:0/805858460 conn(0x7f54280fb1f0 msgr2=0x7f54280fba40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:20:59.441 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.433+0000 7f542e196700 1 -- 172.21.15.77:0/805858460 shutdown_connections 2023-12-01T01:20:59.441 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:20:59.433+0000 7f542e196700 1 -- 172.21.15.77:0/805858460 wait complete. 2023-12-01T01:20:59.447 INFO:tasks.cephfs.filesystem:scrub status for tag:085290de-33b6-4527-ab7f-03f7dfd5dbba - {'path': '/', 'tag': '085290de-33b6-4527-ab7f-03f7dfd5dbba', 'options': 'recursive,force'} 2023-12-01T01:21:00.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:20:59 smithi203 ceph-mon[103581]: pgmap v391: 97 pgs: 97 active+clean; 53 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 3.5 MiB/s wr, 702 op/s 2023-12-01T01:21:00.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:20:59 smithi137 ceph-mon[103312]: pgmap v391: 97 pgs: 97 active+clean; 53 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 3.5 MiB/s wr, 702 op/s 2023-12-01T01:21:00.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:20:59 smithi077 ceph-mon[98867]: pgmap v391: 97 pgs: 97 active+clean; 53 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 3.5 MiB/s wr, 702 op/s 2023-12-01T01:21:01.375 INFO:tasks.workunit.client.0.smithi077.stdout:50000 trunc from 0x3a0a6d to 0x51d279 2023-12-01T01:21:01.375 INFO:tasks.workunit.client.0.smithi077.stdout:All 50000 operations completed A-OK! 2023-12-01T01:21:01.376 INFO:tasks.workunit.client.0.smithi077.stderr:+ ./fsx -z 100MB -N 50000 -p 10000 -l 104857600 2023-12-01T01:21:01.377 INFO:tasks.workunit.client.0.smithi077.stdout:Seed set to 1 2023-12-01T01:21:02.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:21:01 smithi203 ceph-mon[103581]: pgmap v392: 97 pgs: 97 active+clean; 52 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 3.9 MiB/s wr, 790 op/s 2023-12-01T01:21:02.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:21:01 smithi137 ceph-mon[103312]: pgmap v392: 97 pgs: 97 active+clean; 52 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 3.9 MiB/s wr, 790 op/s 2023-12-01T01:21:02.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:21:01 smithi077 ceph-mon[98867]: pgmap v392: 97 pgs: 97 active+clean; 52 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 3.9 MiB/s wr, 790 op/s 2023-12-01T01:21:02.587 INFO:tasks.workunit.client.0.smithi077.stderr:main: filesystem does not support fallocate mode 0, disabling! 2023-12-01T01:21:02.587 INFO:tasks.workunit.client.0.smithi077.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2023-12-01T01:21:02.590 INFO:tasks.workunit.client.0.smithi077.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2023-12-01T01:21:02.590 INFO:tasks.workunit.client.0.smithi077.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2023-12-01T01:21:02.590 INFO:tasks.workunit.client.0.smithi077.stderr:main: filesystem does not support clone range, disabling! 2023-12-01T01:21:02.595 INFO:tasks.workunit.client.0.smithi077.stderr:main: filesystem does not support dedupe range, disabling! 2023-12-01T01:21:02.598 INFO:tasks.workunit.client.0.smithi077.stdout:skipping zero size read 2023-12-01T01:21:02.598 INFO:tasks.workunit.client.0.smithi077.stdout:truncating to largest ever: 0x7e0023 2023-12-01T01:21:02.608 INFO:tasks.workunit.client.0.smithi077.stdout:copying to largest ever: 0xe91bf1 2023-12-01T01:21:02.609 INFO:tasks.workunit.client.0.smithi077.stdout:copying to largest ever: 0x5a6de73 2023-12-01T01:21:02.625 INFO:tasks.workunit.client.0.smithi077.stdout:truncating to largest ever: 0x5bdf572 2023-12-01T01:21:02.630 INFO:tasks.workunit.client.0.smithi077.stdout:truncating to largest ever: 0x602fcad 2023-12-01T01:21:02.751 INFO:tasks.workunit.client.0.smithi077.stdout:copying to largest ever: 0x623914d 2023-12-01T01:21:02.801 INFO:tasks.workunit.client.0.smithi077.stdout:copying to largest ever: 0x628f369 2023-12-01T01:21:02.839 INFO:tasks.workunit.client.0.smithi077.stdout:copying to largest ever: 0x6328dee 2023-12-01T01:21:03.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:21:02 smithi203 ceph-mon[103581]: pgmap v393: 97 pgs: 97 active+clean; 52 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 2.9 MiB/s wr, 564 op/s 2023-12-01T01:21:03.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:21:02 smithi137 ceph-mon[103312]: pgmap v393: 97 pgs: 97 active+clean; 52 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 2.9 MiB/s wr, 564 op/s 2023-12-01T01:21:03.042 INFO:tasks.workunit.client.0.smithi077.stdout:truncating to largest ever: 0x635b95a 2023-12-01T01:21:03.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:21:02 smithi077 ceph-mon[98867]: pgmap v393: 97 pgs: 97 active+clean; 52 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 2.9 MiB/s wr, 564 op/s 2023-12-01T01:21:03.286 INFO:tasks.workunit.client.0.smithi077.stdout:copying to largest ever: 0x637b84e 2023-12-01T01:21:03.374 INFO:tasks.workunit.client.0.smithi077.stdout:truncating to largest ever: 0x63b535e 2023-12-01T01:21:05.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:21:05 smithi077 ceph-mon[98867]: pgmap v394: 97 pgs: 97 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.6 MiB/s wr, 729 op/s 2023-12-01T01:21:05.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:21:05 smithi203 ceph-mon[103581]: pgmap v394: 97 pgs: 97 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.6 MiB/s wr, 729 op/s 2023-12-01T01:21:05.778 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:21:05 smithi137 ceph-mon[103312]: pgmap v394: 97 pgs: 97 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.6 MiB/s wr, 729 op/s 2023-12-01T01:21:08.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:21:07 smithi203 ceph-mon[103581]: pgmap v395: 97 pgs: 97 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 2.9 MiB/s wr, 636 op/s 2023-12-01T01:21:08.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:21:07 smithi137 ceph-mon[103312]: pgmap v395: 97 pgs: 97 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 2.9 MiB/s wr, 636 op/s 2023-12-01T01:21:08.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:21:07 smithi077 ceph-mon[98867]: pgmap v395: 97 pgs: 97 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 2.9 MiB/s wr, 636 op/s 2023-12-01T01:21:08.512 INFO:tasks.workunit.client.0.smithi077.stdout:copying to largest ever: 0x63d8bf1 2023-12-01T01:21:10.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:21:09 smithi203 ceph-mon[103581]: pgmap v396: 97 pgs: 97 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.7 MiB/s wr, 692 op/s 2023-12-01T01:21:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.023+0000 7fe05c461700 1 -- 172.21.15.77:0/3893258755 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0540ffdb0 msgr2=0x7fe054100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.023+0000 7fe05c461700 1 --2- 172.21.15.77:0/3893258755 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0540ffdb0 0x7fe054100230 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe0480098f0 tx=0x7fe048009c00 comp rx=0 tx=0).stop 2023-12-01T01:21:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.023+0000 7fe05c461700 1 -- 172.21.15.77:0/3893258755 shutdown_connections 2023-12-01T01:21:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.023+0000 7fe05c461700 1 --2- 172.21.15.77:0/3893258755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe054100770 0x7fe054105010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.023+0000 7fe05c461700 1 --2- 172.21.15.77:0/3893258755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0540ff050 0x7fe0540ff470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.023+0000 7fe05c461700 1 --2- 172.21.15.77:0/3893258755 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0540ffdb0 0x7fe054100230 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.023+0000 7fe05c461700 1 -- 172.21.15.77:0/3893258755 >> 172.21.15.77:0/3893258755 conn(0x7fe0540fa960 msgr2=0x7fe0540fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:21:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.023+0000 7fe05c461700 1 -- 172.21.15.77:0/3893258755 shutdown_connections 2023-12-01T01:21:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.023+0000 7fe05c461700 1 -- 172.21.15.77:0/3893258755 wait complete. 2023-12-01T01:21:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.024+0000 7fe05c461700 1 Processor -- start 2023-12-01T01:21:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.024+0000 7fe05c461700 1 -- start start 2023-12-01T01:21:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.024+0000 7fe05c461700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0540ff050 0x7fe05419d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.024+0000 7fe05c461700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0540ffdb0 0x7fe05419d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.024+0000 7fe05c461700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe054100770 0x7fe05419e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.024+0000 7fe05c461700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05419ec10 con 0x7fe054100770 2023-12-01T01:21:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.024+0000 7fe05c461700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05419ed80 con 0x7fe0540ffdb0 2023-12-01T01:21:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.024+0000 7fe05c461700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05419ef40 con 0x7fe0540ff050 2023-12-01T01:21:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.024+0000 7fe05a9fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe054100770 0x7fe05419e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.024+0000 7fe05a1fd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0540ff050 0x7fe05419d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.024+0000 7fe05a9fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe054100770 0x7fe05419e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60808/0 (socket says 172.21.15.77:60808) 2023-12-01T01:21:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.024+0000 7fe05a9fe700 1 -- 172.21.15.77:0/1345031502 learned_addr learned my addr 172.21.15.77:0/1345031502 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:21:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.024+0000 7fe0599fc700 1 --2- 172.21.15.77:0/1345031502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0540ffdb0 0x7fe05419d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.025+0000 7fe05a1fd700 1 -- 172.21.15.77:0/1345031502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0540ffdb0 msgr2=0x7fe05419d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.025+0000 7fe05a1fd700 1 --2- 172.21.15.77:0/1345031502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0540ffdb0 0x7fe05419d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.025+0000 7fe05a1fd700 1 -- 172.21.15.77:0/1345031502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe054100770 msgr2=0x7fe05419e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.025+0000 7fe05a1fd700 1 --2- 172.21.15.77:0/1345031502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe054100770 0x7fe05419e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.025+0000 7fe05a1fd700 1 -- 172.21.15.77:0/1345031502 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe048009520 con 0x7fe0540ff050 2023-12-01T01:21:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.025+0000 7fe05a9fe700 1 --2- 172.21.15.77:0/1345031502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe054100770 0x7fe05419e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:21:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.025+0000 7fe0599fc700 1 --2- 172.21.15.77:0/1345031502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0540ffdb0 0x7fe05419d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:21:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.025+0000 7fe05a1fd700 1 --2- 172.21.15.77:0/1345031502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0540ff050 0x7fe05419d1f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe05000e9f0 tx=0x7fe05000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:21:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.025+0000 7fe0477fe700 1 -- 172.21.15.77:0/1345031502 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05000c9c0 con 0x7fe0540ff050 2023-12-01T01:21:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.025+0000 7fe05c461700 1 -- 172.21.15.77:0/1345031502 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe05419f220 con 0x7fe0540ff050 2023-12-01T01:21:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.025+0000 7fe0477fe700 1 -- 172.21.15.77:0/1345031502 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe050010430 con 0x7fe0540ff050 2023-12-01T01:21:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.025+0000 7fe05c461700 1 -- 172.21.15.77:0/1345031502 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0541a59e0 con 0x7fe0540ff050 2023-12-01T01:21:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.025+0000 7fe0477fe700 1 -- 172.21.15.77:0/1345031502 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe050018680 con 0x7fe0540ff050 2023-12-01T01:21:10.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:21:09 smithi137 ceph-mon[103312]: pgmap v396: 97 pgs: 97 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.7 MiB/s wr, 692 op/s 2023-12-01T01:21:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.028+0000 7fe0477fe700 1 -- 172.21.15.77:0/1345031502 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fe050010810 con 0x7fe0540ff050 2023-12-01T01:21:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.028+0000 7fe0477fe700 1 --2- 172.21.15.77:0/1345031502 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fe0400a0520 0x7fe0400a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.029+0000 7fe0599fc700 1 --2- 172.21.15.77:0/1345031502 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fe0400a0520 0x7fe0400a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.029+0000 7fe0477fe700 1 -- 172.21.15.77:0/1345031502 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fe050014070 con 0x7fe0540ff050 2023-12-01T01:21:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.029+0000 7fe0599fc700 1 --2- 172.21.15.77:0/1345031502 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fe0400a0520 0x7fe0400a29e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe048005e90 tx=0x7fe048019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:21:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.029+0000 7fe0685f1b80 1 Processor -- start 2023-12-01T01:21:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.029+0000 7fe0685f1b80 1 -- start start 2023-12-01T01:21:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.030+0000 7fe0685f1b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56228d5a8680 0x56228d5a8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.030+0000 7fe0685f1b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56228d5a8fe0 0x56228d5a98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.030+0000 7fe0685f1b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56228d5a9ed0 0x56228d5aa350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.030+0000 7fe0685f1b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228d5009b0 con 0x56228d5a9ed0 2023-12-01T01:21:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.030+0000 7fe0685f1b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228d4fe370 con 0x56228d5a8680 2023-12-01T01:21:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.030+0000 7fe0685f1b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228d4f9c60 con 0x56228d5a8fe0 2023-12-01T01:21:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.030+0000 7fe05a1fd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56228d5a8680 0x56228d5a8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.030+0000 7fe05a1fd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56228d5a8680 0x56228d5a8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:39734/0 (socket says 172.21.15.77:39734) 2023-12-01T01:21:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.030+0000 7fe05a1fd700 1 -- 172.21.15.77:0/4087842600 learned_addr learned my addr 172.21.15.77:0/4087842600 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:21:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.031+0000 7fe05a9fe700 1 --2- 172.21.15.77:0/4087842600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56228d5a9ed0 0x56228d5aa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.031+0000 7fe05a1fd700 1 -- 172.21.15.77:0/4087842600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56228d5a8fe0 msgr2=0x56228d5a98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.031+0000 7fe05a1fd700 1 --2- 172.21.15.77:0/4087842600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56228d5a8fe0 0x56228d5a98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.031+0000 7fe05a1fd700 1 -- 172.21.15.77:0/4087842600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56228d5a9ed0 msgr2=0x56228d5aa350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.031+0000 7fe05a1fd700 1 --2- 172.21.15.77:0/4087842600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56228d5a9ed0 0x56228d5aa350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.031+0000 7fe05a1fd700 1 -- 172.21.15.77:0/4087842600 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56228d327d80 con 0x56228d5a8680 2023-12-01T01:21:10.033 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.031+0000 7fe05a1fd700 1 --2- 172.21.15.77:0/4087842600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56228d5a8680 0x56228d5a8aa0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe050010b60 tx=0x7fe050098f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7aaf9256cea4983d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:21:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.031+0000 7fe044da9700 1 -- 172.21.15.77:0/4087842600 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0500992f0 con 0x56228d5a8680 2023-12-01T01:21:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.031+0000 7fe044da9700 1 -- 172.21.15.77:0/4087842600 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe05009d120 con 0x56228d5a8680 2023-12-01T01:21:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.031+0000 7fe044da9700 1 -- 172.21.15.77:0/4087842600 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe050098a40 con 0x56228d5a8680 2023-12-01T01:21:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.032+0000 7fe0685f1b80 1 -- 172.21.15.77:0/4087842600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56228d5a8680 msgr2=0x56228d5a8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.032+0000 7fe0685f1b80 1 --2- 172.21.15.77:0/4087842600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56228d5a8680 0x56228d5a8aa0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe050010b60 tx=0x7fe050098f00 comp rx=0 tx=0).stop 2023-12-01T01:21:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.032+0000 7fe0685f1b80 1 -- 172.21.15.77:0/4087842600 shutdown_connections 2023-12-01T01:21:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.032+0000 7fe0685f1b80 1 --2- 172.21.15.77:0/4087842600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56228d5a9ed0 0x56228d5aa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.032+0000 7fe0685f1b80 1 --2- 172.21.15.77:0/4087842600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56228d5a8680 0x56228d5a8aa0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.032+0000 7fe0685f1b80 1 --2- 172.21.15.77:0/4087842600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56228d5a8fe0 0x56228d5a98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.032+0000 7fe0685f1b80 1 -- 172.21.15.77:0/4087842600 >> 172.21.15.77:0/4087842600 conn(0x56228d508070 msgr2=0x56228d5a5040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:21:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.032+0000 7fe0685f1b80 1 -- 172.21.15.77:0/4087842600 shutdown_connections 2023-12-01T01:21:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.032+0000 7fe0685f1b80 1 -- 172.21.15.77:0/4087842600 wait complete. 2023-12-01T01:21:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.033+0000 7fe0685f1b80 1 -- start start 2023-12-01T01:21:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.033+0000 7fe0685f1b80 1 Processor -- start 2023-12-01T01:21:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.033+0000 7fe0685f1b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56228d5a8680 0x56228d5c3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.033+0000 7fe0685f1b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56228d5a8fe0 0x56228d5c3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.033+0000 7fe0685f1b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56228d5a9ed0 0x56228d5caab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.033+0000 7fe0685f1b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228d4964b0 con 0x56228d5a9ed0 2023-12-01T01:21:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.033+0000 7fe0685f1b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228d496150 con 0x56228d5a8fe0 2023-12-01T01:21:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.033+0000 7fe0685f1b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228d37d8c0 con 0x56228d5a8680 2023-12-01T01:21:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.034+0000 7fe0685f1b80 10 client.0 _ll_register_callbacks cb 0x56228d3e3a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:21:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.034+0000 7fe05a9fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56228d5a9ed0 0x56228d5caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.034+0000 7fe05a1fd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56228d5a8680 0x56228d5c3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.034+0000 7fe05a9fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56228d5a9ed0 0x56228d5caab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60820/0 (socket says 172.21.15.77:60820) 2023-12-01T01:21:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.034+0000 7fe05a1fd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56228d5a8680 0x56228d5c3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:47006/0 (socket says 172.21.15.77:47006) 2023-12-01T01:21:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.034+0000 7fe05a9fe700 1 -- 172.21.15.77:0/3825045635 learned_addr learned my addr 172.21.15.77:0/3825045635 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:21:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.034+0000 7fe0599fc700 1 --2- 172.21.15.77:0/3825045635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56228d5a8fe0 0x56228d5c3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.034+0000 7fe05a9fe700 1 -- 172.21.15.77:0/3825045635 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56228d5a8680 msgr2=0x56228d5c3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.034+0000 7fe05a9fe700 1 --2- 172.21.15.77:0/3825045635 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56228d5a8680 0x56228d5c3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.034+0000 7fe05a9fe700 1 -- 172.21.15.77:0/3825045635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56228d5a8fe0 msgr2=0x56228d5c3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.034+0000 7fe05a9fe700 1 --2- 172.21.15.77:0/3825045635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56228d5a8fe0 0x56228d5c3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.034+0000 7fe05a9fe700 1 -- 172.21.15.77:0/3825045635 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0541a59e0 con 0x56228d5a9ed0 2023-12-01T01:21:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.034+0000 7fe0599fc700 1 --2- 172.21.15.77:0/3825045635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56228d5a8fe0 0x56228d5c3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:21:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.034+0000 7fe05a9fe700 1 --2- 172.21.15.77:0/3825045635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56228d5a9ed0 0x56228d5caab0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe04c007b20 tx=0x7fe04c007b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e3bccd3f7379ff server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:21:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.034+0000 7fe03dffb700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:21:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.034+0000 7fe03dffb700 1 -- 172.21.15.77:0/3825045635 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04c00e020 con 0x56228d5a9ed0 2023-12-01T01:21:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.034+0000 7fe0685f1b80 1 -- 172.21.15.77:0/3825045635 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56228d524cb0 con 0x56228d5a9ed0 2023-12-01T01:21:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.035+0000 7fe03dffb700 1 -- 172.21.15.77:0/3825045635 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fe04c004bb0 con 0x56228d5a9ed0 2023-12-01T01:21:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.035+0000 7fe03dffb700 1 -- 172.21.15.77:0/3825045635 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe04c004d70 con 0x56228d5a9ed0 2023-12-01T01:21:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.035+0000 7fe0685f1b80 10 client.16101 fetch_fsmap learned FSMap version 31 2023-12-01T01:21:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.035+0000 7fe0685f1b80 1 -- 172.21.15.77:0/3825045635 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56228d310c00 con 0x56228d5a9ed0 2023-12-01T01:21:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.035+0000 7fe03dffb700 1 -- 172.21.15.77:0/3825045635 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56228d524cb0 con 0x56228d5a9ed0 2023-12-01T01:21:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.035+0000 7fe03dffb700 1 -- 172.21.15.77:0/3825045635 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe04c016ac0 con 0x56228d5a9ed0 2023-12-01T01:21:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.035+0000 7fe0685f1b80 10 client.16101 fetch_fsmap finished waiting for FSMap version 31 2023-12-01T01:21:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.035+0000 7fe0685f1b80 10 client.16101 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-01T01:21:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.035+0000 7fe0685f1b80 20 client.16101 populate_metadata read hostname 'smithi077' 2023-12-01T01:21:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.035+0000 7fe0685f1b80 1 --2- 172.21.15.77:0/3825045635 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x56228d5cf470 0x56228d5d1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.035+0000 7fe0685f1b80 4 client.16101 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:21:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.035+0000 7fe0685f1b80 1 -- 172.21.15.77:0/3825045635 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56228d577db0 con 0x56228d5cf470 2023-12-01T01:21:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.035+0000 7fe05a1fd700 1 --2- 172.21.15.77:0/3825045635 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x56228d5cf470 0x56228d5d1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.036+0000 7fe03dffb700 10 client.16101 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:21:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.036+0000 7fe05a1fd700 1 --2- 172.21.15.77:0/3825045635 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x56228d5cf470 0x56228d5d1890 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ec086da08aa42dd server_cookie=e9c594f12e582297 in_seq=0 out_seq=0 2023-12-01T01:21:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.036+0000 7fe03dffb700 1 -- 172.21.15.77:0/3825045635 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56228d577db0 con 0x56228d5cf470 2023-12-01T01:21:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.036+0000 7fe03dffb700 10 client.16101 handle_command_reply: tid=0 2023-12-01T01:21:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.036+0000 7fe0685f1b80 1 client.16101 shutdown 2023-12-01T01:21:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.036+0000 7fe0685f1b80 1 -- 172.21.15.77:0/3825045635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56228d5a9ed0 msgr2=0x56228d5caab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.036+0000 7fe0685f1b80 1 --2- 172.21.15.77:0/3825045635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56228d5a9ed0 0x56228d5caab0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe04c007b20 tx=0x7fe04c007b50 comp rx=0 tx=0).stop 2023-12-01T01:21:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.036+0000 7fe0685f1b80 1 -- 172.21.15.77:0/3825045635 shutdown_connections 2023-12-01T01:21:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.036+0000 7fe0685f1b80 1 --2- 172.21.15.77:0/3825045635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56228d5a9ed0 0x56228d5caab0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.036+0000 7fe0685f1b80 1 --2- 172.21.15.77:0/3825045635 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56228d5a8680 0x56228d5c3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.036+0000 7fe0685f1b80 1 --2- 172.21.15.77:0/3825045635 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x56228d5cf470 0x56228d5d1890 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.037+0000 7fe0685f1b80 1 --2- 172.21.15.77:0/3825045635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56228d5a8fe0 0x56228d5c3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.037+0000 7fe03dffb700 0 client.16101 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:21:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.037+0000 7fe0685f1b80 1 -- 172.21.15.77:0/3825045635 >> 172.21.15.77:0/3825045635 conn(0x56228d508070 msgr2=0x56228d5a33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:21:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.037+0000 7fe0685f1b80 1 -- 172.21.15.77:0/3825045635 shutdown_connections 2023-12-01T01:21:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.037+0000 7fe0685f1b80 1 -- 172.21.15.77:0/3825045635 wait complete. 2023-12-01T01:21:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.037+0000 7fe0685f1b80 20 client.16101 trim_cache size 0 max 16384 2023-12-01T01:21:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.054+0000 7fe0685f1b80 1 Processor -- start 2023-12-01T01:21:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.054+0000 7fe0685f1b80 1 -- start start 2023-12-01T01:21:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.054+0000 7fe0685f1b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56228d5a8680 0x56228d5c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.054+0000 7fe0685f1b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56228d5a8fe0 0x56228d5ad0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.054+0000 7fe0685f1b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56228d5a9ed0 0x56228d5a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.054+0000 7fe0685f1b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228d5c2a10 con 0x56228d5a9ed0 2023-12-01T01:21:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.054+0000 7fe0685f1b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228d59e530 con 0x56228d5a8fe0 2023-12-01T01:21:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.054+0000 7fe0685f1b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228d5cc720 con 0x56228d5a8680 2023-12-01T01:21:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.054+0000 7fe05a9fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56228d5a9ed0 0x56228d5a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.055+0000 7fe05a9fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56228d5a9ed0 0x56228d5a4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60830/0 (socket says 172.21.15.77:60830) 2023-12-01T01:21:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.055+0000 7fe05a9fe700 1 -- 172.21.15.77:0/2529874250 learned_addr learned my addr 172.21.15.77:0/2529874250 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:21:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.055+0000 7fe05a1fd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56228d5a8680 0x56228d5c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.055+0000 7fe0599fc700 1 --2- 172.21.15.77:0/2529874250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56228d5a8fe0 0x56228d5ad0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.055+0000 7fe05a9fe700 1 -- 172.21.15.77:0/2529874250 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56228d5a8680 msgr2=0x56228d5c89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.055+0000 7fe05a9fe700 1 --2- 172.21.15.77:0/2529874250 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56228d5a8680 0x56228d5c89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.055+0000 7fe05a9fe700 1 -- 172.21.15.77:0/2529874250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56228d5a8fe0 msgr2=0x56228d5ad0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.055+0000 7fe05a9fe700 1 --2- 172.21.15.77:0/2529874250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56228d5a8fe0 0x56228d5ad0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.055+0000 7fe05a9fe700 1 -- 172.21.15.77:0/2529874250 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56228d34e1f0 con 0x56228d5a9ed0 2023-12-01T01:21:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.055+0000 7fe05a9fe700 1 --2- 172.21.15.77:0/2529874250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56228d5a9ed0 0x56228d5a4c20 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe04c016e50 tx=0x7fe04c009bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36bc9ad855c24787 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:21:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.055+0000 7fe0455aa700 1 -- 172.21.15.77:0/2529874250 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04c007ea0 con 0x56228d5a9ed0 2023-12-01T01:21:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.055+0000 7fe0455aa700 1 -- 172.21.15.77:0/2529874250 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe04c00f040 con 0x56228d5a9ed0 2023-12-01T01:21:10.058 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.055+0000 7fe0455aa700 1 -- 172.21.15.77:0/2529874250 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04c015850 con 0x56228d5a9ed0 2023-12-01T01:21:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.056+0000 7fe0685f1b80 1 -- 172.21.15.77:0/2529874250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56228d5a9ed0 msgr2=0x56228d5a4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.056+0000 7fe0685f1b80 1 --2- 172.21.15.77:0/2529874250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56228d5a9ed0 0x56228d5a4c20 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe04c016e50 tx=0x7fe04c009bc0 comp rx=0 tx=0).stop 2023-12-01T01:21:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.056+0000 7fe0685f1b80 1 -- 172.21.15.77:0/2529874250 shutdown_connections 2023-12-01T01:21:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.056+0000 7fe0685f1b80 1 --2- 172.21.15.77:0/2529874250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56228d5a9ed0 0x56228d5a4c20 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.056+0000 7fe0685f1b80 1 --2- 172.21.15.77:0/2529874250 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56228d5a8680 0x56228d5c89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.056+0000 7fe0685f1b80 1 --2- 172.21.15.77:0/2529874250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56228d5a8fe0 0x56228d5ad0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.056+0000 7fe0685f1b80 1 -- 172.21.15.77:0/2529874250 >> 172.21.15.77:0/2529874250 conn(0x56228d508070 msgr2=0x56228d5c21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:21:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.056+0000 7fe0685f1b80 1 -- 172.21.15.77:0/2529874250 shutdown_connections 2023-12-01T01:21:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.056+0000 7fe0685f1b80 1 -- 172.21.15.77:0/2529874250 wait complete. 2023-12-01T01:21:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.056+0000 7fe0685f1b80 1 -- start start 2023-12-01T01:21:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.056+0000 7fe0685f1b80 1 Processor -- start 2023-12-01T01:21:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.057+0000 7fe0685f1b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56228d5a8680 0x56228d61af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.057+0000 7fe0685f1b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56228d5a8fe0 0x56228d61d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.057+0000 7fe0685f1b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56228d5a9ed0 0x56228d61faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.057+0000 7fe0685f1b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228d620230 con 0x56228d5a8680 2023-12-01T01:21:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.057+0000 7fe0685f1b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228d6203a0 con 0x56228d5a9ed0 2023-12-01T01:21:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.057+0000 7fe0685f1b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56228d620560 con 0x56228d5a8fe0 2023-12-01T01:21:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.057+0000 7fe0685f1b80 10 client.0 _ll_register_callbacks cb 0x7fe04c009540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:21:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.057+0000 7fe05a1fd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56228d5a8680 0x56228d61af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.057+0000 7fe05a1fd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56228d5a8680 0x56228d61af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60832/0 (socket says 172.21.15.77:60832) 2023-12-01T01:21:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.057+0000 7fe05a1fd700 1 -- 172.21.15.77:0/2685505853 learned_addr learned my addr 172.21.15.77:0/2685505853 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:21:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.057+0000 7fe0599fc700 1 --2- 172.21.15.77:0/2685505853 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56228d5a8fe0 0x56228d61d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.057+0000 7fe05a9fe700 1 --2- 172.21.15.77:0/2685505853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56228d5a9ed0 0x56228d61faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.057+0000 7fe05a1fd700 1 -- 172.21.15.77:0/2685505853 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56228d5a8fe0 msgr2=0x56228d61d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.057+0000 7fe05a1fd700 1 --2- 172.21.15.77:0/2685505853 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56228d5a8fe0 0x56228d61d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.057+0000 7fe05a1fd700 1 -- 172.21.15.77:0/2685505853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56228d5a9ed0 msgr2=0x56228d61faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.057+0000 7fe05a1fd700 1 --2- 172.21.15.77:0/2685505853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56228d5a9ed0 0x56228d61faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.057+0000 7fe05a1fd700 1 -- 172.21.15.77:0/2685505853 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56228d310c00 con 0x56228d5a8680 2023-12-01T01:21:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.057+0000 7fe05a1fd700 1 --2- 172.21.15.77:0/2685505853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56228d5a8680 0x56228d61af10 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe050095d60 tx=0x7fe050098d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54682be934968b2d server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:21:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.057+0000 7fe03dffb700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:21:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.057+0000 7fe03dffb700 1 -- 172.21.15.77:0/2685505853 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe050097700 con 0x56228d5a8680 2023-12-01T01:21:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.057+0000 7fe03dffb700 1 -- 172.21.15.77:0/2685505853 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fe0500987e0 con 0x56228d5a8680 2023-12-01T01:21:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.057+0000 7fe0685f1b80 1 -- 172.21.15.77:0/2685505853 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe05009d120 con 0x56228d5a8680 2023-12-01T01:21:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.058+0000 7fe03dffb700 1 -- 172.21.15.77:0/2685505853 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0500989a0 con 0x56228d5a8680 2023-12-01T01:21:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.058+0000 7fe0685f1b80 10 client.16107 fetch_fsmap learned FSMap version 31 2023-12-01T01:21:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.058+0000 7fe0685f1b80 1 -- 172.21.15.77:0/2685505853 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56228d326250 con 0x56228d5a8680 2023-12-01T01:21:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.058+0000 7fe03dffb700 1 -- 172.21.15.77:0/2685505853 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0500b7020 con 0x56228d5a8680 2023-12-01T01:21:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.058+0000 7fe03dffb700 1 -- 172.21.15.77:0/2685505853 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe05000cb20 con 0x56228d5a8680 2023-12-01T01:21:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.058+0000 7fe0685f1b80 10 client.16107 fetch_fsmap finished waiting for FSMap version 31 2023-12-01T01:21:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.058+0000 7fe0685f1b80 10 client.16107 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-01T01:21:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.058+0000 7fe0685f1b80 20 client.16107 populate_metadata read hostname 'smithi077' 2023-12-01T01:21:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.058+0000 7fe0685f1b80 1 --2- 172.21.15.77:0/2685505853 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x56228d621da0 0x56228d6241c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.058+0000 7fe0685f1b80 4 client.16107 mds_command: new command op to 24500 tid=0[{"prefix": "flush journal"}] 2023-12-01T01:21:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.058+0000 7fe0685f1b80 1 -- 172.21.15.77:0/2685505853 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56228d4f2ce0 con 0x56228d621da0 2023-12-01T01:21:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.059+0000 7fe0599fc700 1 --2- 172.21.15.77:0/2685505853 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x56228d621da0 0x56228d6241c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.061+0000 7fe0599fc700 1 --2- 172.21.15.77:0/2685505853 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x56228d621da0 0x56228d6241c0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c6ccc5e84092647 server_cookie=856a06d152253fce in_seq=0 out_seq=0 2023-12-01T01:21:10.065 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.061+0000 7fe03dffb700 10 client.16107 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:21:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.083+0000 7fe03dffb700 1 -- 172.21.15.77:0/2685505853 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56228d4f2ce0 con 0x56228d621da0 2023-12-01T01:21:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.083+0000 7fe03dffb700 10 client.16107 handle_command_reply: tid=0 2023-12-01T01:21:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.083+0000 7fe0685f1b80 1 client.16107 shutdown 2023-12-01T01:21:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.083+0000 7fe0685f1b80 1 -- 172.21.15.77:0/2685505853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56228d5a8680 msgr2=0x56228d61af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.083+0000 7fe0685f1b80 1 --2- 172.21.15.77:0/2685505853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56228d5a8680 0x56228d61af10 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe050095d60 tx=0x7fe050098d80 comp rx=0 tx=0).stop 2023-12-01T01:21:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.083+0000 7fe0685f1b80 1 -- 172.21.15.77:0/2685505853 shutdown_connections 2023-12-01T01:21:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.083+0000 7fe0685f1b80 1 --2- 172.21.15.77:0/2685505853 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x56228d621da0 0x56228d6241c0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.083+0000 7fe0685f1b80 1 --2- 172.21.15.77:0/2685505853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56228d5a9ed0 0x56228d61faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.083+0000 7fe0685f1b80 1 --2- 172.21.15.77:0/2685505853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56228d5a8680 0x56228d61af10 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.083+0000 7fe03dffb700 0 client.16107 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:21:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.083+0000 7fe0685f1b80 1 --2- 172.21.15.77:0/2685505853 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56228d5a8fe0 0x56228d61d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.083+0000 7fe0685f1b80 1 -- 172.21.15.77:0/2685505853 >> 172.21.15.77:0/2685505853 conn(0x56228d508070 msgr2=0x56228d5ad0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:21:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.083+0000 7fe0685f1b80 1 -- 172.21.15.77:0/2685505853 shutdown_connections 2023-12-01T01:21:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.083+0000 7fe0685f1b80 1 -- 172.21.15.77:0/2685505853 wait complete. 2023-12-01T01:21:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.083+0000 7fe0685f1b80 20 client.16107 trim_cache size 0 max 16384 2023-12-01T01:21:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout:{ 2023-12-01T01:21:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout: "message": "", 2023-12-01T01:21:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout: "return_code": 0 2023-12-01T01:21:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout:} 2023-12-01T01:21:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.084+0000 7fe05c461700 1 -- 172.21.15.77:0/1345031502 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fe0400a0520 msgr2=0x7fe0400a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.084+0000 7fe05c461700 1 --2- 172.21.15.77:0/1345031502 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fe0400a0520 0x7fe0400a29e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe048005e90 tx=0x7fe048019040 comp rx=0 tx=0).stop 2023-12-01T01:21:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.084+0000 7fe05c461700 1 -- 172.21.15.77:0/1345031502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0540ff050 msgr2=0x7fe05419d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.084+0000 7fe05c461700 1 --2- 172.21.15.77:0/1345031502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0540ff050 0x7fe05419d1f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe05000e9f0 tx=0x7fe05000ed00 comp rx=0 tx=0).stop 2023-12-01T01:21:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.084+0000 7fe05c461700 1 -- 172.21.15.77:0/1345031502 shutdown_connections 2023-12-01T01:21:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.084+0000 7fe05c461700 1 --2- 172.21.15.77:0/1345031502 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fe0400a0520 0x7fe0400a29e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.084+0000 7fe05c461700 1 --2- 172.21.15.77:0/1345031502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe054100770 0x7fe05419e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.084+0000 7fe05c461700 1 --2- 172.21.15.77:0/1345031502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0540ff050 0x7fe05419d1f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.084+0000 7fe05c461700 1 --2- 172.21.15.77:0/1345031502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0540ffdb0 0x7fe05419d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.084+0000 7fe05c461700 1 -- 172.21.15.77:0/1345031502 >> 172.21.15.77:0/1345031502 conn(0x7fe0540fa960 msgr2=0x7fe0540fb5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:21:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.084+0000 7fe05c461700 1 -- 172.21.15.77:0/1345031502 shutdown_connections 2023-12-01T01:21:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:10.084+0000 7fe05c461700 1 -- 172.21.15.77:0/1345031502 wait complete. 2023-12-01T01:21:10.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:21:09 smithi077 ceph-mon[98867]: pgmap v396: 97 pgs: 97 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.7 MiB/s wr, 692 op/s 2023-12-01T01:21:10.216 INFO:tasks.workunit.client.0.smithi077.stdout:truncating to largest ever: 0x63e65d6 2023-12-01T01:21:11.960 INFO:tasks.workunit.client.0.smithi077.stdout:truncating to largest ever: 0x63ea964 2023-12-01T01:21:12.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:21:11 smithi203 ceph-mon[103581]: pgmap v397: 97 pgs: 97 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.8 MiB/s wr, 904 op/s 2023-12-01T01:21:12.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:21:11 smithi137 ceph-mon[103312]: pgmap v397: 97 pgs: 97 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.8 MiB/s wr, 904 op/s 2023-12-01T01:21:12.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:21:11 smithi077 ceph-mon[98867]: pgmap v397: 97 pgs: 97 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.8 MiB/s wr, 904 op/s 2023-12-01T01:21:14.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:21:13 smithi203 ceph-mon[103581]: pgmap v398: 97 pgs: 97 active+clean; 62 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.2 MiB/s wr, 842 op/s 2023-12-01T01:21:14.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:21:13 smithi137 ceph-mon[103312]: pgmap v398: 97 pgs: 97 active+clean; 62 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.2 MiB/s wr, 842 op/s 2023-12-01T01:21:14.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:21:13 smithi077 ceph-mon[98867]: pgmap v398: 97 pgs: 97 active+clean; 62 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.2 MiB/s wr, 842 op/s 2023-12-01T01:21:16.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:21:15 smithi203 ceph-mon[103581]: pgmap v399: 97 pgs: 97 active+clean; 54 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.9 MiB/s wr, 1.16k op/s 2023-12-01T01:21:16.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:21:15 smithi137 ceph-mon[103312]: pgmap v399: 97 pgs: 97 active+clean; 54 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.9 MiB/s wr, 1.16k op/s 2023-12-01T01:21:16.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:21:15 smithi077 ceph-mon[98867]: pgmap v399: 97 pgs: 97 active+clean; 54 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.9 MiB/s wr, 1.16k op/s 2023-12-01T01:21:18.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:21:17 smithi203 ceph-mon[103581]: pgmap v400: 97 pgs: 97 active+clean; 56 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.5 MiB/s wr, 1.15k op/s 2023-12-01T01:21:18.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:21:17 smithi137 ceph-mon[103312]: pgmap v400: 97 pgs: 97 active+clean; 56 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.5 MiB/s wr, 1.15k op/s 2023-12-01T01:21:18.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:21:17 smithi077 ceph-mon[98867]: pgmap v400: 97 pgs: 97 active+clean; 56 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.5 MiB/s wr, 1.15k op/s 2023-12-01T01:21:20.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:21:19 smithi203 ceph-mon[103581]: pgmap v401: 97 pgs: 97 active+clean; 53 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.9 MiB/s wr, 1.33k op/s 2023-12-01T01:21:20.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:21:19 smithi137 ceph-mon[103312]: pgmap v401: 97 pgs: 97 active+clean; 53 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.9 MiB/s wr, 1.33k op/s 2023-12-01T01:21:20.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:21:19 smithi077 ceph-mon[98867]: pgmap v401: 97 pgs: 97 active+clean; 53 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.9 MiB/s wr, 1.33k op/s 2023-12-01T01:21:22.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:21:21 smithi203 ceph-mon[103581]: pgmap v402: 97 pgs: 97 active+clean; 58 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 3.1 MiB/s wr, 1.39k op/s 2023-12-01T01:21:22.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:21:21 smithi137 ceph-mon[103312]: pgmap v402: 97 pgs: 97 active+clean; 58 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 3.1 MiB/s wr, 1.39k op/s 2023-12-01T01:21:22.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:21:21 smithi077 ceph-mon[98867]: pgmap v402: 97 pgs: 97 active+clean; 58 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 3.1 MiB/s wr, 1.39k op/s 2023-12-01T01:21:22.578 INFO:tasks.workunit.client.0.smithi077.stdout:copying to largest ever: 0x63fd4d5 2023-12-01T01:21:23.223 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.222+0000 7f11def4d700 1 -- 172.21.15.77:0/1266449857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11d80ffca0 msgr2=0x7f11d81000c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:23.224 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.222+0000 7f11def4d700 1 --2- 172.21.15.77:0/1266449857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11d80ffca0 0x7f11d81000c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f11cc0099c0 tx=0x7f11cc009cd0 comp rx=0 tx=0).stop 2023-12-01T01:21:23.224 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.222+0000 7f11def4d700 1 -- 172.21.15.77:0/1266449857 shutdown_connections 2023-12-01T01:21:23.224 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.222+0000 7f11def4d700 1 --2- 172.21.15.77:0/1266449857 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11d8101830 0x7f11d81080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:23.224 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.222+0000 7f11def4d700 1 --2- 172.21.15.77:0/1266449857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11d80ffca0 0x7f11d81000c0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:23.224 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.222+0000 7f11def4d700 1 --2- 172.21.15.77:0/1266449857 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11d8100e90 0x7f11d81012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:23.224 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.222+0000 7f11def4d700 1 -- 172.21.15.77:0/1266449857 >> 172.21.15.77:0/1266449857 conn(0x7f11d80fb230 msgr2=0x7f11d80fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:21:23.224 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.222+0000 7f11def4d700 1 -- 172.21.15.77:0/1266449857 shutdown_connections 2023-12-01T01:21:23.224 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.222+0000 7f11def4d700 1 -- 172.21.15.77:0/1266449857 wait complete. 2023-12-01T01:21:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.223+0000 7f11def4d700 1 Processor -- start 2023-12-01T01:21:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.223+0000 7f11def4d700 1 -- start start 2023-12-01T01:21:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.223+0000 7f11def4d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11d80ffca0 0x7f11d806f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.223+0000 7f11def4d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11d8100e90 0x7f11d806fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.223+0000 7f11def4d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11d8101830 0x7f11d8070b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.223+0000 7f11def4d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d8071180 con 0x7f11d8101830 2023-12-01T01:21:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.223+0000 7f11def4d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d80712f0 con 0x7f11d80ffca0 2023-12-01T01:21:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.223+0000 7f11def4d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d80714b0 con 0x7f11d8100e90 2023-12-01T01:21:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.223+0000 7f11dd4ea700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11d8101830 0x7f11d8070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.223+0000 7f11d7fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11d8100e90 0x7f11d806fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.223+0000 7f11dd4ea700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11d8101830 0x7f11d8070b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60850/0 (socket says 172.21.15.77:60850) 2023-12-01T01:21:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.223+0000 7f11d7fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11d8100e90 0x7f11d806fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:47036/0 (socket says 172.21.15.77:47036) 2023-12-01T01:21:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.223+0000 7f11dd4ea700 1 -- 172.21.15.77:0/1425637627 learned_addr learned my addr 172.21.15.77:0/1425637627 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:21:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.223+0000 7f11dcce9700 1 --2- 172.21.15.77:0/1425637627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11d80ffca0 0x7f11d806f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.223+0000 7f11dd4ea700 1 -- 172.21.15.77:0/1425637627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11d8100e90 msgr2=0x7f11d806fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.223+0000 7f11dd4ea700 1 --2- 172.21.15.77:0/1425637627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11d8100e90 0x7f11d806fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.223+0000 7f11dd4ea700 1 -- 172.21.15.77:0/1425637627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11d80ffca0 msgr2=0x7f11d806f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.223+0000 7f11dd4ea700 1 --2- 172.21.15.77:0/1425637627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11d80ffca0 0x7f11d806f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.223+0000 7f11dd4ea700 1 -- 172.21.15.77:0/1425637627 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11cc0095f0 con 0x7f11d8101830 2023-12-01T01:21:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.224+0000 7f11dd4ea700 1 --2- 172.21.15.77:0/1425637627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11d8101830 0x7f11d8070b20 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f11d000ef30 tx=0x7f11d000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:21:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.224+0000 7f11d5ffb700 1 -- 172.21.15.77:0/1425637627 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d0004bb0 con 0x7f11d8101830 2023-12-01T01:21:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.224+0000 7f11def4d700 1 -- 172.21.15.77:0/1425637627 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11d8071790 con 0x7f11d8101830 2023-12-01T01:21:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.224+0000 7f11d5ffb700 1 -- 172.21.15.77:0/1425637627 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11d000f040 con 0x7f11d8101830 2023-12-01T01:21:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.224+0000 7f11d5ffb700 1 -- 172.21.15.77:0/1425637627 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d0004020 con 0x7f11d8101830 2023-12-01T01:21:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.224+0000 7f11def4d700 1 -- 172.21.15.77:0/1425637627 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11d8069230 con 0x7f11d8101830 2023-12-01T01:21:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.227+0000 7f11d5ffb700 1 -- 172.21.15.77:0/1425637627 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f11d0003a00 con 0x7f11d8101830 2023-12-01T01:21:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.227+0000 7f11d5ffb700 1 --2- 172.21.15.77:0/1425637627 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f11c80a0520 0x7f11c80a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.228+0000 7f11d5ffb700 1 -- 172.21.15.77:0/1425637627 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f11d0016070 con 0x7f11d8101830 2023-12-01T01:21:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.228+0000 7f11dcce9700 1 --2- 172.21.15.77:0/1425637627 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f11c80a0520 0x7f11c80a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.228+0000 7f11eb0ddb80 1 Processor -- start 2023-12-01T01:21:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.228+0000 7f11dcce9700 1 --2- 172.21.15.77:0/1425637627 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f11c80a0520 0x7f11c80a29e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f11cc009990 tx=0x7f11cc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:21:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.228+0000 7f11eb0ddb80 1 -- start start 2023-12-01T01:21:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.230+0000 7f11eb0ddb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f4667f680 0x563f4667faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.230+0000 7f11eb0ddb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f4667ffe0 0x563f466808f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.230+0000 7f11eb0ddb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563f46680ed0 0x563f46681350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.230+0000 7f11eb0ddb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f465d79b0 con 0x563f4667f680 2023-12-01T01:21:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.230+0000 7f11eb0ddb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f465d5370 con 0x563f4667ffe0 2023-12-01T01:21:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.230+0000 7f11eb0ddb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f465d0c60 con 0x563f46680ed0 2023-12-01T01:21:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.230+0000 7f11d7fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f4667ffe0 0x563f466808f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.230+0000 7f11dcce9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f4667f680 0x563f4667faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.230+0000 7f11d7fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f4667ffe0 0x563f466808f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:39776/0 (socket says 172.21.15.77:39776) 2023-12-01T01:21:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.230+0000 7f11dcce9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f4667f680 0x563f4667faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60854/0 (socket says 172.21.15.77:60854) 2023-12-01T01:21:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.230+0000 7f11d7fff700 1 -- 172.21.15.77:0/221993376 learned_addr learned my addr 172.21.15.77:0/221993376 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:21:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.230+0000 7f11dd4ea700 1 --2- 172.21.15.77:0/221993376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563f46680ed0 0x563f46681350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.230+0000 7f11d7fff700 1 -- 172.21.15.77:0/221993376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563f46680ed0 msgr2=0x563f46681350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.230+0000 7f11d7fff700 1 --2- 172.21.15.77:0/221993376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563f46680ed0 0x563f46681350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.230+0000 7f11d7fff700 1 -- 172.21.15.77:0/221993376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f4667f680 msgr2=0x563f4667faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.230+0000 7f11d7fff700 1 --2- 172.21.15.77:0/221993376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f4667f680 0x563f4667faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.231+0000 7f11d7fff700 1 -- 172.21.15.77:0/221993376 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f463fed80 con 0x563f4667ffe0 2023-12-01T01:21:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.231+0000 7f11d7fff700 1 --2- 172.21.15.77:0/221993376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f4667ffe0 0x563f466808f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f11c4009d20 tx=0x7f11c4011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddd8c8508f2ece77 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:21:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.231+0000 7f11c2ffd700 1 -- 172.21.15.77:0/221993376 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c4019630 con 0x563f4667ffe0 2023-12-01T01:21:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.231+0000 7f11c2ffd700 1 -- 172.21.15.77:0/221993376 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11c4019c50 con 0x563f4667ffe0 2023-12-01T01:21:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.231+0000 7f11c2ffd700 1 -- 172.21.15.77:0/221993376 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c400ebe0 con 0x563f4667ffe0 2023-12-01T01:21:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.232+0000 7f11eb0ddb80 1 -- 172.21.15.77:0/221993376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f4667ffe0 msgr2=0x563f466808f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.232+0000 7f11eb0ddb80 1 --2- 172.21.15.77:0/221993376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f4667ffe0 0x563f466808f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f11c4009d20 tx=0x7f11c4011720 comp rx=0 tx=0).stop 2023-12-01T01:21:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.232+0000 7f11eb0ddb80 1 -- 172.21.15.77:0/221993376 shutdown_connections 2023-12-01T01:21:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.232+0000 7f11eb0ddb80 1 --2- 172.21.15.77:0/221993376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563f46680ed0 0x563f46681350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.232+0000 7f11eb0ddb80 1 --2- 172.21.15.77:0/221993376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f4667f680 0x563f4667faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.232+0000 7f11eb0ddb80 1 --2- 172.21.15.77:0/221993376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f4667ffe0 0x563f466808f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.232+0000 7f11eb0ddb80 1 -- 172.21.15.77:0/221993376 >> 172.21.15.77:0/221993376 conn(0x563f465df070 msgr2=0x563f4667c040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:21:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.232+0000 7f11eb0ddb80 1 -- 172.21.15.77:0/221993376 shutdown_connections 2023-12-01T01:21:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.232+0000 7f11eb0ddb80 1 -- 172.21.15.77:0/221993376 wait complete. 2023-12-01T01:21:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.232+0000 7f11eb0ddb80 1 -- start start 2023-12-01T01:21:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.233+0000 7f11eb0ddb80 1 Processor -- start 2023-12-01T01:21:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.233+0000 7f11eb0ddb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f4667f680 0x563f466894a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.233+0000 7f11eb0ddb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563f4667ffe0 0x563f466899e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.233+0000 7f11eb0ddb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f46680ed0 0x563f46690a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.233+0000 7f11eb0ddb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f4656d4b0 con 0x563f4667f680 2023-12-01T01:21:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.233+0000 7f11eb0ddb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f4656d150 con 0x563f46680ed0 2023-12-01T01:21:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.233+0000 7f11eb0ddb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f464548c0 con 0x563f4667ffe0 2023-12-01T01:21:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.233+0000 7f11eb0ddb80 10 client.0 _ll_register_callbacks cb 0x563f464baa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:21:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.233+0000 7f11dcce9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f4667f680 0x563f466894a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.233+0000 7f11d7fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563f4667ffe0 0x563f466899e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.233+0000 7f11d7fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563f4667ffe0 0x563f466899e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:47050/0 (socket says 172.21.15.77:47050) 2023-12-01T01:21:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.233+0000 7f11d7fff700 1 -- 172.21.15.77:0/3478100048 learned_addr learned my addr 172.21.15.77:0/3478100048 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:21:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.233+0000 7f11dd4ea700 1 --2- 172.21.15.77:0/3478100048 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f46680ed0 0x563f46690a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.233+0000 7f11dd4ea700 1 -- 172.21.15.77:0/3478100048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563f4667ffe0 msgr2=0x563f466899e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.233+0000 7f11dd4ea700 1 --2- 172.21.15.77:0/3478100048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563f4667ffe0 0x563f466899e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.233+0000 7f11dd4ea700 1 -- 172.21.15.77:0/3478100048 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f4667f680 msgr2=0x563f466894a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.233+0000 7f11dd4ea700 1 --2- 172.21.15.77:0/3478100048 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f4667f680 0x563f466894a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.233+0000 7f11dd4ea700 1 -- 172.21.15.77:0/3478100048 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c4009980 con 0x563f46680ed0 2023-12-01T01:21:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.233+0000 7f11dcce9700 1 --2- 172.21.15.77:0/3478100048 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f4667f680 0x563f466894a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:21:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.234+0000 7f11dd4ea700 1 --2- 172.21.15.77:0/3478100048 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f46680ed0 0x563f46690a20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f11d00cadc0 tx=0x7f11d009b1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc4212637a85adae server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:21:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.234+0000 7f11b7fff700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2023-12-01T01:21:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.234+0000 7f11b7fff700 1 -- 172.21.15.77:0/3478100048 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d009e900 con 0x563f46680ed0 2023-12-01T01:21:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.234+0000 7f11eb0ddb80 1 -- 172.21.15.77:0/3478100048 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563f465fbcb0 con 0x563f46680ed0 2023-12-01T01:21:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.234+0000 7f11b7fff700 1 -- 172.21.15.77:0/3478100048 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f11d00cc070 con 0x563f46680ed0 2023-12-01T01:21:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.234+0000 7f11b7fff700 1 -- 172.21.15.77:0/3478100048 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11d00a2b40 con 0x563f46680ed0 2023-12-01T01:21:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.234+0000 7f11eb0ddb80 10 client.25700 fetch_fsmap learned FSMap version 31 2023-12-01T01:21:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.234+0000 7f11eb0ddb80 1 -- 172.21.15.77:0/3478100048 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x563f463e7c00 con 0x563f46680ed0 2023-12-01T01:21:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.234+0000 7f11b7fff700 1 -- 172.21.15.77:0/3478100048 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563f465fbcb0 con 0x563f46680ed0 2023-12-01T01:21:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.234+0000 7f11b7fff700 1 -- 172.21.15.77:0/3478100048 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f11d00a26b0 con 0x563f46680ed0 2023-12-01T01:21:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.234+0000 7f11eb0ddb80 10 client.25700 fetch_fsmap finished waiting for FSMap version 31 2023-12-01T01:21:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.234+0000 7f11eb0ddb80 10 client.25700 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-01T01:21:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.234+0000 7f11eb0ddb80 20 client.25700 populate_metadata read hostname 'smithi077' 2023-12-01T01:21:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.234+0000 7f11eb0ddb80 1 --2- 172.21.15.77:0/3478100048 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x563f46699760 0x563f4669bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.234+0000 7f11eb0ddb80 4 client.25700 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:21:23.241 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.234+0000 7f11eb0ddb80 1 -- 172.21.15.77:0/3478100048 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563f4664edb0 con 0x563f46699760 2023-12-01T01:21:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.235+0000 7f11dcce9700 1 --2- 172.21.15.77:0/3478100048 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x563f46699760 0x563f4669bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.235+0000 7f11b7fff700 10 client.25700 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:21:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.235+0000 7f11dcce9700 1 --2- 172.21.15.77:0/3478100048 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x563f46699760 0x563f4669bb80 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93568507dd1a3de server_cookie=ca2d06f87b988d34 in_seq=0 out_seq=0 2023-12-01T01:21:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.236+0000 7f11b7fff700 1 -- 172.21.15.77:0/3478100048 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x563f4664edb0 con 0x563f46699760 2023-12-01T01:21:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.236+0000 7f11b7fff700 10 client.25700 handle_command_reply: tid=0 2023-12-01T01:21:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.236+0000 7f11eb0ddb80 1 client.25700 shutdown 2023-12-01T01:21:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.236+0000 7f11eb0ddb80 1 -- 172.21.15.77:0/3478100048 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f46680ed0 msgr2=0x563f46690a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.236+0000 7f11eb0ddb80 1 --2- 172.21.15.77:0/3478100048 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f46680ed0 0x563f46690a20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f11d00cadc0 tx=0x7f11d009b1d0 comp rx=0 tx=0).stop 2023-12-01T01:21:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.236+0000 7f11eb0ddb80 1 -- 172.21.15.77:0/3478100048 shutdown_connections 2023-12-01T01:21:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.236+0000 7f11eb0ddb80 1 --2- 172.21.15.77:0/3478100048 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x563f46699760 0x563f4669bb80 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.236+0000 7f11eb0ddb80 1 --2- 172.21.15.77:0/3478100048 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f46680ed0 0x563f46690a20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.236+0000 7f11b7fff700 0 client.25700 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:21:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.236+0000 7f11eb0ddb80 1 --2- 172.21.15.77:0/3478100048 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f4667f680 0x563f466894a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.236+0000 7f11eb0ddb80 1 --2- 172.21.15.77:0/3478100048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563f4667ffe0 0x563f466899e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:23.243 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.236+0000 7f11eb0ddb80 1 -- 172.21.15.77:0/3478100048 >> 172.21.15.77:0/3478100048 conn(0x563f465df070 msgr2=0x563f4667a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:21:23.243 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.236+0000 7f11eb0ddb80 1 -- 172.21.15.77:0/3478100048 shutdown_connections 2023-12-01T01:21:23.243 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.236+0000 7f11eb0ddb80 1 -- 172.21.15.77:0/3478100048 wait complete. 2023-12-01T01:21:23.243 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.236+0000 7f11eb0ddb80 20 client.25700 trim_cache size 0 max 16384 2023-12-01T01:21:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.253+0000 7f11eb0ddb80 1 Processor -- start 2023-12-01T01:21:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.253+0000 7f11eb0ddb80 1 -- start start 2023-12-01T01:21:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.254+0000 7f11eb0ddb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f4667f680 0x563f466a04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.254+0000 7f11eb0ddb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f4667ffe0 0x563f4668e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.254+0000 7f11eb0ddb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563f46680ed0 0x563f466812b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.254+0000 7f11eb0ddb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f46675500 con 0x563f4667ffe0 2023-12-01T01:21:23.257 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.254+0000 7f11eb0ddb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f46689010 con 0x563f4667f680 2023-12-01T01:21:23.257 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.254+0000 7f11eb0ddb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f46696ad0 con 0x563f46680ed0 2023-12-01T01:21:23.257 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.254+0000 7f11d7fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f4667ffe0 0x563f4668e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:23.257 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.254+0000 7f11d7fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f4667ffe0 0x563f4668e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60868/0 (socket says 172.21.15.77:60868) 2023-12-01T01:21:23.257 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.254+0000 7f11d7fff700 1 -- 172.21.15.77:0/1899212830 learned_addr learned my addr 172.21.15.77:0/1899212830 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:21:23.257 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.254+0000 7f11dcce9700 1 --2- 172.21.15.77:0/1899212830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f4667f680 0x563f466a04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:23.257 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.254+0000 7f11dd4ea700 1 --2- 172.21.15.77:0/1899212830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563f46680ed0 0x563f466812b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:23.257 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.254+0000 7f11d7fff700 1 -- 172.21.15.77:0/1899212830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563f46680ed0 msgr2=0x563f466812b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:23.257 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.254+0000 7f11d7fff700 1 --2- 172.21.15.77:0/1899212830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563f46680ed0 0x563f466812b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:23.257 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.254+0000 7f11d7fff700 1 -- 172.21.15.77:0/1899212830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f4667f680 msgr2=0x563f466a04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:23.257 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.254+0000 7f11d7fff700 1 --2- 172.21.15.77:0/1899212830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f4667f680 0x563f466a04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:23.257 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.254+0000 7f11d7fff700 1 -- 172.21.15.77:0/1899212830 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f464251f0 con 0x563f4667ffe0 2023-12-01T01:21:23.258 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.254+0000 7f11dcce9700 1 --2- 172.21.15.77:0/1899212830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f4667f680 0x563f466a04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:21:23.258 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.254+0000 7f11d7fff700 1 --2- 172.21.15.77:0/1899212830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f4667ffe0 0x563f4668e930 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f11c4011a70 tx=0x7f11c4009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0d95a8cf82d686a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:21:23.258 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.255+0000 7f11c37fe700 1 -- 172.21.15.77:0/1899212830 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c400e5b0 con 0x563f4667ffe0 2023-12-01T01:21:23.258 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.255+0000 7f11c37fe700 1 -- 172.21.15.77:0/1899212830 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11c40198f0 con 0x563f4667ffe0 2023-12-01T01:21:23.258 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.255+0000 7f11c37fe700 1 -- 172.21.15.77:0/1899212830 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c400ea10 con 0x563f4667ffe0 2023-12-01T01:21:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.255+0000 7f11eb0ddb80 1 -- 172.21.15.77:0/1899212830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f4667ffe0 msgr2=0x563f4668e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.255+0000 7f11eb0ddb80 1 --2- 172.21.15.77:0/1899212830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f4667ffe0 0x563f4668e930 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f11c4011a70 tx=0x7f11c4009440 comp rx=0 tx=0).stop 2023-12-01T01:21:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.255+0000 7f11eb0ddb80 1 -- 172.21.15.77:0/1899212830 shutdown_connections 2023-12-01T01:21:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.255+0000 7f11eb0ddb80 1 --2- 172.21.15.77:0/1899212830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563f46680ed0 0x563f466812b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.255+0000 7f11eb0ddb80 1 --2- 172.21.15.77:0/1899212830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f4667f680 0x563f466a04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.255+0000 7f11eb0ddb80 1 --2- 172.21.15.77:0/1899212830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f4667ffe0 0x563f4668e930 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.255+0000 7f11eb0ddb80 1 -- 172.21.15.77:0/1899212830 >> 172.21.15.77:0/1899212830 conn(0x563f465df070 msgr2=0x563f46688140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:21:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.256+0000 7f11eb0ddb80 1 -- 172.21.15.77:0/1899212830 shutdown_connections 2023-12-01T01:21:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.256+0000 7f11eb0ddb80 1 -- 172.21.15.77:0/1899212830 wait complete. 2023-12-01T01:21:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.256+0000 7f11eb0ddb80 1 -- start start 2023-12-01T01:21:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.256+0000 7f11eb0ddb80 1 Processor -- start 2023-12-01T01:21:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.256+0000 7f11eb0ddb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f4667f680 0x563f466e5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.256+0000 7f11eb0ddb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f4667ffe0 0x563f466e7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.256+0000 7f11eb0ddb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563f46680ed0 0x563f466e9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.256+0000 7f11eb0ddb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f466ea4a0 con 0x563f4667f680 2023-12-01T01:21:23.262 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.256+0000 7f11eb0ddb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f466ea610 con 0x563f4667ffe0 2023-12-01T01:21:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.256+0000 7f11eb0ddb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f466ea7d0 con 0x563f46680ed0 2023-12-01T01:21:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.256+0000 7f11eb0ddb80 10 client.0 _ll_register_callbacks cb 0x7f11d009a610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:21:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.257+0000 7f11dcce9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f4667f680 0x563f466e5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.257+0000 7f11dd4ea700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563f46680ed0 0x563f466e9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.257+0000 7f11dcce9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f4667f680 0x563f466e5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60874/0 (socket says 172.21.15.77:60874) 2023-12-01T01:21:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.257+0000 7f11dd4ea700 1 --2- 172.21.15.77:0/2077641300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563f46680ed0 0x563f466e9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:47066/0 (socket says 172.21.15.77:47066) 2023-12-01T01:21:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.257+0000 7f11dcce9700 1 -- 172.21.15.77:0/2077641300 learned_addr learned my addr 172.21.15.77:0/2077641300 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:21:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.257+0000 7f11d7fff700 1 --2- 172.21.15.77:0/2077641300 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f4667ffe0 0x563f466e7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.257+0000 7f11dd4ea700 1 -- 172.21.15.77:0/2077641300 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f4667ffe0 msgr2=0x563f466e7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.257+0000 7f11dd4ea700 1 --2- 172.21.15.77:0/2077641300 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f4667ffe0 0x563f466e7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.257+0000 7f11dd4ea700 1 -- 172.21.15.77:0/2077641300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f4667f680 msgr2=0x563f466e5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.257+0000 7f11dd4ea700 1 --2- 172.21.15.77:0/2077641300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f4667f680 0x563f466e5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.257+0000 7f11dd4ea700 1 -- 172.21.15.77:0/2077641300 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11d8108a30 con 0x563f46680ed0 2023-12-01T01:21:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.257+0000 7f11d7fff700 1 --2- 172.21.15.77:0/2077641300 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f4667ffe0 0x563f466e7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:21:23.264 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.257+0000 7f11dcce9700 1 --2- 172.21.15.77:0/2077641300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f4667f680 0x563f466e5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:21:23.264 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.257+0000 7f11dd4ea700 1 --2- 172.21.15.77:0/2077641300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563f46680ed0 0x563f466e9d90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f11d000c6f0 tx=0x7f11d009ab90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=307c5bfa2704394c server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:21:23.264 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.257+0000 7f11b7fff700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2023-12-01T01:21:23.264 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.257+0000 7f11b7fff700 1 -- 172.21.15.77:0/2077641300 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d0096e10 con 0x563f46680ed0 2023-12-01T01:21:23.264 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.257+0000 7f11b7fff700 1 -- 172.21.15.77:0/2077641300 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f11d00cc070 con 0x563f46680ed0 2023-12-01T01:21:23.264 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.257+0000 7f11eb0ddb80 1 -- 172.21.15.77:0/2077641300 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f11c4019c50 con 0x563f46680ed0 2023-12-01T01:21:23.264 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.258+0000 7f11b7fff700 1 -- 172.21.15.77:0/2077641300 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11d009e900 con 0x563f46680ed0 2023-12-01T01:21:23.264 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.258+0000 7f11eb0ddb80 10 client.25717 fetch_fsmap learned FSMap version 31 2023-12-01T01:21:23.264 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.258+0000 7f11eb0ddb80 1 -- 172.21.15.77:0/2077641300 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x563f463fed80 con 0x563f46680ed0 2023-12-01T01:21:23.264 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.258+0000 7f11b7fff700 1 -- 172.21.15.77:0/2077641300 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11c4019c50 con 0x563f46680ed0 2023-12-01T01:21:23.264 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.258+0000 7f11b7fff700 1 -- 172.21.15.77:0/2077641300 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f11d0099ba0 con 0x563f46680ed0 2023-12-01T01:21:23.264 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.258+0000 7f11eb0ddb80 10 client.25717 fetch_fsmap finished waiting for FSMap version 31 2023-12-01T01:21:23.264 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.258+0000 7f11eb0ddb80 10 client.25717 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-01T01:21:23.264 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.258+0000 7f11eb0ddb80 20 client.25717 populate_metadata read hostname 'smithi077' 2023-12-01T01:21:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.258+0000 7f11eb0ddb80 1 --2- 172.21.15.77:0/2077641300 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x563f466ebce0 0x563f466ee100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.258+0000 7f11eb0ddb80 4 client.25717 mds_command: new command op to 24500 tid=0[{"prefix": "flush journal"}] 2023-12-01T01:21:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.258+0000 7f11eb0ddb80 1 -- 172.21.15.77:0/2077641300 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563f465c9ce0 con 0x563f466ebce0 2023-12-01T01:21:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.258+0000 7f11dcce9700 1 --2- 172.21.15.77:0/2077641300 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x563f466ebce0 0x563f466ee100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.259+0000 7f11dcce9700 1 --2- 172.21.15.77:0/2077641300 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x563f466ebce0 0x563f466ee100 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78458d105b4bfbe2 server_cookie=48d7a18165fb4540 in_seq=0 out_seq=0 2023-12-01T01:21:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.259+0000 7f11b7fff700 10 client.25717 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:21:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.274+0000 7f11b7fff700 1 -- 172.21.15.77:0/2077641300 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563f465c9ce0 con 0x563f466ebce0 2023-12-01T01:21:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.274+0000 7f11b7fff700 10 client.25717 handle_command_reply: tid=0 2023-12-01T01:21:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.274+0000 7f11eb0ddb80 1 client.25717 shutdown 2023-12-01T01:21:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.275+0000 7f11eb0ddb80 1 -- 172.21.15.77:0/2077641300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563f46680ed0 msgr2=0x563f466e9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.275+0000 7f11eb0ddb80 1 --2- 172.21.15.77:0/2077641300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563f46680ed0 0x563f466e9d90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f11d000c6f0 tx=0x7f11d009ab90 comp rx=0 tx=0).stop 2023-12-01T01:21:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.275+0000 7f11eb0ddb80 1 -- 172.21.15.77:0/2077641300 shutdown_connections 2023-12-01T01:21:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.275+0000 7f11eb0ddb80 1 --2- 172.21.15.77:0/2077641300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563f46680ed0 0x563f466e9d90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.275+0000 7f11eb0ddb80 1 --2- 172.21.15.77:0/2077641300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f4667f680 0x563f466e5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.275+0000 7f11eb0ddb80 1 --2- 172.21.15.77:0/2077641300 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x563f466ebce0 0x563f466ee100 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.275+0000 7f11eb0ddb80 1 --2- 172.21.15.77:0/2077641300 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f4667ffe0 0x563f466e7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.275+0000 7f11b7fff700 0 client.25717 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:21:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.275+0000 7f11eb0ddb80 1 -- 172.21.15.77:0/2077641300 >> 172.21.15.77:0/2077641300 conn(0x563f465df070 msgr2=0x563f4667bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:21:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.276+0000 7f11eb0ddb80 1 -- 172.21.15.77:0/2077641300 shutdown_connections 2023-12-01T01:21:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.276+0000 7f11eb0ddb80 1 -- 172.21.15.77:0/2077641300 wait complete. 2023-12-01T01:21:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.276+0000 7f11eb0ddb80 20 client.25717 trim_cache size 0 max 16384 2023-12-01T01:21:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout:{ 2023-12-01T01:21:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout: "message": "", 2023-12-01T01:21:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout: "return_code": 0 2023-12-01T01:21:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout:} 2023-12-01T01:21:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.277+0000 7f11def4d700 1 -- 172.21.15.77:0/1425637627 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f11c80a0520 msgr2=0x7f11c80a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.277+0000 7f11def4d700 1 --2- 172.21.15.77:0/1425637627 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f11c80a0520 0x7f11c80a29e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f11cc009990 tx=0x7f11cc005fb0 comp rx=0 tx=0).stop 2023-12-01T01:21:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.277+0000 7f11def4d700 1 -- 172.21.15.77:0/1425637627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11d8101830 msgr2=0x7f11d8070b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.277+0000 7f11def4d700 1 --2- 172.21.15.77:0/1425637627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11d8101830 0x7f11d8070b20 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f11d000ef30 tx=0x7f11d000c3a0 comp rx=0 tx=0).stop 2023-12-01T01:21:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.277+0000 7f11def4d700 1 -- 172.21.15.77:0/1425637627 shutdown_connections 2023-12-01T01:21:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.277+0000 7f11def4d700 1 --2- 172.21.15.77:0/1425637627 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f11c80a0520 0x7f11c80a29e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.277+0000 7f11def4d700 1 --2- 172.21.15.77:0/1425637627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11d8101830 0x7f11d8070b20 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.277+0000 7f11def4d700 1 --2- 172.21.15.77:0/1425637627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11d80ffca0 0x7f11d806f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.277+0000 7f11def4d700 1 --2- 172.21.15.77:0/1425637627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11d8100e90 0x7f11d806fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.277+0000 7f11def4d700 1 -- 172.21.15.77:0/1425637627 >> 172.21.15.77:0/1425637627 conn(0x7f11d80fb230 msgr2=0x7f11d81043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:21:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.277+0000 7f11def4d700 1 -- 172.21.15.77:0/1425637627 shutdown_connections 2023-12-01T01:21:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:23.277+0000 7f11def4d700 1 -- 172.21.15.77:0/1425637627 wait complete. 2023-12-01T01:21:24.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:21:23 smithi203 ceph-mon[103581]: pgmap v403: 97 pgs: 97 active+clean; 59 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.7 MiB/s wr, 1.20k op/s 2023-12-01T01:21:24.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:21:23 smithi137 ceph-mon[103312]: pgmap v403: 97 pgs: 97 active+clean; 59 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.7 MiB/s wr, 1.20k op/s 2023-12-01T01:21:24.153 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:21:23 smithi077 ceph-mon[98867]: pgmap v403: 97 pgs: 97 active+clean; 59 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.7 MiB/s wr, 1.20k op/s 2023-12-01T01:21:26.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:21:25 smithi203 ceph-mon[103581]: pgmap v404: 97 pgs: 97 active+clean; 60 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 3.3 MiB/s wr, 1.45k op/s 2023-12-01T01:21:26.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:21:25 smithi137 ceph-mon[103312]: pgmap v404: 97 pgs: 97 active+clean; 60 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 3.3 MiB/s wr, 1.45k op/s 2023-12-01T01:21:26.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:21:25 smithi077 ceph-mon[98867]: pgmap v404: 97 pgs: 97 active+clean; 60 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 3.3 MiB/s wr, 1.45k op/s 2023-12-01T01:21:28.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:21:27 smithi203 ceph-mon[103581]: pgmap v405: 97 pgs: 97 active+clean; 59 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.8 MiB/s wr, 1.23k op/s 2023-12-01T01:21:28.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:21:27 smithi137 ceph-mon[103312]: pgmap v405: 97 pgs: 97 active+clean; 59 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.8 MiB/s wr, 1.23k op/s 2023-12-01T01:21:28.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:21:27 smithi077 ceph-mon[98867]: pgmap v405: 97 pgs: 97 active+clean; 59 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.8 MiB/s wr, 1.23k op/s 2023-12-01T01:21:29.448 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-01T01:21:29.615 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.614+0000 7f247e6d0700 1 -- 172.21.15.77:0/1527451141 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f247810d4e0 msgr2=0x7f247810f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:29.616 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.614+0000 7f247e6d0700 1 --2- 172.21.15.77:0/1527451141 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f247810d4e0 0x7f247810f9a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f247000b260 tx=0x7f2470007d00 comp rx=0 tx=0).stop 2023-12-01T01:21:29.616 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.614+0000 7f247e6d0700 1 -- 172.21.15.77:0/1527451141 shutdown_connections 2023-12-01T01:21:29.616 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.614+0000 7f247e6d0700 1 --2- 172.21.15.77:0/1527451141 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f247810d4e0 0x7f247810f9a0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:29.616 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.614+0000 7f247e6d0700 1 --2- 172.21.15.77:0/1527451141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2478108000 0x7f247810a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:29.616 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.614+0000 7f247e6d0700 1 --2- 172.21.15.77:0/1527451141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f247810a9c0 0x7f247810ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:29.616 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.614+0000 7f247e6d0700 1 -- 172.21.15.77:0/1527451141 >> 172.21.15.77:0/1527451141 conn(0x7f247806df90 msgr2=0x7f24780703b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:21:29.616 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.614+0000 7f247e6d0700 1 -- 172.21.15.77:0/1527451141 shutdown_connections 2023-12-01T01:21:29.616 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.614+0000 7f247e6d0700 1 -- 172.21.15.77:0/1527451141 wait complete. 2023-12-01T01:21:29.618 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.615+0000 7f247e6d0700 1 Processor -- start 2023-12-01T01:21:29.618 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.615+0000 7f247e6d0700 1 -- start start 2023-12-01T01:21:29.619 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.615+0000 7f247e6d0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2478108000 0x7f247819f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:29.619 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.615+0000 7f247e6d0700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f247810a9c0 0x7f247819f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:29.619 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.615+0000 7f247e6d0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f247810d4e0 0x7f24781a6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:29.619 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.615+0000 7f247e6d0700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24781a6ee0 con 0x7f247810d4e0 2023-12-01T01:21:29.619 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.615+0000 7f247e6d0700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24781a7020 con 0x7f2478108000 2023-12-01T01:21:29.619 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.615+0000 7f247e6d0700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24781a71b0 con 0x7f247810a9c0 2023-12-01T01:21:29.619 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.615+0000 7f2477fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2478108000 0x7f247819f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:29.619 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.615+0000 7f247cc6d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f247810d4e0 0x7f24781a6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:29.619 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.615+0000 7f2477fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2478108000 0x7f247819f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:39808/0 (socket says 172.21.15.77:39808) 2023-12-01T01:21:29.619 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.615+0000 7f24777fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f247810a9c0 0x7f247819f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:29.619 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.615+0000 7f2477fff700 1 -- 172.21.15.77:0/4244335880 learned_addr learned my addr 172.21.15.77:0/4244335880 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:21:29.619 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.616+0000 7f2477fff700 1 -- 172.21.15.77:0/4244335880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f247810a9c0 msgr2=0x7f247819f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:29.619 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.616+0000 7f2477fff700 1 --2- 172.21.15.77:0/4244335880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f247810a9c0 0x7f247819f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:29.620 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.616+0000 7f2477fff700 1 -- 172.21.15.77:0/4244335880 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f247810d4e0 msgr2=0x7f24781a6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:29.620 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.616+0000 7f2477fff700 1 --2- 172.21.15.77:0/4244335880 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f247810d4e0 0x7f24781a6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:29.620 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.616+0000 7f2477fff700 1 -- 172.21.15.77:0/4244335880 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24700079e0 con 0x7f2478108000 2023-12-01T01:21:29.620 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.616+0000 7f24777fe700 1 --2- 172.21.15.77:0/4244335880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f247810a9c0 0x7f247819f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:21:29.620 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.616+0000 7f247cc6d700 1 --2- 172.21.15.77:0/4244335880 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f247810d4e0 0x7f24781a6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:21:29.620 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.616+0000 7f2477fff700 1 --2- 172.21.15.77:0/4244335880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2478108000 0x7f247819f360 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f246800ea70 tx=0x7f246800ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:21:29.620 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.616+0000 7f24757fa700 1 -- 172.21.15.77:0/4244335880 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246800ccd0 con 0x7f2478108000 2023-12-01T01:21:29.620 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.616+0000 7f24757fa700 1 -- 172.21.15.77:0/4244335880 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f246800ce30 con 0x7f2478108000 2023-12-01T01:21:29.620 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.616+0000 7f247e6d0700 1 -- 172.21.15.77:0/4244335880 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24781a7490 con 0x7f2478108000 2023-12-01T01:21:29.620 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.616+0000 7f24757fa700 1 -- 172.21.15.77:0/4244335880 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2468015390 con 0x7f2478108000 2023-12-01T01:21:29.620 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.616+0000 7f247e6d0700 1 -- 172.21.15.77:0/4244335880 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24781a7950 con 0x7f2478108000 2023-12-01T01:21:29.621 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.619+0000 7f24757fa700 1 -- 172.21.15.77:0/4244335880 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f2468004830 con 0x7f2478108000 2023-12-01T01:21:29.621 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.619+0000 7f24757fa700 1 --2- 172.21.15.77:0/4244335880 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f24600a0520 0x7f24600a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:29.621 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.619+0000 7f24757fa700 1 -- 172.21.15.77:0/4244335880 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f24680cb8c0 con 0x7f2478108000 2023-12-01T01:21:29.621 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.619+0000 7f24777fe700 1 --2- 172.21.15.77:0/4244335880 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f24600a0520 0x7f24600a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:29.621 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.620+0000 7f24777fe700 1 --2- 172.21.15.77:0/4244335880 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f24600a0520 0x7f24600a29e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f246c011490 tx=0x7f246c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:21:29.623 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.621+0000 7f248a860b80 1 Processor -- start 2023-12-01T01:21:29.623 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.621+0000 7f248a860b80 1 -- start start 2023-12-01T01:21:29.623 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.621+0000 7f248a860b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562cb1ac6500 0x562cb1ac6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:29.623 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.621+0000 7f248a860b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cb1ac6e80 0x562cb1ac5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:29.623 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.621+0000 7f248a860b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562cb1ac5800 0x562cb1ac5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:29.623 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.621+0000 7f248a860b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cb1a18a10 con 0x562cb1ac6e80 2023-12-01T01:21:29.623 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.621+0000 7f248a860b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cb1a163d0 con 0x562cb1ac6500 2023-12-01T01:21:29.624 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.621+0000 7f248a860b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cb1a11cc0 con 0x562cb1ac5800 2023-12-01T01:21:29.624 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.621+0000 7f247cc6d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562cb1ac5800 0x562cb1ac5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:29.624 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.621+0000 7f24777fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cb1ac6e80 0x562cb1ac5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:29.624 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.621+0000 7f2477fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562cb1ac6500 0x562cb1ac6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:29.624 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.621+0000 7f247cc6d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562cb1ac5800 0x562cb1ac5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:47086/0 (socket says 172.21.15.77:47086) 2023-12-01T01:21:29.624 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.621+0000 7f247cc6d700 1 -- 172.21.15.77:0/1390603842 learned_addr learned my addr 172.21.15.77:0/1390603842 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:21:29.624 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.621+0000 7f247cc6d700 1 -- 172.21.15.77:0/1390603842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562cb1ac6500 msgr2=0x562cb1ac6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:29.624 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.621+0000 7f247cc6d700 1 --2- 172.21.15.77:0/1390603842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562cb1ac6500 0x562cb1ac6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:29.624 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.621+0000 7f247cc6d700 1 -- 172.21.15.77:0/1390603842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cb1ac6e80 msgr2=0x562cb1ac5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:29.624 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.621+0000 7f247cc6d700 1 --2- 172.21.15.77:0/1390603842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cb1ac6e80 0x562cb1ac5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:29.624 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.622+0000 7f247cc6d700 1 -- 172.21.15.77:0/1390603842 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562cb183fde0 con 0x562cb1ac5800 2023-12-01T01:21:29.624 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.622+0000 7f2477fff700 1 --2- 172.21.15.77:0/1390603842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562cb1ac6500 0x562cb1ac6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:21:29.624 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.622+0000 7f247cc6d700 1 --2- 172.21.15.77:0/1390603842 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562cb1ac5800 0x562cb1ac5c80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f247000b260 tx=0x7f24700060c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21e79e53d2f924b6 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:21:29.624 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.622+0000 7f245e7fc700 1 -- 172.21.15.77:0/1390603842 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247001f040 con 0x562cb1ac5800 2023-12-01T01:21:29.625 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.622+0000 7f245e7fc700 1 -- 172.21.15.77:0/1390603842 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f247000be10 con 0x562cb1ac5800 2023-12-01T01:21:29.625 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.622+0000 7f245e7fc700 1 -- 172.21.15.77:0/1390603842 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2470010920 con 0x562cb1ac5800 2023-12-01T01:21:29.629 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.623+0000 7f248a860b80 1 -- 172.21.15.77:0/1390603842 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562cb1ac5800 msgr2=0x562cb1ac5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:29.629 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.623+0000 7f248a860b80 1 --2- 172.21.15.77:0/1390603842 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562cb1ac5800 0x562cb1ac5c80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f247000b260 tx=0x7f24700060c0 comp rx=0 tx=0).stop 2023-12-01T01:21:29.629 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.623+0000 7f248a860b80 1 -- 172.21.15.77:0/1390603842 shutdown_connections 2023-12-01T01:21:29.629 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.623+0000 7f248a860b80 1 --2- 172.21.15.77:0/1390603842 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562cb1ac5800 0x562cb1ac5c80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:29.629 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.623+0000 7f248a860b80 1 --2- 172.21.15.77:0/1390603842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562cb1ac6500 0x562cb1ac6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:29.629 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.623+0000 7f248a860b80 1 --2- 172.21.15.77:0/1390603842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cb1ac6e80 0x562cb1ac5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:29.629 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.623+0000 7f248a860b80 1 -- 172.21.15.77:0/1390603842 >> 172.21.15.77:0/1390603842 conn(0x562cb1a200d0 msgr2=0x562cb1abb940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:21:29.629 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.623+0000 7f248a860b80 1 -- 172.21.15.77:0/1390603842 shutdown_connections 2023-12-01T01:21:29.629 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.623+0000 7f248a860b80 1 -- 172.21.15.77:0/1390603842 wait complete. 2023-12-01T01:21:29.629 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.623+0000 7f248a860b80 1 -- start start 2023-12-01T01:21:29.629 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.624+0000 7f248a860b80 1 Processor -- start 2023-12-01T01:21:29.629 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.624+0000 7f248a860b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cb1ac5800 0x562cb1aced50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:29.630 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.624+0000 7f248a860b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562cb1ac6500 0x562cb1acf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:29.630 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.624+0000 7f248a860b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562cb1ac6e80 0x562cb1ad62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:29.630 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.624+0000 7f248a860b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cb19ae510 con 0x562cb1ac5800 2023-12-01T01:21:29.630 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.624+0000 7f248a860b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cb19ae1b0 con 0x562cb1ac6e80 2023-12-01T01:21:29.630 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.624+0000 7f248a860b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cb1895920 con 0x562cb1ac6500 2023-12-01T01:21:29.630 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.624+0000 7f248a860b80 10 client.0 _ll_register_callbacks cb 0x562cb18fba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:21:29.630 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.624+0000 7f2477fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cb1ac5800 0x562cb1aced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:29.630 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.624+0000 7f2477fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cb1ac5800 0x562cb1aced50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60900/0 (socket says 172.21.15.77:60900) 2023-12-01T01:21:29.630 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.624+0000 7f2477fff700 1 -- 172.21.15.77:0/648614087 learned_addr learned my addr 172.21.15.77:0/648614087 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:21:29.630 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.624+0000 7f24777fe700 1 --2- 172.21.15.77:0/648614087 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562cb1ac6500 0x562cb1acf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:29.630 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.624+0000 7f247cc6d700 1 --2- 172.21.15.77:0/648614087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562cb1ac6e80 0x562cb1ad62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:29.630 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.624+0000 7f24777fe700 1 -- 172.21.15.77:0/648614087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562cb1ac6e80 msgr2=0x562cb1ad62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:29.630 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.624+0000 7f24777fe700 1 --2- 172.21.15.77:0/648614087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562cb1ac6e80 0x562cb1ad62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:29.631 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.624+0000 7f24777fe700 1 -- 172.21.15.77:0/648614087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cb1ac5800 msgr2=0x562cb1aced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:29.631 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.624+0000 7f24777fe700 1 --2- 172.21.15.77:0/648614087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cb1ac5800 0x562cb1aced50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:29.631 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.624+0000 7f24777fe700 1 -- 172.21.15.77:0/648614087 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24781a7950 con 0x562cb1ac6500 2023-12-01T01:21:29.631 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.624+0000 7f247cc6d700 1 --2- 172.21.15.77:0/648614087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562cb1ac6e80 0x562cb1ad62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:21:29.631 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.624+0000 7f2477fff700 1 --2- 172.21.15.77:0/648614087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cb1ac5800 0x562cb1aced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:21:29.631 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.624+0000 7f24777fe700 1 --2- 172.21.15.77:0/648614087 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562cb1ac6500 0x562cb1acf290 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f246c011be0 tx=0x7f246c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c994ce9feac2f983 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:21:29.631 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.624+0000 7f24577fe700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2023-12-01T01:21:29.631 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.625+0000 7f24577fe700 1 -- 172.21.15.77:0/648614087 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246c01fa90 con 0x562cb1ac6500 2023-12-01T01:21:29.631 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.625+0000 7f24577fe700 1 -- 172.21.15.77:0/648614087 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f246c01fbf0 con 0x562cb1ac6500 2023-12-01T01:21:29.631 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.625+0000 7f248a860b80 1 -- 172.21.15.77:0/648614087 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562cb1a3cd10 con 0x562cb1ac6500 2023-12-01T01:21:29.631 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.625+0000 7f24577fe700 1 -- 172.21.15.77:0/648614087 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f246c01fdf0 con 0x562cb1ac6500 2023-12-01T01:21:29.631 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.625+0000 7f248a860b80 10 client.25729 fetch_fsmap learned FSMap version 31 2023-12-01T01:21:29.631 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.625+0000 7f248a860b80 1 -- 172.21.15.77:0/648614087 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f24781105c0 con 0x562cb1ac6500 2023-12-01T01:21:29.631 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.625+0000 7f24577fe700 1 -- 172.21.15.77:0/648614087 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562cb1a3cd10 con 0x562cb1ac6500 2023-12-01T01:21:29.632 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.625+0000 7f24577fe700 1 -- 172.21.15.77:0/648614087 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f246c032420 con 0x562cb1ac6500 2023-12-01T01:21:29.632 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.625+0000 7f248a860b80 10 client.25729 fetch_fsmap finished waiting for FSMap version 31 2023-12-01T01:21:29.632 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.625+0000 7f248a860b80 10 client.25729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-01T01:21:29.632 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.625+0000 7f248a860b80 20 client.25729 populate_metadata read hostname 'smithi077' 2023-12-01T01:21:29.632 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.625+0000 7f248a860b80 1 --2- 172.21.15.77:0/648614087 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x562cb1adabb0 0x562cb1adcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:29.632 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.625+0000 7f248a860b80 4 client.25729 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:21:29.632 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.625+0000 7f248a860b80 1 -- 172.21.15.77:0/648614087 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562cb1a8fe10 con 0x562cb1adabb0 2023-12-01T01:21:29.632 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.625+0000 7f2477fff700 1 --2- 172.21.15.77:0/648614087 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x562cb1adabb0 0x562cb1adcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:29.632 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.626+0000 7f24577fe700 10 client.25729 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:21:29.632 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.626+0000 7f2477fff700 1 --2- 172.21.15.77:0/648614087 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x562cb1adabb0 0x562cb1adcfd0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=999f5a72f35b675d server_cookie=db570239ed4dc43e in_seq=0 out_seq=0 2023-12-01T01:21:29.632 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.627+0000 7f24577fe700 1 -- 172.21.15.77:0/648614087 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x562cb1a8fe10 con 0x562cb1adabb0 2023-12-01T01:21:29.632 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.627+0000 7f24577fe700 10 client.25729 handle_command_reply: tid=0 2023-12-01T01:21:29.632 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.627+0000 7f248a860b80 1 client.25729 shutdown 2023-12-01T01:21:29.632 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.627+0000 7f248a860b80 1 -- 172.21.15.77:0/648614087 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562cb1ac6500 msgr2=0x562cb1acf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:29.633 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.627+0000 7f248a860b80 1 --2- 172.21.15.77:0/648614087 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562cb1ac6500 0x562cb1acf290 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f246c011be0 tx=0x7f246c0047c0 comp rx=0 tx=0).stop 2023-12-01T01:21:29.633 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.627+0000 7f248a860b80 1 -- 172.21.15.77:0/648614087 shutdown_connections 2023-12-01T01:21:29.633 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.627+0000 7f248a860b80 1 --2- 172.21.15.77:0/648614087 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x562cb1adabb0 0x562cb1adcfd0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:29.633 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.627+0000 7f248a860b80 1 --2- 172.21.15.77:0/648614087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562cb1ac6e80 0x562cb1ad62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:29.633 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.627+0000 7f248a860b80 1 --2- 172.21.15.77:0/648614087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cb1ac5800 0x562cb1aced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:29.633 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.627+0000 7f24577fe700 0 client.25729 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:21:29.633 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.627+0000 7f248a860b80 1 --2- 172.21.15.77:0/648614087 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562cb1ac6500 0x562cb1acf290 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:29.633 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.627+0000 7f248a860b80 1 -- 172.21.15.77:0/648614087 >> 172.21.15.77:0/648614087 conn(0x562cb1a200d0 msgr2=0x562cb1abb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:21:29.633 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.627+0000 7f248a860b80 1 -- 172.21.15.77:0/648614087 shutdown_connections 2023-12-01T01:21:29.633 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.627+0000 7f248a860b80 1 -- 172.21.15.77:0/648614087 wait complete. 2023-12-01T01:21:29.633 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.627+0000 7f248a860b80 20 client.25729 trim_cache size 0 max 16384 2023-12-01T01:21:29.651 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.648+0000 7f248a860b80 1 Processor -- start 2023-12-01T01:21:29.651 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.648+0000 7f248a860b80 1 -- start start 2023-12-01T01:21:29.651 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.648+0000 7f248a860b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cb1ac5800 0x562cb1ad4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:29.651 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.648+0000 7f248a860b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562cb1ac6500 0x562cb1ac96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:29.651 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.648+0000 7f248a860b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562cb1ac6e80 0x562cb1ac9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:29.651 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.648+0000 7f248a860b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cb1a3cd10 con 0x562cb1ac5800 2023-12-01T01:21:29.651 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.648+0000 7f248a860b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cb1acd420 con 0x562cb1ac6500 2023-12-01T01:21:29.651 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.648+0000 7f248a860b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cb1abc9e0 con 0x562cb1ac6e80 2023-12-01T01:21:29.651 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.648+0000 7f2477fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cb1ac5800 0x562cb1ad4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:29.651 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.648+0000 7f2477fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cb1ac5800 0x562cb1ad4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60908/0 (socket says 172.21.15.77:60908) 2023-12-01T01:21:29.651 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.648+0000 7f2477fff700 1 -- 172.21.15.77:0/3552454167 learned_addr learned my addr 172.21.15.77:0/3552454167 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:21:29.651 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.648+0000 7f24777fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562cb1ac6500 0x562cb1ac96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:29.652 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.648+0000 7f247cc6d700 1 --2- 172.21.15.77:0/3552454167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562cb1ac6e80 0x562cb1ac9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:29.652 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.649+0000 7f2477fff700 1 -- 172.21.15.77:0/3552454167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562cb1ac6e80 msgr2=0x562cb1ac9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:29.652 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.649+0000 7f2477fff700 1 --2- 172.21.15.77:0/3552454167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562cb1ac6e80 0x562cb1ac9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:29.652 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.649+0000 7f2477fff700 1 -- 172.21.15.77:0/3552454167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562cb1ac6500 msgr2=0x562cb1ac96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:29.652 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.649+0000 7f2477fff700 1 --2- 172.21.15.77:0/3552454167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562cb1ac6500 0x562cb1ac96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:29.652 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.649+0000 7f2477fff700 1 -- 172.21.15.77:0/3552454167 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562cb1828c60 con 0x562cb1ac5800 2023-12-01T01:21:29.652 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.649+0000 7f2477fff700 1 --2- 172.21.15.77:0/3552454167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cb1ac5800 0x562cb1ad4280 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f2468095be0 tx=0x7f24680c9330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df1ab687292b822 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:21:29.652 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.649+0000 7f245effd700 1 -- 172.21.15.77:0/3552454167 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2468063070 con 0x562cb1ac5800 2023-12-01T01:21:29.652 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.649+0000 7f245effd700 1 -- 172.21.15.77:0/3552454167 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24680991c0 con 0x562cb1ac5800 2023-12-01T01:21:29.652 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.649+0000 7f245effd700 1 -- 172.21.15.77:0/3552454167 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2468097650 con 0x562cb1ac5800 2023-12-01T01:21:29.654 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.650+0000 7f248a860b80 1 -- 172.21.15.77:0/3552454167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cb1ac5800 msgr2=0x562cb1ad4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:29.654 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.650+0000 7f248a860b80 1 --2- 172.21.15.77:0/3552454167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cb1ac5800 0x562cb1ad4280 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f2468095be0 tx=0x7f24680c9330 comp rx=0 tx=0).stop 2023-12-01T01:21:29.654 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.650+0000 7f248a860b80 1 -- 172.21.15.77:0/3552454167 shutdown_connections 2023-12-01T01:21:29.654 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.650+0000 7f248a860b80 1 --2- 172.21.15.77:0/3552454167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562cb1ac6e80 0x562cb1ac9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:29.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.650+0000 7f248a860b80 1 --2- 172.21.15.77:0/3552454167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cb1ac5800 0x562cb1ad4280 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:29.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.650+0000 7f248a860b80 1 --2- 172.21.15.77:0/3552454167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562cb1ac6500 0x562cb1ac96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:29.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.650+0000 7f248a860b80 1 -- 172.21.15.77:0/3552454167 >> 172.21.15.77:0/3552454167 conn(0x562cb1a200d0 msgr2=0x562cb1ad6bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:21:29.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.650+0000 7f248a860b80 1 -- 172.21.15.77:0/3552454167 shutdown_connections 2023-12-01T01:21:29.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.650+0000 7f248a860b80 1 -- 172.21.15.77:0/3552454167 wait complete. 2023-12-01T01:21:29.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.651+0000 7f248a860b80 1 -- start start 2023-12-01T01:21:29.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.651+0000 7f248a860b80 1 Processor -- start 2023-12-01T01:21:29.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.651+0000 7f248a860b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562cb1ac5800 0x562cb1ad3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:29.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.651+0000 7f248a860b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cb1ac6500 0x562cb1ad2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:29.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.651+0000 7f248a860b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562cb1ac6e80 0x562cb1ad29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:29.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.651+0000 7f248a860b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cb1ad2f10 con 0x562cb1ac6500 2023-12-01T01:21:29.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.651+0000 7f248a860b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cb1ad3080 con 0x562cb1ac6e80 2023-12-01T01:21:29.655 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.651+0000 7f248a860b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cb1ad3240 con 0x562cb1ac5800 2023-12-01T01:21:29.656 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.651+0000 7f248a860b80 10 client.0 _ll_register_callbacks cb 0x7f246c003680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:21:29.656 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.651+0000 7f24777fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cb1ac6500 0x562cb1ad2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:29.656 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.651+0000 7f247cc6d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562cb1ac6e80 0x562cb1ad29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:29.656 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.651+0000 7f247cc6d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562cb1ac6e80 0x562cb1ad29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:39840/0 (socket says 172.21.15.77:39840) 2023-12-01T01:21:29.656 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.651+0000 7f24777fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cb1ac6500 0x562cb1ad2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60916/0 (socket says 172.21.15.77:60916) 2023-12-01T01:21:29.656 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.651+0000 7f247cc6d700 1 -- 172.21.15.77:0/894860924 learned_addr learned my addr 172.21.15.77:0/894860924 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:21:29.656 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.651+0000 7f2477fff700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562cb1ac5800 0x562cb1ad3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:29.656 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.652+0000 7f24777fe700 1 -- 172.21.15.77:0/894860924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562cb1ac5800 msgr2=0x562cb1ad3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:29.656 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.652+0000 7f24777fe700 1 --2- 172.21.15.77:0/894860924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562cb1ac5800 0x562cb1ad3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:29.656 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.652+0000 7f24777fe700 1 -- 172.21.15.77:0/894860924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562cb1ac6e80 msgr2=0x562cb1ad29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:29.656 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.652+0000 7f24777fe700 1 --2- 172.21.15.77:0/894860924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562cb1ac6e80 0x562cb1ad29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:29.656 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.652+0000 7f24777fe700 1 -- 172.21.15.77:0/894860924 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24781a7490 con 0x562cb1ac6500 2023-12-01T01:21:29.656 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.652+0000 7f24777fe700 1 --2- 172.21.15.77:0/894860924 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cb1ac6500 0x562cb1ad2490 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f246c032880 tx=0x7f246c032a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=717055f1c6756a8f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:21:29.656 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.652+0000 7f24577fe700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:21:29.657 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.652+0000 7f24577fe700 1 -- 172.21.15.77:0/894860924 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246c01fbb0 con 0x562cb1ac6500 2023-12-01T01:21:29.657 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.652+0000 7f24577fe700 1 -- 172.21.15.77:0/894860924 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f246c02e030 con 0x562cb1ac6500 2023-12-01T01:21:29.657 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.653+0000 7f248a860b80 1 -- 172.21.15.77:0/894860924 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f247000be10 con 0x562cb1ac6500 2023-12-01T01:21:29.657 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.653+0000 7f24577fe700 1 -- 172.21.15.77:0/894860924 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f246c01fd10 con 0x562cb1ac6500 2023-12-01T01:21:29.657 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.653+0000 7f248a860b80 10 client.16143 fetch_fsmap learned FSMap version 31 2023-12-01T01:21:29.657 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.653+0000 7f248a860b80 1 -- 172.21.15.77:0/894860924 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x562cb1866250 con 0x562cb1ac6500 2023-12-01T01:21:29.658 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.653+0000 7f24577fe700 1 -- 172.21.15.77:0/894860924 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f247000be10 con 0x562cb1ac6500 2023-12-01T01:21:29.658 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.653+0000 7f24577fe700 1 -- 172.21.15.77:0/894860924 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f246c03a400 con 0x562cb1ac6500 2023-12-01T01:21:29.659 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.653+0000 7f248a860b80 10 client.16143 fetch_fsmap finished waiting for FSMap version 31 2023-12-01T01:21:29.659 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.653+0000 7f248a860b80 10 client.16143 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-01T01:21:29.659 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.653+0000 7f248a860b80 20 client.16143 populate_metadata read hostname 'smithi077' 2023-12-01T01:21:29.659 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.653+0000 7f248a860b80 1 --2- 172.21.15.77:0/894860924 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x562cb1b2ddd0 0x562cb1b301f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:29.659 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.653+0000 7f248a860b80 4 client.16143 mds_command: new command op to 24500 tid=0[{"prefix": "scrub status"}] 2023-12-01T01:21:29.659 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.653+0000 7f248a860b80 1 -- 172.21.15.77:0/894860924 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562cb1a0ad40 con 0x562cb1b2ddd0 2023-12-01T01:21:29.659 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.654+0000 7f2477fff700 1 --2- 172.21.15.77:0/894860924 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x562cb1b2ddd0 0x562cb1b301f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:29.659 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.654+0000 7f24577fe700 10 client.16143 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:21:29.659 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.655+0000 7f2477fff700 1 --2- 172.21.15.77:0/894860924 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x562cb1b2ddd0 0x562cb1b301f0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc26dc83e616ac0e server_cookie=83083ba916eb0bcd in_seq=0 out_seq=0 2023-12-01T01:21:29.659 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.655+0000 7f24577fe700 1 -- 172.21.15.77:0/894860924 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x562cb1a0ad40 con 0x562cb1b2ddd0 2023-12-01T01:21:29.659 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.655+0000 7f24577fe700 10 client.16143 handle_command_reply: tid=0 2023-12-01T01:21:29.659 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.655+0000 7f248a860b80 1 client.16143 shutdown 2023-12-01T01:21:29.659 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.655+0000 7f248a860b80 1 -- 172.21.15.77:0/894860924 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cb1ac6500 msgr2=0x562cb1ad2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:29.660 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.655+0000 7f248a860b80 1 --2- 172.21.15.77:0/894860924 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cb1ac6500 0x562cb1ad2490 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f246c032880 tx=0x7f246c032a50 comp rx=0 tx=0).stop 2023-12-01T01:21:29.660 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.655+0000 7f248a860b80 1 -- 172.21.15.77:0/894860924 shutdown_connections 2023-12-01T01:21:29.660 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.655+0000 7f248a860b80 1 --2- 172.21.15.77:0/894860924 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x562cb1b2ddd0 0x562cb1b301f0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:29.660 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.655+0000 7f248a860b80 1 --2- 172.21.15.77:0/894860924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562cb1ac6e80 0x562cb1ad29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:29.660 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.655+0000 7f248a860b80 1 --2- 172.21.15.77:0/894860924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562cb1ac5800 0x562cb1ad3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:29.660 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.655+0000 7f248a860b80 1 --2- 172.21.15.77:0/894860924 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562cb1ac6500 0x562cb1ad2490 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:29.660 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.655+0000 7f248a860b80 1 -- 172.21.15.77:0/894860924 >> 172.21.15.77:0/894860924 conn(0x562cb1a200d0 msgr2=0x562cb1ac96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:21:29.660 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.655+0000 7f24577fe700 0 client.16143 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:21:29.660 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.656+0000 7f248a860b80 1 -- 172.21.15.77:0/894860924 shutdown_connections 2023-12-01T01:21:29.660 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.656+0000 7f248a860b80 1 -- 172.21.15.77:0/894860924 wait complete. 2023-12-01T01:21:29.660 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.656+0000 7f248a860b80 20 client.16143 trim_cache size 0 max 16384 2023-12-01T01:21:29.660 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2023-12-01T01:21:29.661 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2023-12-01T01:21:29.661 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2023-12-01T01:21:29.661 INFO:teuthology.orchestra.run.smithi077.stdout: "085290de-33b6-4527-ab7f-03f7dfd5dbba": { 2023-12-01T01:21:29.661 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2023-12-01T01:21:29.661 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "085290de-33b6-4527-ab7f-03f7dfd5dbba", 2023-12-01T01:21:29.661 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2023-12-01T01:21:29.661 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:21:29.661 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:21:29.661 INFO:teuthology.orchestra.run.smithi077.stdout:} 2023-12-01T01:21:29.662 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.656+0000 7f247e6d0700 1 -- 172.21.15.77:0/4244335880 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f24600a0520 msgr2=0x7f24600a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:29.662 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.656+0000 7f247e6d0700 1 --2- 172.21.15.77:0/4244335880 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f24600a0520 0x7f24600a29e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f246c011490 tx=0x7f246c009260 comp rx=0 tx=0).stop 2023-12-01T01:21:29.662 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.656+0000 7f247e6d0700 1 -- 172.21.15.77:0/4244335880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2478108000 msgr2=0x7f247819f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:29.662 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.656+0000 7f247e6d0700 1 --2- 172.21.15.77:0/4244335880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2478108000 0x7f247819f360 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f246800ea70 tx=0x7f246800ed80 comp rx=0 tx=0).stop 2023-12-01T01:21:29.662 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.657+0000 7f247e6d0700 1 -- 172.21.15.77:0/4244335880 shutdown_connections 2023-12-01T01:21:29.662 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.657+0000 7f247e6d0700 1 --2- 172.21.15.77:0/4244335880 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f24600a0520 0x7f24600a29e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:29.662 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.657+0000 7f247e6d0700 1 --2- 172.21.15.77:0/4244335880 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f247810d4e0 0x7f24781a6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:29.662 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.657+0000 7f247e6d0700 1 --2- 172.21.15.77:0/4244335880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2478108000 0x7f247819f360 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:29.663 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.657+0000 7f247e6d0700 1 --2- 172.21.15.77:0/4244335880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f247810a9c0 0x7f247819f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:29.663 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.657+0000 7f247e6d0700 1 -- 172.21.15.77:0/4244335880 >> 172.21.15.77:0/4244335880 conn(0x7f247806df90 msgr2=0x7f2478108c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:21:29.663 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.657+0000 7f247e6d0700 1 -- 172.21.15.77:0/4244335880 shutdown_connections 2023-12-01T01:21:29.663 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:29.657+0000 7f247e6d0700 1 -- 172.21.15.77:0/4244335880 wait complete. 2023-12-01T01:21:29.675 INFO:tasks.cephfs.filesystem:scrub status for tag:085290de-33b6-4527-ab7f-03f7dfd5dbba - {'path': '/', 'tag': '085290de-33b6-4527-ab7f-03f7dfd5dbba', 'options': 'recursive,force'} 2023-12-01T01:21:29.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:21:29 smithi077 ceph-mon[98867]: pgmap v406: 97 pgs: 97 active+clean; 58 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 3.1 MiB/s wr, 1.35k op/s 2023-12-01T01:21:30.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:21:29 smithi203 ceph-mon[103581]: pgmap v406: 97 pgs: 97 active+clean; 58 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 3.1 MiB/s wr, 1.35k op/s 2023-12-01T01:21:30.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:21:29 smithi137 ceph-mon[103312]: pgmap v406: 97 pgs: 97 active+clean; 58 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 3.1 MiB/s wr, 1.35k op/s 2023-12-01T01:21:32.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:21:31 smithi203 ceph-mon[103581]: pgmap v407: 97 pgs: 97 active+clean; 62 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 3.0 MiB/s wr, 1.32k op/s 2023-12-01T01:21:32.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:21:31 smithi137 ceph-mon[103312]: pgmap v407: 97 pgs: 97 active+clean; 62 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 3.0 MiB/s wr, 1.32k op/s 2023-12-01T01:21:32.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:21:31 smithi077 ceph-mon[98867]: pgmap v407: 97 pgs: 97 active+clean; 62 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 3.0 MiB/s wr, 1.32k op/s 2023-12-01T01:21:34.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:21:33 smithi203 ceph-mon[103581]: pgmap v408: 97 pgs: 97 active+clean; 60 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.7 MiB/s wr, 1.17k op/s 2023-12-01T01:21:34.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:21:33 smithi137 ceph-mon[103312]: pgmap v408: 97 pgs: 97 active+clean; 60 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.7 MiB/s wr, 1.17k op/s 2023-12-01T01:21:34.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:21:33 smithi077 ceph-mon[98867]: pgmap v408: 97 pgs: 97 active+clean; 60 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.7 MiB/s wr, 1.17k op/s 2023-12-01T01:21:35.283 INFO:tasks.workunit.client.0.smithi077.stdout:30000 mapread 0x11b12cb thru 0x11b885f (0x7595 bytes) 2023-12-01T01:21:36.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:21:35 smithi203 ceph-mon[103581]: pgmap v409: 97 pgs: 97 active+clean; 70 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 3.2 MiB/s wr, 1.40k op/s 2023-12-01T01:21:36.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:21:35 smithi137 ceph-mon[103312]: pgmap v409: 97 pgs: 97 active+clean; 70 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 3.2 MiB/s wr, 1.40k op/s 2023-12-01T01:21:36.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:21:35 smithi077 ceph-mon[98867]: pgmap v409: 97 pgs: 97 active+clean; 70 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 3.2 MiB/s wr, 1.40k op/s 2023-12-01T01:21:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.412+0000 7fc775696700 1 -- 172.21.15.77:0/1773432644 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc770101240 msgr2=0x7fc770101660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.412+0000 7fc775696700 1 --2- 172.21.15.77:0/1773432644 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc770101240 0x7fc770101660 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fc760009990 tx=0x7fc760009ca0 comp rx=0 tx=0).stop 2023-12-01T01:21:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.412+0000 7fc775696700 1 -- 172.21.15.77:0/1773432644 shutdown_connections 2023-12-01T01:21:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.412+0000 7fc775696700 1 --2- 172.21.15.77:0/1773432644 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc770106c00 0x7fc770108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.412+0000 7fc775696700 1 --2- 172.21.15.77:0/1773432644 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc770101240 0x7fc770101660 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.412+0000 7fc775696700 1 --2- 172.21.15.77:0/1773432644 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7701042d0 0x7fc7701066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.412+0000 7fc775696700 1 -- 172.21.15.77:0/1773432644 >> 172.21.15.77:0/1773432644 conn(0x7fc7700fa960 msgr2=0x7fc7700fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:21:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.412+0000 7fc775696700 1 -- 172.21.15.77:0/1773432644 shutdown_connections 2023-12-01T01:21:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.412+0000 7fc775696700 1 -- 172.21.15.77:0/1773432644 wait complete. 2023-12-01T01:21:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.413+0000 7fc775696700 1 Processor -- start 2023-12-01T01:21:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.413+0000 7fc775696700 1 -- start start 2023-12-01T01:21:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.413+0000 7fc775696700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc770101240 0x7fc770198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.413+0000 7fc775696700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7701042d0 0x7fc7701992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.413+0000 7fc775696700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc770106c00 0x7fc77019a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.413+0000 7fc775696700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77019a7a0 con 0x7fc770101240 2023-12-01T01:21:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.413+0000 7fc775696700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77019a910 con 0x7fc770106c00 2023-12-01T01:21:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.413+0000 7fc775696700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77019aad0 con 0x7fc7701042d0 2023-12-01T01:21:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.414+0000 7fc76effd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc770101240 0x7fc770198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.414+0000 7fc76effd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc770101240 0x7fc770198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60930/0 (socket says 172.21.15.77:60930) 2023-12-01T01:21:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.414+0000 7fc76effd700 1 -- 172.21.15.77:0/1894375365 learned_addr learned my addr 172.21.15.77:0/1894375365 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:21:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.414+0000 7fc76f7fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc770106c00 0x7fc77019a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.414+0000 7fc76e7fc700 1 --2- 172.21.15.77:0/1894375365 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7701042d0 0x7fc7701992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.414+0000 7fc76f7fe700 1 -- 172.21.15.77:0/1894375365 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7701042d0 msgr2=0x7fc7701992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.414+0000 7fc76f7fe700 1 --2- 172.21.15.77:0/1894375365 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7701042d0 0x7fc7701992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:36.417 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.414+0000 7fc76f7fe700 1 -- 172.21.15.77:0/1894375365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc770101240 msgr2=0x7fc770198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.414+0000 7fc76f7fe700 1 --2- 172.21.15.77:0/1894375365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc770101240 0x7fc770198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.414+0000 7fc76f7fe700 1 -- 172.21.15.77:0/1894375365 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7600095f0 con 0x7fc770106c00 2023-12-01T01:21:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.414+0000 7fc76effd700 1 --2- 172.21.15.77:0/1894375365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc770101240 0x7fc770198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:21:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.414+0000 7fc76f7fe700 1 --2- 172.21.15.77:0/1894375365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc770106c00 0x7fc77019a0c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc768007f10 tx=0x7fc76800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:21:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.414+0000 7fc767fff700 1 -- 172.21.15.77:0/1894375365 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc768004db0 con 0x7fc770106c00 2023-12-01T01:21:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.415+0000 7fc775696700 1 -- 172.21.15.77:0/1894375365 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc77019adb0 con 0x7fc770106c00 2023-12-01T01:21:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.415+0000 7fc767fff700 1 -- 172.21.15.77:0/1894375365 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc76800f040 con 0x7fc770106c00 2023-12-01T01:21:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.415+0000 7fc767fff700 1 -- 172.21.15.77:0/1894375365 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc768005990 con 0x7fc770106c00 2023-12-01T01:21:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.415+0000 7fc775696700 1 -- 172.21.15.77:0/1894375365 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7701a14f0 con 0x7fc770106c00 2023-12-01T01:21:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.417+0000 7fc767fff700 1 -- 172.21.15.77:0/1894375365 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fc76800db20 con 0x7fc770106c00 2023-12-01T01:21:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.418+0000 7fc767fff700 1 --2- 172.21.15.77:0/1894375365 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc75c0a0520 0x7fc75c0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.418+0000 7fc76effd700 1 --2- 172.21.15.77:0/1894375365 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc75c0a0520 0x7fc75c0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:36.419 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.418+0000 7fc767fff700 1 -- 172.21.15.77:0/1894375365 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fc7680cb1d0 con 0x7fc770106c00 2023-12-01T01:21:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.418+0000 7fc76effd700 1 --2- 172.21.15.77:0/1894375365 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc75c0a0520 0x7fc75c0a29e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc76000b680 tx=0x7fc76000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:21:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.419+0000 7fc781826b80 1 Processor -- start 2023-12-01T01:21:36.420 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.419+0000 7fc781826b80 1 -- start start 2023-12-01T01:21:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.420+0000 7fc781826b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55becabdb680 0x55becabdbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:36.421 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.420+0000 7fc781826b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55becabdbfe0 0x55becabdc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.420+0000 7fc781826b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55becabdced0 0x55becabdd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.420+0000 7fc781826b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55becab339b0 con 0x55becabdb680 2023-12-01T01:21:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.420+0000 7fc781826b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55becab31370 con 0x55becabdbfe0 2023-12-01T01:21:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.420+0000 7fc781826b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55becab2cc60 con 0x55becabdced0 2023-12-01T01:21:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.420+0000 7fc76effd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55becabdb680 0x55becabdbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.420+0000 7fc76effd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55becabdb680 0x55becabdbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60938/0 (socket says 172.21.15.77:60938) 2023-12-01T01:21:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.420+0000 7fc76effd700 1 -- 172.21.15.77:0/3445961561 learned_addr learned my addr 172.21.15.77:0/3445961561 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:21:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.420+0000 7fc76f7fe700 1 --2- 172.21.15.77:0/3445961561 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55becabdced0 0x55becabdd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.420+0000 7fc76e7fc700 1 --2- 172.21.15.77:0/3445961561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55becabdbfe0 0x55becabdc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:36.423 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.421+0000 7fc76f7fe700 1 -- 172.21.15.77:0/3445961561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55becabdbfe0 msgr2=0x55becabdc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.421+0000 7fc76f7fe700 1 --2- 172.21.15.77:0/3445961561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55becabdbfe0 0x55becabdc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.421+0000 7fc76f7fe700 1 -- 172.21.15.77:0/3445961561 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55becabdb680 msgr2=0x55becabdbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.421+0000 7fc76f7fe700 1 --2- 172.21.15.77:0/3445961561 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55becabdb680 0x55becabdbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.421+0000 7fc76f7fe700 1 -- 172.21.15.77:0/3445961561 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55beca95ad80 con 0x55becabdced0 2023-12-01T01:21:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.421+0000 7fc76e7fc700 1 --2- 172.21.15.77:0/3445961561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55becabdbfe0 0x55becabdc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:21:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.421+0000 7fc76f7fe700 1 --2- 172.21.15.77:0/3445961561 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55becabdced0 0x55becabdd350 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc7680c8dc0 tx=0x7fc7680053b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6eee84381ed6d07 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:21:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.421+0000 7fc7657fa700 1 -- 172.21.15.77:0/3445961561 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7680991d0 con 0x55becabdced0 2023-12-01T01:21:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.421+0000 7fc7657fa700 1 -- 172.21.15.77:0/3445961561 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc768098ad0 con 0x55becabdced0 2023-12-01T01:21:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.421+0000 7fc7657fa700 1 -- 172.21.15.77:0/3445961561 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7680b8070 con 0x55becabdced0 2023-12-01T01:21:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.422+0000 7fc781826b80 1 -- 172.21.15.77:0/3445961561 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55becabdced0 msgr2=0x55becabdd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.422+0000 7fc781826b80 1 --2- 172.21.15.77:0/3445961561 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55becabdced0 0x55becabdd350 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc7680c8dc0 tx=0x7fc7680053b0 comp rx=0 tx=0).stop 2023-12-01T01:21:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.422+0000 7fc781826b80 1 -- 172.21.15.77:0/3445961561 shutdown_connections 2023-12-01T01:21:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.422+0000 7fc781826b80 1 --2- 172.21.15.77:0/3445961561 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55becabdced0 0x55becabdd350 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:36.424 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.422+0000 7fc781826b80 1 --2- 172.21.15.77:0/3445961561 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55becabdb680 0x55becabdbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.422+0000 7fc781826b80 1 --2- 172.21.15.77:0/3445961561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55becabdbfe0 0x55becabdc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.422+0000 7fc781826b80 1 -- 172.21.15.77:0/3445961561 >> 172.21.15.77:0/3445961561 conn(0x55becab3b070 msgr2=0x55becabd8040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:21:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.422+0000 7fc781826b80 1 -- 172.21.15.77:0/3445961561 shutdown_connections 2023-12-01T01:21:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.422+0000 7fc781826b80 1 -- 172.21.15.77:0/3445961561 wait complete. 2023-12-01T01:21:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.423+0000 7fc781826b80 1 -- start start 2023-12-01T01:21:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.423+0000 7fc781826b80 1 Processor -- start 2023-12-01T01:21:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.423+0000 7fc781826b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55becabdb680 0x55becabe5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.423+0000 7fc781826b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55becabdbfe0 0x55becabe5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.423+0000 7fc781826b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55becabdced0 0x55becabecad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.423+0000 7fc781826b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55becaac94b0 con 0x55becabdbfe0 2023-12-01T01:21:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.423+0000 7fc781826b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55becaac9150 con 0x55becabdced0 2023-12-01T01:21:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.423+0000 7fc781826b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beca9b08c0 con 0x55becabdb680 2023-12-01T01:21:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.423+0000 7fc781826b80 10 client.0 _ll_register_callbacks cb 0x55becaa16a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:21:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.423+0000 7fc76effd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55becabdb680 0x55becabe5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.423+0000 7fc76effd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55becabdb680 0x55becabe5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:47130/0 (socket says 172.21.15.77:47130) 2023-12-01T01:21:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.423+0000 7fc76effd700 1 -- 172.21.15.77:0/3187283504 learned_addr learned my addr 172.21.15.77:0/3187283504 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:21:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.423+0000 7fc76f7fe700 1 --2- 172.21.15.77:0/3187283504 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55becabdced0 0x55becabecad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.424+0000 7fc76e7fc700 1 --2- 172.21.15.77:0/3187283504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55becabdbfe0 0x55becabe5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.424+0000 7fc76effd700 1 -- 172.21.15.77:0/3187283504 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55becabdced0 msgr2=0x55becabecad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.424+0000 7fc76effd700 1 --2- 172.21.15.77:0/3187283504 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55becabdced0 0x55becabecad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.424+0000 7fc76effd700 1 -- 172.21.15.77:0/3187283504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55becabdbfe0 msgr2=0x55becabe5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.424+0000 7fc76effd700 1 --2- 172.21.15.77:0/3187283504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55becabdbfe0 0x55becabe5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.424+0000 7fc76effd700 1 -- 172.21.15.77:0/3187283504 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7701a14f0 con 0x55becabdb680 2023-12-01T01:21:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.424+0000 7fc76f7fe700 1 --2- 172.21.15.77:0/3187283504 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55becabdced0 0x55becabecad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:21:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.424+0000 7fc76effd700 1 --2- 172.21.15.77:0/3187283504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55becabdb680 0x55becabe5550 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc760004c60 tx=0x7fc760020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b4940f59f6c6fd1 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:21:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.424+0000 7fc7567fc700 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2023-12-01T01:21:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.424+0000 7fc7567fc700 1 -- 172.21.15.77:0/3187283504 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76002d040 con 0x55becabdb680 2023-12-01T01:21:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.424+0000 7fc7567fc700 1 -- 172.21.15.77:0/3187283504 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fc76000fa70 con 0x55becabdb680 2023-12-01T01:21:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.424+0000 7fc781826b80 1 -- 172.21.15.77:0/3187283504 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55becab57cb0 con 0x55becabdb680 2023-12-01T01:21:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.425+0000 7fc7567fc700 1 -- 172.21.15.77:0/3187283504 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7600056d0 con 0x55becabdb680 2023-12-01T01:21:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.425+0000 7fc781826b80 10 client.25747 fetch_fsmap learned FSMap version 31 2023-12-01T01:21:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.425+0000 7fc781826b80 1 -- 172.21.15.77:0/3187283504 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55beca943c00 con 0x55becabdb680 2023-12-01T01:21:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.425+0000 7fc7567fc700 1 -- 172.21.15.77:0/3187283504 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55becab57cb0 con 0x55becabdb680 2023-12-01T01:21:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.425+0000 7fc7567fc700 1 -- 172.21.15.77:0/3187283504 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc760024410 con 0x55becabdb680 2023-12-01T01:21:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.425+0000 7fc781826b80 10 client.25747 fetch_fsmap finished waiting for FSMap version 31 2023-12-01T01:21:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.425+0000 7fc781826b80 10 client.25747 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-01T01:21:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.425+0000 7fc781826b80 20 client.25747 populate_metadata read hostname 'smithi077' 2023-12-01T01:21:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.425+0000 7fc781826b80 1 --2- 172.21.15.77:0/3187283504 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55becabf9c40 0x55becabfc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.425+0000 7fc781826b80 4 client.25747 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:21:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.425+0000 7fc781826b80 1 -- 172.21.15.77:0/3187283504 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55becabaadb0 con 0x55becabf9c40 2023-12-01T01:21:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.425+0000 7fc76e7fc700 1 --2- 172.21.15.77:0/3187283504 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55becabf9c40 0x55becabfc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.426+0000 7fc7567fc700 10 client.25747 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:21:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.426+0000 7fc76e7fc700 1 --2- 172.21.15.77:0/3187283504 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55becabf9c40 0x55becabfc060 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d656bb4ad6ecf107 server_cookie=b0aa61c9414d2327 in_seq=0 out_seq=0 2023-12-01T01:21:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.426+0000 7fc7567fc700 1 -- 172.21.15.77:0/3187283504 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55becabaadb0 con 0x55becabf9c40 2023-12-01T01:21:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.426+0000 7fc7567fc700 10 client.25747 handle_command_reply: tid=0 2023-12-01T01:21:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.426+0000 7fc781826b80 1 client.25747 shutdown 2023-12-01T01:21:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.426+0000 7fc781826b80 1 -- 172.21.15.77:0/3187283504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55becabdb680 msgr2=0x55becabe5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.426+0000 7fc781826b80 1 --2- 172.21.15.77:0/3187283504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55becabdb680 0x55becabe5550 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc760004c60 tx=0x7fc760020000 comp rx=0 tx=0).stop 2023-12-01T01:21:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.426+0000 7fc781826b80 1 -- 172.21.15.77:0/3187283504 shutdown_connections 2023-12-01T01:21:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.426+0000 7fc781826b80 1 --2- 172.21.15.77:0/3187283504 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55becabf9c40 0x55becabfc060 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.426+0000 7fc781826b80 1 --2- 172.21.15.77:0/3187283504 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55becabdced0 0x55becabecad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.426+0000 7fc781826b80 1 --2- 172.21.15.77:0/3187283504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55becabdb680 0x55becabe5550 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.426+0000 7fc7567fc700 0 client.25747 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:21:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.426+0000 7fc781826b80 1 --2- 172.21.15.77:0/3187283504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55becabdbfe0 0x55becabe5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.426+0000 7fc781826b80 1 -- 172.21.15.77:0/3187283504 >> 172.21.15.77:0/3187283504 conn(0x55becab3b070 msgr2=0x55becabd63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:21:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.427+0000 7fc781826b80 1 -- 172.21.15.77:0/3187283504 shutdown_connections 2023-12-01T01:21:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.427+0000 7fc781826b80 1 -- 172.21.15.77:0/3187283504 wait complete. 2023-12-01T01:21:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.427+0000 7fc781826b80 20 client.25747 trim_cache size 0 max 16384 2023-12-01T01:21:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.445+0000 7fc781826b80 1 Processor -- start 2023-12-01T01:21:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.445+0000 7fc781826b80 1 -- start start 2023-12-01T01:21:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.446+0000 7fc781826b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55becabdb680 0x55becabd7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.446+0000 7fc781826b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55becabdbfe0 0x55becabeaa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.446+0000 7fc781826b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55becabdced0 0x55becabe4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.446+0000 7fc781826b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55becabce7d0 con 0x55becabdbfe0 2023-12-01T01:21:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.446+0000 7fc781826b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55becabcea20 con 0x55becabdb680 2023-12-01T01:21:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.446+0000 7fc781826b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55becabe4a00 con 0x55becabdced0 2023-12-01T01:21:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.446+0000 7fc76e7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55becabdbfe0 0x55becabeaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.446+0000 7fc76e7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55becabdbfe0 0x55becabeaa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60950/0 (socket says 172.21.15.77:60950) 2023-12-01T01:21:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.446+0000 7fc76e7fc700 1 -- 172.21.15.77:0/3029629679 learned_addr learned my addr 172.21.15.77:0/3029629679 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:21:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.446+0000 7fc76effd700 1 --2- 172.21.15.77:0/3029629679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55becabdb680 0x55becabd7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.446+0000 7fc76f7fe700 1 --2- 172.21.15.77:0/3029629679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55becabdced0 0x55becabe4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.447+0000 7fc76effd700 1 -- 172.21.15.77:0/3029629679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55becabdced0 msgr2=0x55becabe4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.447+0000 7fc76effd700 1 --2- 172.21.15.77:0/3029629679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55becabdced0 0x55becabe4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.447+0000 7fc76effd700 1 -- 172.21.15.77:0/3029629679 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55becabdbfe0 msgr2=0x55becabeaa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.447+0000 7fc76effd700 1 --2- 172.21.15.77:0/3029629679 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55becabdbfe0 0x55becabeaa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.447+0000 7fc76effd700 1 -- 172.21.15.77:0/3029629679 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55beca9811f0 con 0x55becabdb680 2023-12-01T01:21:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.447+0000 7fc76e7fc700 1 --2- 172.21.15.77:0/3029629679 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55becabdbfe0 0x55becabeaa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:21:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.447+0000 7fc76effd700 1 --2- 172.21.15.77:0/3029629679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55becabdb680 0x55becabd7490 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc7600247f0 tx=0x7fc760005ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c15fff7c857fe655 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:21:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.447+0000 7fc765ffb700 1 -- 172.21.15.77:0/3029629679 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76002d040 con 0x55becabdb680 2023-12-01T01:21:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.447+0000 7fc765ffb700 1 -- 172.21.15.77:0/3029629679 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc76001e040 con 0x55becabdb680 2023-12-01T01:21:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.447+0000 7fc765ffb700 1 -- 172.21.15.77:0/3029629679 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc760032690 con 0x55becabdb680 2023-12-01T01:21:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.448+0000 7fc781826b80 1 -- 172.21.15.77:0/3029629679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55becabdb680 msgr2=0x55becabd7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.448+0000 7fc781826b80 1 --2- 172.21.15.77:0/3029629679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55becabdb680 0x55becabd7490 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc7600247f0 tx=0x7fc760005ea0 comp rx=0 tx=0).stop 2023-12-01T01:21:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.448+0000 7fc781826b80 1 -- 172.21.15.77:0/3029629679 shutdown_connections 2023-12-01T01:21:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.448+0000 7fc781826b80 1 --2- 172.21.15.77:0/3029629679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55becabdced0 0x55becabe4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.448+0000 7fc781826b80 1 --2- 172.21.15.77:0/3029629679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55becabdb680 0x55becabd7490 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.448+0000 7fc781826b80 1 --2- 172.21.15.77:0/3029629679 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55becabdbfe0 0x55becabeaa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.448+0000 7fc781826b80 1 -- 172.21.15.77:0/3029629679 >> 172.21.15.77:0/3029629679 conn(0x55becab3b070 msgr2=0x55becabe4170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:21:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.448+0000 7fc781826b80 1 -- 172.21.15.77:0/3029629679 shutdown_connections 2023-12-01T01:21:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.448+0000 7fc781826b80 1 -- 172.21.15.77:0/3029629679 wait complete. 2023-12-01T01:21:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.449+0000 7fc781826b80 1 -- start start 2023-12-01T01:21:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.449+0000 7fc781826b80 1 Processor -- start 2023-12-01T01:21:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.449+0000 7fc781826b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55becabdb680 0x55becabe9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.449+0000 7fc781826b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55becabdbfe0 0x55becabea410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.449+0000 7fc781826b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55becabdced0 0x55becabf3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.449+0000 7fc781826b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55becabf39a0 con 0x55becabdbfe0 2023-12-01T01:21:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.449+0000 7fc781826b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55becabf3ae0 con 0x55becabdb680 2023-12-01T01:21:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.449+0000 7fc781826b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55becabf3ca0 con 0x55becabdced0 2023-12-01T01:21:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.449+0000 7fc781826b80 10 client.0 _ll_register_callbacks cb 0x7fc760004810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:21:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.449+0000 7fc76effd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55becabdb680 0x55becabe9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.450+0000 7fc76effd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55becabdb680 0x55becabe9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:39878/0 (socket says 172.21.15.77:39878) 2023-12-01T01:21:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.450+0000 7fc76effd700 1 -- 172.21.15.77:0/3099233030 learned_addr learned my addr 172.21.15.77:0/3099233030 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:21:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.450+0000 7fc76e7fc700 1 --2- 172.21.15.77:0/3099233030 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55becabdbfe0 0x55becabea410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.450+0000 7fc76f7fe700 1 --2- 172.21.15.77:0/3099233030 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55becabdced0 0x55becabf3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.450+0000 7fc76effd700 1 -- 172.21.15.77:0/3099233030 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55becabdced0 msgr2=0x55becabf3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.450+0000 7fc76effd700 1 --2- 172.21.15.77:0/3099233030 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55becabdced0 0x55becabf3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.450+0000 7fc76effd700 1 -- 172.21.15.77:0/3099233030 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55becabdbfe0 msgr2=0x55becabea410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.450+0000 7fc76effd700 1 --2- 172.21.15.77:0/3099233030 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55becabdbfe0 0x55becabea410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.450+0000 7fc76effd700 1 -- 172.21.15.77:0/3099233030 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc758009dc0 con 0x55becabdb680 2023-12-01T01:21:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.450+0000 7fc76effd700 1 --2- 172.21.15.77:0/3099233030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55becabdb680 0x55becabe9ed0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc7600098e0 tx=0x7fc7600092d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23f43f1560e2a33f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:21:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.450+0000 7fc7567fc700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2023-12-01T01:21:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.450+0000 7fc7567fc700 1 -- 172.21.15.77:0/3099233030 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76002d040 con 0x55becabdb680 2023-12-01T01:21:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.451+0000 7fc781826b80 1 -- 172.21.15.77:0/3099233030 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc768098ad0 con 0x55becabdb680 2023-12-01T01:21:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.450+0000 7fc7567fc700 1 -- 172.21.15.77:0/3099233030 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fc760038030 con 0x55becabdb680 2023-12-01T01:21:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.451+0000 7fc7567fc700 1 -- 172.21.15.77:0/3099233030 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc76000f940 con 0x55becabdb680 2023-12-01T01:21:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.451+0000 7fc781826b80 10 client.25742 fetch_fsmap learned FSMap version 31 2023-12-01T01:21:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.451+0000 7fc781826b80 1 -- 172.21.15.77:0/3099233030 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55beca943c00 con 0x55becabdb680 2023-12-01T01:21:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.451+0000 7fc7567fc700 1 -- 172.21.15.77:0/3099233030 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc768098ad0 con 0x55becabdb680 2023-12-01T01:21:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.451+0000 7fc7567fc700 1 -- 172.21.15.77:0/3099233030 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc760024d20 con 0x55becabdb680 2023-12-01T01:21:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.451+0000 7fc781826b80 10 client.25742 fetch_fsmap finished waiting for FSMap version 31 2023-12-01T01:21:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.451+0000 7fc781826b80 10 client.25742 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-01T01:21:36.456 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.451+0000 7fc781826b80 20 client.25742 populate_metadata read hostname 'smithi077' 2023-12-01T01:21:36.456 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.451+0000 7fc781826b80 1 --2- 172.21.15.77:0/3099233030 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55becabf51f0 0x55becac456e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:36.456 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.451+0000 7fc781826b80 4 client.25742 mds_command: new command op to 24500 tid=0[{"prefix": "flush journal"}] 2023-12-01T01:21:36.456 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.451+0000 7fc781826b80 1 -- 172.21.15.77:0/3099233030 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55becab25ce0 con 0x55becabf51f0 2023-12-01T01:21:36.456 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.452+0000 7fc76e7fc700 1 --2- 172.21.15.77:0/3099233030 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55becabf51f0 0x55becac456e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:36.456 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.452+0000 7fc7567fc700 10 client.25742 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:21:36.456 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.452+0000 7fc76e7fc700 1 --2- 172.21.15.77:0/3099233030 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55becabf51f0 0x55becac456e0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dc9c1a11a685fdd server_cookie=b986c4af979d74b9 in_seq=0 out_seq=0 2023-12-01T01:21:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.470+0000 7fc7567fc700 1 -- 172.21.15.77:0/3099233030 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55becab25ce0 con 0x55becabf51f0 2023-12-01T01:21:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.470+0000 7fc7567fc700 10 client.25742 handle_command_reply: tid=0 2023-12-01T01:21:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.470+0000 7fc781826b80 1 client.25742 shutdown 2023-12-01T01:21:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.470+0000 7fc781826b80 1 -- 172.21.15.77:0/3099233030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55becabdb680 msgr2=0x55becabe9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.470+0000 7fc781826b80 1 --2- 172.21.15.77:0/3099233030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55becabdb680 0x55becabe9ed0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc7600098e0 tx=0x7fc7600092d0 comp rx=0 tx=0).stop 2023-12-01T01:21:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.470+0000 7fc781826b80 1 -- 172.21.15.77:0/3099233030 shutdown_connections 2023-12-01T01:21:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.470+0000 7fc781826b80 1 --2- 172.21.15.77:0/3099233030 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55becabdced0 0x55becabf3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:36.472 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.470+0000 7fc781826b80 1 --2- 172.21.15.77:0/3099233030 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55becabf51f0 0x55becac456e0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.470+0000 7fc781826b80 1 --2- 172.21.15.77:0/3099233030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55becabdb680 0x55becabe9ed0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.470+0000 7fc781826b80 1 --2- 172.21.15.77:0/3099233030 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55becabdbfe0 0x55becabea410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.470+0000 7fc781826b80 1 -- 172.21.15.77:0/3099233030 >> 172.21.15.77:0/3099233030 conn(0x55becab3b070 msgr2=0x55becabe4b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:21:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.470+0000 7fc7567fc700 0 client.25742 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:21:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.470+0000 7fc781826b80 1 -- 172.21.15.77:0/3099233030 shutdown_connections 2023-12-01T01:21:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.470+0000 7fc781826b80 1 -- 172.21.15.77:0/3099233030 wait complete. 2023-12-01T01:21:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.470+0000 7fc781826b80 20 client.25742 trim_cache size 0 max 16384 2023-12-01T01:21:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout:{ 2023-12-01T01:21:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout: "message": "", 2023-12-01T01:21:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout: "return_code": 0 2023-12-01T01:21:36.473 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout:} 2023-12-01T01:21:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.474+0000 7fc775696700 1 -- 172.21.15.77:0/1894375365 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc75c0a0520 msgr2=0x7fc75c0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.474+0000 7fc775696700 1 --2- 172.21.15.77:0/1894375365 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc75c0a0520 0x7fc75c0a29e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc76000b680 tx=0x7fc76000b260 comp rx=0 tx=0).stop 2023-12-01T01:21:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.474+0000 7fc775696700 1 -- 172.21.15.77:0/1894375365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc770106c00 msgr2=0x7fc77019a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.474+0000 7fc775696700 1 --2- 172.21.15.77:0/1894375365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc770106c00 0x7fc77019a0c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc768007f10 tx=0x7fc76800d3b0 comp rx=0 tx=0).stop 2023-12-01T01:21:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.474+0000 7fc775696700 1 -- 172.21.15.77:0/1894375365 shutdown_connections 2023-12-01T01:21:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.474+0000 7fc775696700 1 --2- 172.21.15.77:0/1894375365 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc75c0a0520 0x7fc75c0a29e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.474+0000 7fc775696700 1 --2- 172.21.15.77:0/1894375365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc770106c00 0x7fc77019a0c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.474+0000 7fc775696700 1 --2- 172.21.15.77:0/1894375365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc770101240 0x7fc770198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.474+0000 7fc775696700 1 --2- 172.21.15.77:0/1894375365 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7701042d0 0x7fc7701992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.474+0000 7fc775696700 1 -- 172.21.15.77:0/1894375365 >> 172.21.15.77:0/1894375365 conn(0x7fc7700fa960 msgr2=0x7fc770104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:21:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.474+0000 7fc775696700 1 -- 172.21.15.77:0/1894375365 shutdown_connections 2023-12-01T01:21:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:36.474+0000 7fc775696700 1 -- 172.21.15.77:0/1894375365 wait complete. 2023-12-01T01:21:37.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:21:36 smithi203 ceph-mon[103581]: pgmap v410: 97 pgs: 97 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.7 MiB/s wr, 1.17k op/s 2023-12-01T01:21:37.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:21:36 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:21:37.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:21:36 smithi137 ceph-mon[103312]: pgmap v410: 97 pgs: 97 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.7 MiB/s wr, 1.17k op/s 2023-12-01T01:21:37.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:21:36 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:21:37.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:21:36 smithi077 ceph-mon[98867]: pgmap v410: 97 pgs: 97 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.7 MiB/s wr, 1.17k op/s 2023-12-01T01:21:37.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:21:36 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:21:38.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:21:37 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:21:38.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:21:37 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:21:38.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:21:37 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:21:38.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:21:37 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:21:38.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:21:37 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:21:38.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:21:37 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:21:38.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:21:37 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:21:38.262 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:21:37 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:21:38.262 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:21:37 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:21:38.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:21:37 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:21:38.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:21:37 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:21:38.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:21:37 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:21:38.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:21:37 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:21:38.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:21:37 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:21:38.281 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:21:37 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:21:38.281 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:21:37 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:21:38.281 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:21:37 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:21:38.281 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:21:37 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:21:38.281 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:21:37 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:21:38.281 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:21:37 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:21:38.281 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:21:37 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:21:39.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:21:38 smithi077 ceph-mon[98867]: pgmap v411: 97 pgs: 97 active+clean; 68 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 3.1 MiB/s wr, 1.31k op/s 2023-12-01T01:21:39.262 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:21:38 smithi203 ceph-mon[103581]: pgmap v411: 97 pgs: 97 active+clean; 68 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 3.1 MiB/s wr, 1.31k op/s 2023-12-01T01:21:39.278 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:21:38 smithi137 ceph-mon[103312]: pgmap v411: 97 pgs: 97 active+clean; 68 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 3.1 MiB/s wr, 1.31k op/s 2023-12-01T01:21:41.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:21:41 smithi077 ceph-mon[98867]: pgmap v412: 97 pgs: 97 active+clean; 66 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.9 MiB/s wr, 1.23k op/s 2023-12-01T01:21:41.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:21:41 smithi203 ceph-mon[103581]: pgmap v412: 97 pgs: 97 active+clean; 66 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.9 MiB/s wr, 1.23k op/s 2023-12-01T01:21:41.778 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:21:41 smithi137 ceph-mon[103312]: pgmap v412: 97 pgs: 97 active+clean; 66 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.9 MiB/s wr, 1.23k op/s 2023-12-01T01:21:44.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:21:43 smithi203 ceph-mon[103581]: pgmap v413: 97 pgs: 97 active+clean; 69 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 3.0 MiB/s wr, 1.22k op/s 2023-12-01T01:21:44.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:21:43 smithi137 ceph-mon[103312]: pgmap v413: 97 pgs: 97 active+clean; 69 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 3.0 MiB/s wr, 1.22k op/s 2023-12-01T01:21:44.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:21:43 smithi077 ceph-mon[98867]: pgmap v413: 97 pgs: 97 active+clean; 69 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 3.0 MiB/s wr, 1.22k op/s 2023-12-01T01:21:45.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:21:44 smithi203 ceph-mon[103581]: pgmap v414: 97 pgs: 97 active+clean; 62 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 3.3 MiB/s wr, 1.41k op/s 2023-12-01T01:21:45.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:21:44 smithi137 ceph-mon[103312]: pgmap v414: 97 pgs: 97 active+clean; 62 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 3.3 MiB/s wr, 1.41k op/s 2023-12-01T01:21:45.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:21:44 smithi077 ceph-mon[98867]: pgmap v414: 97 pgs: 97 active+clean; 62 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 3.3 MiB/s wr, 1.41k op/s 2023-12-01T01:21:46.626 INFO:tasks.workunit.client.0.smithi077.stdout:40000 copy from 0x98ca82 to 0x993e9d, (0x741b bytes) at 0xfd3925 2023-12-01T01:21:48.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:21:47 smithi203 ceph-mon[103581]: pgmap v415: 97 pgs: 97 active+clean; 68 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.9 MiB/s wr, 1.21k op/s 2023-12-01T01:21:48.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:21:47 smithi137 ceph-mon[103312]: pgmap v415: 97 pgs: 97 active+clean; 68 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.9 MiB/s wr, 1.21k op/s 2023-12-01T01:21:48.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:21:47 smithi077 ceph-mon[98867]: pgmap v415: 97 pgs: 97 active+clean; 68 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.9 MiB/s wr, 1.21k op/s 2023-12-01T01:21:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.616+0000 7fa5a34a9700 1 -- 172.21.15.77:0/2818535437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa59c101820 msgr2=0x7fa59c1080d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.616+0000 7fa5a34a9700 1 --2- 172.21.15.77:0/2818535437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa59c101820 0x7fa59c1080d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa58c0099c0 tx=0x7fa58c009cd0 comp rx=0 tx=0).stop 2023-12-01T01:21:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.616+0000 7fa5a34a9700 1 -- 172.21.15.77:0/2818535437 shutdown_connections 2023-12-01T01:21:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.616+0000 7fa5a34a9700 1 --2- 172.21.15.77:0/2818535437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa59c101820 0x7fa59c1080d0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.616+0000 7fa5a34a9700 1 --2- 172.21.15.77:0/2818535437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa59c0ffc90 0x7fa59c1000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.616+0000 7fa5a34a9700 1 --2- 172.21.15.77:0/2818535437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa59c100e80 0x7fa59c1012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.616+0000 7fa5a34a9700 1 -- 172.21.15.77:0/2818535437 >> 172.21.15.77:0/2818535437 conn(0x7fa59c0fb1e0 msgr2=0x7fa59c0fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:21:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.616+0000 7fa5a34a9700 1 -- 172.21.15.77:0/2818535437 shutdown_connections 2023-12-01T01:21:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.616+0000 7fa5a34a9700 1 -- 172.21.15.77:0/2818535437 wait complete. 2023-12-01T01:21:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.617+0000 7fa5a34a9700 1 Processor -- start 2023-12-01T01:21:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.617+0000 7fa5a34a9700 1 -- start start 2023-12-01T01:21:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.617+0000 7fa5a34a9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa59c0ffc90 0x7fa59c196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.617+0000 7fa5a34a9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa59c100e80 0x7fa59c196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.617+0000 7fa5a34a9700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa59c101820 0x7fa59c197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.617+0000 7fa5a34a9700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c1984c0 con 0x7fa59c100e80 2023-12-01T01:21:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.617+0000 7fa5a34a9700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c198630 con 0x7fa59c101820 2023-12-01T01:21:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.617+0000 7fa5a34a9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c1987f0 con 0x7fa59c0ffc90 2023-12-01T01:21:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.617+0000 7fa5a0a44700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa59c100e80 0x7fa59c196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.617+0000 7fa5a0a44700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa59c100e80 0x7fa59c196fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60972/0 (socket says 172.21.15.77:60972) 2023-12-01T01:21:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.617+0000 7fa5a0a44700 1 -- 172.21.15.77:0/3132903429 learned_addr learned my addr 172.21.15.77:0/3132903429 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:21:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.617+0000 7fa5a1245700 1 --2- 172.21.15.77:0/3132903429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa59c0ffc90 0x7fa59c196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.617+0000 7fa5a1a46700 1 --2- 172.21.15.77:0/3132903429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa59c101820 0x7fa59c197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.617+0000 7fa5a0a44700 1 -- 172.21.15.77:0/3132903429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa59c0ffc90 msgr2=0x7fa59c196aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.617+0000 7fa5a0a44700 1 --2- 172.21.15.77:0/3132903429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa59c0ffc90 0x7fa59c196aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.617+0000 7fa5a0a44700 1 -- 172.21.15.77:0/3132903429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa59c101820 msgr2=0x7fa59c197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.617+0000 7fa5a0a44700 1 --2- 172.21.15.77:0/3132903429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa59c101820 0x7fa59c197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.617+0000 7fa5a0a44700 1 -- 172.21.15.77:0/3132903429 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa58c0095f0 con 0x7fa59c100e80 2023-12-01T01:21:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.617+0000 7fa5a1a46700 1 --2- 172.21.15.77:0/3132903429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa59c101820 0x7fa59c197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:21:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.617+0000 7fa5a1245700 1 --2- 172.21.15.77:0/3132903429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa59c0ffc90 0x7fa59c196aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:21:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.618+0000 7fa5a0a44700 1 --2- 172.21.15.77:0/3132903429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa59c100e80 0x7fa59c196fe0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa59000c9b0 tx=0x7fa59000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:21:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.618+0000 7fa59a7fc700 1 -- 172.21.15.77:0/3132903429 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa590007b00 con 0x7fa59c100e80 2023-12-01T01:21:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.618+0000 7fa5a34a9700 1 -- 172.21.15.77:0/3132903429 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa59c198ad0 con 0x7fa59c100e80 2023-12-01T01:21:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.618+0000 7fa59a7fc700 1 -- 172.21.15.77:0/3132903429 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa59000ce50 con 0x7fa59c100e80 2023-12-01T01:21:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.618+0000 7fa5a34a9700 1 -- 172.21.15.77:0/3132903429 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa59c19f290 con 0x7fa59c100e80 2023-12-01T01:21:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.618+0000 7fa59a7fc700 1 -- 172.21.15.77:0/3132903429 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa590018660 con 0x7fa59c100e80 2023-12-01T01:21:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.621+0000 7fa59a7fc700 1 -- 172.21.15.77:0/3132903429 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fa59000f450 con 0x7fa59c100e80 2023-12-01T01:21:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.621+0000 7fa59a7fc700 1 --2- 172.21.15.77:0/3132903429 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa5840a0520 0x7fa5840a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.621+0000 7fa5a1245700 1 --2- 172.21.15.77:0/3132903429 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa5840a0520 0x7fa5840a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.621+0000 7fa59a7fc700 1 -- 172.21.15.77:0/3132903429 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fa5900cb810 con 0x7fa59c100e80 2023-12-01T01:21:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.622+0000 7fa5a1245700 1 --2- 172.21.15.77:0/3132903429 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa5840a0520 0x7fa5840a29e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa59c100ce0 tx=0x7fa59400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:21:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.622+0000 7fa5af639b80 1 Processor -- start 2023-12-01T01:21:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.622+0000 7fa5af639b80 1 -- start start 2023-12-01T01:21:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.622+0000 7fa5af639b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a07c644680 0x55a07c644aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.622+0000 7fa5af639b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a07c644fe0 0x55a07c6458f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.622+0000 7fa5af639b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a07c645ed0 0x55a07c646350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.622+0000 7fa5af639b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a07c59c9b0 con 0x55a07c644680 2023-12-01T01:21:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.622+0000 7fa5af639b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a07c59a370 con 0x55a07c645ed0 2023-12-01T01:21:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.622+0000 7fa5af639b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a07c595c60 con 0x55a07c644fe0 2023-12-01T01:21:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.622+0000 7fa5a1245700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a07c644680 0x55a07c644aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.622+0000 7fa5a0a44700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a07c644fe0 0x55a07c6458f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.622+0000 7fa5a1a46700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a07c645ed0 0x55a07c646350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.622+0000 7fa5a0a44700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a07c644fe0 0x55a07c6458f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:47168/0 (socket says 172.21.15.77:47168) 2023-12-01T01:21:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.622+0000 7fa5a0a44700 1 -- 172.21.15.77:0/2172822741 learned_addr learned my addr 172.21.15.77:0/2172822741 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:21:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.623+0000 7fa5a0a44700 1 -- 172.21.15.77:0/2172822741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a07c645ed0 msgr2=0x55a07c646350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.623+0000 7fa5a0a44700 1 --2- 172.21.15.77:0/2172822741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a07c645ed0 0x55a07c646350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.623+0000 7fa5a0a44700 1 -- 172.21.15.77:0/2172822741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a07c644680 msgr2=0x55a07c644aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.623+0000 7fa5a0a44700 1 --2- 172.21.15.77:0/2172822741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a07c644680 0x55a07c644aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.623+0000 7fa5a0a44700 1 -- 172.21.15.77:0/2172822741 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a07c3c3d80 con 0x55a07c644fe0 2023-12-01T01:21:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.623+0000 7fa5a1a46700 1 --2- 172.21.15.77:0/2172822741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a07c645ed0 0x55a07c646350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:21:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.623+0000 7fa5a0a44700 1 --2- 172.21.15.77:0/2172822741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a07c644fe0 0x55a07c6458f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa590095610 tx=0x7fa590007e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f06ce55338f2fc2 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:21:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.623+0000 7fa5837fe700 1 -- 172.21.15.77:0/2172822741 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa590007c60 con 0x55a07c644fe0 2023-12-01T01:21:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.623+0000 7fa5837fe700 1 -- 172.21.15.77:0/2172822741 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa590099110 con 0x55a07c644fe0 2023-12-01T01:21:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.623+0000 7fa5837fe700 1 -- 172.21.15.77:0/2172822741 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59000f700 con 0x55a07c644fe0 2023-12-01T01:21:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.624+0000 7fa5af639b80 1 -- 172.21.15.77:0/2172822741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a07c644fe0 msgr2=0x55a07c6458f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.624+0000 7fa5af639b80 1 --2- 172.21.15.77:0/2172822741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a07c644fe0 0x55a07c6458f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa590095610 tx=0x7fa590007e60 comp rx=0 tx=0).stop 2023-12-01T01:21:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.624+0000 7fa5af639b80 1 -- 172.21.15.77:0/2172822741 shutdown_connections 2023-12-01T01:21:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.624+0000 7fa5af639b80 1 --2- 172.21.15.77:0/2172822741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a07c645ed0 0x55a07c646350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.624+0000 7fa5af639b80 1 --2- 172.21.15.77:0/2172822741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a07c644680 0x55a07c644aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.624+0000 7fa5af639b80 1 --2- 172.21.15.77:0/2172822741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a07c644fe0 0x55a07c6458f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.624+0000 7fa5af639b80 1 -- 172.21.15.77:0/2172822741 >> 172.21.15.77:0/2172822741 conn(0x55a07c5a4070 msgr2=0x55a07c641040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:21:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.624+0000 7fa5af639b80 1 -- 172.21.15.77:0/2172822741 shutdown_connections 2023-12-01T01:21:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.624+0000 7fa5af639b80 1 -- 172.21.15.77:0/2172822741 wait complete. 2023-12-01T01:21:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.624+0000 7fa5af639b80 1 -- start start 2023-12-01T01:21:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.624+0000 7fa5af639b80 1 Processor -- start 2023-12-01T01:21:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.625+0000 7fa5af639b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a07c644680 0x55a07c64e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.625+0000 7fa5af639b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a07c644fe0 0x55a07c64e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.625+0000 7fa5af639b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a07c645ed0 0x55a07c6558a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.625+0000 7fa5af639b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a07c5324b0 con 0x55a07c644680 2023-12-01T01:21:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.625+0000 7fa5af639b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a07c532150 con 0x55a07c644fe0 2023-12-01T01:21:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.625+0000 7fa5af639b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a07c4198c0 con 0x55a07c645ed0 2023-12-01T01:21:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.625+0000 7fa5af639b80 10 client.0 _ll_register_callbacks cb 0x55a07c47fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:21:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.625+0000 7fa5a1245700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a07c644680 0x55a07c64e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.625+0000 7fa5a1245700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a07c644680 0x55a07c64e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60984/0 (socket says 172.21.15.77:60984) 2023-12-01T01:21:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.625+0000 7fa5a1245700 1 -- 172.21.15.77:0/4106641250 learned_addr learned my addr 172.21.15.77:0/4106641250 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:21:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.625+0000 7fa5a0a44700 1 --2- 172.21.15.77:0/4106641250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a07c644fe0 0x55a07c64e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.625+0000 7fa5a1a46700 1 --2- 172.21.15.77:0/4106641250 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a07c645ed0 0x55a07c6558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.625+0000 7fa5a1245700 1 -- 172.21.15.77:0/4106641250 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a07c645ed0 msgr2=0x55a07c6558a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.625+0000 7fa5a1245700 1 --2- 172.21.15.77:0/4106641250 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a07c645ed0 0x55a07c6558a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.625+0000 7fa5a1245700 1 -- 172.21.15.77:0/4106641250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a07c644fe0 msgr2=0x55a07c64e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.625+0000 7fa5a1245700 1 --2- 172.21.15.77:0/4106641250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a07c644fe0 0x55a07c64e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.625+0000 7fa5a1245700 1 -- 172.21.15.77:0/4106641250 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa59c108a20 con 0x55a07c644680 2023-12-01T01:21:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.625+0000 7fa5a0a44700 1 --2- 172.21.15.77:0/4106641250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a07c644fe0 0x55a07c64e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:21:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.626+0000 7fa5a1245700 1 --2- 172.21.15.77:0/4106641250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a07c644680 0x55a07c64e320 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa59400eef0 tx=0x7fa594004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69761a21ebad6e6 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:21:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.626+0000 7fa580ff9700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:21:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.626+0000 7fa580ff9700 1 -- 172.21.15.77:0/4106641250 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5940048f0 con 0x55a07c644680 2023-12-01T01:21:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.626+0000 7fa580ff9700 1 -- 172.21.15.77:0/4106641250 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fa59401c040 con 0x55a07c644680 2023-12-01T01:21:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.626+0000 7fa5af639b80 1 -- 172.21.15.77:0/4106641250 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a07c5c0cb0 con 0x55a07c644680 2023-12-01T01:21:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.626+0000 7fa580ff9700 1 -- 172.21.15.77:0/4106641250 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa594017dc0 con 0x55a07c644680 2023-12-01T01:21:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.626+0000 7fa5af639b80 10 client.16173 fetch_fsmap learned FSMap version 31 2023-12-01T01:21:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.626+0000 7fa5af639b80 1 -- 172.21.15.77:0/4106641250 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fa59c19f290 con 0x55a07c644680 2023-12-01T01:21:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.626+0000 7fa580ff9700 1 -- 172.21.15.77:0/4106641250 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a07c5c0cb0 con 0x55a07c644680 2023-12-01T01:21:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.626+0000 7fa580ff9700 1 -- 172.21.15.77:0/4106641250 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa59402c530 con 0x55a07c644680 2023-12-01T01:21:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.626+0000 7fa5af639b80 10 client.16173 fetch_fsmap finished waiting for FSMap version 31 2023-12-01T01:21:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.626+0000 7fa5af639b80 10 client.16173 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-01T01:21:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.626+0000 7fa5af639b80 20 client.16173 populate_metadata read hostname 'smithi077' 2023-12-01T01:21:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.626+0000 7fa5af639b80 1 --2- 172.21.15.77:0/4106641250 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55a07c65a330 0x55a07c65c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.626+0000 7fa5af639b80 4 client.16173 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:21:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.626+0000 7fa5af639b80 1 -- 172.21.15.77:0/4106641250 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a07c613db0 con 0x55a07c65a330 2023-12-01T01:21:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.627+0000 7fa5a0a44700 1 --2- 172.21.15.77:0/4106641250 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55a07c65a330 0x55a07c65c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.627+0000 7fa5a0a44700 1 --2- 172.21.15.77:0/4106641250 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55a07c65a330 0x55a07c65c750 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ae86f523827c29a server_cookie=c7e522dd8916ca0f in_seq=0 out_seq=0 2023-12-01T01:21:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.627+0000 7fa580ff9700 10 client.16173 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:21:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.628+0000 7fa580ff9700 1 -- 172.21.15.77:0/4106641250 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55a07c613db0 con 0x55a07c65a330 2023-12-01T01:21:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.628+0000 7fa580ff9700 10 client.16173 handle_command_reply: tid=0 2023-12-01T01:21:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.628+0000 7fa5af639b80 1 client.16173 shutdown 2023-12-01T01:21:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.629+0000 7fa5af639b80 1 -- 172.21.15.77:0/4106641250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a07c644680 msgr2=0x55a07c64e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.629+0000 7fa5af639b80 1 --2- 172.21.15.77:0/4106641250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a07c644680 0x55a07c64e320 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa59400eef0 tx=0x7fa594004ad0 comp rx=0 tx=0).stop 2023-12-01T01:21:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.629+0000 7fa5af639b80 1 -- 172.21.15.77:0/4106641250 shutdown_connections 2023-12-01T01:21:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.629+0000 7fa5af639b80 1 --2- 172.21.15.77:0/4106641250 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a07c645ed0 0x55a07c6558a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.629+0000 7fa5af639b80 1 --2- 172.21.15.77:0/4106641250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a07c644680 0x55a07c64e320 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.629+0000 7fa5af639b80 1 --2- 172.21.15.77:0/4106641250 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55a07c65a330 0x55a07c65c750 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.629+0000 7fa5af639b80 1 --2- 172.21.15.77:0/4106641250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a07c644fe0 0x55a07c64e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.629+0000 7fa5af639b80 1 -- 172.21.15.77:0/4106641250 >> 172.21.15.77:0/4106641250 conn(0x55a07c5a4070 msgr2=0x55a07c63f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:21:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.631+0000 7fa580ff9700 0 client.16173 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:21:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.631+0000 7fa5af639b80 1 -- 172.21.15.77:0/4106641250 shutdown_connections 2023-12-01T01:21:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.631+0000 7fa5af639b80 1 -- 172.21.15.77:0/4106641250 wait complete. 2023-12-01T01:21:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.631+0000 7fa5af639b80 20 client.16173 trim_cache size 0 max 16384 2023-12-01T01:21:49.653 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.650+0000 7fa5af639b80 1 Processor -- start 2023-12-01T01:21:49.653 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.650+0000 7fa5af639b80 1 -- start start 2023-12-01T01:21:49.653 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.650+0000 7fa5af639b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a07c644680 0x55a07c6462b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:49.653 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.650+0000 7fa5af639b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a07c644fe0 0x55a07c655860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:49.653 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.650+0000 7fa5af639b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a07c645ed0 0x55a07c6490a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:49.653 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.650+0000 7fa5af639b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a07c6453c0 con 0x55a07c644fe0 2023-12-01T01:21:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.650+0000 7fa5af639b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a07c64c000 con 0x55a07c645ed0 2023-12-01T01:21:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.650+0000 7fa5af639b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a07c644280 con 0x55a07c644680 2023-12-01T01:21:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.650+0000 7fa5a0a44700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a07c644fe0 0x55a07c655860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.650+0000 7fa5a1245700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a07c644680 0x55a07c6462b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.650+0000 7fa5a1a46700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a07c645ed0 0x55a07c6490a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.650+0000 7fa5a1245700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a07c644680 0x55a07c6462b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:47180/0 (socket says 172.21.15.77:47180) 2023-12-01T01:21:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.650+0000 7fa5a1245700 1 -- 172.21.15.77:0/1719750903 learned_addr learned my addr 172.21.15.77:0/1719750903 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:21:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.650+0000 7fa5a1a46700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a07c645ed0 0x55a07c6490a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:39918/0 (socket says 172.21.15.77:39918) 2023-12-01T01:21:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.650+0000 7fa5a0a44700 1 -- 172.21.15.77:0/1719750903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a07c644680 msgr2=0x55a07c6462b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.650+0000 7fa5a0a44700 1 --2- 172.21.15.77:0/1719750903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a07c644680 0x55a07c6462b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.650+0000 7fa5a0a44700 1 -- 172.21.15.77:0/1719750903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a07c645ed0 msgr2=0x55a07c6490a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.650+0000 7fa5a0a44700 1 --2- 172.21.15.77:0/1719750903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a07c645ed0 0x55a07c6490a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.650+0000 7fa5a0a44700 1 -- 172.21.15.77:0/1719750903 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a07c3acc00 con 0x55a07c644fe0 2023-12-01T01:21:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.651+0000 7fa5a0a44700 1 --2- 172.21.15.77:0/1719750903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a07c644fe0 0x55a07c655860 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa590000c00 tx=0x7fa5900cb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=280bb0158e7c9317 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:21:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.651+0000 7fa583fff700 1 -- 172.21.15.77:0/1719750903 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa590007c60 con 0x55a07c644fe0 2023-12-01T01:21:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.651+0000 7fa583fff700 1 -- 172.21.15.77:0/1719750903 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa59009cf40 con 0x55a07c644fe0 2023-12-01T01:21:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.651+0000 7fa583fff700 1 -- 172.21.15.77:0/1719750903 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa590099370 con 0x55a07c644fe0 2023-12-01T01:21:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.652+0000 7fa5af639b80 1 -- 172.21.15.77:0/1719750903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a07c644fe0 msgr2=0x55a07c655860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.652+0000 7fa5af639b80 1 --2- 172.21.15.77:0/1719750903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a07c644fe0 0x55a07c655860 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa590000c00 tx=0x7fa5900cb040 comp rx=0 tx=0).stop 2023-12-01T01:21:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.652+0000 7fa5af639b80 1 -- 172.21.15.77:0/1719750903 shutdown_connections 2023-12-01T01:21:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.652+0000 7fa5af639b80 1 --2- 172.21.15.77:0/1719750903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a07c645ed0 0x55a07c6490a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.652+0000 7fa5af639b80 1 --2- 172.21.15.77:0/1719750903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a07c644680 0x55a07c6462b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.652+0000 7fa5af639b80 1 --2- 172.21.15.77:0/1719750903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a07c644fe0 0x55a07c655860 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.652+0000 7fa5af639b80 1 -- 172.21.15.77:0/1719750903 >> 172.21.15.77:0/1719750903 conn(0x55a07c5a4070 msgr2=0x55a07c640c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:21:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.652+0000 7fa5af639b80 1 -- 172.21.15.77:0/1719750903 shutdown_connections 2023-12-01T01:21:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.652+0000 7fa5af639b80 1 -- 172.21.15.77:0/1719750903 wait complete. 2023-12-01T01:21:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.652+0000 7fa5af639b80 1 -- start start 2023-12-01T01:21:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.652+0000 7fa5af639b80 1 Processor -- start 2023-12-01T01:21:49.656 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.652+0000 7fa5af639b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a07c644680 0x55a07c652ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.653+0000 7fa5af639b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a07c644fe0 0x55a07c6531e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.653+0000 7fa5af639b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a07c645ed0 0x55a07c6541f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.653+0000 7fa5af639b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a07c654730 con 0x55a07c644fe0 2023-12-01T01:21:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.653+0000 7fa5af639b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a07c6548a0 con 0x55a07c644680 2023-12-01T01:21:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.653+0000 7fa5af639b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a07c654a60 con 0x55a07c645ed0 2023-12-01T01:21:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.653+0000 7fa5af639b80 10 client.0 _ll_register_callbacks cb 0x7fa59400e890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:21:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.653+0000 7fa5a1245700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a07c644680 0x55a07c652ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.653+0000 7fa5a0a44700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a07c644fe0 0x55a07c6531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.653+0000 7fa5a1245700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a07c644680 0x55a07c652ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:39920/0 (socket says 172.21.15.77:39920) 2023-12-01T01:21:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.653+0000 7fa5a1245700 1 -- 172.21.15.77:0/1910612702 learned_addr learned my addr 172.21.15.77:0/1910612702 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:21:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.653+0000 7fa5a0a44700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a07c644fe0 0x55a07c6531e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32768/0 (socket says 172.21.15.77:32768) 2023-12-01T01:21:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.653+0000 7fa5a1a46700 1 --2- 172.21.15.77:0/1910612702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a07c645ed0 0x55a07c6541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:49.657 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.653+0000 7fa5a1245700 1 -- 172.21.15.77:0/1910612702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a07c645ed0 msgr2=0x55a07c6541f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.653+0000 7fa5a1245700 1 --2- 172.21.15.77:0/1910612702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a07c645ed0 0x55a07c6541f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.653+0000 7fa5a1245700 1 -- 172.21.15.77:0/1910612702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a07c644fe0 msgr2=0x55a07c6531e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.653+0000 7fa5a1245700 1 --2- 172.21.15.77:0/1910612702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a07c644fe0 0x55a07c6531e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.653+0000 7fa5a1245700 1 -- 172.21.15.77:0/1910612702 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5900b5040 con 0x55a07c644680 2023-12-01T01:21:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.653+0000 7fa5a0a44700 1 --2- 172.21.15.77:0/1910612702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a07c644fe0 0x55a07c6531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:21:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.655+0000 7fa5a1245700 1 --2- 172.21.15.77:0/1910612702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a07c644680 0x55a07c652ca0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa5940046b0 tx=0x7fa5940099c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7603e68d1c1c8d0e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:21:49.658 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.655+0000 7fa580ff9700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2023-12-01T01:21:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.657+0000 7fa580ff9700 1 -- 172.21.15.77:0/1910612702 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59402cce0 con 0x55a07c644680 2023-12-01T01:21:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.657+0000 7fa580ff9700 1 -- 172.21.15.77:0/1910612702 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fa594004b40 con 0x55a07c644680 2023-12-01T01:21:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.657+0000 7fa5af639b80 1 -- 172.21.15.77:0/1910612702 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa590099110 con 0x55a07c644680 2023-12-01T01:21:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.657+0000 7fa580ff9700 1 -- 172.21.15.77:0/1910612702 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa594004d00 con 0x55a07c644680 2023-12-01T01:21:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.657+0000 7fa5af639b80 10 client.25760 fetch_fsmap learned FSMap version 31 2023-12-01T01:21:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.657+0000 7fa5af639b80 1 -- 172.21.15.77:0/1910612702 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55a07c3ea1f0 con 0x55a07c644680 2023-12-01T01:21:49.659 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.658+0000 7fa580ff9700 1 -- 172.21.15.77:0/1910612702 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa590099110 con 0x55a07c644680 2023-12-01T01:21:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.659+0000 7fa580ff9700 1 -- 172.21.15.77:0/1910612702 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa594040050 con 0x55a07c644680 2023-12-01T01:21:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.659+0000 7fa5af639b80 10 client.25760 fetch_fsmap finished waiting for FSMap version 31 2023-12-01T01:21:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.659+0000 7fa5af639b80 10 client.25760 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-01T01:21:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.659+0000 7fa5af639b80 20 client.25760 populate_metadata read hostname 'smithi077' 2023-12-01T01:21:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.659+0000 7fa5af639b80 1 --2- 172.21.15.77:0/1910612702 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55a07c6b0920 0x55a07c6b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.659+0000 7fa5af639b80 4 client.25760 mds_command: new command op to 24500 tid=0[{"prefix": "flush journal"}] 2023-12-01T01:21:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.659+0000 7fa5af639b80 1 -- 172.21.15.77:0/1910612702 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a07c58ece0 con 0x55a07c6b0920 2023-12-01T01:21:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.659+0000 7fa5a0a44700 1 --2- 172.21.15.77:0/1910612702 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55a07c6b0920 0x55a07c6b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.660+0000 7fa580ff9700 10 client.25760 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:21:49.661 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.660+0000 7fa5a0a44700 1 --2- 172.21.15.77:0/1910612702 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55a07c6b0920 0x55a07c6b2d40 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e39c7ec4e01eede8 server_cookie=1b9ef1af519f0937 in_seq=0 out_seq=0 2023-12-01T01:21:49.683 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout:{ 2023-12-01T01:21:49.684 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout: "message": "", 2023-12-01T01:21:49.684 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout: "return_code": 0 2023-12-01T01:21:49.684 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout:} 2023-12-01T01:21:49.684 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.679+0000 7fa580ff9700 1 -- 172.21.15.77:0/1910612702 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a07c58ece0 con 0x55a07c6b0920 2023-12-01T01:21:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.679+0000 7fa580ff9700 10 client.25760 handle_command_reply: tid=0 2023-12-01T01:21:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.679+0000 7fa5af639b80 1 client.25760 shutdown 2023-12-01T01:21:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.679+0000 7fa5af639b80 1 -- 172.21.15.77:0/1910612702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a07c644680 msgr2=0x55a07c652ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.680+0000 7fa5af639b80 1 --2- 172.21.15.77:0/1910612702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a07c644680 0x55a07c652ca0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa5940046b0 tx=0x7fa5940099c0 comp rx=0 tx=0).stop 2023-12-01T01:21:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.680+0000 7fa5af639b80 1 -- 172.21.15.77:0/1910612702 shutdown_connections 2023-12-01T01:21:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.680+0000 7fa5af639b80 1 --2- 172.21.15.77:0/1910612702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a07c645ed0 0x55a07c6541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.680+0000 7fa5af639b80 1 --2- 172.21.15.77:0/1910612702 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55a07c6b0920 0x55a07c6b2d40 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.680+0000 7fa5af639b80 1 --2- 172.21.15.77:0/1910612702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a07c644680 0x55a07c652ca0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.680+0000 7fa5af639b80 1 --2- 172.21.15.77:0/1910612702 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a07c644fe0 0x55a07c6531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.680+0000 7fa5af639b80 1 -- 172.21.15.77:0/1910612702 >> 172.21.15.77:0/1910612702 conn(0x55a07c5a4070 msgr2=0x55a07c6490a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:21:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.680+0000 7fa580ff9700 0 client.25760 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:21:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.681+0000 7fa5af639b80 1 -- 172.21.15.77:0/1910612702 shutdown_connections 2023-12-01T01:21:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.681+0000 7fa5af639b80 1 -- 172.21.15.77:0/1910612702 wait complete. 2023-12-01T01:21:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.682+0000 7fa5af639b80 20 client.25760 trim_cache size 0 max 16384 2023-12-01T01:21:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.682+0000 7fa5a34a9700 1 -- 172.21.15.77:0/3132903429 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa5840a0520 msgr2=0x7fa5840a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.683+0000 7fa5a34a9700 1 --2- 172.21.15.77:0/3132903429 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa5840a0520 0x7fa5840a29e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa59c100ce0 tx=0x7fa59400b040 comp rx=0 tx=0).stop 2023-12-01T01:21:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.683+0000 7fa5a34a9700 1 -- 172.21.15.77:0/3132903429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa59c100e80 msgr2=0x7fa59c196fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.683+0000 7fa5a34a9700 1 --2- 172.21.15.77:0/3132903429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa59c100e80 0x7fa59c196fe0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa59000c9b0 tx=0x7fa59000ccc0 comp rx=0 tx=0).stop 2023-12-01T01:21:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.683+0000 7fa5a34a9700 1 -- 172.21.15.77:0/3132903429 shutdown_connections 2023-12-01T01:21:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.683+0000 7fa5a34a9700 1 --2- 172.21.15.77:0/3132903429 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa5840a0520 0x7fa5840a29e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.683+0000 7fa5a34a9700 1 --2- 172.21.15.77:0/3132903429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa59c101820 0x7fa59c197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.683+0000 7fa5a34a9700 1 --2- 172.21.15.77:0/3132903429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa59c0ffc90 0x7fa59c196aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.683+0000 7fa5a34a9700 1 --2- 172.21.15.77:0/3132903429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa59c100e80 0x7fa59c196fe0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.683+0000 7fa5a34a9700 1 -- 172.21.15.77:0/3132903429 >> 172.21.15.77:0/3132903429 conn(0x7fa59c0fb1e0 msgr2=0x7fa59c0fb980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:21:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.683+0000 7fa5a34a9700 1 -- 172.21.15.77:0/3132903429 shutdown_connections 2023-12-01T01:21:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:21:49.683+0000 7fa5a34a9700 1 -- 172.21.15.77:0/3132903429 wait complete. 2023-12-01T01:21:50.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:21:49 smithi203 ceph-mon[103581]: pgmap v416: 97 pgs: 97 active+clean; 63 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 3.0 MiB/s wr, 1.28k op/s 2023-12-01T01:21:50.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:21:49 smithi137 ceph-mon[103312]: pgmap v416: 97 pgs: 97 active+clean; 63 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 3.0 MiB/s wr, 1.28k op/s 2023-12-01T01:21:50.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:21:49 smithi077 ceph-mon[98867]: pgmap v416: 97 pgs: 97 active+clean; 63 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 3.0 MiB/s wr, 1.28k op/s 2023-12-01T01:21:50.676 INFO:tasks.workunit.client.0.smithi077.stdout:truncating to largest ever: 0x63ff6b1 2023-12-01T01:21:51.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:21:50 smithi203 ceph-mon[103581]: pgmap v417: 97 pgs: 97 active+clean; 69 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.9 MiB/s wr, 1.25k op/s 2023-12-01T01:21:51.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:21:50 smithi137 ceph-mon[103312]: pgmap v417: 97 pgs: 97 active+clean; 69 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.9 MiB/s wr, 1.25k op/s 2023-12-01T01:21:51.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:21:50 smithi077 ceph-mon[98867]: pgmap v417: 97 pgs: 97 active+clean; 69 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.9 MiB/s wr, 1.25k op/s 2023-12-01T01:21:54.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:21:53 smithi203 ceph-mon[103581]: pgmap v418: 97 pgs: 97 active+clean; 70 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.8 MiB/s wr, 1.22k op/s 2023-12-01T01:21:54.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:21:53 smithi137 ceph-mon[103312]: pgmap v418: 97 pgs: 97 active+clean; 70 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.8 MiB/s wr, 1.22k op/s 2023-12-01T01:21:54.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:21:53 smithi077 ceph-mon[98867]: pgmap v418: 97 pgs: 97 active+clean; 70 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.8 MiB/s wr, 1.22k op/s 2023-12-01T01:21:56.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:21:55 smithi203 ceph-mon[103581]: pgmap v419: 97 pgs: 97 active+clean; 58 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 3.0 MiB/s wr, 1.33k op/s 2023-12-01T01:21:56.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:21:55 smithi137 ceph-mon[103312]: pgmap v419: 97 pgs: 97 active+clean; 58 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 3.0 MiB/s wr, 1.33k op/s 2023-12-01T01:21:56.155 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:21:55 smithi077 ceph-mon[98867]: pgmap v419: 97 pgs: 97 active+clean; 58 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 3.0 MiB/s wr, 1.33k op/s 2023-12-01T01:21:58.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:21:57 smithi203 ceph-mon[103581]: pgmap v420: 97 pgs: 97 active+clean; 64 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.7 MiB/s wr, 1.18k op/s 2023-12-01T01:21:58.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:21:57 smithi137 ceph-mon[103312]: pgmap v420: 97 pgs: 97 active+clean; 64 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.7 MiB/s wr, 1.18k op/s 2023-12-01T01:21:58.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:21:57 smithi077 ceph-mon[98867]: pgmap v420: 97 pgs: 97 active+clean; 64 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.7 MiB/s wr, 1.18k op/s 2023-12-01T01:21:58.213 INFO:tasks.workunit.client.0.smithi077.stdout:All 50000 operations completed A-OK! 2023-12-01T01:21:58.214 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-12-01T01:21:58.215 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-12-01T01:21:59.676 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-01T01:21:59.814 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.811+0000 7f9670148700 1 -- 172.21.15.77:0/593629888 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9668100e70 msgr2=0x7f96681012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:59.814 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.811+0000 7f9670148700 1 --2- 172.21.15.77:0/593629888 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9668100e70 0x7f96681012d0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f96640099c0 tx=0x7f9664009cd0 comp rx=0 tx=0).stop 2023-12-01T01:21:59.815 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.811+0000 7f9670148700 1 -- 172.21.15.77:0/593629888 shutdown_connections 2023-12-01T01:21:59.815 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.811+0000 7f9670148700 1 --2- 172.21.15.77:0/593629888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9668101810 0x7f96681080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:59.815 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.811+0000 7f9670148700 1 --2- 172.21.15.77:0/593629888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96680ffc80 0x7f96681000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:59.815 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.811+0000 7f9670148700 1 --2- 172.21.15.77:0/593629888 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9668100e70 0x7f96681012d0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:59.815 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.811+0000 7f9670148700 1 -- 172.21.15.77:0/593629888 >> 172.21.15.77:0/593629888 conn(0x7f96680fb1f0 msgr2=0x7f96680fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:21:59.815 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.811+0000 7f9670148700 1 -- 172.21.15.77:0/593629888 shutdown_connections 2023-12-01T01:21:59.815 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.811+0000 7f9670148700 1 -- 172.21.15.77:0/593629888 wait complete. 2023-12-01T01:21:59.815 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.812+0000 7f9670148700 1 Processor -- start 2023-12-01T01:21:59.815 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.812+0000 7f9670148700 1 -- start start 2023-12-01T01:21:59.815 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.812+0000 7f9670148700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96680ffc80 0x7f9668195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:59.815 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.812+0000 7f9670148700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9668100e70 0x7f9668195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:59.815 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.812+0000 7f9670148700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9668101810 0x7f966819cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:59.815 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.812+0000 7f9670148700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966819d4f0 con 0x7f9668100e70 2023-12-01T01:21:59.816 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.812+0000 7f9670148700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966819d630 con 0x7f9668101810 2023-12-01T01:21:59.816 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.812+0000 7f9670148700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966819d7c0 con 0x7f96680ffc80 2023-12-01T01:21:59.816 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.812+0000 7f966d6e3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9668100e70 0x7f9668195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:59.816 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.812+0000 7f966d6e3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9668100e70 0x7f9668195eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32782/0 (socket says 172.21.15.77:32782) 2023-12-01T01:21:59.816 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.812+0000 7f966dee4700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96680ffc80 0x7f9668195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:59.816 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.812+0000 7f966d6e3700 1 -- 172.21.15.77:0/2691247029 learned_addr learned my addr 172.21.15.77:0/2691247029 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:21:59.816 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.812+0000 7f966e6e5700 1 --2- 172.21.15.77:0/2691247029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9668101810 0x7f966819cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:59.816 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.812+0000 7f966d6e3700 1 -- 172.21.15.77:0/2691247029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96680ffc80 msgr2=0x7f9668195970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:59.816 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.812+0000 7f966d6e3700 1 --2- 172.21.15.77:0/2691247029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96680ffc80 0x7f9668195970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:59.816 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.812+0000 7f966d6e3700 1 -- 172.21.15.77:0/2691247029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9668101810 msgr2=0x7f966819cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:59.816 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.812+0000 7f966d6e3700 1 --2- 172.21.15.77:0/2691247029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9668101810 0x7f966819cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:59.816 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.812+0000 7f966d6e3700 1 -- 172.21.15.77:0/2691247029 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96640095f0 con 0x7f9668100e70 2023-12-01T01:21:59.816 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.812+0000 7f966d6e3700 1 --2- 172.21.15.77:0/2691247029 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9668100e70 0x7f9668195eb0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9664009fd0 tx=0x7f9664009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:21:59.816 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.812+0000 7f965effd700 1 -- 172.21.15.77:0/2691247029 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f966401d070 con 0x7f9668100e70 2023-12-01T01:21:59.816 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.813+0000 7f9670148700 1 -- 172.21.15.77:0/2691247029 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f966819da40 con 0x7f9668100e70 2023-12-01T01:21:59.817 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.813+0000 7f965effd700 1 -- 172.21.15.77:0/2691247029 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9664005030 con 0x7f9668100e70 2023-12-01T01:21:59.817 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.813+0000 7f965effd700 1 -- 172.21.15.77:0/2691247029 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9664004a90 con 0x7f9668100e70 2023-12-01T01:21:59.817 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.813+0000 7f9670148700 1 -- 172.21.15.77:0/2691247029 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f966819dec0 con 0x7f9668100e70 2023-12-01T01:21:59.820 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.815+0000 7f965effd700 1 -- 172.21.15.77:0/2691247029 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f96640051a0 con 0x7f9668100e70 2023-12-01T01:21:59.820 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.815+0000 7f965effd700 1 --2- 172.21.15.77:0/2691247029 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f96540a0520 0x7f96540a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:59.820 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.816+0000 7f966dee4700 1 --2- 172.21.15.77:0/2691247029 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f96540a0520 0x7f96540a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:59.820 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.816+0000 7f965effd700 1 -- 172.21.15.77:0/2691247029 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f96640cc6b0 con 0x7f9668100e70 2023-12-01T01:21:59.820 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.816+0000 7f966dee4700 1 --2- 172.21.15.77:0/2691247029 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f96540a0520 0x7f96540a29e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9668100cd0 tx=0x7f9658009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:21:59.820 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.816+0000 7f967c2d8b80 1 Processor -- start 2023-12-01T01:21:59.820 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.817+0000 7f967c2d8b80 1 -- start start 2023-12-01T01:21:59.820 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.817+0000 7f967c2d8b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55735d89c0f0 0x55735d89c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:59.820 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.817+0000 7f967c2d8b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55735d89ca70 0x55735d89ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:59.820 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.817+0000 7f967c2d8b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55735d89b3f0 0x55735d89b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:59.820 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.817+0000 7f967c2d8b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55735d7f2a10 con 0x55735d89b3f0 2023-12-01T01:21:59.821 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.817+0000 7f967c2d8b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55735d7f03d0 con 0x55735d89c0f0 2023-12-01T01:21:59.821 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.817+0000 7f967c2d8b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55735d7ebcc0 con 0x55735d89ca70 2023-12-01T01:21:59.821 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.817+0000 7f966e6e5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55735d89b3f0 0x55735d89b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:59.821 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.817+0000 7f966d6e3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55735d89ca70 0x55735d89ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:59.821 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.817+0000 7f966e6e5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55735d89b3f0 0x55735d89b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32792/0 (socket says 172.21.15.77:32792) 2023-12-01T01:21:59.821 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.817+0000 7f966d6e3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55735d89ca70 0x55735d89ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:47210/0 (socket says 172.21.15.77:47210) 2023-12-01T01:21:59.821 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.817+0000 7f966e6e5700 1 -- 172.21.15.77:0/1280322971 learned_addr learned my addr 172.21.15.77:0/1280322971 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:21:59.821 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.817+0000 7f966dee4700 1 --2- 172.21.15.77:0/1280322971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55735d89c0f0 0x55735d89c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:59.821 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.817+0000 7f966e6e5700 1 -- 172.21.15.77:0/1280322971 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55735d89ca70 msgr2=0x55735d89ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:59.821 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.817+0000 7f966e6e5700 1 --2- 172.21.15.77:0/1280322971 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55735d89ca70 0x55735d89ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:59.821 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.817+0000 7f966e6e5700 1 -- 172.21.15.77:0/1280322971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55735d89c0f0 msgr2=0x55735d89c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:59.821 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.817+0000 7f966e6e5700 1 --2- 172.21.15.77:0/1280322971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55735d89c0f0 0x55735d89c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:59.821 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.817+0000 7f966e6e5700 1 -- 172.21.15.77:0/1280322971 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55735d619de0 con 0x55735d89b3f0 2023-12-01T01:21:59.821 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.817+0000 7f966d6e3700 1 --2- 172.21.15.77:0/1280322971 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55735d89ca70 0x55735d89ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:21:59.822 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.817+0000 7f966e6e5700 1 --2- 172.21.15.77:0/1280322971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55735d89b3f0 0x55735d89b870 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f9660011680 tx=0x7f9660009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1593fc3776d7a654 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:21:59.822 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.817+0000 7f9653fff700 1 -- 172.21.15.77:0/1280322971 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f966000e040 con 0x55735d89b3f0 2023-12-01T01:21:59.822 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.818+0000 7f9653fff700 1 -- 172.21.15.77:0/1280322971 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f966000f040 con 0x55735d89b3f0 2023-12-01T01:21:59.822 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.818+0000 7f9653fff700 1 -- 172.21.15.77:0/1280322971 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9660004860 con 0x55735d89b3f0 2023-12-01T01:21:59.822 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.818+0000 7f967c2d8b80 1 -- 172.21.15.77:0/1280322971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55735d89b3f0 msgr2=0x55735d89b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:59.822 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.818+0000 7f967c2d8b80 1 --2- 172.21.15.77:0/1280322971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55735d89b3f0 0x55735d89b870 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f9660011680 tx=0x7f9660009920 comp rx=0 tx=0).stop 2023-12-01T01:21:59.822 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.818+0000 7f967c2d8b80 1 -- 172.21.15.77:0/1280322971 shutdown_connections 2023-12-01T01:21:59.822 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.818+0000 7f967c2d8b80 1 --2- 172.21.15.77:0/1280322971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55735d89b3f0 0x55735d89b870 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:59.822 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.818+0000 7f967c2d8b80 1 --2- 172.21.15.77:0/1280322971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55735d89c0f0 0x55735d89c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:59.822 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.818+0000 7f967c2d8b80 1 --2- 172.21.15.77:0/1280322971 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55735d89ca70 0x55735d89ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:59.822 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.818+0000 7f967c2d8b80 1 -- 172.21.15.77:0/1280322971 >> 172.21.15.77:0/1280322971 conn(0x55735d7fa0d0 msgr2=0x55735d895940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:21:59.822 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.818+0000 7f967c2d8b80 1 -- 172.21.15.77:0/1280322971 shutdown_connections 2023-12-01T01:21:59.822 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.818+0000 7f967c2d8b80 1 -- 172.21.15.77:0/1280322971 wait complete. 2023-12-01T01:21:59.822 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.819+0000 7f967c2d8b80 1 -- start start 2023-12-01T01:21:59.823 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.819+0000 7f967c2d8b80 1 Processor -- start 2023-12-01T01:21:59.826 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.819+0000 7f967c2d8b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55735d89b3f0 0x55735d8a4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:59.826 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.819+0000 7f967c2d8b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55735d89c0f0 0x55735d8a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:59.826 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.819+0000 7f967c2d8b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55735d89ca70 0x55735d8abed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:59.826 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.819+0000 7f967c2d8b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55735d788510 con 0x55735d89c0f0 2023-12-01T01:21:59.826 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.819+0000 7f967c2d8b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55735d7881b0 con 0x55735d89b3f0 2023-12-01T01:21:59.827 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.819+0000 7f967c2d8b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55735d66f920 con 0x55735d89ca70 2023-12-01T01:21:59.827 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.819+0000 7f967c2d8b80 10 client.0 _ll_register_callbacks cb 0x55735d6d5a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:21:59.827 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.819+0000 7f966d6e3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55735d89c0f0 0x55735d8a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:59.827 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.819+0000 7f966d6e3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55735d89c0f0 0x55735d8a4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32796/0 (socket says 172.21.15.77:32796) 2023-12-01T01:21:59.827 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.819+0000 7f966d6e3700 1 -- 172.21.15.77:0/559530870 learned_addr learned my addr 172.21.15.77:0/559530870 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:21:59.827 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.819+0000 7f966e6e5700 1 --2- 172.21.15.77:0/559530870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55735d89ca70 0x55735d8abed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:59.827 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.820+0000 7f966dee4700 1 --2- 172.21.15.77:0/559530870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55735d89b3f0 0x55735d8a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:59.827 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.820+0000 7f966d6e3700 1 -- 172.21.15.77:0/559530870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55735d89ca70 msgr2=0x55735d8abed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:59.827 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.820+0000 7f966d6e3700 1 --2- 172.21.15.77:0/559530870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55735d89ca70 0x55735d8abed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:59.827 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.820+0000 7f966d6e3700 1 -- 172.21.15.77:0/559530870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55735d89b3f0 msgr2=0x55735d8a4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:59.827 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.820+0000 7f966d6e3700 1 --2- 172.21.15.77:0/559530870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55735d89b3f0 0x55735d8a4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:59.827 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.820+0000 7f966d6e3700 1 -- 172.21.15.77:0/559530870 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f966819dec0 con 0x55735d89c0f0 2023-12-01T01:21:59.827 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.820+0000 7f966dee4700 1 --2- 172.21.15.77:0/559530870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55735d89b3f0 0x55735d8a4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:21:59.827 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.820+0000 7f966d6e3700 1 --2- 172.21.15.77:0/559530870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55735d89c0f0 0x55735d8a4e90 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f9664099d40 tx=0x7f9664005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c7949cccbd5251 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:21:59.827 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.820+0000 7f96517fa700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:21:59.828 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.820+0000 7f96517fa700 1 -- 172.21.15.77:0/559530870 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96640b1070 con 0x55735d89c0f0 2023-12-01T01:21:59.828 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.820+0000 7f96517fa700 1 -- 172.21.15.77:0/559530870 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f96640b9040 con 0x55735d89c0f0 2023-12-01T01:21:59.828 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.820+0000 7f967c2d8b80 1 -- 172.21.15.77:0/559530870 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55735d816d10 con 0x55735d89c0f0 2023-12-01T01:21:59.828 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.820+0000 7f96517fa700 1 -- 172.21.15.77:0/559530870 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96640a2020 con 0x55735d89c0f0 2023-12-01T01:21:59.828 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.820+0000 7f967c2d8b80 10 client.16191 fetch_fsmap learned FSMap version 31 2023-12-01T01:21:59.828 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.820+0000 7f967c2d8b80 1 -- 172.21.15.77:0/559530870 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f9660011330 con 0x55735d89c0f0 2023-12-01T01:21:59.828 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.820+0000 7f96517fa700 1 -- 172.21.15.77:0/559530870 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9664064020 con 0x55735d89c0f0 2023-12-01T01:21:59.828 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.820+0000 7f96517fa700 1 -- 172.21.15.77:0/559530870 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f96640a1b90 con 0x55735d89c0f0 2023-12-01T01:21:59.828 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.820+0000 7f967c2d8b80 10 client.16191 fetch_fsmap finished waiting for FSMap version 31 2023-12-01T01:21:59.828 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.820+0000 7f967c2d8b80 10 client.16191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-01T01:21:59.828 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.820+0000 7f967c2d8b80 20 client.16191 populate_metadata read hostname 'smithi077' 2023-12-01T01:21:59.828 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.820+0000 7f967c2d8b80 1 --2- 172.21.15.77:0/559530870 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55735d8b0300 0x55735d8b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:59.828 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.820+0000 7f967c2d8b80 4 client.16191 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:21:59.828 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.820+0000 7f967c2d8b80 1 -- 172.21.15.77:0/559530870 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55735d869e10 con 0x55735d8b0300 2023-12-01T01:21:59.828 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.821+0000 7f966dee4700 1 --2- 172.21.15.77:0/559530870 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55735d8b0300 0x55735d8b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:59.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.821+0000 7f966dee4700 1 --2- 172.21.15.77:0/559530870 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55735d8b0300 0x55735d8b2720 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1f7f12ad21881c4 server_cookie=4dcbf5fb5bd62399 in_seq=0 out_seq=0 2023-12-01T01:21:59.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.821+0000 7f96517fa700 10 client.16191 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:21:59.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.822+0000 7f96517fa700 1 -- 172.21.15.77:0/559530870 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55735d869e10 con 0x55735d8b0300 2023-12-01T01:21:59.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.822+0000 7f96517fa700 10 client.16191 handle_command_reply: tid=0 2023-12-01T01:21:59.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.822+0000 7f967c2d8b80 1 client.16191 shutdown 2023-12-01T01:21:59.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.822+0000 7f967c2d8b80 1 -- 172.21.15.77:0/559530870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55735d89c0f0 msgr2=0x55735d8a4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:59.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.822+0000 7f967c2d8b80 1 --2- 172.21.15.77:0/559530870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55735d89c0f0 0x55735d8a4e90 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f9664099d40 tx=0x7f9664005760 comp rx=0 tx=0).stop 2023-12-01T01:21:59.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.822+0000 7f967c2d8b80 1 -- 172.21.15.77:0/559530870 shutdown_connections 2023-12-01T01:21:59.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.822+0000 7f967c2d8b80 1 --2- 172.21.15.77:0/559530870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55735d89ca70 0x55735d8abed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:59.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.822+0000 7f967c2d8b80 1 --2- 172.21.15.77:0/559530870 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55735d8b0300 0x55735d8b2720 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:59.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.822+0000 7f967c2d8b80 1 --2- 172.21.15.77:0/559530870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55735d89b3f0 0x55735d8a4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:59.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.822+0000 7f96517fa700 0 client.16191 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:21:59.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.822+0000 7f967c2d8b80 1 --2- 172.21.15.77:0/559530870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55735d89c0f0 0x55735d8a4e90 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:59.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.822+0000 7f967c2d8b80 1 -- 172.21.15.77:0/559530870 >> 172.21.15.77:0/559530870 conn(0x55735d7fa0d0 msgr2=0x55735d8950b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:21:59.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.822+0000 7f967c2d8b80 1 -- 172.21.15.77:0/559530870 shutdown_connections 2023-12-01T01:21:59.830 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.822+0000 7f967c2d8b80 1 -- 172.21.15.77:0/559530870 wait complete. 2023-12-01T01:21:59.830 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.822+0000 7f967c2d8b80 20 client.16191 trim_cache size 0 max 16384 2023-12-01T01:21:59.841 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.839+0000 7f967c2d8b80 1 Processor -- start 2023-12-01T01:21:59.841 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.839+0000 7f967c2d8b80 1 -- start start 2023-12-01T01:21:59.841 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.839+0000 7f967c2d8b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55735d89b3f0 0x55735d89ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:59.841 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.839+0000 7f967c2d8b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55735d89c0f0 0x55735d89f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:59.841 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.839+0000 7f967c2d8b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55735d89ca70 0x55735d89f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:59.841 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.839+0000 7f967c2d8b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55735d88db50 con 0x55735d89ca70 2023-12-01T01:21:59.841 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.839+0000 7f967c2d8b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55735d8a7e70 con 0x55735d89b3f0 2023-12-01T01:21:59.841 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.839+0000 7f967c2d8b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55735d8a2510 con 0x55735d89c0f0 2023-12-01T01:21:59.841 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.839+0000 7f966e6e5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55735d89ca70 0x55735d89f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:59.841 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.839+0000 7f966d6e3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55735d89c0f0 0x55735d89f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:59.841 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.839+0000 7f966e6e5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55735d89ca70 0x55735d89f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32806/0 (socket says 172.21.15.77:32806) 2023-12-01T01:21:59.842 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.839+0000 7f966e6e5700 1 -- 172.21.15.77:0/1607099521 learned_addr learned my addr 172.21.15.77:0/1607099521 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:21:59.842 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.839+0000 7f966dee4700 1 --2- 172.21.15.77:0/1607099521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55735d89b3f0 0x55735d89ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:59.842 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.839+0000 7f966e6e5700 1 -- 172.21.15.77:0/1607099521 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55735d89c0f0 msgr2=0x55735d89f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:59.842 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.839+0000 7f966e6e5700 1 --2- 172.21.15.77:0/1607099521 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55735d89c0f0 0x55735d89f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:59.842 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.839+0000 7f966e6e5700 1 -- 172.21.15.77:0/1607099521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55735d89b3f0 msgr2=0x55735d89ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:59.842 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.839+0000 7f966e6e5700 1 --2- 172.21.15.77:0/1607099521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55735d89b3f0 0x55735d89ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:59.842 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.839+0000 7f966e6e5700 1 -- 172.21.15.77:0/1607099521 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55735d602c60 con 0x55735d89ca70 2023-12-01T01:21:59.842 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.839+0000 7f966e6e5700 1 --2- 172.21.15.77:0/1607099521 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55735d89ca70 0x55735d89f7d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f966000b010 tx=0x7f96600060b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58d9b7039797133f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:21:59.842 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.840+0000 7f965cff9700 1 -- 172.21.15.77:0/1607099521 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f966000e050 con 0x55735d89ca70 2023-12-01T01:21:59.842 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.840+0000 7f965cff9700 1 -- 172.21.15.77:0/1607099521 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9660009360 con 0x55735d89ca70 2023-12-01T01:21:59.842 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.840+0000 7f965cff9700 1 -- 172.21.15.77:0/1607099521 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9660007a90 con 0x55735d89ca70 2023-12-01T01:21:59.846 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.840+0000 7f967c2d8b80 1 -- 172.21.15.77:0/1607099521 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55735d89ca70 msgr2=0x55735d89f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:59.846 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.840+0000 7f967c2d8b80 1 --2- 172.21.15.77:0/1607099521 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55735d89ca70 0x55735d89f7d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f966000b010 tx=0x7f96600060b0 comp rx=0 tx=0).stop 2023-12-01T01:21:59.846 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.840+0000 7f967c2d8b80 1 -- 172.21.15.77:0/1607099521 shutdown_connections 2023-12-01T01:21:59.846 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.840+0000 7f967c2d8b80 1 --2- 172.21.15.77:0/1607099521 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55735d89ca70 0x55735d89f7d0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:59.846 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.840+0000 7f967c2d8b80 1 --2- 172.21.15.77:0/1607099521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55735d89b3f0 0x55735d89ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:59.847 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.840+0000 7f967c2d8b80 1 --2- 172.21.15.77:0/1607099521 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55735d89c0f0 0x55735d89f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:59.847 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.840+0000 7f967c2d8b80 1 -- 172.21.15.77:0/1607099521 >> 172.21.15.77:0/1607099521 conn(0x55735d7fa0d0 msgr2=0x55735d8a2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:21:59.847 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.840+0000 7f967c2d8b80 1 -- 172.21.15.77:0/1607099521 shutdown_connections 2023-12-01T01:21:59.847 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.840+0000 7f967c2d8b80 1 -- 172.21.15.77:0/1607099521 wait complete. 2023-12-01T01:21:59.847 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.840+0000 7f967c2d8b80 1 -- start start 2023-12-01T01:21:59.847 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.841+0000 7f967c2d8b80 1 Processor -- start 2023-12-01T01:21:59.847 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.841+0000 7f967c2d8b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55735d89b3f0 0x55735d8a8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:59.847 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.841+0000 7f967c2d8b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55735d89c0f0 0x55735d8a90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:59.847 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.841+0000 7f967c2d8b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55735d89ca70 0x55735d906240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:59.847 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.841+0000 7f967c2d8b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55735d8ac0d0 con 0x55735d89ca70 2023-12-01T01:21:59.847 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.841+0000 7f967c2d8b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55735d8ac240 con 0x55735d89b3f0 2023-12-01T01:21:59.847 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.841+0000 7f967c2d8b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55735d906780 con 0x55735d89c0f0 2023-12-01T01:21:59.847 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.841+0000 7f967c2d8b80 10 client.0 _ll_register_callbacks cb 0x7f96640999a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:21:59.847 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.841+0000 7f966e6e5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55735d89ca70 0x55735d906240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:59.848 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.841+0000 7f966d6e3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55735d89c0f0 0x55735d8a90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:59.848 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.841+0000 7f966e6e5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55735d89ca70 0x55735d906240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32812/0 (socket says 172.21.15.77:32812) 2023-12-01T01:21:59.848 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.841+0000 7f966e6e5700 1 -- 172.21.15.77:0/1831775669 learned_addr learned my addr 172.21.15.77:0/1831775669 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:21:59.848 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.841+0000 7f966e6e5700 1 -- 172.21.15.77:0/1831775669 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55735d89c0f0 msgr2=0x55735d8a90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:21:59.848 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.841+0000 7f966e6e5700 1 --2- 172.21.15.77:0/1831775669 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55735d89c0f0 0x55735d8a90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:59.848 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.841+0000 7f966e6e5700 1 -- 172.21.15.77:0/1831775669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55735d89b3f0 msgr2=0x55735d8a8b70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-01T01:21:59.848 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.841+0000 7f966e6e5700 1 --2- 172.21.15.77:0/1831775669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55735d89b3f0 0x55735d8a8b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:59.848 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.841+0000 7f966e6e5700 1 -- 172.21.15.77:0/1831775669 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55735d619de0 con 0x55735d89ca70 2023-12-01T01:21:59.848 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.841+0000 7f966d6e3700 1 --2- 172.21.15.77:0/1831775669 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55735d89c0f0 0x55735d8a90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:21:59.848 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.842+0000 7f966e6e5700 1 --2- 172.21.15.77:0/1831775669 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55735d89ca70 0x55735d906240 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f9660000f80 tx=0x7f9660007e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7870d6148e5de31 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:21:59.848 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.842+0000 7f96517fa700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:21:59.848 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.842+0000 7f96517fa700 1 -- 172.21.15.77:0/1831775669 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9660003cd0 con 0x55735d89ca70 2023-12-01T01:21:59.848 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.842+0000 7f96517fa700 1 -- 172.21.15.77:0/1831775669 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f9660003e30 con 0x55735d89ca70 2023-12-01T01:21:59.848 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.842+0000 7f967c2d8b80 1 -- 172.21.15.77:0/1831775669 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f966000f040 con 0x55735d89ca70 2023-12-01T01:21:59.849 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.842+0000 7f96517fa700 1 -- 172.21.15.77:0/1831775669 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f966000e020 con 0x55735d89ca70 2023-12-01T01:21:59.849 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.842+0000 7f967c2d8b80 10 client.16197 fetch_fsmap learned FSMap version 31 2023-12-01T01:21:59.849 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.842+0000 7f967c2d8b80 1 -- 172.21.15.77:0/1831775669 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55735d640250 con 0x55735d89ca70 2023-12-01T01:21:59.849 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.842+0000 7f96517fa700 1 -- 172.21.15.77:0/1831775669 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f966000f040 con 0x55735d89ca70 2023-12-01T01:21:59.849 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.842+0000 7f96517fa700 1 -- 172.21.15.77:0/1831775669 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 31) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9660027570 con 0x55735d89ca70 2023-12-01T01:21:59.849 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.842+0000 7f967c2d8b80 10 client.16197 fetch_fsmap finished waiting for FSMap version 31 2023-12-01T01:21:59.849 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.842+0000 7f967c2d8b80 10 client.16197 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-01T01:21:59.849 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.842+0000 7f967c2d8b80 20 client.16197 populate_metadata read hostname 'smithi077' 2023-12-01T01:21:59.849 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.842+0000 7f967c2d8b80 1 --2- 172.21.15.77:0/1831775669 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55735d907da0 0x55735d90a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:21:59.849 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.842+0000 7f967c2d8b80 4 client.16197 mds_command: new command op to 24500 tid=0[{"prefix": "scrub status"}] 2023-12-01T01:21:59.849 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.842+0000 7f967c2d8b80 1 -- 172.21.15.77:0/1831775669 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55735d7e4d40 con 0x55735d907da0 2023-12-01T01:21:59.849 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.843+0000 7f966dee4700 1 --2- 172.21.15.77:0/1831775669 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55735d907da0 0x55735d90a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:21:59.849 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.843+0000 7f96517fa700 10 client.16197 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:21:59.849 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.843+0000 7f966dee4700 1 --2- 172.21.15.77:0/1831775669 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55735d907da0 0x55735d90a1c0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b221122869f6205 server_cookie=636333837e0adcb4 in_seq=0 out_seq=0 2023-12-01T01:21:59.850 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.843+0000 7f96517fa700 1 -- 172.21.15.77:0/1831775669 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55735d7e4d40 con 0x55735d907da0 2023-12-01T01:21:59.850 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.843+0000 7f96517fa700 10 client.16197 handle_command_reply: tid=0 2023-12-01T01:21:59.850 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.843+0000 7f967c2d8b80 1 client.16197 shutdown 2023-12-01T01:21:59.850 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.843+0000 7f967c2d8b80 1 -- 172.21.15.77:0/1831775669 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55735d89ca70 msgr2=0x55735d906240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:59.850 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.843+0000 7f967c2d8b80 1 --2- 172.21.15.77:0/1831775669 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55735d89ca70 0x55735d906240 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f9660000f80 tx=0x7f9660007e70 comp rx=0 tx=0).stop 2023-12-01T01:21:59.850 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.843+0000 7f967c2d8b80 1 -- 172.21.15.77:0/1831775669 shutdown_connections 2023-12-01T01:21:59.850 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.843+0000 7f967c2d8b80 1 --2- 172.21.15.77:0/1831775669 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55735d89ca70 0x55735d906240 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:59.850 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.843+0000 7f967c2d8b80 1 --2- 172.21.15.77:0/1831775669 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x55735d907da0 0x55735d90a1c0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:59.850 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.843+0000 7f967c2d8b80 1 --2- 172.21.15.77:0/1831775669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55735d89b3f0 0x55735d8a8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:59.850 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.843+0000 7f96517fa700 0 client.16197 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:21:59.850 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.843+0000 7f967c2d8b80 1 --2- 172.21.15.77:0/1831775669 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55735d89c0f0 0x55735d8a90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:59.850 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.843+0000 7f967c2d8b80 1 -- 172.21.15.77:0/1831775669 >> 172.21.15.77:0/1831775669 conn(0x55735d7fa0d0 msgr2=0x55735d8a2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:21:59.850 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.843+0000 7f967c2d8b80 1 -- 172.21.15.77:0/1831775669 shutdown_connections 2023-12-01T01:21:59.850 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.844+0000 7f967c2d8b80 1 -- 172.21.15.77:0/1831775669 wait complete. 2023-12-01T01:21:59.851 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.844+0000 7f967c2d8b80 20 client.16197 trim_cache size 0 max 16384 2023-12-01T01:21:59.851 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2023-12-01T01:21:59.851 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "IDLE (waiting for more scrubs)", 2023-12-01T01:21:59.851 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": { 2023-12-01T01:21:59.851 INFO:teuthology.orchestra.run.smithi077.stdout: "085290de-33b6-4527-ab7f-03f7dfd5dbba": { 2023-12-01T01:21:59.851 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/", 2023-12-01T01:21:59.851 INFO:teuthology.orchestra.run.smithi077.stdout: "tag": "085290de-33b6-4527-ab7f-03f7dfd5dbba", 2023-12-01T01:21:59.851 INFO:teuthology.orchestra.run.smithi077.stdout: "options": "recursive,force" 2023-12-01T01:21:59.851 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:21:59.851 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:21:59.851 INFO:teuthology.orchestra.run.smithi077.stdout:} 2023-12-01T01:21:59.852 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.844+0000 7f9670148700 1 -- 172.21.15.77:0/2691247029 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f96540a0520 msgr2=0x7f96540a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:59.852 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.844+0000 7f9670148700 1 --2- 172.21.15.77:0/2691247029 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f96540a0520 0x7f96540a29e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9668100cd0 tx=0x7f9658009260 comp rx=0 tx=0).stop 2023-12-01T01:21:59.852 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.844+0000 7f9670148700 1 -- 172.21.15.77:0/2691247029 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9668100e70 msgr2=0x7f9668195eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:21:59.852 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.844+0000 7f9670148700 1 --2- 172.21.15.77:0/2691247029 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9668100e70 0x7f9668195eb0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9664009fd0 tx=0x7f9664009580 comp rx=0 tx=0).stop 2023-12-01T01:21:59.853 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.844+0000 7f9670148700 1 -- 172.21.15.77:0/2691247029 shutdown_connections 2023-12-01T01:21:59.853 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.844+0000 7f9670148700 1 --2- 172.21.15.77:0/2691247029 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f96540a0520 0x7f96540a29e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:59.853 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.844+0000 7f9670148700 1 --2- 172.21.15.77:0/2691247029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9668101810 0x7f966819cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:59.853 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.844+0000 7f9670148700 1 --2- 172.21.15.77:0/2691247029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96680ffc80 0x7f9668195970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:59.853 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.844+0000 7f9670148700 1 --2- 172.21.15.77:0/2691247029 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9668100e70 0x7f9668195eb0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:21:59.853 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.844+0000 7f9670148700 1 -- 172.21.15.77:0/2691247029 >> 172.21.15.77:0/2691247029 conn(0x7f96680fb1f0 msgr2=0x7f96680fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:21:59.853 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.844+0000 7f9670148700 1 -- 172.21.15.77:0/2691247029 shutdown_connections 2023-12-01T01:21:59.853 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:21:59.844+0000 7f9670148700 1 -- 172.21.15.77:0/2691247029 wait complete. 2023-12-01T01:21:59.857 INFO:tasks.cephfs.filesystem:scrub status for tag:085290de-33b6-4527-ab7f-03f7dfd5dbba - {'path': '/', 'tag': '085290de-33b6-4527-ab7f-03f7dfd5dbba', 'options': 'recursive,force'} 2023-12-01T01:22:00.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:21:59 smithi203 ceph-mon[103581]: pgmap v421: 97 pgs: 97 active+clean; 64 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.7 MiB/s wr, 1.24k op/s 2023-12-01T01:22:00.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:21:59 smithi137 ceph-mon[103312]: pgmap v421: 97 pgs: 97 active+clean; 64 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.7 MiB/s wr, 1.24k op/s 2023-12-01T01:22:00.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:21:59 smithi077 ceph-mon[98867]: pgmap v421: 97 pgs: 97 active+clean; 64 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.7 MiB/s wr, 1.24k op/s 2023-12-01T01:22:01.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:00 smithi203 ceph-mon[103581]: pgmap v422: 97 pgs: 97 active+clean; 65 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.7 MiB/s wr, 1.18k op/s 2023-12-01T01:22:01.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:00 smithi137 ceph-mon[103312]: pgmap v422: 97 pgs: 97 active+clean; 65 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.7 MiB/s wr, 1.18k op/s 2023-12-01T01:22:01.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:00 smithi077 ceph-mon[98867]: pgmap v422: 97 pgs: 97 active+clean; 65 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.7 MiB/s wr, 1.18k op/s 2023-12-01T01:22:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.815+0000 7f46a7e97700 1 -- 172.21.15.77:0/3318446958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46a01018b0 msgr2=0x7f46a0103ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:02.819 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.815+0000 7f46a7e97700 1 --2- 172.21.15.77:0/3318446958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46a01018b0 0x7f46a0103ca0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4698009f90 tx=0x7f469800b330 comp rx=0 tx=0).stop 2023-12-01T01:22:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.816+0000 7f46a7e97700 1 -- 172.21.15.77:0/3318446958 shutdown_connections 2023-12-01T01:22:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.816+0000 7f46a7e97700 1 --2- 172.21.15.77:0/3318446958 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46a0106b10 0x7f46a0108f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.816+0000 7f46a7e97700 1 --2- 172.21.15.77:0/3318446958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46a01018b0 0x7f46a0103ca0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.816+0000 7f46a7e97700 1 --2- 172.21.15.77:0/3318446958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46a01041e0 0x7f46a01065d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.816+0000 7f46a7e97700 1 -- 172.21.15.77:0/3318446958 >> 172.21.15.77:0/3318446958 conn(0x7f46a00fb1e0 msgr2=0x7f46a00fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.816+0000 7f46a7e97700 1 -- 172.21.15.77:0/3318446958 shutdown_connections 2023-12-01T01:22:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.816+0000 7f46a7e97700 1 -- 172.21.15.77:0/3318446958 wait complete. 2023-12-01T01:22:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.817+0000 7f46a7e97700 1 Processor -- start 2023-12-01T01:22:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.817+0000 7f46a7e97700 1 -- start start 2023-12-01T01:22:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.817+0000 7f46a7e97700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46a01018b0 0x7f46a0196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.817+0000 7f46a7e97700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46a01041e0 0x7f46a01970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.817+0000 7f46a7e97700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46a0106b10 0x7f46a0197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.817+0000 7f46a7e97700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a0198580 con 0x7f46a0106b10 2023-12-01T01:22:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.817+0000 7f46a7e97700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a01986f0 con 0x7f46a01041e0 2023-12-01T01:22:02.820 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.817+0000 7f46a7e97700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a01988b0 con 0x7f46a01018b0 2023-12-01T01:22:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.817+0000 7f46a6434700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46a0106b10 0x7f46a0197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.817+0000 7f46a6434700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46a0106b10 0x7f46a0197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32826/0 (socket says 172.21.15.77:32826) 2023-12-01T01:22:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.817+0000 7f46a6434700 1 -- 172.21.15.77:0/1807913480 learned_addr learned my addr 172.21.15.77:0/1807913480 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:22:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.817+0000 7f46a5432700 1 --2- 172.21.15.77:0/1807913480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46a01041e0 0x7f46a01970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.817+0000 7f46a5c33700 1 --2- 172.21.15.77:0/1807913480 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46a01018b0 0x7f46a0196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.818+0000 7f46a6434700 1 -- 172.21.15.77:0/1807913480 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46a01018b0 msgr2=0x7f46a0196b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.818+0000 7f46a6434700 1 --2- 172.21.15.77:0/1807913480 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46a01018b0 0x7f46a0196b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.818+0000 7f46a6434700 1 -- 172.21.15.77:0/1807913480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46a01041e0 msgr2=0x7f46a01970a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.818+0000 7f46a6434700 1 --2- 172.21.15.77:0/1807913480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46a01041e0 0x7f46a01970a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.818+0000 7f46a6434700 1 -- 172.21.15.77:0/1807913480 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4698009c40 con 0x7f46a0106b10 2023-12-01T01:22:02.822 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.818+0000 7f46a6434700 1 --2- 172.21.15.77:0/1807913480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46a0106b10 0x7f46a0197f20 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f469400d980 tx=0x7f469400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:02.822 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.818+0000 7f4692ffd700 1 -- 172.21.15.77:0/1807913480 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4694004a30 con 0x7f46a0106b10 2023-12-01T01:22:02.822 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.819+0000 7f4692ffd700 1 -- 172.21.15.77:0/1807913480 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46940054e0 con 0x7f46a0106b10 2023-12-01T01:22:02.822 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.819+0000 7f4692ffd700 1 -- 172.21.15.77:0/1807913480 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469400fc50 con 0x7f46a0106b10 2023-12-01T01:22:02.822 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.820+0000 7f46a7e97700 1 -- 172.21.15.77:0/1807913480 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46a0198b90 con 0x7f46a0106b10 2023-12-01T01:22:02.822 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.820+0000 7f46a7e97700 1 -- 172.21.15.77:0/1807913480 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46a00ff880 con 0x7f46a0106b10 2023-12-01T01:22:02.830 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.825+0000 7f46b4027b80 1 Processor -- start 2023-12-01T01:22:02.830 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.826+0000 7f4692ffd700 1 -- 172.21.15.77:0/1807913480 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f4694005650 con 0x7f46a0106b10 2023-12-01T01:22:02.830 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.827+0000 7f46b4027b80 1 -- start start 2023-12-01T01:22:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.827+0000 7f46b4027b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561a5cea680 0x5561a5ceaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.827+0000 7f46b4027b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561a5ceafe0 0x5561a5ceb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.827+0000 7f46b4027b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561a5cebed0 0x5561a5cec350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.827+0000 7f46b4027b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a5c429b0 con 0x5561a5cebed0 2023-12-01T01:22:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.827+0000 7f46b4027b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a5c40370 con 0x5561a5cea680 2023-12-01T01:22:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.827+0000 7f46b4027b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a5c3bc60 con 0x5561a5ceafe0 2023-12-01T01:22:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.827+0000 7f46a5432700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561a5cea680 0x5561a5ceaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.827+0000 7f46a5432700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561a5cea680 0x5561a5ceaaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:39982/0 (socket says 172.21.15.77:39982) 2023-12-01T01:22:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.827+0000 7f46a5432700 1 -- 172.21.15.77:0/2503952376 learned_addr learned my addr 172.21.15.77:0/2503952376 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:22:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.828+0000 7f4692ffd700 1 --2- 172.21.15.77:0/1807913480 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f468c0a05a0 0x7f468c0a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.828+0000 7f46a6434700 1 --2- 172.21.15.77:0/1807913480 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f468c0a05a0 0x7f468c0a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.828+0000 7f4692ffd700 1 -- 172.21.15.77:0/1807913480 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f46940cb6b0 con 0x7f46a0106b10 2023-12-01T01:22:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.829+0000 7f46a5c33700 1 --2- 172.21.15.77:0/2503952376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561a5ceafe0 0x5561a5ceb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.829+0000 7f46a6434700 1 --2- 172.21.15.77:0/1807913480 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f468c0a05a0 0x7f468c0a2a60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4694098f70 tx=0x7f4694068700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.829+0000 7f46a5432700 1 --2- 172.21.15.77:0/2503952376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561a5cebed0 0x5561a5cec350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:02.832 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.829+0000 7f46a5432700 1 -- 172.21.15.77:0/2503952376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561a5ceafe0 msgr2=0x5561a5ceb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:02.832 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.829+0000 7f46a5432700 1 --2- 172.21.15.77:0/2503952376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561a5ceafe0 0x5561a5ceb8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:02.832 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.829+0000 7f46a5432700 1 -- 172.21.15.77:0/2503952376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561a5cebed0 msgr2=0x5561a5cec350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:02.832 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.829+0000 7f46a5432700 1 --2- 172.21.15.77:0/2503952376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561a5cebed0 0x5561a5cec350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:02.832 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.829+0000 7f46a5432700 1 -- 172.21.15.77:0/2503952376 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561a5a69d80 con 0x5561a5cea680 2023-12-01T01:22:02.832 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.830+0000 7f46a5432700 1 --2- 172.21.15.77:0/2503952376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561a5cea680 0x5561a5ceaaa0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f469c010710 tx=0x7f469c012040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36577379755f48a9 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:02.832 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.831+0000 7f468bfff700 1 -- 172.21.15.77:0/2503952376 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469c015040 con 0x5561a5cea680 2023-12-01T01:22:02.833 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.831+0000 7f468bfff700 1 -- 172.21.15.77:0/2503952376 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f469c016040 con 0x5561a5cea680 2023-12-01T01:22:02.833 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.831+0000 7f468bfff700 1 -- 172.21.15.77:0/2503952376 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469c01d480 con 0x5561a5cea680 2023-12-01T01:22:02.833 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.832+0000 7f46b4027b80 1 -- 172.21.15.77:0/2503952376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561a5cea680 msgr2=0x5561a5ceaaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:02.833 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.832+0000 7f46b4027b80 1 --2- 172.21.15.77:0/2503952376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561a5cea680 0x5561a5ceaaa0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f469c010710 tx=0x7f469c012040 comp rx=0 tx=0).stop 2023-12-01T01:22:02.833 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.832+0000 7f46b4027b80 1 -- 172.21.15.77:0/2503952376 shutdown_connections 2023-12-01T01:22:02.833 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.832+0000 7f46b4027b80 1 --2- 172.21.15.77:0/2503952376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561a5cebed0 0x5561a5cec350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:02.834 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.832+0000 7f46b4027b80 1 --2- 172.21.15.77:0/2503952376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561a5cea680 0x5561a5ceaaa0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:02.834 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.832+0000 7f46b4027b80 1 --2- 172.21.15.77:0/2503952376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561a5ceafe0 0x5561a5ceb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:02.834 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.832+0000 7f46b4027b80 1 -- 172.21.15.77:0/2503952376 >> 172.21.15.77:0/2503952376 conn(0x5561a5c4a070 msgr2=0x5561a5ce7040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:02.834 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.832+0000 7f46b4027b80 1 -- 172.21.15.77:0/2503952376 shutdown_connections 2023-12-01T01:22:02.834 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.832+0000 7f46b4027b80 1 -- 172.21.15.77:0/2503952376 wait complete. 2023-12-01T01:22:02.834 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.833+0000 7f46b4027b80 1 -- start start 2023-12-01T01:22:02.834 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.833+0000 7f46b4027b80 1 Processor -- start 2023-12-01T01:22:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.835+0000 7f46b4027b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561a5cea680 0x5561a5d05560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.835+0000 7f46b4027b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561a5ceafe0 0x5561a5d05aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.835+0000 7f46b4027b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561a5cebed0 0x5561a5d0cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.835+0000 7f46b4027b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a5bd84b0 con 0x5561a5cebed0 2023-12-01T01:22:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.835+0000 7f46b4027b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a5bd8150 con 0x5561a5ceafe0 2023-12-01T01:22:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.835+0000 7f46b4027b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a5abf8c0 con 0x5561a5cea680 2023-12-01T01:22:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.835+0000 7f46b4027b80 10 client.0 _ll_register_callbacks cb 0x5561a5b25a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:22:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.836+0000 7f46a5432700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561a5cea680 0x5561a5d05560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.836+0000 7f46a5432700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561a5cea680 0x5561a5d05560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:47258/0 (socket says 172.21.15.77:47258) 2023-12-01T01:22:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.836+0000 7f46a5432700 1 -- 172.21.15.77:0/633222180 learned_addr learned my addr 172.21.15.77:0/633222180 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:22:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.836+0000 7f46a5c33700 1 --2- 172.21.15.77:0/633222180 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561a5ceafe0 0x5561a5d05aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.836+0000 7f46a5c33700 1 -- 172.21.15.77:0/633222180 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561a5cea680 msgr2=0x5561a5d05560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:02.839 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.836+0000 7f46a5c33700 1 --2- 172.21.15.77:0/633222180 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561a5cea680 0x5561a5d05560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:02.839 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.836+0000 7f46a5c33700 1 -- 172.21.15.77:0/633222180 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561a5cebed0 msgr2=0x5561a5d0cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:02.839 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.836+0000 7f46a5c33700 1 --2- 172.21.15.77:0/633222180 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561a5cebed0 0x5561a5d0cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:02.839 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.836+0000 7f46a5c33700 1 -- 172.21.15.77:0/633222180 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f469c010c20 con 0x5561a5ceafe0 2023-12-01T01:22:02.839 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.837+0000 7f46a5432700 1 --2- 172.21.15.77:0/633222180 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561a5cea680 0x5561a5d05560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-01T01:22:02.840 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.837+0000 7f46a5c33700 1 --2- 172.21.15.77:0/633222180 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561a5ceafe0 0x5561a5d05aa0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4698000f80 tx=0x7f4698003c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b36a648d4e4d043 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:02.840 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.837+0000 7f46897fa700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2023-12-01T01:22:02.840 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.837+0000 7f46897fa700 1 -- 172.21.15.77:0/633222180 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469800e040 con 0x5561a5ceafe0 2023-12-01T01:22:02.840 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.837+0000 7f46b4027b80 1 -- 172.21.15.77:0/633222180 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5561a5c66cb0 con 0x5561a5ceafe0 2023-12-01T01:22:02.840 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.837+0000 7f46897fa700 1 -- 172.21.15.77:0/633222180 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f46980052b0 con 0x5561a5ceafe0 2023-12-01T01:22:02.840 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.838+0000 7f46897fa700 1 -- 172.21.15.77:0/633222180 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4698003e50 con 0x5561a5ceafe0 2023-12-01T01:22:02.840 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.838+0000 7f46b4027b80 10 client.25784 fetch_fsmap learned FSMap version 32 2023-12-01T01:22:02.840 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.838+0000 7f46b4027b80 1 -- 172.21.15.77:0/633222180 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5561a5a52c00 con 0x5561a5ceafe0 2023-12-01T01:22:02.840 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.838+0000 7f46897fa700 1 -- 172.21.15.77:0/633222180 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5561a5c66cb0 con 0x5561a5ceafe0 2023-12-01T01:22:02.840 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.838+0000 7f46897fa700 1 -- 172.21.15.77:0/633222180 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f4698019070 con 0x5561a5ceafe0 2023-12-01T01:22:02.841 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.838+0000 7f46b4027b80 10 client.25784 fetch_fsmap finished waiting for FSMap version 32 2023-12-01T01:22:02.841 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.838+0000 7f46b4027b80 10 client.25784 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-01T01:22:02.841 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.838+0000 7f46b4027b80 20 client.25784 populate_metadata read hostname 'smithi077' 2023-12-01T01:22:02.841 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.838+0000 7f46b4027b80 1 --2- 172.21.15.77:0/633222180 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5561a5d11470 0x5561a5d13890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:02.841 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.838+0000 7f46b4027b80 4 client.25784 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:22:02.841 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.838+0000 7f46b4027b80 1 -- 172.21.15.77:0/633222180 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5561a5cb9db0 con 0x5561a5d11470 2023-12-01T01:22:02.841 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.839+0000 7f46a5432700 1 --2- 172.21.15.77:0/633222180 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5561a5d11470 0x5561a5d13890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:02.844 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.843+0000 7f46a5432700 1 --2- 172.21.15.77:0/633222180 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5561a5d11470 0x5561a5d13890 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=936d92edafde63eb server_cookie=597b94441a9a10b in_seq=0 out_seq=0 2023-12-01T01:22:02.844 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.843+0000 7f46897fa700 10 client.25784 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:22:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.845+0000 7f46897fa700 1 -- 172.21.15.77:0/633222180 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5561a5cb9db0 con 0x5561a5d11470 2023-12-01T01:22:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.845+0000 7f46897fa700 10 client.25784 handle_command_reply: tid=0 2023-12-01T01:22:02.847 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.846+0000 7f46b4027b80 1 client.25784 shutdown 2023-12-01T01:22:02.848 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.847+0000 7f46b4027b80 1 -- 172.21.15.77:0/633222180 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561a5ceafe0 msgr2=0x5561a5d05aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:02.848 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.847+0000 7f46b4027b80 1 --2- 172.21.15.77:0/633222180 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561a5ceafe0 0x5561a5d05aa0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4698000f80 tx=0x7f4698003c30 comp rx=0 tx=0).stop 2023-12-01T01:22:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.849+0000 7f46b4027b80 1 -- 172.21.15.77:0/633222180 shutdown_connections 2023-12-01T01:22:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.849+0000 7f46b4027b80 1 --2- 172.21.15.77:0/633222180 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561a5cebed0 0x5561a5d0cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.849+0000 7f46b4027b80 1 --2- 172.21.15.77:0/633222180 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561a5cea680 0x5561a5d05560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.849+0000 7f46b4027b80 1 --2- 172.21.15.77:0/633222180 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5561a5d11470 0x5561a5d13890 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.849+0000 7f46b4027b80 1 --2- 172.21.15.77:0/633222180 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561a5ceafe0 0x5561a5d05aa0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.849+0000 7f46b4027b80 1 -- 172.21.15.77:0/633222180 >> 172.21.15.77:0/633222180 conn(0x5561a5c4a070 msgr2=0x5561a5ce53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:02.851 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.849+0000 7f46897fa700 0 client.25784 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:22:02.851 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.849+0000 7f46b4027b80 1 -- 172.21.15.77:0/633222180 shutdown_connections 2023-12-01T01:22:02.851 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.849+0000 7f46b4027b80 1 -- 172.21.15.77:0/633222180 wait complete. 2023-12-01T01:22:02.854 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.850+0000 7f46b4027b80 20 client.25784 trim_cache size 0 max 16384 2023-12-01T01:22:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.875+0000 7f46b4027b80 1 Processor -- start 2023-12-01T01:22:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.875+0000 7f46b4027b80 1 -- start start 2023-12-01T01:22:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.875+0000 7f46b4027b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561a5cea680 0x5561a5d0aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.875+0000 7f46b4027b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561a5ceafe0 0x5561a5d13480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.875+0000 7f46b4027b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561a5cebed0 0x5561a5d05560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.875+0000 7f46b4027b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a5d11000 con 0x5561a5cebed0 2023-12-01T01:22:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.875+0000 7f46b4027b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a5cdd9f0 con 0x5561a5cea680 2023-12-01T01:22:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.875+0000 7f46b4027b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a5d04a10 con 0x5561a5ceafe0 2023-12-01T01:22:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.875+0000 7f46a5432700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561a5cebed0 0x5561a5d05560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.875+0000 7f46a5c33700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561a5ceafe0 0x5561a5d13480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.875+0000 7f46a5c33700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561a5ceafe0 0x5561a5d13480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.77:47266/0 (socket says 172.21.15.77:47266) 2023-12-01T01:22:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.875+0000 7f46a5432700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561a5cebed0 0x5561a5d05560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32848/0 (socket says 172.21.15.77:32848) 2023-12-01T01:22:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.875+0000 7f46a5c33700 1 -- 172.21.15.77:0/1158553310 learned_addr learned my addr 172.21.15.77:0/1158553310 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:22:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.875+0000 7f46a5432700 1 --2- 172.21.15.77:0/1158553310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561a5cea680 0x5561a5d0aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.876+0000 7f46a5432700 1 -- 172.21.15.77:0/1158553310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561a5ceafe0 msgr2=0x5561a5d13480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.876+0000 7f46a5432700 1 --2- 172.21.15.77:0/1158553310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561a5ceafe0 0x5561a5d13480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.876+0000 7f46a5432700 1 -- 172.21.15.77:0/1158553310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561a5cea680 msgr2=0x5561a5d0aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.876+0000 7f46a5432700 1 --2- 172.21.15.77:0/1158553310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561a5cea680 0x5561a5d0aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.876+0000 7f46a5432700 1 -- 172.21.15.77:0/1158553310 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561a5a901f0 con 0x5561a5cebed0 2023-12-01T01:22:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.876+0000 7f46a5432700 1 --2- 172.21.15.77:0/1158553310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561a5cebed0 0x5561a5d05560 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f469c012010 tx=0x7f469c0106d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8906b3b60a1cfeb server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.876+0000 7f4690ff9700 1 -- 172.21.15.77:0/1158553310 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469c015070 con 0x5561a5cebed0 2023-12-01T01:22:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.876+0000 7f4690ff9700 1 -- 172.21.15.77:0/1158553310 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f469c032040 con 0x5561a5cebed0 2023-12-01T01:22:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.876+0000 7f4690ff9700 1 -- 172.21.15.77:0/1158553310 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469c031070 con 0x5561a5cebed0 2023-12-01T01:22:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.877+0000 7f46b4027b80 1 -- 172.21.15.77:0/1158553310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561a5cebed0 msgr2=0x5561a5d05560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.877+0000 7f46b4027b80 1 --2- 172.21.15.77:0/1158553310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561a5cebed0 0x5561a5d05560 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f469c012010 tx=0x7f469c0106d0 comp rx=0 tx=0).stop 2023-12-01T01:22:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.879+0000 7f46b4027b80 1 -- 172.21.15.77:0/1158553310 shutdown_connections 2023-12-01T01:22:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.879+0000 7f46b4027b80 1 --2- 172.21.15.77:0/1158553310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561a5cebed0 0x5561a5d05560 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.879+0000 7f46b4027b80 1 --2- 172.21.15.77:0/1158553310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561a5cea680 0x5561a5d0aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.879+0000 7f46b4027b80 1 --2- 172.21.15.77:0/1158553310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561a5ceafe0 0x5561a5d13480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.879+0000 7f46b4027b80 1 -- 172.21.15.77:0/1158553310 >> 172.21.15.77:0/1158553310 conn(0x5561a5c4a070 msgr2=0x5561a5d0d310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.882+0000 7f46b4027b80 1 -- 172.21.15.77:0/1158553310 shutdown_connections 2023-12-01T01:22:02.884 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.883+0000 7f46b4027b80 1 -- 172.21.15.77:0/1158553310 wait complete. 2023-12-01T01:22:02.884 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.883+0000 7f46b4027b80 1 -- start start 2023-12-01T01:22:02.884 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.883+0000 7f46b4027b80 1 Processor -- start 2023-12-01T01:22:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.884+0000 7f46b4027b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561a5cea680 0x5561a5d0aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.884+0000 7f46b4027b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561a5ceafe0 0x5561a5d5ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.884+0000 7f46b4027b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561a5cebed0 0x5561a5d61520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.884+0000 7f46b4027b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a5d61c30 con 0x5561a5ceafe0 2023-12-01T01:22:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.884+0000 7f46b4027b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a5d61da0 con 0x5561a5cebed0 2023-12-01T01:22:02.886 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.884+0000 7f46b4027b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a5d61f60 con 0x5561a5cea680 2023-12-01T01:22:02.886 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.884+0000 7f46b4027b80 10 client.0 _ll_register_callbacks cb 0x7f4698007730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:22:02.886 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.884+0000 7f46a5432700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561a5cebed0 0x5561a5d61520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:02.886 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.884+0000 7f46a5432700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561a5cebed0 0x5561a5d61520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.77:40006/0 (socket says 172.21.15.77:40006) 2023-12-01T01:22:02.886 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.884+0000 7f46a5432700 1 -- 172.21.15.77:0/2849854453 learned_addr learned my addr 172.21.15.77:0/2849854453 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:22:02.886 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.884+0000 7f46a5432700 1 --2- 172.21.15.77:0/2849854453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561a5cea680 0x5561a5d0aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.886+0000 7f46a5432700 1 -- 172.21.15.77:0/2849854453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561a5cea680 msgr2=0x5561a5d0aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.886+0000 7f46a5432700 1 --2- 172.21.15.77:0/2849854453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561a5cea680 0x5561a5d0aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.886+0000 7f46a5432700 1 -- 172.21.15.77:0/2849854453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561a5ceafe0 msgr2=0x5561a5d5ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.886+0000 7f46a5432700 1 --2- 172.21.15.77:0/2849854453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561a5ceafe0 0x5561a5d5ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:02.888 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.886+0000 7f46a5432700 1 -- 172.21.15.77:0/2849854453 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5561a5a69d80 con 0x5561a5cebed0 2023-12-01T01:22:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.887+0000 7f46a5432700 1 --2- 172.21.15.77:0/2849854453 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561a5cebed0 0x5561a5d61520 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f469c02c580 tx=0x7f469c005a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3263ab4611750b38 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:02.889 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.887+0000 7f46897fa700 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2023-12-01T01:22:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.887+0000 7f46897fa700 1 -- 172.21.15.77:0/2849854453 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469c015030 con 0x5561a5cebed0 2023-12-01T01:22:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.887+0000 7f46897fa700 1 -- 172.21.15.77:0/2849854453 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f469c000e40 con 0x5561a5cebed0 2023-12-01T01:22:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.887+0000 7f46b4027b80 1 -- 172.21.15.77:0/2849854453 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f469c016040 con 0x5561a5cebed0 2023-12-01T01:22:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.887+0000 7f46897fa700 1 -- 172.21.15.77:0/2849854453 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f469c031070 con 0x5561a5cebed0 2023-12-01T01:22:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.887+0000 7f46b4027b80 10 client.25790 fetch_fsmap learned FSMap version 32 2023-12-01T01:22:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.887+0000 7f46b4027b80 1 -- 172.21.15.77:0/2849854453 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5561a5a68250 con 0x5561a5cebed0 2023-12-01T01:22:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.888+0000 7f46897fa700 1 -- 172.21.15.77:0/2849854453 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f469c04a020 con 0x5561a5cebed0 2023-12-01T01:22:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.888+0000 7f46897fa700 1 -- 172.21.15.77:0/2849854453 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f469c009a20 con 0x5561a5cebed0 2023-12-01T01:22:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.888+0000 7f46b4027b80 10 client.25790 fetch_fsmap finished waiting for FSMap version 32 2023-12-01T01:22:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.888+0000 7f46b4027b80 10 client.25790 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-01T01:22:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.888+0000 7f46b4027b80 20 client.25790 populate_metadata read hostname 'smithi077' 2023-12-01T01:22:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.888+0000 7f46b4027b80 1 --2- 172.21.15.77:0/2849854453 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5561a5d63530 0x5561a5d65950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.888+0000 7f46b4027b80 4 client.25790 mds_command: new command op to 24500 tid=0[{"prefix": "flush journal"}] 2023-12-01T01:22:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.888+0000 7f46b4027b80 1 -- 172.21.15.77:0/2849854453 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5561a5c34ce0 con 0x5561a5d63530 2023-12-01T01:22:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.891+0000 7f46a5c33700 1 --2- 172.21.15.77:0/2849854453 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5561a5d63530 0x5561a5d65950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.892+0000 7f46a5c33700 1 --2- 172.21.15.77:0/2849854453 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5561a5d63530 0x5561a5d65950 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d1490086da65b70 server_cookie=1247627b5007fe9a in_seq=0 out_seq=0 2023-12-01T01:22:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.892+0000 7f46897fa700 10 client.25790 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:22:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.923+0000 7f46897fa700 1 -- 172.21.15.77:0/2849854453 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5561a5c34ce0 con 0x5561a5d63530 2023-12-01T01:22:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.923+0000 7f46897fa700 10 client.25790 handle_command_reply: tid=0 2023-12-01T01:22:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.923+0000 7f46b4027b80 1 client.25790 shutdown 2023-12-01T01:22:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.923+0000 7f46b4027b80 1 -- 172.21.15.77:0/2849854453 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561a5cebed0 msgr2=0x5561a5d61520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.923+0000 7f46b4027b80 1 --2- 172.21.15.77:0/2849854453 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561a5cebed0 0x5561a5d61520 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f469c02c580 tx=0x7f469c005a00 comp rx=0 tx=0).stop 2023-12-01T01:22:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.923+0000 7f46b4027b80 1 -- 172.21.15.77:0/2849854453 shutdown_connections 2023-12-01T01:22:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.923+0000 7f46b4027b80 1 --2- 172.21.15.77:0/2849854453 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5561a5d63530 0x5561a5d65950 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.923+0000 7f46b4027b80 1 --2- 172.21.15.77:0/2849854453 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5561a5cebed0 0x5561a5d61520 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.923+0000 7f46897fa700 0 client.25790 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:22:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.923+0000 7f46b4027b80 1 --2- 172.21.15.77:0/2849854453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5561a5cea680 0x5561a5d0aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.923+0000 7f46b4027b80 1 --2- 172.21.15.77:0/2849854453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561a5ceafe0 0x5561a5d5ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.923+0000 7f46b4027b80 1 -- 172.21.15.77:0/2849854453 >> 172.21.15.77:0/2849854453 conn(0x5561a5c4a070 msgr2=0x5561a5d0d310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.923+0000 7f46b4027b80 1 -- 172.21.15.77:0/2849854453 shutdown_connections 2023-12-01T01:22:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.923+0000 7f46b4027b80 1 -- 172.21.15.77:0/2849854453 wait complete. 2023-12-01T01:22:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.923+0000 7f46b4027b80 20 client.25790 trim_cache size 0 max 16384 2023-12-01T01:22:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout:{ 2023-12-01T01:22:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout: "message": "", 2023-12-01T01:22:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout: "return_code": 0 2023-12-01T01:22:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stdout:} 2023-12-01T01:22:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.924+0000 7f46a7e97700 1 -- 172.21.15.77:0/1807913480 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f468c0a05a0 msgr2=0x7f468c0a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.924+0000 7f46a7e97700 1 --2- 172.21.15.77:0/1807913480 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f468c0a05a0 0x7f468c0a2a60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4694098f70 tx=0x7f4694068700 comp rx=0 tx=0).stop 2023-12-01T01:22:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.924+0000 7f46a7e97700 1 -- 172.21.15.77:0/1807913480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46a0106b10 msgr2=0x7f46a0197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.924+0000 7f46a7e97700 1 --2- 172.21.15.77:0/1807913480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46a0106b10 0x7f46a0197f20 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f469400d980 tx=0x7f469400dc90 comp rx=0 tx=0).stop 2023-12-01T01:22:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.924+0000 7f46a7e97700 1 -- 172.21.15.77:0/1807913480 shutdown_connections 2023-12-01T01:22:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.924+0000 7f46a7e97700 1 --2- 172.21.15.77:0/1807913480 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f468c0a05a0 0x7f468c0a2a60 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.924+0000 7f46a7e97700 1 --2- 172.21.15.77:0/1807913480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46a0106b10 0x7f46a0197f20 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.924+0000 7f46a7e97700 1 --2- 172.21.15.77:0/1807913480 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46a01018b0 0x7f46a0196b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.924+0000 7f46a7e97700 1 --2- 172.21.15.77:0/1807913480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46a01041e0 0x7f46a01970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.924+0000 7f46a7e97700 1 -- 172.21.15.77:0/1807913480 >> 172.21.15.77:0/1807913480 conn(0x7f46a00fb1e0 msgr2=0x7f46a0102d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.924+0000 7f46a7e97700 1 -- 172.21.15.77:0/1807913480 shutdown_connections 2023-12-01T01:22:02.930 INFO:teuthology.task.background_exec.ubuntu@smithi077.front.sepia.ceph.com.smithi077.stderr:2023-12-01T01:22:02.924+0000 7f46a7e97700 1 -- 172.21.15.77:0/1807913480 wait complete. 2023-12-01T01:22:04.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:03 smithi203 ceph-mon[103581]: pgmap v423: 97 pgs: 97 active+clean; 62 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.4 MiB/s wr, 1.06k op/s 2023-12-01T01:22:04.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:03 smithi203 ceph-mon[103581]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:22:04.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:03 smithi137 ceph-mon[103312]: pgmap v423: 97 pgs: 97 active+clean; 62 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.4 MiB/s wr, 1.06k op/s 2023-12-01T01:22:04.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:03 smithi137 ceph-mon[103312]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:22:04.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:03 smithi077 ceph-mon[98867]: pgmap v423: 97 pgs: 97 active+clean; 62 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.4 MiB/s wr, 1.06k op/s 2023-12-01T01:22:04.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:03 smithi077 ceph-mon[98867]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:22:05.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:04 smithi203 ceph-mon[103581]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:22:05.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:04 smithi203 ceph-mon[103581]: pgmap v424: 97 pgs: 97 active+clean; 57 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.1 MiB/s wr, 962 op/s 2023-12-01T01:22:05.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:04 smithi137 ceph-mon[103312]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:22:05.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:04 smithi137 ceph-mon[103312]: pgmap v424: 97 pgs: 97 active+clean; 57 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.1 MiB/s wr, 962 op/s 2023-12-01T01:22:05.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:04 smithi077 ceph-mon[98867]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:22:05.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:04 smithi077 ceph-mon[98867]: pgmap v424: 97 pgs: 97 active+clean; 57 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.1 MiB/s wr, 962 op/s 2023-12-01T01:22:06.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:05 smithi203 ceph-mon[103581]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:22:06.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:05 smithi203 ceph-mon[103581]: scrub summary: idle+waiting paths [/] 2023-12-01T01:22:06.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:05 smithi137 ceph-mon[103312]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:22:06.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:05 smithi137 ceph-mon[103312]: scrub summary: idle+waiting paths [/] 2023-12-01T01:22:06.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:05 smithi077 ceph-mon[98867]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:22:06.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:05 smithi077 ceph-mon[98867]: scrub summary: idle+waiting paths [/] 2023-12-01T01:22:08.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:07 smithi203 ceph-mon[103581]: pgmap v425: 97 pgs: 97 active+clean; 59 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 1.9 MiB/s wr, 742 op/s 2023-12-01T01:22:08.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:07 smithi203 ceph-mon[103581]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:22:08.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:07 smithi137 ceph-mon[103312]: pgmap v425: 97 pgs: 97 active+clean; 59 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 1.9 MiB/s wr, 742 op/s 2023-12-01T01:22:08.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:07 smithi137 ceph-mon[103312]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:22:08.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:07 smithi077 ceph-mon[98867]: pgmap v425: 97 pgs: 97 active+clean; 59 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 1.9 MiB/s wr, 742 op/s 2023-12-01T01:22:08.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:07 smithi077 ceph-mon[98867]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:22:08.206 INFO:tasks.workunit:Stopping ['suites/fsx.sh'] on client.0... 2023-12-01T01:22:08.206 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-12-01T01:22:08.508 DEBUG:teuthology.parallel:result is None 2023-12-01T01:22:08.508 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2023-12-01T01:22:08.542 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2023-12-01T01:22:08.542 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2023-12-01T01:22:08.551 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi077.front.sepia.ceph.com 2023-12-01T01:22:08.590 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-12-01T01:22:08.600 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-12-01T01:22:08.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:08 smithi077 ceph-mon[98867]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:22:08.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:08 smithi077 ceph-mon[98867]: pgmap v426: 97 pgs: 97 active+clean; 57 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1.8 MiB/s wr, 702 op/s 2023-12-01T01:22:09.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:08 smithi203 ceph-mon[103581]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:22:09.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:08 smithi203 ceph-mon[103581]: pgmap v426: 97 pgs: 97 active+clean; 57 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1.8 MiB/s wr, 702 op/s 2023-12-01T01:22:09.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:08 smithi137 ceph-mon[103312]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:22:09.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:08 smithi137 ceph-mon[103312]: pgmap v426: 97 pgs: 97 active+clean; 57 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1.8 MiB/s wr, 702 op/s 2023-12-01T01:22:10.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:09 smithi203 ceph-mon[103581]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:22:10.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:09 smithi137 ceph-mon[103312]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:22:10.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:09 smithi077 ceph-mon[98867]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:22:11.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:10 smithi203 ceph-mon[103581]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:22:11.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:10 smithi203 ceph-mon[103581]: pgmap v427: 97 pgs: 97 active+clean; 43 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.7 MiB/s wr, 578 op/s 2023-12-01T01:22:11.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:10 smithi137 ceph-mon[103312]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:22:11.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:10 smithi137 ceph-mon[103312]: pgmap v427: 97 pgs: 97 active+clean; 43 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.7 MiB/s wr, 578 op/s 2023-12-01T01:22:11.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:10 smithi077 ceph-mon[98867]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:22:11.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:10 smithi077 ceph-mon[98867]: pgmap v427: 97 pgs: 97 active+clean; 43 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.7 MiB/s wr, 578 op/s 2023-12-01T01:22:13.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:12 smithi203 ceph-mon[103581]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:22:13.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:12 smithi203 ceph-mon[103581]: pgmap v428: 97 pgs: 97 active+clean; 39 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.3 MiB/s wr, 471 op/s 2023-12-01T01:22:13.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:12 smithi137 ceph-mon[103312]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:22:13.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:12 smithi137 ceph-mon[103312]: pgmap v428: 97 pgs: 97 active+clean; 39 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.3 MiB/s wr, 471 op/s 2023-12-01T01:22:13.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:12 smithi077 ceph-mon[98867]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:22:13.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:12 smithi077 ceph-mon[98867]: pgmap v428: 97 pgs: 97 active+clean; 39 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.3 MiB/s wr, 471 op/s 2023-12-01T01:22:15.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:15 smithi203 ceph-mon[103581]: pgmap v429: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 412 op/s 2023-12-01T01:22:15.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:15 smithi203 ceph-mon[103581]: scrub summary: idle 2023-12-01T01:22:15.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:15 smithi203 ceph-mon[103581]: scrub summary: idle 2023-12-01T01:22:15.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:15 smithi203 ceph-mon[103581]: scrub summary: idle 2023-12-01T01:22:15.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:15 smithi203 ceph-mon[103581]: scrub summary: idle 2023-12-01T01:22:15.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:15 smithi137 ceph-mon[103312]: pgmap v429: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 412 op/s 2023-12-01T01:22:15.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:15 smithi137 ceph-mon[103312]: scrub summary: idle 2023-12-01T01:22:15.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:15 smithi137 ceph-mon[103312]: scrub summary: idle 2023-12-01T01:22:15.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:15 smithi137 ceph-mon[103312]: scrub summary: idle 2023-12-01T01:22:15.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:15 smithi137 ceph-mon[103312]: scrub summary: idle 2023-12-01T01:22:15.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:15 smithi077 ceph-mon[98867]: pgmap v429: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 412 op/s 2023-12-01T01:22:15.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:15 smithi077 ceph-mon[98867]: scrub summary: idle 2023-12-01T01:22:15.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:15 smithi077 ceph-mon[98867]: scrub summary: idle 2023-12-01T01:22:15.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:15 smithi077 ceph-mon[98867]: scrub summary: idle 2023-12-01T01:22:15.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:15 smithi077 ceph-mon[98867]: scrub summary: idle 2023-12-01T01:22:16.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:16 smithi203 ceph-mon[103581]: scrub summary: idle 2023-12-01T01:22:16.778 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:16 smithi137 ceph-mon[103312]: scrub summary: idle 2023-12-01T01:22:16.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:16 smithi077 ceph-mon[98867]: scrub summary: idle 2023-12-01T01:22:17.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:17 smithi203 ceph-mon[103581]: pgmap v430: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 1006 KiB/s wr, 336 op/s 2023-12-01T01:22:17.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:17 smithi137 ceph-mon[103312]: pgmap v430: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 1006 KiB/s wr, 336 op/s 2023-12-01T01:22:17.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:17 smithi077 ceph-mon[98867]: pgmap v430: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 1006 KiB/s wr, 336 op/s 2023-12-01T01:22:19.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:19 smithi203 ceph-mon[103581]: pgmap v431: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 631 KiB/s wr, 262 op/s 2023-12-01T01:22:19.778 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:19 smithi137 ceph-mon[103312]: pgmap v431: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 631 KiB/s wr, 262 op/s 2023-12-01T01:22:19.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:19 smithi077 ceph-mon[98867]: pgmap v431: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 631 KiB/s wr, 262 op/s 2023-12-01T01:22:21.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:21 smithi203 ceph-mon[103581]: pgmap v432: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 451 KiB/s wr, 161 op/s 2023-12-01T01:22:21.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:21 smithi203 ceph-mon[103581]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:22:21.778 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:21 smithi137 ceph-mon[103312]: pgmap v432: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 451 KiB/s wr, 161 op/s 2023-12-01T01:22:21.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:21 smithi137 ceph-mon[103312]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:22:21.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:21 smithi077 ceph-mon[98867]: pgmap v432: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 451 KiB/s wr, 161 op/s 2023-12-01T01:22:21.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:21 smithi077 ceph-mon[98867]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:22:23.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:22 smithi203 ceph-mon[103581]: pgmap v433: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 78 KiB/s wr, 79 op/s 2023-12-01T01:22:23.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:22 smithi137 ceph-mon[103312]: pgmap v433: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 78 KiB/s wr, 79 op/s 2023-12-01T01:22:23.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:22 smithi077 ceph-mon[98867]: pgmap v433: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 78 KiB/s wr, 79 op/s 2023-12-01T01:22:25.765 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:25 smithi203 ceph-mon[103581]: pgmap v434: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.7 KiB/s wr, 31 op/s 2023-12-01T01:22:25.778 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:25 smithi137 ceph-mon[103312]: pgmap v434: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.7 KiB/s wr, 31 op/s 2023-12-01T01:22:25.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:25 smithi077 ceph-mon[98867]: pgmap v434: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.7 KiB/s wr, 31 op/s 2023-12-01T01:22:26.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:26 smithi203 ceph-mon[103581]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:22:26.778 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:26 smithi137 ceph-mon[103312]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:22:26.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:26 smithi077 ceph-mon[98867]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-01T01:22:27.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:27 smithi203 ceph-mon[103581]: pgmap v435: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 2.2 KiB/s wr, 14 op/s 2023-12-01T01:22:27.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:27 smithi137 ceph-mon[103312]: pgmap v435: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 2.2 KiB/s wr, 14 op/s 2023-12-01T01:22:27.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:27 smithi077 ceph-mon[98867]: pgmap v435: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 2.2 KiB/s wr, 14 op/s 2023-12-01T01:22:29.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:29 smithi203 ceph-mon[103581]: pgmap v436: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1023 B/s wr, 13 op/s 2023-12-01T01:22:29.778 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:29 smithi137 ceph-mon[103312]: pgmap v436: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1023 B/s wr, 13 op/s 2023-12-01T01:22:29.858 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-01T01:22:29.887 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:29 smithi077 ceph-mon[98867]: pgmap v436: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1023 B/s wr, 13 op/s 2023-12-01T01:22:29.996 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:29.994+0000 7fcd0a8d6700 1 -- 172.21.15.77:0/216924128 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd040ffc80 msgr2=0x7fcd041000a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:29.996 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:29.994+0000 7fcd0a8d6700 1 --2- 172.21.15.77:0/216924128 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd040ffc80 0x7fcd041000a0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fccec0099c0 tx=0x7fccec009cd0 comp rx=0 tx=0).stop 2023-12-01T01:22:29.996 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:29.994+0000 7fcd0a8d6700 1 -- 172.21.15.77:0/216924128 shutdown_connections 2023-12-01T01:22:29.996 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:29.994+0000 7fcd0a8d6700 1 --2- 172.21.15.77:0/216924128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd04101810 0x7fcd041080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:29.996 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:29.994+0000 7fcd0a8d6700 1 --2- 172.21.15.77:0/216924128 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd040ffc80 0x7fcd041000a0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:29.996 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:29.994+0000 7fcd0a8d6700 1 --2- 172.21.15.77:0/216924128 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd04100e70 0x7fcd041012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:29.997 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:29.994+0000 7fcd0a8d6700 1 -- 172.21.15.77:0/216924128 >> 172.21.15.77:0/216924128 conn(0x7fcd040fb1f0 msgr2=0x7fcd040fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:29.997 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:29.994+0000 7fcd0a8d6700 1 -- 172.21.15.77:0/216924128 shutdown_connections 2023-12-01T01:22:29.997 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:29.994+0000 7fcd0a8d6700 1 -- 172.21.15.77:0/216924128 wait complete. 2023-12-01T01:22:29.997 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:29.995+0000 7fcd0a8d6700 1 Processor -- start 2023-12-01T01:22:29.997 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:29.995+0000 7fcd0a8d6700 1 -- start start 2023-12-01T01:22:29.999 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:29.995+0000 7fcd0a8d6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd040ffc80 0x7fcd04068ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:29.999 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:29.995+0000 7fcd0a8d6700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd04100e70 0x7fcd0419b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:29.999 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:29.995+0000 7fcd0a8d6700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd04101810 0x7fcd0419dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:29.999 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:29.995+0000 7fcd0a8d6700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0419e520 con 0x7fcd040ffc80 2023-12-01T01:22:29.999 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:29.995+0000 7fcd0a8d6700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0419e690 con 0x7fcd04100e70 2023-12-01T01:22:29.999 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:29.995+0000 7fcd0a8d6700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0419e850 con 0x7fcd04101810 2023-12-01T01:22:29.999 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:29.996+0000 7fcd03fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd040ffc80 0x7fcd04068ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:29.999 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:29.996+0000 7fcd03fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd040ffc80 0x7fcd04068ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32862/0 (socket says 172.21.15.77:32862) 2023-12-01T01:22:30.000 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:29.996+0000 7fcd03fff700 1 -- 172.21.15.77:0/2304815630 learned_addr learned my addr 172.21.15.77:0/2304815630 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:22:30.000 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:29.996+0000 7fcd08e73700 1 --2- 172.21.15.77:0/2304815630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd04101810 0x7fcd0419dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.000 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:29.996+0000 7fcd037fe700 1 --2- 172.21.15.77:0/2304815630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd04100e70 0x7fcd0419b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.000 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:29.996+0000 7fcd03fff700 1 -- 172.21.15.77:0/2304815630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd04101810 msgr2=0x7fcd0419dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:30.000 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:29.996+0000 7fcd03fff700 1 --2- 172.21.15.77:0/2304815630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd04101810 0x7fcd0419dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.000 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:29.996+0000 7fcd03fff700 1 -- 172.21.15.77:0/2304815630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd04100e70 msgr2=0x7fcd0419b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:30.000 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:29.996+0000 7fcd03fff700 1 --2- 172.21.15.77:0/2304815630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd04100e70 0x7fcd0419b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.000 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:29.996+0000 7fcd03fff700 1 -- 172.21.15.77:0/2304815630 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccec0095f0 con 0x7fcd040ffc80 2023-12-01T01:22:30.000 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:29.996+0000 7fcd08e73700 1 --2- 172.21.15.77:0/2304815630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd04101810 0x7fcd0419dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:22:30.000 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:29.996+0000 7fcd03fff700 1 --2- 172.21.15.77:0/2304815630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd040ffc80 0x7fcd04068ac0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fccec009f50 tx=0x7fccec0057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:30.000 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:29.996+0000 7fcd017fa700 1 -- 172.21.15.77:0/2304815630 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccec01d070 con 0x7fcd040ffc80 2023-12-01T01:22:30.000 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:29.996+0000 7fcd017fa700 1 -- 172.21.15.77:0/2304815630 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccec004110 con 0x7fcd040ffc80 2023-12-01T01:22:30.000 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:29.996+0000 7fcd0a8d6700 1 -- 172.21.15.77:0/2304815630 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd0419ead0 con 0x7fcd040ffc80 2023-12-01T01:22:30.000 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:29.996+0000 7fcd017fa700 1 -- 172.21.15.77:0/2304815630 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccec022dd0 con 0x7fcd040ffc80 2023-12-01T01:22:30.001 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:29.996+0000 7fcd0a8d6700 1 -- 172.21.15.77:0/2304815630 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd0419ef80 con 0x7fcd040ffc80 2023-12-01T01:22:30.007 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:29.998+0000 7fcd017fa700 1 -- 172.21.15.77:0/2304815630 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fccec022500 con 0x7fcd040ffc80 2023-12-01T01:22:30.007 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:29.999+0000 7fcd017fa700 1 --2- 172.21.15.77:0/2304815630 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fccf00a0520 0x7fccf00a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.007 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:29.999+0000 7fcd037fe700 1 --2- 172.21.15.77:0/2304815630 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fccf00a0520 0x7fccf00a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.007 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:29.999+0000 7fcd017fa700 1 -- 172.21.15.77:0/2304815630 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fccec0ccd10 con 0x7fcd040ffc80 2023-12-01T01:22:30.007 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:29.999+0000 7fcd037fe700 1 --2- 172.21.15.77:0/2304815630 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fccf00a0520 0x7fccf00a29e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fccf400ba60 tx=0x7fccf4005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:30.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.000+0000 7fcd16a66b80 1 Processor -- start 2023-12-01T01:22:30.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.000+0000 7fcd16a66b80 1 -- start start 2023-12-01T01:22:30.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.000+0000 7fcd16a66b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578cb0ae0f0 0x5578cb0ae530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.000+0000 7fcd16a66b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578cb0aea70 0x5578cb0ace60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.000+0000 7fcd16a66b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578cb0ad3f0 0x5578cb0ad870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.000+0000 7fcd16a66b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578cb004a10 con 0x5578cb0aea70 2023-12-01T01:22:30.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.000+0000 7fcd16a66b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578cb0023d0 con 0x5578cb0ae0f0 2023-12-01T01:22:30.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.000+0000 7fcd16a66b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578caffdcc0 con 0x5578cb0ad3f0 2023-12-01T01:22:30.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.000+0000 7fcd037fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578cb0aea70 0x5578cb0ace60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.000+0000 7fcd037fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578cb0aea70 0x5578cb0ace60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32872/0 (socket says 172.21.15.77:32872) 2023-12-01T01:22:30.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.000+0000 7fcd037fe700 1 -- 172.21.15.77:0/496411332 learned_addr learned my addr 172.21.15.77:0/496411332 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:22:30.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.000+0000 7fcd08e73700 1 --2- 172.21.15.77:0/496411332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578cb0ad3f0 0x5578cb0ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.000+0000 7fcd03fff700 1 --2- 172.21.15.77:0/496411332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578cb0ae0f0 0x5578cb0ae530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.000+0000 7fcd037fe700 1 -- 172.21.15.77:0/496411332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578cb0ad3f0 msgr2=0x5578cb0ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:30.008 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.000+0000 7fcd037fe700 1 --2- 172.21.15.77:0/496411332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578cb0ad3f0 0x5578cb0ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.000+0000 7fcd037fe700 1 -- 172.21.15.77:0/496411332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578cb0ae0f0 msgr2=0x5578cb0ae530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:30.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.000+0000 7fcd037fe700 1 --2- 172.21.15.77:0/496411332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578cb0ae0f0 0x5578cb0ae530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.000+0000 7fcd037fe700 1 -- 172.21.15.77:0/496411332 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578cae2bde0 con 0x5578cb0aea70 2023-12-01T01:22:30.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.000+0000 7fcd037fe700 1 --2- 172.21.15.77:0/496411332 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578cb0aea70 0x5578cb0ace60 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fccf400dc20 tx=0x7fccf4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ab9b747e283824 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:30.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.001+0000 7fccfa7fc700 1 -- 172.21.15.77:0/496411332 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf400f9f0 con 0x5578cb0aea70 2023-12-01T01:22:30.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.001+0000 7fccfa7fc700 1 -- 172.21.15.77:0/496411332 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccf400fb50 con 0x5578cb0aea70 2023-12-01T01:22:30.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.001+0000 7fccfa7fc700 1 -- 172.21.15.77:0/496411332 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf401b5b0 con 0x5578cb0aea70 2023-12-01T01:22:30.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.001+0000 7fcd16a66b80 1 -- 172.21.15.77:0/496411332 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578cb0aea70 msgr2=0x5578cb0ace60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:30.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.001+0000 7fcd16a66b80 1 --2- 172.21.15.77:0/496411332 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578cb0aea70 0x5578cb0ace60 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fccf400dc20 tx=0x7fccf4010000 comp rx=0 tx=0).stop 2023-12-01T01:22:30.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.001+0000 7fcd16a66b80 1 -- 172.21.15.77:0/496411332 shutdown_connections 2023-12-01T01:22:30.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.001+0000 7fcd16a66b80 1 --2- 172.21.15.77:0/496411332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578cb0ad3f0 0x5578cb0ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.001+0000 7fcd16a66b80 1 --2- 172.21.15.77:0/496411332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578cb0ae0f0 0x5578cb0ae530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.001+0000 7fcd16a66b80 1 --2- 172.21.15.77:0/496411332 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578cb0aea70 0x5578cb0ace60 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.009 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.001+0000 7fcd16a66b80 1 -- 172.21.15.77:0/496411332 >> 172.21.15.77:0/496411332 conn(0x5578cb00c0d0 msgr2=0x5578cb0a7940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:30.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.001+0000 7fcd16a66b80 1 -- 172.21.15.77:0/496411332 shutdown_connections 2023-12-01T01:22:30.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.001+0000 7fcd16a66b80 1 -- 172.21.15.77:0/496411332 wait complete. 2023-12-01T01:22:30.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.002+0000 7fcd16a66b80 1 -- start start 2023-12-01T01:22:30.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.002+0000 7fcd16a66b80 1 Processor -- start 2023-12-01T01:22:30.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.002+0000 7fcd16a66b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578cb0ad3f0 0x5578cb0b6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.002+0000 7fcd16a66b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578cb0ae0f0 0x5578cb0b6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.002+0000 7fcd16a66b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578cb0aea70 0x5578cb0bded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.002+0000 7fcd16a66b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578caf9a510 con 0x5578cb0aea70 2023-12-01T01:22:30.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.002+0000 7fcd16a66b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578caf9a1b0 con 0x5578cb0ae0f0 2023-12-01T01:22:30.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.002+0000 7fcd16a66b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578cae81920 con 0x5578cb0ad3f0 2023-12-01T01:22:30.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.002+0000 7fcd16a66b80 10 client.0 _ll_register_callbacks cb 0x5578caee7a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:22:30.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.002+0000 7fcd08e73700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578cb0aea70 0x5578cb0bded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.011 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.002+0000 7fcd08e73700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578cb0aea70 0x5578cb0bded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32880/0 (socket says 172.21.15.77:32880) 2023-12-01T01:22:30.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.002+0000 7fcd08e73700 1 -- 172.21.15.77:0/4178820574 learned_addr learned my addr 172.21.15.77:0/4178820574 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:22:30.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.002+0000 7fcd03fff700 1 --2- 172.21.15.77:0/4178820574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578cb0ad3f0 0x5578cb0b6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.002+0000 7fcd037fe700 1 --2- 172.21.15.77:0/4178820574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578cb0ae0f0 0x5578cb0b6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.003+0000 7fcd08e73700 1 -- 172.21.15.77:0/4178820574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578cb0ad3f0 msgr2=0x5578cb0b6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:30.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.003+0000 7fcd08e73700 1 --2- 172.21.15.77:0/4178820574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578cb0ad3f0 0x5578cb0b6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.003+0000 7fcd08e73700 1 -- 172.21.15.77:0/4178820574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578cb0ae0f0 msgr2=0x5578cb0b6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:30.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.003+0000 7fcd08e73700 1 --2- 172.21.15.77:0/4178820574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578cb0ae0f0 0x5578cb0b6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.003+0000 7fcd08e73700 1 -- 172.21.15.77:0/4178820574 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccf400d8d0 con 0x5578cb0aea70 2023-12-01T01:22:30.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.003+0000 7fcd08e73700 1 --2- 172.21.15.77:0/4178820574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578cb0aea70 0x5578cb0bded0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fccfc007c40 tx=0x7fccfc007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e8c45565c1d193 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:30.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.003+0000 7fcce37fe700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:22:30.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.003+0000 7fcce37fe700 1 -- 172.21.15.77:0/4178820574 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc009a70 con 0x5578cb0aea70 2023-12-01T01:22:30.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.003+0000 7fcce37fe700 1 -- 172.21.15.77:0/4178820574 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fccfc012cb0 con 0x5578cb0aea70 2023-12-01T01:22:30.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.003+0000 7fcd16a66b80 1 -- 172.21.15.77:0/4178820574 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5578cb028d10 con 0x5578cb0aea70 2023-12-01T01:22:30.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.003+0000 7fcce37fe700 1 -- 172.21.15.77:0/4178820574 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccfc012e70 con 0x5578cb0aea70 2023-12-01T01:22:30.012 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.003+0000 7fcd16a66b80 10 client.16221 fetch_fsmap learned FSMap version 41 2023-12-01T01:22:30.013 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.003+0000 7fcd16a66b80 1 -- 172.21.15.77:0/4178820574 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5578cae14c60 con 0x5578cb0aea70 2023-12-01T01:22:30.013 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.003+0000 7fcce37fe700 1 -- 172.21.15.77:0/4178820574 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5578cb028d10 con 0x5578cb0aea70 2023-12-01T01:22:30.013 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.003+0000 7fcce37fe700 1 -- 172.21.15.77:0/4178820574 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 41) v1 ==== 7514+0+0 (secure 0 0 0) 0x7fccfc014040 con 0x5578cb0aea70 2023-12-01T01:22:30.013 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.003+0000 7fcd16a66b80 10 client.16221 fetch_fsmap finished waiting for FSMap version 41 2023-12-01T01:22:30.013 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.003+0000 7fcd16a66b80 10 client.16221 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-01T01:22:30.013 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.003+0000 7fcd16a66b80 20 client.16221 populate_metadata read hostname 'smithi077' 2023-12-01T01:22:30.013 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.003+0000 7fcd16a66b80 1 --2- 172.21.15.77:0/4178820574 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5578cb0c26f0 0x5578cb0c4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.013 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.003+0000 7fcd16a66b80 4 client.16221 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:22:30.013 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.003+0000 7fcd16a66b80 1 -- 172.21.15.77:0/4178820574 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5578cb07be10 con 0x5578cb0c26f0 2023-12-01T01:22:30.013 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.004+0000 7fcd03fff700 1 --2- 172.21.15.77:0/4178820574 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5578cb0c26f0 0x5578cb0c4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.013 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.004+0000 7fcce37fe700 10 client.16221 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:22:30.013 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.004+0000 7fcd03fff700 1 --2- 172.21.15.77:0/4178820574 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5578cb0c26f0 0x5578cb0c4b10 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ad6d9e5b39d1243 server_cookie=a31faccd5c978ec5 in_seq=0 out_seq=0 2023-12-01T01:22:30.014 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.005+0000 7fcce37fe700 1 -- 172.21.15.77:0/4178820574 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5578cb07be10 con 0x5578cb0c26f0 2023-12-01T01:22:30.014 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.005+0000 7fcce37fe700 10 client.16221 handle_command_reply: tid=0 2023-12-01T01:22:30.014 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.005+0000 7fcd16a66b80 1 client.16221 shutdown 2023-12-01T01:22:30.014 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.005+0000 7fcd16a66b80 1 -- 172.21.15.77:0/4178820574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578cb0aea70 msgr2=0x5578cb0bded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:30.014 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.005+0000 7fcd16a66b80 1 --2- 172.21.15.77:0/4178820574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578cb0aea70 0x5578cb0bded0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fccfc007c40 tx=0x7fccfc007f50 comp rx=0 tx=0).stop 2023-12-01T01:22:30.014 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.005+0000 7fcd16a66b80 1 -- 172.21.15.77:0/4178820574 shutdown_connections 2023-12-01T01:22:30.014 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.005+0000 7fcd16a66b80 1 --2- 172.21.15.77:0/4178820574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578cb0aea70 0x5578cb0bded0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.014 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.005+0000 7fcd16a66b80 1 --2- 172.21.15.77:0/4178820574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578cb0ad3f0 0x5578cb0b6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.014 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.005+0000 7fcd16a66b80 1 --2- 172.21.15.77:0/4178820574 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5578cb0c26f0 0x5578cb0c4b10 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.014 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.005+0000 7fcd16a66b80 1 --2- 172.21.15.77:0/4178820574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578cb0ae0f0 0x5578cb0b6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.014 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.005+0000 7fcce37fe700 0 client.16221 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:22:30.014 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.005+0000 7fcd16a66b80 1 -- 172.21.15.77:0/4178820574 >> 172.21.15.77:0/4178820574 conn(0x5578cb00c0d0 msgr2=0x5578cb0a70b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:30.014 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.005+0000 7fcd16a66b80 1 -- 172.21.15.77:0/4178820574 shutdown_connections 2023-12-01T01:22:30.015 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.005+0000 7fcd16a66b80 1 -- 172.21.15.77:0/4178820574 wait complete. 2023-12-01T01:22:30.015 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.005+0000 7fcd16a66b80 20 client.16221 trim_cache size 0 max 16384 2023-12-01T01:22:30.024 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.021+0000 7fcd16a66b80 1 Processor -- start 2023-12-01T01:22:30.025 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.021+0000 7fcd16a66b80 1 -- start start 2023-12-01T01:22:30.025 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.022+0000 7fcd16a66b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578cb0ad3f0 0x5578cb0ace60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.025 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.022+0000 7fcd16a66b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578cb0ae0f0 0x5578cb0bbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.025 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.022+0000 7fcd16a66b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578cb0aea70 0x5578cb0b6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.025 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.022+0000 7fcd16a66b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578cb0bfb30 con 0x5578cb0aea70 2023-12-01T01:22:30.025 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.022+0000 7fcd16a66b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578cb0a8810 con 0x5578cb0ae0f0 2023-12-01T01:22:30.025 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.022+0000 7fcd16a66b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578cb09fc10 con 0x5578cb0ad3f0 2023-12-01T01:22:30.025 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.022+0000 7fcd08e73700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578cb0aea70 0x5578cb0b6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.025 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.022+0000 7fcd08e73700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578cb0aea70 0x5578cb0b6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32888/0 (socket says 172.21.15.77:32888) 2023-12-01T01:22:30.025 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.022+0000 7fcd08e73700 1 -- 172.21.15.77:0/4082033874 learned_addr learned my addr 172.21.15.77:0/4082033874 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:22:30.025 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.022+0000 7fcd037fe700 1 --2- 172.21.15.77:0/4082033874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578cb0ae0f0 0x5578cb0bbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.025 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.022+0000 7fcd03fff700 1 --2- 172.21.15.77:0/4082033874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578cb0ad3f0 0x5578cb0ace60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.025 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.022+0000 7fcd08e73700 1 -- 172.21.15.77:0/4082033874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578cb0ad3f0 msgr2=0x5578cb0ace60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:30.025 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.022+0000 7fcd08e73700 1 --2- 172.21.15.77:0/4082033874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578cb0ad3f0 0x5578cb0ace60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.022+0000 7fcd08e73700 1 -- 172.21.15.77:0/4082033874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578cb0ae0f0 msgr2=0x5578cb0bbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:30.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.022+0000 7fcd08e73700 1 --2- 172.21.15.77:0/4082033874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578cb0ae0f0 0x5578cb0bbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.022+0000 7fcd08e73700 1 -- 172.21.15.77:0/4082033874 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578cae52250 con 0x5578cb0aea70 2023-12-01T01:22:30.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.022+0000 7fcd08e73700 1 --2- 172.21.15.77:0/4082033874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578cb0aea70 0x5578cb0b6950 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fccfc000f80 tx=0x7fccfc009af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0d7ac1c5b6cc644 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:30.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.022+0000 7fccfaffd700 1 -- 172.21.15.77:0/4082033874 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc015990 con 0x5578cb0aea70 2023-12-01T01:22:30.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.023+0000 7fccfaffd700 1 -- 172.21.15.77:0/4082033874 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccfc00f040 con 0x5578cb0aea70 2023-12-01T01:22:30.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.023+0000 7fccfaffd700 1 -- 172.21.15.77:0/4082033874 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc012d10 con 0x5578cb0aea70 2023-12-01T01:22:30.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.023+0000 7fcd16a66b80 1 -- 172.21.15.77:0/4082033874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578cb0aea70 msgr2=0x5578cb0b6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:30.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.023+0000 7fcd16a66b80 1 --2- 172.21.15.77:0/4082033874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578cb0aea70 0x5578cb0b6950 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fccfc000f80 tx=0x7fccfc009af0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.023+0000 7fcd16a66b80 1 -- 172.21.15.77:0/4082033874 shutdown_connections 2023-12-01T01:22:30.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.023+0000 7fcd16a66b80 1 --2- 172.21.15.77:0/4082033874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578cb0aea70 0x5578cb0b6950 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.023+0000 7fcd16a66b80 1 --2- 172.21.15.77:0/4082033874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578cb0ad3f0 0x5578cb0ace60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.026 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.023+0000 7fcd16a66b80 1 --2- 172.21.15.77:0/4082033874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578cb0ae0f0 0x5578cb0bbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.027 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.023+0000 7fcd16a66b80 1 -- 172.21.15.77:0/4082033874 >> 172.21.15.77:0/4082033874 conn(0x5578cb00c0d0 msgr2=0x5578cb0b1360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:30.027 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.023+0000 7fcd16a66b80 1 -- 172.21.15.77:0/4082033874 shutdown_connections 2023-12-01T01:22:30.027 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.023+0000 7fcd16a66b80 1 -- 172.21.15.77:0/4082033874 wait complete. 2023-12-01T01:22:30.027 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.023+0000 7fcd16a66b80 1 -- start start 2023-12-01T01:22:30.031 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.024+0000 7fcd16a66b80 1 Processor -- start 2023-12-01T01:22:30.031 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.024+0000 7fcd16a66b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578cb0ad3f0 0x5578cb0ba980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.031 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.024+0000 7fcd16a66b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578cb0ae0f0 0x5578cb0baec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.031 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.024+0000 7fcd16a66b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578cb0aea70 0x5578cb114200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.031 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.024+0000 7fcd16a66b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578cb0bb6e0 con 0x5578cb0aea70 2023-12-01T01:22:30.031 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.024+0000 7fcd16a66b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578cb0bb850 con 0x5578cb0ad3f0 2023-12-01T01:22:30.032 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.024+0000 7fcd16a66b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578cb1147c0 con 0x5578cb0ae0f0 2023-12-01T01:22:30.032 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.024+0000 7fcd16a66b80 10 client.0 _ll_register_callbacks cb 0x7fccfc007500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:22:30.032 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.024+0000 7fcd08e73700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578cb0aea70 0x5578cb114200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.032 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.024+0000 7fcd08e73700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578cb0aea70 0x5578cb114200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32894/0 (socket says 172.21.15.77:32894) 2023-12-01T01:22:30.032 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.024+0000 7fcd08e73700 1 -- 172.21.15.77:0/2760745743 learned_addr learned my addr 172.21.15.77:0/2760745743 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:22:30.032 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.024+0000 7fcd03fff700 1 --2- 172.21.15.77:0/2760745743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578cb0ad3f0 0x5578cb0ba980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.032 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.024+0000 7fcd037fe700 1 --2- 172.21.15.77:0/2760745743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578cb0ae0f0 0x5578cb0baec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.032 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.024+0000 7fcd08e73700 1 -- 172.21.15.77:0/2760745743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578cb0ae0f0 msgr2=0x5578cb0baec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:30.032 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.024+0000 7fcd08e73700 1 --2- 172.21.15.77:0/2760745743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578cb0ae0f0 0x5578cb0baec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.032 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.024+0000 7fcd08e73700 1 -- 172.21.15.77:0/2760745743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578cb0ad3f0 msgr2=0x5578cb0ba980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:30.032 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.024+0000 7fcd08e73700 1 --2- 172.21.15.77:0/2760745743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578cb0ad3f0 0x5578cb0ba980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.032 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.024+0000 7fcd08e73700 1 -- 172.21.15.77:0/2760745743 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5578cae14c60 con 0x5578cb0aea70 2023-12-01T01:22:30.032 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.024+0000 7fcd03fff700 1 --2- 172.21.15.77:0/2760745743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578cb0ad3f0 0x5578cb0ba980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:22:30.032 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.024+0000 7fcd037fe700 1 --2- 172.21.15.77:0/2760745743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578cb0ae0f0 0x5578cb0baec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:22:30.033 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.025+0000 7fcd08e73700 1 --2- 172.21.15.77:0/2760745743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578cb0aea70 0x5578cb114200 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fccfc0093a0 tx=0x7fccfc0097e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1b4dbe04acee867 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:30.033 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.025+0000 7fcce37fe700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:22:30.033 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.025+0000 7fcce37fe700 1 -- 172.21.15.77:0/2760745743 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc004ea0 con 0x5578cb0aea70 2023-12-01T01:22:30.033 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.025+0000 7fcce37fe700 1 -- 172.21.15.77:0/2760745743 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fccfc012ab0 con 0x5578cb0aea70 2023-12-01T01:22:30.033 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.025+0000 7fcd16a66b80 1 -- 172.21.15.77:0/2760745743 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fccf400fb50 con 0x5578cb0aea70 2023-12-01T01:22:30.033 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.025+0000 7fcce37fe700 1 -- 172.21.15.77:0/2760745743 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccfc0154d0 con 0x5578cb0aea70 2023-12-01T01:22:30.033 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.025+0000 7fcd16a66b80 10 client.16227 fetch_fsmap learned FSMap version 41 2023-12-01T01:22:30.033 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.025+0000 7fcd16a66b80 1 -- 172.21.15.77:0/2760745743 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5578cae2a2b0 con 0x5578cb0aea70 2023-12-01T01:22:30.033 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.025+0000 7fcce37fe700 1 -- 172.21.15.77:0/2760745743 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccf400fb50 con 0x5578cb0aea70 2023-12-01T01:22:30.033 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.025+0000 7fcce37fe700 1 -- 172.21.15.77:0/2760745743 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 41) v1 ==== 7514+0+0 (secure 0 0 0) 0x7fccfc007c70 con 0x5578cb0aea70 2023-12-01T01:22:30.033 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.025+0000 7fcd16a66b80 10 client.16227 fetch_fsmap finished waiting for FSMap version 41 2023-12-01T01:22:30.033 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.025+0000 7fcd16a66b80 10 client.16227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-01T01:22:30.033 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.025+0000 7fcd16a66b80 20 client.16227 populate_metadata read hostname 'smithi077' 2023-12-01T01:22:30.033 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.025+0000 7fcd16a66b80 1 --2- 172.21.15.77:0/2760745743 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5578cb115cc0 0x5578cb1180e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.034 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.025+0000 7fcd16a66b80 4 client.16227 mds_command: new command op to 24500 tid=0[{"prefix": "scrub status"}] 2023-12-01T01:22:30.034 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.025+0000 7fcd16a66b80 1 -- 172.21.15.77:0/2760745743 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5578caff6d40 con 0x5578cb115cc0 2023-12-01T01:22:30.034 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.026+0000 7fcd03fff700 1 --2- 172.21.15.77:0/2760745743 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5578cb115cc0 0x5578cb1180e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.034 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.026+0000 7fcce37fe700 10 client.16227 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:22:30.034 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.026+0000 7fcd03fff700 1 --2- 172.21.15.77:0/2760745743 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5578cb115cc0 0x5578cb1180e0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a40212e77848a2d7 server_cookie=b29ebe515d497c78 in_seq=0 out_seq=0 2023-12-01T01:22:30.034 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.026+0000 7fcce37fe700 1 -- 172.21.15.77:0/2760745743 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5578caff6d40 con 0x5578cb115cc0 2023-12-01T01:22:30.034 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.026+0000 7fcce37fe700 10 client.16227 handle_command_reply: tid=0 2023-12-01T01:22:30.034 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.026+0000 7fcd16a66b80 1 client.16227 shutdown 2023-12-01T01:22:30.034 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.026+0000 7fcd16a66b80 1 -- 172.21.15.77:0/2760745743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578cb0aea70 msgr2=0x5578cb114200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:30.034 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.026+0000 7fcd16a66b80 1 --2- 172.21.15.77:0/2760745743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578cb0aea70 0x5578cb114200 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fccfc0093a0 tx=0x7fccfc0097e0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.034 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.026+0000 7fcd16a66b80 1 -- 172.21.15.77:0/2760745743 shutdown_connections 2023-12-01T01:22:30.035 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.026+0000 7fcd16a66b80 1 --2- 172.21.15.77:0/2760745743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5578cb0aea70 0x5578cb114200 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.035 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.026+0000 7fcd16a66b80 1 --2- 172.21.15.77:0/2760745743 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x5578cb115cc0 0x5578cb1180e0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.035 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.026+0000 7fcd16a66b80 1 --2- 172.21.15.77:0/2760745743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5578cb0ad3f0 0x5578cb0ba980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.035 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.026+0000 7fcce37fe700 0 client.16227 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:22:30.035 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.026+0000 7fcd16a66b80 1 --2- 172.21.15.77:0/2760745743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5578cb0ae0f0 0x5578cb0baec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.035 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.026+0000 7fcd16a66b80 1 -- 172.21.15.77:0/2760745743 >> 172.21.15.77:0/2760745743 conn(0x5578cb00c0d0 msgr2=0x5578cb0b68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:30.035 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.026+0000 7fcd16a66b80 1 -- 172.21.15.77:0/2760745743 shutdown_connections 2023-12-01T01:22:30.035 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.026+0000 7fcd16a66b80 1 -- 172.21.15.77:0/2760745743 wait complete. 2023-12-01T01:22:30.035 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.027+0000 7fcd16a66b80 20 client.16227 trim_cache size 0 max 16384 2023-12-01T01:22:30.035 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2023-12-01T01:22:30.035 INFO:teuthology.orchestra.run.smithi077.stdout: "status": "no active scrubs running", 2023-12-01T01:22:30.036 INFO:teuthology.orchestra.run.smithi077.stdout: "scrubs": {} 2023-12-01T01:22:30.036 INFO:teuthology.orchestra.run.smithi077.stdout:} 2023-12-01T01:22:30.036 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.027+0000 7fcd0a8d6700 1 -- 172.21.15.77:0/2304815630 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fccf00a0520 msgr2=0x7fccf00a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:30.036 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.027+0000 7fcd0a8d6700 1 --2- 172.21.15.77:0/2304815630 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fccf00a0520 0x7fccf00a29e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fccf400ba60 tx=0x7fccf4005d50 comp rx=0 tx=0).stop 2023-12-01T01:22:30.037 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.027+0000 7fcd0a8d6700 1 -- 172.21.15.77:0/2304815630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd040ffc80 msgr2=0x7fcd04068ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:30.037 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.027+0000 7fcd0a8d6700 1 --2- 172.21.15.77:0/2304815630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd040ffc80 0x7fcd04068ac0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fccec009f50 tx=0x7fccec0057d0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.037 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.027+0000 7fcd0a8d6700 1 -- 172.21.15.77:0/2304815630 shutdown_connections 2023-12-01T01:22:30.037 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.027+0000 7fcd0a8d6700 1 --2- 172.21.15.77:0/2304815630 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fccf00a0520 0x7fccf00a29e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.037 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.027+0000 7fcd0a8d6700 1 --2- 172.21.15.77:0/2304815630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd04101810 0x7fcd0419dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.037 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.027+0000 7fcd0a8d6700 1 --2- 172.21.15.77:0/2304815630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcd040ffc80 0x7fcd04068ac0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.037 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.027+0000 7fcd0a8d6700 1 --2- 172.21.15.77:0/2304815630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd04100e70 0x7fcd0419b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.037 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.027+0000 7fcd0a8d6700 1 -- 172.21.15.77:0/2304815630 >> 172.21.15.77:0/2304815630 conn(0x7fcd040fb1f0 msgr2=0x7fcd04104350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:30.037 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.027+0000 7fcd0a8d6700 1 -- 172.21.15.77:0/2304815630 shutdown_connections 2023-12-01T01:22:30.037 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.027+0000 7fcd0a8d6700 1 -- 172.21.15.77:0/2304815630 wait complete. 2023-12-01T01:22:30.040 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-12-01T01:22:30.041 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:22:30.205 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.201+0000 7f4d18887700 1 -- 172.21.15.77:0/377282065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4d14104160 msgr2=0x7f4d14106550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:30.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.201+0000 7f4d18887700 1 --2- 172.21.15.77:0/377282065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4d14104160 0x7f4d14106550 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f4d04009990 tx=0x7f4d04009ca0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.201+0000 7f4d18887700 1 -- 172.21.15.77:0/377282065 shutdown_connections 2023-12-01T01:22:30.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.201+0000 7f4d18887700 1 --2- 172.21.15.77:0/377282065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d14106a90 0x7f4d14108f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.201+0000 7f4d18887700 1 --2- 172.21.15.77:0/377282065 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d14101830 0x7f4d14103c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.201+0000 7f4d18887700 1 --2- 172.21.15.77:0/377282065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4d14104160 0x7f4d14106550 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.201+0000 7f4d18887700 1 -- 172.21.15.77:0/377282065 >> 172.21.15.77:0/377282065 conn(0x7f4d140fb220 msgr2=0x7f4d140fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:30.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.201+0000 7f4d18887700 1 -- 172.21.15.77:0/377282065 shutdown_connections 2023-12-01T01:22:30.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.201+0000 7f4d18887700 1 -- 172.21.15.77:0/377282065 wait complete. 2023-12-01T01:22:30.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.202+0000 7f4d18887700 1 Processor -- start 2023-12-01T01:22:30.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.202+0000 7f4d18887700 1 -- start start 2023-12-01T01:22:30.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.202+0000 7f4d18887700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d14101830 0x7f4d141011a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.202+0000 7f4d18887700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4d14104160 0x7f4d140ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.202+0000 7f4d18887700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d14106a90 0x7f4d140ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.202+0000 7f4d18887700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d141004c0 con 0x7f4d14104160 2023-12-01T01:22:30.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.202+0000 7f4d18887700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d14100630 con 0x7f4d14106a90 2023-12-01T01:22:30.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.202+0000 7f4d18887700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d141a0a10 con 0x7f4d14101830 2023-12-01T01:22:30.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.202+0000 7f4d11d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4d14104160 0x7f4d140ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.202+0000 7f4d11d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4d14104160 0x7f4d140ff7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32906/0 (socket says 172.21.15.77:32906) 2023-12-01T01:22:30.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.202+0000 7f4d11d9b700 1 -- 172.21.15.77:0/3804314151 learned_addr learned my addr 172.21.15.77:0/3804314151 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:22:30.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.203+0000 7f4d1259c700 1 --2- 172.21.15.77:0/3804314151 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d14101830 0x7f4d141011a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.203+0000 7f4d12d9d700 1 --2- 172.21.15.77:0/3804314151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d14106a90 0x7f4d140ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.203+0000 7f4d11d9b700 1 -- 172.21.15.77:0/3804314151 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d14101830 msgr2=0x7f4d141011a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:30.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.203+0000 7f4d11d9b700 1 --2- 172.21.15.77:0/3804314151 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d14101830 0x7f4d141011a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.203+0000 7f4d11d9b700 1 -- 172.21.15.77:0/3804314151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d14106a90 msgr2=0x7f4d140ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:30.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.203+0000 7f4d11d9b700 1 --2- 172.21.15.77:0/3804314151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d14106a90 0x7f4d140ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.203+0000 7f4d11d9b700 1 -- 172.21.15.77:0/3804314151 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d040095f0 con 0x7f4d14104160 2023-12-01T01:22:30.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.203+0000 7f4d11d9b700 1 --2- 172.21.15.77:0/3804314151 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4d14104160 0x7f4d140ff7e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f4d04000c00 tx=0x7f4d0400f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:30.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.203+0000 7f4d0b7fe700 1 -- 172.21.15.77:0/3804314151 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0401d070 con 0x7f4d14104160 2023-12-01T01:22:30.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.203+0000 7f4d0b7fe700 1 -- 172.21.15.77:0/3804314151 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d0400fb50 con 0x7f4d14104160 2023-12-01T01:22:30.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.203+0000 7f4d18887700 1 -- 172.21.15.77:0/3804314151 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d141a0c90 con 0x7f4d14104160 2023-12-01T01:22:30.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.203+0000 7f4d0b7fe700 1 -- 172.21.15.77:0/3804314151 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d04017650 con 0x7f4d14104160 2023-12-01T01:22:30.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.203+0000 7f4d18887700 1 -- 172.21.15.77:0/3804314151 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d141a1140 con 0x7f4d14104160 2023-12-01T01:22:30.208 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.204+0000 7f4d18887700 1 -- 172.21.15.77:0/3804314151 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d140fcda0 con 0x7f4d14104160 2023-12-01T01:22:30.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.205+0000 7f4d0b7fe700 1 -- 172.21.15.77:0/3804314151 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f4d0400fcc0 con 0x7f4d14104160 2023-12-01T01:22:30.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.206+0000 7f4d0b7fe700 1 --2- 172.21.15.77:0/3804314151 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f4d000a0570 0x7f4d000a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.206+0000 7f4d1259c700 1 --2- 172.21.15.77:0/3804314151 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f4d000a0570 0x7f4d000a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.206+0000 7f4d0b7fe700 1 -- 172.21.15.77:0/3804314151 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f4d040ccc40 con 0x7f4d14104160 2023-12-01T01:22:30.209 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.206+0000 7f4d1259c700 1 --2- 172.21.15.77:0/3804314151 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f4d000a0570 0x7f4d000a2a30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc005950 tx=0x7f4cfc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:30.211 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.210+0000 7f4d0b7fe700 1 -- 172.21.15.77:0/3804314151 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4d04096910 con 0x7f4d14104160 2023-12-01T01:22:30.449 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.448+0000 7f4d18887700 1 -- 172.21.15.77:0/3804314151 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d1402cc70 con 0x7f4d14104160 2023-12-01T01:22:30.450 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.449+0000 7f4d0b7fe700 1 -- 172.21.15.77:0/3804314151 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+10866 (secure 0 0 0) 0x7f4d04099f30 con 0x7f4d14104160 2023-12-01T01:22:30.450 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:22:30.450 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1435145620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1435145620},{"type":"v1","addr":"172.21.15.77:6835","nonce":1435145620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/841838908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":841838908},{"type":"v1","addr":"172.21.15.77:6837","nonce":841838908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-01T01:16:25.345276+0000","modified":"2023-12-01T01:22:25.411862+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24496,"mds_2":24475,"mds_3":24473,"mds_4":14649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/3726913446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3726913446},{"type":"v1","addr":"172.21.15.77:6839","nonce":3726913446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14649":{"gid":14649,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":5,"addr":"172.21.15.77:6841/3023184965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3023184965},{"type":"v1","addr":"172.21.15.77:6841","nonce":3023184965}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/2249638445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2249638445},{"type":"v1","addr":"172.21.15.137:6833","nonce":2249638445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6833/1201599774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1201599774},{"type":"v1","addr":"172.21.15.203:6833","nonce":1201599774}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3935354878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3935354878},{"type":"v1","addr":"172.21.15.137:6835","nonce":3935354878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3237538176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3237538176},{"type":"v1","addr":"172.21.15.203:6835","nonce":3237538176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3518881867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3518881867},{"type":"v1","addr":"172.21.15.137:6837","nonce":3518881867}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":46,"addr":"172.21.15.203:6837/2887080977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2887080977},{"type":"v1","addr":"172.21.15.203:6837","nonce":2887080977}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":44,"addr":"172.21.15.203:6839/2888221613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2888221613},{"type":"v1","addr":"172.21.15.203:6839","nonce":2888221613}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/1099966660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1099966660},{"type":"v1","addr":"172.21.15.137:6839","nonce":1099966660}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-01T01:22:30.452 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.451+0000 7f4d18887700 1 -- 172.21.15.77:0/3804314151 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f4d000a0570 msgr2=0x7f4d000a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:30.453 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.451+0000 7f4d18887700 1 --2- 172.21.15.77:0/3804314151 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f4d000a0570 0x7f4d000a2a30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc005950 tx=0x7f4cfc00b410 comp rx=0 tx=0).stop 2023-12-01T01:22:30.453 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.451+0000 7f4d18887700 1 -- 172.21.15.77:0/3804314151 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4d14104160 msgr2=0x7f4d140ff7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:30.453 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.451+0000 7f4d18887700 1 --2- 172.21.15.77:0/3804314151 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4d14104160 0x7f4d140ff7e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f4d04000c00 tx=0x7f4d0400f710 comp rx=0 tx=0).stop 2023-12-01T01:22:30.453 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.451+0000 7f4d18887700 1 -- 172.21.15.77:0/3804314151 shutdown_connections 2023-12-01T01:22:30.453 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.451+0000 7f4d18887700 1 --2- 172.21.15.77:0/3804314151 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f4d000a0570 0x7f4d000a2a30 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.453 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.451+0000 7f4d18887700 1 --2- 172.21.15.77:0/3804314151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d14106a90 0x7f4d140ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.453 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.451+0000 7f4d18887700 1 --2- 172.21.15.77:0/3804314151 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d14101830 0x7f4d141011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.453 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.451+0000 7f4d18887700 1 --2- 172.21.15.77:0/3804314151 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4d14104160 0x7f4d140ff7e0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.453 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.451+0000 7f4d18887700 1 -- 172.21.15.77:0/3804314151 >> 172.21.15.77:0/3804314151 conn(0x7f4d140fb220 msgr2=0x7f4d141024a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:30.453 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.451+0000 7f4d18887700 1 -- 172.21.15.77:0/3804314151 shutdown_connections 2023-12-01T01:22:30.453 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.451+0000 7f4d18887700 1 -- 172.21.15.77:0/3804314151 wait complete. 2023-12-01T01:22:30.454 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 41 2023-12-01T01:22:30.465 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-12-01T01:22:30.591 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.587+0000 7fd6c4ffc700 1 -- 172.21.15.77:0/1706766334 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6c0102930 msgr2=0x7fd6c01091e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:30.591 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.587+0000 7fd6c4ffc700 1 --2- 172.21.15.77:0/1706766334 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6c0102930 0x7fd6c01091e0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd6b800b0d0 tx=0x7fd6b800b3e0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.591 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.587+0000 7fd6c4ffc700 1 -- 172.21.15.77:0/1706766334 shutdown_connections 2023-12-01T01:22:30.591 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.587+0000 7fd6c4ffc700 1 --2- 172.21.15.77:0/1706766334 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6c0102930 0x7fd6c01091e0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.591 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.587+0000 7fd6c4ffc700 1 --2- 172.21.15.77:0/1706766334 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6c0100da0 0x7fd6c01011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.591 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.587+0000 7fd6c4ffc700 1 --2- 172.21.15.77:0/1706766334 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6c0101f90 0x7fd6c01023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.591 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.587+0000 7fd6c4ffc700 1 -- 172.21.15.77:0/1706766334 >> 172.21.15.77:0/1706766334 conn(0x7fd6c00fc310 msgr2=0x7fd6c00fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:30.591 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.588+0000 7fd6c4ffc700 1 -- 172.21.15.77:0/1706766334 shutdown_connections 2023-12-01T01:22:30.592 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.588+0000 7fd6c4ffc700 1 -- 172.21.15.77:0/1706766334 wait complete. 2023-12-01T01:22:30.592 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.588+0000 7fd6c4ffc700 1 Processor -- start 2023-12-01T01:22:30.592 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.588+0000 7fd6c4ffc700 1 -- start start 2023-12-01T01:22:30.592 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.588+0000 7fd6c4ffc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6c0100da0 0x7fd6c0196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.592 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.588+0000 7fd6c4ffc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6c0101f90 0x7fd6c0197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.592 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.588+0000 7fd6c4ffc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6c0102930 0x7fd6c019e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.592 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.588+0000 7fd6c4ffc700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c019e6c0 con 0x7fd6c0101f90 2023-12-01T01:22:30.592 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.588+0000 7fd6c4ffc700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c019e800 con 0x7fd6c0102930 2023-12-01T01:22:30.592 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.588+0000 7fd6c4ffc700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c019e990 con 0x7fd6c0100da0 2023-12-01T01:22:30.592 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.589+0000 7fd6bdd9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6c0101f90 0x7fd6c0197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.592 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.589+0000 7fd6bdd9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6c0101f90 0x7fd6c0197080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32918/0 (socket says 172.21.15.77:32918) 2023-12-01T01:22:30.592 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.589+0000 7fd6bdd9b700 1 -- 172.21.15.77:0/911539046 learned_addr learned my addr 172.21.15.77:0/911539046 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:22:30.592 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.589+0000 7fd6be59c700 1 --2- 172.21.15.77:0/911539046 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6c0100da0 0x7fd6c0196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.592 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.589+0000 7fd6bed9d700 1 --2- 172.21.15.77:0/911539046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6c0102930 0x7fd6c019e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.592 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.589+0000 7fd6bdd9b700 1 -- 172.21.15.77:0/911539046 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6c0100da0 msgr2=0x7fd6c0196b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:30.593 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.589+0000 7fd6bdd9b700 1 --2- 172.21.15.77:0/911539046 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6c0100da0 0x7fd6c0196b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.593 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.589+0000 7fd6bdd9b700 1 -- 172.21.15.77:0/911539046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6c0102930 msgr2=0x7fd6c019e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:30.593 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.589+0000 7fd6bdd9b700 1 --2- 172.21.15.77:0/911539046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6c0102930 0x7fd6c019e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.593 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.589+0000 7fd6bdd9b700 1 -- 172.21.15.77:0/911539046 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b8009d00 con 0x7fd6c0101f90 2023-12-01T01:22:30.593 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.589+0000 7fd6bed9d700 1 --2- 172.21.15.77:0/911539046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6c0102930 0x7fd6c019e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:22:30.593 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.589+0000 7fd6bdd9b700 1 --2- 172.21.15.77:0/911539046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6c0101f90 0x7fd6c0197080 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd6b400d950 tx=0x7fd6b400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:30.593 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.589+0000 7fd6af7fe700 1 -- 172.21.15.77:0/911539046 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b4009940 con 0x7fd6c0101f90 2023-12-01T01:22:30.593 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.589+0000 7fd6af7fe700 1 -- 172.21.15.77:0/911539046 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6b4010460 con 0x7fd6c0101f90 2023-12-01T01:22:30.593 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.589+0000 7fd6c4ffc700 1 -- 172.21.15.77:0/911539046 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6c019ec70 con 0x7fd6c0101f90 2023-12-01T01:22:30.593 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.589+0000 7fd6af7fe700 1 -- 172.21.15.77:0/911539046 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b400f620 con 0x7fd6c0101f90 2023-12-01T01:22:30.593 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.589+0000 7fd6c4ffc700 1 -- 172.21.15.77:0/911539046 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6c019f180 con 0x7fd6c0101f90 2023-12-01T01:22:30.600 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.591+0000 7fd6af7fe700 1 -- 172.21.15.77:0/911539046 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fd6b40108d0 con 0x7fd6c0101f90 2023-12-01T01:22:30.601 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.592+0000 7fd6af7fe700 1 --2- 172.21.15.77:0/911539046 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd6a80a0520 0x7fd6a80a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.601 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.592+0000 7fd6be59c700 1 --2- 172.21.15.77:0/911539046 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd6a80a0520 0x7fd6a80a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.601 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.592+0000 7fd6af7fe700 1 -- 172.21.15.77:0/911539046 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fd6b40cb760 con 0x7fd6c0101f90 2023-12-01T01:22:30.601 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.592+0000 7fd6be59c700 1 --2- 172.21.15.77:0/911539046 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd6a80a0520 0x7fd6a80a29e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0101df0 tx=0x7fd6b000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:30.601 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.593+0000 7fd6d118cb80 1 Processor -- start 2023-12-01T01:22:30.601 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.593+0000 7fd6d118cb80 1 -- start start 2023-12-01T01:22:30.601 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.593+0000 7fd6d118cb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579e174e0f0 0x5579e174e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.601 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.593+0000 7fd6d118cb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e174ea70 0x5579e174ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.601 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.593+0000 7fd6d118cb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579e174d3f0 0x5579e174d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.601 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.593+0000 7fd6d118cb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579e16a4a10 con 0x5579e174ea70 2023-12-01T01:22:30.601 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.593+0000 7fd6d118cb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579e16a23d0 con 0x5579e174d3f0 2023-12-01T01:22:30.601 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.593+0000 7fd6d118cb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579e169dcc0 con 0x5579e174e0f0 2023-12-01T01:22:30.601 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.593+0000 7fd6bdd9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e174ea70 0x5579e174ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.602 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.593+0000 7fd6bdd9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e174ea70 0x5579e174ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32926/0 (socket says 172.21.15.77:32926) 2023-12-01T01:22:30.602 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.593+0000 7fd6bdd9b700 1 -- 172.21.15.77:0/1167336676 learned_addr learned my addr 172.21.15.77:0/1167336676 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:22:30.602 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.593+0000 7fd6be59c700 1 --2- 172.21.15.77:0/1167336676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579e174e0f0 0x5579e174e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.602 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.593+0000 7fd6bed9d700 1 --2- 172.21.15.77:0/1167336676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579e174d3f0 0x5579e174d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.602 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.594+0000 7fd6bdd9b700 1 -- 172.21.15.77:0/1167336676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579e174e0f0 msgr2=0x5579e174e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:30.602 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.594+0000 7fd6bdd9b700 1 --2- 172.21.15.77:0/1167336676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579e174e0f0 0x5579e174e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.602 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.594+0000 7fd6bdd9b700 1 -- 172.21.15.77:0/1167336676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579e174d3f0 msgr2=0x5579e174d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:30.602 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.594+0000 7fd6bdd9b700 1 --2- 172.21.15.77:0/1167336676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579e174d3f0 0x5579e174d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.602 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.594+0000 7fd6bdd9b700 1 -- 172.21.15.77:0/1167336676 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579e14cbde0 con 0x5579e174ea70 2023-12-01T01:22:30.602 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.594+0000 7fd6bed9d700 1 --2- 172.21.15.77:0/1167336676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579e174d3f0 0x5579e174d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:22:30.602 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.594+0000 7fd6bdd9b700 1 --2- 172.21.15.77:0/1167336676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e174ea70 0x5579e174ce60 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fd6b40a0ba0 tx=0x7fd6b4098ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dcd7e1a04f114c server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:30.602 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.594+0000 7fd6acff9700 1 -- 172.21.15.77:0/1167336676 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b409ce90 con 0x5579e174ea70 2023-12-01T01:22:30.602 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.594+0000 7fd6acff9700 1 -- 172.21.15.77:0/1167336676 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6b40a10d0 con 0x5579e174ea70 2023-12-01T01:22:30.602 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.594+0000 7fd6acff9700 1 -- 172.21.15.77:0/1167336676 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b4098820 con 0x5579e174ea70 2023-12-01T01:22:30.603 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.594+0000 7fd6d118cb80 1 -- 172.21.15.77:0/1167336676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e174ea70 msgr2=0x5579e174ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:30.603 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.595+0000 7fd6d118cb80 1 --2- 172.21.15.77:0/1167336676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e174ea70 0x5579e174ce60 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fd6b40a0ba0 tx=0x7fd6b4098ac0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.603 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.595+0000 7fd6d118cb80 1 -- 172.21.15.77:0/1167336676 shutdown_connections 2023-12-01T01:22:30.603 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.595+0000 7fd6d118cb80 1 --2- 172.21.15.77:0/1167336676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579e174d3f0 0x5579e174d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.603 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.595+0000 7fd6d118cb80 1 --2- 172.21.15.77:0/1167336676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579e174e0f0 0x5579e174e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.603 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.595+0000 7fd6d118cb80 1 --2- 172.21.15.77:0/1167336676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e174ea70 0x5579e174ce60 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.603 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.595+0000 7fd6d118cb80 1 -- 172.21.15.77:0/1167336676 >> 172.21.15.77:0/1167336676 conn(0x5579e16ac0d0 msgr2=0x5579e1747940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:30.603 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.595+0000 7fd6d118cb80 1 -- 172.21.15.77:0/1167336676 shutdown_connections 2023-12-01T01:22:30.603 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.595+0000 7fd6d118cb80 1 -- 172.21.15.77:0/1167336676 wait complete. 2023-12-01T01:22:30.603 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.595+0000 7fd6d118cb80 1 -- start start 2023-12-01T01:22:30.603 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.595+0000 7fd6d118cb80 1 Processor -- start 2023-12-01T01:22:30.603 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.595+0000 7fd6d118cb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e174d3f0 0x5579e1756980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.603 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.595+0000 7fd6d118cb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579e174e0f0 0x5579e1756ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.603 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.595+0000 7fd6d118cb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579e174ea70 0x5579e175df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.604 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.595+0000 7fd6d118cb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579e163a510 con 0x5579e174d3f0 2023-12-01T01:22:30.604 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.595+0000 7fd6d118cb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579e163a1b0 con 0x5579e174e0f0 2023-12-01T01:22:30.604 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.595+0000 7fd6d118cb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579e1521920 con 0x5579e174ea70 2023-12-01T01:22:30.604 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.595+0000 7fd6d118cb80 10 client.0 _ll_register_callbacks cb 0x5579e1587a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:22:30.604 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.595+0000 7fd6be59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e174d3f0 0x5579e1756980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.604 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.595+0000 7fd6be59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e174d3f0 0x5579e1756980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32930/0 (socket says 172.21.15.77:32930) 2023-12-01T01:22:30.604 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.595+0000 7fd6be59c700 1 -- 172.21.15.77:0/1434193670 learned_addr learned my addr 172.21.15.77:0/1434193670 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:22:30.604 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.595+0000 7fd6bed9d700 1 --2- 172.21.15.77:0/1434193670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579e174ea70 0x5579e175df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.604 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.595+0000 7fd6bdd9b700 1 --2- 172.21.15.77:0/1434193670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579e174e0f0 0x5579e1756ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.604 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.596+0000 7fd6be59c700 1 -- 172.21.15.77:0/1434193670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579e174ea70 msgr2=0x5579e175df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:30.604 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.596+0000 7fd6be59c700 1 --2- 172.21.15.77:0/1434193670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579e174ea70 0x5579e175df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.604 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.596+0000 7fd6be59c700 1 -- 172.21.15.77:0/1434193670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579e174e0f0 msgr2=0x5579e1756ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:30.604 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.596+0000 7fd6be59c700 1 --2- 172.21.15.77:0/1434193670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579e174e0f0 0x5579e1756ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.604 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.596+0000 7fd6be59c700 1 -- 172.21.15.77:0/1434193670 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6c019f180 con 0x5579e174d3f0 2023-12-01T01:22:30.605 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.596+0000 7fd6bdd9b700 1 --2- 172.21.15.77:0/1434193670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579e174e0f0 0x5579e1756ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:22:30.605 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.596+0000 7fd6be59c700 1 --2- 172.21.15.77:0/1434193670 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e174d3f0 0x5579e1756980 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fd6b00116c0 tx=0x7fd6b00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cb00c88a14f82bf server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:30.605 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.596+0000 7fd6a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:22:30.605 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.596+0000 7fd6a5ffb700 1 -- 172.21.15.77:0/1434193670 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0028670 con 0x5579e174d3f0 2023-12-01T01:22:30.605 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.596+0000 7fd6a5ffb700 1 -- 172.21.15.77:0/1434193670 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fd6b0017070 con 0x5579e174d3f0 2023-12-01T01:22:30.605 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.596+0000 7fd6d118cb80 1 -- 172.21.15.77:0/1434193670 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5579e16c8d10 con 0x5579e174d3f0 2023-12-01T01:22:30.605 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.596+0000 7fd6a5ffb700 1 -- 172.21.15.77:0/1434193670 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6b0011e40 con 0x5579e174d3f0 2023-12-01T01:22:30.605 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.596+0000 7fd6d118cb80 10 client.16245 fetch_fsmap learned FSMap version 41 2023-12-01T01:22:30.605 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.596+0000 7fd6d118cb80 1 -- 172.21.15.77:0/1434193670 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5579e14b4c60 con 0x5579e174d3f0 2023-12-01T01:22:30.605 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.596+0000 7fd6a5ffb700 1 -- 172.21.15.77:0/1434193670 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5579e16c8d10 con 0x5579e174d3f0 2023-12-01T01:22:30.605 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.596+0000 7fd6a5ffb700 1 -- 172.21.15.77:0/1434193670 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 41) v1 ==== 7514+0+0 (secure 0 0 0) 0x7fd6b001e550 con 0x5579e174d3f0 2023-12-01T01:22:30.605 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.596+0000 7fd6d118cb80 10 client.16245 fetch_fsmap finished waiting for FSMap version 41 2023-12-01T01:22:30.605 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.596+0000 7fd6d118cb80 10 client.16245 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2023-12-01T01:22:30.605 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.596+0000 7fd6d118cb80 20 client.16245 populate_metadata read hostname 'smithi077' 2023-12-01T01:22:30.606 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.596+0000 7fd6d118cb80 1 --2- 172.21.15.77:0/1434193670 >> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] conn(0x5579e1762810 0x5579e1764c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.606 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.597+0000 7fd6d118cb80 4 client.16245 mds_command: new command op to 14649 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:22:30.606 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.597+0000 7fd6d118cb80 1 -- 172.21.15.77:0/1434193670 --> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5579e171be10 con 0x5579e1762810 2023-12-01T01:22:30.606 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.597+0000 7fd6bdd9b700 1 --2- 172.21.15.77:0/1434193670 >> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] conn(0x5579e1762810 0x5579e1764c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.606 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.597+0000 7fd6bdd9b700 1 --2- 172.21.15.77:0/1434193670 >> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] conn(0x5579e1762810 0x5579e1764c30 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3cb8036db6f30961 server_cookie=b75bccdefa621ebb in_seq=0 out_seq=0 2023-12-01T01:22:30.606 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.597+0000 7fd6a5ffb700 10 client.16245 ms_handle_connect on v2:172.21.15.77:6840/3023184965 2023-12-01T01:22:30.606 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.597+0000 7fd6a5ffb700 1 -- 172.21.15.77:0/1434193670 <== mds.4 v2:172.21.15.77:6840/3023184965 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5579e171be10 con 0x5579e1762810 2023-12-01T01:22:30.606 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.597+0000 7fd6a5ffb700 10 client.16245 handle_command_reply: tid=0 2023-12-01T01:22:30.606 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.597+0000 7fd6d118cb80 1 client.16245 shutdown 2023-12-01T01:22:30.606 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.598+0000 7fd6d118cb80 1 -- 172.21.15.77:0/1434193670 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e174d3f0 msgr2=0x5579e1756980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:30.606 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.598+0000 7fd6d118cb80 1 --2- 172.21.15.77:0/1434193670 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e174d3f0 0x5579e1756980 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fd6b00116c0 tx=0x7fd6b00047c0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.606 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.598+0000 7fd6d118cb80 1 -- 172.21.15.77:0/1434193670 shutdown_connections 2023-12-01T01:22:30.606 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.598+0000 7fd6d118cb80 1 --2- 172.21.15.77:0/1434193670 >> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] conn(0x5579e1762810 0x5579e1764c30 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.606 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.598+0000 7fd6d118cb80 1 --2- 172.21.15.77:0/1434193670 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579e174ea70 0x5579e175df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.607 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.598+0000 7fd6d118cb80 1 --2- 172.21.15.77:0/1434193670 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e174d3f0 0x5579e1756980 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.607 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.598+0000 7fd6d118cb80 1 --2- 172.21.15.77:0/1434193670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579e174e0f0 0x5579e1756ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.607 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.598+0000 7fd6a5ffb700 0 client.16245 ms_handle_reset on v2:172.21.15.77:6840/3023184965 2023-12-01T01:22:30.607 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.598+0000 7fd6d118cb80 1 -- 172.21.15.77:0/1434193670 >> 172.21.15.77:0/1434193670 conn(0x5579e16ac0d0 msgr2=0x5579e17470b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:30.607 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.598+0000 7fd6d118cb80 1 -- 172.21.15.77:0/1434193670 shutdown_connections 2023-12-01T01:22:30.607 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.598+0000 7fd6d118cb80 1 -- 172.21.15.77:0/1434193670 wait complete. 2023-12-01T01:22:30.607 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.598+0000 7fd6d118cb80 20 client.16245 trim_cache size 0 max 16384 2023-12-01T01:22:30.616 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.610+0000 7fd6d118cb80 1 Processor -- start 2023-12-01T01:22:30.616 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.610+0000 7fd6d118cb80 1 -- start start 2023-12-01T01:22:30.617 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.610+0000 7fd6d118cb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579e174d3f0 0x5579e175beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.617 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.610+0000 7fd6d118cb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579e174e0f0 0x5579e175e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.617 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.610+0000 7fd6d118cb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e174ea70 0x5579e1751310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.617 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.610+0000 7fd6d118cb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579e173fb50 con 0x5579e174ea70 2023-12-01T01:22:30.617 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.610+0000 7fd6d118cb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579e17544e0 con 0x5579e174d3f0 2023-12-01T01:22:30.617 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.610+0000 7fd6d118cb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579e1762560 con 0x5579e174e0f0 2023-12-01T01:22:30.617 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.610+0000 7fd6bed9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e174ea70 0x5579e1751310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.617 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.610+0000 7fd6bed9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e174ea70 0x5579e1751310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32942/0 (socket says 172.21.15.77:32942) 2023-12-01T01:22:30.617 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.610+0000 7fd6bed9d700 1 -- 172.21.15.77:0/2813159050 learned_addr learned my addr 172.21.15.77:0/2813159050 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:22:30.617 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.611+0000 7fd6be59c700 1 --2- 172.21.15.77:0/2813159050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579e174d3f0 0x5579e175beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.617 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.611+0000 7fd6bdd9b700 1 --2- 172.21.15.77:0/2813159050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579e174e0f0 0x5579e175e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.617 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.611+0000 7fd6bed9d700 1 -- 172.21.15.77:0/2813159050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579e174e0f0 msgr2=0x5579e175e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:30.617 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.611+0000 7fd6bed9d700 1 --2- 172.21.15.77:0/2813159050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579e174e0f0 0x5579e175e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.617 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.611+0000 7fd6bed9d700 1 -- 172.21.15.77:0/2813159050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579e174d3f0 msgr2=0x5579e175beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:30.618 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.611+0000 7fd6bed9d700 1 --2- 172.21.15.77:0/2813159050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579e174d3f0 0x5579e175beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.618 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.611+0000 7fd6bed9d700 1 -- 172.21.15.77:0/2813159050 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579e14f2250 con 0x5579e174ea70 2023-12-01T01:22:30.618 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.611+0000 7fd6bed9d700 1 --2- 172.21.15.77:0/2813159050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e174ea70 0x5579e1751310 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8007650 tx=0x7fd6b8007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86eb1da3d02391b2 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:30.618 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.611+0000 7fd6ad7fa700 1 -- 172.21.15.77:0/2813159050 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b800e040 con 0x5579e174ea70 2023-12-01T01:22:30.618 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.611+0000 7fd6ad7fa700 1 -- 172.21.15.77:0/2813159050 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6b8021070 con 0x5579e174ea70 2023-12-01T01:22:30.618 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.611+0000 7fd6ad7fa700 1 -- 172.21.15.77:0/2813159050 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b80056d0 con 0x5579e174ea70 2023-12-01T01:22:30.618 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.612+0000 7fd6d118cb80 1 -- 172.21.15.77:0/2813159050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e174ea70 msgr2=0x5579e1751310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:30.618 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.612+0000 7fd6d118cb80 1 --2- 172.21.15.77:0/2813159050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e174ea70 0x5579e1751310 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8007650 tx=0x7fd6b8007680 comp rx=0 tx=0).stop 2023-12-01T01:22:30.618 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.612+0000 7fd6d118cb80 1 -- 172.21.15.77:0/2813159050 shutdown_connections 2023-12-01T01:22:30.618 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.612+0000 7fd6d118cb80 1 --2- 172.21.15.77:0/2813159050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e174ea70 0x5579e1751310 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.618 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.612+0000 7fd6d118cb80 1 --2- 172.21.15.77:0/2813159050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579e174d3f0 0x5579e175beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.618 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.612+0000 7fd6d118cb80 1 --2- 172.21.15.77:0/2813159050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579e174e0f0 0x5579e175e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.618 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.612+0000 7fd6d118cb80 1 -- 172.21.15.77:0/2813159050 >> 172.21.15.77:0/2813159050 conn(0x5579e16ac0d0 msgr2=0x5579e1747910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:30.619 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.612+0000 7fd6d118cb80 1 -- 172.21.15.77:0/2813159050 shutdown_connections 2023-12-01T01:22:30.619 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.612+0000 7fd6d118cb80 1 -- 172.21.15.77:0/2813159050 wait complete. 2023-12-01T01:22:30.619 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.612+0000 7fd6d118cb80 1 -- start start 2023-12-01T01:22:30.619 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.612+0000 7fd6d118cb80 1 Processor -- start 2023-12-01T01:22:30.619 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.612+0000 7fd6d118cb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579e174d3f0 0x5579e175bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.619 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.612+0000 7fd6d118cb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e174e0f0 0x5579e175a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.619 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.612+0000 7fd6d118cb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579e174ea70 0x5579e175a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.619 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.612+0000 7fd6d118cb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579e175ae00 con 0x5579e174e0f0 2023-12-01T01:22:30.619 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.612+0000 7fd6d118cb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579e175af70 con 0x5579e174d3f0 2023-12-01T01:22:30.619 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.612+0000 7fd6d118cb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579e175b130 con 0x5579e174ea70 2023-12-01T01:22:30.619 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.612+0000 7fd6d118cb80 10 client.0 _ll_register_callbacks cb 0x5579e1587a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:22:30.619 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.612+0000 7fd6bdd9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e174e0f0 0x5579e175a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.619 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.612+0000 7fd6bdd9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e174e0f0 0x5579e175a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32946/0 (socket says 172.21.15.77:32946) 2023-12-01T01:22:30.619 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.612+0000 7fd6bdd9b700 1 -- 172.21.15.77:0/2253709034 learned_addr learned my addr 172.21.15.77:0/2253709034 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:22:30.620 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.612+0000 7fd6bed9d700 1 --2- 172.21.15.77:0/2253709034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579e174ea70 0x5579e175a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.620 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.612+0000 7fd6be59c700 1 --2- 172.21.15.77:0/2253709034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579e174d3f0 0x5579e175bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.620 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.613+0000 7fd6bdd9b700 1 -- 172.21.15.77:0/2253709034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579e174ea70 msgr2=0x5579e175a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:30.620 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.613+0000 7fd6bdd9b700 1 --2- 172.21.15.77:0/2253709034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579e174ea70 0x5579e175a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.620 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.613+0000 7fd6bdd9b700 1 -- 172.21.15.77:0/2253709034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579e174d3f0 msgr2=0x5579e175bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:30.620 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.613+0000 7fd6bdd9b700 1 --2- 172.21.15.77:0/2253709034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579e174d3f0 0x5579e175bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.620 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.613+0000 7fd6bdd9b700 1 -- 172.21.15.77:0/2253709034 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6c0109b30 con 0x5579e174e0f0 2023-12-01T01:22:30.620 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.613+0000 7fd6bdd9b700 1 --2- 172.21.15.77:0/2253709034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e174e0f0 0x5579e175a310 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4098b30 tx=0x7fd6b4009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a894b5dbd40e9e0f server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:30.620 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.613+0000 7fd6a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:22:30.620 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.613+0000 7fd6a5ffb700 1 -- 172.21.15.77:0/2253709034 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b4098e30 con 0x5579e174e0f0 2023-12-01T01:22:30.620 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.613+0000 7fd6a5ffb700 1 -- 172.21.15.77:0/2253709034 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fd6b4079070 con 0x5579e174e0f0 2023-12-01T01:22:30.620 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.613+0000 7fd6d118cb80 1 -- 172.21.15.77:0/2253709034 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd6b40a10d0 con 0x5579e174e0f0 2023-12-01T01:22:30.620 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.613+0000 7fd6a5ffb700 1 -- 172.21.15.77:0/2253709034 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6b40992c0 con 0x5579e174e0f0 2023-12-01T01:22:30.620 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.613+0000 7fd6d118cb80 10 client.16251 fetch_fsmap learned FSMap version 41 2023-12-01T01:22:30.620 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.613+0000 7fd6d118cb80 1 -- 172.21.15.77:0/2253709034 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5579e14ca2b0 con 0x5579e174e0f0 2023-12-01T01:22:30.621 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.613+0000 7fd6a5ffb700 1 -- 172.21.15.77:0/2253709034 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6b40a10d0 con 0x5579e174e0f0 2023-12-01T01:22:30.621 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.613+0000 7fd6a5ffb700 1 -- 172.21.15.77:0/2253709034 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 41) v1 ==== 7514+0+0 (secure 0 0 0) 0x7fd6b40a0c40 con 0x5579e174e0f0 2023-12-01T01:22:30.621 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.613+0000 7fd6d118cb80 10 client.16251 fetch_fsmap finished waiting for FSMap version 41 2023-12-01T01:22:30.621 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.613+0000 7fd6d118cb80 10 client.16251 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2023-12-01T01:22:30.621 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.613+0000 7fd6d118cb80 20 client.16251 populate_metadata read hostname 'smithi077' 2023-12-01T01:22:30.621 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.613+0000 7fd6d118cb80 1 --2- 172.21.15.77:0/2253709034 >> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] conn(0x5579e17b9cf0 0x5579e17bc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.621 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.613+0000 7fd6d118cb80 4 client.16251 mds_command: new command op to 14649 tid=0[{"prefix": "damage ls"}] 2023-12-01T01:22:30.621 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.613+0000 7fd6d118cb80 1 -- 172.21.15.77:0/2253709034 --> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5579e1696d40 con 0x5579e17b9cf0 2023-12-01T01:22:30.621 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.614+0000 7fd6be59c700 1 --2- 172.21.15.77:0/2253709034 >> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] conn(0x5579e17b9cf0 0x5579e17bc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.621 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.614+0000 7fd6be59c700 1 --2- 172.21.15.77:0/2253709034 >> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] conn(0x5579e17b9cf0 0x5579e17bc110 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bccac81457032245 server_cookie=21374d2266755732 in_seq=0 out_seq=0 2023-12-01T01:22:30.621 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.614+0000 7fd6a5ffb700 10 client.16251 ms_handle_connect on v2:172.21.15.77:6840/3023184965 2023-12-01T01:22:30.621 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.614+0000 7fd6a5ffb700 1 -- 172.21.15.77:0/2253709034 <== mds.4 v2:172.21.15.77:6840/3023184965 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5579e1696d40 con 0x5579e17b9cf0 2023-12-01T01:22:30.621 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.614+0000 7fd6a5ffb700 10 client.16251 handle_command_reply: tid=0 2023-12-01T01:22:30.621 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.614+0000 7fd6d118cb80 1 client.16251 shutdown 2023-12-01T01:22:30.622 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.614+0000 7fd6d118cb80 1 -- 172.21.15.77:0/2253709034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e174e0f0 msgr2=0x5579e175a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:30.622 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.614+0000 7fd6d118cb80 1 --2- 172.21.15.77:0/2253709034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e174e0f0 0x5579e175a310 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4098b30 tx=0x7fd6b4009650 comp rx=0 tx=0).stop 2023-12-01T01:22:30.622 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.614+0000 7fd6d118cb80 1 -- 172.21.15.77:0/2253709034 shutdown_connections 2023-12-01T01:22:30.622 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.614+0000 7fd6d118cb80 1 --2- 172.21.15.77:0/2253709034 >> [v2:172.21.15.77:6840/3023184965,v1:172.21.15.77:6841/3023184965] conn(0x5579e17b9cf0 0x5579e17bc110 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.622 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.614+0000 7fd6d118cb80 1 --2- 172.21.15.77:0/2253709034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579e174ea70 0x5579e175a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.622 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.614+0000 7fd6a5ffb700 0 client.16251 ms_handle_reset on v2:172.21.15.77:6840/3023184965 2023-12-01T01:22:30.622 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.614+0000 7fd6d118cb80 1 --2- 172.21.15.77:0/2253709034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5579e174d3f0 0x5579e175bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.622 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.614+0000 7fd6d118cb80 1 --2- 172.21.15.77:0/2253709034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5579e174e0f0 0x5579e175a310 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.622 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.614+0000 7fd6d118cb80 1 -- 172.21.15.77:0/2253709034 >> 172.21.15.77:0/2253709034 conn(0x5579e16ac0d0 msgr2=0x5579e1751310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:30.622 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.614+0000 7fd6d118cb80 1 -- 172.21.15.77:0/2253709034 shutdown_connections 2023-12-01T01:22:30.622 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.614+0000 7fd6d118cb80 1 -- 172.21.15.77:0/2253709034 wait complete. 2023-12-01T01:22:30.622 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.614+0000 7fd6d118cb80 20 client.16251 trim_cache size 0 max 16384 2023-12-01T01:22:30.623 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2023-12-01T01:22:30.623 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.615+0000 7fd6c4ffc700 1 -- 172.21.15.77:0/911539046 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd6a80a0520 msgr2=0x7fd6a80a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:30.623 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.615+0000 7fd6c4ffc700 1 --2- 172.21.15.77:0/911539046 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd6a80a0520 0x7fd6a80a29e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0101df0 tx=0x7fd6b000b410 comp rx=0 tx=0).stop 2023-12-01T01:22:30.623 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.615+0000 7fd6c4ffc700 1 -- 172.21.15.77:0/911539046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6c0101f90 msgr2=0x7fd6c0197080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:30.624 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.615+0000 7fd6c4ffc700 1 --2- 172.21.15.77:0/911539046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6c0101f90 0x7fd6c0197080 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd6b400d950 tx=0x7fd6b400dc60 comp rx=0 tx=0).stop 2023-12-01T01:22:30.624 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.615+0000 7fd6c4ffc700 1 -- 172.21.15.77:0/911539046 shutdown_connections 2023-12-01T01:22:30.624 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.615+0000 7fd6c4ffc700 1 --2- 172.21.15.77:0/911539046 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fd6a80a0520 0x7fd6a80a29e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.624 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.615+0000 7fd6c4ffc700 1 --2- 172.21.15.77:0/911539046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6c0102930 0x7fd6c019e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.624 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.615+0000 7fd6c4ffc700 1 --2- 172.21.15.77:0/911539046 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6c0100da0 0x7fd6c0196b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.624 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.615+0000 7fd6c4ffc700 1 --2- 172.21.15.77:0/911539046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6c0101f90 0x7fd6c0197080 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.624 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.615+0000 7fd6c4ffc700 1 -- 172.21.15.77:0/911539046 >> 172.21.15.77:0/911539046 conn(0x7fd6c00fc310 msgr2=0x7fd6c00fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:30.624 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.615+0000 7fd6c4ffc700 1 -- 172.21.15.77:0/911539046 shutdown_connections 2023-12-01T01:22:30.624 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.615+0000 7fd6c4ffc700 1 -- 172.21.15.77:0/911539046 wait complete. 2023-12-01T01:22:30.628 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-12-01T01:22:30.753 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.750+0000 7f4909d79700 1 -- 172.21.15.77:0/3551813039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49040ffeb0 msgr2=0x7f4904100330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:30.754 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.750+0000 7f4909d79700 1 --2- 172.21.15.77:0/3551813039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49040ffeb0 0x7f4904100330 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f48f4009990 tx=0x7f48f4009ca0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.754 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.750+0000 7f4909d79700 1 -- 172.21.15.77:0/3551813039 shutdown_connections 2023-12-01T01:22:30.754 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.750+0000 7f4909d79700 1 --2- 172.21.15.77:0/3551813039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4904100870 0x7f4904108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.754 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.750+0000 7f4909d79700 1 --2- 172.21.15.77:0/3551813039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4904101260 0x7f4904101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.754 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.750+0000 7f4909d79700 1 --2- 172.21.15.77:0/3551813039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49040ffeb0 0x7f4904100330 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.754 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.750+0000 7f4909d79700 1 -- 172.21.15.77:0/3551813039 >> 172.21.15.77:0/3551813039 conn(0x7f49040fa980 msgr2=0x7f49040fcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:30.754 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.750+0000 7f4909d79700 1 -- 172.21.15.77:0/3551813039 shutdown_connections 2023-12-01T01:22:30.754 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.750+0000 7f4909d79700 1 -- 172.21.15.77:0/3551813039 wait complete. 2023-12-01T01:22:30.754 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.750+0000 7f4909d79700 1 Processor -- start 2023-12-01T01:22:30.754 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.751+0000 7f4909d79700 1 -- start start 2023-12-01T01:22:30.755 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.751+0000 7f4909d79700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49040ffeb0 0x7f4904195910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.755 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.751+0000 7f4909d79700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4904100870 0x7f4904195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.755 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.751+0000 7f4909d79700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4904101260 0x7f490419cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.755 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.751+0000 7f4909d79700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490419d490 con 0x7f49040ffeb0 2023-12-01T01:22:30.755 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.751+0000 7f4909d79700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490419d5d0 con 0x7f4904101260 2023-12-01T01:22:30.755 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.751+0000 7f4909d79700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490419d760 con 0x7f4904100870 2023-12-01T01:22:30.755 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.751+0000 7f49037fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49040ffeb0 0x7f4904195910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.755 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.751+0000 7f49037fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49040ffeb0 0x7f4904195910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32958/0 (socket says 172.21.15.77:32958) 2023-12-01T01:22:30.755 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.751+0000 7f49037fe700 1 -- 172.21.15.77:0/1428661914 learned_addr learned my addr 172.21.15.77:0/1428661914 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:22:30.755 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.751+0000 7f4902ffd700 1 --2- 172.21.15.77:0/1428661914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4904100870 0x7f4904195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.755 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.751+0000 7f49037fe700 1 -- 172.21.15.77:0/1428661914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4904100870 msgr2=0x7f4904195e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:30.755 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.751+0000 7f49037fe700 1 --2- 172.21.15.77:0/1428661914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4904100870 0x7f4904195e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.755 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.751+0000 7f49037fe700 1 -- 172.21.15.77:0/1428661914 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4904101260 msgr2=0x7f490419cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:30.755 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.751+0000 7f4903fff700 1 --2- 172.21.15.77:0/1428661914 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4904101260 0x7f490419cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.756 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.751+0000 7f49037fe700 1 --2- 172.21.15.77:0/1428661914 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4904101260 0x7f490419cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.756 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.751+0000 7f49037fe700 1 -- 172.21.15.77:0/1428661914 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f40095f0 con 0x7f49040ffeb0 2023-12-01T01:22:30.756 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.752+0000 7f49037fe700 1 --2- 172.21.15.77:0/1428661914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49040ffeb0 0x7f4904195910 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f48f800d950 tx=0x7f48f800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:30.756 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.752+0000 7f4900ff9700 1 -- 172.21.15.77:0/1428661914 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f8009940 con 0x7f49040ffeb0 2023-12-01T01:22:30.756 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.752+0000 7f4900ff9700 1 -- 172.21.15.77:0/1428661914 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48f8010460 con 0x7f49040ffeb0 2023-12-01T01:22:30.756 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.752+0000 7f4909d79700 1 -- 172.21.15.77:0/1428661914 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f490419da40 con 0x7f49040ffeb0 2023-12-01T01:22:30.756 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.752+0000 7f4900ff9700 1 -- 172.21.15.77:0/1428661914 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f800f620 con 0x7f49040ffeb0 2023-12-01T01:22:30.756 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.752+0000 7f4909d79700 1 -- 172.21.15.77:0/1428661914 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f490419df50 con 0x7f49040ffeb0 2023-12-01T01:22:30.762 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.754+0000 7f4900ff9700 1 -- 172.21.15.77:0/1428661914 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f48f80108d0 con 0x7f49040ffeb0 2023-12-01T01:22:30.762 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.754+0000 7f4900ff9700 1 --2- 172.21.15.77:0/1428661914 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f48ec0a0520 0x7f48ec0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.762 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.755+0000 7f4902ffd700 1 --2- 172.21.15.77:0/1428661914 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f48ec0a0520 0x7f48ec0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.762 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.755+0000 7f4900ff9700 1 -- 172.21.15.77:0/1428661914 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f48f80cb800 con 0x7f49040ffeb0 2023-12-01T01:22:30.762 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.755+0000 7f4902ffd700 1 --2- 172.21.15.77:0/1428661914 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f48ec0a0520 0x7f48ec0a29e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f48f400b5c0 tx=0x7f48f4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:30.763 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.755+0000 7f4915f09b80 1 Processor -- start 2023-12-01T01:22:30.763 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.755+0000 7f4915f09b80 1 -- start start 2023-12-01T01:22:30.763 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.755+0000 7f4915f09b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f95ba9300 0x556f95ba9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.763 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.755+0000 7f4915f09b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f95ba9c80 0x556f95ba8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.763 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.755+0000 7f4915f09b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f95ba8600 0x556f95ba8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.763 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.755+0000 7f4915f09b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f95afda10 con 0x556f95ba8600 2023-12-01T01:22:30.763 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.755+0000 7f4915f09b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f95afb3d0 con 0x556f95ba9300 2023-12-01T01:22:30.763 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.755+0000 7f4915f09b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f95af6cc0 con 0x556f95ba9c80 2023-12-01T01:22:30.763 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.756+0000 7f4903fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f95ba8600 0x556f95ba8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.763 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.756+0000 7f4903fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f95ba8600 0x556f95ba8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32970/0 (socket says 172.21.15.77:32970) 2023-12-01T01:22:30.763 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.756+0000 7f4903fff700 1 -- 172.21.15.77:0/2041651864 learned_addr learned my addr 172.21.15.77:0/2041651864 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:22:30.763 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.756+0000 7f4902ffd700 1 --2- 172.21.15.77:0/2041651864 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f95ba9c80 0x556f95ba8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.763 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.756+0000 7f49037fe700 1 --2- 172.21.15.77:0/2041651864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f95ba9300 0x556f95ba9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.763 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.756+0000 7f4903fff700 1 -- 172.21.15.77:0/2041651864 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f95ba9c80 msgr2=0x556f95ba8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:30.764 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.756+0000 7f4903fff700 1 --2- 172.21.15.77:0/2041651864 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f95ba9c80 0x556f95ba8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.764 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.756+0000 7f4903fff700 1 -- 172.21.15.77:0/2041651864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f95ba9300 msgr2=0x556f95ba9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:30.764 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.756+0000 7f4903fff700 1 --2- 172.21.15.77:0/2041651864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f95ba9300 0x556f95ba9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.764 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.756+0000 7f4903fff700 1 -- 172.21.15.77:0/2041651864 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f95924de0 con 0x556f95ba8600 2023-12-01T01:22:30.764 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.756+0000 7f4903fff700 1 --2- 172.21.15.77:0/2041651864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f95ba8600 0x556f95ba8a80 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f48fc0060b0 tx=0x7f48fc007500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d3f55747d0a02e3 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:30.764 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.756+0000 7f48e9ffb700 1 -- 172.21.15.77:0/2041651864 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc003a00 con 0x556f95ba8600 2023-12-01T01:22:30.764 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.756+0000 7f48e9ffb700 1 -- 172.21.15.77:0/2041651864 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48fc003b60 con 0x556f95ba8600 2023-12-01T01:22:30.764 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.756+0000 7f48e9ffb700 1 -- 172.21.15.77:0/2041651864 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc014660 con 0x556f95ba8600 2023-12-01T01:22:30.764 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.757+0000 7f4915f09b80 1 -- 172.21.15.77:0/2041651864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f95ba8600 msgr2=0x556f95ba8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:30.764 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.757+0000 7f4915f09b80 1 --2- 172.21.15.77:0/2041651864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f95ba8600 0x556f95ba8a80 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f48fc0060b0 tx=0x7f48fc007500 comp rx=0 tx=0).stop 2023-12-01T01:22:30.764 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.757+0000 7f4915f09b80 1 -- 172.21.15.77:0/2041651864 shutdown_connections 2023-12-01T01:22:30.764 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.757+0000 7f4915f09b80 1 --2- 172.21.15.77:0/2041651864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f95ba8600 0x556f95ba8a80 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.764 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.757+0000 7f4915f09b80 1 --2- 172.21.15.77:0/2041651864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f95ba9300 0x556f95ba9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.764 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.757+0000 7f4915f09b80 1 --2- 172.21.15.77:0/2041651864 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f95ba9c80 0x556f95ba8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.757+0000 7f4915f09b80 1 -- 172.21.15.77:0/2041651864 >> 172.21.15.77:0/2041651864 conn(0x556f95b050d0 msgr2=0x556f95ba0940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:30.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.757+0000 7f4915f09b80 1 -- 172.21.15.77:0/2041651864 shutdown_connections 2023-12-01T01:22:30.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.757+0000 7f4915f09b80 1 -- 172.21.15.77:0/2041651864 wait complete. 2023-12-01T01:22:30.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.757+0000 7f4915f09b80 1 -- start start 2023-12-01T01:22:30.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.758+0000 7f4915f09b80 1 Processor -- start 2023-12-01T01:22:30.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.758+0000 7f4915f09b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f95ba8600 0x556f95bb1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.758+0000 7f4915f09b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f95ba9300 0x556f95bb2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.758+0000 7f4915f09b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f95ba9c80 0x556f95bb90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.758+0000 7f4915f09b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f95a93510 con 0x556f95ba8600 2023-12-01T01:22:30.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.758+0000 7f4915f09b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f95a931b0 con 0x556f95ba9300 2023-12-01T01:22:30.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.758+0000 7f4915f09b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f9597a920 con 0x556f95ba9c80 2023-12-01T01:22:30.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.758+0000 7f4915f09b80 10 client.0 _ll_register_callbacks cb 0x556f959e0a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:22:30.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.758+0000 7f49037fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f95ba8600 0x556f95bb1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.758+0000 7f49037fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f95ba8600 0x556f95bb1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32972/0 (socket says 172.21.15.77:32972) 2023-12-01T01:22:30.765 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.758+0000 7f49037fe700 1 -- 172.21.15.77:0/1567198252 learned_addr learned my addr 172.21.15.77:0/1567198252 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:22:30.766 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.758+0000 7f4903fff700 1 --2- 172.21.15.77:0/1567198252 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f95ba9c80 0x556f95bb90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.766 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.758+0000 7f4902ffd700 1 --2- 172.21.15.77:0/1567198252 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f95ba9300 0x556f95bb2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.766 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.758+0000 7f49037fe700 1 -- 172.21.15.77:0/1567198252 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f95ba9c80 msgr2=0x556f95bb90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:30.766 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.758+0000 7f49037fe700 1 --2- 172.21.15.77:0/1567198252 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f95ba9c80 0x556f95bb90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.766 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.758+0000 7f49037fe700 1 -- 172.21.15.77:0/1567198252 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f95ba9300 msgr2=0x556f95bb2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:30.766 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.758+0000 7f49037fe700 1 --2- 172.21.15.77:0/1567198252 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f95ba9300 0x556f95bb2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.766 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.758+0000 7f49037fe700 1 -- 172.21.15.77:0/1567198252 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48fc009920 con 0x556f95ba8600 2023-12-01T01:22:30.766 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.758+0000 7f49037fe700 1 --2- 172.21.15.77:0/1567198252 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f95ba8600 0x556f95bb1b30 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f48f8010fd0 tx=0x7f48f8099130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1723b0a4f3b5d815 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:30.766 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.758+0000 7f48e2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:22:30.766 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.758+0000 7f48e2ffd700 1 -- 172.21.15.77:0/1567198252 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f80994c0 con 0x556f95ba8600 2023-12-01T01:22:30.766 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.759+0000 7f48e2ffd700 1 -- 172.21.15.77:0/1567198252 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f48f80a10d0 con 0x556f95ba8600 2023-12-01T01:22:30.766 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.759+0000 7f4915f09b80 1 -- 172.21.15.77:0/1567198252 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556f95b21d10 con 0x556f95ba8600 2023-12-01T01:22:30.766 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.759+0000 7f48e2ffd700 1 -- 172.21.15.77:0/1567198252 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48f8098550 con 0x556f95ba8600 2023-12-01T01:22:30.766 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.759+0000 7f4915f09b80 10 client.16263 fetch_fsmap learned FSMap version 41 2023-12-01T01:22:30.767 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.759+0000 7f4915f09b80 1 -- 172.21.15.77:0/1567198252 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x556f9590dc60 con 0x556f95ba8600 2023-12-01T01:22:30.767 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.759+0000 7f48e2ffd700 1 -- 172.21.15.77:0/1567198252 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556f95b21d10 con 0x556f95ba8600 2023-12-01T01:22:30.767 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.759+0000 7f48e2ffd700 1 -- 172.21.15.77:0/1567198252 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 41) v1 ==== 7514+0+0 (secure 0 0 0) 0x7f48f8094ee0 con 0x556f95ba8600 2023-12-01T01:22:30.767 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.759+0000 7f4915f09b80 10 client.16263 fetch_fsmap finished waiting for FSMap version 41 2023-12-01T01:22:30.767 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.759+0000 7f4915f09b80 10 client.16263 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2023-12-01T01:22:30.767 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.759+0000 7f4915f09b80 20 client.16263 populate_metadata read hostname 'smithi077' 2023-12-01T01:22:30.767 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.759+0000 7f4915f09b80 1 --2- 172.21.15.77:0/1567198252 >> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] conn(0x556f95bbd720 0x556f95bbfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.767 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.759+0000 7f4915f09b80 4 client.16263 mds_command: new command op to 24473 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:22:30.767 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.759+0000 7f4915f09b80 1 -- 172.21.15.77:0/1567198252 --> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556f95b74e10 con 0x556f95bbd720 2023-12-01T01:22:30.767 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.759+0000 7f4902ffd700 1 --2- 172.21.15.77:0/1567198252 >> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] conn(0x556f95bbd720 0x556f95bbfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.767 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.760+0000 7f48e2ffd700 10 client.16263 ms_handle_connect on v2:172.21.15.137:6836/3518881867 2023-12-01T01:22:30.767 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.760+0000 7f4902ffd700 1 --2- 172.21.15.77:0/1567198252 >> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] conn(0x556f95bbd720 0x556f95bbfb40 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cc59942968b97299 server_cookie=9d5fe4630f995c3c in_seq=0 out_seq=0 2023-12-01T01:22:30.767 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.760+0000 7f48e2ffd700 1 -- 172.21.15.77:0/1567198252 <== mds.3 v2:172.21.15.137:6836/3518881867 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x556f95b74e10 con 0x556f95bbd720 2023-12-01T01:22:30.767 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.760+0000 7f48e2ffd700 10 client.16263 handle_command_reply: tid=0 2023-12-01T01:22:30.768 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.761+0000 7f4915f09b80 1 client.16263 shutdown 2023-12-01T01:22:30.768 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.761+0000 7f4915f09b80 1 -- 172.21.15.77:0/1567198252 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f95ba8600 msgr2=0x556f95bb1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:30.768 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.761+0000 7f4915f09b80 1 --2- 172.21.15.77:0/1567198252 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f95ba8600 0x556f95bb1b30 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f48f8010fd0 tx=0x7f48f8099130 comp rx=0 tx=0).stop 2023-12-01T01:22:30.768 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.761+0000 7f4915f09b80 1 -- 172.21.15.77:0/1567198252 shutdown_connections 2023-12-01T01:22:30.768 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.761+0000 7f4915f09b80 1 --2- 172.21.15.77:0/1567198252 >> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] conn(0x556f95bbd720 0x556f95bbfb40 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.768 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.761+0000 7f4915f09b80 1 --2- 172.21.15.77:0/1567198252 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f95ba9c80 0x556f95bb90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.768 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.761+0000 7f48e2ffd700 0 client.16263 ms_handle_reset on v2:172.21.15.137:6836/3518881867 2023-12-01T01:22:30.768 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.761+0000 7f4915f09b80 1 --2- 172.21.15.77:0/1567198252 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f95ba8600 0x556f95bb1b30 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.768 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.761+0000 7f4915f09b80 1 --2- 172.21.15.77:0/1567198252 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f95ba9300 0x556f95bb2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.768 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.761+0000 7f4915f09b80 1 -- 172.21.15.77:0/1567198252 >> 172.21.15.77:0/1567198252 conn(0x556f95b050d0 msgr2=0x556f95ba00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:30.768 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.761+0000 7f4915f09b80 1 -- 172.21.15.77:0/1567198252 shutdown_connections 2023-12-01T01:22:30.768 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.761+0000 7f4915f09b80 1 -- 172.21.15.77:0/1567198252 wait complete. 2023-12-01T01:22:30.768 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.761+0000 7f4915f09b80 20 client.16263 trim_cache size 0 max 16384 2023-12-01T01:22:30.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.774+0000 7f4915f09b80 1 Processor -- start 2023-12-01T01:22:30.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.774+0000 7f4915f09b80 1 -- start start 2023-12-01T01:22:30.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.774+0000 7f4915f09b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f95ba8600 0x556f95bb7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.774+0000 7f4915f09b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f95ba9300 0x556f95bb11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.774+0000 7f4915f09b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f95ba9c80 0x556f95ba0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.774+0000 7f4915f09b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f95ba0e80 con 0x556f95ba8600 2023-12-01T01:22:30.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.774+0000 7f4915f09b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f95bbd210 con 0x556f95ba9300 2023-12-01T01:22:30.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.774+0000 7f4915f09b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f95baf6f0 con 0x556f95ba9c80 2023-12-01T01:22:30.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.774+0000 7f49037fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f95ba8600 0x556f95bb7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.774+0000 7f49037fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f95ba8600 0x556f95bb7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32980/0 (socket says 172.21.15.77:32980) 2023-12-01T01:22:30.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.774+0000 7f49037fe700 1 -- 172.21.15.77:0/1743484099 learned_addr learned my addr 172.21.15.77:0/1743484099 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:22:30.781 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.774+0000 7f4903fff700 1 --2- 172.21.15.77:0/1743484099 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f95ba9c80 0x556f95ba0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.782 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.774+0000 7f49037fe700 1 -- 172.21.15.77:0/1743484099 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f95ba9c80 msgr2=0x556f95ba0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:30.782 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.774+0000 7f49037fe700 1 --2- 172.21.15.77:0/1743484099 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f95ba9c80 0x556f95ba0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.782 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.774+0000 7f49037fe700 1 -- 172.21.15.77:0/1743484099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f95ba9300 msgr2=0x556f95bb11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:30.782 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.774+0000 7f49037fe700 1 --2- 172.21.15.77:0/1743484099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f95ba9300 0x556f95bb11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.782 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.774+0000 7f49037fe700 1 -- 172.21.15.77:0/1743484099 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f9594b250 con 0x556f95ba8600 2023-12-01T01:22:30.782 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.774+0000 7f49037fe700 1 --2- 172.21.15.77:0/1743484099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f95ba8600 0x556f95bb7060 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f48f8098ff0 tx=0x7f48f8098c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b40fc25e1d21ba8 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:30.782 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.774+0000 7f48ea7fc700 1 -- 172.21.15.77:0/1743484099 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f8014030 con 0x556f95ba8600 2023-12-01T01:22:30.782 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.774+0000 7f48ea7fc700 1 -- 172.21.15.77:0/1743484099 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48f8095050 con 0x556f95ba8600 2023-12-01T01:22:30.782 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.774+0000 7f48ea7fc700 1 -- 172.21.15.77:0/1743484099 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f80987b0 con 0x556f95ba8600 2023-12-01T01:22:30.782 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.775+0000 7f4915f09b80 1 -- 172.21.15.77:0/1743484099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f95ba8600 msgr2=0x556f95bb7060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:30.782 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.775+0000 7f4915f09b80 1 --2- 172.21.15.77:0/1743484099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f95ba8600 0x556f95bb7060 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f48f8098ff0 tx=0x7f48f8098c20 comp rx=0 tx=0).stop 2023-12-01T01:22:30.782 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.775+0000 7f4915f09b80 1 -- 172.21.15.77:0/1743484099 shutdown_connections 2023-12-01T01:22:30.782 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.775+0000 7f4915f09b80 1 --2- 172.21.15.77:0/1743484099 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f95ba9c80 0x556f95ba0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.782 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.775+0000 7f4915f09b80 1 --2- 172.21.15.77:0/1743484099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f95ba8600 0x556f95bb7060 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.783 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.775+0000 7f4915f09b80 1 --2- 172.21.15.77:0/1743484099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f95ba9300 0x556f95bb11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.783 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.775+0000 7f4915f09b80 1 -- 172.21.15.77:0/1743484099 >> 172.21.15.77:0/1743484099 conn(0x556f95b050d0 msgr2=0x556f95b9cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:30.783 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.775+0000 7f4915f09b80 1 -- 172.21.15.77:0/1743484099 shutdown_connections 2023-12-01T01:22:30.783 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.775+0000 7f4915f09b80 1 -- 172.21.15.77:0/1743484099 wait complete. 2023-12-01T01:22:30.783 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.775+0000 7f4915f09b80 1 -- start start 2023-12-01T01:22:30.783 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.776+0000 7f4915f09b80 1 Processor -- start 2023-12-01T01:22:30.783 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.776+0000 7f4915f09b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f95ba8600 0x556f95bb5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.783 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.776+0000 7f4915f09b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f95ba9300 0x556f95bb5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.783 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.776+0000 7f4915f09b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f95ba9c80 0x556f95bb6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.783 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.776+0000 7f4915f09b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f95bb6ac0 con 0x556f95ba9c80 2023-12-01T01:22:30.783 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.776+0000 7f4915f09b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f95bb6c30 con 0x556f95ba8600 2023-12-01T01:22:30.783 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.776+0000 7f4915f09b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f95bb6df0 con 0x556f95ba9300 2023-12-01T01:22:30.784 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.776+0000 7f4915f09b80 10 client.0 _ll_register_callbacks cb 0x7f48f8099760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:22:30.784 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.776+0000 7f4903fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f95ba9c80 0x556f95bb6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.784 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.776+0000 7f4903fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f95ba9c80 0x556f95bb6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32990/0 (socket says 172.21.15.77:32990) 2023-12-01T01:22:30.784 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.776+0000 7f4903fff700 1 -- 172.21.15.77:0/1692942324 learned_addr learned my addr 172.21.15.77:0/1692942324 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:22:30.784 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.776+0000 7f4902ffd700 1 --2- 172.21.15.77:0/1692942324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f95ba9300 0x556f95bb5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.784 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.776+0000 7f49037fe700 1 --2- 172.21.15.77:0/1692942324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f95ba8600 0x556f95bb5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.784 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.776+0000 7f4903fff700 1 -- 172.21.15.77:0/1692942324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f95ba9300 msgr2=0x556f95bb5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:30.784 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.776+0000 7f4903fff700 1 --2- 172.21.15.77:0/1692942324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f95ba9300 0x556f95bb5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.784 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.776+0000 7f4903fff700 1 -- 172.21.15.77:0/1692942324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f95ba8600 msgr2=0x556f95bb5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:30.784 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.776+0000 7f4903fff700 1 --2- 172.21.15.77:0/1692942324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f95ba8600 0x556f95bb5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.784 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.776+0000 7f4903fff700 1 -- 172.21.15.77:0/1692942324 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556f9590dc60 con 0x556f95ba9c80 2023-12-01T01:22:30.784 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.776+0000 7f49037fe700 1 --2- 172.21.15.77:0/1692942324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f95ba8600 0x556f95bb5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:22:30.784 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.776+0000 7f4903fff700 1 --2- 172.21.15.77:0/1692942324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f95ba9c80 0x556f95bb6580 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f48fc007850 tx=0x7f48fc014be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee88f49588c2fa87 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:30.785 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.776+0000 7f48e2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:22:30.785 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.776+0000 7f48e2ffd700 1 -- 172.21.15.77:0/1692942324 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc003a00 con 0x556f95ba9c80 2023-12-01T01:22:30.785 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.776+0000 7f48e2ffd700 1 -- 172.21.15.77:0/1692942324 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f48fc004ca0 con 0x556f95ba9c80 2023-12-01T01:22:30.785 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.776+0000 7f4915f09b80 1 -- 172.21.15.77:0/1692942324 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f48fc003b60 con 0x556f95ba9c80 2023-12-01T01:22:30.785 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.777+0000 7f48e2ffd700 1 -- 172.21.15.77:0/1692942324 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48fc004e60 con 0x556f95ba9c80 2023-12-01T01:22:30.785 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.777+0000 7f4915f09b80 10 client.16269 fetch_fsmap learned FSMap version 41 2023-12-01T01:22:30.785 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.777+0000 7f4915f09b80 1 -- 172.21.15.77:0/1692942324 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x556f959232b0 con 0x556f95ba9c80 2023-12-01T01:22:30.785 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.777+0000 7f48e2ffd700 1 -- 172.21.15.77:0/1692942324 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48fc003b60 con 0x556f95ba9c80 2023-12-01T01:22:30.785 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.777+0000 7f48e2ffd700 1 -- 172.21.15.77:0/1692942324 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 41) v1 ==== 7514+0+0 (secure 0 0 0) 0x7f48fc009ce0 con 0x556f95ba9c80 2023-12-01T01:22:30.785 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.777+0000 7f4915f09b80 10 client.16269 fetch_fsmap finished waiting for FSMap version 41 2023-12-01T01:22:30.785 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.777+0000 7f4915f09b80 10 client.16269 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2023-12-01T01:22:30.785 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.777+0000 7f4915f09b80 20 client.16269 populate_metadata read hostname 'smithi077' 2023-12-01T01:22:30.785 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.777+0000 7f4915f09b80 1 --2- 172.21.15.77:0/1692942324 >> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] conn(0x556f95c10500 0x556f95c128f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.785 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.777+0000 7f4915f09b80 4 client.16269 mds_command: new command op to 24473 tid=0[{"prefix": "damage ls"}] 2023-12-01T01:22:30.786 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.777+0000 7f4915f09b80 1 -- 172.21.15.77:0/1692942324 --> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556f95aefd40 con 0x556f95c10500 2023-12-01T01:22:30.786 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.777+0000 7f49037fe700 1 --2- 172.21.15.77:0/1692942324 >> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] conn(0x556f95c10500 0x556f95c128f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.786 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.778+0000 7f48e2ffd700 10 client.16269 ms_handle_connect on v2:172.21.15.137:6836/3518881867 2023-12-01T01:22:30.786 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.778+0000 7f49037fe700 1 --2- 172.21.15.77:0/1692942324 >> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] conn(0x556f95c10500 0x556f95c128f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b1960ec71ef2dddf server_cookie=70488e3615c07907 in_seq=0 out_seq=0 2023-12-01T01:22:30.786 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.778+0000 7f48e2ffd700 1 -- 172.21.15.77:0/1692942324 <== mds.3 v2:172.21.15.137:6836/3518881867 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556f95aefd40 con 0x556f95c10500 2023-12-01T01:22:30.786 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.778+0000 7f48e2ffd700 10 client.16269 handle_command_reply: tid=0 2023-12-01T01:22:30.786 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.778+0000 7f4915f09b80 1 client.16269 shutdown 2023-12-01T01:22:30.786 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.778+0000 7f4915f09b80 1 -- 172.21.15.77:0/1692942324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f95ba9c80 msgr2=0x556f95bb6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:30.786 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.778+0000 7f4915f09b80 1 --2- 172.21.15.77:0/1692942324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f95ba9c80 0x556f95bb6580 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f48fc007850 tx=0x7f48fc014be0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.786 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.778+0000 7f4915f09b80 1 -- 172.21.15.77:0/1692942324 shutdown_connections 2023-12-01T01:22:30.786 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.778+0000 7f4915f09b80 1 --2- 172.21.15.77:0/1692942324 >> [v2:172.21.15.137:6836/3518881867,v1:172.21.15.137:6837/3518881867] conn(0x556f95c10500 0x556f95c128f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.786 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.778+0000 7f4915f09b80 1 --2- 172.21.15.77:0/1692942324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556f95ba9c80 0x556f95bb6580 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.787 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.778+0000 7f48e2ffd700 0 client.16269 ms_handle_reset on v2:172.21.15.137:6836/3518881867 2023-12-01T01:22:30.787 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.778+0000 7f4915f09b80 1 --2- 172.21.15.77:0/1692942324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556f95ba8600 0x556f95bb5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.787 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.778+0000 7f4915f09b80 1 --2- 172.21.15.77:0/1692942324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f95ba9300 0x556f95bb5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.787 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.778+0000 7f4915f09b80 1 -- 172.21.15.77:0/1692942324 >> 172.21.15.77:0/1692942324 conn(0x556f95b050d0 msgr2=0x556f95ba89e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:30.787 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.778+0000 7f4915f09b80 1 -- 172.21.15.77:0/1692942324 shutdown_connections 2023-12-01T01:22:30.787 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.778+0000 7f4915f09b80 1 -- 172.21.15.77:0/1692942324 wait complete. 2023-12-01T01:22:30.787 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.778+0000 7f4915f09b80 20 client.16269 trim_cache size 0 max 16384 2023-12-01T01:22:30.787 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2023-12-01T01:22:30.788 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.779+0000 7f4909d79700 1 -- 172.21.15.77:0/1428661914 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f48ec0a0520 msgr2=0x7f48ec0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:30.788 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.779+0000 7f4909d79700 1 --2- 172.21.15.77:0/1428661914 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f48ec0a0520 0x7f48ec0a29e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f48f400b5c0 tx=0x7f48f4005960 comp rx=0 tx=0).stop 2023-12-01T01:22:30.789 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.779+0000 7f4909d79700 1 -- 172.21.15.77:0/1428661914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49040ffeb0 msgr2=0x7f4904195910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:30.789 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.779+0000 7f4909d79700 1 --2- 172.21.15.77:0/1428661914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49040ffeb0 0x7f4904195910 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f48f800d950 tx=0x7f48f800dc60 comp rx=0 tx=0).stop 2023-12-01T01:22:30.789 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.779+0000 7f4909d79700 1 -- 172.21.15.77:0/1428661914 shutdown_connections 2023-12-01T01:22:30.789 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.779+0000 7f4909d79700 1 --2- 172.21.15.77:0/1428661914 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f48ec0a0520 0x7f48ec0a29e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.789 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.779+0000 7f4909d79700 1 --2- 172.21.15.77:0/1428661914 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4904101260 0x7f490419cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.789 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.779+0000 7f4909d79700 1 --2- 172.21.15.77:0/1428661914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49040ffeb0 0x7f4904195910 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.789 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.779+0000 7f4909d79700 1 --2- 172.21.15.77:0/1428661914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4904100870 0x7f4904195e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.789 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.779+0000 7f4909d79700 1 -- 172.21.15.77:0/1428661914 >> 172.21.15.77:0/1428661914 conn(0x7f49040fa980 msgr2=0x7f49041043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:30.789 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.779+0000 7f4909d79700 1 -- 172.21.15.77:0/1428661914 shutdown_connections 2023-12-01T01:22:30.789 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.779+0000 7f4909d79700 1 -- 172.21.15.77:0/1428661914 wait complete. 2023-12-01T01:22:30.792 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-12-01T01:22:30.918 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.914+0000 7f51707ab700 1 -- 172.21.15.77:0/584356766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5168100da0 msgr2=0x7f51681011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:30.918 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.914+0000 7f51707ab700 1 --2- 172.21.15.77:0/584356766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5168100da0 0x7f51681011c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f51580099c0 tx=0x7f5158009cd0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.918 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.914+0000 7f51707ab700 1 -- 172.21.15.77:0/584356766 shutdown_connections 2023-12-01T01:22:30.918 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.914+0000 7f51707ab700 1 --2- 172.21.15.77:0/584356766 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5168102930 0x7f51681091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.918 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.914+0000 7f51707ab700 1 --2- 172.21.15.77:0/584356766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5168100da0 0x7f51681011c0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.918 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.914+0000 7f51707ab700 1 --2- 172.21.15.77:0/584356766 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5168101f90 0x7f51681023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.919 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.914+0000 7f51707ab700 1 -- 172.21.15.77:0/584356766 >> 172.21.15.77:0/584356766 conn(0x7f51680fc310 msgr2=0x7f51680fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:30.919 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.914+0000 7f51707ab700 1 -- 172.21.15.77:0/584356766 shutdown_connections 2023-12-01T01:22:30.919 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.914+0000 7f51707ab700 1 -- 172.21.15.77:0/584356766 wait complete. 2023-12-01T01:22:30.919 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.915+0000 7f51707ab700 1 Processor -- start 2023-12-01T01:22:30.919 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.915+0000 7f51707ab700 1 -- start start 2023-12-01T01:22:30.919 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.915+0000 7f51707ab700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5168100da0 0x7f5168196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.919 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.915+0000 7f51707ab700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5168101f90 0x7f5168196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.919 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.915+0000 7f51707ab700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5168102930 0x7f516819e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.919 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.915+0000 7f51707ab700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516819e610 con 0x7f5168100da0 2023-12-01T01:22:30.919 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.915+0000 7f51707ab700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516819e750 con 0x7f5168101f90 2023-12-01T01:22:30.919 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.915+0000 7f51707ab700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516819e8e0 con 0x7f5168102930 2023-12-01T01:22:30.919 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.915+0000 7f516e547700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5168100da0 0x7f5168196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.919 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.915+0000 7f516e547700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5168100da0 0x7f5168196a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33000/0 (socket says 172.21.15.77:33000) 2023-12-01T01:22:30.919 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.915+0000 7f516e547700 1 -- 172.21.15.77:0/3542982394 learned_addr learned my addr 172.21.15.77:0/3542982394 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:22:30.919 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.915+0000 7f516ed48700 1 --2- 172.21.15.77:0/3542982394 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5168102930 0x7f516819e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.920 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.915+0000 7f516dd46700 1 --2- 172.21.15.77:0/3542982394 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5168101f90 0x7f5168196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.920 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.915+0000 7f516e547700 1 -- 172.21.15.77:0/3542982394 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5168102930 msgr2=0x7f516819e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:30.920 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.915+0000 7f516e547700 1 --2- 172.21.15.77:0/3542982394 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5168102930 0x7f516819e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.920 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.915+0000 7f516e547700 1 -- 172.21.15.77:0/3542982394 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5168101f90 msgr2=0x7f5168196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:30.920 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.915+0000 7f516e547700 1 --2- 172.21.15.77:0/3542982394 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5168101f90 0x7f5168196fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.920 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.915+0000 7f516e547700 1 -- 172.21.15.77:0/3542982394 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51580095f0 con 0x7f5168100da0 2023-12-01T01:22:30.920 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.915+0000 7f516ed48700 1 --2- 172.21.15.77:0/3542982394 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5168102930 0x7f516819e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:22:30.920 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.915+0000 7f516dd46700 1 --2- 172.21.15.77:0/3542982394 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5168101f90 0x7f5168196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:22:30.921 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.915+0000 7f516e547700 1 --2- 172.21.15.77:0/3542982394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5168100da0 0x7f5168196a90 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f5158009990 tx=0x7f5158005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:30.921 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.915+0000 7f515f7fe700 1 -- 172.21.15.77:0/3542982394 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515801d070 con 0x7f5168100da0 2023-12-01T01:22:30.921 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.915+0000 7f515f7fe700 1 -- 172.21.15.77:0/3542982394 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5158004200 con 0x7f5168100da0 2023-12-01T01:22:30.921 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.915+0000 7f51707ab700 1 -- 172.21.15.77:0/3542982394 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f516819eb60 con 0x7f5168100da0 2023-12-01T01:22:30.921 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.915+0000 7f515f7fe700 1 -- 172.21.15.77:0/3542982394 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5158022e30 con 0x7f5168100da0 2023-12-01T01:22:30.921 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.915+0000 7f51707ab700 1 -- 172.21.15.77:0/3542982394 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f516819f010 con 0x7f5168100da0 2023-12-01T01:22:30.927 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.919+0000 7f517c93bb80 1 Processor -- start 2023-12-01T01:22:30.927 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.919+0000 7f517c93bb80 1 -- start start 2023-12-01T01:22:30.928 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.919+0000 7f517c93bb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c725e040f0 0x55c725e04530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.928 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.919+0000 7f517c93bb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c725e04a70 0x55c725e02e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.928 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.919+0000 7f517c93bb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c725e033f0 0x55c725e03870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.928 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.919+0000 7f517c93bb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c725d5aa10 con 0x55c725e04a70 2023-12-01T01:22:30.928 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.919+0000 7f517c93bb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c725d583d0 con 0x55c725e040f0 2023-12-01T01:22:30.928 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.919+0000 7f517c93bb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c725d53cc0 con 0x55c725e033f0 2023-12-01T01:22:30.928 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.919+0000 7f515f7fe700 1 -- 172.21.15.77:0/3542982394 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f5158003d40 con 0x7f5168100da0 2023-12-01T01:22:30.928 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.920+0000 7f516e547700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c725e04a70 0x55c725e02e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.928 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.920+0000 7f516e547700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c725e04a70 0x55c725e02e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33010/0 (socket says 172.21.15.77:33010) 2023-12-01T01:22:30.928 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.920+0000 7f516e547700 1 -- 172.21.15.77:0/2955078516 learned_addr learned my addr 172.21.15.77:0/2955078516 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:22:30.928 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.920+0000 7f516ed48700 1 --2- 172.21.15.77:0/2955078516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c725e040f0 0x55c725e04530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.928 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.920+0000 7f516dd46700 1 --2- 172.21.15.77:0/2955078516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c725e033f0 0x55c725e03870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.928 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.920+0000 7f516e547700 1 -- 172.21.15.77:0/2955078516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c725e033f0 msgr2=0x55c725e03870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:30.929 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.920+0000 7f516e547700 1 --2- 172.21.15.77:0/2955078516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c725e033f0 0x55c725e03870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.929 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.920+0000 7f516e547700 1 -- 172.21.15.77:0/2955078516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c725e040f0 msgr2=0x55c725e04530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:30.929 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.920+0000 7f516e547700 1 --2- 172.21.15.77:0/2955078516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c725e040f0 0x55c725e04530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.929 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.920+0000 7f516e547700 1 -- 172.21.15.77:0/2955078516 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c725b81de0 con 0x55c725e04a70 2023-12-01T01:22:30.929 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.920+0000 7f516ed48700 1 --2- 172.21.15.77:0/2955078516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c725e040f0 0x55c725e04530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:22:30.929 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.920+0000 7f516e547700 1 --2- 172.21.15.77:0/2955078516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c725e04a70 0x55c725e02e60 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f51580d0b80 tx=0x7f51580d0e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fff8162d71138a59 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:30.929 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.920+0000 7f515f7fe700 1 --2- 172.21.15.77:0/3542982394 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f51540a0520 0x7f51540a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.929 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.920+0000 7f516ed48700 1 --2- 172.21.15.77:0/3542982394 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f51540a0520 0x7f51540a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.929 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.920+0000 7f515f7fe700 1 -- 172.21.15.77:0/3542982394 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f51580ccc20 con 0x7f5168100da0 2023-12-01T01:22:30.929 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.920+0000 7f515cff9700 1 -- 172.21.15.77:0/2955078516 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5158096850 con 0x55c725e04a70 2023-12-01T01:22:30.929 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.920+0000 7f515cff9700 1 -- 172.21.15.77:0/2955078516 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51580d07c0 con 0x55c725e04a70 2023-12-01T01:22:30.929 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.920+0000 7f515cff9700 1 -- 172.21.15.77:0/2955078516 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5158099e70 con 0x55c725e04a70 2023-12-01T01:22:30.929 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.921+0000 7f516ed48700 1 --2- 172.21.15.77:0/3542982394 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f51540a0520 0x7f51540a29e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55c725e03e90 tx=0x7f5160009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:30.929 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.921+0000 7f517c93bb80 1 -- 172.21.15.77:0/2955078516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c725e04a70 msgr2=0x55c725e02e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:30.929 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.921+0000 7f517c93bb80 1 --2- 172.21.15.77:0/2955078516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c725e04a70 0x55c725e02e60 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f51580d0b80 tx=0x7f51580d0e90 comp rx=0 tx=0).stop 2023-12-01T01:22:30.930 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.921+0000 7f517c93bb80 1 -- 172.21.15.77:0/2955078516 shutdown_connections 2023-12-01T01:22:30.930 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.921+0000 7f517c93bb80 1 --2- 172.21.15.77:0/2955078516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c725e033f0 0x55c725e03870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.930 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.921+0000 7f517c93bb80 1 --2- 172.21.15.77:0/2955078516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c725e040f0 0x55c725e04530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.930 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.921+0000 7f517c93bb80 1 --2- 172.21.15.77:0/2955078516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c725e04a70 0x55c725e02e60 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.930 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.921+0000 7f517c93bb80 1 -- 172.21.15.77:0/2955078516 >> 172.21.15.77:0/2955078516 conn(0x55c725d620d0 msgr2=0x55c725dfd940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:30.930 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.921+0000 7f517c93bb80 1 -- 172.21.15.77:0/2955078516 shutdown_connections 2023-12-01T01:22:30.930 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.921+0000 7f517c93bb80 1 -- 172.21.15.77:0/2955078516 wait complete. 2023-12-01T01:22:30.930 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.921+0000 7f517c93bb80 1 -- start start 2023-12-01T01:22:30.930 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.922+0000 7f517c93bb80 1 Processor -- start 2023-12-01T01:22:30.930 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.922+0000 7f517c93bb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c725e033f0 0x55c725e0c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.930 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.922+0000 7f517c93bb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c725e040f0 0x55c725e0cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.930 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.922+0000 7f517c93bb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c725e04a70 0x55c725e13c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.930 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.922+0000 7f517c93bb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c725cf0510 con 0x55c725e04a70 2023-12-01T01:22:30.931 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.922+0000 7f517c93bb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c725cf01b0 con 0x55c725e040f0 2023-12-01T01:22:30.931 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.922+0000 7f517c93bb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c725bd7920 con 0x55c725e033f0 2023-12-01T01:22:30.931 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.922+0000 7f517c93bb80 10 client.0 _ll_register_callbacks cb 0x55c725c3da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:22:30.931 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.922+0000 7f516ed48700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c725e04a70 0x55c725e13c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.931 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.922+0000 7f516ed48700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c725e04a70 0x55c725e13c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33018/0 (socket says 172.21.15.77:33018) 2023-12-01T01:22:30.931 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.922+0000 7f516ed48700 1 -- 172.21.15.77:0/4138168103 learned_addr learned my addr 172.21.15.77:0/4138168103 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:22:30.931 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.922+0000 7f516e547700 1 --2- 172.21.15.77:0/4138168103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c725e033f0 0x55c725e0c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.931 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.922+0000 7f516dd46700 1 --2- 172.21.15.77:0/4138168103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c725e040f0 0x55c725e0cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.931 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.922+0000 7f516ed48700 1 -- 172.21.15.77:0/4138168103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c725e033f0 msgr2=0x55c725e0c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:30.931 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.922+0000 7f516ed48700 1 --2- 172.21.15.77:0/4138168103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c725e033f0 0x55c725e0c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.931 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.922+0000 7f516ed48700 1 -- 172.21.15.77:0/4138168103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c725e040f0 msgr2=0x55c725e0cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:30.933 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.922+0000 7f516ed48700 1 --2- 172.21.15.77:0/4138168103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c725e040f0 0x55c725e0cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.933 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.922+0000 7f516ed48700 1 -- 172.21.15.77:0/4138168103 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f516819f010 con 0x55c725e04a70 2023-12-01T01:22:30.933 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.922+0000 7f516dd46700 1 --2- 172.21.15.77:0/4138168103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c725e040f0 0x55c725e0cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:22:30.933 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.923+0000 7f516ed48700 1 --2- 172.21.15.77:0/4138168103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c725e04a70 0x55c725e13c30 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f5160016e50 tx=0x7f5160004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f73051502d978276 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:30.933 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.923+0000 7f5151ffb700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:22:30.933 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.923+0000 7f5151ffb700 1 -- 172.21.15.77:0/4138168103 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5160003a00 con 0x55c725e04a70 2023-12-01T01:22:30.934 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.923+0000 7f5151ffb700 1 -- 172.21.15.77:0/4138168103 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f5160003b60 con 0x55c725e04a70 2023-12-01T01:22:30.934 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.923+0000 7f517c93bb80 1 -- 172.21.15.77:0/4138168103 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c725dd1e10 con 0x55c725e04a70 2023-12-01T01:22:30.934 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.923+0000 7f5151ffb700 1 -- 172.21.15.77:0/4138168103 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5160003d20 con 0x55c725e04a70 2023-12-01T01:22:30.934 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.923+0000 7f517c93bb80 10 client.16281 fetch_fsmap learned FSMap version 41 2023-12-01T01:22:30.934 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.923+0000 7f517c93bb80 1 -- 172.21.15.77:0/4138168103 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55c725b6ac60 con 0x55c725e04a70 2023-12-01T01:22:30.934 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.923+0000 7f5151ffb700 1 -- 172.21.15.77:0/4138168103 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c725dd1e10 con 0x55c725e04a70 2023-12-01T01:22:30.934 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.923+0000 7f5151ffb700 1 -- 172.21.15.77:0/4138168103 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 41) v1 ==== 7514+0+0 (secure 0 0 0) 0x7f5160004500 con 0x55c725e04a70 2023-12-01T01:22:30.934 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.923+0000 7f517c93bb80 10 client.16281 fetch_fsmap finished waiting for FSMap version 41 2023-12-01T01:22:30.934 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.923+0000 7f517c93bb80 10 client.16281 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-01T01:22:30.934 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.923+0000 7f517c93bb80 20 client.16281 populate_metadata read hostname 'smithi077' 2023-12-01T01:22:30.934 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.923+0000 7f517c93bb80 1 --2- 172.21.15.77:0/4138168103 >> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] conn(0x55c725e185e0 0x55c725e1aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.934 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.923+0000 7f517c93bb80 4 client.16281 mds_command: new command op to 24475 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:22:30.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.923+0000 7f517c93bb80 1 -- 172.21.15.77:0/4138168103 --> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c725d4cd40 con 0x55c725e185e0 2023-12-01T01:22:30.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.924+0000 7f516e547700 1 --2- 172.21.15.77:0/4138168103 >> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] conn(0x55c725e185e0 0x55c725e1aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.924+0000 7f516e547700 1 --2- 172.21.15.77:0/4138168103 >> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] conn(0x55c725e185e0 0x55c725e1aa00 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cdf6bfd04d7e8787 server_cookie=ff1b57a4bbc76eb in_seq=0 out_seq=0 2023-12-01T01:22:30.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.924+0000 7f5151ffb700 10 client.16281 ms_handle_connect on v2:172.21.15.203:6836/2887080977 2023-12-01T01:22:30.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.925+0000 7f5151ffb700 1 -- 172.21.15.77:0/4138168103 <== mds.2 v2:172.21.15.203:6836/2887080977 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55c725d4cd40 con 0x55c725e185e0 2023-12-01T01:22:30.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.925+0000 7f5151ffb700 10 client.16281 handle_command_reply: tid=0 2023-12-01T01:22:30.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.925+0000 7f517c93bb80 1 client.16281 shutdown 2023-12-01T01:22:30.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.925+0000 7f517c93bb80 1 -- 172.21.15.77:0/4138168103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c725e04a70 msgr2=0x55c725e13c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:30.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.925+0000 7f517c93bb80 1 --2- 172.21.15.77:0/4138168103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c725e04a70 0x55c725e13c30 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f5160016e50 tx=0x7f5160004b40 comp rx=0 tx=0).stop 2023-12-01T01:22:30.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.925+0000 7f517c93bb80 1 -- 172.21.15.77:0/4138168103 shutdown_connections 2023-12-01T01:22:30.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.925+0000 7f517c93bb80 1 --2- 172.21.15.77:0/4138168103 >> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] conn(0x55c725e185e0 0x55c725e1aa00 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.925+0000 7f517c93bb80 1 --2- 172.21.15.77:0/4138168103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c725e04a70 0x55c725e13c30 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.925+0000 7f5151ffb700 0 client.16281 ms_handle_reset on v2:172.21.15.203:6836/2887080977 2023-12-01T01:22:30.935 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.925+0000 7f517c93bb80 1 --2- 172.21.15.77:0/4138168103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c725e033f0 0x55c725e0c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.936 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.925+0000 7f517c93bb80 1 --2- 172.21.15.77:0/4138168103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c725e040f0 0x55c725e0cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.936 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.925+0000 7f517c93bb80 1 -- 172.21.15.77:0/4138168103 >> 172.21.15.77:0/4138168103 conn(0x55c725d620d0 msgr2=0x55c725dfd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:30.936 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.925+0000 7f517c93bb80 1 -- 172.21.15.77:0/4138168103 shutdown_connections 2023-12-01T01:22:30.936 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.925+0000 7f517c93bb80 1 -- 172.21.15.77:0/4138168103 wait complete. 2023-12-01T01:22:30.936 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.925+0000 7f517c93bb80 20 client.16281 trim_cache size 0 max 16384 2023-12-01T01:22:30.944 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.937+0000 7f517c93bb80 1 Processor -- start 2023-12-01T01:22:30.944 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.937+0000 7f517c93bb80 1 -- start start 2023-12-01T01:22:30.945 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.937+0000 7f517c93bb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c725e033f0 0x55c725e02e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.945 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.937+0000 7f517c93bb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c725e040f0 0x55c725e11b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.945 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.937+0000 7f517c93bb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c725e04a70 0x55c725e0c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.945 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.937+0000 7f517c93bb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c725e0bb60 con 0x55c725e04a70 2023-12-01T01:22:30.945 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.937+0000 7f517c93bb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c725dff470 con 0x55c725e040f0 2023-12-01T01:22:30.945 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.937+0000 7f517c93bb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c725e0c220 con 0x55c725e033f0 2023-12-01T01:22:30.945 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.938+0000 7f516ed48700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c725e04a70 0x55c725e0c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.945 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.938+0000 7f516ed48700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c725e04a70 0x55c725e0c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33026/0 (socket says 172.21.15.77:33026) 2023-12-01T01:22:30.945 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.938+0000 7f516ed48700 1 -- 172.21.15.77:0/104119959 learned_addr learned my addr 172.21.15.77:0/104119959 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:22:30.945 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.938+0000 7f516e547700 1 --2- 172.21.15.77:0/104119959 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c725e033f0 0x55c725e02e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.945 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.938+0000 7f516dd46700 1 --2- 172.21.15.77:0/104119959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c725e040f0 0x55c725e11b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.945 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.938+0000 7f516ed48700 1 -- 172.21.15.77:0/104119959 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c725e033f0 msgr2=0x55c725e02e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:30.945 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.938+0000 7f516ed48700 1 --2- 172.21.15.77:0/104119959 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c725e033f0 0x55c725e02e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.945 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.938+0000 7f516ed48700 1 -- 172.21.15.77:0/104119959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c725e040f0 msgr2=0x55c725e11b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:30.946 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.938+0000 7f516ed48700 1 --2- 172.21.15.77:0/104119959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c725e040f0 0x55c725e11b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.946 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.938+0000 7f516ed48700 1 -- 172.21.15.77:0/104119959 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c725ba8250 con 0x55c725e04a70 2023-12-01T01:22:30.946 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.938+0000 7f516dd46700 1 --2- 172.21.15.77:0/104119959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c725e040f0 0x55c725e11b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:22:30.946 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.938+0000 7f516ed48700 1 --2- 172.21.15.77:0/104119959 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c725e04a70 0x55c725e0c6b0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f5160004860 tx=0x7f5160009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2897fb8181a5771 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:30.946 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.938+0000 7f515d7fa700 1 -- 172.21.15.77:0/104119959 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51600227e0 con 0x55c725e04a70 2023-12-01T01:22:30.946 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.938+0000 7f515d7fa700 1 -- 172.21.15.77:0/104119959 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f516000f050 con 0x55c725e04a70 2023-12-01T01:22:30.946 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.938+0000 7f515d7fa700 1 -- 172.21.15.77:0/104119959 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5160038480 con 0x55c725e04a70 2023-12-01T01:22:30.946 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.939+0000 7f517c93bb80 1 -- 172.21.15.77:0/104119959 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c725e04a70 msgr2=0x55c725e0c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:30.946 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.939+0000 7f517c93bb80 1 --2- 172.21.15.77:0/104119959 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c725e04a70 0x55c725e0c6b0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f5160004860 tx=0x7f5160009c80 comp rx=0 tx=0).stop 2023-12-01T01:22:30.946 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.939+0000 7f517c93bb80 1 -- 172.21.15.77:0/104119959 shutdown_connections 2023-12-01T01:22:30.946 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.939+0000 7f517c93bb80 1 --2- 172.21.15.77:0/104119959 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c725e04a70 0x55c725e0c6b0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.946 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.939+0000 7f517c93bb80 1 --2- 172.21.15.77:0/104119959 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c725e033f0 0x55c725e02e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.946 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.939+0000 7f517c93bb80 1 --2- 172.21.15.77:0/104119959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c725e040f0 0x55c725e11b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.946 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.939+0000 7f517c93bb80 1 -- 172.21.15.77:0/104119959 >> 172.21.15.77:0/104119959 conn(0x55c725d620d0 msgr2=0x55c725e07330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:30.946 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.939+0000 7f517c93bb80 1 -- 172.21.15.77:0/104119959 shutdown_connections 2023-12-01T01:22:30.947 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.939+0000 7f517c93bb80 1 -- 172.21.15.77:0/104119959 wait complete. 2023-12-01T01:22:30.947 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.939+0000 7f517c93bb80 1 -- start start 2023-12-01T01:22:30.947 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.939+0000 7f517c93bb80 1 Processor -- start 2023-12-01T01:22:30.947 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.939+0000 7f517c93bb80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c725e033f0 0x55c725e116a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.947 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.939+0000 7f517c93bb80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c725e040f0 0x55c725e0fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.947 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.939+0000 7f517c93bb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c725e04a70 0x55c725e101f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.947 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.939+0000 7f517c93bb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c725e11ec0 con 0x55c725e04a70 2023-12-01T01:22:30.947 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.940+0000 7f517c93bb80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c725e10730 con 0x55c725e040f0 2023-12-01T01:22:30.947 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.940+0000 7f517c93bb80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c725e108a0 con 0x55c725e033f0 2023-12-01T01:22:30.947 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.940+0000 7f517c93bb80 10 client.0 _ll_register_callbacks cb 0x7f51600167f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:22:30.947 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.940+0000 7f516ed48700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c725e04a70 0x55c725e101f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.947 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.940+0000 7f516ed48700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c725e04a70 0x55c725e101f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33032/0 (socket says 172.21.15.77:33032) 2023-12-01T01:22:30.947 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.940+0000 7f516ed48700 1 -- 172.21.15.77:0/2886775213 learned_addr learned my addr 172.21.15.77:0/2886775213 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:22:30.947 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.940+0000 7f516e547700 1 --2- 172.21.15.77:0/2886775213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c725e033f0 0x55c725e116a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.948 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.940+0000 7f516dd46700 1 --2- 172.21.15.77:0/2886775213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c725e040f0 0x55c725e0fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.948 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.940+0000 7f516ed48700 1 -- 172.21.15.77:0/2886775213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c725e033f0 msgr2=0x55c725e116a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:30.948 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.940+0000 7f516ed48700 1 --2- 172.21.15.77:0/2886775213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c725e033f0 0x55c725e116a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.948 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.940+0000 7f516ed48700 1 -- 172.21.15.77:0/2886775213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c725e040f0 msgr2=0x55c725e0fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:30.948 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.940+0000 7f516ed48700 1 --2- 172.21.15.77:0/2886775213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c725e040f0 0x55c725e0fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.948 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.940+0000 7f516ed48700 1 -- 172.21.15.77:0/2886775213 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c725b6ac60 con 0x55c725e04a70 2023-12-01T01:22:30.948 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.940+0000 7f516ed48700 1 --2- 172.21.15.77:0/2886775213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c725e04a70 0x55c725e101f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5160007fd0 tx=0x7f5160023910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f101bdaf8e9e24a server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:30.948 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.940+0000 7f5151ffb700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:22:30.948 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.940+0000 7f5151ffb700 1 -- 172.21.15.77:0/2886775213 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5160023d40 con 0x55c725e04a70 2023-12-01T01:22:30.948 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.940+0000 7f5151ffb700 1 -- 172.21.15.77:0/2886775213 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f5160004a90 con 0x55c725e04a70 2023-12-01T01:22:30.948 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.940+0000 7f517c93bb80 1 -- 172.21.15.77:0/2886775213 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c725d7ed10 con 0x55c725e04a70 2023-12-01T01:22:30.948 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.940+0000 7f5151ffb700 1 -- 172.21.15.77:0/2886775213 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5160023ea0 con 0x55c725e04a70 2023-12-01T01:22:30.948 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.940+0000 7f517c93bb80 10 client.16287 fetch_fsmap learned FSMap version 41 2023-12-01T01:22:30.948 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.940+0000 7f517c93bb80 1 -- 172.21.15.77:0/2886775213 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55c725b802b0 con 0x55c725e04a70 2023-12-01T01:22:30.948 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.940+0000 7f5151ffb700 1 -- 172.21.15.77:0/2886775213 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c725d7ed10 con 0x55c725e04a70 2023-12-01T01:22:30.949 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.941+0000 7f5151ffb700 1 -- 172.21.15.77:0/2886775213 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 41) v1 ==== 7514+0+0 (secure 0 0 0) 0x7f5160038480 con 0x55c725e04a70 2023-12-01T01:22:30.949 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.941+0000 7f517c93bb80 10 client.16287 fetch_fsmap finished waiting for FSMap version 41 2023-12-01T01:22:30.949 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.941+0000 7f517c93bb80 10 client.16287 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-01T01:22:30.949 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.941+0000 7f517c93bb80 20 client.16287 populate_metadata read hostname 'smithi077' 2023-12-01T01:22:30.949 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.941+0000 7f517c93bb80 1 --2- 172.21.15.77:0/2886775213 >> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] conn(0x55c725e185e0 0x55c725e6d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:30.949 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.941+0000 7f517c93bb80 4 client.16287 mds_command: new command op to 24475 tid=0[{"prefix": "damage ls"}] 2023-12-01T01:22:30.949 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.941+0000 7f517c93bb80 1 -- 172.21.15.77:0/2886775213 --> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c725d4ca60 con 0x55c725e185e0 2023-12-01T01:22:30.949 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.941+0000 7f516e547700 1 --2- 172.21.15.77:0/2886775213 >> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] conn(0x55c725e185e0 0x55c725e6d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:30.949 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.941+0000 7f5151ffb700 10 client.16287 ms_handle_connect on v2:172.21.15.203:6836/2887080977 2023-12-01T01:22:30.949 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.941+0000 7f516e547700 1 --2- 172.21.15.77:0/2886775213 >> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] conn(0x55c725e185e0 0x55c725e6d360 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=915dfebd1e88ba49 server_cookie=d0caa2720bb6b19c in_seq=0 out_seq=0 2023-12-01T01:22:30.949 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.942+0000 7f5151ffb700 1 -- 172.21.15.77:0/2886775213 <== mds.2 v2:172.21.15.203:6836/2887080977 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+8851 (crc 0 0 0) 0x55c725d4ca60 con 0x55c725e185e0 2023-12-01T01:22:30.949 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.942+0000 7f5151ffb700 10 client.16287 handle_command_reply: tid=0 2023-12-01T01:22:30.949 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.942+0000 7f517c93bb80 1 client.16287 shutdown 2023-12-01T01:22:30.949 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.942+0000 7f517c93bb80 1 -- 172.21.15.77:0/2886775213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c725e04a70 msgr2=0x55c725e101f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:30.950 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.942+0000 7f517c93bb80 1 --2- 172.21.15.77:0/2886775213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c725e04a70 0x55c725e101f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5160007fd0 tx=0x7f5160023910 comp rx=0 tx=0).stop 2023-12-01T01:22:30.950 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.942+0000 7f517c93bb80 1 -- 172.21.15.77:0/2886775213 shutdown_connections 2023-12-01T01:22:30.950 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.942+0000 7f517c93bb80 1 --2- 172.21.15.77:0/2886775213 >> [v2:172.21.15.203:6836/2887080977,v1:172.21.15.203:6837/2887080977] conn(0x55c725e185e0 0x55c725e6d360 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.950 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.942+0000 7f517c93bb80 1 --2- 172.21.15.77:0/2886775213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c725e04a70 0x55c725e101f0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.950 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.942+0000 7f517c93bb80 1 --2- 172.21.15.77:0/2886775213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c725e033f0 0x55c725e116a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.950 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.942+0000 7f517c93bb80 1 --2- 172.21.15.77:0/2886775213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c725e040f0 0x55c725e0fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.950 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.942+0000 7f517c93bb80 1 -- 172.21.15.77:0/2886775213 >> 172.21.15.77:0/2886775213 conn(0x55c725d620d0 msgr2=0x55c725e0c650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:30.950 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.942+0000 7f5151ffb700 0 client.16287 ms_handle_reset on v2:172.21.15.203:6836/2887080977 2023-12-01T01:22:30.950 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.942+0000 7f517c93bb80 1 -- 172.21.15.77:0/2886775213 shutdown_connections 2023-12-01T01:22:30.950 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.942+0000 7f517c93bb80 1 -- 172.21.15.77:0/2886775213 wait complete. 2023-12-01T01:22:30.950 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.942+0000 7f517c93bb80 20 client.16287 trim_cache size 0 max 16384 2023-12-01T01:22:30.951 INFO:teuthology.orchestra.run.smithi077.stdout:[ 2023-12-01T01:22:30.951 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:30.951 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:30.951 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 199831713, 2023-12-01T01:22:30.951 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628268, 2023-12-01T01:22:30.951 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/048.out" 2023-12-01T01:22:30.951 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:30.951 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:30.951 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:30.951 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 314450054, 2023-12-01T01:22:30.951 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628249, 2023-12-01T01:22:30.951 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/039" 2023-12-01T01:22:30.951 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:30.951 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:30.952 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:30.952 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 394327943, 2023-12-01T01:22:30.952 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628228, 2023-12-01T01:22:30.952 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/028.out" 2023-12-01T01:22:30.952 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:30.952 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:30.952 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:30.952 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 430370064, 2023-12-01T01:22:30.952 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629834, 2023-12-01T01:22:30.952 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/070.out" 2023-12-01T01:22:30.952 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:30.952 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:30.952 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:30.952 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 449008516, 2023-12-01T01:22:30.953 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629839, 2023-12-01T01:22:30.953 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/073" 2023-12-01T01:22:30.953 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:30.953 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:30.953 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:30.953 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 930141062, 2023-12-01T01:22:30.953 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628173, 2023-12-01T01:22:30.953 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/001" 2023-12-01T01:22:30.953 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:30.953 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:30.953 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:30.953 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 986724124, 2023-12-01T01:22:30.953 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628224, 2023-12-01T01:22:30.953 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/026.out" 2023-12-01T01:22:30.953 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:30.954 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:30.954 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:30.954 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 1073715324, 2023-12-01T01:22:30.954 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628192, 2023-12-01T01:22:30.954 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/010.out" 2023-12-01T01:22:30.954 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:30.954 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:30.954 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:30.954 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 1094384904, 2023-12-01T01:22:30.954 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629821, 2023-12-01T01:22:30.954 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/064" 2023-12-01T01:22:30.954 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:30.954 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:30.954 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:30.955 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 1130381686, 2023-12-01T01:22:30.955 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628236, 2023-12-01T01:22:30.955 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/032.out" 2023-12-01T01:22:30.955 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:30.955 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:30.955 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:30.955 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 1152559890, 2023-12-01T01:22:30.955 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629812, 2023-12-01T01:22:30.955 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/059.out" 2023-12-01T01:22:30.955 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:30.955 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:30.955 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:30.955 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 1160278782, 2023-12-01T01:22:30.955 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629814, 2023-12-01T01:22:30.955 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/060.out" 2023-12-01T01:22:30.956 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:30.956 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:30.956 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:30.956 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 1160732478, 2023-12-01T01:22:30.956 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628257, 2023-12-01T01:22:30.956 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/043" 2023-12-01T01:22:30.956 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:30.956 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:30.956 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:30.956 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 1195723880, 2023-12-01T01:22:30.956 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628245, 2023-12-01T01:22:30.956 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/037" 2023-12-01T01:22:30.956 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:30.956 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:30.956 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:30.957 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 1538021286, 2023-12-01T01:22:30.957 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629844, 2023-12-01T01:22:30.957 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/075.out" 2023-12-01T01:22:30.957 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:30.957 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:30.957 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:30.957 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 1677118401, 2023-12-01T01:22:30.957 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628223, 2023-12-01T01:22:30.957 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/026" 2023-12-01T01:22:30.957 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:30.957 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:30.957 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:30.957 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 1961928516, 2023-12-01T01:22:30.957 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629843, 2023-12-01T01:22:30.957 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/075" 2023-12-01T01:22:30.958 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:30.958 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:30.958 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:30.958 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 1992333326, 2023-12-01T01:22:30.958 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628263, 2023-12-01T01:22:30.958 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/046" 2023-12-01T01:22:30.958 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:30.958 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:30.958 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:30.958 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 2031349638, 2023-12-01T01:22:30.958 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629845, 2023-12-01T01:22:30.958 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/076" 2023-12-01T01:22:30.958 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:30.958 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:30.959 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:30.959 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 2149702231, 2023-12-01T01:22:30.959 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628256, 2023-12-01T01:22:30.959 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/042.out" 2023-12-01T01:22:30.959 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:30.959 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:30.959 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:30.959 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 2273843009, 2023-12-01T01:22:30.959 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628174, 2023-12-01T01:22:30.959 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/001.out" 2023-12-01T01:22:30.959 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:30.959 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:30.959 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:30.959 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 2369318845, 2023-12-01T01:22:30.960 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628197, 2023-12-01T01:22:30.960 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/013" 2023-12-01T01:22:30.960 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:30.960 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:30.960 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:30.960 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 2423172278, 2023-12-01T01:22:30.960 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628276, 2023-12-01T01:22:30.960 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/052.out" 2023-12-01T01:22:30.960 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:30.960 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:30.960 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:30.960 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 2521698041, 2023-12-01T01:22:30.960 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629836, 2023-12-01T01:22:30.960 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/071.out" 2023-12-01T01:22:30.960 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:30.961 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:30.961 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:30.961 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 2714493690, 2023-12-01T01:22:30.961 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628189, 2023-12-01T01:22:30.961 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/009" 2023-12-01T01:22:30.961 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:30.961 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:30.961 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:30.961 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 2853027489, 2023-12-01T01:22:30.961 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628288, 2023-12-01T01:22:30.961 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/056" 2023-12-01T01:22:30.961 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:30.961 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:30.961 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:30.961 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 3061654771, 2023-12-01T01:22:30.962 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628225, 2023-12-01T01:22:30.962 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/027" 2023-12-01T01:22:30.962 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:30.962 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:30.962 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:30.962 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 3378493807, 2023-12-01T01:22:30.962 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628273, 2023-12-01T01:22:30.962 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/051" 2023-12-01T01:22:30.962 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:30.962 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:30.962 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:30.962 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 3680175837, 2023-12-01T01:22:30.962 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628179, 2023-12-01T01:22:30.962 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/004" 2023-12-01T01:22:30.963 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:30.963 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:30.963 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:30.963 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 3772566160, 2023-12-01T01:22:30.963 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628218, 2023-12-01T01:22:30.963 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/023.out" 2023-12-01T01:22:30.963 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:30.963 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:30.963 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:30.963 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 3782971795, 2023-12-01T01:22:30.963 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628200, 2023-12-01T01:22:30.963 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/014.out" 2023-12-01T01:22:30.963 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:30.963 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:30.963 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:30.964 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 3789597338, 2023-12-01T01:22:30.964 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629825, 2023-12-01T01:22:30.964 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/066" 2023-12-01T01:22:30.964 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:30.964 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:30.964 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:30.964 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 3951329273, 2023-12-01T01:22:30.964 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628272, 2023-12-01T01:22:30.964 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/050.out" 2023-12-01T01:22:30.964 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:30.964 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:30.964 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:30.964 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 4060739507, 2023-12-01T01:22:30.964 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628201, 2023-12-01T01:22:30.965 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/015" 2023-12-01T01:22:30.965 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:30.965 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:30.965 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:30.965 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 4061408592, 2023-12-01T01:22:30.965 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629829, 2023-12-01T01:22:30.965 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/068" 2023-12-01T01:22:30.965 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:30.965 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:30.965 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:30.965 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 4070008490, 2023-12-01T01:22:30.965 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628220, 2023-12-01T01:22:30.965 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/024.out" 2023-12-01T01:22:30.965 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:30.965 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:30.966 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:30.966 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 4076756610, 2023-12-01T01:22:30.966 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628238, 2023-12-01T01:22:30.966 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/033.out" 2023-12-01T01:22:30.966 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:30.966 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:30.966 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:30.966 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 4101696094, 2023-12-01T01:22:30.966 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628222, 2023-12-01T01:22:30.966 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/025.out" 2023-12-01T01:22:30.966 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:30.966 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:30.966 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:30.966 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 4270462330, 2023-12-01T01:22:30.967 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628262, 2023-12-01T01:22:30.967 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/045.out" 2023-12-01T01:22:30.967 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:30.967 INFO:teuthology.orchestra.run.smithi077.stdout:] 2023-12-01T01:22:30.968 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.943+0000 7f51707ab700 1 -- 172.21.15.77:0/3542982394 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f51540a0520 msgr2=0x7f51540a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:30.968 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.943+0000 7f51707ab700 1 --2- 172.21.15.77:0/3542982394 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f51540a0520 0x7f51540a29e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55c725e03e90 tx=0x7f5160009040 comp rx=0 tx=0).stop 2023-12-01T01:22:30.968 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.943+0000 7f51707ab700 1 -- 172.21.15.77:0/3542982394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5168100da0 msgr2=0x7f5168196a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:30.968 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.943+0000 7f51707ab700 1 --2- 172.21.15.77:0/3542982394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5168100da0 0x7f5168196a90 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f5158009990 tx=0x7f5158005e80 comp rx=0 tx=0).stop 2023-12-01T01:22:30.968 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.943+0000 7f51707ab700 1 -- 172.21.15.77:0/3542982394 shutdown_connections 2023-12-01T01:22:30.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.943+0000 7f51707ab700 1 --2- 172.21.15.77:0/3542982394 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f51540a0520 0x7f51540a29e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.943+0000 7f51707ab700 1 --2- 172.21.15.77:0/3542982394 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5168102930 0x7f516819e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.943+0000 7f51707ab700 1 --2- 172.21.15.77:0/3542982394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5168100da0 0x7f5168196a90 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.943+0000 7f51707ab700 1 --2- 172.21.15.77:0/3542982394 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5168101f90 0x7f5168196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:30.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.943+0000 7f51707ab700 1 -- 172.21.15.77:0/3542982394 >> 172.21.15.77:0/3542982394 conn(0x7f51680fc310 msgr2=0x7f51680fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:30.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.943+0000 7f51707ab700 1 -- 172.21.15.77:0/3542982394 shutdown_connections 2023-12-01T01:22:30.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:30.943+0000 7f51707ab700 1 -- 172.21.15.77:0/3542982394 wait complete. 2023-12-01T01:22:30.969 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-12-01T01:22:31.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.092+0000 7eff77755700 1 -- 172.21.15.77:0/1439700380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff70101810 msgr2=0x7eff701080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:31.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.092+0000 7eff77755700 1 --2- 172.21.15.77:0/1439700380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff70101810 0x7eff701080c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7eff6800b0d0 tx=0x7eff6800b3e0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.092+0000 7eff77755700 1 -- 172.21.15.77:0/1439700380 shutdown_connections 2023-12-01T01:22:31.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.092+0000 7eff77755700 1 --2- 172.21.15.77:0/1439700380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff70101810 0x7eff701080c0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.092+0000 7eff77755700 1 --2- 172.21.15.77:0/1439700380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff700ffc80 0x7eff701000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.092+0000 7eff77755700 1 --2- 172.21.15.77:0/1439700380 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff70100e70 0x7eff701012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.092+0000 7eff77755700 1 -- 172.21.15.77:0/1439700380 >> 172.21.15.77:0/1439700380 conn(0x7eff700fb1f0 msgr2=0x7eff700fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:31.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.092+0000 7eff77755700 1 -- 172.21.15.77:0/1439700380 shutdown_connections 2023-12-01T01:22:31.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.092+0000 7eff77755700 1 -- 172.21.15.77:0/1439700380 wait complete. 2023-12-01T01:22:31.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.092+0000 7eff77755700 1 Processor -- start 2023-12-01T01:22:31.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.093+0000 7eff77755700 1 -- start start 2023-12-01T01:22:31.096 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.093+0000 7eff77755700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff700ffc80 0x7eff70068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.097 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.093+0000 7eff77755700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff70100e70 0x7eff7019b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.097 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.093+0000 7eff77755700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff70101810 0x7eff7019df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.097 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.093+0000 7eff77755700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7019e6d0 con 0x7eff70101810 2023-12-01T01:22:31.097 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.093+0000 7eff77755700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7019e840 con 0x7eff700ffc80 2023-12-01T01:22:31.097 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.093+0000 7eff77755700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7019ea00 con 0x7eff70100e70 2023-12-01T01:22:31.097 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.093+0000 7eff75cf2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff70101810 0x7eff7019df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:31.097 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.093+0000 7eff75cf2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff70101810 0x7eff7019df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33046/0 (socket says 172.21.15.77:33046) 2023-12-01T01:22:31.097 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.093+0000 7eff75cf2700 1 -- 172.21.15.77:0/343880724 learned_addr learned my addr 172.21.15.77:0/343880724 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:22:31.097 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.093+0000 7eff754f1700 1 --2- 172.21.15.77:0/343880724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff700ffc80 0x7eff70068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:31.097 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.093+0000 7eff74cf0700 1 --2- 172.21.15.77:0/343880724 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff70100e70 0x7eff7019b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:31.097 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.093+0000 7eff75cf2700 1 -- 172.21.15.77:0/343880724 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff70100e70 msgr2=0x7eff7019b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:31.097 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.093+0000 7eff75cf2700 1 --2- 172.21.15.77:0/343880724 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff70100e70 0x7eff7019b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.097 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.093+0000 7eff75cf2700 1 -- 172.21.15.77:0/343880724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff700ffc80 msgr2=0x7eff70068a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:31.097 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.093+0000 7eff75cf2700 1 --2- 172.21.15.77:0/343880724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff700ffc80 0x7eff70068a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.098 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.093+0000 7eff75cf2700 1 -- 172.21.15.77:0/343880724 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff68009d00 con 0x7eff70101810 2023-12-01T01:22:31.098 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.093+0000 7eff75cf2700 1 --2- 172.21.15.77:0/343880724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff70101810 0x7eff7019df00 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7eff68007a00 tx=0x7eff680094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:31.098 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.093+0000 7eff667fc700 1 -- 172.21.15.77:0/343880724 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6800e040 con 0x7eff70101810 2023-12-01T01:22:31.098 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.093+0000 7eff667fc700 1 -- 172.21.15.77:0/343880724 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff68003e10 con 0x7eff70101810 2023-12-01T01:22:31.098 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.093+0000 7eff667fc700 1 -- 172.21.15.77:0/343880724 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6801b9e0 con 0x7eff70101810 2023-12-01T01:22:31.098 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.093+0000 7eff77755700 1 -- 172.21.15.77:0/343880724 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff7019ec80 con 0x7eff70101810 2023-12-01T01:22:31.098 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.094+0000 7eff77755700 1 -- 172.21.15.77:0/343880724 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff7019f050 con 0x7eff70101810 2023-12-01T01:22:31.104 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.096+0000 7eff667fc700 1 -- 172.21.15.77:0/343880724 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7eff68005630 con 0x7eff70101810 2023-12-01T01:22:31.104 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.096+0000 7eff667fc700 1 --2- 172.21.15.77:0/343880724 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7eff5c0a0520 0x7eff5c0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.104 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.096+0000 7eff667fc700 1 -- 172.21.15.77:0/343880724 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7eff680ccee0 con 0x7eff70101810 2023-12-01T01:22:31.104 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.096+0000 7eff754f1700 1 --2- 172.21.15.77:0/343880724 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7eff5c0a0520 0x7eff5c0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:31.104 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.097+0000 7eff838e5b80 1 Processor -- start 2023-12-01T01:22:31.104 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.097+0000 7eff838e5b80 1 -- start start 2023-12-01T01:22:31.104 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.097+0000 7eff754f1700 1 --2- 172.21.15.77:0/343880724 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7eff5c0a0520 0x7eff5c0a29e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7eff70100cd0 tx=0x7eff6000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:31.104 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.097+0000 7eff838e5b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ffef2db0f0 0x55ffef2db530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.104 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.097+0000 7eff838e5b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffef2dba70 0x55ffef2d9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.105 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.097+0000 7eff838e5b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffef2da3f0 0x55ffef2da870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.105 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.097+0000 7eff838e5b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffef231a10 con 0x55ffef2da3f0 2023-12-01T01:22:31.105 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.097+0000 7eff838e5b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffef22f3d0 con 0x55ffef2dba70 2023-12-01T01:22:31.105 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.097+0000 7eff838e5b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffef22acc0 con 0x55ffef2db0f0 2023-12-01T01:22:31.105 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.097+0000 7eff75cf2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffef2da3f0 0x55ffef2da870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:31.105 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.097+0000 7eff75cf2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffef2da3f0 0x55ffef2da870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33054/0 (socket says 172.21.15.77:33054) 2023-12-01T01:22:31.105 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.097+0000 7eff75cf2700 1 -- 172.21.15.77:0/2146678436 learned_addr learned my addr 172.21.15.77:0/2146678436 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:22:31.105 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.098+0000 7eff754f1700 1 --2- 172.21.15.77:0/2146678436 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ffef2db0f0 0x55ffef2db530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:31.105 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.098+0000 7eff74cf0700 1 --2- 172.21.15.77:0/2146678436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffef2dba70 0x55ffef2d9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:31.105 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.098+0000 7eff75cf2700 1 -- 172.21.15.77:0/2146678436 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ffef2db0f0 msgr2=0x55ffef2db530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:31.105 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.098+0000 7eff75cf2700 1 --2- 172.21.15.77:0/2146678436 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ffef2db0f0 0x55ffef2db530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.105 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.098+0000 7eff75cf2700 1 -- 172.21.15.77:0/2146678436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffef2dba70 msgr2=0x55ffef2d9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:31.106 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.098+0000 7eff75cf2700 1 --2- 172.21.15.77:0/2146678436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffef2dba70 0x55ffef2d9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.106 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.098+0000 7eff75cf2700 1 -- 172.21.15.77:0/2146678436 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffef058de0 con 0x55ffef2da3f0 2023-12-01T01:22:31.106 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.098+0000 7eff74cf0700 1 --2- 172.21.15.77:0/2146678436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffef2dba70 0x55ffef2d9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:22:31.107 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.098+0000 7eff75cf2700 1 --2- 172.21.15.77:0/2146678436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffef2da3f0 0x55ffef2da870 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7eff68096d70 tx=0x7eff6801b470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bacfe8c7643a2cfb server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:31.107 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.098+0000 7eff5b7fe700 1 -- 172.21.15.77:0/2146678436 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff680caea0 con 0x55ffef2da3f0 2023-12-01T01:22:31.107 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.098+0000 7eff5b7fe700 1 -- 172.21.15.77:0/2146678436 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff6809af70 con 0x55ffef2da3f0 2023-12-01T01:22:31.107 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.098+0000 7eff5b7fe700 1 -- 172.21.15.77:0/2146678436 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6809ad30 con 0x55ffef2da3f0 2023-12-01T01:22:31.107 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.099+0000 7eff838e5b80 1 -- 172.21.15.77:0/2146678436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffef2da3f0 msgr2=0x55ffef2da870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:31.107 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.099+0000 7eff838e5b80 1 --2- 172.21.15.77:0/2146678436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffef2da3f0 0x55ffef2da870 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7eff68096d70 tx=0x7eff6801b470 comp rx=0 tx=0).stop 2023-12-01T01:22:31.107 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.099+0000 7eff838e5b80 1 -- 172.21.15.77:0/2146678436 shutdown_connections 2023-12-01T01:22:31.107 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.099+0000 7eff838e5b80 1 --2- 172.21.15.77:0/2146678436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffef2da3f0 0x55ffef2da870 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.107 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.099+0000 7eff838e5b80 1 --2- 172.21.15.77:0/2146678436 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ffef2db0f0 0x55ffef2db530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.108 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.099+0000 7eff838e5b80 1 --2- 172.21.15.77:0/2146678436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffef2dba70 0x55ffef2d9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.108 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.099+0000 7eff838e5b80 1 -- 172.21.15.77:0/2146678436 >> 172.21.15.77:0/2146678436 conn(0x55ffef2390d0 msgr2=0x55ffef2d4940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:31.108 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.099+0000 7eff838e5b80 1 -- 172.21.15.77:0/2146678436 shutdown_connections 2023-12-01T01:22:31.108 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.099+0000 7eff838e5b80 1 -- 172.21.15.77:0/2146678436 wait complete. 2023-12-01T01:22:31.108 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.099+0000 7eff838e5b80 1 -- start start 2023-12-01T01:22:31.108 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.099+0000 7eff838e5b80 1 Processor -- start 2023-12-01T01:22:31.108 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.099+0000 7eff838e5b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffef2da3f0 0x55ffef2e3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.108 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.099+0000 7eff838e5b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ffef2db0f0 0x55ffef2e3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.108 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.099+0000 7eff838e5b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffef2dba70 0x55ffef2eacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.108 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.100+0000 7eff838e5b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffef1c7510 con 0x55ffef2da3f0 2023-12-01T01:22:31.108 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.100+0000 7eff838e5b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffef1c71b0 con 0x55ffef2dba70 2023-12-01T01:22:31.108 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.100+0000 7eff838e5b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffef0ae920 con 0x55ffef2db0f0 2023-12-01T01:22:31.108 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.100+0000 7eff838e5b80 10 client.0 _ll_register_callbacks cb 0x55ffef114a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:22:31.108 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.100+0000 7eff754f1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffef2da3f0 0x55ffef2e3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:31.109 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.100+0000 7eff754f1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffef2da3f0 0x55ffef2e3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33056/0 (socket says 172.21.15.77:33056) 2023-12-01T01:22:31.109 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.100+0000 7eff754f1700 1 -- 172.21.15.77:0/2133896012 learned_addr learned my addr 172.21.15.77:0/2133896012 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:22:31.109 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.100+0000 7eff75cf2700 1 --2- 172.21.15.77:0/2133896012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffef2dba70 0x55ffef2eacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:31.109 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.100+0000 7eff74cf0700 1 --2- 172.21.15.77:0/2133896012 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ffef2db0f0 0x55ffef2e3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:31.109 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.100+0000 7eff754f1700 1 -- 172.21.15.77:0/2133896012 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ffef2db0f0 msgr2=0x55ffef2e3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:31.109 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.100+0000 7eff754f1700 1 --2- 172.21.15.77:0/2133896012 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ffef2db0f0 0x55ffef2e3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.109 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.100+0000 7eff754f1700 1 -- 172.21.15.77:0/2133896012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffef2dba70 msgr2=0x55ffef2eacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:31.109 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.100+0000 7eff754f1700 1 --2- 172.21.15.77:0/2133896012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffef2dba70 0x55ffef2eacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.109 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.100+0000 7eff754f1700 1 -- 172.21.15.77:0/2133896012 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff7019f050 con 0x55ffef2da3f0 2023-12-01T01:22:31.109 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.100+0000 7eff74cf0700 1 --2- 172.21.15.77:0/2133896012 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ffef2db0f0 0x55ffef2e3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:22:31.109 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.100+0000 7eff754f1700 1 --2- 172.21.15.77:0/2133896012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffef2da3f0 0x55ffef2e3740 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7eff600116f0 tx=0x7eff600047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7312c961e07d1a9c server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:31.109 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.100+0000 7eff58ff9700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:22:31.109 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.100+0000 7eff58ff9700 1 -- 172.21.15.77:0/2133896012 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6001f840 con 0x55ffef2da3f0 2023-12-01T01:22:31.109 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.100+0000 7eff58ff9700 1 -- 172.21.15.77:0/2133896012 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7eff60017070 con 0x55ffef2da3f0 2023-12-01T01:22:31.109 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.100+0000 7eff838e5b80 1 -- 172.21.15.77:0/2133896012 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ffef255d10 con 0x55ffef2da3f0 2023-12-01T01:22:31.110 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.100+0000 7eff58ff9700 1 -- 172.21.15.77:0/2133896012 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff60010e80 con 0x55ffef2da3f0 2023-12-01T01:22:31.110 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.100+0000 7eff838e5b80 10 client.16299 fetch_fsmap learned FSMap version 41 2023-12-01T01:22:31.110 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.100+0000 7eff838e5b80 1 -- 172.21.15.77:0/2133896012 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55ffef041c60 con 0x55ffef2da3f0 2023-12-01T01:22:31.110 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.101+0000 7eff58ff9700 1 -- 172.21.15.77:0/2133896012 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ffef255d10 con 0x55ffef2da3f0 2023-12-01T01:22:31.110 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.101+0000 7eff58ff9700 1 -- 172.21.15.77:0/2133896012 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 41) v1 ==== 7514+0+0 (secure 0 0 0) 0x7eff60028b20 con 0x55ffef2da3f0 2023-12-01T01:22:31.110 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.101+0000 7eff838e5b80 10 client.16299 fetch_fsmap finished waiting for FSMap version 41 2023-12-01T01:22:31.110 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.101+0000 7eff838e5b80 10 client.16299 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-12-01T01:22:31.110 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.101+0000 7eff838e5b80 20 client.16299 populate_metadata read hostname 'smithi077' 2023-12-01T01:22:31.110 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.101+0000 7eff838e5b80 1 --2- 172.21.15.77:0/2133896012 >> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] conn(0x55ffef2ef4f0 0x55ffef2f1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.110 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.101+0000 7eff838e5b80 4 client.16299 mds_command: new command op to 24496 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:22:31.110 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.101+0000 7eff838e5b80 1 -- 172.21.15.77:0/2133896012 --> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ffef2a8e10 con 0x55ffef2ef4f0 2023-12-01T01:22:31.110 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.101+0000 7eff74cf0700 1 --2- 172.21.15.77:0/2133896012 >> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] conn(0x55ffef2ef4f0 0x55ffef2f1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:31.110 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.101+0000 7eff58ff9700 10 client.16299 ms_handle_connect on v2:172.21.15.203:6838/2888221613 2023-12-01T01:22:31.110 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.101+0000 7eff74cf0700 1 --2- 172.21.15.77:0/2133896012 >> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] conn(0x55ffef2ef4f0 0x55ffef2f1910 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=43798a40b2e243f4 server_cookie=682d83cb679aee6e in_seq=0 out_seq=0 2023-12-01T01:22:31.111 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.102+0000 7eff58ff9700 1 -- 172.21.15.77:0/2133896012 <== mds.1 v2:172.21.15.203:6838/2888221613 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55ffef2a8e10 con 0x55ffef2ef4f0 2023-12-01T01:22:31.111 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.102+0000 7eff58ff9700 10 client.16299 handle_command_reply: tid=0 2023-12-01T01:22:31.111 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.102+0000 7eff838e5b80 1 client.16299 shutdown 2023-12-01T01:22:31.111 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.102+0000 7eff838e5b80 1 -- 172.21.15.77:0/2133896012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffef2da3f0 msgr2=0x55ffef2e3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:31.111 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.102+0000 7eff838e5b80 1 --2- 172.21.15.77:0/2133896012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffef2da3f0 0x55ffef2e3740 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7eff600116f0 tx=0x7eff600047c0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.111 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.102+0000 7eff838e5b80 1 -- 172.21.15.77:0/2133896012 shutdown_connections 2023-12-01T01:22:31.111 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.102+0000 7eff838e5b80 1 --2- 172.21.15.77:0/2133896012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffef2dba70 0x55ffef2eacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.111 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.102+0000 7eff838e5b80 1 --2- 172.21.15.77:0/2133896012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffef2da3f0 0x55ffef2e3740 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.111 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.102+0000 7eff838e5b80 1 --2- 172.21.15.77:0/2133896012 >> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] conn(0x55ffef2ef4f0 0x55ffef2f1910 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.111 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.102+0000 7eff838e5b80 1 --2- 172.21.15.77:0/2133896012 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ffef2db0f0 0x55ffef2e3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.111 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.102+0000 7eff58ff9700 0 client.16299 ms_handle_reset on v2:172.21.15.203:6838/2888221613 2023-12-01T01:22:31.111 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.102+0000 7eff838e5b80 1 -- 172.21.15.77:0/2133896012 >> 172.21.15.77:0/2133896012 conn(0x55ffef2390d0 msgr2=0x55ffef2d40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:31.111 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.102+0000 7eff838e5b80 1 -- 172.21.15.77:0/2133896012 shutdown_connections 2023-12-01T01:22:31.111 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.102+0000 7eff838e5b80 1 -- 172.21.15.77:0/2133896012 wait complete. 2023-12-01T01:22:31.111 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.102+0000 7eff838e5b80 20 client.16299 trim_cache size 0 max 16384 2023-12-01T01:22:31.123 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.115+0000 7eff838e5b80 1 Processor -- start 2023-12-01T01:22:31.124 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.115+0000 7eff838e5b80 1 -- start start 2023-12-01T01:22:31.124 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.115+0000 7eff838e5b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffef2da3f0 0x55ffef2d9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.124 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.115+0000 7eff838e5b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffef2db0f0 0x55ffef2e8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.124 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.115+0000 7eff838e5b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ffef2dba70 0x55ffef2e2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.124 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.115+0000 7eff838e5b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffef2d6170 con 0x55ffef2da3f0 2023-12-01T01:22:31.124 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.115+0000 7eff838e5b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffef2e14d0 con 0x55ffef2db0f0 2023-12-01T01:22:31.124 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.115+0000 7eff838e5b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffef2ec8a0 con 0x55ffef2dba70 2023-12-01T01:22:31.124 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.115+0000 7eff754f1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffef2da3f0 0x55ffef2d9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:31.124 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.115+0000 7eff754f1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffef2da3f0 0x55ffef2d9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33064/0 (socket says 172.21.15.77:33064) 2023-12-01T01:22:31.125 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.115+0000 7eff754f1700 1 -- 172.21.15.77:0/1955096448 learned_addr learned my addr 172.21.15.77:0/1955096448 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:22:31.125 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.115+0000 7eff74cf0700 1 --2- 172.21.15.77:0/1955096448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffef2db0f0 0x55ffef2e8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:31.125 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.115+0000 7eff754f1700 1 -- 172.21.15.77:0/1955096448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ffef2dba70 msgr2=0x55ffef2e2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:31.125 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.115+0000 7eff754f1700 1 --2- 172.21.15.77:0/1955096448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ffef2dba70 0x55ffef2e2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.125 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.115+0000 7eff754f1700 1 -- 172.21.15.77:0/1955096448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffef2db0f0 msgr2=0x55ffef2e8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:31.125 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.115+0000 7eff754f1700 1 --2- 172.21.15.77:0/1955096448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffef2db0f0 0x55ffef2e8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.125 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.115+0000 7eff754f1700 1 -- 172.21.15.77:0/1955096448 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffef07f250 con 0x55ffef2da3f0 2023-12-01T01:22:31.125 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.115+0000 7eff74cf0700 1 --2- 172.21.15.77:0/1955096448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffef2db0f0 0x55ffef2e8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:22:31.125 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.115+0000 7eff754f1700 1 --2- 172.21.15.77:0/1955096448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffef2da3f0 0x55ffef2d9e60 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7eff6001e020 tx=0x7eff60010af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75ba311fcef75441 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:31.125 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.116+0000 7eff5bfff700 1 -- 172.21.15.77:0/1955096448 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6001e990 con 0x55ffef2da3f0 2023-12-01T01:22:31.125 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.116+0000 7eff5bfff700 1 -- 172.21.15.77:0/1955096448 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff60015070 con 0x55ffef2da3f0 2023-12-01T01:22:31.125 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.116+0000 7eff5bfff700 1 -- 172.21.15.77:0/1955096448 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6001f9c0 con 0x55ffef2da3f0 2023-12-01T01:22:31.126 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.116+0000 7eff838e5b80 1 -- 172.21.15.77:0/1955096448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffef2da3f0 msgr2=0x55ffef2d9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:31.126 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.116+0000 7eff838e5b80 1 --2- 172.21.15.77:0/1955096448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffef2da3f0 0x55ffef2d9e60 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7eff6001e020 tx=0x7eff60010af0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.126 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.116+0000 7eff838e5b80 1 -- 172.21.15.77:0/1955096448 shutdown_connections 2023-12-01T01:22:31.126 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.116+0000 7eff838e5b80 1 --2- 172.21.15.77:0/1955096448 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ffef2dba70 0x55ffef2e2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.126 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.116+0000 7eff838e5b80 1 --2- 172.21.15.77:0/1955096448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffef2da3f0 0x55ffef2d9e60 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.126 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.116+0000 7eff838e5b80 1 --2- 172.21.15.77:0/1955096448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffef2db0f0 0x55ffef2e8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.126 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.116+0000 7eff838e5b80 1 -- 172.21.15.77:0/1955096448 >> 172.21.15.77:0/1955096448 conn(0x55ffef2390d0 msgr2=0x55ffef2d47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:31.126 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.116+0000 7eff838e5b80 1 -- 172.21.15.77:0/1955096448 shutdown_connections 2023-12-01T01:22:31.126 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.116+0000 7eff838e5b80 1 -- 172.21.15.77:0/1955096448 wait complete. 2023-12-01T01:22:31.126 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.117+0000 7eff838e5b80 1 -- start start 2023-12-01T01:22:31.126 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.117+0000 7eff838e5b80 1 Processor -- start 2023-12-01T01:22:31.126 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.117+0000 7eff838e5b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ffef2da3f0 0x55ffef2e8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.126 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.117+0000 7eff838e5b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffef2db0f0 0x55ffef2e8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.126 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.117+0000 7eff838e5b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffef2dba70 0x55ffef2e6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.127 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.117+0000 7eff838e5b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffef2e7560 con 0x55ffef2dba70 2023-12-01T01:22:31.127 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.117+0000 7eff838e5b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffef2e76d0 con 0x55ffef2db0f0 2023-12-01T01:22:31.127 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.117+0000 7eff838e5b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffef2e7890 con 0x55ffef2da3f0 2023-12-01T01:22:31.127 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.117+0000 7eff838e5b80 10 client.0 _ll_register_callbacks cb 0x7eff60010c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:22:31.127 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.117+0000 7eff75cf2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffef2dba70 0x55ffef2e6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:31.127 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.117+0000 7eff75cf2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffef2dba70 0x55ffef2e6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33074/0 (socket says 172.21.15.77:33074) 2023-12-01T01:22:31.127 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.117+0000 7eff75cf2700 1 -- 172.21.15.77:0/2628880279 learned_addr learned my addr 172.21.15.77:0/2628880279 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:22:31.127 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.117+0000 7eff754f1700 1 --2- 172.21.15.77:0/2628880279 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ffef2da3f0 0x55ffef2e8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:31.127 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.117+0000 7eff75cf2700 1 -- 172.21.15.77:0/2628880279 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ffef2da3f0 msgr2=0x55ffef2e8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:31.127 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.117+0000 7eff75cf2700 1 --2- 172.21.15.77:0/2628880279 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ffef2da3f0 0x55ffef2e8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.127 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.117+0000 7eff75cf2700 1 -- 172.21.15.77:0/2628880279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffef2db0f0 msgr2=0x55ffef2e8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:31.127 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.117+0000 7eff75cf2700 1 --2- 172.21.15.77:0/2628880279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffef2db0f0 0x55ffef2e8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.127 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.117+0000 7eff75cf2700 1 -- 172.21.15.77:0/2628880279 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ffef041c60 con 0x55ffef2dba70 2023-12-01T01:22:31.128 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.117+0000 7eff754f1700 1 --2- 172.21.15.77:0/2628880279 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ffef2da3f0 0x55ffef2e8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:22:31.128 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.117+0000 7eff75cf2700 1 --2- 172.21.15.77:0/2628880279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffef2dba70 0x55ffef2e6e80 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7eff6809a570 tx=0x7eff6809a3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a4c9e5e4f328ca2 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:31.128 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.117+0000 7eff58ff9700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:22:31.128 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.118+0000 7eff58ff9700 1 -- 172.21.15.77:0/2628880279 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff680caea0 con 0x55ffef2dba70 2023-12-01T01:22:31.128 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.118+0000 7eff58ff9700 1 -- 172.21.15.77:0/2628880279 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7eff6809a870 con 0x55ffef2dba70 2023-12-01T01:22:31.128 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.118+0000 7eff838e5b80 1 -- 172.21.15.77:0/2628880279 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff6809af70 con 0x55ffef2dba70 2023-12-01T01:22:31.128 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.118+0000 7eff58ff9700 1 -- 172.21.15.77:0/2628880279 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff680a2850 con 0x55ffef2dba70 2023-12-01T01:22:31.128 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.118+0000 7eff838e5b80 10 client.16305 fetch_fsmap learned FSMap version 41 2023-12-01T01:22:31.128 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.118+0000 7eff838e5b80 1 -- 172.21.15.77:0/2628880279 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55ffef0572b0 con 0x55ffef2dba70 2023-12-01T01:22:31.128 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.118+0000 7eff58ff9700 1 -- 172.21.15.77:0/2628880279 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff6809af70 con 0x55ffef2dba70 2023-12-01T01:22:31.129 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.118+0000 7eff58ff9700 1 -- 172.21.15.77:0/2628880279 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 41) v1 ==== 7514+0+0 (secure 0 0 0) 0x7eff68060b80 con 0x55ffef2dba70 2023-12-01T01:22:31.129 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.118+0000 7eff838e5b80 10 client.16305 fetch_fsmap finished waiting for FSMap version 41 2023-12-01T01:22:31.129 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.118+0000 7eff838e5b80 10 client.16305 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-12-01T01:22:31.129 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.118+0000 7eff838e5b80 20 client.16305 populate_metadata read hostname 'smithi077' 2023-12-01T01:22:31.129 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.118+0000 7eff838e5b80 1 --2- 172.21.15.77:0/2628880279 >> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] conn(0x55ffef342160 0x55ffef344580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.129 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.118+0000 7eff838e5b80 4 client.16305 mds_command: new command op to 24496 tid=0[{"prefix": "damage ls"}] 2023-12-01T01:22:31.129 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.118+0000 7eff838e5b80 1 -- 172.21.15.77:0/2628880279 --> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ffef223d40 con 0x55ffef342160 2023-12-01T01:22:31.129 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.118+0000 7eff754f1700 1 --2- 172.21.15.77:0/2628880279 >> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] conn(0x55ffef342160 0x55ffef344580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:31.129 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.119+0000 7eff754f1700 1 --2- 172.21.15.77:0/2628880279 >> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] conn(0x55ffef342160 0x55ffef344580 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4c74e8426ca67fbf server_cookie=1a43d5684dd0864d in_seq=0 out_seq=0 2023-12-01T01:22:31.129 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.119+0000 7eff58ff9700 10 client.16305 ms_handle_connect on v2:172.21.15.203:6838/2888221613 2023-12-01T01:22:31.129 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.120+0000 7eff58ff9700 1 -- 172.21.15.77:0/2628880279 <== mds.1 v2:172.21.15.203:6838/2888221613 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18408 (crc 0 0 0) 0x55ffef223d40 con 0x55ffef342160 2023-12-01T01:22:31.129 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.120+0000 7eff58ff9700 10 client.16305 handle_command_reply: tid=0 2023-12-01T01:22:31.129 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.120+0000 7eff838e5b80 1 client.16305 shutdown 2023-12-01T01:22:31.130 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.120+0000 7eff838e5b80 1 -- 172.21.15.77:0/2628880279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffef2dba70 msgr2=0x55ffef2e6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:31.130 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.120+0000 7eff838e5b80 1 --2- 172.21.15.77:0/2628880279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffef2dba70 0x55ffef2e6e80 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7eff6809a570 tx=0x7eff6809a3b0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.130 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.120+0000 7eff838e5b80 1 -- 172.21.15.77:0/2628880279 shutdown_connections 2023-12-01T01:22:31.130 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.120+0000 7eff838e5b80 1 --2- 172.21.15.77:0/2628880279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffef2dba70 0x55ffef2e6e80 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.130 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.120+0000 7eff838e5b80 1 --2- 172.21.15.77:0/2628880279 >> [v2:172.21.15.203:6838/2888221613,v1:172.21.15.203:6839/2888221613] conn(0x55ffef342160 0x55ffef344580 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.130 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.120+0000 7eff838e5b80 1 --2- 172.21.15.77:0/2628880279 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ffef2da3f0 0x55ffef2e8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.130 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.120+0000 7eff838e5b80 1 --2- 172.21.15.77:0/2628880279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ffef2db0f0 0x55ffef2e8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.130 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.120+0000 7eff58ff9700 0 client.16305 ms_handle_reset on v2:172.21.15.203:6838/2888221613 2023-12-01T01:22:31.130 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.120+0000 7eff838e5b80 1 -- 172.21.15.77:0/2628880279 >> 172.21.15.77:0/2628880279 conn(0x55ffef2390d0 msgr2=0x55ffef2de270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:31.130 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.120+0000 7eff838e5b80 1 -- 172.21.15.77:0/2628880279 shutdown_connections 2023-12-01T01:22:31.130 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.120+0000 7eff838e5b80 1 -- 172.21.15.77:0/2628880279 wait complete. 2023-12-01T01:22:31.130 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.120+0000 7eff838e5b80 20 client.16305 trim_cache size 0 max 16384 2023-12-01T01:22:31.131 INFO:teuthology.orchestra.run.smithi077.stdout:[ 2023-12-01T01:22:31.131 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.131 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.131 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 14572913, 2023-12-01T01:22:31.131 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629837, 2023-12-01T01:22:31.131 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/072" 2023-12-01T01:22:31.131 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.131 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.131 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.132 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 203980746, 2023-12-01T01:22:31.132 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629791, 2023-12-01T01:22:31.132 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/prepare-commit-msg.sample" 2023-12-01T01:22:31.132 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.132 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.132 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.132 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 211808904, 2023-12-01T01:22:31.132 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628193, 2023-12-01T01:22:31.132 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/011" 2023-12-01T01:22:31.132 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.132 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.132 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.133 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 235331874, 2023-12-01T01:22:31.133 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629816, 2023-12-01T01:22:31.133 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/061.out" 2023-12-01T01:22:31.133 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.133 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.133 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.133 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 263176013, 2023-12-01T01:22:31.133 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511627811, 2023-12-01T01:22:31.133 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/HEAD" 2023-12-01T01:22:31.133 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.133 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.133 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.133 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 278320937, 2023-12-01T01:22:31.134 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629803, 2023-12-01T01:22:31.134 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/push-to-checkout.sample" 2023-12-01T01:22:31.134 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.134 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.134 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.134 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 293252552, 2023-12-01T01:22:31.134 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629809, 2023-12-01T01:22:31.134 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/058" 2023-12-01T01:22:31.134 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.134 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.134 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.134 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 352602362, 2023-12-01T01:22:31.135 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629818, 2023-12-01T01:22:31.135 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/062.out" 2023-12-01T01:22:31.135 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.135 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.135 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.135 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 410053489, 2023-12-01T01:22:31.135 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629802, 2023-12-01T01:22:31.135 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/pre-receive.sample" 2023-12-01T01:22:31.135 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.135 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.135 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.135 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 436778939, 2023-12-01T01:22:31.135 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628289, 2023-12-01T01:22:31.135 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/056.out" 2023-12-01T01:22:31.135 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.136 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.137 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.137 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 477235739, 2023-12-01T01:22:31.137 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628278, 2023-12-01T01:22:31.137 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/053.out" 2023-12-01T01:22:31.137 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.137 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.137 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.137 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 555654069, 2023-12-01T01:22:31.137 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628186, 2023-12-01T01:22:31.138 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/007.out" 2023-12-01T01:22:31.138 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.138 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.138 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.138 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 557636540, 2023-12-01T01:22:31.138 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629799, 2023-12-01T01:22:31.138 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/fsmonitor-watchman.sample" 2023-12-01T01:22:31.138 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.138 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.138 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.138 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 658139213, 2023-12-01T01:22:31.138 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629826, 2023-12-01T01:22:31.139 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/066.out" 2023-12-01T01:22:31.139 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.139 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.139 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.139 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 660664312, 2023-12-01T01:22:31.139 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628209, 2023-12-01T01:22:31.139 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/019" 2023-12-01T01:22:31.139 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.139 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.139 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.139 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 675523175, 2023-12-01T01:22:31.139 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628215, 2023-12-01T01:22:31.140 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/022" 2023-12-01T01:22:31.140 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.140 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.140 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.140 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 734519730, 2023-12-01T01:22:31.140 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511627815, 2023-12-01T01:22:31.141 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/config" 2023-12-01T01:22:31.141 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.141 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.141 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.141 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 774827360, 2023-12-01T01:22:31.141 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628187, 2023-12-01T01:22:31.141 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/008" 2023-12-01T01:22:31.141 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.141 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.141 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.142 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 836908857, 2023-12-01T01:22:31.142 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629800, 2023-12-01T01:22:31.142 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/pre-applypatch.sample" 2023-12-01T01:22:31.142 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.142 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.142 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.142 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 841532776, 2023-12-01T01:22:31.142 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628279, 2023-12-01T01:22:31.142 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/054" 2023-12-01T01:22:31.142 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.143 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.143 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.143 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 874423599, 2023-12-01T01:22:31.143 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629806, 2023-12-01T01:22:31.143 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/refs/tags" 2023-12-01T01:22:31.143 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.143 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.143 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.143 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 1007783187, 2023-12-01T01:22:31.143 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629287, 2023-12-01T01:22:31.143 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/description" 2023-12-01T01:22:31.143 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.143 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.143 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.144 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 1025184374, 2023-12-01T01:22:31.144 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511627786, 2023-12-01T01:22:31.144 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/objects" 2023-12-01T01:22:31.144 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.144 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.144 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.144 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 1061002300, 2023-12-01T01:22:31.144 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629286, 2023-12-01T01:22:31.144 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/branches" 2023-12-01T01:22:31.144 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.144 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.145 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.145 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 1064816424, 2023-12-01T01:22:31.145 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628221, 2023-12-01T01:22:31.145 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/025" 2023-12-01T01:22:31.145 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.145 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.145 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.145 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 1067250946, 2023-12-01T01:22:31.146 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511627800, 2023-12-01T01:22:31.146 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/refs/remotes" 2023-12-01T01:22:31.146 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.146 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.146 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.146 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 1104971984, 2023-12-01T01:22:31.146 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628191, 2023-12-01T01:22:31.146 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/010" 2023-12-01T01:22:31.146 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.146 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.146 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.146 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 1119684513, 2023-12-01T01:22:31.147 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629792, 2023-12-01T01:22:31.147 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/pre-push.sample" 2023-12-01T01:22:31.147 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.147 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.147 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.147 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 1124264560, 2023-12-01T01:22:31.147 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511627797, 2023-12-01T01:22:31.147 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/shallow" 2023-12-01T01:22:31.147 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.147 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.147 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.147 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 1242214721, 2023-12-01T01:22:31.147 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629797, 2023-12-01T01:22:31.148 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/post-update.sample" 2023-12-01T01:22:31.148 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.148 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.148 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.148 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 1244434151, 2023-12-01T01:22:31.148 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629827, 2023-12-01T01:22:31.148 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/067" 2023-12-01T01:22:31.148 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.148 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.148 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.148 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 1259009207, 2023-12-01T01:22:31.149 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511627803, 2023-12-01T01:22:31.149 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs" 2023-12-01T01:22:31.149 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.149 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.149 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.149 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 1422637408, 2023-12-01T01:22:31.149 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628261, 2023-12-01T01:22:31.149 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/045" 2023-12-01T01:22:31.149 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.149 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.149 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.150 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 1428076455, 2023-12-01T01:22:31.150 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629804, 2023-12-01T01:22:31.150 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/refs" 2023-12-01T01:22:31.150 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.150 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.150 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.150 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 1695969736, 2023-12-01T01:22:31.150 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629841, 2023-12-01T01:22:31.151 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/074" 2023-12-01T01:22:31.151 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.151 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.151 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.151 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 1761071705, 2023-12-01T01:22:31.151 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629795, 2023-12-01T01:22:31.151 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/commit-msg.sample" 2023-12-01T01:22:31.151 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.151 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.152 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.152 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 1805161389, 2023-12-01T01:22:31.152 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629796, 2023-12-01T01:22:31.152 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/pre-rebase.sample" 2023-12-01T01:22:31.152 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.152 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.152 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.152 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 1877675049, 2023-12-01T01:22:31.152 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628176, 2023-12-01T01:22:31.152 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/002.out" 2023-12-01T01:22:31.152 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.152 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.153 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.153 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 1890627858, 2023-12-01T01:22:31.153 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511627802, 2023-12-01T01:22:31.153 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/refs/remotes/origin/HEAD" 2023-12-01T01:22:31.153 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.153 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.153 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.153 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 1998787905, 2023-12-01T01:22:31.153 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629801, 2023-12-01T01:22:31.153 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/applypatch-msg.sample" 2023-12-01T01:22:31.153 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.153 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.154 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.154 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 2067168348, 2023-12-01T01:22:31.154 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511627794, 2023-12-01T01:22:31.154 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/objects/pack/pack-fe9e56eb4d11faa0800e85314d6db62e368e37e2.pack" 2023-12-01T01:22:31.154 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.154 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.154 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.154 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 2067441283, 2023-12-01T01:22:31.154 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628267, 2023-12-01T01:22:31.154 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/048" 2023-12-01T01:22:31.154 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.154 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.154 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.155 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 2314649064, 2023-12-01T01:22:31.155 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511627808, 2023-12-01T01:22:31.155 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/HEAD" 2023-12-01T01:22:31.155 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.155 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.155 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.155 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 2316055598, 2023-12-01T01:22:31.155 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629793, 2023-12-01T01:22:31.155 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/pre-commit.sample" 2023-12-01T01:22:31.155 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.155 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.155 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.155 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 2418999225, 2023-12-01T01:22:31.155 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629790, 2023-12-01T01:22:31.156 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks" 2023-12-01T01:22:31.156 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.156 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.156 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.156 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 2574389908, 2023-12-01T01:22:31.156 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629815, 2023-12-01T01:22:31.156 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/061" 2023-12-01T01:22:31.156 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.156 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.156 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.156 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 2642785167, 2023-12-01T01:22:31.156 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511627804, 2023-12-01T01:22:31.156 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/refs" 2023-12-01T01:22:31.156 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.157 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.157 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.157 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 2671866383, 2023-12-01T01:22:31.157 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629805, 2023-12-01T01:22:31.157 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/refs/heads" 2023-12-01T01:22:31.157 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.157 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.157 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.157 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 2757735776, 2023-12-01T01:22:31.157 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629794, 2023-12-01T01:22:31.157 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/update.sample" 2023-12-01T01:22:31.157 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.157 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.157 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.158 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 2773009882, 2023-12-01T01:22:31.158 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511627813, 2023-12-01T01:22:31.158 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/refs/heads/master" 2023-12-01T01:22:31.158 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.158 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.158 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.158 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 2838053543, 2023-12-01T01:22:31.158 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511627787, 2023-12-01T01:22:31.158 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/objects/pack" 2023-12-01T01:22:31.158 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.158 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.158 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.159 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 2841150211, 2023-12-01T01:22:31.159 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629813, 2023-12-01T01:22:31.159 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/060" 2023-12-01T01:22:31.159 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.159 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.159 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.159 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 2844842988, 2023-12-01T01:22:31.159 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511627788, 2023-12-01T01:22:31.159 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/objects/info" 2023-12-01T01:22:31.159 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.159 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.159 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.160 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 2847010026, 2023-12-01T01:22:31.160 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629838, 2023-12-01T01:22:31.160 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/072.out" 2023-12-01T01:22:31.160 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.160 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.160 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.160 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 2892705002, 2023-12-01T01:22:31.160 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511627812, 2023-12-01T01:22:31.160 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/refs/heads" 2023-12-01T01:22:31.160 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.160 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.160 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.160 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 3086244248, 2023-12-01T01:22:31.160 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628247, 2023-12-01T01:22:31.161 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/038" 2023-12-01T01:22:31.161 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.161 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.161 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.161 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 3161524395, 2023-12-01T01:22:31.161 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511627799, 2023-12-01T01:22:31.161 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/packed-refs" 2023-12-01T01:22:31.161 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.161 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.161 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.161 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 3241200341, 2023-12-01T01:22:31.161 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628219, 2023-12-01T01:22:31.161 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/024" 2023-12-01T01:22:31.162 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.162 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.162 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.162 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 3464379870, 2023-12-01T01:22:31.162 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628198, 2023-12-01T01:22:31.162 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/013.out" 2023-12-01T01:22:31.162 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.162 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.162 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.162 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 3522942619, 2023-12-01T01:22:31.162 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511627806, 2023-12-01T01:22:31.162 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/refs/remotes/origin" 2023-12-01T01:22:31.162 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.163 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.163 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.163 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 3544605118, 2023-12-01T01:22:31.163 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628243, 2023-12-01T01:22:31.163 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/036" 2023-12-01T01:22:31.163 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.163 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.163 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.163 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 3555931254, 2023-12-01T01:22:31.163 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628226, 2023-12-01T01:22:31.163 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/027.out" 2023-12-01T01:22:31.163 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.163 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.163 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.164 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 3581624321, 2023-12-01T01:22:31.164 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629789, 2023-12-01T01:22:31.164 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/info/exclude" 2023-12-01T01:22:31.164 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.164 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.164 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.164 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 3597285760, 2023-12-01T01:22:31.164 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629820, 2023-12-01T01:22:31.164 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/063.out" 2023-12-01T01:22:31.164 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.164 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.164 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.165 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 3651198720, 2023-12-01T01:22:31.165 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511627810, 2023-12-01T01:22:31.165 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/refs/heads/master" 2023-12-01T01:22:31.165 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.165 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.165 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.165 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 3786557964, 2023-12-01T01:22:31.165 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629788, 2023-12-01T01:22:31.165 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/info" 2023-12-01T01:22:31.165 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.165 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.165 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.165 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 3929051858, 2023-12-01T01:22:31.166 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629830, 2023-12-01T01:22:31.166 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/068.out" 2023-12-01T01:22:31.166 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.166 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.166 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.166 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 3959738088, 2023-12-01T01:22:31.166 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628290, 2023-12-01T01:22:31.166 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/057" 2023-12-01T01:22:31.166 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.166 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.166 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.166 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 4006946962, 2023-12-01T01:22:31.166 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628270, 2023-12-01T01:22:31.167 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/049.out" 2023-12-01T01:22:31.167 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.167 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.167 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.167 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 4074317491, 2023-12-01T01:22:31.167 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628280, 2023-12-01T01:22:31.167 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/054.out" 2023-12-01T01:22:31.167 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.167 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.167 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.167 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 4095850631, 2023-12-01T01:22:31.168 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629819, 2023-12-01T01:22:31.168 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/063" 2023-12-01T01:22:31.168 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.168 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.168 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.168 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 4129769534, 2023-12-01T01:22:31.168 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628286, 2023-12-01T01:22:31.168 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/055" 2023-12-01T01:22:31.168 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.168 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.168 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.168 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 4141656461, 2023-12-01T01:22:31.169 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628244, 2023-12-01T01:22:31.169 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/036.out" 2023-12-01T01:22:31.169 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.169 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.169 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.169 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 4143748041, 2023-12-01T01:22:31.169 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628287, 2023-12-01T01:22:31.169 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/055.out" 2023-12-01T01:22:31.169 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.169 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.169 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.169 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 4168634955, 2023-12-01T01:22:31.169 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511629798, 2023-12-01T01:22:31.169 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/pre-merge-commit.sample" 2023-12-01T01:22:31.170 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.170 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.170 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.170 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 4184368615, 2023-12-01T01:22:31.170 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511627807, 2023-12-01T01:22:31.170 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/refs/remotes/origin/HEAD" 2023-12-01T01:22:31.170 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.170 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.170 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.170 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 4215518723, 2023-12-01T01:22:31.170 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511627795, 2023-12-01T01:22:31.170 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/objects/pack/pack-fe9e56eb4d11faa0800e85314d6db62e368e37e2.idx" 2023-12-01T01:22:31.171 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.171 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.171 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.171 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 4215586282, 2023-12-01T01:22:31.171 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511627801, 2023-12-01T01:22:31.171 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/refs/remotes/origin" 2023-12-01T01:22:31.171 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.171 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.171 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.171 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 4245207642, 2023-12-01T01:22:31.171 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511627805, 2023-12-01T01:22:31.171 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/refs/remotes" 2023-12-01T01:22:31.171 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:31.171 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:31.172 INFO:teuthology.orchestra.run.smithi077.stdout: "damage_type": "backtrace", 2023-12-01T01:22:31.172 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 4262423772, 2023-12-01T01:22:31.172 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1099511628291, 2023-12-01T01:22:31.172 INFO:teuthology.orchestra.run.smithi077.stdout: "path": "/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/057.out" 2023-12-01T01:22:31.172 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:31.172 INFO:teuthology.orchestra.run.smithi077.stdout:] 2023-12-01T01:22:31.174 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.120+0000 7eff77755700 1 -- 172.21.15.77:0/343880724 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7eff5c0a0520 msgr2=0x7eff5c0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:31.174 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.120+0000 7eff77755700 1 --2- 172.21.15.77:0/343880724 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7eff5c0a0520 0x7eff5c0a29e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7eff70100cd0 tx=0x7eff6000b410 comp rx=0 tx=0).stop 2023-12-01T01:22:31.174 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.120+0000 7eff77755700 1 -- 172.21.15.77:0/343880724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff70101810 msgr2=0x7eff7019df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:31.174 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.120+0000 7eff77755700 1 --2- 172.21.15.77:0/343880724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff70101810 0x7eff7019df00 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7eff68007a00 tx=0x7eff680094c0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.174 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.120+0000 7eff77755700 1 -- 172.21.15.77:0/343880724 shutdown_connections 2023-12-01T01:22:31.174 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.120+0000 7eff77755700 1 --2- 172.21.15.77:0/343880724 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7eff5c0a0520 0x7eff5c0a29e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.174 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.120+0000 7eff77755700 1 --2- 172.21.15.77:0/343880724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff70101810 0x7eff7019df00 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.174 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.120+0000 7eff77755700 1 --2- 172.21.15.77:0/343880724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff700ffc80 0x7eff70068a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.174 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.120+0000 7eff77755700 1 --2- 172.21.15.77:0/343880724 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff70100e70 0x7eff7019b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.174 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.120+0000 7eff77755700 1 -- 172.21.15.77:0/343880724 >> 172.21.15.77:0/343880724 conn(0x7eff700fb1f0 msgr2=0x7eff700fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:31.174 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.120+0000 7eff77755700 1 -- 172.21.15.77:0/343880724 shutdown_connections 2023-12-01T01:22:31.175 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.121+0000 7eff77755700 1 -- 172.21.15.77:0/343880724 wait complete. 2023-12-01T01:22:31.175 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-12-01T01:22:31.300 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.296+0000 7f9522167700 1 -- 172.21.15.77:0/2975533177 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f951c104130 msgr2=0x7f951c106520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:31.300 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.296+0000 7f9522167700 1 --2- 172.21.15.77:0/2975533177 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f951c104130 0x7f951c106520 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f9510009990 tx=0x7f9510009ca0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.300 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.296+0000 7f9522167700 1 -- 172.21.15.77:0/2975533177 shutdown_connections 2023-12-01T01:22:31.300 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.296+0000 7f9522167700 1 --2- 172.21.15.77:0/2975533177 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f951c106a60 0x7f951c108ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.300 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.296+0000 7f9522167700 1 --2- 172.21.15.77:0/2975533177 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f951c101800 0x7f951c103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.300 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.296+0000 7f9522167700 1 --2- 172.21.15.77:0/2975533177 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f951c104130 0x7f951c106520 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.300 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.296+0000 7f9522167700 1 -- 172.21.15.77:0/2975533177 >> 172.21.15.77:0/2975533177 conn(0x7f951c0fb1f0 msgr2=0x7f951c0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:31.300 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.296+0000 7f9522167700 1 -- 172.21.15.77:0/2975533177 shutdown_connections 2023-12-01T01:22:31.301 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.297+0000 7f9522167700 1 -- 172.21.15.77:0/2975533177 wait complete. 2023-12-01T01:22:31.301 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.297+0000 7f9522167700 1 Processor -- start 2023-12-01T01:22:31.301 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.297+0000 7f9522167700 1 -- start start 2023-12-01T01:22:31.301 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.297+0000 7f9522167700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f951c101800 0x7f951c198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.302 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.297+0000 7f9522167700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f951c104130 0x7f951c1991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.302 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.297+0000 7f9522167700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f951c106a60 0x7f951c1a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.302 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.297+0000 7f9522167700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951c1a0800 con 0x7f951c106a60 2023-12-01T01:22:31.302 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.297+0000 7f9522167700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951c1a0970 con 0x7f951c101800 2023-12-01T01:22:31.302 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.297+0000 7f9522167700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951c1a0b30 con 0x7f951c104130 2023-12-01T01:22:31.302 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.298+0000 7f951bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f951c106a60 0x7f951c1a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:31.302 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.298+0000 7f951bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f951c106a60 0x7f951c1a0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33088/0 (socket says 172.21.15.77:33088) 2023-12-01T01:22:31.302 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.298+0000 7f951bfff700 1 -- 172.21.15.77:0/806302584 learned_addr learned my addr 172.21.15.77:0/806302584 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:22:31.302 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.298+0000 7f951affd700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f951c104130 0x7f951c1991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:31.302 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.298+0000 7f951b7fe700 1 --2- 172.21.15.77:0/806302584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f951c101800 0x7f951c198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:31.302 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.298+0000 7f951bfff700 1 -- 172.21.15.77:0/806302584 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f951c104130 msgr2=0x7f951c1991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:31.302 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.298+0000 7f951bfff700 1 --2- 172.21.15.77:0/806302584 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f951c104130 0x7f951c1991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.302 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.298+0000 7f951bfff700 1 -- 172.21.15.77:0/806302584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f951c101800 msgr2=0x7f951c198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:31.303 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.298+0000 7f951bfff700 1 --2- 172.21.15.77:0/806302584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f951c101800 0x7f951c198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.303 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.298+0000 7f951bfff700 1 -- 172.21.15.77:0/806302584 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95100095f0 con 0x7f951c106a60 2023-12-01T01:22:31.303 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.298+0000 7f951bfff700 1 --2- 172.21.15.77:0/806302584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f951c106a60 0x7f951c1a0230 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f951400ef30 tx=0x7f951400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:31.303 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.298+0000 7f9518ff9700 1 -- 172.21.15.77:0/806302584 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9514018070 con 0x7f951c106a60 2023-12-01T01:22:31.303 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.298+0000 7f9522167700 1 -- 172.21.15.77:0/806302584 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f951c1a0db0 con 0x7f951c106a60 2023-12-01T01:22:31.303 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.298+0000 7f9518ff9700 1 -- 172.21.15.77:0/806302584 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f951400f040 con 0x7f951c106a60 2023-12-01T01:22:31.303 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.298+0000 7f9518ff9700 1 -- 172.21.15.77:0/806302584 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951400cdb0 con 0x7f951c106a60 2023-12-01T01:22:31.303 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.298+0000 7f9522167700 1 -- 172.21.15.77:0/806302584 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f951c1a1210 con 0x7f951c106a60 2023-12-01T01:22:31.310 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.300+0000 7f9518ff9700 1 -- 172.21.15.77:0/806302584 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f9514003a00 con 0x7f951c106a60 2023-12-01T01:22:31.310 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.301+0000 7f9518ff9700 1 --2- 172.21.15.77:0/806302584 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f95040a0520 0x7f95040a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.310 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.301+0000 7f951b7fe700 1 --2- 172.21.15.77:0/806302584 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f95040a0520 0x7f95040a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:31.311 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.301+0000 7f9518ff9700 1 -- 172.21.15.77:0/806302584 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f95140cb4d0 con 0x7f951c106a60 2023-12-01T01:22:31.311 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.302+0000 7f951b7fe700 1 --2- 172.21.15.77:0/806302584 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f95040a0520 0x7f95040a29e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f951c199b60 tx=0x7f950c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:31.311 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.302+0000 7f952e2f7b80 1 Processor -- start 2023-12-01T01:22:31.311 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.302+0000 7f952e2f7b80 1 -- start start 2023-12-01T01:22:31.311 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.302+0000 7f952e2f7b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564c0ce380f0 0x564c0ce38530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.311 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.302+0000 7f952e2f7b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c0ce38a70 0x564c0ce36e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.311 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.302+0000 7f952e2f7b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c0ce373f0 0x564c0ce37870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.311 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.302+0000 7f952e2f7b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0cd8ea10 con 0x564c0ce38a70 2023-12-01T01:22:31.311 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.302+0000 7f952e2f7b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0cd8c3d0 con 0x564c0ce373f0 2023-12-01T01:22:31.311 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.302+0000 7f952e2f7b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0cd87cc0 con 0x564c0ce380f0 2023-12-01T01:22:31.312 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.302+0000 7f951affd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c0ce38a70 0x564c0ce36e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:31.312 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.302+0000 7f951affd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c0ce38a70 0x564c0ce36e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33094/0 (socket says 172.21.15.77:33094) 2023-12-01T01:22:31.312 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.302+0000 7f951affd700 1 -- 172.21.15.77:0/1984392845 learned_addr learned my addr 172.21.15.77:0/1984392845 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:22:31.312 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.302+0000 7f951b7fe700 1 --2- 172.21.15.77:0/1984392845 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564c0ce380f0 0x564c0ce38530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:31.312 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.302+0000 7f951bfff700 1 --2- 172.21.15.77:0/1984392845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c0ce373f0 0x564c0ce37870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:31.312 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.302+0000 7f951affd700 1 -- 172.21.15.77:0/1984392845 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564c0ce380f0 msgr2=0x564c0ce38530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:31.312 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.302+0000 7f951affd700 1 --2- 172.21.15.77:0/1984392845 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564c0ce380f0 0x564c0ce38530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.312 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.302+0000 7f951affd700 1 -- 172.21.15.77:0/1984392845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c0ce373f0 msgr2=0x564c0ce37870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:31.312 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.302+0000 7f951affd700 1 --2- 172.21.15.77:0/1984392845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c0ce373f0 0x564c0ce37870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.312 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.302+0000 7f951affd700 1 -- 172.21.15.77:0/1984392845 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c0cbb5de0 con 0x564c0ce38a70 2023-12-01T01:22:31.312 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.303+0000 7f951affd700 1 --2- 172.21.15.77:0/1984392845 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c0ce38a70 0x564c0ce36e60 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f9510009990 tx=0x7f9510005ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81106746b3993dcf server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:31.312 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.303+0000 7f9501ffb700 1 -- 172.21.15.77:0/1984392845 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951001d070 con 0x564c0ce38a70 2023-12-01T01:22:31.312 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.303+0000 7f9501ffb700 1 -- 172.21.15.77:0/1984392845 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f951001e040 con 0x564c0ce38a70 2023-12-01T01:22:31.312 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.303+0000 7f9501ffb700 1 -- 172.21.15.77:0/1984392845 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9510004910 con 0x564c0ce38a70 2023-12-01T01:22:31.313 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.303+0000 7f952e2f7b80 1 -- 172.21.15.77:0/1984392845 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c0ce38a70 msgr2=0x564c0ce36e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:31.313 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.303+0000 7f952e2f7b80 1 --2- 172.21.15.77:0/1984392845 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c0ce38a70 0x564c0ce36e60 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f9510009990 tx=0x7f9510005ea0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.313 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.304+0000 7f952e2f7b80 1 -- 172.21.15.77:0/1984392845 shutdown_connections 2023-12-01T01:22:31.313 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.304+0000 7f952e2f7b80 1 --2- 172.21.15.77:0/1984392845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c0ce373f0 0x564c0ce37870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.313 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.304+0000 7f952e2f7b80 1 --2- 172.21.15.77:0/1984392845 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564c0ce380f0 0x564c0ce38530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.313 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.304+0000 7f952e2f7b80 1 --2- 172.21.15.77:0/1984392845 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c0ce38a70 0x564c0ce36e60 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.313 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.304+0000 7f952e2f7b80 1 -- 172.21.15.77:0/1984392845 >> 172.21.15.77:0/1984392845 conn(0x564c0cd960d0 msgr2=0x564c0ce31940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:31.313 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.304+0000 7f952e2f7b80 1 -- 172.21.15.77:0/1984392845 shutdown_connections 2023-12-01T01:22:31.313 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.304+0000 7f952e2f7b80 1 -- 172.21.15.77:0/1984392845 wait complete. 2023-12-01T01:22:31.313 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.304+0000 7f952e2f7b80 1 -- start start 2023-12-01T01:22:31.313 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.304+0000 7f952e2f7b80 1 Processor -- start 2023-12-01T01:22:31.313 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.305+0000 7f952e2f7b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c0ce373f0 0x564c0ce408f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.313 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.305+0000 7f952e2f7b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564c0ce380f0 0x564c0ce40e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.313 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.305+0000 7f952e2f7b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c0ce38a70 0x564c0ce47e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.314 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.305+0000 7f952e2f7b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0cd24510 con 0x564c0ce373f0 2023-12-01T01:22:31.314 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.305+0000 7f952e2f7b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0cd241b0 con 0x564c0ce38a70 2023-12-01T01:22:31.314 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.305+0000 7f952e2f7b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0cc0b920 con 0x564c0ce380f0 2023-12-01T01:22:31.314 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.305+0000 7f951b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c0ce373f0 0x564c0ce408f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:31.314 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.305+0000 7f952e2f7b80 10 client.0 _ll_register_callbacks cb 0x564c0cc71a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:22:31.314 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.305+0000 7f951b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c0ce373f0 0x564c0ce408f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33098/0 (socket says 172.21.15.77:33098) 2023-12-01T01:22:31.314 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.305+0000 7f951b7fe700 1 -- 172.21.15.77:0/2948115397 learned_addr learned my addr 172.21.15.77:0/2948115397 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:22:31.314 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.305+0000 7f951affd700 1 --2- 172.21.15.77:0/2948115397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564c0ce380f0 0x564c0ce40e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:31.314 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.305+0000 7f951bfff700 1 --2- 172.21.15.77:0/2948115397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c0ce38a70 0x564c0ce47e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:31.314 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.305+0000 7f951b7fe700 1 -- 172.21.15.77:0/2948115397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564c0ce380f0 msgr2=0x564c0ce40e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:31.314 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.305+0000 7f951b7fe700 1 --2- 172.21.15.77:0/2948115397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564c0ce380f0 0x564c0ce40e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.314 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.305+0000 7f951b7fe700 1 -- 172.21.15.77:0/2948115397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c0ce38a70 msgr2=0x564c0ce47e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:31.314 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.305+0000 7f951b7fe700 1 --2- 172.21.15.77:0/2948115397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c0ce38a70 0x564c0ce47e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.314 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.305+0000 7f951b7fe700 1 -- 172.21.15.77:0/2948115397 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f951c109ac0 con 0x564c0ce373f0 2023-12-01T01:22:31.315 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.305+0000 7f951bfff700 1 --2- 172.21.15.77:0/2948115397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c0ce38a70 0x564c0ce47e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:22:31.315 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.305+0000 7f951b7fe700 1 --2- 172.21.15.77:0/2948115397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c0ce373f0 0x564c0ce408f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f950c012910 tx=0x7f950c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca73fabbded78b4 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:31.315 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.305+0000 7f94faffd700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:22:31.315 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.305+0000 7f94faffd700 1 -- 172.21.15.77:0/2948115397 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950c020840 con 0x564c0ce373f0 2023-12-01T01:22:31.315 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.305+0000 7f952e2f7b80 1 -- 172.21.15.77:0/2948115397 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c0cdb2d10 con 0x564c0ce373f0 2023-12-01T01:22:31.315 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.305+0000 7f94faffd700 1 -- 172.21.15.77:0/2948115397 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f950c018070 con 0x564c0ce373f0 2023-12-01T01:22:31.315 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.305+0000 7f94faffd700 1 -- 172.21.15.77:0/2948115397 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f950c0209a0 con 0x564c0ce373f0 2023-12-01T01:22:31.315 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.305+0000 7f952e2f7b80 10 client.16317 fetch_fsmap learned FSMap version 41 2023-12-01T01:22:31.315 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.305+0000 7f952e2f7b80 1 -- 172.21.15.77:0/2948115397 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x564c0cb9ec60 con 0x564c0ce373f0 2023-12-01T01:22:31.315 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.306+0000 7f94faffd700 1 -- 172.21.15.77:0/2948115397 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564c0cdb2d10 con 0x564c0ce373f0 2023-12-01T01:22:31.315 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.306+0000 7f94faffd700 1 -- 172.21.15.77:0/2948115397 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 41) v1 ==== 7514+0+0 (secure 0 0 0) 0x7f950c029c30 con 0x564c0ce373f0 2023-12-01T01:22:31.315 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.306+0000 7f952e2f7b80 10 client.16317 fetch_fsmap finished waiting for FSMap version 41 2023-12-01T01:22:31.315 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.306+0000 7f952e2f7b80 10 client.16317 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-01T01:22:31.315 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.306+0000 7f952e2f7b80 20 client.16317 populate_metadata read hostname 'smithi077' 2023-12-01T01:22:31.316 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.306+0000 7f952e2f7b80 1 --2- 172.21.15.77:0/2948115397 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x564c0ce4c6a0 0x564c0ce4eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.316 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.306+0000 7f952e2f7b80 4 client.16317 mds_command: new command op to 24500 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-01T01:22:31.316 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.306+0000 7f952e2f7b80 1 -- 172.21.15.77:0/2948115397 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c0ce05e10 con 0x564c0ce4c6a0 2023-12-01T01:22:31.316 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.306+0000 7f951affd700 1 --2- 172.21.15.77:0/2948115397 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x564c0ce4c6a0 0x564c0ce4eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:31.316 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.306+0000 7f94faffd700 10 client.16317 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:22:31.316 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.306+0000 7f951affd700 1 --2- 172.21.15.77:0/2948115397 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x564c0ce4c6a0 0x564c0ce4eac0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23b8c6ede135b117 server_cookie=f686a75b2a9401f6 in_seq=0 out_seq=0 2023-12-01T01:22:31.316 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.307+0000 7f94faffd700 1 -- 172.21.15.77:0/2948115397 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x564c0ce05e10 con 0x564c0ce4c6a0 2023-12-01T01:22:31.316 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.307+0000 7f94faffd700 10 client.16317 handle_command_reply: tid=0 2023-12-01T01:22:31.316 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.307+0000 7f952e2f7b80 1 client.16317 shutdown 2023-12-01T01:22:31.316 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.307+0000 7f952e2f7b80 1 -- 172.21.15.77:0/2948115397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c0ce373f0 msgr2=0x564c0ce408f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:31.316 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.307+0000 7f952e2f7b80 1 --2- 172.21.15.77:0/2948115397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c0ce373f0 0x564c0ce408f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f950c012910 tx=0x7f950c0047c0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.316 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.307+0000 7f952e2f7b80 1 -- 172.21.15.77:0/2948115397 shutdown_connections 2023-12-01T01:22:31.316 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.307+0000 7f952e2f7b80 1 --2- 172.21.15.77:0/2948115397 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x564c0ce4c6a0 0x564c0ce4eac0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.316 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.307+0000 7f952e2f7b80 1 --2- 172.21.15.77:0/2948115397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c0ce38a70 0x564c0ce47e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.317 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.307+0000 7f952e2f7b80 1 --2- 172.21.15.77:0/2948115397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c0ce373f0 0x564c0ce408f0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.317 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.307+0000 7f94faffd700 0 client.16317 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:22:31.317 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.307+0000 7f952e2f7b80 1 --2- 172.21.15.77:0/2948115397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564c0ce380f0 0x564c0ce40e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.317 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.307+0000 7f952e2f7b80 1 -- 172.21.15.77:0/2948115397 >> 172.21.15.77:0/2948115397 conn(0x564c0cd960d0 msgr2=0x564c0ce310b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:31.317 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.307+0000 7f952e2f7b80 1 -- 172.21.15.77:0/2948115397 shutdown_connections 2023-12-01T01:22:31.317 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.307+0000 7f952e2f7b80 1 -- 172.21.15.77:0/2948115397 wait complete. 2023-12-01T01:22:31.317 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.308+0000 7f952e2f7b80 20 client.16317 trim_cache size 0 max 16384 2023-12-01T01:22:31.326 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.320+0000 7f952e2f7b80 1 Processor -- start 2023-12-01T01:22:31.326 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.320+0000 7f952e2f7b80 1 -- start start 2023-12-01T01:22:31.327 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.320+0000 7f952e2f7b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c0ce373f0 0x564c0ce45e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.327 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.320+0000 7f952e2f7b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c0ce380f0 0x564c0ce3ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.327 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.320+0000 7f952e2f7b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564c0ce38a70 0x564c0ce3b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.327 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.320+0000 7f952e2f7b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0ce3e4e0 con 0x564c0ce380f0 2023-12-01T01:22:31.327 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.320+0000 7f952e2f7b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0ce333d0 con 0x564c0ce373f0 2023-12-01T01:22:31.327 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.320+0000 7f952e2f7b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0ce53440 con 0x564c0ce38a70 2023-12-01T01:22:31.327 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.320+0000 7f951affd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c0ce380f0 0x564c0ce3ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:31.327 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.320+0000 7f951affd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c0ce380f0 0x564c0ce3ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33108/0 (socket says 172.21.15.77:33108) 2023-12-01T01:22:31.327 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.320+0000 7f951affd700 1 -- 172.21.15.77:0/3951045187 learned_addr learned my addr 172.21.15.77:0/3951045187 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:22:31.327 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.320+0000 7f951bfff700 1 --2- 172.21.15.77:0/3951045187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564c0ce38a70 0x564c0ce3b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:31.327 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.321+0000 7f951b7fe700 1 --2- 172.21.15.77:0/3951045187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c0ce373f0 0x564c0ce45e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:31.327 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.321+0000 7f951affd700 1 -- 172.21.15.77:0/3951045187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564c0ce38a70 msgr2=0x564c0ce3b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:31.327 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.321+0000 7f951affd700 1 --2- 172.21.15.77:0/3951045187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564c0ce38a70 0x564c0ce3b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.328 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.321+0000 7f951affd700 1 -- 172.21.15.77:0/3951045187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c0ce373f0 msgr2=0x564c0ce45e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:31.328 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.321+0000 7f951affd700 1 --2- 172.21.15.77:0/3951045187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c0ce373f0 0x564c0ce45e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.328 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.321+0000 7f951affd700 1 -- 172.21.15.77:0/3951045187 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c0cbdc250 con 0x564c0ce380f0 2023-12-01T01:22:31.328 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.321+0000 7f951affd700 1 --2- 172.21.15.77:0/3951045187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c0ce380f0 0x564c0ce3ffa0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f95100056f0 tx=0x7f951001a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aec542437616a82 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:31.328 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.321+0000 7f95027fc700 1 -- 172.21.15.77:0/3951045187 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951001d040 con 0x564c0ce380f0 2023-12-01T01:22:31.328 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.321+0000 7f95027fc700 1 -- 172.21.15.77:0/3951045187 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f951000b2c0 con 0x564c0ce380f0 2023-12-01T01:22:31.328 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.321+0000 7f95027fc700 1 -- 172.21.15.77:0/3951045187 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951000fa60 con 0x564c0ce380f0 2023-12-01T01:22:31.328 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.321+0000 7f952e2f7b80 1 -- 172.21.15.77:0/3951045187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c0ce380f0 msgr2=0x564c0ce3ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:31.328 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.321+0000 7f952e2f7b80 1 --2- 172.21.15.77:0/3951045187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c0ce380f0 0x564c0ce3ffa0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f95100056f0 tx=0x7f951001a040 comp rx=0 tx=0).stop 2023-12-01T01:22:31.328 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.321+0000 7f952e2f7b80 1 -- 172.21.15.77:0/3951045187 shutdown_connections 2023-12-01T01:22:31.328 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.321+0000 7f952e2f7b80 1 --2- 172.21.15.77:0/3951045187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564c0ce38a70 0x564c0ce3b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.328 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.321+0000 7f952e2f7b80 1 --2- 172.21.15.77:0/3951045187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c0ce373f0 0x564c0ce45e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.328 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.321+0000 7f952e2f7b80 1 --2- 172.21.15.77:0/3951045187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c0ce380f0 0x564c0ce3ffa0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.328 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.321+0000 7f952e2f7b80 1 -- 172.21.15.77:0/3951045187 >> 172.21.15.77:0/3951045187 conn(0x564c0cd960d0 msgr2=0x564c0ce31910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:31.329 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.322+0000 7f952e2f7b80 1 -- 172.21.15.77:0/3951045187 shutdown_connections 2023-12-01T01:22:31.329 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.322+0000 7f952e2f7b80 1 -- 172.21.15.77:0/3951045187 wait complete. 2023-12-01T01:22:31.329 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.322+0000 7f952e2f7b80 1 -- start start 2023-12-01T01:22:31.329 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.322+0000 7f952e2f7b80 1 Processor -- start 2023-12-01T01:22:31.329 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.322+0000 7f952e2f7b80 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564c0ce373f0 0x564c0ce44060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.329 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.322+0000 7f952e2f7b80 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c0ce380f0 0x564c0ce445a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.329 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.322+0000 7f952e2f7b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c0ce38a70 0x564c0ce45170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.329 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.322+0000 7f952e2f7b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0ce456b0 con 0x564c0ce38a70 2023-12-01T01:22:31.329 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.322+0000 7f952e2f7b80 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0ce45820 con 0x564c0ce380f0 2023-12-01T01:22:31.329 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.322+0000 7f952e2f7b80 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0ce459e0 con 0x564c0ce373f0 2023-12-01T01:22:31.329 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.322+0000 7f952e2f7b80 10 client.0 _ll_register_callbacks cb 0x564c0cc71a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-01T01:22:31.329 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.322+0000 7f951bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c0ce38a70 0x564c0ce45170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:31.329 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.322+0000 7f951bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c0ce38a70 0x564c0ce45170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33116/0 (socket says 172.21.15.77:33116) 2023-12-01T01:22:31.329 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.322+0000 7f951bfff700 1 -- 172.21.15.77:0/3141782420 learned_addr learned my addr 172.21.15.77:0/3141782420 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:22:31.330 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.322+0000 7f951b7fe700 1 --2- 172.21.15.77:0/3141782420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564c0ce373f0 0x564c0ce44060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:31.330 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.322+0000 7f951affd700 1 --2- 172.21.15.77:0/3141782420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c0ce380f0 0x564c0ce445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:31.330 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.323+0000 7f951bfff700 1 -- 172.21.15.77:0/3141782420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564c0ce373f0 msgr2=0x564c0ce44060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:31.330 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.323+0000 7f951bfff700 1 --2- 172.21.15.77:0/3141782420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564c0ce373f0 0x564c0ce44060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.330 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.323+0000 7f951bfff700 1 -- 172.21.15.77:0/3141782420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c0ce380f0 msgr2=0x564c0ce445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-01T01:22:31.330 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.323+0000 7f951bfff700 1 --2- 172.21.15.77:0/3141782420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c0ce380f0 0x564c0ce445a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.330 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.323+0000 7f951bfff700 1 -- 172.21.15.77:0/3141782420 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c0cbb5de0 con 0x564c0ce38a70 2023-12-01T01:22:31.330 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.323+0000 7f951affd700 1 --2- 172.21.15.77:0/3141782420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c0ce380f0 0x564c0ce445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:22:31.330 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.323+0000 7f951bfff700 1 --2- 172.21.15.77:0/3141782420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c0ce38a70 0x564c0ce45170 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f95140a0910 tx=0x7f95140987c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecc6994611dae53e server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:31.330 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.323+0000 7f94faffd700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2023-12-01T01:22:31.330 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.323+0000 7f94faffd700 1 -- 172.21.15.77:0/3141782420 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951401b320 con 0x564c0ce38a70 2023-12-01T01:22:31.330 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.323+0000 7f94faffd700 1 -- 172.21.15.77:0/3141782420 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f9514099560 con 0x564c0ce38a70 2023-12-01T01:22:31.330 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.323+0000 7f952e2f7b80 1 -- 172.21.15.77:0/3141782420 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f951001e040 con 0x564c0ce38a70 2023-12-01T01:22:31.331 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.323+0000 7f94faffd700 1 -- 172.21.15.77:0/3141782420 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9514098e60 con 0x564c0ce38a70 2023-12-01T01:22:31.331 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.323+0000 7f952e2f7b80 10 client.16323 fetch_fsmap learned FSMap version 41 2023-12-01T01:22:31.331 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.323+0000 7f952e2f7b80 1 -- 172.21.15.77:0/3141782420 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x564c0cbb42b0 con 0x564c0ce38a70 2023-12-01T01:22:31.331 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.323+0000 7f94faffd700 1 -- 172.21.15.77:0/3141782420 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f951001e040 con 0x564c0ce38a70 2023-12-01T01:22:31.331 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.323+0000 7f94faffd700 1 -- 172.21.15.77:0/3141782420 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 41) v1 ==== 7514+0+0 (secure 0 0 0) 0x7f9514098350 con 0x564c0ce38a70 2023-12-01T01:22:31.331 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.323+0000 7f952e2f7b80 10 client.16323 fetch_fsmap finished waiting for FSMap version 41 2023-12-01T01:22:31.331 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.323+0000 7f952e2f7b80 10 client.16323 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-12-01T01:22:31.331 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.323+0000 7f952e2f7b80 20 client.16323 populate_metadata read hostname 'smithi077' 2023-12-01T01:22:31.331 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.323+0000 7f952e2f7b80 1 --2- 172.21.15.77:0/3141782420 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x564c0ce9f1b0 0x564c0cea15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.331 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.323+0000 7f952e2f7b80 4 client.16323 mds_command: new command op to 24500 tid=0[{"prefix": "damage ls"}] 2023-12-01T01:22:31.331 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.323+0000 7f952e2f7b80 1 -- 172.21.15.77:0/3141782420 --> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564c0cd80d40 con 0x564c0ce9f1b0 2023-12-01T01:22:31.331 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.324+0000 7f951b7fe700 1 --2- 172.21.15.77:0/3141782420 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x564c0ce9f1b0 0x564c0cea15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:31.331 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.324+0000 7f94faffd700 10 client.16323 ms_handle_connect on v2:172.21.15.137:6838/1099966660 2023-12-01T01:22:31.331 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.324+0000 7f951b7fe700 1 --2- 172.21.15.77:0/3141782420 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x564c0ce9f1b0 0x564c0cea15d0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9faf79c07409227c server_cookie=f136cc25c4523e8c in_seq=0 out_seq=0 2023-12-01T01:22:31.332 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.324+0000 7f94faffd700 1 -- 172.21.15.77:0/3141782420 <== mds.0 v2:172.21.15.137:6838/1099966660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564c0cd80d40 con 0x564c0ce9f1b0 2023-12-01T01:22:31.332 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.324+0000 7f94faffd700 10 client.16323 handle_command_reply: tid=0 2023-12-01T01:22:31.332 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.324+0000 7f952e2f7b80 1 client.16323 shutdown 2023-12-01T01:22:31.332 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.325+0000 7f952e2f7b80 1 -- 172.21.15.77:0/3141782420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c0ce38a70 msgr2=0x564c0ce45170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:31.332 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.325+0000 7f952e2f7b80 1 --2- 172.21.15.77:0/3141782420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c0ce38a70 0x564c0ce45170 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f95140a0910 tx=0x7f95140987c0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.332 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.325+0000 7f952e2f7b80 1 -- 172.21.15.77:0/3141782420 shutdown_connections 2023-12-01T01:22:31.332 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.325+0000 7f952e2f7b80 1 --2- 172.21.15.77:0/3141782420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564c0ce38a70 0x564c0ce45170 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.332 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.325+0000 7f952e2f7b80 1 --2- 172.21.15.77:0/3141782420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x564c0ce373f0 0x564c0ce44060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.332 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.325+0000 7f952e2f7b80 1 --2- 172.21.15.77:0/3141782420 >> [v2:172.21.15.137:6838/1099966660,v1:172.21.15.137:6839/1099966660] conn(0x564c0ce9f1b0 0x564c0cea15d0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.332 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.325+0000 7f952e2f7b80 1 --2- 172.21.15.77:0/3141782420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c0ce380f0 0x564c0ce445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.332 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.325+0000 7f94faffd700 0 client.16323 ms_handle_reset on v2:172.21.15.137:6838/1099966660 2023-12-01T01:22:31.332 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.325+0000 7f952e2f7b80 1 -- 172.21.15.77:0/3141782420 >> 172.21.15.77:0/3141782420 conn(0x564c0cd960d0 msgr2=0x564c0ce3b310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:31.332 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.325+0000 7f952e2f7b80 1 -- 172.21.15.77:0/3141782420 shutdown_connections 2023-12-01T01:22:31.332 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.325+0000 7f952e2f7b80 1 -- 172.21.15.77:0/3141782420 wait complete. 2023-12-01T01:22:31.333 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.325+0000 7f952e2f7b80 20 client.16323 trim_cache size 0 max 16384 2023-12-01T01:22:31.333 INFO:teuthology.orchestra.run.smithi077.stdout:[] 2023-12-01T01:22:31.334 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.325+0000 7f9522167700 1 -- 172.21.15.77:0/806302584 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f95040a0520 msgr2=0x7f95040a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:31.334 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.325+0000 7f9522167700 1 --2- 172.21.15.77:0/806302584 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f95040a0520 0x7f95040a29e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f951c199b60 tx=0x7f950c009260 comp rx=0 tx=0).stop 2023-12-01T01:22:31.334 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.325+0000 7f9522167700 1 -- 172.21.15.77:0/806302584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f951c106a60 msgr2=0x7f951c1a0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:31.334 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.325+0000 7f9522167700 1 --2- 172.21.15.77:0/806302584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f951c106a60 0x7f951c1a0230 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f951400ef30 tx=0x7f951400c3a0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.334 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.325+0000 7f9522167700 1 -- 172.21.15.77:0/806302584 shutdown_connections 2023-12-01T01:22:31.334 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.325+0000 7f9522167700 1 --2- 172.21.15.77:0/806302584 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f95040a0520 0x7f95040a29e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.334 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.325+0000 7f9522167700 1 --2- 172.21.15.77:0/806302584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f951c106a60 0x7f951c1a0230 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.334 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.325+0000 7f9522167700 1 --2- 172.21.15.77:0/806302584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f951c101800 0x7f951c198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.334 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.325+0000 7f9522167700 1 --2- 172.21.15.77:0/806302584 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f951c104130 0x7f951c1991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.334 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.325+0000 7f9522167700 1 -- 172.21.15.77:0/806302584 >> 172.21.15.77:0/806302584 conn(0x7f951c0fb1f0 msgr2=0x7f951c0ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:31.334 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.325+0000 7f9522167700 1 -- 172.21.15.77:0/806302584 shutdown_connections 2023-12-01T01:22:31.334 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.326+0000 7f9522167700 1 -- 172.21.15.77:0/806302584 wait complete. 2023-12-01T01:22:31.339 ERROR:tasks.fwd_scrub:rank 2 damaged: [{'damage_type': 'backtrace', 'id': 199831713, 'ino': 1099511628268, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/048.out'}, {'damage_type': 'backtrace', 'id': 314450054, 'ino': 1099511628249, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/039'}, {'damage_type': 'backtrace', 'id': 394327943, 'ino': 1099511628228, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/028.out'}, {'damage_type': 'backtrace', 'id': 430370064, 'ino': 1099511629834, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/070.out'}, {'damage_type': 'backtrace', 'id': 449008516, 'ino': 1099511629839, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/073'}, {'damage_type': 'backtrace', 'id': 930141062, 'ino': 1099511628173, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/001'}, {'damage_type': 'backtrace', 'id': 986724124, 'ino': 1099511628224, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/026.out'}, {'damage_type': 'backtrace', 'id': 1073715324, 'ino': 1099511628192, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/010.out'}, {'damage_type': 'backtrace', 'id': 1094384904, 'ino': 1099511629821, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/064'}, {'damage_type': 'backtrace', 'id': 1130381686, 'ino': 1099511628236, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/032.out'}, {'damage_type': 'backtrace', 'id': 1152559890, 'ino': 1099511629812, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/059.out'}, {'damage_type': 'backtrace', 'id': 1160278782, 'ino': 1099511629814, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/060.out'}, {'damage_type': 'backtrace', 'id': 1160732478, 'ino': 1099511628257, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/043'}, {'damage_type': 'backtrace', 'id': 1195723880, 'ino': 1099511628245, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/037'}, {'damage_type': 'backtrace', 'id': 1538021286, 'ino': 1099511629844, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/075.out'}, {'damage_type': 'backtrace', 'id': 1677118401, 'ino': 1099511628223, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/026'}, {'damage_type': 'backtrace', 'id': 1961928516, 'ino': 1099511629843, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/075'}, {'damage_type': 'backtrace', 'id': 1992333326, 'ino': 1099511628263, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/046'}, {'damage_type': 'backtrace', 'id': 2031349638, 'ino': 1099511629845, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/076'}, {'damage_type': 'backtrace', 'id': 2149702231, 'ino': 1099511628256, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/042.out'}, {'damage_type': 'backtrace', 'id': 2273843009, 'ino': 1099511628174, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/001.out'}, {'damage_type': 'backtrace', 'id': 2369318845, 'ino': 1099511628197, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/013'}, {'damage_type': 'backtrace', 'id': 2423172278, 'ino': 1099511628276, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/052.out'}, {'damage_type': 'backtrace', 'id': 2521698041, 'ino': 1099511629836, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/071.out'}, {'damage_type': 'backtrace', 'id': 2714493690, 'ino': 1099511628189, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/009'}, {'damage_type': 'backtrace', 'id': 2853027489, 'ino': 1099511628288, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/056'}, {'damage_type': 'backtrace', 'id': 3061654771, 'ino': 1099511628225, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/027'}, {'damage_type': 'backtrace', 'id': 3378493807, 'ino': 1099511628273, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/051'}, {'damage_type': 'backtrace', 'id': 3680175837, 'ino': 1099511628179, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/004'}, {'damage_type': 'backtrace', 'id': 3772566160, 'ino': 1099511628218, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/023.out'}, {'damage_type': 'backtrace', 'id': 3782971795, 'ino': 1099511628200, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/014.out'}, {'damage_type': 'backtrace', 'id': 3789597338, 'ino': 1099511629825, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/066'}, {'damage_type': 'backtrace', 'id': 3951329273, 'ino': 1099511628272, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/050.out'}, {'damage_type': 'backtrace', 'id': 4060739507, 'ino': 1099511628201, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/015'}, {'damage_type': 'backtrace', 'id': 4061408592, 'ino': 1099511629829, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/068'}, {'damage_type': 'backtrace', 'id': 4070008490, 'ino': 1099511628220, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/024.out'}, {'damage_type': 'backtrace', 'id': 4076756610, 'ino': 1099511628238, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/033.out'}, {'damage_type': 'backtrace', 'id': 4101696094, 'ino': 1099511628222, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/025.out'}, {'damage_type': 'backtrace', 'id': 4270462330, 'ino': 1099511628262, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/045.out'}] 2023-12-01T01:22:31.339 ERROR:tasks.fwd_scrub:rank 1 damaged: [{'damage_type': 'backtrace', 'id': 14572913, 'ino': 1099511629837, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/072'}, {'damage_type': 'backtrace', 'id': 203980746, 'ino': 1099511629791, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/prepare-commit-msg.sample'}, {'damage_type': 'backtrace', 'id': 211808904, 'ino': 1099511628193, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/011'}, {'damage_type': 'backtrace', 'id': 235331874, 'ino': 1099511629816, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/061.out'}, {'damage_type': 'backtrace', 'id': 263176013, 'ino': 1099511627811, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/HEAD'}, {'damage_type': 'backtrace', 'id': 278320937, 'ino': 1099511629803, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/push-to-checkout.sample'}, {'damage_type': 'backtrace', 'id': 293252552, 'ino': 1099511629809, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/058'}, {'damage_type': 'backtrace', 'id': 352602362, 'ino': 1099511629818, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/062.out'}, {'damage_type': 'backtrace', 'id': 410053489, 'ino': 1099511629802, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/pre-receive.sample'}, {'damage_type': 'backtrace', 'id': 436778939, 'ino': 1099511628289, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/056.out'}, {'damage_type': 'backtrace', 'id': 477235739, 'ino': 1099511628278, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/053.out'}, {'damage_type': 'backtrace', 'id': 555654069, 'ino': 1099511628186, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/007.out'}, {'damage_type': 'backtrace', 'id': 557636540, 'ino': 1099511629799, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/fsmonitor-watchman.sample'}, {'damage_type': 'backtrace', 'id': 658139213, 'ino': 1099511629826, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/066.out'}, {'damage_type': 'backtrace', 'id': 660664312, 'ino': 1099511628209, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/019'}, {'damage_type': 'backtrace', 'id': 675523175, 'ino': 1099511628215, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/022'}, {'damage_type': 'backtrace', 'id': 734519730, 'ino': 1099511627815, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/config'}, {'damage_type': 'backtrace', 'id': 774827360, 'ino': 1099511628187, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/008'}, {'damage_type': 'backtrace', 'id': 836908857, 'ino': 1099511629800, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/pre-applypatch.sample'}, {'damage_type': 'backtrace', 'id': 841532776, 'ino': 1099511628279, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/054'}, {'damage_type': 'backtrace', 'id': 874423599, 'ino': 1099511629806, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/refs/tags'}, {'damage_type': 'backtrace', 'id': 1007783187, 'ino': 1099511629287, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/description'}, {'damage_type': 'backtrace', 'id': 1025184374, 'ino': 1099511627786, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/objects'}, {'damage_type': 'backtrace', 'id': 1061002300, 'ino': 1099511629286, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/branches'}, {'damage_type': 'backtrace', 'id': 1064816424, 'ino': 1099511628221, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/025'}, {'damage_type': 'backtrace', 'id': 1067250946, 'ino': 1099511627800, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/refs/remotes'}, {'damage_type': 'backtrace', 'id': 1104971984, 'ino': 1099511628191, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/010'}, {'damage_type': 'backtrace', 'id': 1119684513, 'ino': 1099511629792, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/pre-push.sample'}, {'damage_type': 'backtrace', 'id': 1124264560, 'ino': 1099511627797, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/shallow'}, {'damage_type': 'backtrace', 'id': 1242214721, 'ino': 1099511629797, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/post-update.sample'}, {'damage_type': 'backtrace', 'id': 1244434151, 'ino': 1099511629827, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/067'}, {'damage_type': 'backtrace', 'id': 1259009207, 'ino': 1099511627803, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs'}, {'damage_type': 'backtrace', 'id': 1422637408, 'ino': 1099511628261, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/045'}, {'damage_type': 'backtrace', 'id': 1428076455, 'ino': 1099511629804, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/refs'}, {'damage_type': 'backtrace', 'id': 1695969736, 'ino': 1099511629841, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/074'}, {'damage_type': 'backtrace', 'id': 1761071705, 'ino': 1099511629795, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/commit-msg.sample'}, {'damage_type': 'backtrace', 'id': 1805161389, 'ino': 1099511629796, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/pre-rebase.sample'}, {'damage_type': 'backtrace', 'id': 1877675049, 'ino': 1099511628176, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/002.out'}, {'damage_type': 'backtrace', 'id': 1890627858, 'ino': 1099511627802, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/refs/remotes/origin/HEAD'}, {'damage_type': 'backtrace', 'id': 1998787905, 'ino': 1099511629801, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/applypatch-msg.sample'}, {'damage_type': 'backtrace', 'id': 2067168348, 'ino': 1099511627794, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/objects/pack/pack-fe9e56eb4d11faa0800e85314d6db62e368e37e2.pack'}, {'damage_type': 'backtrace', 'id': 2067441283, 'ino': 1099511628267, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/048'}, {'damage_type': 'backtrace', 'id': 2314649064, 'ino': 1099511627808, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/HEAD'}, {'damage_type': 'backtrace', 'id': 2316055598, 'ino': 1099511629793, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/pre-commit.sample'}, {'damage_type': 'backtrace', 'id': 2418999225, 'ino': 1099511629790, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks'}, {'damage_type': 'backtrace', 'id': 2574389908, 'ino': 1099511629815, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/061'}, {'damage_type': 'backtrace', 'id': 2642785167, 'ino': 1099511627804, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/refs'}, {'damage_type': 'backtrace', 'id': 2671866383, 'ino': 1099511629805, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/refs/heads'}, {'damage_type': 'backtrace', 'id': 2757735776, 'ino': 1099511629794, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/update.sample'}, {'damage_type': 'backtrace', 'id': 2773009882, 'ino': 1099511627813, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/refs/heads/master'}, {'damage_type': 'backtrace', 'id': 2838053543, 'ino': 1099511627787, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/objects/pack'}, {'damage_type': 'backtrace', 'id': 2841150211, 'ino': 1099511629813, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/060'}, {'damage_type': 'backtrace', 'id': 2844842988, 'ino': 1099511627788, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/objects/info'}, {'damage_type': 'backtrace', 'id': 2847010026, 'ino': 1099511629838, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/072.out'}, {'damage_type': 'backtrace', 'id': 2892705002, 'ino': 1099511627812, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/refs/heads'}, {'damage_type': 'backtrace', 'id': 3086244248, 'ino': 1099511628247, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/038'}, {'damage_type': 'backtrace', 'id': 3161524395, 'ino': 1099511627799, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/packed-refs'}, {'damage_type': 'backtrace', 'id': 3241200341, 'ino': 1099511628219, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/024'}, {'damage_type': 'backtrace', 'id': 3464379870, 'ino': 1099511628198, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/013.out'}, {'damage_type': 'backtrace', 'id': 3522942619, 'ino': 1099511627806, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/refs/remotes/origin'}, {'damage_type': 'backtrace', 'id': 3544605118, 'ino': 1099511628243, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/036'}, {'damage_type': 'backtrace', 'id': 3555931254, 'ino': 1099511628226, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/027.out'}, {'damage_type': 'backtrace', 'id': 3581624321, 'ino': 1099511629789, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/info/exclude'}, {'damage_type': 'backtrace', 'id': 3597285760, 'ino': 1099511629820, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/063.out'}, {'damage_type': 'backtrace', 'id': 3651198720, 'ino': 1099511627810, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/refs/heads/master'}, {'damage_type': 'backtrace', 'id': 3786557964, 'ino': 1099511629788, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/info'}, {'damage_type': 'backtrace', 'id': 3929051858, 'ino': 1099511629830, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/068.out'}, {'damage_type': 'backtrace', 'id': 3959738088, 'ino': 1099511628290, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/057'}, {'damage_type': 'backtrace', 'id': 4006946962, 'ino': 1099511628270, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/049.out'}, {'damage_type': 'backtrace', 'id': 4074317491, 'ino': 1099511628280, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/054.out'}, {'damage_type': 'backtrace', 'id': 4095850631, 'ino': 1099511629819, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/063'}, {'damage_type': 'backtrace', 'id': 4129769534, 'ino': 1099511628286, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/055'}, {'damage_type': 'backtrace', 'id': 4141656461, 'ino': 1099511628244, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/036.out'}, {'damage_type': 'backtrace', 'id': 4143748041, 'ino': 1099511628287, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/055.out'}, {'damage_type': 'backtrace', 'id': 4168634955, 'ino': 1099511629798, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/hooks/pre-merge-commit.sample'}, {'damage_type': 'backtrace', 'id': 4184368615, 'ino': 1099511627807, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/refs/remotes/origin/HEAD'}, {'damage_type': 'backtrace', 'id': 4215518723, 'ino': 1099511627795, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/objects/pack/pack-fe9e56eb4d11faa0800e85314d6db62e368e37e2.idx'}, {'damage_type': 'backtrace', 'id': 4215586282, 'ino': 1099511627801, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/refs/remotes/origin'}, {'damage_type': 'backtrace', 'id': 4245207642, 'ino': 1099511627805, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/.git/logs/refs/remotes'}, {'damage_type': 'backtrace', 'id': 4262423772, 'ino': 1099511628291, 'path': '/volumes/_nogroup/sv_1/04fb43e4-18d1-43d3-aaf6-e1b2ee3ebc39/client.0/tmp/xfstests-dev/tests/btrfs/057.out'}] 2023-12-01T01:22:31.339 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_a6f82997ac57b70a895455dfed6256360a1e4c32/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/github.com_ceph_ceph-c_a6f82997ac57b70a895455dfed6256360a1e4c32/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/github.com_ceph_ceph-c_a6f82997ac57b70a895455dfed6256360a1e4c32/qa/tasks/fwd_scrub.py", line 79, in _scrub self._check_damage() File "/home/teuthworker/src/github.com_ceph_ceph-c_a6f82997ac57b70a895455dfed6256360a1e4c32/qa/tasks/fwd_scrub.py", line 90, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2023-12-01T01:22:31.340 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_a6f82997ac57b70a895455dfed6256360a1e4c32/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/github.com_ceph_ceph-c_a6f82997ac57b70a895455dfed6256360a1e4c32/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2023-12-01T01:22:31.340 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-12-01T01:22:31.352 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:22:31.477 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.473+0000 7f9112e6d700 1 -- 172.21.15.77:0/2468947213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f910c0ffc70 msgr2=0x7f910c100090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:31.477 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.473+0000 7f9112e6d700 1 --2- 172.21.15.77:0/2468947213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f910c0ffc70 0x7f910c100090 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f91000099c0 tx=0x7f9100009cd0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.477 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.473+0000 7f9112e6d700 1 -- 172.21.15.77:0/2468947213 shutdown_connections 2023-12-01T01:22:31.477 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.473+0000 7f9112e6d700 1 --2- 172.21.15.77:0/2468947213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f910c101800 0x7f910c1080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.477 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.473+0000 7f9112e6d700 1 --2- 172.21.15.77:0/2468947213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f910c0ffc70 0x7f910c100090 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.477 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.473+0000 7f9112e6d700 1 --2- 172.21.15.77:0/2468947213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f910c100e60 0x7f910c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.477 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.473+0000 7f9112e6d700 1 -- 172.21.15.77:0/2468947213 >> 172.21.15.77:0/2468947213 conn(0x7f910c0fb1c0 msgr2=0x7f910c0fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:31.477 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.473+0000 7f9112e6d700 1 -- 172.21.15.77:0/2468947213 shutdown_connections 2023-12-01T01:22:31.477 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.473+0000 7f9112e6d700 1 -- 172.21.15.77:0/2468947213 wait complete. 2023-12-01T01:22:31.477 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.474+0000 7f9112e6d700 1 Processor -- start 2023-12-01T01:22:31.477 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.474+0000 7f9112e6d700 1 -- start start 2023-12-01T01:22:31.478 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.474+0000 7f9112e6d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f910c0ffc70 0x7f910c103c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.478 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.474+0000 7f9112e6d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f910c100e60 0x7f910c102240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.478 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.474+0000 7f9112e6d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f910c101800 0x7f910c102780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.478 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.474+0000 7f9112e6d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910c102f20 con 0x7f910c0ffc70 2023-12-01T01:22:31.478 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.474+0000 7f9112e6d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910c103090 con 0x7f910c100e60 2023-12-01T01:22:31.478 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.474+0000 7f9112e6d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910c1a0ab0 con 0x7f910c101800 2023-12-01T01:22:31.478 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.474+0000 7f9110c09700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f910c0ffc70 0x7f910c103c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:31.478 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.474+0000 7f9110c09700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f910c0ffc70 0x7f910c103c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33126/0 (socket says 172.21.15.77:33126) 2023-12-01T01:22:31.478 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.474+0000 7f9110c09700 1 -- 172.21.15.77:0/154956140 learned_addr learned my addr 172.21.15.77:0/154956140 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:22:31.478 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.474+0000 7f910bfff700 1 --2- 172.21.15.77:0/154956140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f910c100e60 0x7f910c102240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:31.478 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.474+0000 7f911140a700 1 --2- 172.21.15.77:0/154956140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f910c101800 0x7f910c102780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:31.478 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.474+0000 7f9110c09700 1 -- 172.21.15.77:0/154956140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f910c101800 msgr2=0x7f910c102780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:31.478 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.474+0000 7f9110c09700 1 --2- 172.21.15.77:0/154956140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f910c101800 0x7f910c102780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.478 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.474+0000 7f9110c09700 1 -- 172.21.15.77:0/154956140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f910c100e60 msgr2=0x7f910c102240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:31.479 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.474+0000 7f9110c09700 1 --2- 172.21.15.77:0/154956140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f910c100e60 0x7f910c102240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.479 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.474+0000 7f9110c09700 1 -- 172.21.15.77:0/154956140 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91000095f0 con 0x7f910c0ffc70 2023-12-01T01:22:31.479 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.474+0000 7f910bfff700 1 --2- 172.21.15.77:0/154956140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f910c100e60 0x7f910c102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:22:31.479 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.475+0000 7f9110c09700 1 --2- 172.21.15.77:0/154956140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f910c0ffc70 0x7f910c103c00 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f9100009f50 tx=0x7f91000057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:31.479 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.475+0000 7f9109ffb700 1 -- 172.21.15.77:0/154956140 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910001d070 con 0x7f910c0ffc70 2023-12-01T01:22:31.479 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.475+0000 7f9109ffb700 1 -- 172.21.15.77:0/154956140 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9100004110 con 0x7f910c0ffc70 2023-12-01T01:22:31.479 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.475+0000 7f9112e6d700 1 -- 172.21.15.77:0/154956140 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f910c1a0d30 con 0x7f910c0ffc70 2023-12-01T01:22:31.479 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.475+0000 7f9109ffb700 1 -- 172.21.15.77:0/154956140 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9100022dd0 con 0x7f910c0ffc70 2023-12-01T01:22:31.479 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.475+0000 7f9112e6d700 1 -- 172.21.15.77:0/154956140 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f910c1a1190 con 0x7f910c0ffc70 2023-12-01T01:22:31.480 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.476+0000 7f9112e6d700 1 -- 172.21.15.77:0/154956140 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f910c10b480 con 0x7f910c0ffc70 2023-12-01T01:22:31.480 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.477+0000 7f9109ffb700 1 -- 172.21.15.77:0/154956140 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f9100022500 con 0x7f910c0ffc70 2023-12-01T01:22:31.480 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.478+0000 7f9109ffb700 1 --2- 172.21.15.77:0/154956140 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f90fc0a0570 0x7f90fc0a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.480 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.478+0000 7f910bfff700 1 --2- 172.21.15.77:0/154956140 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f90fc0a0570 0x7f90fc0a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:31.480 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.478+0000 7f9109ffb700 1 -- 172.21.15.77:0/154956140 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f91000ccde0 con 0x7f910c0ffc70 2023-12-01T01:22:31.480 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.478+0000 7f910bfff700 1 --2- 172.21.15.77:0/154956140 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f90fc0a0570 0x7f90fc0a2a30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f90f8005950 tx=0x7f90f80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:31.483 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.482+0000 7f9109ffb700 1 -- 172.21.15.77:0/154956140 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9100096a10 con 0x7f910c0ffc70 2023-12-01T01:22:31.677 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.676+0000 7f9112e6d700 1 -- 172.21.15.77:0/154956140 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f910c1a1800 con 0x7f910c0ffc70 2023-12-01T01:22:31.679 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.678+0000 7f9109ffb700 1 -- 172.21.15.77:0/154956140 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22472 (secure 0 0 0) 0x7f910009a030 con 0x7f910c0ffc70 2023-12-01T01:22:31.679 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:22:31.680 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":84,"fsid":"2a06e29a-8fe6-11ee-95a2-87774f69a715","created":"2023-12-01T01:09:30.210398+0000","modified":"2023-12-01T01:17:25.609766+0000","last_up_change":"2023-12-01T01:15:44.586092+0000","last_in_change":"2023-12-01T01:15:26.305439+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:12:30.514229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:16:23.568860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-01T01:16:24.531137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fc651e62-a103-4ab8-a76d-d3708ea6aeec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6803","nonce":2244799094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6805","nonce":2244799094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6809","nonce":2244799094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6807","nonce":2244799094}]},"public_addr":"172.21.15.77:6803/2244799094","cluster_addr":"172.21.15.77:6805/2244799094","heartbeat_back_addr":"172.21.15.77:6809/2244799094","heartbeat_front_addr":"172.21.15.77:6807/2244799094","state":["exists","up"]},{"osd":1,"uuid":"0adf8c54-49ec-49a0-8bf1-af1f20e60729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6801","nonce":224194890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6803","nonce":224194890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6807","nonce":224194890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6805","nonce":224194890}]},"public_addr":"172.21.15.137:6801/224194890","cluster_addr":"172.21.15.137:6803/224194890","heartbeat_back_addr":"172.21.15.137:6807/224194890","heartbeat_front_addr":"172.21.15.137:6805/224194890","state":["exists","up"]},{"osd":2,"uuid":"5a69802c-dede-4544-9a12-436f5c1c0038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6801","nonce":4078459110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6803","nonce":4078459110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6807","nonce":4078459110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6805","nonce":4078459110}]},"public_addr":"172.21.15.203:6801/4078459110","cluster_addr":"172.21.15.203:6803/4078459110","heartbeat_back_addr":"172.21.15.203:6807/4078459110","heartbeat_front_addr":"172.21.15.203:6805/4078459110","state":["exists","up"]},{"osd":3,"uuid":"ad325a97-580b-4c99-bf12-cbb0d92a6d0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6811","nonce":1454440477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6813","nonce":1454440477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6817","nonce":1454440477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6815","nonce":1454440477}]},"public_addr":"172.21.15.77:6811/1454440477","cluster_addr":"172.21.15.77:6813/1454440477","heartbeat_back_addr":"172.21.15.77:6817/1454440477","heartbeat_front_addr":"172.21.15.77:6815/1454440477","state":["exists","up"]},{"osd":4,"uuid":"1e3ac1ba-98bb-4b8b-8e26-50064c66bf2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6809","nonce":1103241451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6811","nonce":1103241451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6815","nonce":1103241451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6813","nonce":1103241451}]},"public_addr":"172.21.15.137:6809/1103241451","cluster_addr":"172.21.15.137:6811/1103241451","heartbeat_back_addr":"172.21.15.137:6815/1103241451","heartbeat_front_addr":"172.21.15.137:6813/1103241451","state":["exists","up"]},{"osd":5,"uuid":"eb32c016-abeb-44ab-abf0-19dd09235736","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6809","nonce":2648569482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6811","nonce":2648569482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6815","nonce":2648569482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6813","nonce":2648569482}]},"public_addr":"172.21.15.203:6809/2648569482","cluster_addr":"172.21.15.203:6811/2648569482","heartbeat_back_addr":"172.21.15.203:6815/2648569482","heartbeat_front_addr":"172.21.15.203:6813/2648569482","state":["exists","up"]},{"osd":6,"uuid":"0e3fa97e-2ca6-418c-9e8d-a6fe3c56adb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6819","nonce":2463414651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6821","nonce":2463414651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6825","nonce":2463414651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6823","nonce":2463414651}]},"public_addr":"172.21.15.77:6819/2463414651","cluster_addr":"172.21.15.77:6821/2463414651","heartbeat_back_addr":"172.21.15.77:6825/2463414651","heartbeat_front_addr":"172.21.15.77:6823/2463414651","state":["exists","up"]},{"osd":7,"uuid":"d7398c01-0f98-4138-8a49-c3ff0aae7e88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6817","nonce":3108330766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6819","nonce":3108330766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6823","nonce":3108330766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6821","nonce":3108330766}]},"public_addr":"172.21.15.137:6817/3108330766","cluster_addr":"172.21.15.137:6819/3108330766","heartbeat_back_addr":"172.21.15.137:6823/3108330766","heartbeat_front_addr":"172.21.15.137:6821/3108330766","state":["exists","up"]},{"osd":8,"uuid":"0c09050d-3e43-410c-8cd7-bcb891af56b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6817","nonce":1161158776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6819","nonce":1161158776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6823","nonce":1161158776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6821","nonce":1161158776}]},"public_addr":"172.21.15.203:6817/1161158776","cluster_addr":"172.21.15.203:6819/1161158776","heartbeat_back_addr":"172.21.15.203:6823/1161158776","heartbeat_front_addr":"172.21.15.203:6821/1161158776","state":["exists","up"]},{"osd":9,"uuid":"527ecb49-95ec-48bd-9872-3b07fae5c5ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6827","nonce":2409880685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6829","nonce":2409880685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6833","nonce":2409880685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6831","nonce":2409880685}]},"public_addr":"172.21.15.77:6827/2409880685","cluster_addr":"172.21.15.77:6829/2409880685","heartbeat_back_addr":"172.21.15.77:6833/2409880685","heartbeat_front_addr":"172.21.15.77:6831/2409880685","state":["exists","up"]},{"osd":10,"uuid":"1680bb3e-a3c8-427b-af68-e2214f32bed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6825","nonce":374920146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6827","nonce":374920146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6831","nonce":374920146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6829","nonce":374920146}]},"public_addr":"172.21.15.137:6825/374920146","cluster_addr":"172.21.15.137:6827/374920146","heartbeat_back_addr":"172.21.15.137:6831/374920146","heartbeat_front_addr":"172.21.15.137:6829/374920146","state":["exists","up"]},{"osd":11,"uuid":"8a533466-023d-4803-ae36-9a9f1715e49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6825","nonce":2161588962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6827","nonce":2161588962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6831","nonce":2161588962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6829","nonce":2161588962}]},"public_addr":"172.21.15.203:6825/2161588962","cluster_addr":"172.21.15.203:6827/2161588962","heartbeat_back_addr":"172.21.15.203:6831/2161588962","heartbeat_front_addr":"172.21.15.203:6829/2161588962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:11:45.420509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:06.210115+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:28.191307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:49.594116+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:11.261689+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:33.376541+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:54.968336+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:16.775310+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:38.733792+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:59.456699+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:15:20.924059+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:15:43.094773+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/2260035254":"2023-12-02T01:10:24.191835+0000","172.21.15.77:6800/1456621747":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/219838828":"2023-12-02T01:10:24.191835+0000","172.21.15.77:6801/1456621747":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/2937149266":"2023-12-02T01:10:02.232948+0000","172.21.15.77:0/807520018":"2023-12-02T01:10:02.232948+0000","172.21.15.77:6801/1661788026":"2023-12-02T01:10:02.232948+0000","172.21.15.77:6800/1661788026":"2023-12-02T01:10:02.232948+0000","172.21.15.77:0/3551282473":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/1202222558":"2023-12-02T01:10:02.232948+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:22:31.682 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.680+0000 7f9112e6d700 1 -- 172.21.15.77:0/154956140 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f90fc0a0570 msgr2=0x7f90fc0a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:31.682 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.680+0000 7f9112e6d700 1 --2- 172.21.15.77:0/154956140 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f90fc0a0570 0x7f90fc0a2a30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f90f8005950 tx=0x7f90f80058e0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.682 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.680+0000 7f9112e6d700 1 -- 172.21.15.77:0/154956140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f910c0ffc70 msgr2=0x7f910c103c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:31.682 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.680+0000 7f9112e6d700 1 --2- 172.21.15.77:0/154956140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f910c0ffc70 0x7f910c103c00 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f9100009f50 tx=0x7f91000057d0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.682 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.680+0000 7f9112e6d700 1 -- 172.21.15.77:0/154956140 shutdown_connections 2023-12-01T01:22:31.682 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.680+0000 7f9112e6d700 1 --2- 172.21.15.77:0/154956140 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f90fc0a0570 0x7f90fc0a2a30 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.682 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.680+0000 7f9112e6d700 1 --2- 172.21.15.77:0/154956140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f910c101800 0x7f910c102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.682 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.680+0000 7f9112e6d700 1 --2- 172.21.15.77:0/154956140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f910c0ffc70 0x7f910c103c00 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.682 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.680+0000 7f9112e6d700 1 --2- 172.21.15.77:0/154956140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f910c100e60 0x7f910c102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.682 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.680+0000 7f9112e6d700 1 -- 172.21.15.77:0/154956140 >> 172.21.15.77:0/154956140 conn(0x7f910c0fb1c0 msgr2=0x7f910c067f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:31.682 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.680+0000 7f9112e6d700 1 -- 172.21.15.77:0/154956140 shutdown_connections 2023-12-01T01:22:31.682 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.680+0000 7f9112e6d700 1 -- 172.21.15.77:0/154956140 wait complete. 2023-12-01T01:22:31.693 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:31 smithi077 ceph-mon[98867]: pgmap v437: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 12 op/s 2023-12-01T01:22:31.693 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:31 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/3804314151' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:22:31.695 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:12:30.514229+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T01:16:23.568860+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T01:16:24.531137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:22:31.695 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:22:31.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:31 smithi203 ceph-mon[103581]: pgmap v437: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 12 op/s 2023-12-01T01:22:31.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:31 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/3804314151' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:22:31.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:31 smithi137 ceph-mon[103312]: pgmap v437: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 12 op/s 2023-12-01T01:22:31.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:31 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/3804314151' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:22:31.822 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.818+0000 7f3babaa9700 1 -- 172.21.15.77:0/4132529932 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ba4101880 msgr2=0x7f3ba4108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:31.822 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.818+0000 7f3babaa9700 1 --2- 172.21.15.77:0/4132529932 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ba4101880 0x7f3ba4108130 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c00b1d0 tx=0x7f3b9c00b4e0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.822 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.818+0000 7f3babaa9700 1 -- 172.21.15.77:0/4132529932 shutdown_connections 2023-12-01T01:22:31.822 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.818+0000 7f3babaa9700 1 --2- 172.21.15.77:0/4132529932 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ba4101880 0x7f3ba4108130 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.822 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.818+0000 7f3babaa9700 1 --2- 172.21.15.77:0/4132529932 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ba40ffcf0 0x7f3ba4100110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.822 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.818+0000 7f3babaa9700 1 --2- 172.21.15.77:0/4132529932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ba4100ee0 0x7f3ba4101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.822 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.818+0000 7f3babaa9700 1 -- 172.21.15.77:0/4132529932 >> 172.21.15.77:0/4132529932 conn(0x7f3ba40fb260 msgr2=0x7f3ba40fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:31.822 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.818+0000 7f3babaa9700 1 -- 172.21.15.77:0/4132529932 shutdown_connections 2023-12-01T01:22:31.822 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.818+0000 7f3babaa9700 1 -- 172.21.15.77:0/4132529932 wait complete. 2023-12-01T01:22:31.822 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.819+0000 7f3babaa9700 1 Processor -- start 2023-12-01T01:22:31.822 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.819+0000 7f3babaa9700 1 -- start start 2023-12-01T01:22:31.822 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.819+0000 7f3babaa9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ba40ffcf0 0x7f3ba4194800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.823 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.819+0000 7f3babaa9700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ba4100ee0 0x7f3ba4194d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.823 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.819+0000 7f3babaa9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ba4101880 0x7f3ba419bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.823 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.819+0000 7f3babaa9700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba419c380 con 0x7f3ba40ffcf0 2023-12-01T01:22:31.823 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.819+0000 7f3babaa9700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba419c4c0 con 0x7f3ba4100ee0 2023-12-01T01:22:31.823 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.819+0000 7f3babaa9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba419c650 con 0x7f3ba4101880 2023-12-01T01:22:31.823 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.819+0000 7f3ba9845700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ba40ffcf0 0x7f3ba4194800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:31.823 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.819+0000 7f3ba9845700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ba40ffcf0 0x7f3ba4194800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33140/0 (socket says 172.21.15.77:33140) 2023-12-01T01:22:31.823 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.819+0000 7f3ba9845700 1 -- 172.21.15.77:0/892062394 learned_addr learned my addr 172.21.15.77:0/892062394 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:22:31.823 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.819+0000 7f3baa046700 1 --2- 172.21.15.77:0/892062394 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ba4101880 0x7f3ba419bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:31.823 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.819+0000 7f3ba9044700 1 --2- 172.21.15.77:0/892062394 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ba4100ee0 0x7f3ba4194d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:31.823 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.819+0000 7f3ba9845700 1 -- 172.21.15.77:0/892062394 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ba4101880 msgr2=0x7f3ba419bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:31.823 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.819+0000 7f3ba9845700 1 --2- 172.21.15.77:0/892062394 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ba4101880 0x7f3ba419bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.823 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.819+0000 7f3ba9845700 1 -- 172.21.15.77:0/892062394 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ba4100ee0 msgr2=0x7f3ba4194d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:31.824 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.819+0000 7f3ba9845700 1 --2- 172.21.15.77:0/892062394 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ba4100ee0 0x7f3ba4194d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:31.824 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.819+0000 7f3ba9845700 1 -- 172.21.15.77:0/892062394 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b9c009d00 con 0x7f3ba40ffcf0 2023-12-01T01:22:31.824 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.819+0000 7f3ba9845700 1 --2- 172.21.15.77:0/892062394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ba40ffcf0 0x7f3ba4194800 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f3ba000d980 tx=0x7f3ba000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:31.824 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.820+0000 7f3b9affd700 1 -- 172.21.15.77:0/892062394 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0004020 con 0x7f3ba40ffcf0 2023-12-01T01:22:31.824 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.820+0000 7f3b9affd700 1 -- 172.21.15.77:0/892062394 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ba0009730 con 0x7f3ba40ffcf0 2023-12-01T01:22:31.824 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.820+0000 7f3babaa9700 1 -- 172.21.15.77:0/892062394 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ba419c930 con 0x7f3ba40ffcf0 2023-12-01T01:22:31.824 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.820+0000 7f3b9affd700 1 -- 172.21.15.77:0/892062394 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0003a80 con 0x7f3ba40ffcf0 2023-12-01T01:22:31.824 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.820+0000 7f3babaa9700 1 -- 172.21.15.77:0/892062394 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba419ce40 con 0x7f3ba40ffcf0 2023-12-01T01:22:31.825 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.821+0000 7f3babaa9700 1 -- 172.21.15.77:0/892062394 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba4107210 con 0x7f3ba40ffcf0 2023-12-01T01:22:31.825 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.822+0000 7f3b9affd700 1 -- 172.21.15.77:0/892062394 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f3ba00098a0 con 0x7f3ba40ffcf0 2023-12-01T01:22:31.825 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.822+0000 7f3b9affd700 1 --2- 172.21.15.77:0/892062394 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f3b900a0520 0x7f3b900a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:31.826 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.823+0000 7f3ba9044700 1 --2- 172.21.15.77:0/892062394 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f3b900a0520 0x7f3b900a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:31.826 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.823+0000 7f3b9affd700 1 -- 172.21.15.77:0/892062394 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f3ba00cb610 con 0x7f3ba40ffcf0 2023-12-01T01:22:31.826 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.823+0000 7f3ba9044700 1 --2- 172.21.15.77:0/892062394 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f3b900a0520 0x7f3b900a29e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3b94005fd0 tx=0x7f3b94005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:31.828 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:31.827+0000 7f3b9affd700 1 -- 172.21.15.77:0/892062394 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3ba0095250 con 0x7f3ba40ffcf0 2023-12-01T01:22:32.018 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.017+0000 7f3babaa9700 1 -- 172.21.15.77:0/892062394 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3ba402cee0 con 0x7f3ba40ffcf0 2023-12-01T01:22:32.018 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.017+0000 7f3b9affd700 1 -- 172.21.15.77:0/892062394 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3ba0098870 con 0x7f3ba40ffcf0 2023-12-01T01:22:32.019 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 1 2023-12-01T01:22:32.021 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.019+0000 7f3babaa9700 1 -- 172.21.15.77:0/892062394 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f3b900a0520 msgr2=0x7f3b900a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:32.021 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.019+0000 7f3babaa9700 1 --2- 172.21.15.77:0/892062394 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f3b900a0520 0x7f3b900a29e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3b94005fd0 tx=0x7f3b94005f60 comp rx=0 tx=0).stop 2023-12-01T01:22:32.021 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.019+0000 7f3babaa9700 1 -- 172.21.15.77:0/892062394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ba40ffcf0 msgr2=0x7f3ba4194800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:32.021 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.019+0000 7f3babaa9700 1 --2- 172.21.15.77:0/892062394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ba40ffcf0 0x7f3ba4194800 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f3ba000d980 tx=0x7f3ba000dc90 comp rx=0 tx=0).stop 2023-12-01T01:22:32.021 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.019+0000 7f3babaa9700 1 -- 172.21.15.77:0/892062394 shutdown_connections 2023-12-01T01:22:32.021 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.019+0000 7f3babaa9700 1 --2- 172.21.15.77:0/892062394 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f3b900a0520 0x7f3b900a29e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:32.021 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.019+0000 7f3babaa9700 1 --2- 172.21.15.77:0/892062394 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ba4101880 0x7f3ba419bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:32.021 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.019+0000 7f3babaa9700 1 --2- 172.21.15.77:0/892062394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ba40ffcf0 0x7f3ba4194800 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:32.021 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.019+0000 7f3babaa9700 1 --2- 172.21.15.77:0/892062394 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ba4100ee0 0x7f3ba4194d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:32.021 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.019+0000 7f3babaa9700 1 -- 172.21.15.77:0/892062394 >> 172.21.15.77:0/892062394 conn(0x7f3ba40fb260 msgr2=0x7f3ba4104350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:32.022 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.020+0000 7f3babaa9700 1 -- 172.21.15.77:0/892062394 shutdown_connections 2023-12-01T01:22:32.022 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.020+0000 7f3babaa9700 1 -- 172.21.15.77:0/892062394 wait complete. 2023-12-01T01:22:32.033 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T01:22:32.160 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.156+0000 7fc482156700 1 -- 172.21.15.77:0/3836969959 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc47c101290 msgr2=0x7fc47c1016b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:32.160 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.156+0000 7fc482156700 1 --2- 172.21.15.77:0/3836969959 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc47c101290 0x7fc47c1016b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fc4700099c0 tx=0x7fc470009cd0 comp rx=0 tx=0).stop 2023-12-01T01:22:32.160 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.156+0000 7fc482156700 1 -- 172.21.15.77:0/3836969959 shutdown_connections 2023-12-01T01:22:32.160 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.156+0000 7fc482156700 1 --2- 172.21.15.77:0/3836969959 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc47c100830 0x7fc47c1080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:32.160 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.156+0000 7fc482156700 1 --2- 172.21.15.77:0/3836969959 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc47c101290 0x7fc47c1016b0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:32.160 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.156+0000 7fc482156700 1 --2- 172.21.15.77:0/3836969959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc47c0ffe90 0x7fc47c1002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:32.160 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.156+0000 7fc482156700 1 -- 172.21.15.77:0/3836969959 >> 172.21.15.77:0/3836969959 conn(0x7fc47c0fa980 msgr2=0x7fc47c0fcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:32.160 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.156+0000 7fc482156700 1 -- 172.21.15.77:0/3836969959 shutdown_connections 2023-12-01T01:22:32.161 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.156+0000 7fc482156700 1 -- 172.21.15.77:0/3836969959 wait complete. 2023-12-01T01:22:32.161 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.157+0000 7fc482156700 1 Processor -- start 2023-12-01T01:22:32.161 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.157+0000 7fc482156700 1 -- start start 2023-12-01T01:22:32.161 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.157+0000 7fc482156700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc47c0ffe90 0x7fc47c0ff910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:32.161 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.157+0000 7fc482156700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc47c100830 0x7fc47c19dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:32.161 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.157+0000 7fc482156700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc47c101290 0x7fc47c1a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:32.161 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.157+0000 7fc482156700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c1a0820 con 0x7fc47c0ffe90 2023-12-01T01:22:32.161 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.157+0000 7fc482156700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c1a0990 con 0x7fc47c100830 2023-12-01T01:22:32.161 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.157+0000 7fc482156700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c1a0b50 con 0x7fc47c101290 2023-12-01T01:22:32.161 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.157+0000 7fc47b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc47c0ffe90 0x7fc47c0ff910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:32.161 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.157+0000 7fc47b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc47c0ffe90 0x7fc47c0ff910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33154/0 (socket says 172.21.15.77:33154) 2023-12-01T01:22:32.161 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.157+0000 7fc47b7fe700 1 -- 172.21.15.77:0/3889522151 learned_addr learned my addr 172.21.15.77:0/3889522151 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:22:32.161 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.158+0000 7fc47bfff700 1 --2- 172.21.15.77:0/3889522151 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc47c101290 0x7fc47c1a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:32.161 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.158+0000 7fc47affd700 1 --2- 172.21.15.77:0/3889522151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc47c100830 0x7fc47c19dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:32.162 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.158+0000 7fc47b7fe700 1 -- 172.21.15.77:0/3889522151 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc47c101290 msgr2=0x7fc47c1a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:32.162 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.158+0000 7fc47b7fe700 1 --2- 172.21.15.77:0/3889522151 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc47c101290 0x7fc47c1a0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:32.162 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.158+0000 7fc47b7fe700 1 -- 172.21.15.77:0/3889522151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc47c100830 msgr2=0x7fc47c19dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:32.162 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.158+0000 7fc47b7fe700 1 --2- 172.21.15.77:0/3889522151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc47c100830 0x7fc47c19dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:32.162 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.158+0000 7fc47b7fe700 1 -- 172.21.15.77:0/3889522151 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4700095f0 con 0x7fc47c0ffe90 2023-12-01T01:22:32.162 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.158+0000 7fc47affd700 1 --2- 172.21.15.77:0/3889522151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc47c100830 0x7fc47c19dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:22:32.162 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.158+0000 7fc47b7fe700 1 --2- 172.21.15.77:0/3889522151 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc47c0ffe90 0x7fc47c0ff910 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fc4700036f0 tx=0x7fc470009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:32.162 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.158+0000 7fc478ff9700 1 -- 172.21.15.77:0/3889522151 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47001d070 con 0x7fc47c0ffe90 2023-12-01T01:22:32.162 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.158+0000 7fc482156700 1 -- 172.21.15.77:0/3889522151 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc47c1a0dd0 con 0x7fc47c0ffe90 2023-12-01T01:22:32.162 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.158+0000 7fc478ff9700 1 -- 172.21.15.77:0/3889522151 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc470004110 con 0x7fc47c0ffe90 2023-12-01T01:22:32.162 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.158+0000 7fc482156700 1 -- 172.21.15.77:0/3889522151 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc47c1a11a0 con 0x7fc47c0ffe90 2023-12-01T01:22:32.162 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.158+0000 7fc478ff9700 1 -- 172.21.15.77:0/3889522151 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc470021410 con 0x7fc47c0ffe90 2023-12-01T01:22:32.163 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.159+0000 7fc482156700 1 -- 172.21.15.77:0/3889522151 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc47c10b580 con 0x7fc47c0ffe90 2023-12-01T01:22:32.163 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.161+0000 7fc478ff9700 1 -- 172.21.15.77:0/3889522151 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fc47000f660 con 0x7fc47c0ffe90 2023-12-01T01:22:32.163 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.161+0000 7fc478ff9700 1 --2- 172.21.15.77:0/3889522151 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc4640a0520 0x7fc4640a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:32.163 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.162+0000 7fc47affd700 1 --2- 172.21.15.77:0/3889522151 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc4640a0520 0x7fc4640a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:32.164 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.162+0000 7fc478ff9700 1 -- 172.21.15.77:0/3889522151 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fc4700cc860 con 0x7fc47c0ffe90 2023-12-01T01:22:32.164 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.162+0000 7fc47affd700 1 --2- 172.21.15.77:0/3889522151 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc4640a0520 0x7fc4640a29e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc46c005fd0 tx=0x7fc46c005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:32.165 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.164+0000 7fc478ff9700 1 -- 172.21.15.77:0/3889522151 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc47005c0e0 con 0x7fc47c0ffe90 2023-12-01T01:22:32.360 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.358+0000 7fc482156700 1 -- 172.21.15.77:0/3889522151 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc47c106bb0 con 0x7fc47c0ffe90 2023-12-01T01:22:32.360 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.359+0000 7fc478ff9700 1 -- 172.21.15.77:0/3889522151 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc470096490 con 0x7fc47c0ffe90 2023-12-01T01:22:32.360 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 32 2023-12-01T01:22:32.365 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.361+0000 7fc482156700 1 -- 172.21.15.77:0/3889522151 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc4640a0520 msgr2=0x7fc4640a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:32.365 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.361+0000 7fc482156700 1 --2- 172.21.15.77:0/3889522151 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc4640a0520 0x7fc4640a29e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc46c005fd0 tx=0x7fc46c005dc0 comp rx=0 tx=0).stop 2023-12-01T01:22:32.365 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.361+0000 7fc482156700 1 -- 172.21.15.77:0/3889522151 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc47c0ffe90 msgr2=0x7fc47c0ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:32.365 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.361+0000 7fc482156700 1 --2- 172.21.15.77:0/3889522151 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc47c0ffe90 0x7fc47c0ff910 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fc4700036f0 tx=0x7fc470009580 comp rx=0 tx=0).stop 2023-12-01T01:22:32.365 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.361+0000 7fc482156700 1 -- 172.21.15.77:0/3889522151 shutdown_connections 2023-12-01T01:22:32.366 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.361+0000 7fc482156700 1 --2- 172.21.15.77:0/3889522151 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fc4640a0520 0x7fc4640a29e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:32.366 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.361+0000 7fc482156700 1 --2- 172.21.15.77:0/3889522151 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc47c101290 0x7fc47c1a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:32.366 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.361+0000 7fc482156700 1 --2- 172.21.15.77:0/3889522151 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc47c0ffe90 0x7fc47c0ff910 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:32.366 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.361+0000 7fc482156700 1 --2- 172.21.15.77:0/3889522151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc47c100830 0x7fc47c19dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:32.366 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.361+0000 7fc482156700 1 -- 172.21.15.77:0/3889522151 >> 172.21.15.77:0/3889522151 conn(0x7fc47c0fa980 msgr2=0x7fc47c0fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:32.366 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.361+0000 7fc482156700 1 -- 172.21.15.77:0/3889522151 shutdown_connections 2023-12-01T01:22:32.366 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.361+0000 7fc482156700 1 -- 172.21.15.77:0/3889522151 wait complete. 2023-12-01T01:22:32.374 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T01:22:32.503 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.500+0000 7f23999f3700 1 -- 172.21.15.77:0/2754015508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23940ff3f0 msgr2=0x7f23940ff810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:32.503 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.500+0000 7f23999f3700 1 --2- 172.21.15.77:0/2754015508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23940ff3f0 0x7f23940ff810 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f23880099c0 tx=0x7f2388009cd0 comp rx=0 tx=0).stop 2023-12-01T01:22:32.503 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.500+0000 7f23999f3700 1 -- 172.21.15.77:0/2754015508 shutdown_connections 2023-12-01T01:22:32.503 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.500+0000 7f23999f3700 1 --2- 172.21.15.77:0/2754015508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2394100f80 0x7f2394107830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:32.504 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.500+0000 7f23999f3700 1 --2- 172.21.15.77:0/2754015508 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23940ff3f0 0x7f23940ff810 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:32.504 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.500+0000 7f23999f3700 1 --2- 172.21.15.77:0/2754015508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23941005e0 0x7f2394100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:32.504 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.500+0000 7f23999f3700 1 -- 172.21.15.77:0/2754015508 >> 172.21.15.77:0/2754015508 conn(0x7f23940fa960 msgr2=0x7f23940fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:32.504 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.500+0000 7f23999f3700 1 -- 172.21.15.77:0/2754015508 shutdown_connections 2023-12-01T01:22:32.504 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.500+0000 7f23999f3700 1 -- 172.21.15.77:0/2754015508 wait complete. 2023-12-01T01:22:32.504 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.500+0000 7f23999f3700 1 Processor -- start 2023-12-01T01:22:32.504 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.501+0000 7f23999f3700 1 -- start start 2023-12-01T01:22:32.504 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.501+0000 7f23999f3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23940ff3f0 0x7f2394196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:32.504 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.501+0000 7f23999f3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23941005e0 0x7f2394196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:32.504 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.501+0000 7f23999f3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2394100f80 0x7f239419e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:32.504 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.501+0000 7f23999f3700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239419e5f0 con 0x7f23941005e0 2023-12-01T01:22:32.504 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.501+0000 7f23999f3700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239419e730 con 0x7f2394100f80 2023-12-01T01:22:32.504 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.501+0000 7f23999f3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239419e8c0 con 0x7f23940ff3f0 2023-12-01T01:22:32.505 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.501+0000 7f23927fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23941005e0 0x7f2394196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:32.505 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.501+0000 7f23927fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23941005e0 0x7f2394196fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33170/0 (socket says 172.21.15.77:33170) 2023-12-01T01:22:32.505 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.501+0000 7f23927fc700 1 -- 172.21.15.77:0/1447368769 learned_addr learned my addr 172.21.15.77:0/1447368769 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:22:32.505 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.501+0000 7f2392ffd700 1 --2- 172.21.15.77:0/1447368769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23940ff3f0 0x7f2394196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:32.505 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.501+0000 7f23937fe700 1 --2- 172.21.15.77:0/1447368769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2394100f80 0x7f239419e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:32.505 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.501+0000 7f23927fc700 1 -- 172.21.15.77:0/1447368769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23940ff3f0 msgr2=0x7f2394196a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:32.505 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.501+0000 7f23927fc700 1 --2- 172.21.15.77:0/1447368769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23940ff3f0 0x7f2394196a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:32.505 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.501+0000 7f23927fc700 1 -- 172.21.15.77:0/1447368769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2394100f80 msgr2=0x7f239419e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:32.505 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.501+0000 7f23927fc700 1 --2- 172.21.15.77:0/1447368769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2394100f80 0x7f239419e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:32.505 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.501+0000 7f23927fc700 1 -- 172.21.15.77:0/1447368769 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23880095f0 con 0x7f23941005e0 2023-12-01T01:22:32.505 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.501+0000 7f23927fc700 1 --2- 172.21.15.77:0/1447368769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23941005e0 0x7f2394196fb0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f238400dbe0 tx=0x7f23840072a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:32.505 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.501+0000 7f23989f1700 1 -- 172.21.15.77:0/1447368769 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2384007d80 con 0x7f23941005e0 2023-12-01T01:22:32.505 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.502+0000 7f23989f1700 1 -- 172.21.15.77:0/1447368769 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23840044e0 con 0x7f23941005e0 2023-12-01T01:22:32.505 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.502+0000 7f23999f3700 1 -- 172.21.15.77:0/1447368769 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f239419eba0 con 0x7f23941005e0 2023-12-01T01:22:32.506 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.502+0000 7f23989f1700 1 -- 172.21.15.77:0/1447368769 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2384005280 con 0x7f23941005e0 2023-12-01T01:22:32.506 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.502+0000 7f23999f3700 1 -- 172.21.15.77:0/1447368769 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f239419f0b0 con 0x7f23941005e0 2023-12-01T01:22:32.506 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.503+0000 7f23999f3700 1 -- 172.21.15.77:0/1447368769 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2394106370 con 0x7f23941005e0 2023-12-01T01:22:32.507 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.503+0000 7f23989f1700 1 -- 172.21.15.77:0/1447368769 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f2384004750 con 0x7f23941005e0 2023-12-01T01:22:32.507 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.504+0000 7f23989f1700 1 --2- 172.21.15.77:0/1447368769 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f237c0a0520 0x7f237c0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:32.507 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.504+0000 7f2392ffd700 1 --2- 172.21.15.77:0/1447368769 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f237c0a0520 0x7f237c0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:32.507 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.504+0000 7f23989f1700 1 -- 172.21.15.77:0/1447368769 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f23840cb7a0 con 0x7f23941005e0 2023-12-01T01:22:32.507 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.504+0000 7f2392ffd700 1 --2- 172.21.15.77:0/1447368769 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f237c0a0520 0x7f237c0a29e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2388009990 tx=0x7f238801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:32.509 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.508+0000 7f23989f1700 1 -- 172.21.15.77:0/1447368769 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f23840953e0 con 0x7f23941005e0 2023-12-01T01:22:32.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:32 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/154956140' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:22:32.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:32 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/892062394' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-01T01:22:32.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:32 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/3889522151' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-01T01:22:32.703 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.701+0000 7f23999f3700 1 -- 172.21.15.77:0/1447368769 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f239402cc70 con 0x7f23941005e0 2023-12-01T01:22:32.703 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.702+0000 7f23989f1700 1 -- 172.21.15.77:0/1447368769 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7f238401a020 con 0x7f23941005e0 2023-12-01T01:22:32.703 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 64 2023-12-01T01:22:32.706 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.704+0000 7f23999f3700 1 -- 172.21.15.77:0/1447368769 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f237c0a0520 msgr2=0x7f237c0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:32.706 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.704+0000 7f23999f3700 1 --2- 172.21.15.77:0/1447368769 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f237c0a0520 0x7f237c0a29e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2388009990 tx=0x7f238801a040 comp rx=0 tx=0).stop 2023-12-01T01:22:32.706 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.704+0000 7f23999f3700 1 -- 172.21.15.77:0/1447368769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23941005e0 msgr2=0x7f2394196fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:32.706 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.704+0000 7f23999f3700 1 --2- 172.21.15.77:0/1447368769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23941005e0 0x7f2394196fb0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f238400dbe0 tx=0x7f23840072a0 comp rx=0 tx=0).stop 2023-12-01T01:22:32.706 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.704+0000 7f23999f3700 1 -- 172.21.15.77:0/1447368769 shutdown_connections 2023-12-01T01:22:32.706 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.704+0000 7f23999f3700 1 --2- 172.21.15.77:0/1447368769 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f237c0a0520 0x7f237c0a29e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:32.706 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.704+0000 7f23999f3700 1 --2- 172.21.15.77:0/1447368769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2394100f80 0x7f239419e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:32.706 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.704+0000 7f23999f3700 1 --2- 172.21.15.77:0/1447368769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23940ff3f0 0x7f2394196a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:32.706 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.704+0000 7f23999f3700 1 --2- 172.21.15.77:0/1447368769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23941005e0 0x7f2394196fb0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:32.706 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.704+0000 7f23999f3700 1 -- 172.21.15.77:0/1447368769 >> 172.21.15.77:0/1447368769 conn(0x7f23940fa960 msgr2=0x7f23940fcc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:32.706 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.704+0000 7f23999f3700 1 -- 172.21.15.77:0/1447368769 shutdown_connections 2023-12-01T01:22:32.707 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.704+0000 7f23999f3700 1 -- 172.21.15.77:0/1447368769 wait complete. 2023-12-01T01:22:32.718 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-12-01T01:22:32.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:32 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/154956140' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:22:32.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:32 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/892062394' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-01T01:22:32.763 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:32 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/3889522151' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-01T01:22:32.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:32 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/154956140' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:22:32.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:32 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/892062394' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-01T01:22:32.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:32 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/3889522151' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-01T01:22:32.843 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.839+0000 7f45892d1700 1 -- 172.21.15.77:0/932128916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4584100960 msgr2=0x7f4584100de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:32.843 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.839+0000 7f45892d1700 1 --2- 172.21.15.77:0/932128916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4584100960 0x7f4584100de0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f457c00b0d0 tx=0x7f457c00b3e0 comp rx=0 tx=0).stop 2023-12-01T01:22:32.843 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.839+0000 7f45892d1700 1 -- 172.21.15.77:0/932128916 shutdown_connections 2023-12-01T01:22:32.843 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.839+0000 7f45892d1700 1 --2- 172.21.15.77:0/932128916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4584100960 0x7f4584100de0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:32.843 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.839+0000 7f45892d1700 1 --2- 172.21.15.77:0/932128916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45840ff640 0x7f45840ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:32.843 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.839+0000 7f45892d1700 1 --2- 172.21.15.77:0/932128916 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45840fffa0 0x7f4584100420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:32.843 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.839+0000 7f45892d1700 1 -- 172.21.15.77:0/932128916 >> 172.21.15.77:0/932128916 conn(0x7f45840fb1c0 msgr2=0x7f45840fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:32.844 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.839+0000 7f45892d1700 1 -- 172.21.15.77:0/932128916 shutdown_connections 2023-12-01T01:22:32.844 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.839+0000 7f45892d1700 1 -- 172.21.15.77:0/932128916 wait complete. 2023-12-01T01:22:32.844 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.839+0000 7f45892d1700 1 Processor -- start 2023-12-01T01:22:32.844 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.840+0000 7f45892d1700 1 -- start start 2023-12-01T01:22:32.844 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.840+0000 7f45892d1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45840ff640 0x7f4584198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:32.844 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.840+0000 7f45892d1700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45840fffa0 0x7f4584199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:32.844 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.840+0000 7f45892d1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4584100960 0x7f45841a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:32.844 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.840+0000 7f45892d1700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45841a0840 con 0x7f45840ff640 2023-12-01T01:22:32.844 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.840+0000 7f45892d1700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45841a09b0 con 0x7f4584100960 2023-12-01T01:22:32.844 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.840+0000 7f45892d1700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45841a0b70 con 0x7f45840fffa0 2023-12-01T01:22:32.844 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.840+0000 7f4582ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45840ff640 0x7f4584198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:32.844 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.840+0000 7f4582ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45840ff640 0x7f4584198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33182/0 (socket says 172.21.15.77:33182) 2023-12-01T01:22:32.845 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.840+0000 7f4582ffd700 1 -- 172.21.15.77:0/2024898664 learned_addr learned my addr 172.21.15.77:0/2024898664 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:22:32.845 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.840+0000 7f45827fc700 1 --2- 172.21.15.77:0/2024898664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45840fffa0 0x7f4584199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:32.845 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.840+0000 7f45837fe700 1 --2- 172.21.15.77:0/2024898664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4584100960 0x7f45841a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:32.845 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.840+0000 7f4582ffd700 1 -- 172.21.15.77:0/2024898664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45840fffa0 msgr2=0x7f4584199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:32.845 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.840+0000 7f4582ffd700 1 --2- 172.21.15.77:0/2024898664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45840fffa0 0x7f4584199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:32.845 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.840+0000 7f4582ffd700 1 -- 172.21.15.77:0/2024898664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4584100960 msgr2=0x7f45841a0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:32.845 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.840+0000 7f4582ffd700 1 --2- 172.21.15.77:0/2024898664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4584100960 0x7f45841a0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:32.845 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.840+0000 7f4582ffd700 1 -- 172.21.15.77:0/2024898664 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f457c009d00 con 0x7f45840ff640 2023-12-01T01:22:32.845 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.840+0000 7f45827fc700 1 --2- 172.21.15.77:0/2024898664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45840fffa0 0x7f4584199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:22:32.845 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.840+0000 7f4582ffd700 1 --2- 172.21.15.77:0/2024898664 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45840ff640 0x7f4584198d00 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f457400d950 tx=0x7f457400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:32.845 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.840+0000 7f456bfff700 1 -- 172.21.15.77:0/2024898664 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4574009b40 con 0x7f45840ff640 2023-12-01T01:22:32.846 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.840+0000 7f45892d1700 1 -- 172.21.15.77:0/2024898664 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45841a0e50 con 0x7f45840ff640 2023-12-01T01:22:32.846 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.840+0000 7f456bfff700 1 -- 172.21.15.77:0/2024898664 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4574009ca0 con 0x7f45840ff640 2023-12-01T01:22:32.846 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.841+0000 7f456bfff700 1 -- 172.21.15.77:0/2024898664 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457400f620 con 0x7f45840ff640 2023-12-01T01:22:32.846 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.841+0000 7f45892d1700 1 -- 172.21.15.77:0/2024898664 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45841a1280 con 0x7f45840ff640 2023-12-01T01:22:32.846 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.841+0000 7f45892d1700 1 -- 172.21.15.77:0/2024898664 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f458410a420 con 0x7f45840ff640 2023-12-01T01:22:32.847 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.842+0000 7f456bfff700 1 -- 172.21.15.77:0/2024898664 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f4574010460 con 0x7f45840ff640 2023-12-01T01:22:32.847 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.843+0000 7f456bfff700 1 --2- 172.21.15.77:0/2024898664 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f456c0a0570 0x7f456c0a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:22:32.847 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.843+0000 7f45827fc700 1 --2- 172.21.15.77:0/2024898664 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f456c0a0570 0x7f456c0a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:22:32.847 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.843+0000 7f456bfff700 1 -- 172.21.15.77:0/2024898664 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f45740cbc60 con 0x7f45840ff640 2023-12-01T01:22:32.847 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.843+0000 7f45827fc700 1 --2- 172.21.15.77:0/2024898664 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f456c0a0570 0x7f456c0a2a30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4578005950 tx=0x7f4578005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:22:32.848 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:32.847+0000 7f456bfff700 1 -- 172.21.15.77:0/2024898664 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f45740958a0 con 0x7f45840ff640 2023-12-01T01:22:33.100 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:33.099+0000 7f45892d1700 1 -- 172.21.15.77:0/2024898664 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f458402cec0 con 0x7f45840ff640 2023-12-01T01:22:33.108 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:33.106+0000 7f456bfff700 1 -- 172.21.15.77:0/2024898664 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) v1 ==== 81+0+729705 (secure 0 0 0) 0x7f4574095030 con 0x7f45840ff640 2023-12-01T01:22:33.117 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:22:33.117 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2023-12-01T01:22:33.117 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 18, 2023-12-01T01:22:33.117 INFO:teuthology.orchestra.run.smithi077.stdout: "active_gid": 14152, 2023-12-01T01:22:33.117 INFO:teuthology.orchestra.run.smithi077.stdout: "active_name": "x", 2023-12-01T01:22:33.117 INFO:teuthology.orchestra.run.smithi077.stdout: "active_addrs": { 2023-12-01T01:22:33.117 INFO:teuthology.orchestra.run.smithi077.stdout: "addrvec": [ 2023-12-01T01:22:33.117 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.117 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "v2", 2023-12-01T01:22:33.118 INFO:teuthology.orchestra.run.smithi077.stdout: "addr": "172.21.15.77:6800", 2023-12-01T01:22:33.118 INFO:teuthology.orchestra.run.smithi077.stdout: "nonce": 3838941679 2023-12-01T01:22:33.118 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.118 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.118 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "v1", 2023-12-01T01:22:33.118 INFO:teuthology.orchestra.run.smithi077.stdout: "addr": "172.21.15.77:6801", 2023-12-01T01:22:33.118 INFO:teuthology.orchestra.run.smithi077.stdout: "nonce": 3838941679 2023-12-01T01:22:33.118 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.118 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2023-12-01T01:22:33.118 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.118 INFO:teuthology.orchestra.run.smithi077.stdout: "active_addr": "172.21.15.77:6801/3838941679", 2023-12-01T01:22:33.119 INFO:teuthology.orchestra.run.smithi077.stdout: "active_change": "2023-12-01T01:10:24.191951+0000", 2023-12-01T01:22:33.119 INFO:teuthology.orchestra.run.smithi077.stdout: "active_mgr_features": 4540138322906710015, 2023-12-01T01:22:33.119 INFO:teuthology.orchestra.run.smithi077.stdout: "available": true, 2023-12-01T01:22:33.119 INFO:teuthology.orchestra.run.smithi077.stdout: "standbys": [ 2023-12-01T01:22:33.119 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.119 INFO:teuthology.orchestra.run.smithi077.stdout: "gid": 14223, 2023-12-01T01:22:33.119 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "y", 2023-12-01T01:22:33.119 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr_features": 4540138322906710015, 2023-12-01T01:22:33.119 INFO:teuthology.orchestra.run.smithi077.stdout: "available_modules": [ 2023-12-01T01:22:33.119 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.119 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "alerts", 2023-12-01T01:22:33.120 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.120 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.120 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.120 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2023-12-01T01:22:33.120 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2023-12-01T01:22:33.120 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:33.120 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.120 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.120 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2023-12-01T01:22:33.120 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.120 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.120 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.121 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How frequently to reexamine health status", 2023-12-01T01:22:33.121 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.121 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.121 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.121 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.121 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.121 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.121 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.121 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.121 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.121 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.121 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.121 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.122 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.122 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.122 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.122 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.122 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.122 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.122 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.122 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.122 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.122 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.122 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.122 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.123 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.123 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.123 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.123 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.123 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.123 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.123 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.123 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.123 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.123 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.123 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.123 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.123 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.124 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.124 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.124 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.124 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.124 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.124 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.124 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.124 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.124 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.124 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.124 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.124 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.124 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.125 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.125 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.125 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.125 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.125 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.125 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.125 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.125 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.125 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.126 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.126 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.126 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.126 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.126 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.126 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.126 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.126 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.126 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.126 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.126 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.126 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.126 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.127 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.127 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.127 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_destination": { 2023-12-01T01:22:33.127 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_destination", 2023-12-01T01:22:33.127 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.127 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.127 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.127 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.127 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.127 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.127 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.127 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Email address to send alerts to", 2023-12-01T01:22:33.127 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.128 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.128 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.128 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.128 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_from_name": { 2023-12-01T01:22:33.128 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_from_name", 2023-12-01T01:22:33.128 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.128 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.128 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.128 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "Ceph", 2023-12-01T01:22:33.128 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.128 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.128 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.129 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Email From: name", 2023-12-01T01:22:33.129 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.129 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.129 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.129 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.129 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_host": { 2023-12-01T01:22:33.129 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_host", 2023-12-01T01:22:33.129 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.129 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.129 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.129 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.130 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.130 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.130 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.130 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SMTP server", 2023-12-01T01:22:33.130 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.130 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.131 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.131 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.131 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_password": { 2023-12-01T01:22:33.131 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_password", 2023-12-01T01:22:33.131 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.131 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.131 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.131 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.131 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.131 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.131 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.131 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Password to authenticate with", 2023-12-01T01:22:33.132 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.132 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.132 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.132 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.132 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_port": { 2023-12-01T01:22:33.132 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_port", 2023-12-01T01:22:33.132 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.132 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.132 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.132 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "465", 2023-12-01T01:22:33.132 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.132 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.133 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.133 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SMTP port", 2023-12-01T01:22:33.133 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.133 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.133 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.133 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.133 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_sender": { 2023-12-01T01:22:33.133 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_sender", 2023-12-01T01:22:33.133 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.133 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.133 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.133 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.133 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.134 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.134 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.134 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SMTP envelope sender", 2023-12-01T01:22:33.134 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.134 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.134 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.134 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.134 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_ssl": { 2023-12-01T01:22:33.134 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_ssl", 2023-12-01T01:22:33.134 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.134 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.135 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.135 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.135 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.135 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.135 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.135 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use SSL to connect to SMTP server", 2023-12-01T01:22:33.135 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.135 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.135 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.135 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.135 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_user": { 2023-12-01T01:22:33.135 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_user", 2023-12-01T01:22:33.136 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.136 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.136 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.136 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.136 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.136 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.136 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.136 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "User to authenticate as", 2023-12-01T01:22:33.136 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.136 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.136 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.136 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.137 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.137 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.137 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.137 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "balancer", 2023-12-01T01:22:33.137 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.137 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.137 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.137 INFO:teuthology.orchestra.run.smithi077.stdout: "active": { 2023-12-01T01:22:33.137 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "active", 2023-12-01T01:22:33.137 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.137 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.137 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.137 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.138 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.138 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.138 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.138 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "automatically balance PGs across cluster", 2023-12-01T01:22:33.138 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.138 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.138 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.138 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.138 INFO:teuthology.orchestra.run.smithi077.stdout: "begin_time": { 2023-12-01T01:22:33.138 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "begin_time", 2023-12-01T01:22:33.138 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.139 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.139 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.139 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0000", 2023-12-01T01:22:33.139 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.139 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.139 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.139 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "beginning time of day to automatically balance", 2023-12-01T01:22:33.139 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-12-01T01:22:33.139 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.139 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.139 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.139 INFO:teuthology.orchestra.run.smithi077.stdout: "begin_weekday": { 2023-12-01T01:22:33.140 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "begin_weekday", 2023-12-01T01:22:33.140 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2023-12-01T01:22:33.140 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.140 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.140 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2023-12-01T01:22:33.140 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "0", 2023-12-01T01:22:33.140 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "6", 2023-12-01T01:22:33.140 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.140 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-12-01T01:22:33.140 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-12-01T01:22:33.140 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.140 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.141 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.141 INFO:teuthology.orchestra.run.smithi077.stdout: "crush_compat_max_iterations": { 2023-12-01T01:22:33.141 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crush_compat_max_iterations", 2023-12-01T01:22:33.141 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2023-12-01T01:22:33.141 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.141 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.141 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "25", 2023-12-01T01:22:33.141 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2023-12-01T01:22:33.141 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "250", 2023-12-01T01:22:33.141 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.141 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-12-01T01:22:33.141 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.141 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.142 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.142 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.142 INFO:teuthology.orchestra.run.smithi077.stdout: "crush_compat_metrics": { 2023-12-01T01:22:33.142 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crush_compat_metrics", 2023-12-01T01:22:33.142 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.142 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.142 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.142 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "pgs,objects,bytes", 2023-12-01T01:22:33.142 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.142 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.142 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.142 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-12-01T01:22:33.143 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-12-01T01:22:33.143 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.143 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.143 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.143 INFO:teuthology.orchestra.run.smithi077.stdout: "crush_compat_step": { 2023-12-01T01:22:33.143 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crush_compat_step", 2023-12-01T01:22:33.143 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2023-12-01T01:22:33.143 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.143 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.143 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0.5", 2023-12-01T01:22:33.143 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "0.001", 2023-12-01T01:22:33.143 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "0.999", 2023-12-01T01:22:33.144 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.144 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "aggressiveness of optimization", 2023-12-01T01:22:33.144 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-12-01T01:22:33.144 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.144 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.144 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.144 INFO:teuthology.orchestra.run.smithi077.stdout: "end_time": { 2023-12-01T01:22:33.144 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "end_time", 2023-12-01T01:22:33.144 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.144 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.144 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.144 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "2359", 2023-12-01T01:22:33.144 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.145 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.145 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.145 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "ending time of day to automatically balance", 2023-12-01T01:22:33.145 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-12-01T01:22:33.145 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.145 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.145 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.145 INFO:teuthology.orchestra.run.smithi077.stdout: "end_weekday": { 2023-12-01T01:22:33.145 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "end_weekday", 2023-12-01T01:22:33.145 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2023-12-01T01:22:33.145 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.145 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.145 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2023-12-01T01:22:33.146 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "0", 2023-12-01T01:22:33.146 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "6", 2023-12-01T01:22:33.146 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.146 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-12-01T01:22:33.146 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-12-01T01:22:33.146 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.146 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.146 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.146 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.146 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.146 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.146 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.147 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.147 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.147 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.147 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.147 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.147 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.147 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.147 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.147 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.147 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.147 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.147 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.148 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.148 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.148 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.148 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.148 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.148 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.148 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.148 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.148 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.148 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.148 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.148 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.149 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.149 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.149 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.149 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.149 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.149 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.149 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.149 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.149 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.149 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.149 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.149 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.150 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.150 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.150 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.150 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.150 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.150 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.150 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.150 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.150 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.150 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.150 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.151 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.151 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.151 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.151 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.151 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.151 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.151 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.151 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.151 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.151 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.151 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.152 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.152 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.152 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.152 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.152 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.152 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.152 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.152 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.152 INFO:teuthology.orchestra.run.smithi077.stdout: "min_score": { 2023-12-01T01:22:33.152 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "min_score", 2023-12-01T01:22:33.152 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2023-12-01T01:22:33.152 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.153 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.153 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2023-12-01T01:22:33.153 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.153 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.153 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.153 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-12-01T01:22:33.153 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.153 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.153 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.153 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.153 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": { 2023-12-01T01:22:33.153 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mode", 2023-12-01T01:22:33.154 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.154 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.154 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.154 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "upmap", 2023-12-01T01:22:33.154 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.154 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.154 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.154 INFO:teuthology.orchestra.run.smithi077.stdout: "crush-compat", 2023-12-01T01:22:33.154 INFO:teuthology.orchestra.run.smithi077.stdout: "none", 2023-12-01T01:22:33.154 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap" 2023-12-01T01:22:33.154 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.155 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Balancer mode", 2023-12-01T01:22:33.155 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.155 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.155 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.155 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.155 INFO:teuthology.orchestra.run.smithi077.stdout: "pool_ids": { 2023-12-01T01:22:33.155 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pool_ids", 2023-12-01T01:22:33.155 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.155 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.155 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.155 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.155 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.156 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.156 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.156 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-12-01T01:22:33.156 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.156 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.156 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.156 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.156 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2023-12-01T01:22:33.156 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2023-12-01T01:22:33.156 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:33.156 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.157 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.157 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2023-12-01T01:22:33.157 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.157 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.157 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.157 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-12-01T01:22:33.157 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.157 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.157 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.157 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.157 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap_max_deviation": { 2023-12-01T01:22:33.157 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "upmap_max_deviation", 2023-12-01T01:22:33.158 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.158 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.158 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.158 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2023-12-01T01:22:33.158 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2023-12-01T01:22:33.158 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.158 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.158 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "deviation below which no optimization is attempted", 2023-12-01T01:22:33.158 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-12-01T01:22:33.158 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.158 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.158 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.159 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap_max_optimizations": { 2023-12-01T01:22:33.159 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "upmap_max_optimizations", 2023-12-01T01:22:33.159 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2023-12-01T01:22:33.159 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.159 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.159 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2023-12-01T01:22:33.159 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.159 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.159 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.159 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-12-01T01:22:33.159 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.159 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.160 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.160 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.160 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.160 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.160 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.160 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cephadm", 2023-12-01T01:22:33.160 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.160 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.160 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.160 INFO:teuthology.orchestra.run.smithi077.stdout: "agent_down_multiplier": { 2023-12-01T01:22:33.160 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "agent_down_multiplier", 2023-12-01T01:22:33.160 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2023-12-01T01:22:33.160 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.161 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.161 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "3.0", 2023-12-01T01:22:33.161 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.161 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.161 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.161 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-12-01T01:22:33.161 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.161 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.161 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.161 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.161 INFO:teuthology.orchestra.run.smithi077.stdout: "agent_refresh_rate": { 2023-12-01T01:22:33.161 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "agent_refresh_rate", 2023-12-01T01:22:33.162 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:33.162 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.162 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.162 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "20", 2023-12-01T01:22:33.162 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.162 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.162 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.162 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-12-01T01:22:33.162 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.162 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.162 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.162 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.163 INFO:teuthology.orchestra.run.smithi077.stdout: "agent_starting_port": { 2023-12-01T01:22:33.163 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "agent_starting_port", 2023-12-01T01:22:33.163 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.163 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.163 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.163 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "4721", 2023-12-01T01:22:33.163 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.163 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.163 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.163 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-12-01T01:22:33.163 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.163 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.164 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.164 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.164 INFO:teuthology.orchestra.run.smithi077.stdout: "allow_ptrace": { 2023-12-01T01:22:33.164 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "allow_ptrace", 2023-12-01T01:22:33.164 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.164 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.164 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.164 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.164 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.164 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.164 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.164 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-12-01T01:22:33.165 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-12-01T01:22:33.165 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.165 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.165 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.165 INFO:teuthology.orchestra.run.smithi077.stdout: "autotune_interval": { 2023-12-01T01:22:33.165 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "autotune_interval", 2023-12-01T01:22:33.165 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:33.165 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.165 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.165 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2023-12-01T01:22:33.165 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.165 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.166 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.166 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to autotune daemon memory", 2023-12-01T01:22:33.166 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.166 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.166 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.166 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.166 INFO:teuthology.orchestra.run.smithi077.stdout: "autotune_memory_target_ratio": { 2023-12-01T01:22:33.166 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "autotune_memory_target_ratio", 2023-12-01T01:22:33.166 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2023-12-01T01:22:33.166 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.166 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.166 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0.7", 2023-12-01T01:22:33.167 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.167 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.167 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.167 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-12-01T01:22:33.167 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.167 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.167 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.167 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.167 INFO:teuthology.orchestra.run.smithi077.stdout: "cgroups_split": { 2023-12-01T01:22:33.167 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cgroups_split", 2023-12-01T01:22:33.167 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.167 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.168 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.168 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.168 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.168 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.168 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.168 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-12-01T01:22:33.168 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.168 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.168 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.168 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.168 INFO:teuthology.orchestra.run.smithi077.stdout: "config_checks_enabled": { 2023-12-01T01:22:33.168 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "config_checks_enabled", 2023-12-01T01:22:33.168 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.169 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.169 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.169 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.169 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.169 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.169 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.169 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-12-01T01:22:33.169 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.169 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.169 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.169 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.169 INFO:teuthology.orchestra.run.smithi077.stdout: "config_dashboard": { 2023-12-01T01:22:33.169 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "config_dashboard", 2023-12-01T01:22:33.170 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.170 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.170 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.170 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.170 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.170 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.170 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.170 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-12-01T01:22:33.170 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.170 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.170 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.170 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.171 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_alertmanager": { 2023-12-01T01:22:33.171 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_alertmanager", 2023-12-01T01:22:33.171 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.171 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.171 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.171 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-12-01T01:22:33.171 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.171 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.171 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.171 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2023-12-01T01:22:33.171 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.171 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.172 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.172 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.172 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_base": { 2023-12-01T01:22:33.172 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_base", 2023-12-01T01:22:33.172 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.172 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.172 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.172 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/ceph", 2023-12-01T01:22:33.172 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.172 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.172 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.172 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Container image name, without the tag", 2023-12-01T01:22:33.172 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.173 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.173 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.173 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.173 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_elasticsearch": { 2023-12-01T01:22:33.173 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_elasticsearch", 2023-12-01T01:22:33.173 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.173 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.173 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.173 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-12-01T01:22:33.173 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.173 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.173 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.173 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "elasticsearch container image", 2023-12-01T01:22:33.174 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.174 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.174 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.174 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.174 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_grafana": { 2023-12-01T01:22:33.174 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_grafana", 2023-12-01T01:22:33.174 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.174 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.174 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.174 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-12-01T01:22:33.174 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.174 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.174 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.175 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2023-12-01T01:22:33.175 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.175 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.175 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.175 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.175 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_haproxy": { 2023-12-01T01:22:33.175 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_haproxy", 2023-12-01T01:22:33.175 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.175 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.175 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.175 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-12-01T01:22:33.175 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.175 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.176 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.176 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "HAproxy container image", 2023-12-01T01:22:33.176 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.176 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.176 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.176 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.176 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_jaeger_agent": { 2023-12-01T01:22:33.176 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_jaeger_agent", 2023-12-01T01:22:33.176 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.176 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.176 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.176 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-12-01T01:22:33.176 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.177 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.177 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.177 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Jaeger agent container image", 2023-12-01T01:22:33.177 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.177 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.177 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.177 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.177 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_jaeger_collector": { 2023-12-01T01:22:33.177 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_jaeger_collector", 2023-12-01T01:22:33.177 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.178 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.178 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.178 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-12-01T01:22:33.178 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.178 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.178 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.178 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Jaeger collector container image", 2023-12-01T01:22:33.178 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.178 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.178 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.178 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.178 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_jaeger_query": { 2023-12-01T01:22:33.178 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_jaeger_query", 2023-12-01T01:22:33.179 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.179 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.179 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.179 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-12-01T01:22:33.179 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.179 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.179 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.179 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Jaeger query container image", 2023-12-01T01:22:33.179 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.179 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.179 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.179 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.180 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_keepalived": { 2023-12-01T01:22:33.180 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_keepalived", 2023-12-01T01:22:33.180 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.180 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.180 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.180 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-12-01T01:22:33.180 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.180 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.180 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.180 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Keepalived container image", 2023-12-01T01:22:33.180 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.180 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.180 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.181 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.181 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_loki": { 2023-12-01T01:22:33.181 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_loki", 2023-12-01T01:22:33.181 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.181 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.181 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.181 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-12-01T01:22:33.181 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.181 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.181 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.181 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Loki container image", 2023-12-01T01:22:33.181 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.181 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.182 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.182 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.182 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_node_exporter": { 2023-12-01T01:22:33.182 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_node_exporter", 2023-12-01T01:22:33.182 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.182 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.182 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.182 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-12-01T01:22:33.182 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.182 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.182 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.182 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2023-12-01T01:22:33.183 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.183 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.183 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.183 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.183 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_nvmeof": { 2023-12-01T01:22:33.183 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_nvmeof", 2023-12-01T01:22:33.183 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.183 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.183 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.183 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.2", 2023-12-01T01:22:33.183 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.183 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.184 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.184 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Nvme-of container image", 2023-12-01T01:22:33.184 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.184 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.184 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.184 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.184 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_prometheus": { 2023-12-01T01:22:33.184 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_prometheus", 2023-12-01T01:22:33.184 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.184 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.184 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.184 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-12-01T01:22:33.184 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.185 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.185 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.185 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2023-12-01T01:22:33.185 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.185 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.185 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.185 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.185 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_promtail": { 2023-12-01T01:22:33.185 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_promtail", 2023-12-01T01:22:33.185 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.185 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.185 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.186 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-12-01T01:22:33.186 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.186 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.186 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.186 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Promtail container image", 2023-12-01T01:22:33.186 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.186 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.186 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.186 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.186 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_snmp_gateway": { 2023-12-01T01:22:33.186 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_snmp_gateway", 2023-12-01T01:22:33.186 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.186 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.187 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.187 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-12-01T01:22:33.187 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.187 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.187 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.187 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SNMP Gateway container image", 2023-12-01T01:22:33.187 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.187 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.187 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.187 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.187 INFO:teuthology.orchestra.run.smithi077.stdout: "container_init": { 2023-12-01T01:22:33.187 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_init", 2023-12-01T01:22:33.188 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.188 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.188 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.188 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.188 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.188 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.188 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.188 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Run podman/docker with `--init`", 2023-12-01T01:22:33.188 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.188 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.188 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.188 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.188 INFO:teuthology.orchestra.run.smithi077.stdout: "daemon_cache_timeout": { 2023-12-01T01:22:33.189 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "daemon_cache_timeout", 2023-12-01T01:22:33.189 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:33.189 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.189 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.189 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2023-12-01T01:22:33.189 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.189 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.189 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.189 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-12-01T01:22:33.189 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.189 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.189 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.190 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.190 INFO:teuthology.orchestra.run.smithi077.stdout: "default_cephadm_command_timeout": { 2023-12-01T01:22:33.190 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "default_cephadm_command_timeout", 2023-12-01T01:22:33.190 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:33.190 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.190 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.190 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "900", 2023-12-01T01:22:33.190 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.190 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.190 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.190 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-12-01T01:22:33.191 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.191 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.191 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.191 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.191 INFO:teuthology.orchestra.run.smithi077.stdout: "default_registry": { 2023-12-01T01:22:33.191 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "default_registry", 2023-12-01T01:22:33.191 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.191 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.191 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.191 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io", 2023-12-01T01:22:33.191 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.192 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.192 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.192 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-12-01T01:22:33.192 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.192 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.192 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.192 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.192 INFO:teuthology.orchestra.run.smithi077.stdout: "device_cache_timeout": { 2023-12-01T01:22:33.192 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "device_cache_timeout", 2023-12-01T01:22:33.192 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:33.192 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.192 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.192 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "1800", 2023-12-01T01:22:33.193 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.193 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.193 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.193 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "seconds to cache device inventory", 2023-12-01T01:22:33.193 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.193 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.193 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.193 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.193 INFO:teuthology.orchestra.run.smithi077.stdout: "device_enhanced_scan": { 2023-12-01T01:22:33.193 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "device_enhanced_scan", 2023-12-01T01:22:33.193 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.193 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.193 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.194 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.194 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.194 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.194 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.194 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use libstoragemgmt during device scans", 2023-12-01T01:22:33.194 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.194 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.194 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.194 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.194 INFO:teuthology.orchestra.run.smithi077.stdout: "facts_cache_timeout": { 2023-12-01T01:22:33.194 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "facts_cache_timeout", 2023-12-01T01:22:33.194 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:33.194 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.195 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.195 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2023-12-01T01:22:33.195 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.195 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.195 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.195 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "seconds to cache host facts data", 2023-12-01T01:22:33.195 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.195 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.195 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.195 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.195 INFO:teuthology.orchestra.run.smithi077.stdout: "host_check_interval": { 2023-12-01T01:22:33.195 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "host_check_interval", 2023-12-01T01:22:33.196 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:33.196 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.196 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.196 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2023-12-01T01:22:33.196 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.196 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.196 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.196 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to perform a host check", 2023-12-01T01:22:33.196 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.196 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.196 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.196 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.197 INFO:teuthology.orchestra.run.smithi077.stdout: "inventory_list_all": { 2023-12-01T01:22:33.197 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "inventory_list_all", 2023-12-01T01:22:33.197 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.197 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.197 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.197 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.197 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.197 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.197 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.197 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-12-01T01:22:33.197 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.197 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.197 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.198 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.198 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.198 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.198 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.198 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.198 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.198 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.198 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.198 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.198 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.198 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.198 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.198 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.199 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.199 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.199 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.199 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.199 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.199 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.199 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.199 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.199 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.199 INFO:teuthology.orchestra.run.smithi077.stdout: "log_refresh_metadata": { 2023-12-01T01:22:33.199 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_refresh_metadata", 2023-12-01T01:22:33.199 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.199 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.200 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.200 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.200 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.200 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.200 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.200 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-12-01T01:22:33.200 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.200 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.200 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.200 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.200 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.200 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.201 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.201 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.201 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.201 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.201 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.201 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.201 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.201 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-12-01T01:22:33.201 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.201 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.201 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.201 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.201 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.202 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.202 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.202 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.202 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.202 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.202 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.202 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.202 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.202 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.202 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.202 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.202 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.203 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.203 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.203 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.203 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.203 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.203 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.203 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.203 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.203 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.203 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.203 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.203 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.203 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.204 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.204 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.204 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.204 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.204 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.204 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.204 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.204 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.204 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.204 INFO:teuthology.orchestra.run.smithi077.stdout: "manage_etc_ceph_ceph_conf": { 2023-12-01T01:22:33.204 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-12-01T01:22:33.204 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.204 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.205 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.205 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.205 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.205 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.205 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.205 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-12-01T01:22:33.205 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.205 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.205 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.205 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.205 INFO:teuthology.orchestra.run.smithi077.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-12-01T01:22:33.206 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-12-01T01:22:33.206 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.206 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.206 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.206 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "*", 2023-12-01T01:22:33.206 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.206 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.206 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.206 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-12-01T01:22:33.206 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.206 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.206 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.207 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.207 INFO:teuthology.orchestra.run.smithi077.stdout: "max_count_per_host": { 2023-12-01T01:22:33.207 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_count_per_host", 2023-12-01T01:22:33.207 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.207 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.207 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.207 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2023-12-01T01:22:33.207 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.207 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.207 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.207 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "max number of daemons per service per host", 2023-12-01T01:22:33.207 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.207 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.208 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.208 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.208 INFO:teuthology.orchestra.run.smithi077.stdout: "max_osd_draining_count": { 2023-12-01T01:22:33.208 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_osd_draining_count", 2023-12-01T01:22:33.208 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.208 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.208 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.208 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2023-12-01T01:22:33.208 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.208 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.208 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.208 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-12-01T01:22:33.208 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.209 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.209 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.209 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.209 INFO:teuthology.orchestra.run.smithi077.stdout: "migration_current": { 2023-12-01T01:22:33.209 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "migration_current", 2023-12-01T01:22:33.209 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.209 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.209 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.209 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.209 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.209 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.209 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.209 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "internal - do not modify", 2023-12-01T01:22:33.210 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.210 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.210 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.210 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.210 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": { 2023-12-01T01:22:33.210 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mode", 2023-12-01T01:22:33.210 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.210 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.210 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.210 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "root", 2023-12-01T01:22:33.210 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.210 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.211 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.211 INFO:teuthology.orchestra.run.smithi077.stdout: "cephadm-package", 2023-12-01T01:22:33.211 INFO:teuthology.orchestra.run.smithi077.stdout: "root" 2023-12-01T01:22:33.211 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.211 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "mode for remote execution of cephadm", 2023-12-01T01:22:33.211 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.211 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.211 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.211 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.211 INFO:teuthology.orchestra.run.smithi077.stdout: "prometheus_alerts_path": { 2023-12-01T01:22:33.211 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "prometheus_alerts_path", 2023-12-01T01:22:33.211 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.211 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.212 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.212 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-12-01T01:22:33.212 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.212 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.212 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.212 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-12-01T01:22:33.212 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.212 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.212 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.212 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.212 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_insecure": { 2023-12-01T01:22:33.212 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_insecure", 2023-12-01T01:22:33.212 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.213 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.213 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.213 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.213 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.213 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.213 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.213 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-12-01T01:22:33.213 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.213 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.213 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.213 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.213 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_password": { 2023-12-01T01:22:33.213 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_password", 2023-12-01T01:22:33.214 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.214 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.214 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.214 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.214 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.214 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.214 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.214 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-12-01T01:22:33.214 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.214 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.214 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.214 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.215 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_url": { 2023-12-01T01:22:33.215 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_url", 2023-12-01T01:22:33.215 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.215 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.215 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.215 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.215 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.215 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.215 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.215 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-12-01T01:22:33.215 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.215 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.215 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.216 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.216 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_username": { 2023-12-01T01:22:33.216 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_username", 2023-12-01T01:22:33.216 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.216 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.216 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.216 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.216 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.216 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.216 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.216 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-12-01T01:22:33.216 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.216 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.217 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.217 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.217 INFO:teuthology.orchestra.run.smithi077.stdout: "secure_monitoring_stack": { 2023-12-01T01:22:33.217 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "secure_monitoring_stack", 2023-12-01T01:22:33.217 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.217 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.217 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.217 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.217 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.217 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.217 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.217 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-12-01T01:22:33.218 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.218 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.218 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.218 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.218 INFO:teuthology.orchestra.run.smithi077.stdout: "service_discovery_port": { 2023-12-01T01:22:33.218 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "service_discovery_port", 2023-12-01T01:22:33.218 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.218 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.218 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.218 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8765", 2023-12-01T01:22:33.218 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.219 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.219 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.219 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "cephadm service discovery port", 2023-12-01T01:22:33.219 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.219 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.219 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.219 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.219 INFO:teuthology.orchestra.run.smithi077.stdout: "ssh_config_file": { 2023-12-01T01:22:33.219 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssh_config_file", 2023-12-01T01:22:33.219 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.219 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.219 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.219 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.220 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.220 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.220 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.220 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-12-01T01:22:33.220 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.220 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.220 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.220 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.220 INFO:teuthology.orchestra.run.smithi077.stdout: "use_agent": { 2023-12-01T01:22:33.220 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "use_agent", 2023-12-01T01:22:33.220 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.220 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.221 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.221 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.221 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.221 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.221 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.221 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-12-01T01:22:33.221 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.221 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.221 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.221 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.221 INFO:teuthology.orchestra.run.smithi077.stdout: "use_repo_digest": { 2023-12-01T01:22:33.221 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "use_repo_digest", 2023-12-01T01:22:33.221 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.222 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.222 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.222 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.222 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.222 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.222 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.222 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-12-01T01:22:33.222 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.222 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.222 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.222 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.222 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_on_failed_host_check": { 2023-12-01T01:22:33.223 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_on_failed_host_check", 2023-12-01T01:22:33.223 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.223 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.223 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.223 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.223 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.223 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.223 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.223 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise a health warning if the host check fails", 2023-12-01T01:22:33.223 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.223 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.223 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.224 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.224 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_on_stray_daemons": { 2023-12-01T01:22:33.224 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_on_stray_daemons", 2023-12-01T01:22:33.224 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.224 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.224 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.224 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.224 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.224 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.224 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.224 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-12-01T01:22:33.224 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.225 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.225 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.225 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.225 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_on_stray_hosts": { 2023-12-01T01:22:33.225 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_on_stray_hosts", 2023-12-01T01:22:33.225 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.225 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.225 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.225 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.225 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.225 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.225 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.226 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-12-01T01:22:33.226 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.226 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.226 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.226 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.226 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.226 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.226 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.226 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crash", 2023-12-01T01:22:33.226 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.226 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.227 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.227 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.227 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.227 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.227 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.227 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.227 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.227 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.227 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.227 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.227 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.227 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.228 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.228 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.228 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.228 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.228 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.228 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.228 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.228 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.228 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.228 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.228 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.228 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.228 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.229 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.229 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.229 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.229 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.229 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.229 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.229 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.229 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.229 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.229 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.229 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.229 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.230 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.230 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.230 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.230 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.230 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.230 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.230 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.230 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.230 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.230 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.230 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.230 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.231 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.231 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.231 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.231 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.231 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.231 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.231 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.231 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.231 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.231 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.231 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.231 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.232 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.232 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.232 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.232 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.232 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.232 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.232 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.232 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.232 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.232 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.232 INFO:teuthology.orchestra.run.smithi077.stdout: "retain_interval": { 2023-12-01T01:22:33.232 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "retain_interval", 2023-12-01T01:22:33.233 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:33.233 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.233 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.233 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "31536000", 2023-12-01T01:22:33.233 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.233 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.233 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.233 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how long to retain crashes before pruning them", 2023-12-01T01:22:33.233 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.233 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.233 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.233 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.234 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_recent_interval": { 2023-12-01T01:22:33.234 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_recent_interval", 2023-12-01T01:22:33.234 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:33.234 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.234 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.234 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "1209600", 2023-12-01T01:22:33.234 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.234 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.234 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.234 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "time interval in which to warn about recent crashes", 2023-12-01T01:22:33.234 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.234 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.235 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.235 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.235 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.235 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.235 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.235 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "dashboard", 2023-12-01T01:22:33.235 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.235 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.235 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.235 INFO:teuthology.orchestra.run.smithi077.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-12-01T01:22:33.235 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-12-01T01:22:33.235 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.235 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.236 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.236 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2023-12-01T01:22:33.236 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.236 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.236 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.236 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.236 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.236 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.236 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.236 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.236 INFO:teuthology.orchestra.run.smithi077.stdout: "ALERTMANAGER_API_HOST": { 2023-12-01T01:22:33.237 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ALERTMANAGER_API_HOST", 2023-12-01T01:22:33.237 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.237 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.237 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.237 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.237 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.237 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.237 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.237 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.237 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.237 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.237 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.238 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.238 INFO:teuthology.orchestra.run.smithi077.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-12-01T01:22:33.238 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-12-01T01:22:33.238 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.238 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.238 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.238 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.238 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.238 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.238 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.238 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.238 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.239 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.239 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.239 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.239 INFO:teuthology.orchestra.run.smithi077.stdout: "AUDIT_API_ENABLED": { 2023-12-01T01:22:33.239 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "AUDIT_API_ENABLED", 2023-12-01T01:22:33.239 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.239 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.239 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.239 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.239 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.239 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.239 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.240 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.240 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.240 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.240 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.240 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.240 INFO:teuthology.orchestra.run.smithi077.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-12-01T01:22:33.240 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-12-01T01:22:33.240 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.240 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.240 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.240 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.240 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.241 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.241 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.241 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.241 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.241 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.241 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.241 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.241 INFO:teuthology.orchestra.run.smithi077.stdout: "ENABLE_BROWSABLE_API": { 2023-12-01T01:22:33.241 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ENABLE_BROWSABLE_API", 2023-12-01T01:22:33.241 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.241 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.241 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.242 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.242 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.242 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.242 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.242 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.242 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.242 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.242 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.242 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.242 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-12-01T01:22:33.242 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-12-01T01:22:33.242 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.243 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.243 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.243 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.243 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.243 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.243 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.243 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.243 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.243 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.243 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.243 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.243 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-12-01T01:22:33.243 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-12-01T01:22:33.244 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.244 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.244 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.244 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.244 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.244 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.244 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.244 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.244 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.244 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.244 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.244 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.244 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-12-01T01:22:33.245 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-12-01T01:22:33.245 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.245 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.245 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.245 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.245 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.245 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.245 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.245 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.245 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.245 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.245 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.245 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.246 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-12-01T01:22:33.246 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-12-01T01:22:33.246 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.246 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.246 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.246 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.246 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.246 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.246 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.246 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.246 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.246 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.246 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.247 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.247 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_NFS": { 2023-12-01T01:22:33.247 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-12-01T01:22:33.247 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.247 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.247 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.247 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.247 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.247 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.247 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.247 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.247 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.247 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.248 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.248 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.248 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_RBD": { 2023-12-01T01:22:33.248 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-12-01T01:22:33.248 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.248 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.248 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.248 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.248 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.248 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.248 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.248 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.249 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.249 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.249 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.249 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.249 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_RGW": { 2023-12-01T01:22:33.249 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-12-01T01:22:33.249 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.249 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.249 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.249 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.249 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.249 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.250 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.250 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.250 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.250 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.250 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.250 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.250 INFO:teuthology.orchestra.run.smithi077.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-12-01T01:22:33.250 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-12-01T01:22:33.250 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.250 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.250 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.250 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.250 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.251 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.251 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.251 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.251 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.251 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.251 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.251 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.251 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_PASSWORD": { 2023-12-01T01:22:33.251 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_PASSWORD", 2023-12-01T01:22:33.251 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.251 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.251 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.251 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "admin", 2023-12-01T01:22:33.252 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.252 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.252 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.252 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.252 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.252 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.252 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.252 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.252 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-12-01T01:22:33.252 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-12-01T01:22:33.252 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.252 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.252 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.253 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.253 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.253 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.253 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.253 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.253 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.253 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.253 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.253 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.253 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_URL": { 2023-12-01T01:22:33.253 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_URL", 2023-12-01T01:22:33.253 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.253 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.254 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.254 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.254 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.254 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.254 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.254 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.254 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.254 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.254 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.254 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.254 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_USERNAME": { 2023-12-01T01:22:33.254 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_USERNAME", 2023-12-01T01:22:33.255 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.255 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.255 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.255 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "admin", 2023-12-01T01:22:33.255 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.255 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.255 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.255 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.255 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.255 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.255 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.255 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.255 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-12-01T01:22:33.256 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-12-01T01:22:33.256 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.256 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.256 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.256 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.256 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.256 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.256 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.256 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.256 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.256 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.256 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.256 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.257 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-12-01T01:22:33.257 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-12-01T01:22:33.257 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.257 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.257 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.257 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.257 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.257 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.257 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.257 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.257 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.257 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.257 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.258 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.258 INFO:teuthology.orchestra.run.smithi077.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-12-01T01:22:33.258 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-12-01T01:22:33.258 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.258 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.258 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.258 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.258 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.258 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.258 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.258 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.258 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.258 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.259 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.259 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.259 INFO:teuthology.orchestra.run.smithi077.stdout: "ISSUE_TRACKER_API_KEY": { 2023-12-01T01:22:33.259 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-12-01T01:22:33.259 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.259 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.259 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.259 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.259 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.259 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.259 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.259 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.259 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.260 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.260 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.260 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.260 INFO:teuthology.orchestra.run.smithi077.stdout: "PROMETHEUS_API_HOST": { 2023-12-01T01:22:33.260 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PROMETHEUS_API_HOST", 2023-12-01T01:22:33.260 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.260 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.260 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.260 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.260 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.260 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.260 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.260 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.261 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.261 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.261 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.261 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.261 INFO:teuthology.orchestra.run.smithi077.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-12-01T01:22:33.261 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-12-01T01:22:33.261 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.261 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.261 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.261 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.261 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.261 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.261 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.262 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.262 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.262 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.262 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.262 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.262 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-12-01T01:22:33.262 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-12-01T01:22:33.262 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.262 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.262 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.262 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.262 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.263 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.263 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.263 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.263 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.263 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.263 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.263 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.263 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-12-01T01:22:33.263 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-12-01T01:22:33.263 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.263 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.264 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.264 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.264 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.264 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.264 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.264 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.264 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.264 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.264 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.264 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.264 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-12-01T01:22:33.264 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-12-01T01:22:33.264 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.265 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.265 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.265 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.265 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.265 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.265 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.265 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.265 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.265 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.265 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.265 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.265 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-12-01T01:22:33.265 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-12-01T01:22:33.266 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.266 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.266 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.266 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.266 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.266 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.266 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.266 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.266 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.266 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.266 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.266 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.267 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-12-01T01:22:33.267 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-12-01T01:22:33.267 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.267 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.267 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.267 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.267 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.267 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.267 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.267 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.267 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.267 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.268 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.268 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.268 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-12-01T01:22:33.268 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-12-01T01:22:33.268 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.268 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.268 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.268 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.268 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.268 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.268 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.268 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.268 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.269 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.269 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.269 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.269 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-12-01T01:22:33.269 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-12-01T01:22:33.269 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.269 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.269 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.269 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.269 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.269 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.269 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.270 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.270 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.270 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.270 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.270 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.270 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_ENABLED": { 2023-12-01T01:22:33.270 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_ENABLED", 2023-12-01T01:22:33.270 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.270 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.270 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.270 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.270 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.270 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.271 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.271 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.271 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.271 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.271 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.271 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.271 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-12-01T01:22:33.271 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-12-01T01:22:33.271 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.271 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.271 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.271 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-12-01T01:22:33.272 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.272 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.272 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.272 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.272 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.272 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.272 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.272 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.272 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-12-01T01:22:33.272 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-12-01T01:22:33.272 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.272 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.273 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.273 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2023-12-01T01:22:33.273 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.273 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.273 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.273 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.273 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.273 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.273 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.273 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.273 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-12-01T01:22:33.273 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-12-01T01:22:33.274 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.274 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.274 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.274 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8", 2023-12-01T01:22:33.274 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.274 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.274 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.274 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.274 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.274 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.274 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.274 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.274 INFO:teuthology.orchestra.run.smithi077.stdout: "REST_REQUESTS_TIMEOUT": { 2023-12-01T01:22:33.275 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-12-01T01:22:33.275 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.275 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.275 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.275 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "45", 2023-12-01T01:22:33.275 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.275 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.275 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.275 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.275 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.275 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.275 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.276 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.276 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_ACCESS_KEY": { 2023-12-01T01:22:33.276 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_ACCESS_KEY", 2023-12-01T01:22:33.276 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.276 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.276 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.276 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.276 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.276 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.276 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.276 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.277 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.277 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.277 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.277 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.277 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-12-01T01:22:33.277 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-12-01T01:22:33.277 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.277 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.277 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.277 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "admin", 2023-12-01T01:22:33.277 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.277 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.277 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.277 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.278 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.278 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.278 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.278 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.278 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_SECRET_KEY": { 2023-12-01T01:22:33.278 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_SECRET_KEY", 2023-12-01T01:22:33.278 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.278 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.278 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.278 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.278 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.279 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.279 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.279 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.279 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.279 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.279 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.279 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.279 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_SSL_VERIFY": { 2023-12-01T01:22:33.279 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_SSL_VERIFY", 2023-12-01T01:22:33.279 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.279 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.280 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.280 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.280 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.280 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.280 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.280 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.280 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.280 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.280 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.280 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.280 INFO:teuthology.orchestra.run.smithi077.stdout: "UNSAFE_TLS_v1_2": { 2023-12-01T01:22:33.280 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "UNSAFE_TLS_v1_2", 2023-12-01T01:22:33.280 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.281 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.281 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.281 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.281 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.281 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.281 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.281 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.281 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.281 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.281 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.281 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.281 INFO:teuthology.orchestra.run.smithi077.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-12-01T01:22:33.281 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-12-01T01:22:33.282 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.282 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.282 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.282 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2023-12-01T01:22:33.282 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.282 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.282 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.282 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.282 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.282 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.282 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.282 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.283 INFO:teuthology.orchestra.run.smithi077.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-12-01T01:22:33.283 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-12-01T01:22:33.283 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.283 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.283 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.283 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2023-12-01T01:22:33.283 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.283 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.283 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.283 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.283 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.283 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.283 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.284 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.284 INFO:teuthology.orchestra.run.smithi077.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-12-01T01:22:33.284 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-12-01T01:22:33.284 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.284 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.284 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.284 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2023-12-01T01:22:33.284 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.284 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.284 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.284 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.284 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.285 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.285 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.285 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.285 INFO:teuthology.orchestra.run.smithi077.stdout: "cross_origin_url": { 2023-12-01T01:22:33.285 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cross_origin_url", 2023-12-01T01:22:33.285 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.285 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.285 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.285 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.285 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.285 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.285 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.286 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.286 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.286 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.286 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.286 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.286 INFO:teuthology.orchestra.run.smithi077.stdout: "crt_file": { 2023-12-01T01:22:33.286 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crt_file", 2023-12-01T01:22:33.286 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.286 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.286 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.286 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.287 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.287 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.287 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.287 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.287 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.287 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.287 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.287 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.287 INFO:teuthology.orchestra.run.smithi077.stdout: "debug": { 2023-12-01T01:22:33.287 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "debug", 2023-12-01T01:22:33.287 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.287 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.288 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.288 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.288 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.288 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.288 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.288 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Enable/disable debug options", 2023-12-01T01:22:33.288 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.288 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.288 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.288 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.288 INFO:teuthology.orchestra.run.smithi077.stdout: "jwt_token_ttl": { 2023-12-01T01:22:33.288 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "jwt_token_ttl", 2023-12-01T01:22:33.288 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.289 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.289 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.289 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "28800", 2023-12-01T01:22:33.289 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.289 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.289 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.289 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.289 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.289 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.289 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.289 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.289 INFO:teuthology.orchestra.run.smithi077.stdout: "key_file": { 2023-12-01T01:22:33.290 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "key_file", 2023-12-01T01:22:33.290 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.290 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.290 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.290 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.290 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.290 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.290 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.290 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.290 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.290 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.290 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.291 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.291 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.291 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.291 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.291 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.291 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.291 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.291 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.291 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.291 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.291 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.291 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.292 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.292 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.292 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.292 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.292 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.292 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.292 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.292 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.292 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.292 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.292 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.292 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.292 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.293 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.293 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.293 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.293 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.293 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.293 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.293 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.293 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.293 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.293 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.294 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.294 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.294 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.294 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.294 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.294 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.294 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.294 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.294 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.294 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.294 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.294 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.294 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.295 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.295 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.295 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.295 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.295 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.295 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.295 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.295 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.295 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.295 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.295 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.295 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.296 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.296 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.296 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.296 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.296 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.296 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.296 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.296 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.296 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.296 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.296 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.296 INFO:teuthology.orchestra.run.smithi077.stdout: "motd": { 2023-12-01T01:22:33.297 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "motd", 2023-12-01T01:22:33.297 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.297 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.297 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.297 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.297 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.297 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.297 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.297 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "The message of the day", 2023-12-01T01:22:33.297 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.297 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.297 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.297 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.298 INFO:teuthology.orchestra.run.smithi077.stdout: "redirect_resolve_ip_addr": { 2023-12-01T01:22:33.298 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "redirect_resolve_ip_addr", 2023-12-01T01:22:33.298 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.298 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.298 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.298 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.298 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.298 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.298 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.298 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.298 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.298 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.299 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.299 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.299 INFO:teuthology.orchestra.run.smithi077.stdout: "server_addr": { 2023-12-01T01:22:33.299 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_addr", 2023-12-01T01:22:33.299 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.299 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.299 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.299 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "::", 2023-12-01T01:22:33.299 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.299 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.299 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.299 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.300 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.300 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.300 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.300 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.300 INFO:teuthology.orchestra.run.smithi077.stdout: "server_port": { 2023-12-01T01:22:33.300 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_port", 2023-12-01T01:22:33.300 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.300 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.300 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.300 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8080", 2023-12-01T01:22:33.300 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.300 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.300 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.301 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.301 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.301 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.301 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.301 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.301 INFO:teuthology.orchestra.run.smithi077.stdout: "ssl": { 2023-12-01T01:22:33.301 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssl", 2023-12-01T01:22:33.301 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.301 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.301 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.301 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.301 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.302 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.302 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.302 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.302 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.302 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.302 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.302 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.302 INFO:teuthology.orchestra.run.smithi077.stdout: "ssl_server_port": { 2023-12-01T01:22:33.302 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssl_server_port", 2023-12-01T01:22:33.302 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.302 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.302 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.302 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8443", 2023-12-01T01:22:33.303 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.303 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.303 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.303 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.303 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.303 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.303 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.303 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.303 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_behaviour": { 2023-12-01T01:22:33.303 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_behaviour", 2023-12-01T01:22:33.303 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.303 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.303 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.304 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "redirect", 2023-12-01T01:22:33.304 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.304 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.304 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.304 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.304 INFO:teuthology.orchestra.run.smithi077.stdout: "redirect" 2023-12-01T01:22:33.304 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.304 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.304 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.304 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.304 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.305 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.305 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_error_status_code": { 2023-12-01T01:22:33.305 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_error_status_code", 2023-12-01T01:22:33.305 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.305 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.305 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.305 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "500", 2023-12-01T01:22:33.305 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "400", 2023-12-01T01:22:33.305 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "599", 2023-12-01T01:22:33.305 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.305 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.305 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.306 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.306 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.306 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.306 INFO:teuthology.orchestra.run.smithi077.stdout: "url_prefix": { 2023-12-01T01:22:33.306 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "url_prefix", 2023-12-01T01:22:33.306 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.306 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.306 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.306 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.306 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.306 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.306 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.307 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.307 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.307 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.307 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.307 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.307 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.307 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.307 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.307 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "devicehealth", 2023-12-01T01:22:33.307 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.307 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.307 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.308 INFO:teuthology.orchestra.run.smithi077.stdout: "enable_monitoring": { 2023-12-01T01:22:33.308 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enable_monitoring", 2023-12-01T01:22:33.308 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.308 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.308 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.308 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.308 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.308 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.308 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.308 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "monitor device health metrics", 2023-12-01T01:22:33.308 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.308 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.308 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.309 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.309 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.309 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.309 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.309 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.309 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.309 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.309 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.309 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.309 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.309 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.310 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.310 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.310 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.310 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.310 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.310 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.310 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.310 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.310 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.310 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.310 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.310 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.310 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.311 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.311 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.311 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.311 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.311 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.311 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.311 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.311 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.311 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.311 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.311 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.312 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.312 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.312 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.312 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.312 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.312 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.312 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.312 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.312 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.312 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.312 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.312 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.312 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.313 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.313 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.313 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.313 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.313 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.313 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.313 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.313 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.313 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.313 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.313 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.313 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.313 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.314 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.314 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.314 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.314 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.314 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.314 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.314 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.314 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.314 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.314 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.314 INFO:teuthology.orchestra.run.smithi077.stdout: "mark_out_threshold": { 2023-12-01T01:22:33.314 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mark_out_threshold", 2023-12-01T01:22:33.314 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:33.315 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.315 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.315 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "2419200", 2023-12-01T01:22:33.315 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.315 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.315 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.315 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-12-01T01:22:33.315 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.315 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.315 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.315 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.315 INFO:teuthology.orchestra.run.smithi077.stdout: "pool_name": { 2023-12-01T01:22:33.315 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pool_name", 2023-12-01T01:22:33.316 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.316 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.316 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.316 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "device_health_metrics", 2023-12-01T01:22:33.316 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.316 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.316 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.316 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "name of pool in which to store device health metrics", 2023-12-01T01:22:33.316 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.316 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.316 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.316 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.316 INFO:teuthology.orchestra.run.smithi077.stdout: "retention_period": { 2023-12-01T01:22:33.317 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "retention_period", 2023-12-01T01:22:33.317 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:33.317 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.317 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.317 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "15552000", 2023-12-01T01:22:33.317 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.317 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.317 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.317 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how long to retain device health metrics", 2023-12-01T01:22:33.317 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.317 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.317 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.318 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.318 INFO:teuthology.orchestra.run.smithi077.stdout: "scrape_frequency": { 2023-12-01T01:22:33.318 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "scrape_frequency", 2023-12-01T01:22:33.318 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:33.318 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.318 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.318 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "86400", 2023-12-01T01:22:33.318 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.318 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.318 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.318 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to scrape device health metrics", 2023-12-01T01:22:33.318 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.319 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.319 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.319 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.319 INFO:teuthology.orchestra.run.smithi077.stdout: "self_heal": { 2023-12-01T01:22:33.319 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "self_heal", 2023-12-01T01:22:33.319 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.319 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.319 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.319 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.319 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.319 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.319 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.320 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-12-01T01:22:33.320 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.320 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.320 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.320 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.320 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2023-12-01T01:22:33.320 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2023-12-01T01:22:33.320 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:33.320 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.320 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.320 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2023-12-01T01:22:33.320 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.321 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.321 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.321 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to wake up and check device health", 2023-12-01T01:22:33.321 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.321 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.321 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.321 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.321 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_threshold": { 2023-12-01T01:22:33.321 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_threshold", 2023-12-01T01:22:33.321 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:33.321 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.322 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.322 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "7257600", 2023-12-01T01:22:33.322 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.322 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.322 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.322 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-12-01T01:22:33.322 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.322 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.322 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.322 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.322 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.322 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.322 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.323 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "diskprediction_local", 2023-12-01T01:22:33.323 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.323 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.323 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.323 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.323 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.323 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.323 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.323 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.323 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.323 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.323 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.323 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.324 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.324 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.324 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.324 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.324 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.324 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.324 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.324 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.324 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.324 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.324 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.324 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.325 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.325 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.325 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.325 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.325 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.325 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.325 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.325 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.325 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.325 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.325 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.326 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.326 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.326 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.326 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.326 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.326 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.326 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.326 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.326 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.326 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.326 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.327 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.327 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.327 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.327 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.327 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.327 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.327 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.327 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.327 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.327 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.327 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.327 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.327 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.328 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.328 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.328 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.328 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.328 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.328 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.328 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.328 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.328 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.328 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.328 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.328 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.329 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.329 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.329 INFO:teuthology.orchestra.run.smithi077.stdout: "predict_interval": { 2023-12-01T01:22:33.329 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "predict_interval", 2023-12-01T01:22:33.329 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.329 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.329 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.329 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "86400", 2023-12-01T01:22:33.329 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.329 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.329 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.329 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.330 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.330 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.330 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.330 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.330 INFO:teuthology.orchestra.run.smithi077.stdout: "predictor_model": { 2023-12-01T01:22:33.330 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "predictor_model", 2023-12-01T01:22:33.330 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.330 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.330 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.330 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "prophetstor", 2023-12-01T01:22:33.330 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.330 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.330 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.331 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.331 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.331 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.331 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.331 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.331 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2023-12-01T01:22:33.331 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2023-12-01T01:22:33.331 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.331 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.331 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.331 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2023-12-01T01:22:33.331 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.331 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.332 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.332 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.332 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.332 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.332 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.332 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.332 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.332 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.332 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.332 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "influx", 2023-12-01T01:22:33.332 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": false, 2023-12-01T01:22:33.332 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "influxdb python module not found", 2023-12-01T01:22:33.333 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.333 INFO:teuthology.orchestra.run.smithi077.stdout: "batch_size": { 2023-12-01T01:22:33.333 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "batch_size", 2023-12-01T01:22:33.333 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.333 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.333 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.333 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5000", 2023-12-01T01:22:33.333 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.333 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.333 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.333 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-12-01T01:22:33.333 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.334 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.334 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.334 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.334 INFO:teuthology.orchestra.run.smithi077.stdout: "database": { 2023-12-01T01:22:33.334 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "database", 2023-12-01T01:22:33.334 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.334 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.334 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.334 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "ceph", 2023-12-01T01:22:33.334 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.334 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.334 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.335 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-12-01T01:22:33.335 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.335 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.335 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.335 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.335 INFO:teuthology.orchestra.run.smithi077.stdout: "hostname": { 2023-12-01T01:22:33.335 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "hostname", 2023-12-01T01:22:33.335 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.335 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.335 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.335 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.335 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.336 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.336 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.336 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "InfluxDB server hostname", 2023-12-01T01:22:33.336 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.336 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.336 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.336 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.336 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2023-12-01T01:22:33.336 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2023-12-01T01:22:33.336 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:33.336 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.336 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.337 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "30", 2023-12-01T01:22:33.337 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "5", 2023-12-01T01:22:33.337 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.337 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.337 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-12-01T01:22:33.337 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.337 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.337 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.337 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.337 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.338 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.338 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.338 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.338 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.338 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.338 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.338 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.338 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.338 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.338 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.338 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.338 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.339 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.339 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.339 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.339 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.339 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.339 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.339 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.339 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.339 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.339 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.339 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.339 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.340 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.340 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.340 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.340 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.340 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.340 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.340 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.340 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.340 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.340 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.340 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.340 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.341 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.341 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.341 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.341 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.341 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.341 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.341 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.341 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.341 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.341 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.341 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.341 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.342 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.342 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.342 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.342 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.342 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.342 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.342 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.342 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.342 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.342 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.342 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.343 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.343 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.343 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.343 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.343 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.343 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.343 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.343 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.343 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.343 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.343 INFO:teuthology.orchestra.run.smithi077.stdout: "password": { 2023-12-01T01:22:33.343 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "password", 2023-12-01T01:22:33.343 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.344 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.344 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.344 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.344 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.344 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.344 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.344 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "password of InfluxDB server user", 2023-12-01T01:22:33.344 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.344 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.344 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.344 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.344 INFO:teuthology.orchestra.run.smithi077.stdout: "port": { 2023-12-01T01:22:33.345 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "port", 2023-12-01T01:22:33.345 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.345 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.345 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.345 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8086", 2023-12-01T01:22:33.345 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.345 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.345 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.345 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "InfluxDB server port", 2023-12-01T01:22:33.345 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.345 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.345 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.346 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.346 INFO:teuthology.orchestra.run.smithi077.stdout: "ssl": { 2023-12-01T01:22:33.346 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssl", 2023-12-01T01:22:33.346 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.346 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.346 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.346 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "false", 2023-12-01T01:22:33.346 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.346 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.346 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.346 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-12-01T01:22:33.347 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.347 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.347 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.347 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.347 INFO:teuthology.orchestra.run.smithi077.stdout: "threads": { 2023-12-01T01:22:33.347 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "threads", 2023-12-01T01:22:33.347 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.347 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.347 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.347 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2023-12-01T01:22:33.347 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2023-12-01T01:22:33.347 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "32", 2023-12-01T01:22:33.347 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.348 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-12-01T01:22:33.348 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.348 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.348 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.348 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.348 INFO:teuthology.orchestra.run.smithi077.stdout: "username": { 2023-12-01T01:22:33.348 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "username", 2023-12-01T01:22:33.348 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.348 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.348 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.348 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.348 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.348 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.349 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.349 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "username of InfluxDB server user", 2023-12-01T01:22:33.349 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.349 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.349 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.349 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.349 INFO:teuthology.orchestra.run.smithi077.stdout: "verify_ssl": { 2023-12-01T01:22:33.349 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "verify_ssl", 2023-12-01T01:22:33.349 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.349 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.349 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.349 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "true", 2023-12-01T01:22:33.350 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.350 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.350 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.350 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-12-01T01:22:33.350 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.350 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.350 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.350 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.350 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.350 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.350 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.350 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "insights", 2023-12-01T01:22:33.351 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.351 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.351 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.351 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.351 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.351 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.351 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.351 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.351 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.351 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.351 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.351 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.352 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.352 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.352 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.352 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.352 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.352 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.352 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.352 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.352 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.352 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.352 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.352 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.352 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.353 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.353 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.353 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.353 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.353 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.353 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.353 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.353 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.353 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.353 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.353 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.353 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.353 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.354 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.354 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.354 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.354 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.354 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.354 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.354 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.354 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.354 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.354 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.354 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.354 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.355 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.355 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.355 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.355 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.355 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.355 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.355 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.355 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.355 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.355 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.355 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.355 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.356 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.356 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.356 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.356 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.356 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.356 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.356 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.356 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.356 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.356 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.356 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.356 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.357 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.357 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.357 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "iostat", 2023-12-01T01:22:33.357 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.357 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.357 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.357 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.357 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.357 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.357 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.357 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.357 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.357 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.358 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.358 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.358 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.358 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.358 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.358 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.358 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.358 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.358 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.358 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.358 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.358 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.359 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.359 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.359 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.359 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.359 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.359 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.359 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.359 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.359 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.359 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.359 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.359 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.359 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.360 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.360 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.360 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.360 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.360 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.360 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.360 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.360 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.360 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.360 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.360 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.360 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.361 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.361 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.361 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.361 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.361 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.361 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.361 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.361 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.361 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.361 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.361 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.361 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.362 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.362 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.362 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.362 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.362 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.362 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.362 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.362 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.362 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.362 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.362 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.362 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.363 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.363 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.363 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.363 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.363 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.363 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "k8sevents", 2023-12-01T01:22:33.363 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.363 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.363 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.363 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_event_retention_days": { 2023-12-01T01:22:33.363 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ceph_event_retention_days", 2023-12-01T01:22:33.363 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.364 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.364 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.364 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "7", 2023-12-01T01:22:33.364 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.364 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.364 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.364 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Days to hold ceph event information within local cache", 2023-12-01T01:22:33.364 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.364 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.364 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.364 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.364 INFO:teuthology.orchestra.run.smithi077.stdout: "config_check_secs": { 2023-12-01T01:22:33.365 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "config_check_secs", 2023-12-01T01:22:33.365 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.365 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.365 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.365 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2023-12-01T01:22:33.365 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "10", 2023-12-01T01:22:33.365 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.365 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.365 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-12-01T01:22:33.365 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.365 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.365 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.365 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.366 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.366 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.366 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.366 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.366 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.366 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.366 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.366 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.366 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.366 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.366 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.366 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.367 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.367 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.367 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.367 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.367 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.367 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.367 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.367 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.367 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.367 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.367 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.367 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.368 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.368 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.368 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.368 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.368 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.368 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.368 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.368 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.368 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.368 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.368 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.368 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.368 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.369 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.369 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.369 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.369 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.369 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.369 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.369 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.369 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.369 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.369 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.369 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.369 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.369 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.370 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.370 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.370 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.370 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.370 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.370 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.370 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.370 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.370 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.370 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.370 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.370 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.371 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.371 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.371 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.371 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.371 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.371 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.371 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.371 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.371 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.371 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.371 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.372 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "localpool", 2023-12-01T01:22:33.372 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.372 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.372 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.372 INFO:teuthology.orchestra.run.smithi077.stdout: "failure_domain": { 2023-12-01T01:22:33.372 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "failure_domain", 2023-12-01T01:22:33.372 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.372 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.372 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.372 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "host", 2023-12-01T01:22:33.372 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.372 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.373 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.373 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "failure domain for any created local pool", 2023-12-01T01:22:33.373 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-12-01T01:22:33.373 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.373 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.373 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.373 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.373 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.373 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.373 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.373 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.373 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.373 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.374 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.374 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.374 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.374 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.374 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.374 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.374 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.374 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.374 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.374 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.374 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.374 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.375 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.375 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.375 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.375 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.375 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.375 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.375 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.375 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.375 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.375 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.375 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.376 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.376 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.376 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.376 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.376 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.376 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.376 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.376 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.376 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.376 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.376 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.376 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.376 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.377 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.377 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.377 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.377 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.377 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.377 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.377 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.377 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.377 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.377 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.377 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.377 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.378 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.378 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.378 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.378 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.378 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.378 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.378 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.378 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.378 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.378 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.378 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.379 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.379 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.379 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.379 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.379 INFO:teuthology.orchestra.run.smithi077.stdout: "min_size": { 2023-12-01T01:22:33.379 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "min_size", 2023-12-01T01:22:33.379 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.379 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.379 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.379 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.379 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.379 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.379 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.380 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "default min_size for any created local pool", 2023-12-01T01:22:33.380 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-12-01T01:22:33.380 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.380 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.380 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.380 INFO:teuthology.orchestra.run.smithi077.stdout: "num_rep": { 2023-12-01T01:22:33.380 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "num_rep", 2023-12-01T01:22:33.380 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.380 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.380 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.380 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "3", 2023-12-01T01:22:33.380 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.381 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.381 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.381 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "default replica count for any created local pool", 2023-12-01T01:22:33.381 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.381 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.381 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.381 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.381 INFO:teuthology.orchestra.run.smithi077.stdout: "pg_num": { 2023-12-01T01:22:33.381 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pg_num", 2023-12-01T01:22:33.381 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.381 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.381 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.382 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "128", 2023-12-01T01:22:33.382 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.382 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.382 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.382 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "default pg_num for any created local pool", 2023-12-01T01:22:33.382 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.382 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.382 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.382 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.382 INFO:teuthology.orchestra.run.smithi077.stdout: "prefix": { 2023-12-01T01:22:33.382 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "prefix", 2023-12-01T01:22:33.382 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.382 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.383 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.383 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.383 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.383 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.383 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.383 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "name prefix for any created local pool", 2023-12-01T01:22:33.383 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.383 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.383 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.383 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.383 INFO:teuthology.orchestra.run.smithi077.stdout: "subtree": { 2023-12-01T01:22:33.383 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "subtree", 2023-12-01T01:22:33.383 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.384 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.384 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.384 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "rack", 2023-12-01T01:22:33.384 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.384 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.384 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.384 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "CRUSH level for which to create a local pool", 2023-12-01T01:22:33.384 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-12-01T01:22:33.384 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.384 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.384 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.384 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.384 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.385 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.385 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mds_autoscaler", 2023-12-01T01:22:33.385 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.385 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.385 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.385 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.385 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.385 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.385 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.385 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.385 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.385 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.385 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.386 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.386 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.386 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.386 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.386 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.386 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.386 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.386 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.386 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.386 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.386 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.387 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.387 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.387 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.387 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.387 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.387 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.387 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.387 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.387 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.387 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.389 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.389 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.389 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.389 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.389 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.389 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.389 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.389 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.389 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.389 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.390 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.390 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.390 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.390 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.390 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.390 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.390 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.390 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.390 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.390 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.390 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.390 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.391 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.391 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.391 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.391 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.391 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.391 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.391 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.391 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.391 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.391 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.391 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.391 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.391 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.392 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.392 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.392 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.392 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.392 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.392 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.392 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.392 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.392 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.392 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mirroring", 2023-12-01T01:22:33.392 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.392 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.392 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.393 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.393 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.393 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.393 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.393 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.393 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.393 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.393 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.393 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.393 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.393 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.393 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.393 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.394 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.394 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.394 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.394 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.394 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.394 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.394 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.394 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.394 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.394 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.394 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.394 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.394 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.395 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.395 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.395 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.395 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.395 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.395 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.395 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.395 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.395 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.395 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.395 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.395 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.395 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.396 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.396 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.396 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.396 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.396 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.396 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.396 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.396 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.396 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.396 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.396 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.396 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.396 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.397 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.397 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.397 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.397 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.397 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.397 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.397 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.397 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.397 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.397 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.397 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.397 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.398 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.398 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.398 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.398 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.398 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.398 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.398 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.398 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.398 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.398 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "nfs", 2023-12-01T01:22:33.398 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.398 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.399 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.399 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.399 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.399 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.399 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.399 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.399 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.399 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.399 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.399 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.399 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.399 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.400 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.400 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.400 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.400 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.400 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.400 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.400 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.400 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.400 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.400 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.400 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.400 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.400 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.401 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.401 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.401 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.401 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.401 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.401 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.401 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.401 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.401 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.401 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.401 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.401 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.402 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.402 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.402 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.402 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.402 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.402 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.402 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.402 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.402 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.402 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.402 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.402 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.402 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.403 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.403 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.403 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.403 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.403 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.403 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.403 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.403 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.403 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.403 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.403 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.404 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.404 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.404 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.404 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.404 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.404 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.404 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.404 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.404 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.404 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.404 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.404 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.404 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.405 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "orchestrator", 2023-12-01T01:22:33.405 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.405 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.405 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.405 INFO:teuthology.orchestra.run.smithi077.stdout: "fail_fs": { 2023-12-01T01:22:33.405 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "fail_fs", 2023-12-01T01:22:33.405 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.405 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.405 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.405 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.405 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.405 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.405 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.406 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-12-01T01:22:33.406 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.406 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.406 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.406 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.406 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.406 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.406 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.406 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.406 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.406 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.406 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.407 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.407 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.407 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.407 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.407 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.407 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.407 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.407 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.407 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.407 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.407 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.407 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.408 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.408 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.408 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.408 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.408 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.408 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.408 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.408 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.408 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.408 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.408 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.408 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.409 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.409 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.409 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.409 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.409 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.409 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.409 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.409 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.409 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.409 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.409 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.410 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.410 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.410 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.410 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.410 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.410 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.410 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.410 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.410 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.410 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.410 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.410 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.411 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.411 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.411 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.411 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.411 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.411 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.411 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.411 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.411 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.411 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.411 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.411 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.411 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.412 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.412 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.412 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.412 INFO:teuthology.orchestra.run.smithi077.stdout: "orchestrator": { 2023-12-01T01:22:33.412 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "orchestrator", 2023-12-01T01:22:33.412 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.412 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.412 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.412 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.412 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.412 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.412 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.413 INFO:teuthology.orchestra.run.smithi077.stdout: "cephadm", 2023-12-01T01:22:33.413 INFO:teuthology.orchestra.run.smithi077.stdout: "rook", 2023-12-01T01:22:33.413 INFO:teuthology.orchestra.run.smithi077.stdout: "test_orchestrator" 2023-12-01T01:22:33.413 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.413 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Orchestrator backend", 2023-12-01T01:22:33.413 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.413 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.413 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.413 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.413 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.413 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.413 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.414 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "osd_perf_query", 2023-12-01T01:22:33.414 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.414 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.414 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.414 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.414 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.414 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.414 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.414 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.414 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.414 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.415 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.415 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.415 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.415 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.415 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.415 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.415 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.415 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.415 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.415 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.415 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.415 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.416 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.416 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.416 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.416 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.416 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.416 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.416 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.416 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.416 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.416 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.416 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.416 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.417 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.417 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.417 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.417 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.417 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.417 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.417 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.417 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.417 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.417 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.417 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.418 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.418 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.418 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.418 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.418 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.418 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.418 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.418 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.418 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.418 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.419 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.419 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.419 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.419 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.419 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.419 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.419 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.419 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.419 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.419 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.419 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.419 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.419 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.420 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.420 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.420 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.420 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.420 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.420 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.420 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.420 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.420 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "osd_support", 2023-12-01T01:22:33.420 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.421 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.421 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.421 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.421 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.421 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.421 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.421 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.421 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.421 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.421 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.422 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.422 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.422 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.422 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.422 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.422 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.422 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.422 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.422 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.422 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.422 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.422 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.423 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.423 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.423 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.423 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.423 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.423 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.423 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.423 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.423 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.423 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.423 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.424 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.424 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.424 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.424 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.424 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.424 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.424 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.424 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.424 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.424 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.424 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.425 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.425 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.425 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.425 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.425 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.425 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.425 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.425 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.425 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.425 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.425 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.425 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.425 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.426 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.426 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.426 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.426 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.426 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.426 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.426 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.426 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.426 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.426 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.426 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.426 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.427 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.427 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.427 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.427 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.427 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.427 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.427 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pg_autoscaler", 2023-12-01T01:22:33.427 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.427 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.427 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.427 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.427 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.427 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.428 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.428 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.428 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.428 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.428 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.428 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.428 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.428 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.428 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.428 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.428 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.428 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.429 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.429 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.429 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.429 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.429 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.429 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.429 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.429 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.429 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.429 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.429 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.429 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.429 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.430 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.430 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.430 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.430 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.430 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.430 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.430 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.430 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.430 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.430 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.430 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.430 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.431 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.431 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.431 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.431 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.431 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.431 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.431 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.431 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.431 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.431 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.431 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.431 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.431 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.432 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.432 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.432 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.432 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.432 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.432 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.432 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.432 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.432 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.432 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.432 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.432 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.432 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.433 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.433 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.433 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.433 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.433 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2023-12-01T01:22:33.433 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2023-12-01T01:22:33.433 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:33.433 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.433 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.433 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2023-12-01T01:22:33.433 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.433 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.434 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.434 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.434 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.434 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.434 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.434 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.434 INFO:teuthology.orchestra.run.smithi077.stdout: "threshold": { 2023-12-01T01:22:33.434 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "threshold", 2023-12-01T01:22:33.434 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2023-12-01T01:22:33.434 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.434 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.434 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "3.0", 2023-12-01T01:22:33.434 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1.0", 2023-12-01T01:22:33.435 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.435 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.435 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "scaling threshold", 2023-12-01T01:22:33.435 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-12-01T01:22:33.435 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.435 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.435 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.435 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.435 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.435 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.435 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "progress", 2023-12-01T01:22:33.435 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.435 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.436 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.436 INFO:teuthology.orchestra.run.smithi077.stdout: "allow_pg_recovery_event": { 2023-12-01T01:22:33.436 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "allow_pg_recovery_event", 2023-12-01T01:22:33.436 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.436 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.436 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.436 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.436 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.436 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.436 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.436 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "allow the module to show pg recovery progress", 2023-12-01T01:22:33.437 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.437 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.437 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.437 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.437 INFO:teuthology.orchestra.run.smithi077.stdout: "enabled": { 2023-12-01T01:22:33.437 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enabled", 2023-12-01T01:22:33.437 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.437 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.437 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.437 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.437 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.437 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.437 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.438 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.438 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.438 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.438 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.438 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.438 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.438 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.438 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.438 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.438 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.438 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.438 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.439 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.439 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.439 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.439 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.439 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.439 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.439 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.439 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.439 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.439 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.439 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.439 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.439 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.440 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.440 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.440 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.440 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.440 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.440 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.440 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.440 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.440 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.440 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.440 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.440 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.441 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.441 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.441 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.441 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.441 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.441 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.441 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.441 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.441 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.441 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.441 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.441 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.441 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.442 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.442 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.442 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.442 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.442 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.442 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.442 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.442 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.442 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.442 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.442 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.442 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.443 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.443 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.443 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.443 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.443 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.443 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.443 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.443 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.443 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.443 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.443 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.443 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.443 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.444 INFO:teuthology.orchestra.run.smithi077.stdout: "max_completed_events": { 2023-12-01T01:22:33.444 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_completed_events", 2023-12-01T01:22:33.444 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.444 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.444 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.444 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "50", 2023-12-01T01:22:33.444 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.444 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.444 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.444 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "number of past completed events to remember", 2023-12-01T01:22:33.444 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.444 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.445 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.445 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.445 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2023-12-01T01:22:33.445 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2023-12-01T01:22:33.445 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:33.445 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.445 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.445 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2023-12-01T01:22:33.445 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.445 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.445 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.445 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how long the module is going to sleep", 2023-12-01T01:22:33.446 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.446 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.446 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.446 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.446 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.446 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.446 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.446 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "prometheus", 2023-12-01T01:22:33.446 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.446 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.446 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.446 INFO:teuthology.orchestra.run.smithi077.stdout: "cache": { 2023-12-01T01:22:33.447 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cache", 2023-12-01T01:22:33.447 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.447 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.447 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.447 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.447 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.447 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.447 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.447 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.447 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.447 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.447 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.447 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.448 INFO:teuthology.orchestra.run.smithi077.stdout: "exclude_perf_counters": { 2023-12-01T01:22:33.448 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "exclude_perf_counters", 2023-12-01T01:22:33.448 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.448 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.448 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.448 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.448 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.448 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.448 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.448 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-12-01T01:22:33.448 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-12-01T01:22:33.449 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.449 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.449 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.449 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.449 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.449 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.449 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.449 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.449 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.449 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.449 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.449 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.449 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.450 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.450 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.450 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.450 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.450 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.450 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.450 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.450 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.450 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.450 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.450 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.450 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.451 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.451 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.451 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.451 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.451 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.451 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.451 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.451 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.451 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.451 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.452 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.452 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.452 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.452 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.452 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.452 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.452 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.452 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.452 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.452 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.452 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.452 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.452 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.453 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.453 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.453 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.453 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.453 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.453 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.453 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.453 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.453 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.453 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.453 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.453 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.453 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.454 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.454 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.454 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.454 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.454 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.454 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.454 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.454 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.454 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.454 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.454 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.454 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.455 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_stats_pools": { 2023-12-01T01:22:33.455 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rbd_stats_pools", 2023-12-01T01:22:33.455 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.455 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.455 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.455 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.455 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.455 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.455 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.455 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.455 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.455 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.456 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.456 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.456 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_stats_pools_refresh_interval": { 2023-12-01T01:22:33.456 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-12-01T01:22:33.456 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.456 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.456 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.456 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "300", 2023-12-01T01:22:33.456 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.456 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.456 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.456 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.457 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.457 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.457 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.457 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.457 INFO:teuthology.orchestra.run.smithi077.stdout: "scrape_interval": { 2023-12-01T01:22:33.457 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "scrape_interval", 2023-12-01T01:22:33.457 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2023-12-01T01:22:33.457 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.457 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.457 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "15.0", 2023-12-01T01:22:33.457 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.458 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.458 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.458 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.458 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.458 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.458 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.458 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.458 INFO:teuthology.orchestra.run.smithi077.stdout: "server_addr": { 2023-12-01T01:22:33.458 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_addr", 2023-12-01T01:22:33.458 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.458 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.458 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.459 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "::", 2023-12-01T01:22:33.459 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.459 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.459 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.459 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-12-01T01:22:33.459 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.459 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.459 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.459 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.459 INFO:teuthology.orchestra.run.smithi077.stdout: "server_port": { 2023-12-01T01:22:33.459 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_port", 2023-12-01T01:22:33.459 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.460 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.460 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.460 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "9283", 2023-12-01T01:22:33.460 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.460 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.460 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.460 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-12-01T01:22:33.460 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.460 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.460 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.460 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.461 INFO:teuthology.orchestra.run.smithi077.stdout: "stale_cache_strategy": { 2023-12-01T01:22:33.461 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "stale_cache_strategy", 2023-12-01T01:22:33.461 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.461 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.461 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.461 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "log", 2023-12-01T01:22:33.461 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.461 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.461 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.461 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.461 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.462 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.462 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.462 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.462 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_behaviour": { 2023-12-01T01:22:33.462 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_behaviour", 2023-12-01T01:22:33.462 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.462 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.462 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.462 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "default", 2023-12-01T01:22:33.462 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.463 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.463 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.463 INFO:teuthology.orchestra.run.smithi077.stdout: "default", 2023-12-01T01:22:33.463 INFO:teuthology.orchestra.run.smithi077.stdout: "error" 2023-12-01T01:22:33.463 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.463 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.463 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.463 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.463 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.463 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.463 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_error_status_code": { 2023-12-01T01:22:33.463 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_error_status_code", 2023-12-01T01:22:33.463 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.464 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.464 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.464 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "500", 2023-12-01T01:22:33.464 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "400", 2023-12-01T01:22:33.464 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "599", 2023-12-01T01:22:33.464 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.464 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.464 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.464 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.464 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.464 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.464 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.465 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.465 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.465 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rbd_support", 2023-12-01T01:22:33.465 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.465 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.465 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.465 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.465 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.465 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.465 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.465 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.465 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.466 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.466 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.466 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.466 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.466 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.466 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.466 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.466 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.466 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.466 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.466 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.466 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.466 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.467 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.467 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.467 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.467 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.467 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.467 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.467 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.467 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.467 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.467 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.467 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.468 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.468 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.468 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.468 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.468 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.468 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.468 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.468 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.468 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.468 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.468 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.468 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.468 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.469 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.469 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.469 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.469 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.469 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.469 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.469 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.469 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.469 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.469 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.469 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.469 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.470 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.470 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.470 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.470 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.470 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.470 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.470 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.470 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.470 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.470 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.470 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.470 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.470 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.471 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.471 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.471 INFO:teuthology.orchestra.run.smithi077.stdout: "max_concurrent_snap_create": { 2023-12-01T01:22:33.471 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_concurrent_snap_create", 2023-12-01T01:22:33.471 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.471 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.471 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.471 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2023-12-01T01:22:33.471 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.471 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.471 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.471 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.472 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.472 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.472 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.472 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.472 INFO:teuthology.orchestra.run.smithi077.stdout: "mirror_snapshot_schedule": { 2023-12-01T01:22:33.472 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mirror_snapshot_schedule", 2023-12-01T01:22:33.472 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.472 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.472 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.472 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.472 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.472 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.473 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.473 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.473 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.473 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.473 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.473 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.473 INFO:teuthology.orchestra.run.smithi077.stdout: "trash_purge_schedule": { 2023-12-01T01:22:33.473 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "trash_purge_schedule", 2023-12-01T01:22:33.473 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.473 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.473 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.473 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.474 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.474 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.474 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.474 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.474 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.474 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.474 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.474 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.474 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.474 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.474 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.474 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "restful", 2023-12-01T01:22:33.475 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.475 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.475 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.475 INFO:teuthology.orchestra.run.smithi077.stdout: "enable_auth": { 2023-12-01T01:22:33.475 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enable_auth", 2023-12-01T01:22:33.475 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.475 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.475 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.475 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.475 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.475 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.475 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.475 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.476 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.476 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.476 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.476 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.476 INFO:teuthology.orchestra.run.smithi077.stdout: "key_file": { 2023-12-01T01:22:33.476 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "key_file", 2023-12-01T01:22:33.476 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.476 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.476 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.476 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.476 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.476 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.476 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.477 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.477 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.477 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.477 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.477 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.477 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.477 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.477 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.477 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.477 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.477 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.477 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.478 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.478 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.478 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.478 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.478 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.478 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.478 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.478 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.478 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.478 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.478 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.478 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.479 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.479 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.479 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.479 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.479 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.479 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.479 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.479 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.479 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.479 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.479 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.479 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.480 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.480 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.480 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.480 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.480 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.480 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.480 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.480 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.480 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.480 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.480 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.480 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.480 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.481 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.481 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.481 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.481 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.481 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.481 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.481 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.481 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.481 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.481 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.481 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.481 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.482 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.482 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.482 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.482 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.482 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.482 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.482 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.482 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.482 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.482 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.482 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.482 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.483 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.483 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.483 INFO:teuthology.orchestra.run.smithi077.stdout: "server_addr": { 2023-12-01T01:22:33.483 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_addr", 2023-12-01T01:22:33.483 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.483 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.483 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.483 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.483 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.484 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.484 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.484 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.484 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.484 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.484 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.484 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.484 INFO:teuthology.orchestra.run.smithi077.stdout: "server_port": { 2023-12-01T01:22:33.484 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_port", 2023-12-01T01:22:33.484 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.484 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.484 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.484 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.485 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.485 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.485 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.485 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.485 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.485 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.485 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.485 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.485 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.485 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.485 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.485 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rgw", 2023-12-01T01:22:33.486 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.486 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.486 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.486 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.486 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.486 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.486 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.486 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.486 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.486 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.486 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.486 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.487 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.487 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.487 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.487 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.487 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.487 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.487 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.487 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.487 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.487 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.487 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.487 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.487 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.488 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.488 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.488 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.488 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.488 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.488 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.488 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.488 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.488 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.488 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.488 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.488 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.488 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.489 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.489 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.489 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.489 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.489 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.489 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.489 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.489 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.489 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.489 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.489 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.489 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.490 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.490 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.490 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.490 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.490 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.490 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.490 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.490 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.490 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.490 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.490 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.490 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.491 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.491 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.491 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.491 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.491 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.491 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.491 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.491 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.491 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.491 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.491 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.491 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.491 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.492 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.492 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rook", 2023-12-01T01:22:33.492 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.492 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.492 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.492 INFO:teuthology.orchestra.run.smithi077.stdout: "drive_group_interval": { 2023-12-01T01:22:33.492 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "drive_group_interval", 2023-12-01T01:22:33.492 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2023-12-01T01:22:33.492 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.492 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.492 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "300.0", 2023-12-01T01:22:33.493 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.493 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.493 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.493 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-12-01T01:22:33.493 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.493 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.493 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.493 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.493 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.493 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.493 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.493 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.493 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.494 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.494 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.494 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.494 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.494 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.494 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.494 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.494 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.494 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.494 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.494 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.494 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.495 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.495 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.495 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.495 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.495 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.495 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.495 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.495 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.495 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.495 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.495 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.495 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.495 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.496 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.496 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.496 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.496 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.496 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.496 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.496 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.496 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.496 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.496 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.496 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.496 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.497 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.497 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.497 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.497 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.497 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.497 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.497 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.497 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.497 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.497 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.497 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.497 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.497 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.498 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.498 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.498 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.498 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.498 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.498 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.498 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.498 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.498 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.498 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.498 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.498 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.499 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.499 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.499 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.499 INFO:teuthology.orchestra.run.smithi077.stdout: "storage_class": { 2023-12-01T01:22:33.499 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "storage_class", 2023-12-01T01:22:33.499 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.499 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.499 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.499 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "local", 2023-12-01T01:22:33.499 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.499 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.499 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.500 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-12-01T01:22:33.500 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.500 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.500 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.500 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.500 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.500 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.500 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.500 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "selftest", 2023-12-01T01:22:33.500 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.500 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.500 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.500 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.501 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.501 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.501 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.501 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.501 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.501 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.501 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.501 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.501 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.501 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.501 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.501 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.502 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.502 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.502 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.502 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.502 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.502 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.502 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.502 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.502 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.502 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.502 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.502 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.503 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.503 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.503 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.503 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.503 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.503 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.503 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.503 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.503 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.503 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.503 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.503 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.504 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.504 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.504 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.504 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.504 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.504 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.504 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.504 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.504 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.504 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.504 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.504 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.505 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.505 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.505 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.505 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.505 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.505 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.505 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.505 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.505 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.505 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.505 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.505 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.505 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.506 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.506 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.506 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.506 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.506 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.506 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.506 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.506 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.506 INFO:teuthology.orchestra.run.smithi077.stdout: "roption1": { 2023-12-01T01:22:33.506 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "roption1", 2023-12-01T01:22:33.506 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.507 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.507 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.507 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.507 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.507 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.507 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.507 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.507 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.507 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.507 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.507 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.507 INFO:teuthology.orchestra.run.smithi077.stdout: "roption2": { 2023-12-01T01:22:33.508 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "roption2", 2023-12-01T01:22:33.508 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.508 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.508 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.508 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "xyz", 2023-12-01T01:22:33.508 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.508 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.508 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.508 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.508 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.508 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.508 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.508 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.509 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption1": { 2023-12-01T01:22:33.509 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption1", 2023-12-01T01:22:33.509 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.509 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.509 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.509 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.509 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.509 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.509 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.509 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.509 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.509 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.509 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.510 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.510 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption2": { 2023-12-01T01:22:33.510 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption2", 2023-12-01T01:22:33.510 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.510 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.510 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.510 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.510 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.510 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.510 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.510 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.510 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.511 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.511 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.511 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.511 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption3": { 2023-12-01T01:22:33.511 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption3", 2023-12-01T01:22:33.511 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2023-12-01T01:22:33.511 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.511 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.511 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.511 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.511 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.511 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.511 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.512 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.512 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.512 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.512 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.512 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption4": { 2023-12-01T01:22:33.512 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption4", 2023-12-01T01:22:33.512 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.512 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.512 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.512 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.512 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.512 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.512 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.513 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.513 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.513 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.513 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.513 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.513 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption5": { 2023-12-01T01:22:33.513 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption5", 2023-12-01T01:22:33.513 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.513 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.513 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.513 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.513 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.514 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.514 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.514 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.514 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.514 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.514 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.514 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.514 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption6": { 2023-12-01T01:22:33.514 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption6", 2023-12-01T01:22:33.514 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.514 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.515 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.515 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.515 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.515 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.515 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.515 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.515 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.515 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.515 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.515 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.515 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption7": { 2023-12-01T01:22:33.516 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption7", 2023-12-01T01:22:33.516 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.516 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.516 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.516 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.516 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2023-12-01T01:22:33.516 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "42", 2023-12-01T01:22:33.516 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.516 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.516 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.516 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.516 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.517 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.517 INFO:teuthology.orchestra.run.smithi077.stdout: "testkey": { 2023-12-01T01:22:33.517 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "testkey", 2023-12-01T01:22:33.517 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.517 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.517 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.517 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.517 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.517 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.517 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.517 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.517 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.517 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.518 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.518 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.518 INFO:teuthology.orchestra.run.smithi077.stdout: "testlkey": { 2023-12-01T01:22:33.518 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "testlkey", 2023-12-01T01:22:33.518 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.518 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.518 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.518 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.518 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.518 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.518 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.519 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.519 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.519 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.519 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.519 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.519 INFO:teuthology.orchestra.run.smithi077.stdout: "testnewline": { 2023-12-01T01:22:33.519 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "testnewline", 2023-12-01T01:22:33.519 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.519 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.519 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.519 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.519 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.519 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.520 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.520 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.520 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.520 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.520 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.520 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.520 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.520 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.520 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.520 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "snap_schedule", 2023-12-01T01:22:33.520 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.520 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.521 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.521 INFO:teuthology.orchestra.run.smithi077.stdout: "allow_m_granularity": { 2023-12-01T01:22:33.521 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "allow_m_granularity", 2023-12-01T01:22:33.521 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.521 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.521 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.521 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.521 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.521 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.521 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.521 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "allow minute scheduled snapshots", 2023-12-01T01:22:33.521 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.522 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.522 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.522 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.522 INFO:teuthology.orchestra.run.smithi077.stdout: "dump_on_update": { 2023-12-01T01:22:33.522 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "dump_on_update", 2023-12-01T01:22:33.522 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.522 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.522 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.522 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.522 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.522 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.522 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.523 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "dump database to debug log on update", 2023-12-01T01:22:33.523 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.523 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.523 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.523 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.523 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.523 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.523 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.523 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.523 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.523 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.523 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.523 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.524 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.524 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.524 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.524 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.524 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.524 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.524 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.524 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.524 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.524 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.524 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.524 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.525 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.525 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.525 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.525 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.525 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.525 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.525 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.525 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.525 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.525 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.525 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.525 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.525 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.526 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.526 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.526 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.526 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.526 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.526 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.526 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.526 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.526 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.526 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.526 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.526 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.527 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.527 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.527 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.527 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.527 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.527 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.527 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.527 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.527 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.527 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.527 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.527 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.527 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.528 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.528 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.528 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.528 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.528 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.528 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.528 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.528 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.528 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.528 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.528 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.528 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.529 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.529 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.529 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.529 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "stats", 2023-12-01T01:22:33.529 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.529 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.529 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.529 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.529 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.529 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.529 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.530 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.530 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.530 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.530 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.530 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.530 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.530 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.530 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.530 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.530 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.530 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.530 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.531 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.531 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.531 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.531 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.531 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.531 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.531 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.531 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.531 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.531 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.531 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.532 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.532 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.532 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.532 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.532 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.532 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.532 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.532 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.532 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.532 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.532 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.532 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.532 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.533 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.533 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.533 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.533 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.533 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.533 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.533 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.533 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.533 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.533 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.533 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.533 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.534 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.534 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.534 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.534 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.534 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.534 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.534 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.534 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.534 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.534 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.534 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.534 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.534 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.535 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.535 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.535 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.535 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.535 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.535 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.535 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.535 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.535 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "status", 2023-12-01T01:22:33.535 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.535 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.535 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.535 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.536 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.536 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.536 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.536 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.536 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.536 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.536 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.536 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.536 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.536 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.536 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.536 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.536 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.537 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.537 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.537 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.537 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.537 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.537 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.537 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.537 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.537 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.537 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.537 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.537 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.538 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.538 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.538 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.538 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.538 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.538 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.538 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.538 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.538 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.538 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.538 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.538 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.539 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.539 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.539 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.539 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.539 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.539 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.539 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.539 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.539 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.539 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.539 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.539 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.539 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.540 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.540 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.540 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.540 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.540 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.540 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.540 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.540 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.540 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.540 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.540 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.540 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.540 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.541 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.541 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.541 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.541 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.541 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.541 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.541 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.541 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.541 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.541 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "telegraf", 2023-12-01T01:22:33.541 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.541 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.541 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.542 INFO:teuthology.orchestra.run.smithi077.stdout: "address": { 2023-12-01T01:22:33.542 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "address", 2023-12-01T01:22:33.542 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.542 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.542 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.542 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-12-01T01:22:33.542 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.542 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.542 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.542 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.542 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.542 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.542 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.543 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.543 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2023-12-01T01:22:33.543 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2023-12-01T01:22:33.543 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:33.543 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.543 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.543 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "15", 2023-12-01T01:22:33.543 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.543 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.543 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.543 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.543 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.543 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.544 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.544 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.544 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.544 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.544 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.544 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.544 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.544 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.544 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.544 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.544 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.544 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.545 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.545 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.545 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.545 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.545 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.545 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.545 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.545 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.545 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.545 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.545 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.545 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.546 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.546 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.546 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.546 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.546 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.546 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.546 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.546 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.546 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.546 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.546 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.547 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.547 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.547 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.547 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.547 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.547 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.547 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.547 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.547 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.547 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.547 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.547 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.548 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.548 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.548 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.548 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.548 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.548 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.548 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.548 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.548 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.548 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.548 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.549 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.549 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.549 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.549 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.549 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.549 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.549 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.549 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.549 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.549 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.549 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.549 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.550 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.550 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.550 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.550 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.550 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.550 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "telemetry", 2023-12-01T01:22:33.550 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.550 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.550 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.550 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_basic": { 2023-12-01T01:22:33.550 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_basic", 2023-12-01T01:22:33.550 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.550 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.551 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.551 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.551 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.551 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.551 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.551 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share basic cluster information (size, version)", 2023-12-01T01:22:33.551 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.551 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.551 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.551 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.551 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_crash": { 2023-12-01T01:22:33.551 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_crash", 2023-12-01T01:22:33.552 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.552 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.552 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.552 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.552 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.552 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.552 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.552 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-12-01T01:22:33.552 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.552 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.552 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.552 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.553 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_device": { 2023-12-01T01:22:33.553 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_device", 2023-12-01T01:22:33.553 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.553 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.553 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.553 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.553 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.553 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.553 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.553 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-12-01T01:22:33.553 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.553 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.553 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.554 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.554 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_ident": { 2023-12-01T01:22:33.554 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_ident", 2023-12-01T01:22:33.554 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.554 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.554 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.554 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.554 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.554 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.554 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.554 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-12-01T01:22:33.554 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.554 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.555 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.555 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.555 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_perf": { 2023-12-01T01:22:33.555 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_perf", 2023-12-01T01:22:33.555 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.555 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.555 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.555 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.555 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.555 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.555 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.555 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share various performance metrics of a cluster", 2023-12-01T01:22:33.555 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.556 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.556 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.556 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.556 INFO:teuthology.orchestra.run.smithi077.stdout: "contact": { 2023-12-01T01:22:33.556 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "contact", 2023-12-01T01:22:33.556 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.556 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.556 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.556 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.556 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.556 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.556 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.556 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.557 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.557 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.557 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.557 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.557 INFO:teuthology.orchestra.run.smithi077.stdout: "description": { 2023-12-01T01:22:33.557 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "description", 2023-12-01T01:22:33.557 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.557 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.557 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.557 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.557 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.557 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.557 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.558 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.558 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.558 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.558 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.558 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.558 INFO:teuthology.orchestra.run.smithi077.stdout: "device_url": { 2023-12-01T01:22:33.558 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "device_url", 2023-12-01T01:22:33.558 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.558 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.558 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.558 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-12-01T01:22:33.558 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.559 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.559 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.559 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.559 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.559 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.559 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.559 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.559 INFO:teuthology.orchestra.run.smithi077.stdout: "enabled": { 2023-12-01T01:22:33.559 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enabled", 2023-12-01T01:22:33.559 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.559 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.559 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.559 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.560 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.560 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.560 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.560 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.560 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.560 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.560 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.560 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.560 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2023-12-01T01:22:33.560 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2023-12-01T01:22:33.560 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.560 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.560 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.561 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "24", 2023-12-01T01:22:33.561 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "8", 2023-12-01T01:22:33.561 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.561 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.561 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.561 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.561 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.561 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.561 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.561 INFO:teuthology.orchestra.run.smithi077.stdout: "last_opt_revision": { 2023-12-01T01:22:33.561 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "last_opt_revision", 2023-12-01T01:22:33.561 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.562 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.562 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.562 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "1", 2023-12-01T01:22:33.562 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.562 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.562 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.562 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.562 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.562 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.562 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.562 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.562 INFO:teuthology.orchestra.run.smithi077.stdout: "leaderboard": { 2023-12-01T01:22:33.563 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "leaderboard", 2023-12-01T01:22:33.563 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.563 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.563 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.563 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.563 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.563 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.563 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.563 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.563 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.563 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.563 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.563 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.564 INFO:teuthology.orchestra.run.smithi077.stdout: "leaderboard_description": { 2023-12-01T01:22:33.564 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "leaderboard_description", 2023-12-01T01:22:33.564 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.564 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.564 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.564 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.564 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.564 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.564 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.564 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.564 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.564 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.565 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.565 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.565 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.565 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.565 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.565 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.565 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.565 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.565 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.565 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.565 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.565 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.565 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.566 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.566 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.566 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.566 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.566 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.566 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.566 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.566 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.566 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.566 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.566 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.566 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.567 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.567 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.567 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.567 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.567 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.567 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.567 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.567 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.567 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.567 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.567 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.567 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.567 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.568 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.568 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.568 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.568 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.568 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.568 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.568 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.568 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.568 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.568 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.568 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.568 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.568 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.569 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.569 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.569 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.569 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.569 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.569 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.569 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.569 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.569 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.569 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.569 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.569 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.569 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.570 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.570 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.570 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.570 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.570 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.570 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.570 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.570 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.570 INFO:teuthology.orchestra.run.smithi077.stdout: "organization": { 2023-12-01T01:22:33.570 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "organization", 2023-12-01T01:22:33.570 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.570 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.571 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.571 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.571 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.571 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.571 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.571 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.571 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.571 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.571 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.571 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.571 INFO:teuthology.orchestra.run.smithi077.stdout: "proxy": { 2023-12-01T01:22:33.571 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "proxy", 2023-12-01T01:22:33.572 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.572 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.572 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.572 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.572 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.572 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.572 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.572 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.572 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.572 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.572 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.572 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.572 INFO:teuthology.orchestra.run.smithi077.stdout: "url": { 2023-12-01T01:22:33.572 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "url", 2023-12-01T01:22:33.573 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.573 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.573 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.573 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-12-01T01:22:33.573 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.573 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.573 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.573 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.573 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.573 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.573 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.573 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.573 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.574 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.574 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.574 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "test_orchestrator", 2023-12-01T01:22:33.574 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.574 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.574 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.574 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.574 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.574 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.574 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.574 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.574 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.575 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.575 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.575 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.575 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.575 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.575 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.575 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.575 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.575 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.575 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.575 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.575 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.575 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.576 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.576 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.576 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.576 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.576 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.576 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.576 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.576 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.576 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.576 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.576 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.576 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.576 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.577 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.577 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.577 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.577 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.577 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.577 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.577 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.577 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.577 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.577 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.577 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.578 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.578 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.578 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.578 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.578 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.578 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.578 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.578 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.578 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.578 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.578 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.578 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.579 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.579 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.579 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.579 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.579 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.579 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.579 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.579 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.579 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.579 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.579 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.579 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.580 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.580 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.580 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.580 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.580 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.580 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.580 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "volumes", 2023-12-01T01:22:33.580 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.580 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.580 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.580 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.580 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.581 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.581 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.581 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.581 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.581 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.581 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.581 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.581 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.581 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.581 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.581 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.581 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.581 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.582 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.582 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.582 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.582 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.582 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.582 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.582 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.582 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.582 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.582 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.582 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.582 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.582 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.583 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.583 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.583 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.583 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.583 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.583 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.583 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.583 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.583 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.583 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.583 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.583 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.583 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.584 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.584 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.584 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.584 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.584 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.584 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.584 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.584 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.584 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.584 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.584 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.584 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.585 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.585 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.585 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.585 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.585 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.585 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.585 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.585 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.585 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.585 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.585 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.585 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.586 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.586 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.586 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.586 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.586 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.586 INFO:teuthology.orchestra.run.smithi077.stdout: "max_concurrent_clones": { 2023-12-01T01:22:33.586 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_concurrent_clones", 2023-12-01T01:22:33.586 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.586 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.586 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.586 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "4", 2023-12-01T01:22:33.586 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.587 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.587 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.587 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Number of asynchronous cloner threads", 2023-12-01T01:22:33.587 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.587 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.587 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.587 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.587 INFO:teuthology.orchestra.run.smithi077.stdout: "snapshot_clone_delay": { 2023-12-01T01:22:33.587 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "snapshot_clone_delay", 2023-12-01T01:22:33.587 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.587 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.587 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.588 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2023-12-01T01:22:33.588 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.588 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.588 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.588 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-12-01T01:22:33.588 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.588 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.588 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.588 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.588 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.588 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.588 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.588 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix", 2023-12-01T01:22:33.589 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.589 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.589 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.589 INFO:teuthology.orchestra.run.smithi077.stdout: "discovery_interval": { 2023-12-01T01:22:33.589 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "discovery_interval", 2023-12-01T01:22:33.589 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2023-12-01T01:22:33.589 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.589 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.589 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "100", 2023-12-01T01:22:33.589 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.589 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.589 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.589 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.590 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.590 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.590 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.590 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.590 INFO:teuthology.orchestra.run.smithi077.stdout: "identifier": { 2023-12-01T01:22:33.590 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "identifier", 2023-12-01T01:22:33.590 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.590 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.590 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.590 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.590 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.590 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.591 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.591 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.591 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.591 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.591 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.591 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.591 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2023-12-01T01:22:33.593 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2023-12-01T01:22:33.593 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:33.593 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.593 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.593 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2023-12-01T01:22:33.593 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.593 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.593 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.594 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.594 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.594 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.594 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.594 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.594 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.594 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.594 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.594 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.594 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.595 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.595 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.595 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.595 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.595 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.595 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.595 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.595 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.595 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.595 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.595 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.596 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.596 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.596 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.596 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.596 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.596 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.596 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.596 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.596 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.596 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.596 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.596 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.597 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.597 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.597 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.597 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.597 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.597 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.597 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.597 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.597 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.597 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.597 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.597 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.598 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.598 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.598 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.598 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.598 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.598 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.598 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.598 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.598 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.598 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.598 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.598 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.599 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.599 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.599 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.599 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.599 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.599 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.599 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.599 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.599 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.599 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.599 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.599 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.600 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.600 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.600 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.600 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.600 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.600 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.600 INFO:teuthology.orchestra.run.smithi077.stdout: "zabbix_host": { 2023-12-01T01:22:33.600 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix_host", 2023-12-01T01:22:33.600 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.600 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.600 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.601 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.601 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.601 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.601 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.601 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.601 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.601 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.601 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.601 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.601 INFO:teuthology.orchestra.run.smithi077.stdout: "zabbix_port": { 2023-12-01T01:22:33.601 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix_port", 2023-12-01T01:22:33.601 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.602 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.602 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.602 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10051", 2023-12-01T01:22:33.602 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.602 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.602 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.602 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.602 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.602 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.602 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.602 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.603 INFO:teuthology.orchestra.run.smithi077.stdout: "zabbix_sender": { 2023-12-01T01:22:33.603 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix_sender", 2023-12-01T01:22:33.603 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.603 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.603 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.603 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-12-01T01:22:33.603 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.603 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.603 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.603 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.604 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.604 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.604 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.604 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.604 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.604 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.604 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2023-12-01T01:22:33.604 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.604 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.604 INFO:teuthology.orchestra.run.smithi077.stdout: "gid": 24106, 2023-12-01T01:22:33.604 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "z", 2023-12-01T01:22:33.604 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr_features": 4540138322906710015, 2023-12-01T01:22:33.605 INFO:teuthology.orchestra.run.smithi077.stdout: "available_modules": [ 2023-12-01T01:22:33.605 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.605 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "alerts", 2023-12-01T01:22:33.605 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.605 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.605 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.605 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2023-12-01T01:22:33.605 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2023-12-01T01:22:33.605 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:33.605 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.605 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.606 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2023-12-01T01:22:33.606 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.606 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.606 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.606 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How frequently to reexamine health status", 2023-12-01T01:22:33.606 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.606 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.606 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.606 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.606 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.606 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.606 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.606 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.607 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.607 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.607 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.607 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.607 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.607 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.607 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.607 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.607 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.607 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.607 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.607 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.608 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.608 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.608 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.608 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.608 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.608 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.608 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.608 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.608 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.608 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.608 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.608 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.609 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.609 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.609 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.609 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.609 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.609 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.609 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.609 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.609 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.609 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.609 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.609 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.609 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.610 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.610 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.610 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.610 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.610 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.610 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.610 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.610 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.610 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.610 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.610 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.610 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.610 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.611 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.611 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.611 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.611 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.611 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.611 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.611 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.611 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.611 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.611 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.611 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.612 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.612 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.612 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.612 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.612 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.612 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_destination": { 2023-12-01T01:22:33.612 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_destination", 2023-12-01T01:22:33.612 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.612 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.612 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.612 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.612 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.613 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.613 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.613 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Email address to send alerts to", 2023-12-01T01:22:33.613 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.613 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.613 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.613 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.613 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_from_name": { 2023-12-01T01:22:33.613 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_from_name", 2023-12-01T01:22:33.613 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.613 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.613 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.613 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "Ceph", 2023-12-01T01:22:33.614 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.614 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.614 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.614 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Email From: name", 2023-12-01T01:22:33.614 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.614 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.614 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.614 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.614 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_host": { 2023-12-01T01:22:33.614 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_host", 2023-12-01T01:22:33.614 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.614 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.615 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.615 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.615 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.615 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.615 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.615 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SMTP server", 2023-12-01T01:22:33.615 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.615 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.615 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.615 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.615 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_password": { 2023-12-01T01:22:33.615 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_password", 2023-12-01T01:22:33.616 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.616 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.616 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.616 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.616 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.616 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.616 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.616 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Password to authenticate with", 2023-12-01T01:22:33.616 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.616 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.616 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.616 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.616 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_port": { 2023-12-01T01:22:33.617 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_port", 2023-12-01T01:22:33.617 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.617 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.617 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.617 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "465", 2023-12-01T01:22:33.617 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.617 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.617 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.617 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SMTP port", 2023-12-01T01:22:33.617 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.617 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.617 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.617 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.618 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_sender": { 2023-12-01T01:22:33.618 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_sender", 2023-12-01T01:22:33.618 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.618 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.618 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.618 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.618 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.618 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.618 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.618 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SMTP envelope sender", 2023-12-01T01:22:33.618 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.619 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.619 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.619 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.619 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_ssl": { 2023-12-01T01:22:33.619 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_ssl", 2023-12-01T01:22:33.619 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.619 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.619 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.619 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.619 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.619 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.619 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.620 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use SSL to connect to SMTP server", 2023-12-01T01:22:33.620 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.620 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.620 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.620 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.620 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_user": { 2023-12-01T01:22:33.620 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_user", 2023-12-01T01:22:33.620 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.620 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.620 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.620 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.621 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.621 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.621 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.621 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "User to authenticate as", 2023-12-01T01:22:33.621 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.621 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.621 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.621 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.621 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.621 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.621 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.621 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "balancer", 2023-12-01T01:22:33.621 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.622 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.622 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.622 INFO:teuthology.orchestra.run.smithi077.stdout: "active": { 2023-12-01T01:22:33.622 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "active", 2023-12-01T01:22:33.622 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.622 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.622 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.622 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.622 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.622 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.622 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.622 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "automatically balance PGs across cluster", 2023-12-01T01:22:33.623 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.623 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.623 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.623 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.623 INFO:teuthology.orchestra.run.smithi077.stdout: "begin_time": { 2023-12-01T01:22:33.623 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "begin_time", 2023-12-01T01:22:33.623 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.623 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.623 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.623 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0000", 2023-12-01T01:22:33.623 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.624 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.624 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.624 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "beginning time of day to automatically balance", 2023-12-01T01:22:33.624 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-12-01T01:22:33.624 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.624 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.624 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.624 INFO:teuthology.orchestra.run.smithi077.stdout: "begin_weekday": { 2023-12-01T01:22:33.624 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "begin_weekday", 2023-12-01T01:22:33.624 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2023-12-01T01:22:33.624 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.624 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.625 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2023-12-01T01:22:33.625 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "0", 2023-12-01T01:22:33.625 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "6", 2023-12-01T01:22:33.625 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.625 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-12-01T01:22:33.625 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-12-01T01:22:33.625 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.625 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.625 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.625 INFO:teuthology.orchestra.run.smithi077.stdout: "crush_compat_max_iterations": { 2023-12-01T01:22:33.625 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crush_compat_max_iterations", 2023-12-01T01:22:33.625 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2023-12-01T01:22:33.625 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.626 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.626 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "25", 2023-12-01T01:22:33.626 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2023-12-01T01:22:33.626 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "250", 2023-12-01T01:22:33.626 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.626 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-12-01T01:22:33.626 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.626 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.626 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.626 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.626 INFO:teuthology.orchestra.run.smithi077.stdout: "crush_compat_metrics": { 2023-12-01T01:22:33.627 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crush_compat_metrics", 2023-12-01T01:22:33.627 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.627 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.627 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.627 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "pgs,objects,bytes", 2023-12-01T01:22:33.627 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.627 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.627 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.627 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-12-01T01:22:33.627 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-12-01T01:22:33.627 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.627 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.628 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.628 INFO:teuthology.orchestra.run.smithi077.stdout: "crush_compat_step": { 2023-12-01T01:22:33.628 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crush_compat_step", 2023-12-01T01:22:33.628 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2023-12-01T01:22:33.628 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.628 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.628 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0.5", 2023-12-01T01:22:33.628 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "0.001", 2023-12-01T01:22:33.628 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "0.999", 2023-12-01T01:22:33.628 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.628 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "aggressiveness of optimization", 2023-12-01T01:22:33.628 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-12-01T01:22:33.629 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.629 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.629 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.629 INFO:teuthology.orchestra.run.smithi077.stdout: "end_time": { 2023-12-01T01:22:33.629 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "end_time", 2023-12-01T01:22:33.629 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.629 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.629 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.629 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "2359", 2023-12-01T01:22:33.629 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.629 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.630 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.630 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "ending time of day to automatically balance", 2023-12-01T01:22:33.630 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-12-01T01:22:33.630 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.630 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.630 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.630 INFO:teuthology.orchestra.run.smithi077.stdout: "end_weekday": { 2023-12-01T01:22:33.630 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "end_weekday", 2023-12-01T01:22:33.630 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2023-12-01T01:22:33.630 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.630 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.630 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2023-12-01T01:22:33.631 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "0", 2023-12-01T01:22:33.631 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "6", 2023-12-01T01:22:33.631 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.631 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-12-01T01:22:33.631 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-12-01T01:22:33.631 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.631 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.631 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.631 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.631 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.631 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.632 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.632 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.632 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.632 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.632 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.632 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.632 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.632 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.632 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.632 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.632 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.633 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.633 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.633 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.633 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.633 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.633 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.633 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.633 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.633 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.633 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.633 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.633 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.633 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.634 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.634 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.634 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.634 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.634 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.634 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.634 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.634 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.634 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.634 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.634 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.634 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.635 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.635 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.635 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.635 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.635 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.635 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.635 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.635 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.635 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.635 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.635 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.635 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.635 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.636 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.636 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.636 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.636 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.636 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.636 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.636 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.636 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.636 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.636 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.636 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.636 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.636 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.637 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.637 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.637 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.637 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.637 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.637 INFO:teuthology.orchestra.run.smithi077.stdout: "min_score": { 2023-12-01T01:22:33.637 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "min_score", 2023-12-01T01:22:33.637 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2023-12-01T01:22:33.637 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.637 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.637 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2023-12-01T01:22:33.637 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.637 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.638 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.638 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-12-01T01:22:33.638 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.638 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.638 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.638 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.638 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": { 2023-12-01T01:22:33.638 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mode", 2023-12-01T01:22:33.638 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.638 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.638 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.638 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "upmap", 2023-12-01T01:22:33.639 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.639 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.639 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.639 INFO:teuthology.orchestra.run.smithi077.stdout: "crush-compat", 2023-12-01T01:22:33.639 INFO:teuthology.orchestra.run.smithi077.stdout: "none", 2023-12-01T01:22:33.639 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap" 2023-12-01T01:22:33.639 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.639 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Balancer mode", 2023-12-01T01:22:33.639 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.639 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.639 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.639 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.640 INFO:teuthology.orchestra.run.smithi077.stdout: "pool_ids": { 2023-12-01T01:22:33.640 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pool_ids", 2023-12-01T01:22:33.640 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.640 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.640 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.640 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.640 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.640 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.640 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.640 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-12-01T01:22:33.640 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.640 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.640 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.641 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.641 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2023-12-01T01:22:33.641 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2023-12-01T01:22:33.641 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:33.641 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.641 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.641 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2023-12-01T01:22:33.641 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.641 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.641 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.641 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-12-01T01:22:33.641 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.642 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.642 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.642 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.642 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap_max_deviation": { 2023-12-01T01:22:33.642 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "upmap_max_deviation", 2023-12-01T01:22:33.642 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.642 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.642 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.642 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2023-12-01T01:22:33.642 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2023-12-01T01:22:33.642 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.642 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.643 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "deviation below which no optimization is attempted", 2023-12-01T01:22:33.643 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-12-01T01:22:33.643 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.643 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.643 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.643 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap_max_optimizations": { 2023-12-01T01:22:33.643 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "upmap_max_optimizations", 2023-12-01T01:22:33.643 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2023-12-01T01:22:33.643 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.643 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.643 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2023-12-01T01:22:33.643 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.643 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.644 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.644 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-12-01T01:22:33.644 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.644 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.644 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.644 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.644 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.644 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.644 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.644 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cephadm", 2023-12-01T01:22:33.644 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.644 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.645 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.645 INFO:teuthology.orchestra.run.smithi077.stdout: "agent_down_multiplier": { 2023-12-01T01:22:33.645 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "agent_down_multiplier", 2023-12-01T01:22:33.645 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2023-12-01T01:22:33.645 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.645 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.645 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "3.0", 2023-12-01T01:22:33.645 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.645 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.645 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.645 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-12-01T01:22:33.645 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.645 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.646 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.646 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.646 INFO:teuthology.orchestra.run.smithi077.stdout: "agent_refresh_rate": { 2023-12-01T01:22:33.646 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "agent_refresh_rate", 2023-12-01T01:22:33.646 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:33.646 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.646 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.646 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "20", 2023-12-01T01:22:33.646 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.646 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.646 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.647 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-12-01T01:22:33.647 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.647 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.647 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.647 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.647 INFO:teuthology.orchestra.run.smithi077.stdout: "agent_starting_port": { 2023-12-01T01:22:33.647 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "agent_starting_port", 2023-12-01T01:22:33.647 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.647 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.647 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.647 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "4721", 2023-12-01T01:22:33.647 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.648 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.648 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.648 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-12-01T01:22:33.648 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.648 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.648 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.648 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.648 INFO:teuthology.orchestra.run.smithi077.stdout: "allow_ptrace": { 2023-12-01T01:22:33.648 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "allow_ptrace", 2023-12-01T01:22:33.648 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.648 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.648 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.649 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.649 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.649 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.649 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.649 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-12-01T01:22:33.649 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-12-01T01:22:33.649 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.649 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.649 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.649 INFO:teuthology.orchestra.run.smithi077.stdout: "autotune_interval": { 2023-12-01T01:22:33.649 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "autotune_interval", 2023-12-01T01:22:33.649 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:33.650 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.650 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.650 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2023-12-01T01:22:33.650 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.650 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.650 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.650 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to autotune daemon memory", 2023-12-01T01:22:33.650 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.650 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.650 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.650 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.650 INFO:teuthology.orchestra.run.smithi077.stdout: "autotune_memory_target_ratio": { 2023-12-01T01:22:33.651 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "autotune_memory_target_ratio", 2023-12-01T01:22:33.651 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2023-12-01T01:22:33.651 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.651 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.651 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0.7", 2023-12-01T01:22:33.651 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.651 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.651 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.651 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-12-01T01:22:33.651 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.651 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.651 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.651 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.652 INFO:teuthology.orchestra.run.smithi077.stdout: "cgroups_split": { 2023-12-01T01:22:33.652 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cgroups_split", 2023-12-01T01:22:33.652 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.652 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.652 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.652 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.652 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.652 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.652 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.652 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-12-01T01:22:33.652 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.652 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.653 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.653 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.653 INFO:teuthology.orchestra.run.smithi077.stdout: "config_checks_enabled": { 2023-12-01T01:22:33.653 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "config_checks_enabled", 2023-12-01T01:22:33.653 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.653 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.653 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.653 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.653 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.653 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.653 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.653 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-12-01T01:22:33.653 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.654 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.654 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.654 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.654 INFO:teuthology.orchestra.run.smithi077.stdout: "config_dashboard": { 2023-12-01T01:22:33.654 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "config_dashboard", 2023-12-01T01:22:33.654 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.654 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.654 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.654 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.654 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.654 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.654 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.655 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-12-01T01:22:33.655 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.655 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.655 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.655 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.655 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_alertmanager": { 2023-12-01T01:22:33.655 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_alertmanager", 2023-12-01T01:22:33.655 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.655 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.655 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.655 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-12-01T01:22:33.656 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.656 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.656 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.656 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2023-12-01T01:22:33.656 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.656 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.656 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.656 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.656 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_base": { 2023-12-01T01:22:33.656 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_base", 2023-12-01T01:22:33.656 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.656 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.657 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.657 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/ceph", 2023-12-01T01:22:33.657 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.657 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.657 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.657 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Container image name, without the tag", 2023-12-01T01:22:33.657 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.657 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.657 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.657 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.657 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_elasticsearch": { 2023-12-01T01:22:33.657 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_elasticsearch", 2023-12-01T01:22:33.657 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.658 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.658 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.658 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-12-01T01:22:33.658 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.658 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.658 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.658 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "elasticsearch container image", 2023-12-01T01:22:33.658 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.658 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.658 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.658 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.658 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_grafana": { 2023-12-01T01:22:33.659 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_grafana", 2023-12-01T01:22:33.659 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.659 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.659 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.659 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-12-01T01:22:33.659 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.659 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.659 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.659 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2023-12-01T01:22:33.659 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.659 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.659 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.659 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.660 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_haproxy": { 2023-12-01T01:22:33.660 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_haproxy", 2023-12-01T01:22:33.660 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.660 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.660 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.660 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-12-01T01:22:33.660 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.660 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.660 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.660 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "HAproxy container image", 2023-12-01T01:22:33.660 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.660 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.661 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.661 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.661 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_jaeger_agent": { 2023-12-01T01:22:33.661 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_jaeger_agent", 2023-12-01T01:22:33.661 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.661 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.661 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.661 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-12-01T01:22:33.661 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.661 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.661 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.661 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Jaeger agent container image", 2023-12-01T01:22:33.661 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.662 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.662 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.662 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.662 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_jaeger_collector": { 2023-12-01T01:22:33.662 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_jaeger_collector", 2023-12-01T01:22:33.662 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.662 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.662 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.662 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-12-01T01:22:33.662 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.662 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.662 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.662 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Jaeger collector container image", 2023-12-01T01:22:33.663 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.663 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.663 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.663 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.663 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_jaeger_query": { 2023-12-01T01:22:33.663 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_jaeger_query", 2023-12-01T01:22:33.663 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.663 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.663 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.663 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-12-01T01:22:33.663 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.663 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.664 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.664 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Jaeger query container image", 2023-12-01T01:22:33.664 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.664 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.664 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.664 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.664 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_keepalived": { 2023-12-01T01:22:33.664 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_keepalived", 2023-12-01T01:22:33.664 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.664 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.664 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.664 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-12-01T01:22:33.665 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.665 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.665 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.665 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Keepalived container image", 2023-12-01T01:22:33.665 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.665 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.665 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.665 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.665 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_loki": { 2023-12-01T01:22:33.665 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_loki", 2023-12-01T01:22:33.665 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.665 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.665 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.666 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-12-01T01:22:33.666 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.666 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.666 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.666 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Loki container image", 2023-12-01T01:22:33.666 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.666 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.666 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.666 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.666 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_node_exporter": { 2023-12-01T01:22:33.666 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_node_exporter", 2023-12-01T01:22:33.666 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.667 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.667 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.667 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-12-01T01:22:33.667 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.667 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.667 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.667 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2023-12-01T01:22:33.667 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.667 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.667 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.667 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.668 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_nvmeof": { 2023-12-01T01:22:33.668 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_nvmeof", 2023-12-01T01:22:33.668 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.668 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.668 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.668 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.2", 2023-12-01T01:22:33.668 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.668 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.668 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.668 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Nvme-of container image", 2023-12-01T01:22:33.668 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.668 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.669 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.669 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.669 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_prometheus": { 2023-12-01T01:22:33.669 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_prometheus", 2023-12-01T01:22:33.669 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.669 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.669 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.669 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-12-01T01:22:33.669 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.669 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.669 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.670 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2023-12-01T01:22:33.670 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.670 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.670 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.670 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.670 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_promtail": { 2023-12-01T01:22:33.670 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_promtail", 2023-12-01T01:22:33.670 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.670 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.670 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.670 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-12-01T01:22:33.670 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.671 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.671 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.671 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Promtail container image", 2023-12-01T01:22:33.671 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.671 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.671 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.671 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.671 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_snmp_gateway": { 2023-12-01T01:22:33.671 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_snmp_gateway", 2023-12-01T01:22:33.671 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.671 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.671 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.671 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-12-01T01:22:33.672 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.672 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.672 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.672 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SNMP Gateway container image", 2023-12-01T01:22:33.672 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.672 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.672 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.672 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.672 INFO:teuthology.orchestra.run.smithi077.stdout: "container_init": { 2023-12-01T01:22:33.672 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_init", 2023-12-01T01:22:33.672 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.672 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.673 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.673 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.673 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.673 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.673 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.673 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Run podman/docker with `--init`", 2023-12-01T01:22:33.673 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.673 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.673 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.673 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.673 INFO:teuthology.orchestra.run.smithi077.stdout: "daemon_cache_timeout": { 2023-12-01T01:22:33.673 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "daemon_cache_timeout", 2023-12-01T01:22:33.674 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:33.674 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.674 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.674 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2023-12-01T01:22:33.674 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.674 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.674 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.674 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-12-01T01:22:33.674 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.674 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.674 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.674 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.674 INFO:teuthology.orchestra.run.smithi077.stdout: "default_cephadm_command_timeout": { 2023-12-01T01:22:33.675 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "default_cephadm_command_timeout", 2023-12-01T01:22:33.675 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:33.675 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.675 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.675 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "900", 2023-12-01T01:22:33.675 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.675 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.675 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.675 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-12-01T01:22:33.675 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.675 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.675 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.676 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.676 INFO:teuthology.orchestra.run.smithi077.stdout: "default_registry": { 2023-12-01T01:22:33.676 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "default_registry", 2023-12-01T01:22:33.676 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.676 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.676 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.676 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io", 2023-12-01T01:22:33.676 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.676 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.676 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.676 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-12-01T01:22:33.676 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.676 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.677 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.677 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.677 INFO:teuthology.orchestra.run.smithi077.stdout: "device_cache_timeout": { 2023-12-01T01:22:33.677 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "device_cache_timeout", 2023-12-01T01:22:33.677 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:33.677 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.677 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.677 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "1800", 2023-12-01T01:22:33.677 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.677 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.677 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.677 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "seconds to cache device inventory", 2023-12-01T01:22:33.677 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.678 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.678 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.678 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.678 INFO:teuthology.orchestra.run.smithi077.stdout: "device_enhanced_scan": { 2023-12-01T01:22:33.678 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "device_enhanced_scan", 2023-12-01T01:22:33.678 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.678 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.678 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.678 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.678 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.678 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.678 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.678 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use libstoragemgmt during device scans", 2023-12-01T01:22:33.679 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.679 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.679 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.679 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.679 INFO:teuthology.orchestra.run.smithi077.stdout: "facts_cache_timeout": { 2023-12-01T01:22:33.679 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "facts_cache_timeout", 2023-12-01T01:22:33.679 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:33.679 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.679 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.679 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2023-12-01T01:22:33.679 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.680 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.680 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.680 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "seconds to cache host facts data", 2023-12-01T01:22:33.680 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.680 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.680 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.680 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.680 INFO:teuthology.orchestra.run.smithi077.stdout: "host_check_interval": { 2023-12-01T01:22:33.680 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "host_check_interval", 2023-12-01T01:22:33.680 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:33.680 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.680 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.680 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2023-12-01T01:22:33.681 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.681 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.681 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.681 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to perform a host check", 2023-12-01T01:22:33.681 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.681 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.681 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.681 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.681 INFO:teuthology.orchestra.run.smithi077.stdout: "inventory_list_all": { 2023-12-01T01:22:33.681 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "inventory_list_all", 2023-12-01T01:22:33.681 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.681 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.681 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.681 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.682 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.682 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.682 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.682 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-12-01T01:22:33.682 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.682 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.682 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.682 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.682 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.682 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.682 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.683 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.683 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.683 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.683 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.683 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.683 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.683 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.683 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.683 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.683 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.683 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.684 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.684 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.684 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.684 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.684 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.684 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.684 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.684 INFO:teuthology.orchestra.run.smithi077.stdout: "log_refresh_metadata": { 2023-12-01T01:22:33.684 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_refresh_metadata", 2023-12-01T01:22:33.684 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.684 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.684 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.684 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.685 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.685 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.685 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.685 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-12-01T01:22:33.685 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.685 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.685 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.685 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.685 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.685 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.685 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.685 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.685 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.686 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.686 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.686 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.686 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.686 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-12-01T01:22:33.686 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.686 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.686 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.686 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.686 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.686 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.686 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.687 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.687 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.687 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.687 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.687 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.687 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.687 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.687 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.687 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.687 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.687 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.687 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.687 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.688 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.688 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.688 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.688 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.688 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.688 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.688 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.688 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.688 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.688 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.688 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.688 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.688 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.689 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.689 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.689 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.689 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.689 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.689 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.689 INFO:teuthology.orchestra.run.smithi077.stdout: "manage_etc_ceph_ceph_conf": { 2023-12-01T01:22:33.689 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-12-01T01:22:33.689 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.689 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.689 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.689 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.690 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.690 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.690 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.690 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-12-01T01:22:33.690 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.690 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.690 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.690 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.690 INFO:teuthology.orchestra.run.smithi077.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-12-01T01:22:33.690 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-12-01T01:22:33.690 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.690 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.690 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.691 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "*", 2023-12-01T01:22:33.691 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.691 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.691 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.691 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-12-01T01:22:33.691 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.691 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.691 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.691 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.691 INFO:teuthology.orchestra.run.smithi077.stdout: "max_count_per_host": { 2023-12-01T01:22:33.691 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_count_per_host", 2023-12-01T01:22:33.691 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.691 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.692 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.692 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2023-12-01T01:22:33.692 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.692 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.692 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.692 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "max number of daemons per service per host", 2023-12-01T01:22:33.692 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.692 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.692 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.692 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.692 INFO:teuthology.orchestra.run.smithi077.stdout: "max_osd_draining_count": { 2023-12-01T01:22:33.692 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_osd_draining_count", 2023-12-01T01:22:33.693 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.693 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.693 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.693 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2023-12-01T01:22:33.693 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.693 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.693 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.693 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-12-01T01:22:33.693 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.693 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.693 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.693 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.693 INFO:teuthology.orchestra.run.smithi077.stdout: "migration_current": { 2023-12-01T01:22:33.694 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "migration_current", 2023-12-01T01:22:33.694 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.694 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.694 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.694 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.694 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.694 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.694 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.694 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "internal - do not modify", 2023-12-01T01:22:33.694 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.694 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.694 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.695 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.695 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": { 2023-12-01T01:22:33.695 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mode", 2023-12-01T01:22:33.695 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.695 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.695 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.695 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "root", 2023-12-01T01:22:33.695 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.695 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.695 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.695 INFO:teuthology.orchestra.run.smithi077.stdout: "cephadm-package", 2023-12-01T01:22:33.695 INFO:teuthology.orchestra.run.smithi077.stdout: "root" 2023-12-01T01:22:33.695 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.696 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "mode for remote execution of cephadm", 2023-12-01T01:22:33.696 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.696 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.696 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.696 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.696 INFO:teuthology.orchestra.run.smithi077.stdout: "prometheus_alerts_path": { 2023-12-01T01:22:33.696 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "prometheus_alerts_path", 2023-12-01T01:22:33.697 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.697 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.697 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.697 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-12-01T01:22:33.697 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.697 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.697 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.697 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-12-01T01:22:33.697 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.697 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.697 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.698 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.698 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_insecure": { 2023-12-01T01:22:33.698 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_insecure", 2023-12-01T01:22:33.698 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.698 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.698 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.698 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.698 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.698 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.698 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.698 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-12-01T01:22:33.698 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.699 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.699 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.699 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.699 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_password": { 2023-12-01T01:22:33.699 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_password", 2023-12-01T01:22:33.699 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.699 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.699 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.699 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.699 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.699 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.699 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.699 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-12-01T01:22:33.700 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.700 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.700 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.700 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.700 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_url": { 2023-12-01T01:22:33.700 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_url", 2023-12-01T01:22:33.700 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.700 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.700 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.700 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.700 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.700 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.700 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.701 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-12-01T01:22:33.701 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.701 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.701 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.701 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.701 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_username": { 2023-12-01T01:22:33.701 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_username", 2023-12-01T01:22:33.701 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.701 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.701 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.701 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.701 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.701 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.702 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.702 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-12-01T01:22:33.702 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.702 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.702 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.702 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.702 INFO:teuthology.orchestra.run.smithi077.stdout: "secure_monitoring_stack": { 2023-12-01T01:22:33.702 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "secure_monitoring_stack", 2023-12-01T01:22:33.702 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.702 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.702 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.702 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.703 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.703 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.703 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.703 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-12-01T01:22:33.703 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.703 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.703 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.703 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.703 INFO:teuthology.orchestra.run.smithi077.stdout: "service_discovery_port": { 2023-12-01T01:22:33.703 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "service_discovery_port", 2023-12-01T01:22:33.703 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.703 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.703 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.704 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8765", 2023-12-01T01:22:33.704 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.704 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.704 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.704 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "cephadm service discovery port", 2023-12-01T01:22:33.704 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.704 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.704 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.704 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.704 INFO:teuthology.orchestra.run.smithi077.stdout: "ssh_config_file": { 2023-12-01T01:22:33.704 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssh_config_file", 2023-12-01T01:22:33.704 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.704 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.705 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.705 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.705 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.705 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.705 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.705 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-12-01T01:22:33.705 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.705 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.705 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.705 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.705 INFO:teuthology.orchestra.run.smithi077.stdout: "use_agent": { 2023-12-01T01:22:33.705 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "use_agent", 2023-12-01T01:22:33.705 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.706 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.706 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.706 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.706 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.706 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.706 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.706 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-12-01T01:22:33.706 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.706 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.706 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.706 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.706 INFO:teuthology.orchestra.run.smithi077.stdout: "use_repo_digest": { 2023-12-01T01:22:33.707 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "use_repo_digest", 2023-12-01T01:22:33.707 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.707 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.707 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.707 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.707 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.707 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.707 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.707 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-12-01T01:22:33.707 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.707 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.707 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.707 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.708 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_on_failed_host_check": { 2023-12-01T01:22:33.708 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_on_failed_host_check", 2023-12-01T01:22:33.708 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.708 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.708 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.708 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.708 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.708 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.708 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.708 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise a health warning if the host check fails", 2023-12-01T01:22:33.708 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.708 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.709 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.709 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.709 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_on_stray_daemons": { 2023-12-01T01:22:33.709 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_on_stray_daemons", 2023-12-01T01:22:33.709 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.709 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.709 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.709 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.709 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.709 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.709 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.709 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-12-01T01:22:33.709 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.710 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.710 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.710 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.710 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_on_stray_hosts": { 2023-12-01T01:22:33.710 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_on_stray_hosts", 2023-12-01T01:22:33.710 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.710 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.710 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.710 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.710 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.710 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.710 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.711 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-12-01T01:22:33.711 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.711 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.711 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.711 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.711 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.711 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.711 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.711 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crash", 2023-12-01T01:22:33.711 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.711 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.711 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.712 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.712 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.712 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.712 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.712 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.712 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.712 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.712 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.712 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.712 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.712 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.712 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.712 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.713 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.713 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.713 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.713 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.713 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.713 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.713 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.713 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.713 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.713 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.713 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.713 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.713 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.714 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.714 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.714 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.714 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.714 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.714 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.714 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.714 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.714 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.714 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.714 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.715 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.715 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.715 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.715 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.715 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.715 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.715 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.715 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.715 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.715 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.715 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.715 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.716 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.716 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.716 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.716 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.716 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.716 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.716 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.716 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.716 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.716 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.716 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.716 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.716 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.717 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.717 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.717 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.717 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.717 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.717 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.717 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.717 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.717 INFO:teuthology.orchestra.run.smithi077.stdout: "retain_interval": { 2023-12-01T01:22:33.717 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "retain_interval", 2023-12-01T01:22:33.717 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:33.717 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.718 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.718 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "31536000", 2023-12-01T01:22:33.718 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.718 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.718 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.718 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how long to retain crashes before pruning them", 2023-12-01T01:22:33.718 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.718 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.718 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.718 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.718 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_recent_interval": { 2023-12-01T01:22:33.718 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_recent_interval", 2023-12-01T01:22:33.719 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:33.719 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.719 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.719 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "1209600", 2023-12-01T01:22:33.719 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.719 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.719 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.719 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "time interval in which to warn about recent crashes", 2023-12-01T01:22:33.719 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.719 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.719 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.719 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.720 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.720 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.720 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.720 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "dashboard", 2023-12-01T01:22:33.720 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.720 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.720 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.720 INFO:teuthology.orchestra.run.smithi077.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-12-01T01:22:33.720 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-12-01T01:22:33.720 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.720 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.720 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.720 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2023-12-01T01:22:33.721 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.721 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.721 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.721 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.721 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.721 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.721 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.721 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.721 INFO:teuthology.orchestra.run.smithi077.stdout: "ALERTMANAGER_API_HOST": { 2023-12-01T01:22:33.721 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ALERTMANAGER_API_HOST", 2023-12-01T01:22:33.721 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.721 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.722 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.722 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.722 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.722 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.722 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.722 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.722 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.722 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.722 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.722 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.722 INFO:teuthology.orchestra.run.smithi077.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-12-01T01:22:33.722 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-12-01T01:22:33.723 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.723 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.723 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.723 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.723 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.723 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.723 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.723 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.723 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.723 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.723 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.723 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.723 INFO:teuthology.orchestra.run.smithi077.stdout: "AUDIT_API_ENABLED": { 2023-12-01T01:22:33.724 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "AUDIT_API_ENABLED", 2023-12-01T01:22:33.724 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.724 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.724 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.724 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.724 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.724 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.724 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.724 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.724 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.724 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.724 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.725 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.725 INFO:teuthology.orchestra.run.smithi077.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-12-01T01:22:33.725 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-12-01T01:22:33.725 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.725 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.725 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.725 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.725 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.725 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.725 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.725 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.726 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.726 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.726 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.726 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.726 INFO:teuthology.orchestra.run.smithi077.stdout: "ENABLE_BROWSABLE_API": { 2023-12-01T01:22:33.726 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ENABLE_BROWSABLE_API", 2023-12-01T01:22:33.726 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.726 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.726 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.726 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.726 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.726 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.727 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.727 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.727 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.727 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.727 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.727 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.727 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-12-01T01:22:33.727 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-12-01T01:22:33.727 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.727 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.727 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.727 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.727 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.728 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.728 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.728 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.728 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.728 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.728 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.728 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.728 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-12-01T01:22:33.728 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-12-01T01:22:33.728 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.728 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.728 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.729 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.729 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.729 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.729 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.729 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.729 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.729 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.729 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.729 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.729 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-12-01T01:22:33.729 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-12-01T01:22:33.729 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.729 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.730 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.730 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.730 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.730 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.730 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.730 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.730 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.730 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.730 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.730 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.730 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-12-01T01:22:33.730 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-12-01T01:22:33.731 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.731 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.731 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.731 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.731 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.731 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.731 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.731 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.731 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.731 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.731 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.731 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.731 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_NFS": { 2023-12-01T01:22:33.732 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-12-01T01:22:33.732 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.732 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.732 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.732 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.732 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.732 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.732 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.732 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.732 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.732 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.732 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.732 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.733 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_RBD": { 2023-12-01T01:22:33.733 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-12-01T01:22:33.733 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.733 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.733 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.733 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.733 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.733 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.733 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.733 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.733 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.733 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.733 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.734 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.734 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_RGW": { 2023-12-01T01:22:33.734 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-12-01T01:22:33.734 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.734 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.734 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.734 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.734 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.734 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.734 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.734 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.734 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.735 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.735 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.735 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.735 INFO:teuthology.orchestra.run.smithi077.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-12-01T01:22:33.735 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-12-01T01:22:33.735 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.735 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.735 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.735 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.735 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.735 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.735 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.735 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.736 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.736 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.736 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.736 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.736 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_PASSWORD": { 2023-12-01T01:22:33.736 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_PASSWORD", 2023-12-01T01:22:33.736 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.736 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.736 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.736 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "admin", 2023-12-01T01:22:33.736 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.736 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.736 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.737 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.737 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.737 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.737 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.737 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.737 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-12-01T01:22:33.737 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-12-01T01:22:33.737 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.737 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.737 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.737 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.737 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.738 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.738 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.738 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.738 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.738 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.738 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.738 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.738 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_URL": { 2023-12-01T01:22:33.738 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_URL", 2023-12-01T01:22:33.738 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.738 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.738 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.739 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.739 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.739 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.739 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.739 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.739 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.739 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.739 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.739 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.739 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_USERNAME": { 2023-12-01T01:22:33.739 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_USERNAME", 2023-12-01T01:22:33.739 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.739 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.740 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.740 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "admin", 2023-12-01T01:22:33.740 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.740 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.740 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.740 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.740 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.740 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.740 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.740 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.740 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-12-01T01:22:33.740 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-12-01T01:22:33.741 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.741 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.741 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.741 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.741 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.741 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.741 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.741 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.741 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.741 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.741 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.741 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.742 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-12-01T01:22:33.742 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-12-01T01:22:33.742 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.742 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.742 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.742 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.742 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.742 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.742 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.742 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.742 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.743 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.743 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.743 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.743 INFO:teuthology.orchestra.run.smithi077.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-12-01T01:22:33.743 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-12-01T01:22:33.743 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.743 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.743 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.743 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.743 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.743 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.743 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.743 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.744 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.744 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.744 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.744 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.744 INFO:teuthology.orchestra.run.smithi077.stdout: "ISSUE_TRACKER_API_KEY": { 2023-12-01T01:22:33.744 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-12-01T01:22:33.744 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.744 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.744 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.744 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.744 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.744 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.745 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.745 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.745 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.745 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.745 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.745 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.745 INFO:teuthology.orchestra.run.smithi077.stdout: "PROMETHEUS_API_HOST": { 2023-12-01T01:22:33.745 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PROMETHEUS_API_HOST", 2023-12-01T01:22:33.745 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.745 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.745 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.745 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.745 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.745 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.746 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.746 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.746 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.746 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.746 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.746 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.746 INFO:teuthology.orchestra.run.smithi077.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-12-01T01:22:33.746 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-12-01T01:22:33.746 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.746 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.746 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.746 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.747 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.747 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.747 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.747 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.747 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.747 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.747 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.747 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.747 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-12-01T01:22:33.747 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-12-01T01:22:33.747 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.747 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.748 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.748 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.748 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.748 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.748 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.748 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.748 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.748 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.748 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.748 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.748 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-12-01T01:22:33.748 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-12-01T01:22:33.749 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.749 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.749 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.749 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.749 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.749 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.749 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.749 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.749 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.749 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.749 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.749 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.750 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-12-01T01:22:33.750 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-12-01T01:22:33.750 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.750 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.750 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.750 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.750 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.750 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.750 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.750 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.750 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.751 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.751 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.751 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.751 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-12-01T01:22:33.751 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-12-01T01:22:33.751 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.751 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.751 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.751 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.751 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.751 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.751 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.752 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.752 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.752 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.752 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.752 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.752 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-12-01T01:22:33.752 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-12-01T01:22:33.752 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.752 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.752 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.752 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.752 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.753 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.753 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.753 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.753 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.753 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.753 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.753 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.753 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-12-01T01:22:33.753 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-12-01T01:22:33.753 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.753 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.753 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.753 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.754 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.754 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.754 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.754 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.754 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.754 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.754 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.754 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.754 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-12-01T01:22:33.754 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-12-01T01:22:33.754 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.755 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.755 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.755 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.755 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.755 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.755 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.755 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.755 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.755 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.755 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.755 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.755 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_ENABLED": { 2023-12-01T01:22:33.755 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_ENABLED", 2023-12-01T01:22:33.756 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.756 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.756 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.756 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.756 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.756 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.756 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.756 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.756 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.756 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.756 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.756 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.757 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-12-01T01:22:33.757 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-12-01T01:22:33.757 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.757 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.757 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.757 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-12-01T01:22:33.757 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.757 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.757 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.757 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.757 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.757 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.758 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.758 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.758 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-12-01T01:22:33.758 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-12-01T01:22:33.758 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.758 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.758 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.758 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2023-12-01T01:22:33.758 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.758 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.758 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.758 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.759 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.759 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.759 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.759 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.759 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-12-01T01:22:33.759 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-12-01T01:22:33.759 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.759 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.759 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.759 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8", 2023-12-01T01:22:33.759 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.759 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.760 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.760 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.760 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.760 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.760 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.760 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.760 INFO:teuthology.orchestra.run.smithi077.stdout: "REST_REQUESTS_TIMEOUT": { 2023-12-01T01:22:33.760 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-12-01T01:22:33.760 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.760 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.760 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.760 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "45", 2023-12-01T01:22:33.760 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.761 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.761 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.761 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.761 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.761 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.761 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.761 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.761 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_ACCESS_KEY": { 2023-12-01T01:22:33.761 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_ACCESS_KEY", 2023-12-01T01:22:33.761 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.761 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.761 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.762 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.762 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.762 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.762 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.762 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.762 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.762 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.762 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.762 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.762 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-12-01T01:22:33.762 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-12-01T01:22:33.762 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.763 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.763 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.763 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "admin", 2023-12-01T01:22:33.763 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.763 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.763 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.763 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.763 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.763 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.763 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.763 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.763 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_SECRET_KEY": { 2023-12-01T01:22:33.763 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_SECRET_KEY", 2023-12-01T01:22:33.764 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.764 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.764 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.764 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.764 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.764 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.764 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.764 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.764 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.764 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.764 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.765 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.765 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_SSL_VERIFY": { 2023-12-01T01:22:33.765 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_SSL_VERIFY", 2023-12-01T01:22:33.765 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.765 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.765 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.765 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.765 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.765 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.765 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.765 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.765 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.765 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.766 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.766 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.766 INFO:teuthology.orchestra.run.smithi077.stdout: "UNSAFE_TLS_v1_2": { 2023-12-01T01:22:33.766 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "UNSAFE_TLS_v1_2", 2023-12-01T01:22:33.766 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.766 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.766 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.766 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.766 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.766 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.766 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.766 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.767 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.767 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.767 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.767 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.767 INFO:teuthology.orchestra.run.smithi077.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-12-01T01:22:33.767 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-12-01T01:22:33.767 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.767 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.767 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.767 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2023-12-01T01:22:33.767 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.767 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.767 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.768 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.768 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.768 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.768 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.768 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.768 INFO:teuthology.orchestra.run.smithi077.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-12-01T01:22:33.768 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-12-01T01:22:33.768 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.768 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.768 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.768 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2023-12-01T01:22:33.768 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.769 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.769 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.769 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.769 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.769 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.769 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.769 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.769 INFO:teuthology.orchestra.run.smithi077.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-12-01T01:22:33.769 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-12-01T01:22:33.769 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.769 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.769 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.770 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2023-12-01T01:22:33.770 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.770 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.770 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.770 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.770 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.770 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.770 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.770 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.770 INFO:teuthology.orchestra.run.smithi077.stdout: "cross_origin_url": { 2023-12-01T01:22:33.770 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cross_origin_url", 2023-12-01T01:22:33.771 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.771 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.771 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.771 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.771 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.771 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.771 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.771 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.771 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.771 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.772 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.772 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.772 INFO:teuthology.orchestra.run.smithi077.stdout: "crt_file": { 2023-12-01T01:22:33.772 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crt_file", 2023-12-01T01:22:33.772 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.772 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.772 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.772 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.772 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.772 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.772 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.773 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.773 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.773 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.773 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.773 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.773 INFO:teuthology.orchestra.run.smithi077.stdout: "debug": { 2023-12-01T01:22:33.773 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "debug", 2023-12-01T01:22:33.773 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.773 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.773 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.773 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.773 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.774 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.774 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.774 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Enable/disable debug options", 2023-12-01T01:22:33.774 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.774 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.774 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.774 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.774 INFO:teuthology.orchestra.run.smithi077.stdout: "jwt_token_ttl": { 2023-12-01T01:22:33.774 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "jwt_token_ttl", 2023-12-01T01:22:33.774 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.774 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.775 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.775 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "28800", 2023-12-01T01:22:33.775 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.775 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.775 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.775 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.775 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.775 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.775 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.776 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.776 INFO:teuthology.orchestra.run.smithi077.stdout: "key_file": { 2023-12-01T01:22:33.776 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "key_file", 2023-12-01T01:22:33.776 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.776 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.776 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.776 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.776 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.776 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.776 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.776 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.776 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.777 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.777 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.777 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.777 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.777 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.777 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.777 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.777 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.777 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.777 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.777 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.777 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.777 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.778 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.778 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.778 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.778 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.778 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.778 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.778 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.778 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.778 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.778 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.778 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.778 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.779 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.779 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.779 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.779 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.779 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.779 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.779 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.779 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.779 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.779 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.779 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.779 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.780 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.780 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.780 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.780 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.780 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.780 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.780 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.780 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.780 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.780 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.780 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.780 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.781 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.781 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.781 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.781 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.781 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.781 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.781 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.781 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.781 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.781 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.781 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.781 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.782 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.782 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.782 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.782 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.782 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.782 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.782 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.782 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.782 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.782 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.782 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.782 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.782 INFO:teuthology.orchestra.run.smithi077.stdout: "motd": { 2023-12-01T01:22:33.783 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "motd", 2023-12-01T01:22:33.783 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.783 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.783 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.783 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.783 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.783 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.783 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.783 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "The message of the day", 2023-12-01T01:22:33.783 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.783 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.783 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.783 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.784 INFO:teuthology.orchestra.run.smithi077.stdout: "redirect_resolve_ip_addr": { 2023-12-01T01:22:33.784 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "redirect_resolve_ip_addr", 2023-12-01T01:22:33.784 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.784 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.784 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.784 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.784 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.784 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.784 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.784 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.784 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.784 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.785 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.785 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.785 INFO:teuthology.orchestra.run.smithi077.stdout: "server_addr": { 2023-12-01T01:22:33.785 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_addr", 2023-12-01T01:22:33.785 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.785 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.785 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.785 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "::", 2023-12-01T01:22:33.785 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.785 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.785 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.785 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.785 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.786 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.786 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.786 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.786 INFO:teuthology.orchestra.run.smithi077.stdout: "server_port": { 2023-12-01T01:22:33.786 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_port", 2023-12-01T01:22:33.786 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.786 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.786 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.786 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8080", 2023-12-01T01:22:33.786 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.786 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.786 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.786 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.787 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.787 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.787 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.787 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.787 INFO:teuthology.orchestra.run.smithi077.stdout: "ssl": { 2023-12-01T01:22:33.787 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssl", 2023-12-01T01:22:33.787 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.787 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.787 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.787 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.787 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.788 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.788 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.788 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.788 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.788 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.788 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.788 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.788 INFO:teuthology.orchestra.run.smithi077.stdout: "ssl_server_port": { 2023-12-01T01:22:33.788 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssl_server_port", 2023-12-01T01:22:33.788 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.788 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.788 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.788 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8443", 2023-12-01T01:22:33.789 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.789 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.789 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.789 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.789 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.789 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.789 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.789 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.789 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_behaviour": { 2023-12-01T01:22:33.789 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_behaviour", 2023-12-01T01:22:33.789 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.789 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.789 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.790 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "redirect", 2023-12-01T01:22:33.790 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.790 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.790 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.790 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.790 INFO:teuthology.orchestra.run.smithi077.stdout: "redirect" 2023-12-01T01:22:33.790 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.790 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.790 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.790 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.790 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.790 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.790 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_error_status_code": { 2023-12-01T01:22:33.791 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_error_status_code", 2023-12-01T01:22:33.791 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.791 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.791 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.791 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "500", 2023-12-01T01:22:33.791 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "400", 2023-12-01T01:22:33.791 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "599", 2023-12-01T01:22:33.791 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.791 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.791 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.791 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.791 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.792 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.792 INFO:teuthology.orchestra.run.smithi077.stdout: "url_prefix": { 2023-12-01T01:22:33.792 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "url_prefix", 2023-12-01T01:22:33.792 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.792 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.792 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.792 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.792 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.792 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.792 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.792 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.792 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.793 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.793 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.793 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.793 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.793 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.793 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.793 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "devicehealth", 2023-12-01T01:22:33.793 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.793 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.793 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.793 INFO:teuthology.orchestra.run.smithi077.stdout: "enable_monitoring": { 2023-12-01T01:22:33.793 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enable_monitoring", 2023-12-01T01:22:33.794 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.794 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.794 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.794 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.794 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.794 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.794 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.794 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "monitor device health metrics", 2023-12-01T01:22:33.794 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.794 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.794 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.794 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.795 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.795 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.795 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.795 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.795 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.795 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.795 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.795 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.795 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.795 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.795 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.795 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.796 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.796 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.796 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.796 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.796 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.796 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.796 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.796 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.796 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.796 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.796 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.796 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.797 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.797 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.797 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.797 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.797 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.797 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.797 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.797 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.797 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.797 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.797 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.797 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.797 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.797 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.798 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.798 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.798 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.798 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.798 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.798 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.798 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.798 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.798 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.798 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.798 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.798 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.799 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.799 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.799 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.799 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.799 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.799 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.799 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.799 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.799 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.799 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.801 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.801 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.801 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.801 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.801 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.801 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.801 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.801 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.802 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.802 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.802 INFO:teuthology.orchestra.run.smithi077.stdout: "mark_out_threshold": { 2023-12-01T01:22:33.802 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mark_out_threshold", 2023-12-01T01:22:33.802 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:33.802 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.802 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.802 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "2419200", 2023-12-01T01:22:33.802 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.802 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.802 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.802 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-12-01T01:22:33.802 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.802 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.803 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.803 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.803 INFO:teuthology.orchestra.run.smithi077.stdout: "pool_name": { 2023-12-01T01:22:33.803 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pool_name", 2023-12-01T01:22:33.803 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.803 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.803 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.803 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "device_health_metrics", 2023-12-01T01:22:33.803 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.803 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.803 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.803 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "name of pool in which to store device health metrics", 2023-12-01T01:22:33.803 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.803 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.803 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.804 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.804 INFO:teuthology.orchestra.run.smithi077.stdout: "retention_period": { 2023-12-01T01:22:33.804 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "retention_period", 2023-12-01T01:22:33.804 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:33.804 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.804 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.804 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "15552000", 2023-12-01T01:22:33.804 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.804 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.804 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.804 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how long to retain device health metrics", 2023-12-01T01:22:33.804 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.805 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.805 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.805 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.805 INFO:teuthology.orchestra.run.smithi077.stdout: "scrape_frequency": { 2023-12-01T01:22:33.805 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "scrape_frequency", 2023-12-01T01:22:33.805 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:33.805 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.805 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.805 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "86400", 2023-12-01T01:22:33.805 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.805 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.805 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.805 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to scrape device health metrics", 2023-12-01T01:22:33.805 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.806 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.806 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.806 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.806 INFO:teuthology.orchestra.run.smithi077.stdout: "self_heal": { 2023-12-01T01:22:33.806 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "self_heal", 2023-12-01T01:22:33.806 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.806 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.806 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.806 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.806 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.806 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.806 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.806 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-12-01T01:22:33.806 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.807 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.807 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.807 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.807 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2023-12-01T01:22:33.807 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2023-12-01T01:22:33.807 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:33.807 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.807 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.807 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2023-12-01T01:22:33.807 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.807 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.807 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.807 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to wake up and check device health", 2023-12-01T01:22:33.807 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.807 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.808 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.808 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.808 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_threshold": { 2023-12-01T01:22:33.808 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_threshold", 2023-12-01T01:22:33.808 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:33.808 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.808 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.808 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "7257600", 2023-12-01T01:22:33.808 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.808 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.808 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.808 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-12-01T01:22:33.808 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.809 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.809 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.809 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.809 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.809 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.809 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.809 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "diskprediction_local", 2023-12-01T01:22:33.809 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.809 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.809 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.809 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.809 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.809 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.809 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.809 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.810 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.810 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.810 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.810 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.810 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.810 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.810 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.810 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.810 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.810 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.810 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.810 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.810 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.810 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.811 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.811 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.811 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.811 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.811 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.811 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.811 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.811 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.811 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.811 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.811 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.811 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.811 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.811 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.812 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.812 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.812 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.812 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.812 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.812 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.812 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.812 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.812 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.812 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.812 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.812 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.812 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.812 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.812 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.813 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.813 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.813 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.813 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.813 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.813 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.813 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.813 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.813 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.813 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.813 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.813 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.813 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.813 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.814 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.814 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.814 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.814 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.814 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.814 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.814 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.814 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.814 INFO:teuthology.orchestra.run.smithi077.stdout: "predict_interval": { 2023-12-01T01:22:33.814 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "predict_interval", 2023-12-01T01:22:33.814 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.814 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.814 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.814 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "86400", 2023-12-01T01:22:33.815 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.815 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.815 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.815 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.815 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.815 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.815 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.815 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.815 INFO:teuthology.orchestra.run.smithi077.stdout: "predictor_model": { 2023-12-01T01:22:33.815 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "predictor_model", 2023-12-01T01:22:33.815 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.815 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.815 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.815 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "prophetstor", 2023-12-01T01:22:33.815 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.816 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.816 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.816 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.816 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.816 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.816 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.816 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.816 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2023-12-01T01:22:33.816 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2023-12-01T01:22:33.816 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.816 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.816 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.816 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2023-12-01T01:22:33.816 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.817 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.817 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.817 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.817 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.817 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.817 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.817 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.817 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.817 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.817 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.817 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "influx", 2023-12-01T01:22:33.817 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": false, 2023-12-01T01:22:33.817 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "influxdb python module not found", 2023-12-01T01:22:33.818 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.818 INFO:teuthology.orchestra.run.smithi077.stdout: "batch_size": { 2023-12-01T01:22:33.818 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "batch_size", 2023-12-01T01:22:33.818 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.818 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.818 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.818 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5000", 2023-12-01T01:22:33.818 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.818 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.818 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.818 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-12-01T01:22:33.818 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.818 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.818 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.819 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.819 INFO:teuthology.orchestra.run.smithi077.stdout: "database": { 2023-12-01T01:22:33.819 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "database", 2023-12-01T01:22:33.819 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.819 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.819 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.819 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "ceph", 2023-12-01T01:22:33.819 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.819 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.819 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.819 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-12-01T01:22:33.819 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.819 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.819 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.820 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.820 INFO:teuthology.orchestra.run.smithi077.stdout: "hostname": { 2023-12-01T01:22:33.820 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "hostname", 2023-12-01T01:22:33.820 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.820 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.820 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.820 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.820 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.820 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.820 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.820 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "InfluxDB server hostname", 2023-12-01T01:22:33.820 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.820 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.820 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.820 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.821 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2023-12-01T01:22:33.821 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2023-12-01T01:22:33.821 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:33.821 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.821 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.821 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "30", 2023-12-01T01:22:33.821 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "5", 2023-12-01T01:22:33.821 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.821 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.821 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-12-01T01:22:33.821 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.821 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.821 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.821 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.821 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.822 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.822 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.822 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.822 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.822 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.822 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.822 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.822 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.822 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.822 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.822 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.822 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.822 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.823 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.823 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.823 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.823 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.823 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.823 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.823 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.823 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.823 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.823 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.823 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.823 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.823 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.823 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.824 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.824 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.824 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.824 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.824 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.824 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.824 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.824 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.824 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.824 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.824 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.824 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.824 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.824 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.824 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.825 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.825 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.825 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.825 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.825 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.825 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.825 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.825 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.825 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.825 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.825 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.825 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.825 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.825 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.826 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.826 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.826 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.826 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.826 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.826 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.826 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.826 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.826 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.826 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.826 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.826 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.826 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.826 INFO:teuthology.orchestra.run.smithi077.stdout: "password": { 2023-12-01T01:22:33.827 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "password", 2023-12-01T01:22:33.827 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.827 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.827 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.827 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.827 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.827 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.827 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.827 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "password of InfluxDB server user", 2023-12-01T01:22:33.827 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.827 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.827 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.827 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.827 INFO:teuthology.orchestra.run.smithi077.stdout: "port": { 2023-12-01T01:22:33.827 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "port", 2023-12-01T01:22:33.828 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.828 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.828 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.828 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8086", 2023-12-01T01:22:33.828 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.828 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.828 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.828 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "InfluxDB server port", 2023-12-01T01:22:33.828 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.828 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.828 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.828 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.828 INFO:teuthology.orchestra.run.smithi077.stdout: "ssl": { 2023-12-01T01:22:33.828 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssl", 2023-12-01T01:22:33.829 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.829 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.829 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.829 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "false", 2023-12-01T01:22:33.829 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.829 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.829 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.829 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-12-01T01:22:33.829 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.829 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.829 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.829 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.829 INFO:teuthology.orchestra.run.smithi077.stdout: "threads": { 2023-12-01T01:22:33.829 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "threads", 2023-12-01T01:22:33.830 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.830 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.830 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.830 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2023-12-01T01:22:33.830 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2023-12-01T01:22:33.830 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "32", 2023-12-01T01:22:33.830 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.830 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-12-01T01:22:33.830 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.830 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.830 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.830 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.830 INFO:teuthology.orchestra.run.smithi077.stdout: "username": { 2023-12-01T01:22:33.831 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "username", 2023-12-01T01:22:33.831 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.831 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.831 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.831 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.831 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.831 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.831 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.831 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "username of InfluxDB server user", 2023-12-01T01:22:33.831 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.831 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.831 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.831 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.832 INFO:teuthology.orchestra.run.smithi077.stdout: "verify_ssl": { 2023-12-01T01:22:33.832 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "verify_ssl", 2023-12-01T01:22:33.832 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.832 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.832 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.832 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "true", 2023-12-01T01:22:33.832 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.832 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.832 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.832 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-12-01T01:22:33.832 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.832 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.832 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.832 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.832 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.833 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.833 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.833 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "insights", 2023-12-01T01:22:33.833 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.833 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.833 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.833 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.833 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.833 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.833 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.833 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.833 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.833 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.833 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.834 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.834 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.834 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.834 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.834 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.834 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.834 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.834 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.834 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.834 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.834 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.834 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.834 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.834 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.835 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.835 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.835 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.835 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.835 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.835 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.835 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.835 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.835 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.835 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.835 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.835 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.835 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.835 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.836 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.836 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.836 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.836 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.836 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.836 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.836 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.836 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.836 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.836 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.836 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.836 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.836 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.836 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.837 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.837 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.837 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.837 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.837 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.837 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.837 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.837 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.837 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.837 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.837 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.837 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.837 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.837 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.837 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.838 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.838 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.838 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.838 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.838 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.838 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.838 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.838 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.838 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "iostat", 2023-12-01T01:22:33.838 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.838 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.838 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.838 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.838 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.839 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.839 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.839 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.839 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.839 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.839 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.839 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.839 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.839 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.839 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.839 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.839 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.839 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.839 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.839 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.840 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.840 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.840 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.840 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.840 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.840 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.840 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.840 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.840 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.840 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.840 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.840 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.840 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.840 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.840 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.841 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.841 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.841 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.841 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.841 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.841 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.841 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.841 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.841 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.841 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.841 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.841 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.841 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.841 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.842 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.842 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.842 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.842 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.842 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.842 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.842 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.842 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.842 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.842 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.842 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.842 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.842 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.842 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.842 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.843 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.843 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.843 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.843 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.843 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.843 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.843 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.843 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.843 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.843 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.843 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.843 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.843 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "k8sevents", 2023-12-01T01:22:33.843 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.844 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.844 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.844 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_event_retention_days": { 2023-12-01T01:22:33.844 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ceph_event_retention_days", 2023-12-01T01:22:33.844 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.844 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.844 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.844 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "7", 2023-12-01T01:22:33.844 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.844 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.844 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.844 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Days to hold ceph event information within local cache", 2023-12-01T01:22:33.844 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.844 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.845 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.845 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.845 INFO:teuthology.orchestra.run.smithi077.stdout: "config_check_secs": { 2023-12-01T01:22:33.845 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "config_check_secs", 2023-12-01T01:22:33.845 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.845 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.845 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.845 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2023-12-01T01:22:33.845 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "10", 2023-12-01T01:22:33.845 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.845 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.845 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-12-01T01:22:33.845 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.845 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.846 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.846 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.846 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.846 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.846 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.846 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.846 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.846 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.846 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.846 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.846 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.846 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.846 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.846 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.847 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.847 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.847 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.847 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.847 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.847 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.847 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.847 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.847 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.847 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.847 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.847 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.847 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.847 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.847 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.848 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.848 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.848 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.848 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.848 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.848 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.848 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.848 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.848 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.848 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.848 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.848 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.848 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.848 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.848 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.849 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.849 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.849 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.849 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.849 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.849 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.849 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.849 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.849 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.849 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.849 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.849 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.849 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.849 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.849 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.850 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.850 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.850 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.850 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.850 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.850 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.850 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.850 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.850 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.850 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.850 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.850 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.850 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.850 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.851 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.851 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.851 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "localpool", 2023-12-01T01:22:33.851 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.851 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.851 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.851 INFO:teuthology.orchestra.run.smithi077.stdout: "failure_domain": { 2023-12-01T01:22:33.851 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "failure_domain", 2023-12-01T01:22:33.851 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.851 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.851 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.851 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "host", 2023-12-01T01:22:33.851 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.851 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.851 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.852 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "failure domain for any created local pool", 2023-12-01T01:22:33.852 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-12-01T01:22:33.852 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.852 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.852 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.852 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.852 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.852 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.852 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.852 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.852 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.852 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.852 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.852 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.852 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.853 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.853 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.853 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.853 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.853 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.853 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.853 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.853 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.853 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.853 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.853 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.853 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.853 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.853 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.854 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.854 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.854 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.854 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.854 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.854 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.854 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.854 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.854 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.854 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.854 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.854 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.854 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.854 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.854 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.855 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.855 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.855 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.855 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.855 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.855 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.855 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.855 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.855 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.855 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.855 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.855 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.855 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.855 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.856 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.856 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.856 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.856 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.856 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.856 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.856 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.856 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.856 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.856 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.856 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.856 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.856 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.856 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.856 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.857 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.857 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.857 INFO:teuthology.orchestra.run.smithi077.stdout: "min_size": { 2023-12-01T01:22:33.857 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "min_size", 2023-12-01T01:22:33.857 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.857 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.857 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.857 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.857 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.857 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.857 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.857 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "default min_size for any created local pool", 2023-12-01T01:22:33.857 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-12-01T01:22:33.858 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.858 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.858 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.858 INFO:teuthology.orchestra.run.smithi077.stdout: "num_rep": { 2023-12-01T01:22:33.858 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "num_rep", 2023-12-01T01:22:33.858 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.858 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.858 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.858 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "3", 2023-12-01T01:22:33.858 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.858 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.858 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.858 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "default replica count for any created local pool", 2023-12-01T01:22:33.858 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.859 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.859 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.859 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.859 INFO:teuthology.orchestra.run.smithi077.stdout: "pg_num": { 2023-12-01T01:22:33.859 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pg_num", 2023-12-01T01:22:33.859 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.859 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.859 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.859 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "128", 2023-12-01T01:22:33.859 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.859 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.859 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.859 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "default pg_num for any created local pool", 2023-12-01T01:22:33.859 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.860 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.860 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.860 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.860 INFO:teuthology.orchestra.run.smithi077.stdout: "prefix": { 2023-12-01T01:22:33.860 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "prefix", 2023-12-01T01:22:33.860 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.860 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.860 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.860 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.860 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.860 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.860 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.860 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "name prefix for any created local pool", 2023-12-01T01:22:33.860 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.860 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.861 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.861 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.861 INFO:teuthology.orchestra.run.smithi077.stdout: "subtree": { 2023-12-01T01:22:33.861 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "subtree", 2023-12-01T01:22:33.861 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.861 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.861 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.861 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "rack", 2023-12-01T01:22:33.861 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.861 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.861 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.861 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "CRUSH level for which to create a local pool", 2023-12-01T01:22:33.861 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-12-01T01:22:33.861 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.862 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.862 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.862 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.862 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.862 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.862 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mds_autoscaler", 2023-12-01T01:22:33.862 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.862 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.862 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.862 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.862 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.862 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.862 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.862 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.862 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.863 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.863 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.863 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.863 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.863 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.863 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.863 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.863 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.863 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.863 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.863 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.863 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.863 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.863 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.864 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.864 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.864 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.864 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.864 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.864 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.864 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.864 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.864 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.864 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.864 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.864 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.864 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.864 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.864 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.865 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.865 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.865 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.865 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.865 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.865 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.865 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.865 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.865 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.865 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.865 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.865 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.865 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.865 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.865 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.866 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.866 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.866 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.866 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.866 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.866 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.866 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.866 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.866 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.866 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.866 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.866 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.866 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.866 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.867 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.867 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.867 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.867 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.867 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.867 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.867 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.867 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.867 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.867 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mirroring", 2023-12-01T01:22:33.867 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.867 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.867 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.867 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.867 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.868 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.868 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.868 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.868 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.868 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.868 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.868 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.868 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.868 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.868 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.868 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.868 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.868 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.868 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.869 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.869 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.869 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.869 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.869 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.869 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.869 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.869 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.869 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.869 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.869 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.869 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.869 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.869 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.869 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.870 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.870 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.870 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.870 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.870 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.870 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.870 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.870 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.870 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.871 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.871 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.871 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.871 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.871 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.871 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.871 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.871 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.871 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.871 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.871 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.871 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.871 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.871 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.871 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.872 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.872 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.872 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.872 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.872 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.872 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.872 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.872 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.872 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.872 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.872 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.872 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.872 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.872 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.873 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.873 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.873 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.873 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.873 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "nfs", 2023-12-01T01:22:33.873 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.873 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.873 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.873 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.873 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.873 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.873 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.873 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.873 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.873 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.874 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.874 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.874 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.874 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.874 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.874 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.874 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.874 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.874 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.874 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.874 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.874 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.874 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.875 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.875 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.875 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.875 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.875 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.875 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.875 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.875 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.875 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.875 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.875 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.875 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.875 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.875 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.875 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.876 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.876 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.876 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.876 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.876 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.876 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.876 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.876 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.876 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.876 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.876 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.876 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.876 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.876 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.877 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.877 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.877 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.877 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.877 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.877 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.877 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.877 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.877 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.877 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.877 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.877 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.877 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.877 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.877 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.878 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.878 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.878 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.878 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.878 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.878 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.878 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.878 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.878 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.878 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "orchestrator", 2023-12-01T01:22:33.878 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.878 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.878 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.878 INFO:teuthology.orchestra.run.smithi077.stdout: "fail_fs": { 2023-12-01T01:22:33.879 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "fail_fs", 2023-12-01T01:22:33.879 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.879 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.879 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.879 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.879 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.879 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.879 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.879 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-12-01T01:22:33.879 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.879 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.879 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.879 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.879 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.880 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.880 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.880 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.880 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.880 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.880 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.880 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.880 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.880 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.880 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.880 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.880 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.880 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.880 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.880 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.881 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.881 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.881 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.881 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.881 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.881 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.881 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.881 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.881 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.881 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.881 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.881 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.881 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.881 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.881 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.882 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.882 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.882 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.882 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.882 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.882 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.882 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.882 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.882 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.882 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.882 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.882 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.882 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.882 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.883 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.883 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.883 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.883 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.883 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.883 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.883 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.883 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.883 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.883 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.883 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.883 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.883 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.883 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.883 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.884 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.884 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.884 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.884 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.884 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.884 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.884 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.884 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.884 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.884 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.884 INFO:teuthology.orchestra.run.smithi077.stdout: "orchestrator": { 2023-12-01T01:22:33.884 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "orchestrator", 2023-12-01T01:22:33.884 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.885 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.885 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.885 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.885 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.885 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.885 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.885 INFO:teuthology.orchestra.run.smithi077.stdout: "cephadm", 2023-12-01T01:22:33.885 INFO:teuthology.orchestra.run.smithi077.stdout: "rook", 2023-12-01T01:22:33.885 INFO:teuthology.orchestra.run.smithi077.stdout: "test_orchestrator" 2023-12-01T01:22:33.885 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.885 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Orchestrator backend", 2023-12-01T01:22:33.885 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.885 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.885 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.885 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.886 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.886 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.886 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.886 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "osd_perf_query", 2023-12-01T01:22:33.886 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.886 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.886 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.886 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.886 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.886 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.886 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.886 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.886 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.886 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.887 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.887 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.887 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.887 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.887 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.887 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.887 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.887 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.887 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.887 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.887 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.887 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.887 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.887 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.887 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.888 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.888 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.888 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.888 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.888 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.888 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.888 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.888 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.888 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.888 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.888 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.888 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.888 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.888 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.889 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.889 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.889 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.889 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.889 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.889 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.889 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.889 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.889 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.889 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.889 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.889 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.889 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.889 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.889 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.890 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.890 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.890 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.890 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.890 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.890 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.890 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.890 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.890 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.890 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.890 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.890 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.890 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.890 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.891 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.891 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.891 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.891 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.891 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.891 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.891 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.891 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.891 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "osd_support", 2023-12-01T01:22:33.891 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.891 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.891 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.891 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.891 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.891 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.892 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.892 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.892 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.892 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.892 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.892 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.892 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.892 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.892 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.892 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.892 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.892 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.892 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.892 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.893 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.893 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.893 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.893 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.893 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.893 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.893 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.893 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.893 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.893 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.893 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.893 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.893 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.893 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.893 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.894 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.894 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.894 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.894 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.894 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.894 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.894 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.894 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.894 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.894 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.894 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.894 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.894 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.894 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.894 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.895 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.895 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.895 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.895 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.895 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.895 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.895 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.895 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.895 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.895 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.895 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.895 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.895 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.895 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.895 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.896 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.896 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.896 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.896 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.896 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.896 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.896 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.896 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.896 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.896 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.896 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.896 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pg_autoscaler", 2023-12-01T01:22:33.896 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.896 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.897 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.897 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.897 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.897 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.897 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.897 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.897 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.897 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.897 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.897 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.897 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.897 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.897 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.897 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.897 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.898 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.898 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.898 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.898 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.898 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.898 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.898 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.898 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.898 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.898 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.898 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.898 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.898 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.899 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.899 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.899 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.899 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.899 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.899 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.899 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.899 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.899 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.899 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.899 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.899 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.899 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.899 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.899 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.900 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.900 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.900 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.900 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.900 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.900 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.900 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.900 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.900 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.900 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.900 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.900 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.900 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.900 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.900 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.901 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.901 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.901 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.901 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.901 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.901 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.901 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.901 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.901 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.901 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.901 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.901 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.901 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.901 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2023-12-01T01:22:33.902 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2023-12-01T01:22:33.902 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:33.902 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.902 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.902 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2023-12-01T01:22:33.902 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.902 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.902 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.902 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.902 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.902 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.902 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.902 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.902 INFO:teuthology.orchestra.run.smithi077.stdout: "threshold": { 2023-12-01T01:22:33.903 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "threshold", 2023-12-01T01:22:33.903 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2023-12-01T01:22:33.903 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.903 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.903 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "3.0", 2023-12-01T01:22:33.903 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1.0", 2023-12-01T01:22:33.903 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.903 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.903 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "scaling threshold", 2023-12-01T01:22:33.903 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-12-01T01:22:33.903 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.903 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.903 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.903 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.903 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.904 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.904 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "progress", 2023-12-01T01:22:33.904 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.904 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.904 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.904 INFO:teuthology.orchestra.run.smithi077.stdout: "allow_pg_recovery_event": { 2023-12-01T01:22:33.904 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "allow_pg_recovery_event", 2023-12-01T01:22:33.904 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.904 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.904 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.904 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.904 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.904 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.904 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.905 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "allow the module to show pg recovery progress", 2023-12-01T01:22:33.905 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.905 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.905 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.905 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.905 INFO:teuthology.orchestra.run.smithi077.stdout: "enabled": { 2023-12-01T01:22:33.905 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enabled", 2023-12-01T01:22:33.905 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.905 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.905 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.905 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.905 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.905 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.905 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.905 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.906 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.906 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.906 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.906 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.906 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.906 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.906 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.906 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.906 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.906 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.906 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.906 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.906 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.906 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.907 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.907 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.907 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.907 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.907 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.907 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.907 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.907 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.907 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.907 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.907 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.907 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.907 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.907 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.907 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.908 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.908 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.908 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.908 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.908 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.908 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.908 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.908 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.908 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.908 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.908 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.908 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.908 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.908 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.909 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.909 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.909 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.909 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.909 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.909 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.909 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.909 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.909 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.909 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.909 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.909 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.909 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.909 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.909 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.910 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.910 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.910 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.910 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.910 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.910 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.910 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.910 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.910 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.910 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.910 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.910 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.910 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.910 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.911 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.911 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.911 INFO:teuthology.orchestra.run.smithi077.stdout: "max_completed_events": { 2023-12-01T01:22:33.911 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_completed_events", 2023-12-01T01:22:33.911 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.911 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.911 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.911 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "50", 2023-12-01T01:22:33.911 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.911 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.911 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.911 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "number of past completed events to remember", 2023-12-01T01:22:33.911 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.911 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.912 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.912 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.912 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2023-12-01T01:22:33.912 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2023-12-01T01:22:33.912 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:33.912 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.912 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.912 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2023-12-01T01:22:33.912 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.912 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.912 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.912 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how long the module is going to sleep", 2023-12-01T01:22:33.912 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.912 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.913 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.913 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.913 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.913 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.913 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.913 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "prometheus", 2023-12-01T01:22:33.913 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.913 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.913 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.913 INFO:teuthology.orchestra.run.smithi077.stdout: "cache": { 2023-12-01T01:22:33.913 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cache", 2023-12-01T01:22:33.913 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.913 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.913 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.913 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.914 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.914 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.914 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.914 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.914 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.914 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.914 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.914 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.914 INFO:teuthology.orchestra.run.smithi077.stdout: "exclude_perf_counters": { 2023-12-01T01:22:33.914 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "exclude_perf_counters", 2023-12-01T01:22:33.914 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.914 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.914 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.914 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.915 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.915 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.915 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.915 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-12-01T01:22:33.915 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-12-01T01:22:33.915 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.915 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.915 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.915 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.915 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.915 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.915 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.915 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.915 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.915 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.916 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.916 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.916 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.916 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.916 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.916 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.916 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.916 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.916 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.916 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.916 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.916 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.916 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.916 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.916 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.917 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.917 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.917 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.917 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.917 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.917 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.917 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.917 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.917 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.917 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.917 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.917 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.917 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.917 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.917 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.918 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.918 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.918 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.918 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.918 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.918 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.918 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.918 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.918 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.918 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.918 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.918 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.918 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.918 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.919 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.919 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.919 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.919 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.919 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.919 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.919 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.919 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.919 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.919 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.919 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.919 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.919 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.919 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.919 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.920 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.920 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.920 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.920 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.920 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_stats_pools": { 2023-12-01T01:22:33.920 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rbd_stats_pools", 2023-12-01T01:22:33.920 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.920 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.920 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.920 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.920 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.920 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.920 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.920 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.920 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.921 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.921 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.921 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.921 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_stats_pools_refresh_interval": { 2023-12-01T01:22:33.921 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-12-01T01:22:33.921 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.921 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.921 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.921 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "300", 2023-12-01T01:22:33.921 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.921 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.921 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.921 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.921 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.921 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.922 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.922 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.922 INFO:teuthology.orchestra.run.smithi077.stdout: "scrape_interval": { 2023-12-01T01:22:33.922 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "scrape_interval", 2023-12-01T01:22:33.922 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2023-12-01T01:22:33.922 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.922 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.922 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "15.0", 2023-12-01T01:22:33.922 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.922 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.922 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.922 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.922 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.922 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.923 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.923 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.923 INFO:teuthology.orchestra.run.smithi077.stdout: "server_addr": { 2023-12-01T01:22:33.923 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_addr", 2023-12-01T01:22:33.923 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.923 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.923 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.923 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "::", 2023-12-01T01:22:33.923 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.923 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.923 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.923 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-12-01T01:22:33.923 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.924 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.924 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.924 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.924 INFO:teuthology.orchestra.run.smithi077.stdout: "server_port": { 2023-12-01T01:22:33.924 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_port", 2023-12-01T01:22:33.924 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.924 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.924 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.924 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "9283", 2023-12-01T01:22:33.924 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.924 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.924 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.924 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-12-01T01:22:33.924 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.925 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.925 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.925 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.925 INFO:teuthology.orchestra.run.smithi077.stdout: "stale_cache_strategy": { 2023-12-01T01:22:33.925 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "stale_cache_strategy", 2023-12-01T01:22:33.925 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.925 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.925 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.925 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "log", 2023-12-01T01:22:33.925 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.925 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.925 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.925 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.925 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.926 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.926 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.926 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.926 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_behaviour": { 2023-12-01T01:22:33.926 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_behaviour", 2023-12-01T01:22:33.926 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.926 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.926 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.926 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "default", 2023-12-01T01:22:33.926 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.926 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.926 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.926 INFO:teuthology.orchestra.run.smithi077.stdout: "default", 2023-12-01T01:22:33.926 INFO:teuthology.orchestra.run.smithi077.stdout: "error" 2023-12-01T01:22:33.927 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.927 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.927 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.927 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.927 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.927 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.927 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_error_status_code": { 2023-12-01T01:22:33.927 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_error_status_code", 2023-12-01T01:22:33.927 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.927 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.927 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.927 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "500", 2023-12-01T01:22:33.927 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "400", 2023-12-01T01:22:33.927 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "599", 2023-12-01T01:22:33.928 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.928 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.928 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.928 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.928 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.928 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.928 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.928 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.928 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.928 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rbd_support", 2023-12-01T01:22:33.928 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.928 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.928 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.928 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.928 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.929 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.929 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.929 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.929 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.929 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.929 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.929 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.929 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.929 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.929 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.929 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.929 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.929 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.929 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.930 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.930 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.930 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.930 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.930 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.930 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.930 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.930 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.930 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.930 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.930 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.930 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.930 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.930 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.931 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.931 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.931 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.931 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.931 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.931 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.931 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.931 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.931 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.931 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.931 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.931 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.931 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.931 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.931 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.932 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.932 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.932 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.932 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.932 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.932 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.932 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.932 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.932 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.932 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.932 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.932 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.932 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.932 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.932 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.933 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.933 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.933 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.933 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.933 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.933 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.933 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.933 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.933 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.933 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.933 INFO:teuthology.orchestra.run.smithi077.stdout: "max_concurrent_snap_create": { 2023-12-01T01:22:33.933 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_concurrent_snap_create", 2023-12-01T01:22:33.933 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.933 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.934 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.934 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2023-12-01T01:22:33.934 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.934 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.934 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.934 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.934 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.934 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.934 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.934 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.934 INFO:teuthology.orchestra.run.smithi077.stdout: "mirror_snapshot_schedule": { 2023-12-01T01:22:33.934 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mirror_snapshot_schedule", 2023-12-01T01:22:33.934 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.934 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.934 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.935 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.935 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.935 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.935 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.935 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.935 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.935 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.935 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.935 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.935 INFO:teuthology.orchestra.run.smithi077.stdout: "trash_purge_schedule": { 2023-12-01T01:22:33.935 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "trash_purge_schedule", 2023-12-01T01:22:33.935 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.935 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.935 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.936 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.936 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.936 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.936 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.936 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.936 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.936 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.936 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.936 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.936 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.936 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.936 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.936 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "restful", 2023-12-01T01:22:33.936 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.936 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.937 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.937 INFO:teuthology.orchestra.run.smithi077.stdout: "enable_auth": { 2023-12-01T01:22:33.937 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enable_auth", 2023-12-01T01:22:33.937 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.937 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.937 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.937 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.937 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.937 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.937 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.937 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.937 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.937 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.938 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.938 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.938 INFO:teuthology.orchestra.run.smithi077.stdout: "key_file": { 2023-12-01T01:22:33.938 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "key_file", 2023-12-01T01:22:33.938 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.938 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.938 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.938 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.938 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.938 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.938 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.938 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.938 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.938 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.939 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.939 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.939 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.939 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.939 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.939 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.939 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.939 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.939 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.939 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.939 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.939 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.939 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.939 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.940 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.940 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.940 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.940 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.940 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.940 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.940 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.940 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.940 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.940 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.940 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.940 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.940 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.940 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.940 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.941 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.941 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.941 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.941 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.941 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.941 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.941 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.941 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.941 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.941 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.941 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.941 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.941 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.941 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.942 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.942 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.942 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.942 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.942 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.942 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.942 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.942 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.942 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.942 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.942 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.942 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.942 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.942 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.943 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.943 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.943 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.943 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.943 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.943 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.943 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.943 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.943 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.943 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.943 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.943 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.943 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.943 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.943 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.944 INFO:teuthology.orchestra.run.smithi077.stdout: "server_addr": { 2023-12-01T01:22:33.944 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_addr", 2023-12-01T01:22:33.944 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.944 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.944 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.944 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.944 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.944 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.944 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.944 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.944 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.944 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.944 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.944 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.944 INFO:teuthology.orchestra.run.smithi077.stdout: "server_port": { 2023-12-01T01:22:33.945 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_port", 2023-12-01T01:22:33.945 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.945 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.945 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.945 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.945 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.945 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.945 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.945 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.945 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.945 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.945 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.945 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.945 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.946 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.946 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.946 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rgw", 2023-12-01T01:22:33.946 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.946 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.946 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.946 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.946 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.946 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.946 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.946 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.946 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.946 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.946 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.946 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.947 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.947 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.947 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.947 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.947 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.947 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.947 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.947 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.947 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.947 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.947 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.947 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.947 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.947 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.948 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.948 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.948 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.948 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.948 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.948 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.948 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.948 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.948 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.948 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.948 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.948 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.948 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.948 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.948 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.949 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.949 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.949 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.949 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.949 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.949 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.949 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.949 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.949 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.949 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.949 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.949 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.949 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.949 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.950 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.950 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.950 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.950 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.950 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.950 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.950 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.950 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.950 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.950 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.950 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.950 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.950 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.950 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.951 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.951 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.951 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.951 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.951 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.951 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.951 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.951 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rook", 2023-12-01T01:22:33.951 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.951 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.951 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.951 INFO:teuthology.orchestra.run.smithi077.stdout: "drive_group_interval": { 2023-12-01T01:22:33.951 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "drive_group_interval", 2023-12-01T01:22:33.952 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2023-12-01T01:22:33.952 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.952 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.952 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "300.0", 2023-12-01T01:22:33.952 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.952 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.952 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.952 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-12-01T01:22:33.952 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.952 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.952 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.952 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.952 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.952 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.952 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.953 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.953 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.953 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.953 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.953 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.953 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.953 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.953 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.953 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.953 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.953 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.953 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.953 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.953 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.954 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.954 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.954 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.954 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.954 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.954 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.954 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.954 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.954 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.954 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.954 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.954 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.954 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.954 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.955 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.955 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.955 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.955 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.955 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.955 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.955 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.955 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.955 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.955 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.955 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.955 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.955 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.955 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.956 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.956 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.956 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.956 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.956 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.956 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.956 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.956 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.956 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.956 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.956 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.956 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.956 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.956 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.957 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.957 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.957 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.957 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.957 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.957 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.957 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.957 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.957 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.957 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.957 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.957 INFO:teuthology.orchestra.run.smithi077.stdout: "storage_class": { 2023-12-01T01:22:33.957 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "storage_class", 2023-12-01T01:22:33.957 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.958 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.958 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.958 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "local", 2023-12-01T01:22:33.958 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.958 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.958 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.958 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-12-01T01:22:33.958 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.958 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.958 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.958 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.958 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.958 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.959 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.959 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "selftest", 2023-12-01T01:22:33.959 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.959 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.959 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.959 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.959 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.959 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.959 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.959 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.959 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.959 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.959 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.959 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.960 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.960 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.960 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.960 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.960 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.960 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.960 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.960 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.960 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.960 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.960 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.960 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.960 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.960 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.960 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.961 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.961 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.961 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.961 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.961 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.961 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.961 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.961 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.961 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.961 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.961 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.961 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.961 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.961 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.962 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.962 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.962 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.962 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.962 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.962 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.962 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.962 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.962 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.962 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.962 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.962 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.962 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.962 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.963 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.963 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.963 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.963 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.963 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.963 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.963 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.963 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.963 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.963 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.963 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.963 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.963 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.963 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.963 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.964 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.964 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.964 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.964 INFO:teuthology.orchestra.run.smithi077.stdout: "roption1": { 2023-12-01T01:22:33.964 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "roption1", 2023-12-01T01:22:33.964 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.964 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.964 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.964 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.964 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.964 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.964 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.964 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.964 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.965 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.965 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.965 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.965 INFO:teuthology.orchestra.run.smithi077.stdout: "roption2": { 2023-12-01T01:22:33.965 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "roption2", 2023-12-01T01:22:33.965 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.965 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.965 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.965 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "xyz", 2023-12-01T01:22:33.965 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.965 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.965 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.965 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.965 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.966 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.966 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.966 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.966 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption1": { 2023-12-01T01:22:33.966 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption1", 2023-12-01T01:22:33.966 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.966 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.966 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.966 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.966 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.966 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.966 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.966 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.966 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.967 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.967 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.967 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.967 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption2": { 2023-12-01T01:22:33.967 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption2", 2023-12-01T01:22:33.967 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.967 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.967 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.967 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.967 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.967 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.967 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.967 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.967 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.968 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.968 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.968 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.968 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption3": { 2023-12-01T01:22:33.968 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption3", 2023-12-01T01:22:33.968 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2023-12-01T01:22:33.968 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.968 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.968 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.968 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.968 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.968 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.968 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.968 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.968 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.969 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.969 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.969 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption4": { 2023-12-01T01:22:33.969 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption4", 2023-12-01T01:22:33.969 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.969 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.969 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.969 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.969 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.969 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.969 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.969 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.969 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.969 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.970 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.970 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.970 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption5": { 2023-12-01T01:22:33.970 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption5", 2023-12-01T01:22:33.970 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.970 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.970 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.970 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.970 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.970 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.970 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.970 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.970 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.970 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.970 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.971 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.971 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption6": { 2023-12-01T01:22:33.971 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption6", 2023-12-01T01:22:33.971 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.971 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.971 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.971 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:33.971 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.971 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.971 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.971 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.971 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.971 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.971 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.971 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.972 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption7": { 2023-12-01T01:22:33.972 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption7", 2023-12-01T01:22:33.972 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:33.972 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.972 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.972 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.972 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2023-12-01T01:22:33.972 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "42", 2023-12-01T01:22:33.972 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.972 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.972 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.972 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.972 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.972 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.973 INFO:teuthology.orchestra.run.smithi077.stdout: "testkey": { 2023-12-01T01:22:33.973 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "testkey", 2023-12-01T01:22:33.973 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.973 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.973 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.973 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.973 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.973 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.973 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.973 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.973 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.973 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.973 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.973 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.973 INFO:teuthology.orchestra.run.smithi077.stdout: "testlkey": { 2023-12-01T01:22:33.974 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "testlkey", 2023-12-01T01:22:33.974 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.974 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.974 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.974 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.974 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.974 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.974 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.974 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.974 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.974 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.974 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.974 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.974 INFO:teuthology.orchestra.run.smithi077.stdout: "testnewline": { 2023-12-01T01:22:33.975 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "testnewline", 2023-12-01T01:22:33.975 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.975 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.975 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.975 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.975 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.975 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.975 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.975 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.975 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.975 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.975 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.975 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.975 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.976 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.976 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.976 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "snap_schedule", 2023-12-01T01:22:33.976 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.976 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.976 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.976 INFO:teuthology.orchestra.run.smithi077.stdout: "allow_m_granularity": { 2023-12-01T01:22:33.976 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "allow_m_granularity", 2023-12-01T01:22:33.976 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.976 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.976 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.976 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.976 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.976 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.976 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.977 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "allow minute scheduled snapshots", 2023-12-01T01:22:33.977 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.977 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.977 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.977 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.977 INFO:teuthology.orchestra.run.smithi077.stdout: "dump_on_update": { 2023-12-01T01:22:33.977 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "dump_on_update", 2023-12-01T01:22:33.977 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.977 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.977 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.977 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.977 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.977 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.977 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.978 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "dump database to debug log on update", 2023-12-01T01:22:33.978 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.978 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.978 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.978 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.978 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.978 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.978 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.978 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.978 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.978 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.978 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.978 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.979 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.979 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.979 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.979 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.979 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.979 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.979 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.979 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.979 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.979 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.979 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.979 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.979 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.979 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.980 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.980 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.980 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.980 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.980 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.980 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.980 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.980 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.980 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.980 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.980 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.980 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.980 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.980 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.980 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.981 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.981 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.981 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.981 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.981 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.981 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.981 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.981 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.981 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.981 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.981 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.981 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.981 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.981 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.981 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.982 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.982 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.982 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.982 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.982 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.982 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.982 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.982 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.982 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.982 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.982 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.982 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.982 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.982 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.982 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.983 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.983 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.983 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.983 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.983 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.983 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.983 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "stats", 2023-12-01T01:22:33.983 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.983 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.983 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.983 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.983 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.983 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.983 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.984 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.984 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.984 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.984 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.984 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.984 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.984 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.984 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.984 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.984 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.984 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.984 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.984 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.984 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.985 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.985 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.985 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.985 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.985 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.985 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.985 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.985 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.985 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.985 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.985 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.985 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.985 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.985 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.985 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.986 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.986 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.986 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.986 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.986 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.986 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.986 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.986 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.986 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.986 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.986 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.986 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.986 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.986 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.987 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.987 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.987 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.987 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.987 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.987 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.987 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.987 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.987 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.987 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.987 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.987 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.987 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.987 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.987 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.988 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.988 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.988 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.988 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.988 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.988 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.988 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.988 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.988 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.988 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.988 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.988 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "status", 2023-12-01T01:22:33.988 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.988 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.988 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.989 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.989 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.989 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.989 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.989 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.989 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.989 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.989 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.989 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.989 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.989 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.989 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.989 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.989 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.989 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.990 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.990 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.990 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.990 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.990 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.990 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.990 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.990 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.990 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.990 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.990 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.990 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.990 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.990 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.991 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.991 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.991 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.991 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.991 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.991 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.991 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.991 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.991 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.991 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.991 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.991 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.991 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.991 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.992 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.992 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.992 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.992 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.992 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.992 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.992 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.992 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.992 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.992 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.992 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.992 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.992 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.992 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:33.992 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:33.993 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.993 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.993 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.993 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.993 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.993 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.993 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.993 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.993 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.993 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.993 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.993 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.993 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:33.993 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.994 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:33.994 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "telegraf", 2023-12-01T01:22:33.994 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:33.994 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:33.994 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:33.994 INFO:teuthology.orchestra.run.smithi077.stdout: "address": { 2023-12-01T01:22:33.994 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "address", 2023-12-01T01:22:33.994 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.994 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.994 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.994 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-12-01T01:22:33.994 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.994 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.994 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.995 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.995 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.995 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.995 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.995 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.995 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2023-12-01T01:22:33.995 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2023-12-01T01:22:33.995 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:33.995 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.995 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:33.995 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "15", 2023-12-01T01:22:33.995 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.995 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.995 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.995 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.996 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.996 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.996 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.996 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.996 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:33.996 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:33.996 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.996 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.996 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.996 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:33.996 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.996 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.996 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.996 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.996 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.997 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.997 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.997 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.997 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.997 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.997 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.997 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.997 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.997 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.997 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.997 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:33.997 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:33.997 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:33.997 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.998 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.998 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:33.998 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.998 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.998 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:33.998 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.998 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.998 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:33.998 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:33.998 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:33.998 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:33.998 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:33.998 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:33.998 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:33.999 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:33.999 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:33.999 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:33.999 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:33.999 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:33.999 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:33.999 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:33.999 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:33.999 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:33.999 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:33.999 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:33.999 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:33.999 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:33.999 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:33.999 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.000 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.000 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.000 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:34.000 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:34.000 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.000 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.000 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.000 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.000 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.000 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.000 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.000 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.000 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.000 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.001 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.001 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.001 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.001 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.001 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.001 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "telemetry", 2023-12-01T01:22:34.001 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:34.001 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:34.001 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:34.002 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_basic": { 2023-12-01T01:22:34.002 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_basic", 2023-12-01T01:22:34.002 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.002 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.002 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.002 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:34.002 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.002 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.002 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.002 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share basic cluster information (size, version)", 2023-12-01T01:22:34.002 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.002 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.002 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.002 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.003 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_crash": { 2023-12-01T01:22:34.003 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_crash", 2023-12-01T01:22:34.003 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.003 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.003 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.003 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:34.003 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.003 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.003 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.003 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-12-01T01:22:34.003 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.003 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.003 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.003 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.004 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_device": { 2023-12-01T01:22:34.004 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_device", 2023-12-01T01:22:34.004 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.004 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.005 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.005 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:34.005 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.006 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.006 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.006 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-12-01T01:22:34.006 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.006 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.006 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.006 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.006 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_ident": { 2023-12-01T01:22:34.006 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_ident", 2023-12-01T01:22:34.006 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.006 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.006 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.006 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.006 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.007 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.007 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.007 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-12-01T01:22:34.007 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.007 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.007 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.007 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.007 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_perf": { 2023-12-01T01:22:34.007 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_perf", 2023-12-01T01:22:34.007 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.007 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.007 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.007 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.008 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.008 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.008 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.008 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share various performance metrics of a cluster", 2023-12-01T01:22:34.008 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.008 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.008 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.008 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.008 INFO:teuthology.orchestra.run.smithi077.stdout: "contact": { 2023-12-01T01:22:34.008 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "contact", 2023-12-01T01:22:34.008 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.008 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.008 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.008 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.008 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.009 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.009 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.009 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.009 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.009 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.009 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.009 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.009 INFO:teuthology.orchestra.run.smithi077.stdout: "description": { 2023-12-01T01:22:34.009 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "description", 2023-12-01T01:22:34.009 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.009 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.009 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.009 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.009 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.009 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.010 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.010 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.010 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.010 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.010 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.010 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.010 INFO:teuthology.orchestra.run.smithi077.stdout: "device_url": { 2023-12-01T01:22:34.010 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "device_url", 2023-12-01T01:22:34.010 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.010 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.010 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.010 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-12-01T01:22:34.010 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.010 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.011 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.011 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.011 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.011 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.011 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.011 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.011 INFO:teuthology.orchestra.run.smithi077.stdout: "enabled": { 2023-12-01T01:22:34.011 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enabled", 2023-12-01T01:22:34.011 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.011 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.011 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.011 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.011 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.011 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.011 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.012 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.012 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.012 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.012 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.012 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.012 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2023-12-01T01:22:34.012 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2023-12-01T01:22:34.012 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.012 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.012 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.012 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "24", 2023-12-01T01:22:34.012 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "8", 2023-12-01T01:22:34.012 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.012 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.013 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.013 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.013 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.013 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.013 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.013 INFO:teuthology.orchestra.run.smithi077.stdout: "last_opt_revision": { 2023-12-01T01:22:34.013 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "last_opt_revision", 2023-12-01T01:22:34.013 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.013 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.013 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.013 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "1", 2023-12-01T01:22:34.013 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.013 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.013 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.013 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.014 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.014 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.014 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.014 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.014 INFO:teuthology.orchestra.run.smithi077.stdout: "leaderboard": { 2023-12-01T01:22:34.014 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "leaderboard", 2023-12-01T01:22:34.014 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.014 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.014 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.014 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.014 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.014 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.014 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.014 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.015 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.015 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.015 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.015 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.015 INFO:teuthology.orchestra.run.smithi077.stdout: "leaderboard_description": { 2023-12-01T01:22:34.015 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "leaderboard_description", 2023-12-01T01:22:34.015 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.015 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.015 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.015 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.015 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.015 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.015 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.015 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.015 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.016 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.016 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.016 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.016 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:34.016 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:34.016 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.016 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.016 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.016 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.016 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.016 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.016 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.016 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.016 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.016 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.017 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.017 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.017 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.017 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.017 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.017 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.017 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.017 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.017 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.017 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:34.017 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:34.017 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.017 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.017 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.017 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.018 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.018 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.018 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.018 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.018 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.018 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.018 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.018 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.018 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:34.018 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:34.018 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.018 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.018 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.018 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:34.019 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.019 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.019 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.019 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.019 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.019 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.019 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.019 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.019 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.019 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.019 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.019 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.019 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.019 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.020 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.020 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:34.020 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:34.020 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.020 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.020 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.020 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.020 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.020 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.020 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.020 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.020 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.020 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.020 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.021 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.021 INFO:teuthology.orchestra.run.smithi077.stdout: "organization": { 2023-12-01T01:22:34.021 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "organization", 2023-12-01T01:22:34.021 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.021 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.021 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.021 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.021 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.021 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.021 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.021 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.021 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.021 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.021 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.021 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.022 INFO:teuthology.orchestra.run.smithi077.stdout: "proxy": { 2023-12-01T01:22:34.022 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "proxy", 2023-12-01T01:22:34.022 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.022 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.022 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.022 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.022 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.022 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.022 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.022 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.022 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.022 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.022 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.022 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.023 INFO:teuthology.orchestra.run.smithi077.stdout: "url": { 2023-12-01T01:22:34.023 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "url", 2023-12-01T01:22:34.023 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.023 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.023 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.023 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-12-01T01:22:34.023 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.023 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.023 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.023 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.023 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.023 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.023 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.023 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.023 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.024 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.024 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.024 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "test_orchestrator", 2023-12-01T01:22:34.024 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:34.024 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:34.024 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:34.024 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:34.024 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:34.024 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.024 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.024 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.024 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.024 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.024 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.025 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.025 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.025 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.025 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.025 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.025 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.025 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.025 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.025 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.025 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.025 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.025 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.025 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.025 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:34.025 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:34.026 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.026 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.026 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.026 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.026 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.026 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.026 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.026 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.026 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.026 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.026 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.026 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.026 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:34.026 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:34.027 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.027 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.027 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.027 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:34.027 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.027 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.027 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.027 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.027 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.027 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.027 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.027 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.027 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.027 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.028 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.028 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.028 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.028 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.028 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.028 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:34.028 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:34.028 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.028 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.028 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.028 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.028 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.028 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.028 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.029 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.029 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.029 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.029 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.029 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.029 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.029 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.029 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.029 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "volumes", 2023-12-01T01:22:34.029 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:34.029 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:34.029 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:34.029 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:34.029 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:34.029 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.030 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.030 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.030 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.030 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.030 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.030 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.030 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.030 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.030 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.030 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.030 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.030 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.030 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.030 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.031 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.031 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.031 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.031 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.031 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:34.031 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:34.031 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.031 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.031 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.031 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.031 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.031 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.031 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.031 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.032 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.032 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.032 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.032 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.032 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:34.032 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:34.032 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.032 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.032 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.032 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:34.032 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.032 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.032 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.032 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.032 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.033 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.033 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.033 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.033 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.033 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.033 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.033 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.033 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.033 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.033 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.033 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:34.033 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:34.033 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.034 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.034 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.034 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.034 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.034 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.034 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.034 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.034 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.034 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.034 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.034 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.034 INFO:teuthology.orchestra.run.smithi077.stdout: "max_concurrent_clones": { 2023-12-01T01:22:34.034 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_concurrent_clones", 2023-12-01T01:22:34.035 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.035 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.035 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.035 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "4", 2023-12-01T01:22:34.035 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.035 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.035 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.035 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Number of asynchronous cloner threads", 2023-12-01T01:22:34.035 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.035 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.035 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.035 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.035 INFO:teuthology.orchestra.run.smithi077.stdout: "snapshot_clone_delay": { 2023-12-01T01:22:34.035 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "snapshot_clone_delay", 2023-12-01T01:22:34.036 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.036 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.036 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.036 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2023-12-01T01:22:34.036 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.036 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.036 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.036 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-12-01T01:22:34.036 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.036 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.036 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.036 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.036 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.036 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.036 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.037 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix", 2023-12-01T01:22:34.037 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:34.037 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:34.037 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:34.037 INFO:teuthology.orchestra.run.smithi077.stdout: "discovery_interval": { 2023-12-01T01:22:34.037 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "discovery_interval", 2023-12-01T01:22:34.037 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2023-12-01T01:22:34.037 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.037 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.037 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "100", 2023-12-01T01:22:34.037 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.037 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.037 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.037 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.038 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.038 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.038 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.038 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.038 INFO:teuthology.orchestra.run.smithi077.stdout: "identifier": { 2023-12-01T01:22:34.038 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "identifier", 2023-12-01T01:22:34.038 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.038 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.038 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.038 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.038 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.038 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.038 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.038 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.038 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.039 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.039 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.039 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.039 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2023-12-01T01:22:34.039 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2023-12-01T01:22:34.039 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:34.039 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.039 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.039 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2023-12-01T01:22:34.039 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.039 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.039 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.039 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.039 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.040 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.040 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.040 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.040 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:34.040 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:34.040 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.040 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.040 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.040 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.040 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.040 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.040 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.040 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.040 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.040 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.041 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.041 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.041 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.041 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.041 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.041 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.041 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.041 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.041 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.041 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:34.041 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:34.041 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.041 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.041 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.042 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.042 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.042 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.042 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.042 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.042 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.042 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.042 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.042 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.042 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:34.042 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:34.042 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.042 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.042 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.043 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:34.043 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.043 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.043 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.043 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.043 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.043 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.043 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.043 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.043 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.043 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.043 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.043 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.043 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.044 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.044 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.044 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:34.044 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:34.044 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.044 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.044 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.044 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.044 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.044 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.044 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.044 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.044 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.044 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.044 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.045 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.045 INFO:teuthology.orchestra.run.smithi077.stdout: "zabbix_host": { 2023-12-01T01:22:34.045 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix_host", 2023-12-01T01:22:34.045 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.045 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.045 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.045 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.045 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.045 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.045 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.045 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.045 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.045 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.045 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.045 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.046 INFO:teuthology.orchestra.run.smithi077.stdout: "zabbix_port": { 2023-12-01T01:22:34.046 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix_port", 2023-12-01T01:22:34.046 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.046 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.046 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.046 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10051", 2023-12-01T01:22:34.046 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.046 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.046 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.046 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.046 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.046 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.046 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.046 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.047 INFO:teuthology.orchestra.run.smithi077.stdout: "zabbix_sender": { 2023-12-01T01:22:34.047 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix_sender", 2023-12-01T01:22:34.047 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.047 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.047 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.047 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-12-01T01:22:34.047 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.047 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.047 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.047 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.047 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.047 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.047 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.047 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.048 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.048 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.048 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2023-12-01T01:22:34.048 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.048 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.048 INFO:teuthology.orchestra.run.smithi077.stdout: "modules": [ 2023-12-01T01:22:34.048 INFO:teuthology.orchestra.run.smithi077.stdout: "cephadm", 2023-12-01T01:22:34.048 INFO:teuthology.orchestra.run.smithi077.stdout: "dashboard", 2023-12-01T01:22:34.048 INFO:teuthology.orchestra.run.smithi077.stdout: "iostat", 2023-12-01T01:22:34.048 INFO:teuthology.orchestra.run.smithi077.stdout: "nfs", 2023-12-01T01:22:34.048 INFO:teuthology.orchestra.run.smithi077.stdout: "restful" 2023-12-01T01:22:34.048 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.048 INFO:teuthology.orchestra.run.smithi077.stdout: "available_modules": [ 2023-12-01T01:22:34.048 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.048 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "alerts", 2023-12-01T01:22:34.049 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:34.049 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:34.049 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:34.049 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2023-12-01T01:22:34.049 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2023-12-01T01:22:34.049 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:34.049 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.049 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.049 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2023-12-01T01:22:34.049 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.049 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.049 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.049 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How frequently to reexamine health status", 2023-12-01T01:22:34.049 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.050 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.050 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.050 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.050 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:34.050 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:34.050 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.050 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.050 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.050 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.050 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.050 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.050 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.050 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.050 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.050 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.051 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.051 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.051 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.051 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.051 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.051 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.051 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.051 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.051 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.051 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:34.051 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:34.051 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.051 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.051 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.052 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.052 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.052 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.052 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.052 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.052 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.052 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.052 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.052 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.052 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:34.052 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:34.052 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.052 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.052 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.052 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:34.053 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.053 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.053 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.053 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.053 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.053 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.053 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.053 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.053 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.053 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.053 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.053 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.053 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.053 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.054 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.054 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:34.054 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:34.054 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.054 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.054 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.054 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.054 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.054 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.054 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.054 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.054 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.054 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.054 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.054 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.055 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_destination": { 2023-12-01T01:22:34.055 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_destination", 2023-12-01T01:22:34.055 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.055 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.055 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.055 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.055 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.055 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.055 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.055 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Email address to send alerts to", 2023-12-01T01:22:34.055 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.055 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.055 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.055 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.056 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_from_name": { 2023-12-01T01:22:34.056 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_from_name", 2023-12-01T01:22:34.056 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.056 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.056 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.056 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "Ceph", 2023-12-01T01:22:34.056 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.056 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.056 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.056 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Email From: name", 2023-12-01T01:22:34.056 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.056 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.056 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.056 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.056 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_host": { 2023-12-01T01:22:34.057 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_host", 2023-12-01T01:22:34.057 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.057 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.057 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.057 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.057 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.057 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.057 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.057 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SMTP server", 2023-12-01T01:22:34.057 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.057 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.057 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.057 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.057 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_password": { 2023-12-01T01:22:34.058 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_password", 2023-12-01T01:22:34.058 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.058 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.058 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.058 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.058 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.058 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.058 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.058 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Password to authenticate with", 2023-12-01T01:22:34.058 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.058 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.058 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.058 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.058 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_port": { 2023-12-01T01:22:34.059 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_port", 2023-12-01T01:22:34.059 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.059 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.059 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.059 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "465", 2023-12-01T01:22:34.059 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.059 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.059 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.059 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SMTP port", 2023-12-01T01:22:34.059 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.059 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.059 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.059 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.059 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_sender": { 2023-12-01T01:22:34.059 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_sender", 2023-12-01T01:22:34.060 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.060 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.060 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.060 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.060 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.060 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.060 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.060 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SMTP envelope sender", 2023-12-01T01:22:34.060 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.060 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.060 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.060 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.060 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_ssl": { 2023-12-01T01:22:34.060 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_ssl", 2023-12-01T01:22:34.060 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.061 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.061 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.061 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:34.061 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.061 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.061 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.061 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use SSL to connect to SMTP server", 2023-12-01T01:22:34.061 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.061 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.061 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.061 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.061 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_user": { 2023-12-01T01:22:34.061 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_user", 2023-12-01T01:22:34.061 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.062 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.062 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.062 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.062 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.062 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.062 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.062 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "User to authenticate as", 2023-12-01T01:22:34.062 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.062 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.062 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.062 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.062 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.062 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.062 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.063 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "balancer", 2023-12-01T01:22:34.063 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:34.063 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:34.063 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:34.063 INFO:teuthology.orchestra.run.smithi077.stdout: "active": { 2023-12-01T01:22:34.063 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "active", 2023-12-01T01:22:34.063 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.063 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.063 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.063 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:34.063 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.063 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.063 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.064 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "automatically balance PGs across cluster", 2023-12-01T01:22:34.064 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.064 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.064 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.064 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.064 INFO:teuthology.orchestra.run.smithi077.stdout: "begin_time": { 2023-12-01T01:22:34.064 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "begin_time", 2023-12-01T01:22:34.064 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.064 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.064 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.064 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0000", 2023-12-01T01:22:34.064 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.064 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.064 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.064 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "beginning time of day to automatically balance", 2023-12-01T01:22:34.065 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-12-01T01:22:34.065 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.065 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.065 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.065 INFO:teuthology.orchestra.run.smithi077.stdout: "begin_weekday": { 2023-12-01T01:22:34.065 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "begin_weekday", 2023-12-01T01:22:34.065 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2023-12-01T01:22:34.065 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.065 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.065 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2023-12-01T01:22:34.065 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "0", 2023-12-01T01:22:34.065 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "6", 2023-12-01T01:22:34.065 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.065 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-12-01T01:22:34.066 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-12-01T01:22:34.066 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.066 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.066 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.066 INFO:teuthology.orchestra.run.smithi077.stdout: "crush_compat_max_iterations": { 2023-12-01T01:22:34.066 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crush_compat_max_iterations", 2023-12-01T01:22:34.066 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2023-12-01T01:22:34.066 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.066 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.066 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "25", 2023-12-01T01:22:34.066 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2023-12-01T01:22:34.066 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "250", 2023-12-01T01:22:34.066 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.066 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-12-01T01:22:34.067 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.067 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.067 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.067 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.067 INFO:teuthology.orchestra.run.smithi077.stdout: "crush_compat_metrics": { 2023-12-01T01:22:34.067 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crush_compat_metrics", 2023-12-01T01:22:34.067 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.067 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.067 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.067 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "pgs,objects,bytes", 2023-12-01T01:22:34.067 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.067 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.067 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.067 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-12-01T01:22:34.067 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-12-01T01:22:34.068 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.068 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.068 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.068 INFO:teuthology.orchestra.run.smithi077.stdout: "crush_compat_step": { 2023-12-01T01:22:34.068 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crush_compat_step", 2023-12-01T01:22:34.068 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2023-12-01T01:22:34.068 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.068 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.068 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0.5", 2023-12-01T01:22:34.068 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "0.001", 2023-12-01T01:22:34.068 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "0.999", 2023-12-01T01:22:34.068 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.068 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "aggressiveness of optimization", 2023-12-01T01:22:34.068 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-12-01T01:22:34.069 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.069 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.069 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.069 INFO:teuthology.orchestra.run.smithi077.stdout: "end_time": { 2023-12-01T01:22:34.069 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "end_time", 2023-12-01T01:22:34.069 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.069 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.069 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.069 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "2359", 2023-12-01T01:22:34.069 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.069 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.069 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.069 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "ending time of day to automatically balance", 2023-12-01T01:22:34.069 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-12-01T01:22:34.069 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.070 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.070 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.070 INFO:teuthology.orchestra.run.smithi077.stdout: "end_weekday": { 2023-12-01T01:22:34.070 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "end_weekday", 2023-12-01T01:22:34.070 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2023-12-01T01:22:34.070 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.070 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.070 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2023-12-01T01:22:34.070 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "0", 2023-12-01T01:22:34.070 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "6", 2023-12-01T01:22:34.070 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.070 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-12-01T01:22:34.070 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-12-01T01:22:34.071 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.071 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.071 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.071 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:34.071 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:34.071 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.071 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.071 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.071 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.071 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.071 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.071 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.071 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.071 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.071 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.072 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.072 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.072 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.072 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.072 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.072 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.072 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.072 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.072 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.072 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:34.072 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:34.072 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.072 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.072 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.072 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.073 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.073 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.073 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.073 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.073 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.073 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.073 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.073 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.073 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:34.073 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:34.073 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.073 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.073 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.073 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:34.074 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.074 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.074 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.074 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.074 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.074 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.074 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.074 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.074 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.074 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.074 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.074 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.074 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.074 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.075 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.075 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:34.075 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:34.075 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.075 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.075 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.075 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.075 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.075 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.075 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.075 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.075 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.075 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.075 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.075 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.076 INFO:teuthology.orchestra.run.smithi077.stdout: "min_score": { 2023-12-01T01:22:34.076 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "min_score", 2023-12-01T01:22:34.076 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2023-12-01T01:22:34.076 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.076 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.076 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2023-12-01T01:22:34.076 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.076 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.076 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.076 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-12-01T01:22:34.076 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.076 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.076 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.076 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.076 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": { 2023-12-01T01:22:34.077 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mode", 2023-12-01T01:22:34.077 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.077 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.077 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.077 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "upmap", 2023-12-01T01:22:34.077 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.077 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.077 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.077 INFO:teuthology.orchestra.run.smithi077.stdout: "crush-compat", 2023-12-01T01:22:34.077 INFO:teuthology.orchestra.run.smithi077.stdout: "none", 2023-12-01T01:22:34.077 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap" 2023-12-01T01:22:34.077 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.077 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Balancer mode", 2023-12-01T01:22:34.077 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.077 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.078 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.078 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.078 INFO:teuthology.orchestra.run.smithi077.stdout: "pool_ids": { 2023-12-01T01:22:34.078 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pool_ids", 2023-12-01T01:22:34.078 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.078 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.078 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.078 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.078 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.078 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.078 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.078 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-12-01T01:22:34.078 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.079 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.079 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.079 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.079 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2023-12-01T01:22:34.079 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2023-12-01T01:22:34.079 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:34.079 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.079 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.079 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2023-12-01T01:22:34.079 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.079 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.079 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.079 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-12-01T01:22:34.079 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.080 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.080 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.080 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.080 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap_max_deviation": { 2023-12-01T01:22:34.080 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "upmap_max_deviation", 2023-12-01T01:22:34.080 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.080 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.080 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.080 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2023-12-01T01:22:34.080 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2023-12-01T01:22:34.080 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.080 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.080 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "deviation below which no optimization is attempted", 2023-12-01T01:22:34.080 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-12-01T01:22:34.080 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.081 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.081 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.081 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap_max_optimizations": { 2023-12-01T01:22:34.081 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "upmap_max_optimizations", 2023-12-01T01:22:34.081 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2023-12-01T01:22:34.081 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.081 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.081 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2023-12-01T01:22:34.081 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.081 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.081 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.081 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-12-01T01:22:34.081 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.081 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.082 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.082 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.082 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.082 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.082 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.082 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cephadm", 2023-12-01T01:22:34.082 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:34.082 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:34.082 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:34.082 INFO:teuthology.orchestra.run.smithi077.stdout: "agent_down_multiplier": { 2023-12-01T01:22:34.082 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "agent_down_multiplier", 2023-12-01T01:22:34.082 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2023-12-01T01:22:34.082 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.082 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.082 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "3.0", 2023-12-01T01:22:34.083 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.083 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.083 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.083 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-12-01T01:22:34.083 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.083 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.083 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.083 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.083 INFO:teuthology.orchestra.run.smithi077.stdout: "agent_refresh_rate": { 2023-12-01T01:22:34.083 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "agent_refresh_rate", 2023-12-01T01:22:34.083 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:34.083 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.083 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.083 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "20", 2023-12-01T01:22:34.084 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.084 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.084 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.084 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-12-01T01:22:34.084 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.084 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.084 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.084 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.084 INFO:teuthology.orchestra.run.smithi077.stdout: "agent_starting_port": { 2023-12-01T01:22:34.084 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "agent_starting_port", 2023-12-01T01:22:34.084 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.084 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.084 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.084 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "4721", 2023-12-01T01:22:34.085 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.085 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.085 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.085 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-12-01T01:22:34.085 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.085 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.085 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.085 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.085 INFO:teuthology.orchestra.run.smithi077.stdout: "allow_ptrace": { 2023-12-01T01:22:34.085 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "allow_ptrace", 2023-12-01T01:22:34.085 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.085 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.085 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.085 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.085 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.086 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.086 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.086 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-12-01T01:22:34.086 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-12-01T01:22:34.086 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.086 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.086 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.086 INFO:teuthology.orchestra.run.smithi077.stdout: "autotune_interval": { 2023-12-01T01:22:34.086 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "autotune_interval", 2023-12-01T01:22:34.086 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:34.086 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.086 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.086 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2023-12-01T01:22:34.087 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.087 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.087 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.087 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to autotune daemon memory", 2023-12-01T01:22:34.087 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.087 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.087 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.087 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.087 INFO:teuthology.orchestra.run.smithi077.stdout: "autotune_memory_target_ratio": { 2023-12-01T01:22:34.087 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "autotune_memory_target_ratio", 2023-12-01T01:22:34.087 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2023-12-01T01:22:34.087 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.087 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.087 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0.7", 2023-12-01T01:22:34.087 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.088 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.088 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.088 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-12-01T01:22:34.088 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.088 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.088 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.088 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.088 INFO:teuthology.orchestra.run.smithi077.stdout: "cgroups_split": { 2023-12-01T01:22:34.088 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cgroups_split", 2023-12-01T01:22:34.088 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.088 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.088 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.088 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:34.088 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.089 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.089 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.089 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-12-01T01:22:34.089 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.089 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.089 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.089 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.089 INFO:teuthology.orchestra.run.smithi077.stdout: "config_checks_enabled": { 2023-12-01T01:22:34.089 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "config_checks_enabled", 2023-12-01T01:22:34.089 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.089 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.089 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.089 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.089 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.090 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.090 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.090 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-12-01T01:22:34.090 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.090 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.090 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.090 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.090 INFO:teuthology.orchestra.run.smithi077.stdout: "config_dashboard": { 2023-12-01T01:22:34.090 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "config_dashboard", 2023-12-01T01:22:34.090 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.090 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.090 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.090 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:34.090 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.091 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.091 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.091 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-12-01T01:22:34.091 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.091 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.091 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.091 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.091 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_alertmanager": { 2023-12-01T01:22:34.091 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_alertmanager", 2023-12-01T01:22:34.091 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.091 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.091 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.091 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-12-01T01:22:34.091 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.091 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.092 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.092 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2023-12-01T01:22:34.092 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.092 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.092 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.092 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.092 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_base": { 2023-12-01T01:22:34.092 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_base", 2023-12-01T01:22:34.092 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.092 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.092 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.092 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/ceph", 2023-12-01T01:22:34.092 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.093 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.093 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.093 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Container image name, without the tag", 2023-12-01T01:22:34.093 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.093 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.093 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.093 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.093 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_elasticsearch": { 2023-12-01T01:22:34.093 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_elasticsearch", 2023-12-01T01:22:34.093 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.093 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.093 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.093 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-12-01T01:22:34.093 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.093 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.094 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.094 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "elasticsearch container image", 2023-12-01T01:22:34.094 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.094 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.094 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.094 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.094 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_grafana": { 2023-12-01T01:22:34.094 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_grafana", 2023-12-01T01:22:34.094 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.094 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.094 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.094 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-12-01T01:22:34.094 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.094 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.095 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.095 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2023-12-01T01:22:34.095 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.095 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.095 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.095 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.095 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_haproxy": { 2023-12-01T01:22:34.095 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_haproxy", 2023-12-01T01:22:34.095 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.095 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.095 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.095 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-12-01T01:22:34.095 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.095 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.096 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.096 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "HAproxy container image", 2023-12-01T01:22:34.096 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.096 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.096 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.096 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.096 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_jaeger_agent": { 2023-12-01T01:22:34.096 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_jaeger_agent", 2023-12-01T01:22:34.096 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.096 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.096 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.096 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-12-01T01:22:34.096 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.096 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.096 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.097 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Jaeger agent container image", 2023-12-01T01:22:34.097 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.097 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.097 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.097 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.097 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_jaeger_collector": { 2023-12-01T01:22:34.097 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_jaeger_collector", 2023-12-01T01:22:34.097 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.097 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.097 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.097 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-12-01T01:22:34.097 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.097 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.097 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.098 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Jaeger collector container image", 2023-12-01T01:22:34.098 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.098 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.098 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.098 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.098 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_jaeger_query": { 2023-12-01T01:22:34.098 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_jaeger_query", 2023-12-01T01:22:34.098 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.098 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.098 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.098 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-12-01T01:22:34.098 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.098 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.099 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.099 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Jaeger query container image", 2023-12-01T01:22:34.099 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.099 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.099 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.099 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.099 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_keepalived": { 2023-12-01T01:22:34.099 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_keepalived", 2023-12-01T01:22:34.099 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.099 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.099 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.099 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-12-01T01:22:34.099 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.099 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.100 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.100 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Keepalived container image", 2023-12-01T01:22:34.100 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.100 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.100 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.100 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.100 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_loki": { 2023-12-01T01:22:34.100 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_loki", 2023-12-01T01:22:34.100 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.100 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.100 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.100 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-12-01T01:22:34.100 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.100 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.100 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.101 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Loki container image", 2023-12-01T01:22:34.101 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.101 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.101 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.101 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.101 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_node_exporter": { 2023-12-01T01:22:34.101 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_node_exporter", 2023-12-01T01:22:34.101 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.101 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.101 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.101 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-12-01T01:22:34.101 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.101 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.101 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.101 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2023-12-01T01:22:34.102 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.102 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.102 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.102 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.102 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_nvmeof": { 2023-12-01T01:22:34.102 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_nvmeof", 2023-12-01T01:22:34.102 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.102 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.102 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.102 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.2", 2023-12-01T01:22:34.102 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.102 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.102 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.102 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Nvme-of container image", 2023-12-01T01:22:34.103 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.103 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.103 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.103 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.103 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_prometheus": { 2023-12-01T01:22:34.103 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_prometheus", 2023-12-01T01:22:34.103 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.103 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.103 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.103 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-12-01T01:22:34.103 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.103 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.103 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.103 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2023-12-01T01:22:34.103 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.104 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.104 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.104 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.104 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_promtail": { 2023-12-01T01:22:34.104 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_promtail", 2023-12-01T01:22:34.104 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.104 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.104 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.104 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-12-01T01:22:34.104 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.104 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.104 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.104 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Promtail container image", 2023-12-01T01:22:34.104 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.105 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.105 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.105 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.105 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_snmp_gateway": { 2023-12-01T01:22:34.105 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_snmp_gateway", 2023-12-01T01:22:34.105 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.105 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.105 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.105 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-12-01T01:22:34.105 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.105 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.105 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.105 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SNMP Gateway container image", 2023-12-01T01:22:34.105 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.105 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.106 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.106 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.106 INFO:teuthology.orchestra.run.smithi077.stdout: "container_init": { 2023-12-01T01:22:34.106 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_init", 2023-12-01T01:22:34.106 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.106 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.106 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.106 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:34.106 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.106 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.106 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.106 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Run podman/docker with `--init`", 2023-12-01T01:22:34.106 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.106 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.107 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.107 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.107 INFO:teuthology.orchestra.run.smithi077.stdout: "daemon_cache_timeout": { 2023-12-01T01:22:34.107 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "daemon_cache_timeout", 2023-12-01T01:22:34.107 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:34.107 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.107 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.107 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2023-12-01T01:22:34.107 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.107 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.107 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.107 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-12-01T01:22:34.107 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.107 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.108 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.108 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.108 INFO:teuthology.orchestra.run.smithi077.stdout: "default_cephadm_command_timeout": { 2023-12-01T01:22:34.108 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "default_cephadm_command_timeout", 2023-12-01T01:22:34.108 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:34.108 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.108 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.108 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "900", 2023-12-01T01:22:34.108 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.108 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.108 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.108 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-12-01T01:22:34.108 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.108 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.109 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.109 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.109 INFO:teuthology.orchestra.run.smithi077.stdout: "default_registry": { 2023-12-01T01:22:34.109 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "default_registry", 2023-12-01T01:22:34.109 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.109 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.109 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.109 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io", 2023-12-01T01:22:34.109 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.109 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.109 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.109 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-12-01T01:22:34.109 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.109 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.109 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.110 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.110 INFO:teuthology.orchestra.run.smithi077.stdout: "device_cache_timeout": { 2023-12-01T01:22:34.110 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "device_cache_timeout", 2023-12-01T01:22:34.110 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:34.110 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.110 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.110 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "1800", 2023-12-01T01:22:34.110 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.110 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.110 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.110 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "seconds to cache device inventory", 2023-12-01T01:22:34.110 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.110 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.110 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.111 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.111 INFO:teuthology.orchestra.run.smithi077.stdout: "device_enhanced_scan": { 2023-12-01T01:22:34.111 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "device_enhanced_scan", 2023-12-01T01:22:34.111 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.111 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.111 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.111 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.111 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.111 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.111 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.111 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use libstoragemgmt during device scans", 2023-12-01T01:22:34.111 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.111 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.111 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.112 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.112 INFO:teuthology.orchestra.run.smithi077.stdout: "facts_cache_timeout": { 2023-12-01T01:22:34.112 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "facts_cache_timeout", 2023-12-01T01:22:34.112 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:34.112 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.112 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.112 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2023-12-01T01:22:34.112 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.112 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.112 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.112 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "seconds to cache host facts data", 2023-12-01T01:22:34.112 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.112 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.112 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.113 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.113 INFO:teuthology.orchestra.run.smithi077.stdout: "host_check_interval": { 2023-12-01T01:22:34.113 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "host_check_interval", 2023-12-01T01:22:34.113 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:34.113 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.113 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.113 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2023-12-01T01:22:34.113 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.113 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.113 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.113 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to perform a host check", 2023-12-01T01:22:34.113 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.113 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.113 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.113 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.114 INFO:teuthology.orchestra.run.smithi077.stdout: "inventory_list_all": { 2023-12-01T01:22:34.114 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "inventory_list_all", 2023-12-01T01:22:34.114 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.114 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.114 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.114 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.114 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.114 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.114 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.114 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-12-01T01:22:34.114 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.114 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.114 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.114 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.115 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:34.115 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:34.115 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.115 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.115 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.115 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.115 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.115 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.115 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.115 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.115 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.115 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.115 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.115 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.115 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.116 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.116 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.116 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.116 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.116 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.116 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.116 INFO:teuthology.orchestra.run.smithi077.stdout: "log_refresh_metadata": { 2023-12-01T01:22:34.116 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_refresh_metadata", 2023-12-01T01:22:34.116 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.116 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.116 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.116 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.116 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.116 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.116 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.117 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-12-01T01:22:34.117 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.117 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.117 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.117 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.117 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:34.117 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:34.117 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.117 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.117 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.117 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:34.117 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.117 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.117 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.118 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-12-01T01:22:34.118 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.118 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.118 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.118 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.118 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:34.118 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:34.118 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.118 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.118 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.118 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:34.118 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.118 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.118 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.118 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.119 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.119 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.119 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.119 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.119 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.119 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.119 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.119 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.119 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.119 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.119 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.119 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:34.119 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:34.119 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.119 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.120 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.120 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.120 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.120 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.120 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.120 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.120 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.120 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.120 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.120 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.120 INFO:teuthology.orchestra.run.smithi077.stdout: "manage_etc_ceph_ceph_conf": { 2023-12-01T01:22:34.120 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-12-01T01:22:34.120 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.120 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.120 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.121 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.121 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.121 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.121 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.121 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-12-01T01:22:34.121 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.121 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.121 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.121 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.121 INFO:teuthology.orchestra.run.smithi077.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-12-01T01:22:34.121 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-12-01T01:22:34.121 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.121 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.122 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.122 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "*", 2023-12-01T01:22:34.122 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.122 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.122 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.122 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-12-01T01:22:34.122 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.122 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.122 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.122 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.122 INFO:teuthology.orchestra.run.smithi077.stdout: "max_count_per_host": { 2023-12-01T01:22:34.122 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_count_per_host", 2023-12-01T01:22:34.122 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.123 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.123 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.123 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2023-12-01T01:22:34.123 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.123 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.123 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.123 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "max number of daemons per service per host", 2023-12-01T01:22:34.123 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.123 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.123 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.123 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.123 INFO:teuthology.orchestra.run.smithi077.stdout: "max_osd_draining_count": { 2023-12-01T01:22:34.123 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_osd_draining_count", 2023-12-01T01:22:34.123 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.123 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.124 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.124 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2023-12-01T01:22:34.124 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.124 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.124 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.124 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-12-01T01:22:34.124 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.124 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.124 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.124 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.124 INFO:teuthology.orchestra.run.smithi077.stdout: "migration_current": { 2023-12-01T01:22:34.124 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "migration_current", 2023-12-01T01:22:34.124 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.124 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.124 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.125 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.125 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.125 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.125 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.125 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "internal - do not modify", 2023-12-01T01:22:34.125 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.125 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.125 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.125 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.125 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": { 2023-12-01T01:22:34.125 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mode", 2023-12-01T01:22:34.125 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.125 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.125 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.126 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "root", 2023-12-01T01:22:34.126 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.126 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.126 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.126 INFO:teuthology.orchestra.run.smithi077.stdout: "cephadm-package", 2023-12-01T01:22:34.126 INFO:teuthology.orchestra.run.smithi077.stdout: "root" 2023-12-01T01:22:34.126 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.126 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "mode for remote execution of cephadm", 2023-12-01T01:22:34.126 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.126 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.126 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.126 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.126 INFO:teuthology.orchestra.run.smithi077.stdout: "prometheus_alerts_path": { 2023-12-01T01:22:34.127 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "prometheus_alerts_path", 2023-12-01T01:22:34.127 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.127 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.127 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.127 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-12-01T01:22:34.127 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.127 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.127 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.127 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-12-01T01:22:34.127 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.127 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.127 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.127 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.127 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_insecure": { 2023-12-01T01:22:34.128 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_insecure", 2023-12-01T01:22:34.128 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.128 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.128 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.128 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.128 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.128 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.128 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.128 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-12-01T01:22:34.128 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.128 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.128 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.128 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.128 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_password": { 2023-12-01T01:22:34.128 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_password", 2023-12-01T01:22:34.129 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.129 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.129 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.129 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.129 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.129 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.129 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.129 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-12-01T01:22:34.129 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.129 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.129 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.129 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.129 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_url": { 2023-12-01T01:22:34.129 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_url", 2023-12-01T01:22:34.129 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.130 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.130 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.130 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.130 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.130 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.130 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.130 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-12-01T01:22:34.130 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.130 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.130 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.130 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.130 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_username": { 2023-12-01T01:22:34.130 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_username", 2023-12-01T01:22:34.130 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.131 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.131 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.131 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.131 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.131 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.131 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.131 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-12-01T01:22:34.131 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.131 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.131 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.131 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.131 INFO:teuthology.orchestra.run.smithi077.stdout: "secure_monitoring_stack": { 2023-12-01T01:22:34.131 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "secure_monitoring_stack", 2023-12-01T01:22:34.131 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.132 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.132 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.132 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.132 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.132 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.132 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.132 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-12-01T01:22:34.132 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.132 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.132 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.132 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.132 INFO:teuthology.orchestra.run.smithi077.stdout: "service_discovery_port": { 2023-12-01T01:22:34.132 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "service_discovery_port", 2023-12-01T01:22:34.132 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.132 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.133 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.133 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8765", 2023-12-01T01:22:34.133 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.133 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.133 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.133 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "cephadm service discovery port", 2023-12-01T01:22:34.133 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.133 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.133 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.133 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.133 INFO:teuthology.orchestra.run.smithi077.stdout: "ssh_config_file": { 2023-12-01T01:22:34.133 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssh_config_file", 2023-12-01T01:22:34.133 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.133 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.133 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.134 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.134 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.134 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.134 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.134 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-12-01T01:22:34.134 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.134 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.134 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.134 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.134 INFO:teuthology.orchestra.run.smithi077.stdout: "use_agent": { 2023-12-01T01:22:34.134 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "use_agent", 2023-12-01T01:22:34.134 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.134 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.134 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.135 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.135 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.135 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.135 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.135 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-12-01T01:22:34.135 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.135 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.135 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.135 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.135 INFO:teuthology.orchestra.run.smithi077.stdout: "use_repo_digest": { 2023-12-01T01:22:34.135 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "use_repo_digest", 2023-12-01T01:22:34.135 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.135 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.135 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.136 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:34.136 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.136 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.136 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.136 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-12-01T01:22:34.136 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.136 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.136 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.136 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.136 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_on_failed_host_check": { 2023-12-01T01:22:34.136 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_on_failed_host_check", 2023-12-01T01:22:34.136 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.136 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.136 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.137 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:34.137 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.137 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.137 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.137 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise a health warning if the host check fails", 2023-12-01T01:22:34.137 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.137 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.137 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.137 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.137 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_on_stray_daemons": { 2023-12-01T01:22:34.137 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_on_stray_daemons", 2023-12-01T01:22:34.137 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.137 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.137 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.138 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:34.138 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.138 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.138 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.138 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-12-01T01:22:34.138 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.138 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.138 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.138 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.138 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_on_stray_hosts": { 2023-12-01T01:22:34.138 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_on_stray_hosts", 2023-12-01T01:22:34.138 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.138 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.138 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.139 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:34.139 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.139 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.139 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.139 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-12-01T01:22:34.139 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.139 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.139 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.139 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.139 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.139 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.139 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.139 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crash", 2023-12-01T01:22:34.139 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:34.139 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:34.140 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:34.140 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:34.140 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:34.140 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.140 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.140 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.140 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.140 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.140 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.140 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.140 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.140 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.140 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.140 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.141 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.141 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.141 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.141 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.141 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.141 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.141 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.141 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.141 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:34.141 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:34.141 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.141 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.141 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.141 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.141 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.142 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.142 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.142 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.142 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.142 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.142 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.142 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.142 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:34.142 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:34.142 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.142 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.142 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.142 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:34.142 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.143 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.143 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.143 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.143 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.143 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.143 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.143 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.143 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.143 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.143 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.143 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.143 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.143 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.143 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.143 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:34.144 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:34.144 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.144 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.144 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.144 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.144 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.144 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.144 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.144 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.144 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.144 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.144 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.144 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.144 INFO:teuthology.orchestra.run.smithi077.stdout: "retain_interval": { 2023-12-01T01:22:34.145 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "retain_interval", 2023-12-01T01:22:34.145 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:34.145 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.145 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.145 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "31536000", 2023-12-01T01:22:34.145 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.145 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.145 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.145 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how long to retain crashes before pruning them", 2023-12-01T01:22:34.145 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.145 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.145 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.145 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.145 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_recent_interval": { 2023-12-01T01:22:34.145 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_recent_interval", 2023-12-01T01:22:34.146 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:34.146 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.146 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.146 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "1209600", 2023-12-01T01:22:34.146 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.146 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.146 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.146 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "time interval in which to warn about recent crashes", 2023-12-01T01:22:34.146 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.146 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.146 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.146 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.146 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.146 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.147 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.147 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "dashboard", 2023-12-01T01:22:34.147 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:34.147 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:34.147 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:34.147 INFO:teuthology.orchestra.run.smithi077.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-12-01T01:22:34.147 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-12-01T01:22:34.147 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.147 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.147 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.147 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2023-12-01T01:22:34.147 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.147 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.147 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.147 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.148 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.148 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.148 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.148 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.148 INFO:teuthology.orchestra.run.smithi077.stdout: "ALERTMANAGER_API_HOST": { 2023-12-01T01:22:34.148 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ALERTMANAGER_API_HOST", 2023-12-01T01:22:34.148 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.148 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.148 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.148 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.148 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.148 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.148 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.148 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.149 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.149 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.149 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.149 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.149 INFO:teuthology.orchestra.run.smithi077.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-12-01T01:22:34.149 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-12-01T01:22:34.149 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.149 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.149 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.149 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:34.149 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.149 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.149 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.149 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.149 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.150 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.150 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.150 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.150 INFO:teuthology.orchestra.run.smithi077.stdout: "AUDIT_API_ENABLED": { 2023-12-01T01:22:34.150 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "AUDIT_API_ENABLED", 2023-12-01T01:22:34.150 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.150 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.150 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.150 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.150 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.150 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.150 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.150 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.150 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.151 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.151 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.151 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.151 INFO:teuthology.orchestra.run.smithi077.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-12-01T01:22:34.151 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-12-01T01:22:34.151 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.151 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.151 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.151 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:34.151 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.151 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.151 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.151 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.151 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.152 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.152 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.152 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.152 INFO:teuthology.orchestra.run.smithi077.stdout: "ENABLE_BROWSABLE_API": { 2023-12-01T01:22:34.152 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ENABLE_BROWSABLE_API", 2023-12-01T01:22:34.152 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.152 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.152 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.152 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:34.152 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.152 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.152 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.152 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.152 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.153 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.153 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.153 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.153 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-12-01T01:22:34.153 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-12-01T01:22:34.153 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.153 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.153 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.153 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:34.153 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.153 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.153 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.153 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.153 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.153 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.154 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.154 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.154 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-12-01T01:22:34.154 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-12-01T01:22:34.154 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.154 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.154 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.154 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:34.154 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.154 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.154 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.154 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.154 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.154 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.155 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.155 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.155 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-12-01T01:22:34.155 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-12-01T01:22:34.155 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.155 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.155 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.155 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:34.155 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.155 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.155 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.155 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.155 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.155 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.156 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.156 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.156 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-12-01T01:22:34.156 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-12-01T01:22:34.156 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.156 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.156 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.156 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:34.156 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.156 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.156 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.156 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.156 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.156 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.156 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.157 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.157 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_NFS": { 2023-12-01T01:22:34.157 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-12-01T01:22:34.157 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.157 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.157 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.157 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:34.157 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.157 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.157 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.157 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.157 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.157 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.157 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.157 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.158 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_RBD": { 2023-12-01T01:22:34.158 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-12-01T01:22:34.158 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.158 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.158 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.158 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:34.158 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.158 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.158 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.158 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.158 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.158 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.158 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.158 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.159 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_RGW": { 2023-12-01T01:22:34.159 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-12-01T01:22:34.159 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.159 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.159 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.159 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:34.159 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.159 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.159 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.159 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.159 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.159 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.159 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.159 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.159 INFO:teuthology.orchestra.run.smithi077.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-12-01T01:22:34.160 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-12-01T01:22:34.160 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.160 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.160 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.160 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.160 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.160 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.160 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.160 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.160 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.160 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.160 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.160 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.160 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_PASSWORD": { 2023-12-01T01:22:34.161 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_PASSWORD", 2023-12-01T01:22:34.161 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.161 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.161 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.161 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "admin", 2023-12-01T01:22:34.161 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.161 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.161 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.161 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.161 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.161 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.161 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.161 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.161 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-12-01T01:22:34.161 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-12-01T01:22:34.162 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.162 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.162 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.162 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:34.162 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.162 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.162 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.162 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.162 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.162 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.162 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.162 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.162 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_URL": { 2023-12-01T01:22:34.162 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_URL", 2023-12-01T01:22:34.163 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.163 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.163 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.163 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.163 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.163 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.163 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.163 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.163 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.163 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.163 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.163 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.163 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_USERNAME": { 2023-12-01T01:22:34.163 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_USERNAME", 2023-12-01T01:22:34.163 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.164 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.164 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.164 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "admin", 2023-12-01T01:22:34.164 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.164 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.164 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.164 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.164 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.164 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.164 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.164 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.164 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-12-01T01:22:34.164 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-12-01T01:22:34.164 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.164 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.165 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.165 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.165 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.165 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.165 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.165 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.165 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.165 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.165 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.165 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.165 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-12-01T01:22:34.165 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-12-01T01:22:34.165 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.165 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.166 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.166 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.166 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.166 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.166 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.166 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.166 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.166 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.166 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.166 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.166 INFO:teuthology.orchestra.run.smithi077.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-12-01T01:22:34.166 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-12-01T01:22:34.166 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.166 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.166 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.167 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:34.167 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.167 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.167 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.167 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.167 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.167 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.167 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.167 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.167 INFO:teuthology.orchestra.run.smithi077.stdout: "ISSUE_TRACKER_API_KEY": { 2023-12-01T01:22:34.167 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-12-01T01:22:34.167 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.167 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.168 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.168 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.168 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.168 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.168 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.168 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.168 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.168 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.168 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.168 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.168 INFO:teuthology.orchestra.run.smithi077.stdout: "PROMETHEUS_API_HOST": { 2023-12-01T01:22:34.168 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PROMETHEUS_API_HOST", 2023-12-01T01:22:34.168 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.168 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.169 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.169 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.169 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.169 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.169 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.169 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.169 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.169 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.169 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.169 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.169 INFO:teuthology.orchestra.run.smithi077.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-12-01T01:22:34.169 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-12-01T01:22:34.169 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.169 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.170 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.170 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:34.170 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.170 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.170 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.170 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.170 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.170 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.170 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.170 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.170 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-12-01T01:22:34.170 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-12-01T01:22:34.170 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.171 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.171 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.171 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.171 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.171 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.171 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.171 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.171 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.171 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.171 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.171 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.171 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-12-01T01:22:34.171 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-12-01T01:22:34.171 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.172 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.172 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.172 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.172 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.172 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.172 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.172 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.172 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.172 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.172 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.172 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.172 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-12-01T01:22:34.172 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-12-01T01:22:34.172 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.173 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.173 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.173 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:34.173 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.173 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.173 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.173 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.173 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.173 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.173 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.173 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.173 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-12-01T01:22:34.173 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-12-01T01:22:34.173 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.173 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.174 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.174 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:34.174 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.174 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.174 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.174 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.174 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.174 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.174 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.174 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.174 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-12-01T01:22:34.174 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-12-01T01:22:34.174 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.174 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.175 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.175 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.175 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.175 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.175 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.175 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.175 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.175 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.175 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.175 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.175 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-12-01T01:22:34.175 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-12-01T01:22:34.175 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.175 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.176 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.176 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.176 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.176 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.176 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.176 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.176 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.176 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.176 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.176 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.176 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-12-01T01:22:34.176 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-12-01T01:22:34.176 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.176 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.176 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.177 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.177 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.177 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.177 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.177 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.177 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.177 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.177 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.177 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.177 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_ENABLED": { 2023-12-01T01:22:34.177 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_ENABLED", 2023-12-01T01:22:34.177 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.177 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.177 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.177 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:34.178 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.178 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.178 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.178 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.178 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.178 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.178 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.178 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.178 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-12-01T01:22:34.178 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-12-01T01:22:34.178 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.178 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.178 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.178 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-12-01T01:22:34.179 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.179 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.179 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.179 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.179 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.179 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.179 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.179 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.179 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-12-01T01:22:34.179 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-12-01T01:22:34.179 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.179 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.179 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.179 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2023-12-01T01:22:34.179 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.180 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.180 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.180 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.180 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.180 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.180 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.180 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.180 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-12-01T01:22:34.180 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-12-01T01:22:34.180 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.180 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.180 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.180 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8", 2023-12-01T01:22:34.180 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.181 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.181 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.181 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.181 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.181 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.181 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.181 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.181 INFO:teuthology.orchestra.run.smithi077.stdout: "REST_REQUESTS_TIMEOUT": { 2023-12-01T01:22:34.181 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-12-01T01:22:34.181 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.181 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.181 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.181 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "45", 2023-12-01T01:22:34.181 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.181 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.182 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.182 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.182 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.182 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.182 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.182 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.182 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_ACCESS_KEY": { 2023-12-01T01:22:34.182 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_ACCESS_KEY", 2023-12-01T01:22:34.182 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.182 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.182 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.182 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.182 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.183 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.183 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.183 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.183 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.183 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.183 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.183 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.183 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-12-01T01:22:34.183 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-12-01T01:22:34.183 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.183 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.183 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.183 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "admin", 2023-12-01T01:22:34.183 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.183 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.184 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.184 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.184 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.184 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.184 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.184 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.184 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_SECRET_KEY": { 2023-12-01T01:22:34.184 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_SECRET_KEY", 2023-12-01T01:22:34.184 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.184 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.184 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.184 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.184 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.184 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.184 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.185 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.185 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.185 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.185 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.185 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.185 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_SSL_VERIFY": { 2023-12-01T01:22:34.185 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_SSL_VERIFY", 2023-12-01T01:22:34.185 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.185 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.185 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.185 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:34.185 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.185 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.185 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.186 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.186 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.186 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.186 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.186 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.186 INFO:teuthology.orchestra.run.smithi077.stdout: "UNSAFE_TLS_v1_2": { 2023-12-01T01:22:34.186 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "UNSAFE_TLS_v1_2", 2023-12-01T01:22:34.186 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.186 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.186 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.186 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.186 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.186 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.186 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.187 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.187 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.187 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.187 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.187 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.187 INFO:teuthology.orchestra.run.smithi077.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-12-01T01:22:34.187 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-12-01T01:22:34.187 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.187 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.187 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.187 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2023-12-01T01:22:34.187 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.187 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.187 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.188 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.188 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.188 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.188 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.188 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.188 INFO:teuthology.orchestra.run.smithi077.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-12-01T01:22:34.188 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-12-01T01:22:34.188 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.188 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.188 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.188 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2023-12-01T01:22:34.188 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.188 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.188 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.188 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.189 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.189 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.189 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.189 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.189 INFO:teuthology.orchestra.run.smithi077.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-12-01T01:22:34.189 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-12-01T01:22:34.189 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.189 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.189 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.189 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2023-12-01T01:22:34.189 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.189 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.189 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.189 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.189 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.190 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.190 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.190 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.190 INFO:teuthology.orchestra.run.smithi077.stdout: "cross_origin_url": { 2023-12-01T01:22:34.190 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cross_origin_url", 2023-12-01T01:22:34.190 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.190 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.190 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.190 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.190 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.190 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.190 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.190 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.190 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.191 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.191 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.191 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.191 INFO:teuthology.orchestra.run.smithi077.stdout: "crt_file": { 2023-12-01T01:22:34.191 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crt_file", 2023-12-01T01:22:34.191 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.191 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.191 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.191 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.191 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.191 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.191 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.191 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.191 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.191 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.192 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.192 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.192 INFO:teuthology.orchestra.run.smithi077.stdout: "debug": { 2023-12-01T01:22:34.192 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "debug", 2023-12-01T01:22:34.192 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.192 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.192 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.192 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.192 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.192 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.192 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.192 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Enable/disable debug options", 2023-12-01T01:22:34.192 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.192 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.193 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.193 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.193 INFO:teuthology.orchestra.run.smithi077.stdout: "jwt_token_ttl": { 2023-12-01T01:22:34.193 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "jwt_token_ttl", 2023-12-01T01:22:34.193 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.193 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.193 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.193 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "28800", 2023-12-01T01:22:34.193 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.193 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.193 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.193 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.193 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.193 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.194 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.194 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.194 INFO:teuthology.orchestra.run.smithi077.stdout: "key_file": { 2023-12-01T01:22:34.194 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "key_file", 2023-12-01T01:22:34.194 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.194 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.194 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.194 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.194 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.194 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.194 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.194 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.194 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.194 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.194 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.195 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.195 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:34.195 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:34.195 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.195 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.195 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.195 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.195 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.195 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.195 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.195 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.195 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.195 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.196 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.196 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.196 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.196 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.196 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.196 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.196 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.196 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.196 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.196 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:34.196 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:34.196 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.196 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.196 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.196 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.197 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.197 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.197 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.197 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.197 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.197 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.197 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.197 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.197 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:34.197 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:34.197 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.197 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.197 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.197 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:34.197 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.198 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.198 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.198 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.198 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.198 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.198 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.198 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.198 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.198 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.198 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.198 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.198 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.198 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.198 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.199 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:34.199 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:34.199 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.199 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.199 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.199 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.199 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.199 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.199 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.199 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.199 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.199 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.199 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.199 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.200 INFO:teuthology.orchestra.run.smithi077.stdout: "motd": { 2023-12-01T01:22:34.200 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "motd", 2023-12-01T01:22:34.200 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.200 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.200 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.200 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.200 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.200 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.200 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.200 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "The message of the day", 2023-12-01T01:22:34.200 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.200 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.200 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.200 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.201 INFO:teuthology.orchestra.run.smithi077.stdout: "redirect_resolve_ip_addr": { 2023-12-01T01:22:34.201 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "redirect_resolve_ip_addr", 2023-12-01T01:22:34.201 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.201 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.201 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.201 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.201 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.201 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.201 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.201 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.201 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.201 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.201 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.201 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.202 INFO:teuthology.orchestra.run.smithi077.stdout: "server_addr": { 2023-12-01T01:22:34.202 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_addr", 2023-12-01T01:22:34.202 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.202 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.202 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.202 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "::", 2023-12-01T01:22:34.202 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.202 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.202 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.202 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.202 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.202 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.202 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.202 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.202 INFO:teuthology.orchestra.run.smithi077.stdout: "server_port": { 2023-12-01T01:22:34.203 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_port", 2023-12-01T01:22:34.203 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.203 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.203 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.203 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8080", 2023-12-01T01:22:34.203 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.203 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.203 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.203 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.203 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.203 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.203 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.203 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.203 INFO:teuthology.orchestra.run.smithi077.stdout: "ssl": { 2023-12-01T01:22:34.204 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssl", 2023-12-01T01:22:34.204 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.204 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.204 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.204 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:34.204 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.204 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.204 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.204 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.204 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.204 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.204 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.204 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.204 INFO:teuthology.orchestra.run.smithi077.stdout: "ssl_server_port": { 2023-12-01T01:22:34.204 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssl_server_port", 2023-12-01T01:22:34.205 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.205 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.205 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.205 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8443", 2023-12-01T01:22:34.205 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.205 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.205 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.205 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.205 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.205 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.205 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.205 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.205 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_behaviour": { 2023-12-01T01:22:34.205 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_behaviour", 2023-12-01T01:22:34.206 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.206 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.206 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.206 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "redirect", 2023-12-01T01:22:34.206 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.206 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.206 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.206 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.206 INFO:teuthology.orchestra.run.smithi077.stdout: "redirect" 2023-12-01T01:22:34.206 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.206 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.206 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.206 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.206 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.207 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.207 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_error_status_code": { 2023-12-01T01:22:34.207 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_error_status_code", 2023-12-01T01:22:34.207 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.207 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.207 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.207 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "500", 2023-12-01T01:22:34.207 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "400", 2023-12-01T01:22:34.207 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "599", 2023-12-01T01:22:34.207 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.207 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.207 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.207 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.209 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.209 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.209 INFO:teuthology.orchestra.run.smithi077.stdout: "url_prefix": { 2023-12-01T01:22:34.209 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "url_prefix", 2023-12-01T01:22:34.209 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.209 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.209 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.209 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.209 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.209 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.209 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.210 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.210 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.210 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.210 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.210 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.210 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.210 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.210 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.210 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "devicehealth", 2023-12-01T01:22:34.210 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:34.210 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:34.210 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:34.210 INFO:teuthology.orchestra.run.smithi077.stdout: "enable_monitoring": { 2023-12-01T01:22:34.210 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enable_monitoring", 2023-12-01T01:22:34.211 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.211 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.211 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.211 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:34.211 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.211 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.211 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.211 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "monitor device health metrics", 2023-12-01T01:22:34.211 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.211 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.211 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.211 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.211 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:34.211 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:34.212 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.212 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.212 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.212 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.212 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.212 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.212 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.212 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.212 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.212 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.212 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.212 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.212 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.212 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.212 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.213 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.213 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.213 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.213 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.213 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:34.213 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:34.213 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.213 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.213 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.213 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.213 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.213 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.213 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.213 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.213 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.214 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.214 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.214 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.214 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:34.214 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:34.214 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.214 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.214 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.214 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:34.214 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.214 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.214 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.214 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.214 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.214 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.215 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.215 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.215 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.215 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.215 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.215 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.215 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.215 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.215 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.215 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:34.215 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:34.215 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.215 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.215 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.215 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.216 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.216 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.216 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.216 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.216 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.216 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.216 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.216 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.216 INFO:teuthology.orchestra.run.smithi077.stdout: "mark_out_threshold": { 2023-12-01T01:22:34.216 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mark_out_threshold", 2023-12-01T01:22:34.216 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:34.216 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.216 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.216 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "2419200", 2023-12-01T01:22:34.217 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.217 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.217 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.217 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-12-01T01:22:34.217 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.217 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.217 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.217 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.217 INFO:teuthology.orchestra.run.smithi077.stdout: "pool_name": { 2023-12-01T01:22:34.217 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pool_name", 2023-12-01T01:22:34.217 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.217 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.217 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.217 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "device_health_metrics", 2023-12-01T01:22:34.218 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.218 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.218 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.218 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "name of pool in which to store device health metrics", 2023-12-01T01:22:34.218 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.218 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.218 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.218 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.218 INFO:teuthology.orchestra.run.smithi077.stdout: "retention_period": { 2023-12-01T01:22:34.218 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "retention_period", 2023-12-01T01:22:34.218 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:34.218 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.218 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.218 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "15552000", 2023-12-01T01:22:34.219 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.219 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.219 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.219 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how long to retain device health metrics", 2023-12-01T01:22:34.219 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.219 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.219 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.219 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.219 INFO:teuthology.orchestra.run.smithi077.stdout: "scrape_frequency": { 2023-12-01T01:22:34.219 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "scrape_frequency", 2023-12-01T01:22:34.219 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:34.219 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.219 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.219 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "86400", 2023-12-01T01:22:34.219 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.220 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.220 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.220 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to scrape device health metrics", 2023-12-01T01:22:34.220 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.220 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.220 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.220 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.220 INFO:teuthology.orchestra.run.smithi077.stdout: "self_heal": { 2023-12-01T01:22:34.220 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "self_heal", 2023-12-01T01:22:34.220 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.220 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.220 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.220 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:34.220 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.221 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.221 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.221 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-12-01T01:22:34.221 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.221 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.221 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.221 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.221 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2023-12-01T01:22:34.221 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2023-12-01T01:22:34.221 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:34.221 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.221 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.221 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2023-12-01T01:22:34.221 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.222 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.222 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.222 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to wake up and check device health", 2023-12-01T01:22:34.222 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.222 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.222 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.222 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.222 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_threshold": { 2023-12-01T01:22:34.222 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_threshold", 2023-12-01T01:22:34.222 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:34.222 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.222 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.222 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "7257600", 2023-12-01T01:22:34.222 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.223 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.223 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.223 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-12-01T01:22:34.223 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.223 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.223 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.223 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.223 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.223 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.223 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.223 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "diskprediction_local", 2023-12-01T01:22:34.223 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:34.223 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:34.223 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:34.223 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:34.224 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:34.224 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.224 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.224 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.224 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.224 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.224 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.224 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.224 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.224 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.224 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.224 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.224 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.224 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.224 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.225 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.225 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.225 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.225 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.225 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.225 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:34.225 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:34.225 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.225 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.225 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.225 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.225 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.225 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.225 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.226 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.226 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.226 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.226 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.226 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.226 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:34.226 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:34.226 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.226 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.226 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.226 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:34.226 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.226 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.226 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.226 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.227 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.227 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.227 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.227 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.227 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.227 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.227 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.227 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.227 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.227 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.227 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.227 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:34.227 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:34.227 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.228 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.228 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.228 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.228 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.228 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.228 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.228 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.228 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.228 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.228 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.228 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.228 INFO:teuthology.orchestra.run.smithi077.stdout: "predict_interval": { 2023-12-01T01:22:34.228 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "predict_interval", 2023-12-01T01:22:34.228 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.228 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.229 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.229 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "86400", 2023-12-01T01:22:34.229 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.229 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.229 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.229 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.229 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.229 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.229 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.229 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.229 INFO:teuthology.orchestra.run.smithi077.stdout: "predictor_model": { 2023-12-01T01:22:34.229 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "predictor_model", 2023-12-01T01:22:34.229 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.229 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.230 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.230 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "prophetstor", 2023-12-01T01:22:34.230 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.230 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.230 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.230 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.230 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.230 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.230 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.230 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.230 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2023-12-01T01:22:34.230 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2023-12-01T01:22:34.230 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.230 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.230 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.231 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2023-12-01T01:22:34.231 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.231 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.231 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.231 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.231 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.231 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.231 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.231 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.231 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.231 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.231 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.231 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "influx", 2023-12-01T01:22:34.231 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": false, 2023-12-01T01:22:34.232 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "influxdb python module not found", 2023-12-01T01:22:34.232 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:34.232 INFO:teuthology.orchestra.run.smithi077.stdout: "batch_size": { 2023-12-01T01:22:34.232 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "batch_size", 2023-12-01T01:22:34.232 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.232 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.232 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.232 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5000", 2023-12-01T01:22:34.232 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.232 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.232 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.232 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-12-01T01:22:34.232 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.232 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.233 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.233 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.233 INFO:teuthology.orchestra.run.smithi077.stdout: "database": { 2023-12-01T01:22:34.233 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "database", 2023-12-01T01:22:34.233 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.233 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.233 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.233 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "ceph", 2023-12-01T01:22:34.233 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.233 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.233 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.233 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-12-01T01:22:34.233 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.233 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.234 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.234 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.234 INFO:teuthology.orchestra.run.smithi077.stdout: "hostname": { 2023-12-01T01:22:34.234 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "hostname", 2023-12-01T01:22:34.234 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.234 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.234 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.234 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.234 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.234 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.234 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.234 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "InfluxDB server hostname", 2023-12-01T01:22:34.234 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.234 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.235 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.235 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.235 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2023-12-01T01:22:34.235 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2023-12-01T01:22:34.235 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:34.235 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.235 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.235 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "30", 2023-12-01T01:22:34.235 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "5", 2023-12-01T01:22:34.235 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.235 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.235 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-12-01T01:22:34.235 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.235 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.235 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.236 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.236 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:34.236 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:34.236 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.236 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.236 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.236 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.236 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.236 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.236 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.236 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.236 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.236 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.236 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.237 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.237 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.237 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.237 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.237 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.237 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.237 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.237 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.237 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:34.237 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:34.237 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.237 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.237 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.237 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.237 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.238 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.238 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.238 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.238 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.238 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.238 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.238 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.238 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:34.238 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:34.238 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.238 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.238 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.238 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:34.238 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.239 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.239 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.239 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.239 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.239 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.239 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.239 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.239 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.239 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.239 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.239 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.239 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.239 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.239 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.239 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:34.240 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:34.240 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.240 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.240 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.240 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.240 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.240 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.240 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.240 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.240 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.240 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.240 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.240 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.240 INFO:teuthology.orchestra.run.smithi077.stdout: "password": { 2023-12-01T01:22:34.240 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "password", 2023-12-01T01:22:34.241 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.241 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.241 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.241 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.241 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.241 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.241 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.241 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "password of InfluxDB server user", 2023-12-01T01:22:34.241 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.241 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.241 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.241 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.241 INFO:teuthology.orchestra.run.smithi077.stdout: "port": { 2023-12-01T01:22:34.241 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "port", 2023-12-01T01:22:34.241 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.242 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.242 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.242 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8086", 2023-12-01T01:22:34.242 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.242 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.242 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.242 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "InfluxDB server port", 2023-12-01T01:22:34.242 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.242 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.242 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.242 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.242 INFO:teuthology.orchestra.run.smithi077.stdout: "ssl": { 2023-12-01T01:22:34.242 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssl", 2023-12-01T01:22:34.242 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.243 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.243 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.243 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "false", 2023-12-01T01:22:34.243 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.243 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.243 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.243 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-12-01T01:22:34.243 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.243 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.243 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.243 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.243 INFO:teuthology.orchestra.run.smithi077.stdout: "threads": { 2023-12-01T01:22:34.243 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "threads", 2023-12-01T01:22:34.243 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.243 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.244 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.244 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2023-12-01T01:22:34.244 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2023-12-01T01:22:34.244 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "32", 2023-12-01T01:22:34.244 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.244 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-12-01T01:22:34.244 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.244 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.244 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.244 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.244 INFO:teuthology.orchestra.run.smithi077.stdout: "username": { 2023-12-01T01:22:34.244 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "username", 2023-12-01T01:22:34.244 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.244 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.245 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.245 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.245 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.245 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.245 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.245 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "username of InfluxDB server user", 2023-12-01T01:22:34.245 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.245 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.245 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.245 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.245 INFO:teuthology.orchestra.run.smithi077.stdout: "verify_ssl": { 2023-12-01T01:22:34.245 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "verify_ssl", 2023-12-01T01:22:34.245 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.245 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.246 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.246 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "true", 2023-12-01T01:22:34.246 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.246 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.246 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.246 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-12-01T01:22:34.246 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.246 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.246 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.246 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.246 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.246 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.246 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.246 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "insights", 2023-12-01T01:22:34.247 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:34.247 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:34.247 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:34.247 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:34.247 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:34.247 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.247 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.247 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.247 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.247 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.247 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.247 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.247 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.247 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.247 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.248 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.248 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.248 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.248 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.248 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.248 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.248 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.248 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.248 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.248 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:34.249 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:34.249 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.249 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.249 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.249 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.249 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.249 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.249 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.249 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.249 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.249 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.249 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.249 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.249 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:34.249 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:34.250 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.250 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.250 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.250 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:34.250 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.250 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.250 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.250 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.250 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.250 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.250 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.250 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.251 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.251 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.251 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.251 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.251 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.251 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.251 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.251 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:34.251 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:34.251 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.251 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.251 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.251 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.251 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.251 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.252 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.252 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.252 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.252 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.252 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.252 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.252 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.252 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.252 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.252 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "iostat", 2023-12-01T01:22:34.252 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:34.252 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:34.252 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:34.252 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:34.252 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:34.253 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.253 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.253 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.253 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.253 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.253 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.253 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.253 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.253 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.253 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.253 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.253 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.253 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.253 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.253 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.254 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.254 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.254 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.254 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.254 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:34.254 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:34.254 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.254 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.254 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.254 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.254 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.254 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.254 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.254 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.255 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.255 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.255 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.255 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.255 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:34.255 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:34.255 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.255 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.255 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.255 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:34.255 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.255 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.255 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.255 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.256 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.256 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.256 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.256 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.256 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.256 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.256 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.256 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.256 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.256 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.256 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.256 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:34.256 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:34.256 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.256 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.257 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.257 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.257 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.257 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.257 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.257 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.257 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.257 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.257 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.257 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.257 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.257 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.257 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.257 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "k8sevents", 2023-12-01T01:22:34.257 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:34.258 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:34.258 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:34.258 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_event_retention_days": { 2023-12-01T01:22:34.258 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ceph_event_retention_days", 2023-12-01T01:22:34.258 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.258 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.258 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.258 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "7", 2023-12-01T01:22:34.258 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.258 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.258 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.258 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Days to hold ceph event information within local cache", 2023-12-01T01:22:34.258 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.258 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.259 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.259 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.259 INFO:teuthology.orchestra.run.smithi077.stdout: "config_check_secs": { 2023-12-01T01:22:34.259 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "config_check_secs", 2023-12-01T01:22:34.259 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.259 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.259 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.259 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2023-12-01T01:22:34.259 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "10", 2023-12-01T01:22:34.259 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.259 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.259 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-12-01T01:22:34.259 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.259 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.260 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.260 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.260 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:34.260 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:34.260 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.260 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.260 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.260 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.260 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.260 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.260 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.260 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.260 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.260 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.260 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.261 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.261 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.261 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.261 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.261 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.261 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.261 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.261 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.261 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:34.261 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:34.261 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.261 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.261 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.261 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.261 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.262 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.262 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.262 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.262 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.262 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.262 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.262 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.262 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:34.262 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:34.262 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.262 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.262 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.262 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:34.262 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.263 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.263 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.263 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.263 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.263 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.263 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.263 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.263 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.263 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.263 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.263 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.263 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.263 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.263 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.263 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:34.264 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:34.264 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.264 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.264 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.264 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.264 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.264 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.264 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.264 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.264 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.264 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.264 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.264 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.264 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.265 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.265 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.265 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "localpool", 2023-12-01T01:22:34.265 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:34.265 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:34.265 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:34.265 INFO:teuthology.orchestra.run.smithi077.stdout: "failure_domain": { 2023-12-01T01:22:34.265 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "failure_domain", 2023-12-01T01:22:34.265 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.265 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.265 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.265 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "host", 2023-12-01T01:22:34.265 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.265 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.265 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.266 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "failure domain for any created local pool", 2023-12-01T01:22:34.266 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-12-01T01:22:34.266 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.266 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.266 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.266 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:34.266 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:34.266 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.266 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.266 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.266 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.266 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.266 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.266 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.267 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.267 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.267 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.267 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.267 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.267 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.267 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.267 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.267 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.267 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.267 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.267 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.267 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:34.267 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:34.268 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.268 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.268 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.268 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.268 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.268 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.268 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.268 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.268 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.268 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.268 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.268 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.268 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:34.268 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:34.268 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.269 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.269 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.269 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:34.269 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.269 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.269 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.269 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.269 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.269 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.269 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.269 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.269 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.269 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.269 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.270 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.270 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.270 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.270 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.270 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:34.270 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:34.270 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.270 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.270 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.270 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.270 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.270 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.270 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.270 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.271 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.271 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.271 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.271 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.271 INFO:teuthology.orchestra.run.smithi077.stdout: "min_size": { 2023-12-01T01:22:34.271 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "min_size", 2023-12-01T01:22:34.271 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.271 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.271 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.271 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.271 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.271 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.271 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.271 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "default min_size for any created local pool", 2023-12-01T01:22:34.271 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-12-01T01:22:34.272 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.272 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.272 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.272 INFO:teuthology.orchestra.run.smithi077.stdout: "num_rep": { 2023-12-01T01:22:34.272 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "num_rep", 2023-12-01T01:22:34.272 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.272 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.272 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.272 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "3", 2023-12-01T01:22:34.272 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.272 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.272 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.272 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "default replica count for any created local pool", 2023-12-01T01:22:34.272 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.273 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.273 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.273 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.273 INFO:teuthology.orchestra.run.smithi077.stdout: "pg_num": { 2023-12-01T01:22:34.273 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pg_num", 2023-12-01T01:22:34.273 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.273 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.273 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.273 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "128", 2023-12-01T01:22:34.273 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.273 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.273 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.273 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "default pg_num for any created local pool", 2023-12-01T01:22:34.273 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.273 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.274 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.274 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.274 INFO:teuthology.orchestra.run.smithi077.stdout: "prefix": { 2023-12-01T01:22:34.274 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "prefix", 2023-12-01T01:22:34.274 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.274 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.274 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.274 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.274 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.274 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.274 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.274 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "name prefix for any created local pool", 2023-12-01T01:22:34.274 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.274 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.275 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.275 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.275 INFO:teuthology.orchestra.run.smithi077.stdout: "subtree": { 2023-12-01T01:22:34.275 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "subtree", 2023-12-01T01:22:34.275 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.275 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.275 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.275 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "rack", 2023-12-01T01:22:34.275 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.275 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.275 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.275 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "CRUSH level for which to create a local pool", 2023-12-01T01:22:34.275 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-12-01T01:22:34.275 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.275 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.276 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.276 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.276 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.276 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.276 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mds_autoscaler", 2023-12-01T01:22:34.276 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:34.276 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:34.276 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:34.276 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:34.276 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:34.276 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.276 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.276 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.276 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.277 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.277 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.277 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.277 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.277 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.277 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.277 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.277 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.277 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.277 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.277 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.277 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.277 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.277 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.277 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.278 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:34.278 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:34.278 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.278 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.278 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.278 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.278 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.278 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.278 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.278 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.278 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.278 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.278 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.278 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.279 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:34.279 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:34.279 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.279 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.279 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.279 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:34.279 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.279 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.279 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.279 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.279 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.279 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.279 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.279 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.279 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.280 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.280 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.280 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.280 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.280 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.280 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.280 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:34.280 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:34.280 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.280 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.280 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.280 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.280 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.280 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.281 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.281 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.281 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.281 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.281 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.281 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.281 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.281 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.281 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.281 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mirroring", 2023-12-01T01:22:34.281 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:34.281 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:34.281 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:34.281 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:34.282 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:34.282 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.282 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.282 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.282 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.282 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.282 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.282 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.282 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.282 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.282 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.282 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.282 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.282 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.282 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.283 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.283 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.283 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.283 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.283 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.283 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:34.283 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:34.283 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.283 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.283 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.283 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.283 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.283 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.283 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.284 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.284 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.284 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.284 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.284 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.284 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:34.284 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:34.284 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.284 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.284 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.284 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:34.284 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.284 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.284 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.285 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.285 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.285 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.285 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.285 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.285 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.285 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.285 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.285 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.285 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.285 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.285 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.285 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:34.285 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:34.285 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.286 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.286 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.286 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.286 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.286 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.286 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.286 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.286 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.286 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.286 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.286 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.286 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.286 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.286 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.286 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "nfs", 2023-12-01T01:22:34.287 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:34.287 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:34.287 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:34.287 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:34.287 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:34.287 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.287 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.287 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.287 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.287 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.287 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.287 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.287 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.287 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.288 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.288 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.288 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.288 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.288 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.288 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.288 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.288 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.288 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.288 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.288 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:34.288 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:34.288 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.288 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.288 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.289 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.289 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.289 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.289 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.289 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.289 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.289 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.289 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.289 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.289 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:34.289 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:34.289 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.289 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.289 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.290 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:34.290 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.290 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.290 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.290 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.290 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.290 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.290 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.290 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.290 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.290 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.290 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.290 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.290 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.290 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.291 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.291 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:34.291 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:34.291 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.291 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.291 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.291 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.291 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.291 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.291 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.291 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.291 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.291 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.291 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.292 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.292 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.292 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.292 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.292 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "orchestrator", 2023-12-01T01:22:34.292 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:34.292 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:34.292 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:34.292 INFO:teuthology.orchestra.run.smithi077.stdout: "fail_fs": { 2023-12-01T01:22:34.292 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "fail_fs", 2023-12-01T01:22:34.292 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.292 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.292 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.292 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.292 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.293 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.293 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.293 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-12-01T01:22:34.293 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.293 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.293 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.293 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.293 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:34.293 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:34.293 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.293 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.293 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.293 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.293 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.293 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.294 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.294 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.294 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.294 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.294 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.294 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.294 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.294 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.294 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.294 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.294 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.294 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.294 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.294 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:34.295 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:34.295 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.295 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.295 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.295 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.295 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.295 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.295 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.295 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.295 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.295 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.295 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.295 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.295 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:34.296 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:34.296 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.296 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.296 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.296 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:34.296 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.296 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.296 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.296 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.296 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.296 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.296 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.296 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.296 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.296 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.297 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.297 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.297 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.297 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.297 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.297 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:34.297 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:34.297 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.297 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.297 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.297 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.297 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.297 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.298 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.298 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.298 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.298 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.298 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.298 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.298 INFO:teuthology.orchestra.run.smithi077.stdout: "orchestrator": { 2023-12-01T01:22:34.298 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "orchestrator", 2023-12-01T01:22:34.298 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.298 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.298 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.298 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.298 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.298 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.299 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.299 INFO:teuthology.orchestra.run.smithi077.stdout: "cephadm", 2023-12-01T01:22:34.299 INFO:teuthology.orchestra.run.smithi077.stdout: "rook", 2023-12-01T01:22:34.299 INFO:teuthology.orchestra.run.smithi077.stdout: "test_orchestrator" 2023-12-01T01:22:34.299 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.299 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Orchestrator backend", 2023-12-01T01:22:34.299 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.299 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.299 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.299 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.299 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.299 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.299 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.299 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "osd_perf_query", 2023-12-01T01:22:34.299 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:34.300 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:34.300 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:34.300 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:34.300 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:34.300 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.300 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.300 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.300 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.300 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.300 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.300 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.300 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.300 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.300 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.300 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.301 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.301 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.301 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.301 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.301 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.301 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.301 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.301 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.301 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:34.301 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:34.301 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.301 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.301 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.301 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.302 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.302 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.302 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.302 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.302 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.302 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.302 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.302 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.302 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:34.302 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:34.302 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.302 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.302 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.302 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:34.303 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.303 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.303 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.303 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.303 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.303 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.303 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.303 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.303 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.303 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.303 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.303 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.303 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.303 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.303 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.304 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:34.304 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:34.304 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.304 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.304 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.304 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.304 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.304 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.304 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.304 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.304 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.304 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.304 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.304 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.304 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.305 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.305 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.305 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "osd_support", 2023-12-01T01:22:34.305 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:34.305 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:34.305 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:34.305 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:34.305 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:34.305 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.305 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.305 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.305 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.305 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.305 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.306 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.306 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.306 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.306 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.306 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.306 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.306 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.306 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.306 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.306 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.306 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.306 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.306 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.306 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:34.306 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:34.307 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.307 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.307 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.307 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.307 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.307 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.307 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.307 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.307 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.307 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.307 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.307 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.307 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:34.307 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:34.308 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.308 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.308 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.308 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:34.308 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.308 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.308 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.308 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.308 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.308 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.308 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.308 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.308 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.308 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.308 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.309 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.309 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.309 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.309 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.309 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:34.309 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:34.309 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.309 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.309 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.309 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.309 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.309 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.309 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.309 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.309 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.310 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.310 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.310 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.310 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.310 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.310 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.310 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pg_autoscaler", 2023-12-01T01:22:34.310 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:34.310 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:34.310 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:34.310 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:34.310 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:34.310 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.310 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.311 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.311 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.311 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.311 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.311 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.311 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.311 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.311 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.311 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.311 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.311 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.311 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.311 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.311 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.311 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.312 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.312 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.312 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:34.312 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:34.312 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.312 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.312 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.312 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.312 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.312 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.312 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.312 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.312 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.312 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.312 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.313 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.313 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:34.313 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:34.313 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.313 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.313 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.313 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:34.313 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.313 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.313 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.313 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.313 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.313 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.313 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.314 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.314 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.314 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.314 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.314 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.314 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.314 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.314 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.314 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:34.314 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:34.314 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.314 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.314 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.314 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.315 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.315 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.315 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.315 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.315 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.315 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.315 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.315 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.315 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2023-12-01T01:22:34.315 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2023-12-01T01:22:34.315 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:34.315 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.315 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.315 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2023-12-01T01:22:34.315 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.316 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.316 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.316 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.316 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.316 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.316 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.316 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.316 INFO:teuthology.orchestra.run.smithi077.stdout: "threshold": { 2023-12-01T01:22:34.316 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "threshold", 2023-12-01T01:22:34.316 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2023-12-01T01:22:34.316 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.316 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.316 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "3.0", 2023-12-01T01:22:34.316 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1.0", 2023-12-01T01:22:34.317 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.317 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.317 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "scaling threshold", 2023-12-01T01:22:34.317 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-12-01T01:22:34.317 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.317 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.317 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.317 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.317 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.317 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.317 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "progress", 2023-12-01T01:22:34.317 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:34.317 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:34.317 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:34.318 INFO:teuthology.orchestra.run.smithi077.stdout: "allow_pg_recovery_event": { 2023-12-01T01:22:34.318 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "allow_pg_recovery_event", 2023-12-01T01:22:34.318 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.318 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.318 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.318 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.318 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.318 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.318 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.318 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "allow the module to show pg recovery progress", 2023-12-01T01:22:34.318 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.318 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.318 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.318 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.319 INFO:teuthology.orchestra.run.smithi077.stdout: "enabled": { 2023-12-01T01:22:34.319 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enabled", 2023-12-01T01:22:34.319 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.319 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.319 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.319 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:34.319 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.319 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.319 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.319 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.319 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.319 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.319 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.319 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.319 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:34.320 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:34.320 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.320 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.320 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.320 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.320 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.320 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.320 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.320 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.320 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.320 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.320 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.320 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.320 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.320 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.321 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.321 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.321 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.321 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.321 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.321 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:34.321 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:34.321 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.321 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.321 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.321 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.321 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.321 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.321 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.322 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.322 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.322 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.322 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.322 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.322 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:34.322 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:34.322 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.322 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.322 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.322 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:34.322 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.322 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.322 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.323 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.323 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.323 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.323 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.323 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.323 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.323 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.323 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.323 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.323 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.323 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.323 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.323 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:34.323 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:34.323 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.324 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.324 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.324 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.324 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.324 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.324 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.324 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.324 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.324 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.324 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.324 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.324 INFO:teuthology.orchestra.run.smithi077.stdout: "max_completed_events": { 2023-12-01T01:22:34.324 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_completed_events", 2023-12-01T01:22:34.324 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.325 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.325 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.325 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "50", 2023-12-01T01:22:34.325 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.325 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.325 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.325 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "number of past completed events to remember", 2023-12-01T01:22:34.325 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.325 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.325 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.325 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.325 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2023-12-01T01:22:34.325 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2023-12-01T01:22:34.325 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:34.325 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.326 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.326 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2023-12-01T01:22:34.326 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.326 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.326 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.326 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how long the module is going to sleep", 2023-12-01T01:22:34.326 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.326 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.326 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.326 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.326 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.326 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.326 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.327 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "prometheus", 2023-12-01T01:22:34.327 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:34.327 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:34.327 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:34.327 INFO:teuthology.orchestra.run.smithi077.stdout: "cache": { 2023-12-01T01:22:34.327 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cache", 2023-12-01T01:22:34.327 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.327 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.327 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.327 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:34.327 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.327 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.327 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.327 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.327 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.328 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.328 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.328 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.328 INFO:teuthology.orchestra.run.smithi077.stdout: "exclude_perf_counters": { 2023-12-01T01:22:34.328 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "exclude_perf_counters", 2023-12-01T01:22:34.328 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.328 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.328 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.328 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:34.328 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.328 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.328 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.328 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-12-01T01:22:34.328 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-12-01T01:22:34.328 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.329 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.329 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.329 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:34.329 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:34.329 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.329 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.329 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.329 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.329 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.329 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.329 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.329 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.329 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.329 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.330 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.330 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.330 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.330 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.330 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.330 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.330 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.330 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.330 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.330 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:34.330 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:34.330 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.330 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.331 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.331 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.331 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.331 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.331 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.331 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.331 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.331 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.331 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.331 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.331 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:34.331 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:34.331 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.331 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.331 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.332 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:34.332 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.332 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.332 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.332 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.332 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.332 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.332 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.332 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.332 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.332 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.332 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.332 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.332 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.333 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.333 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.333 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:34.333 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:34.333 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.333 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.333 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.333 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.333 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.333 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.333 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.333 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.333 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.333 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.334 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.334 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.334 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_stats_pools": { 2023-12-01T01:22:34.334 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rbd_stats_pools", 2023-12-01T01:22:34.334 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.334 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.334 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.334 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.334 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.334 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.334 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.334 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.334 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.334 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.335 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.335 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.335 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_stats_pools_refresh_interval": { 2023-12-01T01:22:34.335 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-12-01T01:22:34.335 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.335 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.335 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.335 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "300", 2023-12-01T01:22:34.335 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.335 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.335 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.335 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.335 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.335 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.336 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.336 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.336 INFO:teuthology.orchestra.run.smithi077.stdout: "scrape_interval": { 2023-12-01T01:22:34.336 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "scrape_interval", 2023-12-01T01:22:34.336 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2023-12-01T01:22:34.336 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.336 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.336 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "15.0", 2023-12-01T01:22:34.336 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.336 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.336 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.336 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.336 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.336 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.336 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.337 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.337 INFO:teuthology.orchestra.run.smithi077.stdout: "server_addr": { 2023-12-01T01:22:34.337 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_addr", 2023-12-01T01:22:34.337 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.337 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.337 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.337 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "::", 2023-12-01T01:22:34.337 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.337 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.337 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.337 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-12-01T01:22:34.337 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.337 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.337 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.337 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.338 INFO:teuthology.orchestra.run.smithi077.stdout: "server_port": { 2023-12-01T01:22:34.338 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_port", 2023-12-01T01:22:34.338 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.338 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.338 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.338 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "9283", 2023-12-01T01:22:34.338 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.338 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.338 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.338 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-12-01T01:22:34.338 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.338 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.338 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.338 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.339 INFO:teuthology.orchestra.run.smithi077.stdout: "stale_cache_strategy": { 2023-12-01T01:22:34.339 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "stale_cache_strategy", 2023-12-01T01:22:34.339 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.339 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.339 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.339 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "log", 2023-12-01T01:22:34.339 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.339 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.339 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.339 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.339 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.339 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.339 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.339 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.340 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_behaviour": { 2023-12-01T01:22:34.340 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_behaviour", 2023-12-01T01:22:34.340 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.340 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.340 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.340 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "default", 2023-12-01T01:22:34.340 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.340 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.340 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.340 INFO:teuthology.orchestra.run.smithi077.stdout: "default", 2023-12-01T01:22:34.340 INFO:teuthology.orchestra.run.smithi077.stdout: "error" 2023-12-01T01:22:34.340 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.340 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.340 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.340 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.341 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.341 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.341 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_error_status_code": { 2023-12-01T01:22:34.341 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_error_status_code", 2023-12-01T01:22:34.341 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.341 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.341 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.341 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "500", 2023-12-01T01:22:34.341 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "400", 2023-12-01T01:22:34.341 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "599", 2023-12-01T01:22:34.341 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.341 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.341 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.341 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.342 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.342 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.342 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.342 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.342 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.342 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rbd_support", 2023-12-01T01:22:34.342 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:34.342 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:34.342 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:34.342 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:34.342 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:34.342 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.342 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.342 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.342 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.343 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.343 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.343 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.343 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.343 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.343 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.343 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.343 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.343 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.343 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.343 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.343 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.343 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.343 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.343 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.344 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:34.344 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:34.344 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.344 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.344 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.344 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.344 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.344 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.344 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.344 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.344 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.344 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.344 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.344 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.344 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:34.345 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:34.345 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.345 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.345 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.345 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:34.345 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.345 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.345 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.345 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.345 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.345 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.345 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.345 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.345 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.346 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.346 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.346 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.346 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.346 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.346 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.346 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:34.346 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:34.346 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.346 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.346 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.346 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.346 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.346 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.347 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.347 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.347 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.347 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.347 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.347 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.347 INFO:teuthology.orchestra.run.smithi077.stdout: "max_concurrent_snap_create": { 2023-12-01T01:22:34.347 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_concurrent_snap_create", 2023-12-01T01:22:34.347 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.347 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.347 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.347 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2023-12-01T01:22:34.347 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.347 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.348 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.348 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.348 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.348 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.348 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.348 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.348 INFO:teuthology.orchestra.run.smithi077.stdout: "mirror_snapshot_schedule": { 2023-12-01T01:22:34.348 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mirror_snapshot_schedule", 2023-12-01T01:22:34.348 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.348 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.348 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.348 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.348 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.348 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.348 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.349 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.349 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.349 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.349 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.349 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.349 INFO:teuthology.orchestra.run.smithi077.stdout: "trash_purge_schedule": { 2023-12-01T01:22:34.349 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "trash_purge_schedule", 2023-12-01T01:22:34.349 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.349 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.349 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.349 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.349 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.349 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.349 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.350 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.350 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.350 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.350 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.350 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.350 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.350 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.350 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.350 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "restful", 2023-12-01T01:22:34.350 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:34.350 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:34.350 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:34.350 INFO:teuthology.orchestra.run.smithi077.stdout: "enable_auth": { 2023-12-01T01:22:34.350 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enable_auth", 2023-12-01T01:22:34.350 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.351 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.351 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.351 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:34.351 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.351 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.351 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.351 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.351 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.351 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.351 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.351 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.351 INFO:teuthology.orchestra.run.smithi077.stdout: "key_file": { 2023-12-01T01:22:34.351 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "key_file", 2023-12-01T01:22:34.351 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.352 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.352 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.352 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.352 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.352 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.352 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.352 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.352 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.352 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.352 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.352 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.352 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:34.352 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:34.352 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.352 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.353 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.353 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.353 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.353 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.353 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.353 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.353 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.353 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.353 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.353 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.353 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.353 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.353 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.353 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.353 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.354 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.354 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.354 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:34.354 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:34.354 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.354 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.354 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.354 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.354 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.354 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.354 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.354 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.354 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.354 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.355 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.355 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.355 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:34.355 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:34.355 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.355 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.355 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.355 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:34.355 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.355 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.355 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.355 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.355 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.355 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.355 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.356 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.356 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.356 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.356 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.356 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.356 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.356 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.356 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.356 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:34.356 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:34.356 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.356 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.356 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.356 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.356 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.357 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.357 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.357 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.357 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.357 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.357 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.357 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.357 INFO:teuthology.orchestra.run.smithi077.stdout: "server_addr": { 2023-12-01T01:22:34.357 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_addr", 2023-12-01T01:22:34.357 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.357 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.357 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.357 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.357 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.357 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.358 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.358 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.358 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.358 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.358 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.358 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.358 INFO:teuthology.orchestra.run.smithi077.stdout: "server_port": { 2023-12-01T01:22:34.358 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_port", 2023-12-01T01:22:34.358 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.358 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.358 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.358 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.358 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.358 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.359 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.359 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.359 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.359 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.359 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.359 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.359 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.359 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.359 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.359 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rgw", 2023-12-01T01:22:34.359 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:34.359 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:34.359 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:34.359 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:34.359 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:34.360 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.360 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.360 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.360 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.360 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.360 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.360 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.360 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.360 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.360 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.360 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.360 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.360 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.360 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.361 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.361 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.361 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.361 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.361 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.361 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:34.361 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:34.361 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.361 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.361 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.361 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.361 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.361 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.361 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.361 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.362 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.362 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.362 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.362 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.362 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:34.362 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:34.362 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.362 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.362 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.362 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:34.362 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.362 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.362 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.362 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.363 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.363 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.363 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.363 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.363 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.363 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.363 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.363 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.363 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.363 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.363 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.363 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:34.363 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:34.363 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.364 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.364 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.364 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.364 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.364 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.364 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.364 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.364 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.364 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.364 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.364 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.364 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.364 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.364 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.365 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rook", 2023-12-01T01:22:34.365 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:34.365 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:34.365 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:34.365 INFO:teuthology.orchestra.run.smithi077.stdout: "drive_group_interval": { 2023-12-01T01:22:34.365 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "drive_group_interval", 2023-12-01T01:22:34.365 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2023-12-01T01:22:34.365 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.365 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.365 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "300.0", 2023-12-01T01:22:34.365 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.365 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.365 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.365 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-12-01T01:22:34.365 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.366 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.366 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.366 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.366 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:34.366 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:34.366 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.366 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.366 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.366 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.366 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.366 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.366 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.366 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.366 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.367 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.367 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.367 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.367 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.367 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.367 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.367 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.367 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.367 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.367 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.367 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:34.367 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:34.367 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.367 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.368 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.368 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.368 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.368 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.368 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.368 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.368 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.368 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.368 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.368 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.368 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:34.368 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:34.368 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.368 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.368 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.369 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:34.369 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.369 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.369 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.369 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.369 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.369 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.369 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.369 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.369 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.369 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.369 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.369 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.369 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.369 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.370 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.370 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:34.370 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:34.370 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.370 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.370 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.370 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.370 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.370 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.370 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.370 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.370 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.370 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.370 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.371 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.371 INFO:teuthology.orchestra.run.smithi077.stdout: "storage_class": { 2023-12-01T01:22:34.371 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "storage_class", 2023-12-01T01:22:34.371 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.371 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.371 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.371 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "local", 2023-12-01T01:22:34.371 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.371 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.371 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.371 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-12-01T01:22:34.371 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.371 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.371 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.371 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.372 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.372 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.372 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.372 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "selftest", 2023-12-01T01:22:34.372 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:34.372 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:34.372 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:34.372 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:34.372 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:34.372 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.372 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.372 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.372 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.372 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.373 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.373 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.373 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.373 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.373 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.373 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.373 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.373 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.373 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.373 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.373 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.373 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.373 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.373 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.374 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:34.374 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:34.374 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.374 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.374 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.374 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.374 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.374 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.374 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.374 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.374 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.374 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.374 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.374 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.375 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:34.375 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:34.375 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.375 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.375 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.375 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:34.375 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.375 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.375 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.375 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.375 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.375 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.375 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.375 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.376 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.376 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.376 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.376 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.376 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.376 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.376 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.376 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:34.376 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:34.376 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.376 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.376 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.376 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.376 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.376 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.377 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.377 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.377 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.377 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.377 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.377 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.377 INFO:teuthology.orchestra.run.smithi077.stdout: "roption1": { 2023-12-01T01:22:34.377 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "roption1", 2023-12-01T01:22:34.377 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.377 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.377 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.377 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.377 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.377 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.377 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.378 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.378 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.378 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.378 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.378 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.378 INFO:teuthology.orchestra.run.smithi077.stdout: "roption2": { 2023-12-01T01:22:34.378 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "roption2", 2023-12-01T01:22:34.378 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.378 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.378 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.378 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "xyz", 2023-12-01T01:22:34.378 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.378 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.378 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.379 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.379 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.379 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.379 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.379 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.379 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption1": { 2023-12-01T01:22:34.379 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption1", 2023-12-01T01:22:34.379 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.379 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.379 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.379 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.379 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.379 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.380 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.380 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.380 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.380 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.380 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.380 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.380 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption2": { 2023-12-01T01:22:34.380 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption2", 2023-12-01T01:22:34.380 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.380 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.380 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.380 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.380 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.380 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.380 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.381 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.381 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.381 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.381 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.381 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.381 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption3": { 2023-12-01T01:22:34.381 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption3", 2023-12-01T01:22:34.381 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2023-12-01T01:22:34.381 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.381 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.381 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.381 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.381 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.382 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.382 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.382 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.382 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.382 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.382 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.382 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption4": { 2023-12-01T01:22:34.382 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption4", 2023-12-01T01:22:34.382 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.382 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.382 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.382 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.382 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.382 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.383 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.383 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.383 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.383 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.383 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.383 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.383 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption5": { 2023-12-01T01:22:34.383 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption5", 2023-12-01T01:22:34.383 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.383 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.383 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.383 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.383 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.383 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.384 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.384 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.384 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.384 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.384 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.384 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.384 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption6": { 2023-12-01T01:22:34.384 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption6", 2023-12-01T01:22:34.384 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.384 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.384 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.384 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:34.384 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.384 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.385 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.385 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.385 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.385 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.385 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.385 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.385 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption7": { 2023-12-01T01:22:34.385 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption7", 2023-12-01T01:22:34.385 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.385 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.385 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.385 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.385 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2023-12-01T01:22:34.385 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "42", 2023-12-01T01:22:34.386 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.386 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.386 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.386 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.386 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.386 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.386 INFO:teuthology.orchestra.run.smithi077.stdout: "testkey": { 2023-12-01T01:22:34.386 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "testkey", 2023-12-01T01:22:34.386 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.386 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.386 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.386 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.386 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.386 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.387 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.387 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.387 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.387 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.387 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.387 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.387 INFO:teuthology.orchestra.run.smithi077.stdout: "testlkey": { 2023-12-01T01:22:34.387 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "testlkey", 2023-12-01T01:22:34.387 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.387 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.387 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.387 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.387 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.387 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.387 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.388 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.388 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.388 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.388 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.388 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.388 INFO:teuthology.orchestra.run.smithi077.stdout: "testnewline": { 2023-12-01T01:22:34.388 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "testnewline", 2023-12-01T01:22:34.388 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.388 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.388 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.388 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.388 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.388 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.388 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.388 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.389 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.389 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.389 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.389 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.389 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.389 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.389 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.389 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "snap_schedule", 2023-12-01T01:22:34.389 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:34.389 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:34.389 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:34.389 INFO:teuthology.orchestra.run.smithi077.stdout: "allow_m_granularity": { 2023-12-01T01:22:34.389 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "allow_m_granularity", 2023-12-01T01:22:34.389 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.390 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.390 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.390 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.390 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.390 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.390 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.390 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "allow minute scheduled snapshots", 2023-12-01T01:22:34.390 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.390 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.390 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.390 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.390 INFO:teuthology.orchestra.run.smithi077.stdout: "dump_on_update": { 2023-12-01T01:22:34.390 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "dump_on_update", 2023-12-01T01:22:34.390 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.391 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.391 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.391 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.391 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.391 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.391 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.391 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "dump database to debug log on update", 2023-12-01T01:22:34.391 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.391 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.391 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.391 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.391 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:34.391 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:34.391 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.392 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.392 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.392 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.392 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.392 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.392 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.392 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.392 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.392 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.392 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.392 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.392 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.392 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.392 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.392 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.393 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.393 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.393 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.393 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:34.393 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:34.393 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.393 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.393 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.393 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.393 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.393 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.393 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.393 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.393 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.394 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.394 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.394 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.394 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:34.394 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:34.394 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.394 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.394 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.394 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:34.394 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.394 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.394 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.394 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.394 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.394 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.395 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.395 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.395 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.395 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.395 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.395 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.395 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.395 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.395 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.395 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:34.395 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:34.395 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.395 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.395 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.396 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.396 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.396 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.396 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.396 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.396 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.396 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.396 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.396 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.396 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.396 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.396 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.396 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "stats", 2023-12-01T01:22:34.396 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:34.397 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:34.397 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:34.397 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:34.397 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:34.397 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.397 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.397 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.397 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.397 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.397 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.397 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.397 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.397 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.397 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.397 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.398 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.398 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.398 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.398 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.398 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.398 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.398 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.398 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.398 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:34.398 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:34.398 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.398 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.398 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.398 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.399 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.399 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.399 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.399 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.399 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.399 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.399 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.399 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.399 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:34.399 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:34.399 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.399 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.399 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.399 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:34.399 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.400 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.400 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.400 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.400 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.400 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.400 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.400 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.400 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.400 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.400 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.400 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.400 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.400 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.400 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.401 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:34.401 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:34.401 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.401 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.401 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.401 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.401 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.401 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.401 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.401 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.401 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.401 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.401 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.401 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.401 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.402 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.402 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.402 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "status", 2023-12-01T01:22:34.402 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:34.402 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:34.402 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:34.402 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:34.402 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:34.402 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.402 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.402 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.402 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.402 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.402 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.402 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.403 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.403 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.403 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.403 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.403 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.403 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.403 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.403 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.403 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.403 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.403 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.403 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.403 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:34.403 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:34.404 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.404 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.404 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.404 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.404 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.404 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.404 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.404 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.404 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.404 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.404 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.404 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.404 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:34.404 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:34.404 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.405 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.405 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.405 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:34.405 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.405 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.405 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.405 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.405 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.405 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.405 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.405 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.405 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.405 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.405 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.406 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.406 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.406 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.406 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.406 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:34.406 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:34.406 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.406 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.406 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.406 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.406 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.406 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.406 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.406 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.407 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.407 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.407 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.407 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.407 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.407 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.407 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.407 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "telegraf", 2023-12-01T01:22:34.407 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:34.407 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:34.407 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:34.407 INFO:teuthology.orchestra.run.smithi077.stdout: "address": { 2023-12-01T01:22:34.407 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "address", 2023-12-01T01:22:34.407 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.408 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.408 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.408 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-12-01T01:22:34.408 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.408 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.408 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.408 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.408 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.408 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.408 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.408 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.408 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2023-12-01T01:22:34.408 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2023-12-01T01:22:34.408 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:34.408 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.409 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.409 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "15", 2023-12-01T01:22:34.409 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.409 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.409 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.409 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.409 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.409 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.409 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.409 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.409 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:34.409 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:34.409 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.409 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.409 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.410 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.410 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.410 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.410 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.410 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.410 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.410 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.410 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.410 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.410 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.410 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.410 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.410 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.411 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.411 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.411 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.411 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:34.411 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:34.411 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.411 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.411 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.411 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.411 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.411 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.411 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.411 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.411 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.411 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.412 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.412 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.412 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:34.412 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:34.412 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.412 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.412 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.412 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:34.412 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.412 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.412 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.412 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.412 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.412 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.413 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.413 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.413 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.413 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.413 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.413 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.413 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.413 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.413 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.413 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:34.413 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:34.413 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.413 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.413 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.414 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.414 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.414 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.414 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.414 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.414 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.414 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.414 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.414 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.414 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.414 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.414 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.414 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "telemetry", 2023-12-01T01:22:34.414 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:34.415 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:34.417 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:34.417 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_basic": { 2023-12-01T01:22:34.417 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_basic", 2023-12-01T01:22:34.417 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.417 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.417 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.417 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:34.417 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.417 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.417 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.417 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share basic cluster information (size, version)", 2023-12-01T01:22:34.418 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.418 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.418 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.418 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.418 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_crash": { 2023-12-01T01:22:34.418 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_crash", 2023-12-01T01:22:34.418 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.418 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.418 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.418 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:34.418 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.418 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.418 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.418 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-12-01T01:22:34.419 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.419 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.419 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.419 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.419 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_device": { 2023-12-01T01:22:34.419 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_device", 2023-12-01T01:22:34.419 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.419 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.419 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.419 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2023-12-01T01:22:34.419 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.419 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.419 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.419 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-12-01T01:22:34.420 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.420 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.420 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.420 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.420 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_ident": { 2023-12-01T01:22:34.420 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_ident", 2023-12-01T01:22:34.420 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.420 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.420 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.420 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.420 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.420 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.420 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.420 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-12-01T01:22:34.420 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.421 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.421 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.421 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.421 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_perf": { 2023-12-01T01:22:34.421 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_perf", 2023-12-01T01:22:34.421 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.421 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.421 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.421 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.421 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.421 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.421 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.421 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share various performance metrics of a cluster", 2023-12-01T01:22:34.421 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.422 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.422 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.422 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.422 INFO:teuthology.orchestra.run.smithi077.stdout: "contact": { 2023-12-01T01:22:34.422 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "contact", 2023-12-01T01:22:34.422 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.422 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.422 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.422 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.422 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.422 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.422 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.422 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.422 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.423 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.423 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.423 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.423 INFO:teuthology.orchestra.run.smithi077.stdout: "description": { 2023-12-01T01:22:34.423 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "description", 2023-12-01T01:22:34.423 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.423 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.423 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.423 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.423 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.423 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.423 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.423 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.423 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.423 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.424 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.424 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.424 INFO:teuthology.orchestra.run.smithi077.stdout: "device_url": { 2023-12-01T01:22:34.424 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "device_url", 2023-12-01T01:22:34.424 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.424 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.424 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.424 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-12-01T01:22:34.424 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.424 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.424 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.424 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.424 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.424 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.425 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.425 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.425 INFO:teuthology.orchestra.run.smithi077.stdout: "enabled": { 2023-12-01T01:22:34.425 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enabled", 2023-12-01T01:22:34.425 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.425 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.425 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.425 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.425 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.425 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.425 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.425 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.425 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.425 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.425 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.426 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.426 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2023-12-01T01:22:34.426 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2023-12-01T01:22:34.426 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.426 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.426 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.426 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "24", 2023-12-01T01:22:34.426 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "8", 2023-12-01T01:22:34.426 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.426 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.426 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.426 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.426 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.426 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.427 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.427 INFO:teuthology.orchestra.run.smithi077.stdout: "last_opt_revision": { 2023-12-01T01:22:34.427 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "last_opt_revision", 2023-12-01T01:22:34.427 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.427 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.427 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.427 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "1", 2023-12-01T01:22:34.427 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.427 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.427 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.427 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.427 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.427 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.427 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.428 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.428 INFO:teuthology.orchestra.run.smithi077.stdout: "leaderboard": { 2023-12-01T01:22:34.428 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "leaderboard", 2023-12-01T01:22:34.428 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.428 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.428 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.428 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.428 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.428 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.428 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.428 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.428 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.428 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.428 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.428 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.429 INFO:teuthology.orchestra.run.smithi077.stdout: "leaderboard_description": { 2023-12-01T01:22:34.429 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "leaderboard_description", 2023-12-01T01:22:34.429 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.429 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.429 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.429 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.429 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.429 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.429 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.429 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.429 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.429 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.429 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.429 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.429 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:34.430 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:34.430 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.430 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.430 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.430 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.430 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.430 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.430 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.430 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.430 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.430 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.430 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.430 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.430 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.431 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.431 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.431 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.431 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.431 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.431 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.431 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:34.431 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:34.431 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.431 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.431 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.431 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.431 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.431 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.432 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.432 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.432 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.432 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.432 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.432 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.432 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:34.432 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:34.432 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.432 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.432 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.432 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:34.432 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.432 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.433 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.433 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.433 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.433 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.433 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.433 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.433 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.433 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.433 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.433 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.433 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.433 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.433 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.433 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:34.434 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:34.434 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.434 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.434 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.434 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.434 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.434 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.434 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.434 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.434 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.434 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.434 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.434 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.434 INFO:teuthology.orchestra.run.smithi077.stdout: "organization": { 2023-12-01T01:22:34.435 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "organization", 2023-12-01T01:22:34.435 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.435 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.435 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.435 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.435 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.435 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.435 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.435 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.435 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.435 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.435 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.435 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.435 INFO:teuthology.orchestra.run.smithi077.stdout: "proxy": { 2023-12-01T01:22:34.435 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "proxy", 2023-12-01T01:22:34.436 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.436 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.436 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.436 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.436 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.436 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.436 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.436 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.436 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.436 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.436 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.436 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.436 INFO:teuthology.orchestra.run.smithi077.stdout: "url": { 2023-12-01T01:22:34.436 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "url", 2023-12-01T01:22:34.437 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.437 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.437 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.437 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-12-01T01:22:34.437 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.437 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.437 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.437 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.437 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.437 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.437 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.437 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.437 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.437 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.437 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.438 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "test_orchestrator", 2023-12-01T01:22:34.438 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:34.438 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:34.438 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:34.438 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:34.438 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:34.438 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.438 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.438 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.438 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.438 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.438 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.438 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.439 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.439 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.439 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.439 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.439 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.439 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.439 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.439 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.439 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.439 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.439 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.439 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.439 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:34.439 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:34.439 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.440 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.440 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.440 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.440 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.440 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.440 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.440 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.440 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.440 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.440 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.440 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.440 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:34.440 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:34.440 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.440 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.441 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.441 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:34.441 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.441 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.441 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.441 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.441 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.441 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.441 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.441 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.441 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.441 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.441 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.441 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.441 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.442 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.442 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.442 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:34.442 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:34.442 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.442 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.442 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.442 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.442 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.442 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.442 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.442 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.442 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.442 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.443 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.443 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.443 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.443 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.443 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.443 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "volumes", 2023-12-01T01:22:34.443 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:34.443 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:34.443 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:34.443 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:34.443 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:34.443 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.443 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.443 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.444 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.444 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.444 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.444 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.444 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.444 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.444 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.444 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.444 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.444 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.444 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.444 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.444 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.444 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.444 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.445 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.445 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:34.445 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:34.445 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.445 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.445 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.445 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.445 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.445 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.445 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.445 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.445 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.445 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.445 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.445 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.446 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:34.446 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:34.446 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.446 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.446 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.446 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:34.446 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.446 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.446 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.446 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.446 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.446 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.446 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.446 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.447 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.447 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.447 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.447 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.447 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.447 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.447 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.447 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:34.447 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:34.447 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.447 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.447 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.447 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.448 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.448 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.448 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.448 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.448 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.448 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.448 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.448 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.448 INFO:teuthology.orchestra.run.smithi077.stdout: "max_concurrent_clones": { 2023-12-01T01:22:34.448 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_concurrent_clones", 2023-12-01T01:22:34.448 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.448 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.448 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.449 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "4", 2023-12-01T01:22:34.449 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.449 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.449 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.449 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Number of asynchronous cloner threads", 2023-12-01T01:22:34.449 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.449 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.449 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.449 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.449 INFO:teuthology.orchestra.run.smithi077.stdout: "snapshot_clone_delay": { 2023-12-01T01:22:34.449 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "snapshot_clone_delay", 2023-12-01T01:22:34.449 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.449 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.449 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.449 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2023-12-01T01:22:34.450 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.450 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.450 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.450 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-12-01T01:22:34.450 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.450 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.450 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.450 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.450 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.450 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.450 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.450 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix", 2023-12-01T01:22:34.450 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2023-12-01T01:22:34.450 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2023-12-01T01:22:34.451 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2023-12-01T01:22:34.451 INFO:teuthology.orchestra.run.smithi077.stdout: "discovery_interval": { 2023-12-01T01:22:34.451 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "discovery_interval", 2023-12-01T01:22:34.451 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2023-12-01T01:22:34.451 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.451 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.451 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "100", 2023-12-01T01:22:34.451 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.451 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.451 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.451 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.451 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.451 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.451 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.452 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.452 INFO:teuthology.orchestra.run.smithi077.stdout: "identifier": { 2023-12-01T01:22:34.452 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "identifier", 2023-12-01T01:22:34.452 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.452 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.452 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.452 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.452 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.452 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.452 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.452 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.452 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.452 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.452 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.453 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.453 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2023-12-01T01:22:34.453 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2023-12-01T01:22:34.453 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2023-12-01T01:22:34.453 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.453 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.453 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2023-12-01T01:22:34.453 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.453 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.453 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.453 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.453 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.453 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.453 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.453 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.454 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2023-12-01T01:22:34.454 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2023-12-01T01:22:34.454 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.454 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.454 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.454 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.454 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.454 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.454 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.454 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.454 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.454 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.454 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.454 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.455 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.455 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.455 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.455 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.455 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.455 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.455 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.455 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2023-12-01T01:22:34.455 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2023-12-01T01:22:34.455 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.455 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.455 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.455 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.455 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.455 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.456 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.456 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.456 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.456 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.456 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.456 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.456 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2023-12-01T01:22:34.456 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2023-12-01T01:22:34.456 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.456 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.456 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.456 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2023-12-01T01:22:34.456 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.456 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.457 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2023-12-01T01:22:34.457 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2023-12-01T01:22:34.457 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2023-12-01T01:22:34.457 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2023-12-01T01:22:34.457 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2023-12-01T01:22:34.457 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2023-12-01T01:22:34.457 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2023-12-01T01:22:34.457 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.457 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.457 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.457 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.457 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.457 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.457 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2023-12-01T01:22:34.458 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2023-12-01T01:22:34.458 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2023-12-01T01:22:34.458 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.458 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2023-12-01T01:22:34.458 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2023-12-01T01:22:34.458 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.458 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.458 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.458 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.458 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.458 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.458 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.458 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.458 INFO:teuthology.orchestra.run.smithi077.stdout: "zabbix_host": { 2023-12-01T01:22:34.459 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix_host", 2023-12-01T01:22:34.459 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.459 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.459 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.459 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2023-12-01T01:22:34.459 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.459 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.459 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.459 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.459 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.459 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.459 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.459 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.459 INFO:teuthology.orchestra.run.smithi077.stdout: "zabbix_port": { 2023-12-01T01:22:34.459 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix_port", 2023-12-01T01:22:34.460 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2023-12-01T01:22:34.460 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.460 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.460 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10051", 2023-12-01T01:22:34.460 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.460 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.460 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.460 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.460 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.460 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.460 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.460 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.460 INFO:teuthology.orchestra.run.smithi077.stdout: "zabbix_sender": { 2023-12-01T01:22:34.460 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix_sender", 2023-12-01T01:22:34.460 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2023-12-01T01:22:34.461 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2023-12-01T01:22:34.461 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2023-12-01T01:22:34.461 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-12-01T01:22:34.461 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2023-12-01T01:22:34.461 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2023-12-01T01:22:34.461 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2023-12-01T01:22:34.461 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2023-12-01T01:22:34.461 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2023-12-01T01:22:34.461 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2023-12-01T01:22:34.461 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2023-12-01T01:22:34.461 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.461 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.461 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.461 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.462 INFO:teuthology.orchestra.run.smithi077.stdout: "services": { 2023-12-01T01:22:34.462 INFO:teuthology.orchestra.run.smithi077.stdout: "dashboard": "https://172.21.15.77:8443/" 2023-12-01T01:22:34.462 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.462 INFO:teuthology.orchestra.run.smithi077.stdout: "always_on_modules": { 2023-12-01T01:22:34.462 INFO:teuthology.orchestra.run.smithi077.stdout: "octopus": [ 2023-12-01T01:22:34.462 INFO:teuthology.orchestra.run.smithi077.stdout: "balancer", 2023-12-01T01:22:34.462 INFO:teuthology.orchestra.run.smithi077.stdout: "crash", 2023-12-01T01:22:34.462 INFO:teuthology.orchestra.run.smithi077.stdout: "devicehealth", 2023-12-01T01:22:34.462 INFO:teuthology.orchestra.run.smithi077.stdout: "orchestrator", 2023-12-01T01:22:34.462 INFO:teuthology.orchestra.run.smithi077.stdout: "pg_autoscaler", 2023-12-01T01:22:34.462 INFO:teuthology.orchestra.run.smithi077.stdout: "progress", 2023-12-01T01:22:34.462 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_support", 2023-12-01T01:22:34.462 INFO:teuthology.orchestra.run.smithi077.stdout: "status", 2023-12-01T01:22:34.462 INFO:teuthology.orchestra.run.smithi077.stdout: "telemetry", 2023-12-01T01:22:34.462 INFO:teuthology.orchestra.run.smithi077.stdout: "volumes" 2023-12-01T01:22:34.463 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.463 INFO:teuthology.orchestra.run.smithi077.stdout: "pacific": [ 2023-12-01T01:22:34.463 INFO:teuthology.orchestra.run.smithi077.stdout: "balancer", 2023-12-01T01:22:34.463 INFO:teuthology.orchestra.run.smithi077.stdout: "crash", 2023-12-01T01:22:34.463 INFO:teuthology.orchestra.run.smithi077.stdout: "devicehealth", 2023-12-01T01:22:34.463 INFO:teuthology.orchestra.run.smithi077.stdout: "orchestrator", 2023-12-01T01:22:34.463 INFO:teuthology.orchestra.run.smithi077.stdout: "pg_autoscaler", 2023-12-01T01:22:34.463 INFO:teuthology.orchestra.run.smithi077.stdout: "progress", 2023-12-01T01:22:34.463 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_support", 2023-12-01T01:22:34.463 INFO:teuthology.orchestra.run.smithi077.stdout: "status", 2023-12-01T01:22:34.463 INFO:teuthology.orchestra.run.smithi077.stdout: "telemetry", 2023-12-01T01:22:34.463 INFO:teuthology.orchestra.run.smithi077.stdout: "volumes" 2023-12-01T01:22:34.463 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.463 INFO:teuthology.orchestra.run.smithi077.stdout: "quincy": [ 2023-12-01T01:22:34.464 INFO:teuthology.orchestra.run.smithi077.stdout: "balancer", 2023-12-01T01:22:34.464 INFO:teuthology.orchestra.run.smithi077.stdout: "crash", 2023-12-01T01:22:34.464 INFO:teuthology.orchestra.run.smithi077.stdout: "devicehealth", 2023-12-01T01:22:34.464 INFO:teuthology.orchestra.run.smithi077.stdout: "orchestrator", 2023-12-01T01:22:34.464 INFO:teuthology.orchestra.run.smithi077.stdout: "pg_autoscaler", 2023-12-01T01:22:34.464 INFO:teuthology.orchestra.run.smithi077.stdout: "progress", 2023-12-01T01:22:34.464 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_support", 2023-12-01T01:22:34.464 INFO:teuthology.orchestra.run.smithi077.stdout: "status", 2023-12-01T01:22:34.464 INFO:teuthology.orchestra.run.smithi077.stdout: "telemetry", 2023-12-01T01:22:34.464 INFO:teuthology.orchestra.run.smithi077.stdout: "volumes" 2023-12-01T01:22:34.464 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2023-12-01T01:22:34.464 INFO:teuthology.orchestra.run.smithi077.stdout: "reef": [ 2023-12-01T01:22:34.464 INFO:teuthology.orchestra.run.smithi077.stdout: "balancer", 2023-12-01T01:22:34.464 INFO:teuthology.orchestra.run.smithi077.stdout: "crash", 2023-12-01T01:22:34.465 INFO:teuthology.orchestra.run.smithi077.stdout: "devicehealth", 2023-12-01T01:22:34.465 INFO:teuthology.orchestra.run.smithi077.stdout: "orchestrator", 2023-12-01T01:22:34.465 INFO:teuthology.orchestra.run.smithi077.stdout: "pg_autoscaler", 2023-12-01T01:22:34.465 INFO:teuthology.orchestra.run.smithi077.stdout: "progress", 2023-12-01T01:22:34.465 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_support", 2023-12-01T01:22:34.465 INFO:teuthology.orchestra.run.smithi077.stdout: "status", 2023-12-01T01:22:34.465 INFO:teuthology.orchestra.run.smithi077.stdout: "telemetry", 2023-12-01T01:22:34.465 INFO:teuthology.orchestra.run.smithi077.stdout: "volumes" 2023-12-01T01:22:34.465 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2023-12-01T01:22:34.465 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.465 INFO:teuthology.orchestra.run.smithi077.stdout: "last_failure_osd_epoch": 3, 2023-12-01T01:22:34.465 INFO:teuthology.orchestra.run.smithi077.stdout: "active_clients": [ 2023-12-01T01:22:34.465 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.465 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "devicehealth", 2023-12-01T01:22:34.465 INFO:teuthology.orchestra.run.smithi077.stdout: "addrvec": [ 2023-12-01T01:22:34.466 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.466 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "v2", 2023-12-01T01:22:34.466 INFO:teuthology.orchestra.run.smithi077.stdout: "addr": "172.21.15.77:0", 2023-12-01T01:22:34.466 INFO:teuthology.orchestra.run.smithi077.stdout: "nonce": 1827570846 2023-12-01T01:22:34.466 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.466 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2023-12-01T01:22:34.466 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.466 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.466 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "libcephsqlite", 2023-12-01T01:22:34.466 INFO:teuthology.orchestra.run.smithi077.stdout: "addrvec": [ 2023-12-01T01:22:34.466 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.466 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "v2", 2023-12-01T01:22:34.466 INFO:teuthology.orchestra.run.smithi077.stdout: "addr": "172.21.15.77:0", 2023-12-01T01:22:34.466 INFO:teuthology.orchestra.run.smithi077.stdout: "nonce": 2170743527 2023-12-01T01:22:34.467 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.467 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2023-12-01T01:22:34.467 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.467 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.467 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rbd_support", 2023-12-01T01:22:34.467 INFO:teuthology.orchestra.run.smithi077.stdout: "addrvec": [ 2023-12-01T01:22:34.467 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.467 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "v2", 2023-12-01T01:22:34.467 INFO:teuthology.orchestra.run.smithi077.stdout: "addr": "172.21.15.77:0", 2023-12-01T01:22:34.467 INFO:teuthology.orchestra.run.smithi077.stdout: "nonce": 3740941141 2023-12-01T01:22:34.467 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.467 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2023-12-01T01:22:34.467 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:34.467 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.467 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "volumes", 2023-12-01T01:22:34.468 INFO:teuthology.orchestra.run.smithi077.stdout: "addrvec": [ 2023-12-01T01:22:34.468 INFO:teuthology.orchestra.run.smithi077.stdout: { 2023-12-01T01:22:34.468 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "v2", 2023-12-01T01:22:34.468 INFO:teuthology.orchestra.run.smithi077.stdout: "addr": "172.21.15.77:0", 2023-12-01T01:22:34.468 INFO:teuthology.orchestra.run.smithi077.stdout: "nonce": 738416770 2023-12-01T01:22:34.468 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.468 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2023-12-01T01:22:34.468 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:34.468 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2023-12-01T01:22:34.468 INFO:teuthology.orchestra.run.smithi077.stdout:} 2023-12-01T01:22:34.469 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:33 smithi137 ceph-mon[103312]: pgmap v438: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 511 B/s wr, 11 op/s 2023-12-01T01:22:34.470 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:33 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/1447368769' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-01T01:22:34.470 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:33 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/2024898664' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-12-01T01:22:34.470 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:33 smithi203 ceph-mon[103581]: pgmap v438: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 511 B/s wr, 11 op/s 2023-12-01T01:22:34.470 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:33 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/1447368769' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-01T01:22:34.470 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:33 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/2024898664' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-12-01T01:22:34.470 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:33.111+0000 7f45892d1700 1 -- 172.21.15.77:0/2024898664 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f456c0a0570 msgr2=0x7f456c0a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:34.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:33.111+0000 7f45892d1700 1 --2- 172.21.15.77:0/2024898664 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f456c0a0570 0x7f456c0a2a30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4578005950 tx=0x7f4578005f50 comp rx=0 tx=0).stop 2023-12-01T01:22:34.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:33.111+0000 7f45892d1700 1 -- 172.21.15.77:0/2024898664 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45840ff640 msgr2=0x7f4584198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:22:34.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:33.111+0000 7f45892d1700 1 --2- 172.21.15.77:0/2024898664 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45840ff640 0x7f4584198d00 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f457400d950 tx=0x7f457400dd10 comp rx=0 tx=0).stop 2023-12-01T01:22:34.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:33.111+0000 7f45892d1700 1 -- 172.21.15.77:0/2024898664 shutdown_connections 2023-12-01T01:22:34.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:33.111+0000 7f45892d1700 1 --2- 172.21.15.77:0/2024898664 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f456c0a0570 0x7f456c0a2a30 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:34.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:33.111+0000 7f45892d1700 1 --2- 172.21.15.77:0/2024898664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4584100960 0x7f45841a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:34.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:33.111+0000 7f45892d1700 1 --2- 172.21.15.77:0/2024898664 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45840ff640 0x7f4584198d00 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:34.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:33.111+0000 7f45892d1700 1 --2- 172.21.15.77:0/2024898664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45840fffa0 0x7f4584199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:22:34.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:33.111+0000 7f45892d1700 1 -- 172.21.15.77:0/2024898664 >> 172.21.15.77:0/2024898664 conn(0x7f45840fb1c0 msgr2=0x7f45840fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:22:34.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:33.112+0000 7f45892d1700 1 -- 172.21.15.77:0/2024898664 shutdown_connections 2023-12-01T01:22:34.471 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:22:33.112+0000 7f45892d1700 1 -- 172.21.15.77:0/2024898664 wait complete. 2023-12-01T01:22:34.477 DEBUG:tasks.check_counter:Getting stats from a 2023-12-01T01:22:34.477 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph daemon mds.a perf dump 2023-12-01T01:22:34.478 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:33 smithi077 ceph-mon[98867]: pgmap v438: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 511 B/s wr, 11 op/s 2023-12-01T01:22:34.478 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:33 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/1447368769' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-01T01:22:34.478 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:33 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/2024898664' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-12-01T01:22:35.038 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:22:35.526 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:35 smithi077 ceph-mon[98867]: pgmap v439: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 85 B/s wr, 12 op/s 2023-12-01T01:22:35.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:35 smithi203 ceph-mon[103581]: pgmap v439: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 85 B/s wr, 12 op/s 2023-12-01T01:22:35.778 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:35 smithi137 ceph-mon[103312]: pgmap v439: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 85 B/s wr, 12 op/s 2023-12-01T01:22:36.280 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2023-12-01T01:22:36.280 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-0": { 2023-12-01T01:22:36.281 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 155537, 2023-12-01T01:22:36.281 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 155181, 2023-12-01T01:22:36.281 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 28095549, 2023-12-01T01:22:36.281 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 28269324, 2023-12-01T01:22:36.281 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 11, 2023-12-01T01:22:36.281 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 4, 2023-12-01T01:22:36.281 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 5.455795029, 2023-12-01T01:22:36.281 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 2.169671229, 2023-12-01T01:22:36.282 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 3.952249188, 2023-12-01T01:22:36.282 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.014654336, 2023-12-01T01:22:36.282 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2023-12-01T01:22:36.282 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 155179, 2023-12-01T01:22:36.282 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 1.868748960, 2023-12-01T01:22:36.282 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000012042 2023-12-01T01:22:36.282 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.282 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2023-12-01T01:22:36.282 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 309415, 2023-12-01T01:22:36.283 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.030859637, 2023-12-01T01:22:36.283 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000099 2023-12-01T01:22:36.283 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.283 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 0, 2023-12-01T01:22:36.283 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 0 2023-12-01T01:22:36.283 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.283 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-1": { 2023-12-01T01:22:36.284 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 693502, 2023-12-01T01:22:36.284 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 696185, 2023-12-01T01:22:36.284 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 119343880, 2023-12-01T01:22:36.284 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 113033062, 2023-12-01T01:22:36.284 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 14, 2023-12-01T01:22:36.284 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 5, 2023-12-01T01:22:36.284 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 22.890825500, 2023-12-01T01:22:36.284 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 9.078979595, 2023-12-01T01:22:36.284 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 16.598250813, 2023-12-01T01:22:36.284 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.010867991, 2023-12-01T01:22:36.284 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2023-12-01T01:22:36.285 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 696184, 2023-12-01T01:22:36.285 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 7.822012939, 2023-12-01T01:22:36.285 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000011235 2023-12-01T01:22:36.285 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.285 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2023-12-01T01:22:36.285 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1386820, 2023-12-01T01:22:36.285 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.133340975, 2023-12-01T01:22:36.285 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000096 2023-12-01T01:22:36.286 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.286 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 1588784, 2023-12-01T01:22:36.286 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 3731296 2023-12-01T01:22:36.286 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.286 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-2": { 2023-12-01T01:22:36.286 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 1156, 2023-12-01T01:22:36.286 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 1080, 2023-12-01T01:22:36.286 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 379190, 2023-12-01T01:22:36.286 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 1981288, 2023-12-01T01:22:36.286 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 15, 2023-12-01T01:22:36.286 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 7, 2023-12-01T01:22:36.286 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.068061606, 2023-12-01T01:22:36.287 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.025482327, 2023-12-01T01:22:36.287 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.031535593, 2023-12-01T01:22:36.287 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.012610025, 2023-12-01T01:22:36.287 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2023-12-01T01:22:36.287 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1079, 2023-12-01T01:22:36.287 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.077859480, 2023-12-01T01:22:36.287 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000072158 2023-12-01T01:22:36.287 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.287 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2023-12-01T01:22:36.287 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1070, 2023-12-01T01:22:36.287 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000142881, 2023-12-01T01:22:36.287 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000133 2023-12-01T01:22:36.287 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.288 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 3152, 2023-12-01T01:22:36.288 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 214928 2023-12-01T01:22:36.288 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.288 INFO:teuthology.orchestra.run.smithi077.stdout: "cct": { 2023-12-01T01:22:36.288 INFO:teuthology.orchestra.run.smithi077.stdout: "total_workers": 1, 2023-12-01T01:22:36.288 INFO:teuthology.orchestra.run.smithi077.stdout: "unhealthy_workers": 0 2023-12-01T01:22:36.288 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.288 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-MDSRank": { 2023-12-01T01:22:36.288 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2023-12-01T01:22:36.288 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2023-12-01T01:22:36.288 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 385, 2023-12-01T01:22:36.288 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 1.254476348, 2023-12-01T01:22:36.288 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.003258380 2023-12-01T01:22:36.288 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:36.289 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.289 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-PurgeQueue": { 2023-12-01T01:22:36.289 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2023-12-01T01:22:36.289 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2023-12-01T01:22:36.289 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1686, 2023-12-01T01:22:36.289 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.321482955, 2023-12-01T01:22:36.289 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000190677 2023-12-01T01:22:36.289 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:36.289 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.289 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2023-12-01T01:22:36.289 INFO:teuthology.orchestra.run.smithi077.stdout: "request": 2240, 2023-12-01T01:22:36.289 INFO:teuthology.orchestra.run.smithi077.stdout: "reply": 2237, 2023-12-01T01:22:36.290 INFO:teuthology.orchestra.run.smithi077.stdout: "reply_latency": { 2023-12-01T01:22:36.290 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 2237, 2023-12-01T01:22:36.290 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 1.633245601, 2023-12-01T01:22:36.290 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000730105 2023-12-01T01:22:36.290 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.290 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_reply": 0, 2023-12-01T01:22:36.290 INFO:teuthology.orchestra.run.smithi077.stdout: "forward": 3, 2023-12-01T01:22:36.290 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_complete": 0, 2023-12-01T01:22:36.290 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_keys": 0, 2023-12-01T01:22:36.290 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_commit": 11, 2023-12-01T01:22:36.290 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_split": 0, 2023-12-01T01:22:36.290 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_merge": 0, 2023-12-01T01:22:36.290 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes": 20, 2023-12-01T01:22:36.290 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_top": 2, 2023-12-01T01:22:36.291 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_bottom": 8, 2023-12-01T01:22:36.291 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pin_tail": 10, 2023-12-01T01:22:36.291 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pinned": 14, 2023-12-01T01:22:36.291 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_expired": 498, 2023-12-01T01:22:36.291 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_with_caps": 1, 2023-12-01T01:22:36.291 INFO:teuthology.orchestra.run.smithi077.stdout: "caps": 1, 2023-12-01T01:22:36.291 INFO:teuthology.orchestra.run.smithi077.stdout: "subtrees": 5, 2023-12-01T01:22:36.291 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse": 3729, 2023-12-01T01:22:36.291 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_hit": 3543, 2023-12-01T01:22:36.291 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_forward": 3, 2023-12-01T01:22:36.291 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_discover": 123, 2023-12-01T01:22:36.291 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_dir_fetch": 0, 2023-12-01T01:22:36.291 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_remote_ino": 0, 2023-12-01T01:22:36.291 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_lock": 4, 2023-12-01T01:22:36.292 INFO:teuthology.orchestra.run.smithi077.stdout: "load_cent": 8, 2023-12-01T01:22:36.292 INFO:teuthology.orchestra.run.smithi077.stdout: "q": 0, 2023-12-01T01:22:36.292 INFO:teuthology.orchestra.run.smithi077.stdout: "exported": 50, 2023-12-01T01:22:36.292 INFO:teuthology.orchestra.run.smithi077.stdout: "exported_inodes": 271, 2023-12-01T01:22:36.292 INFO:teuthology.orchestra.run.smithi077.stdout: "imported": 50, 2023-12-01T01:22:36.292 INFO:teuthology.orchestra.run.smithi077.stdout: "imported_inodes": 1037, 2023-12-01T01:22:36.292 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_dir_fetch": 0, 2023-12-01T01:22:36.292 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_backtrace_fetch": 211, 2023-12-01T01:22:36.292 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_peer_discover": 0, 2023-12-01T01:22:36.292 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rfiles": 3568, 2023-12-01T01:22:36.292 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rbytes": 50854799, 2023-12-01T01:22:36.292 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rsnaps": 0, 2023-12-01T01:22:36.292 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_fetch": 211, 2023-12-01T01:22:36.292 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_set_tag": 0, 2023-12-01T01:22:36.292 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_repaired": 0, 2023-12-01T01:22:36.293 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_inotable_repaired": 0, 2023-12-01T01:22:36.293 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_inodes": 0, 2023-12-01T01:22:36.293 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_base_inodes": 0, 2023-12-01T01:22:36.293 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dirfrag_rstats": 0, 2023-12-01T01:22:36.293 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_file_inodes": 211, 2023-12-01T01:22:36.293 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_inode_file_caps": 0, 2023-12-01T01:22:36.293 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_revoke": 8, 2023-12-01T01:22:36.293 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_grant": 1111, 2023-12-01T01:22:36.293 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_trunc": 0, 2023-12-01T01:22:36.293 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-01T01:22:36.293 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-01T01:22:36.293 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps": 636, 2023-12-01T01:22:36.293 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps_dirty": 629, 2023-12-01T01:22:36.293 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_cap_release": 17, 2023-12-01T01:22:36.294 INFO:teuthology.orchestra.run.smithi077.stdout: "process_request_cap_release": 1960 2023-12-01T01:22:36.294 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.294 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_cache": { 2023-12-01T01:22:36.294 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update": 158, 2023-12-01T01:22:36.294 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update_receipt": 497, 2023-12-01T01:22:36.294 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_try_discover": 295, 2023-12-01T01:22:36.294 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_send_discover": 416, 2023-12-01T01:22:36.294 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_handle_discover": 190, 2023-12-01T01:22:36.294 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays": 0, 2023-12-01T01:22:36.294 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_delayed": 0, 2023-12-01T01:22:36.294 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_enqueuing": 0, 2023-12-01T01:22:36.294 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_created": 958, 2023-12-01T01:22:36.294 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_enqueued": 958, 2023-12-01T01:22:36.295 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_reintegrated": 0, 2023-12-01T01:22:36.295 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_migrated": 0, 2023-12-01T01:22:36.295 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_processing": 0, 2023-12-01T01:22:36.295 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_enqueued": 0, 2023-12-01T01:22:36.295 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_prioritized": 0, 2023-12-01T01:22:36.295 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_started": 0, 2023-12-01T01:22:36.295 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_completed": 0, 2023-12-01T01:22:36.295 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_enqueue_scrub": 0, 2023-12-01T01:22:36.295 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_exportdir": 50, 2023-12-01T01:22:36.295 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_flush": 0, 2023-12-01T01:22:36.295 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragmentdir": 0, 2023-12-01T01:22:36.295 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragstats": 0, 2023-12-01T01:22:36.295 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_inodestats": 0 2023-12-01T01:22:36.296 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.296 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_log": { 2023-12-01T01:22:36.296 INFO:teuthology.orchestra.run.smithi077.stdout: "evadd": 3439, 2023-12-01T01:22:36.296 INFO:teuthology.orchestra.run.smithi077.stdout: "evex": 0, 2023-12-01T01:22:36.296 INFO:teuthology.orchestra.run.smithi077.stdout: "evtrm": 0, 2023-12-01T01:22:36.296 INFO:teuthology.orchestra.run.smithi077.stdout: "ev": 3439, 2023-12-01T01:22:36.296 INFO:teuthology.orchestra.run.smithi077.stdout: "evexg": 0, 2023-12-01T01:22:36.296 INFO:teuthology.orchestra.run.smithi077.stdout: "evexd": 0, 2023-12-01T01:22:36.296 INFO:teuthology.orchestra.run.smithi077.stdout: "segadd": 5, 2023-12-01T01:22:36.296 INFO:teuthology.orchestra.run.smithi077.stdout: "segex": 0, 2023-12-01T01:22:36.296 INFO:teuthology.orchestra.run.smithi077.stdout: "segtrm": 0, 2023-12-01T01:22:36.296 INFO:teuthology.orchestra.run.smithi077.stdout: "seg": 5, 2023-12-01T01:22:36.296 INFO:teuthology.orchestra.run.smithi077.stdout: "segexg": 0, 2023-12-01T01:22:36.296 INFO:teuthology.orchestra.run.smithi077.stdout: "segexd": 0, 2023-12-01T01:22:36.297 INFO:teuthology.orchestra.run.smithi077.stdout: "expos": 4194304, 2023-12-01T01:22:36.297 INFO:teuthology.orchestra.run.smithi077.stdout: "wrpos": 10051537, 2023-12-01T01:22:36.297 INFO:teuthology.orchestra.run.smithi077.stdout: "rdpos": 0, 2023-12-01T01:22:36.297 INFO:teuthology.orchestra.run.smithi077.stdout: "jlat": { 2023-12-01T01:22:36.297 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 165, 2023-12-01T01:22:36.297 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 2.537841031, 2023-12-01T01:22:36.297 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.015380854 2023-12-01T01:22:36.297 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.297 INFO:teuthology.orchestra.run.smithi077.stdout: "replayed": 0 2023-12-01T01:22:36.297 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.297 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_mem": { 2023-12-01T01:22:36.297 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 26, 2023-12-01T01:22:36.297 INFO:teuthology.orchestra.run.smithi077.stdout: "ino+": 1138, 2023-12-01T01:22:36.297 INFO:teuthology.orchestra.run.smithi077.stdout: "ino-": 1112, 2023-12-01T01:22:36.298 INFO:teuthology.orchestra.run.smithi077.stdout: "dir": 25, 2023-12-01T01:22:36.298 INFO:teuthology.orchestra.run.smithi077.stdout: "dir+": 292, 2023-12-01T01:22:36.298 INFO:teuthology.orchestra.run.smithi077.stdout: "dir-": 267, 2023-12-01T01:22:36.298 INFO:teuthology.orchestra.run.smithi077.stdout: "dn": 20, 2023-12-01T01:22:36.298 INFO:teuthology.orchestra.run.smithi077.stdout: "dn+": 2163, 2023-12-01T01:22:36.298 INFO:teuthology.orchestra.run.smithi077.stdout: "dn-": 2143, 2023-12-01T01:22:36.298 INFO:teuthology.orchestra.run.smithi077.stdout: "cap": 1, 2023-12-01T01:22:36.298 INFO:teuthology.orchestra.run.smithi077.stdout: "cap+": 1774, 2023-12-01T01:22:36.298 INFO:teuthology.orchestra.run.smithi077.stdout: "cap-": 1773, 2023-12-01T01:22:36.298 INFO:teuthology.orchestra.run.smithi077.stdout: "rss": 66084, 2023-12-01T01:22:36.298 INFO:teuthology.orchestra.run.smithi077.stdout: "heap": 198964 2023-12-01T01:22:36.298 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.298 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_server": { 2023-12-01T01:22:36.299 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_client_request": 2394, 2023-12-01T01:22:36.299 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_server_request": 21, 2023-12-01T01:22:36.299 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_request": 2240, 2023-12-01T01:22:36.299 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_session": 4, 2023-12-01T01:22:36.299 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_peer_request": 51, 2023-12-01T01:22:36.299 INFO:teuthology.orchestra.run.smithi077.stdout: "req_create_latency": { 2023-12-01T01:22:36.299 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 147, 2023-12-01T01:22:36.299 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.093389874, 2023-12-01T01:22:36.299 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000635305 2023-12-01T01:22:36.299 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.299 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getattr_latency": { 2023-12-01T01:22:36.299 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:36.299 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:36.300 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:36.300 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.300 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getfilelock_latency": { 2023-12-01T01:22:36.300 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:36.300 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:36.300 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:36.300 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.300 INFO:teuthology.orchestra.run.smithi077.stdout: "req_link_latency": { 2023-12-01T01:22:36.300 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:36.300 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:36.301 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:36.301 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.301 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookup_latency": { 2023-12-01T01:22:36.301 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1086, 2023-12-01T01:22:36.301 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.363349780, 2023-12-01T01:22:36.301 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000334576 2023-12-01T01:22:36.301 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.301 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookuphash_latency": { 2023-12-01T01:22:36.301 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:36.301 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:36.301 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:36.301 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.302 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupino_latency": { 2023-12-01T01:22:36.302 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1, 2023-12-01T01:22:36.302 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000619441, 2023-12-01T01:22:36.302 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000619441 2023-12-01T01:22:36.302 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.302 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupname_latency": { 2023-12-01T01:22:36.302 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:36.302 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:36.302 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:36.302 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.302 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupparent_latency": { 2023-12-01T01:22:36.302 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:36.302 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:36.302 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:36.303 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.303 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupsnap_latency": { 2023-12-01T01:22:36.303 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:36.303 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:36.303 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:36.303 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.303 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lssnap_latency": { 2023-12-01T01:22:36.303 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:36.303 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:36.303 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:36.303 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.303 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mkdir_latency": { 2023-12-01T01:22:36.303 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:36.303 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:36.304 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:36.304 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.304 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mknod_latency": { 2023-12-01T01:22:36.304 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:36.304 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:36.304 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:36.304 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.304 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mksnap_latency": { 2023-12-01T01:22:36.304 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:36.304 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:36.304 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:36.304 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.304 INFO:teuthology.orchestra.run.smithi077.stdout: "req_open_latency": { 2023-12-01T01:22:36.304 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:36.304 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:36.305 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:36.305 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.305 INFO:teuthology.orchestra.run.smithi077.stdout: "req_readdir_latency": { 2023-12-01T01:22:36.305 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 45, 2023-12-01T01:22:36.305 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.027937603, 2023-12-01T01:22:36.305 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000620835 2023-12-01T01:22:36.305 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.305 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rename_latency": { 2023-12-01T01:22:36.305 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:36.305 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:36.305 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:36.305 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.305 INFO:teuthology.orchestra.run.smithi077.stdout: "req_renamesnap_latency": { 2023-12-01T01:22:36.305 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:36.305 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:36.306 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:36.306 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.306 INFO:teuthology.orchestra.run.smithi077.stdout: "req_snapdiff_latency": { 2023-12-01T01:22:36.306 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:36.306 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:36.306 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:36.306 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.306 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmdir_latency": { 2023-12-01T01:22:36.306 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:36.306 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:36.306 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:36.306 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.306 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmsnap_latency": { 2023-12-01T01:22:36.306 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:36.307 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:36.307 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:36.307 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.307 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmxattr_latency": { 2023-12-01T01:22:36.307 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:36.307 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:36.307 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:36.307 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.307 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setattr_latency": { 2023-12-01T01:22:36.307 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:36.307 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:36.307 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:36.307 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.307 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setdirlayout_latency": { 2023-12-01T01:22:36.308 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:36.308 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:36.308 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:36.308 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.308 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setfilelock_latency": { 2023-12-01T01:22:36.308 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:36.308 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:36.308 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:36.308 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.308 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setlayout_latency": { 2023-12-01T01:22:36.308 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:36.308 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:36.308 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:36.309 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.309 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setxattr_latency": { 2023-12-01T01:22:36.309 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:36.309 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:36.309 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:36.309 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.309 INFO:teuthology.orchestra.run.smithi077.stdout: "req_symlink_latency": { 2023-12-01T01:22:36.309 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:36.309 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:36.309 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:36.309 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.309 INFO:teuthology.orchestra.run.smithi077.stdout: "req_unlink_latency": { 2023-12-01T01:22:36.309 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 958, 2023-12-01T01:22:36.309 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 1.147948903, 2023-12-01T01:22:36.310 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.001198276 2023-12-01T01:22:36.310 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.310 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_revoke_eviction": 0, 2023-12-01T01:22:36.310 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_acquisition_throttle": 0, 2023-12-01T01:22:36.310 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getvxattr_latency": { 2023-12-01T01:22:36.310 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:36.310 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:36.310 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:36.310 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:36.310 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.310 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_sessions": { 2023-12-01T01:22:36.311 INFO:teuthology.orchestra.run.smithi077.stdout: "session_count": 1, 2023-12-01T01:22:36.311 INFO:teuthology.orchestra.run.smithi077.stdout: "session_add": 1, 2023-12-01T01:22:36.311 INFO:teuthology.orchestra.run.smithi077.stdout: "session_remove": 0, 2023-12-01T01:22:36.311 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_open": 1, 2023-12-01T01:22:36.311 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_stale": 0, 2023-12-01T01:22:36.311 INFO:teuthology.orchestra.run.smithi077.stdout: "total_load": 1571, 2023-12-01T01:22:36.311 INFO:teuthology.orchestra.run.smithi077.stdout: "average_load": 1571, 2023-12-01T01:22:36.311 INFO:teuthology.orchestra.run.smithi077.stdout: "avg_session_uptime": 280, 2023-12-01T01:22:36.311 INFO:teuthology.orchestra.run.smithi077.stdout: "mdthresh_evicted": 0 2023-12-01T01:22:36.311 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.311 INFO:teuthology.orchestra.run.smithi077.stdout: "mempool": { 2023-12-01T01:22:36.311 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_bytes": 0, 2023-12-01T01:22:36.311 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_items": 0, 2023-12-01T01:22:36.312 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_bytes": 0, 2023-12-01T01:22:36.312 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_items": 0, 2023-12-01T01:22:36.312 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_bytes": 0, 2023-12-01T01:22:36.312 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_items": 0, 2023-12-01T01:22:36.312 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-01T01:22:36.312 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_items": 0, 2023-12-01T01:22:36.312 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-01T01:22:36.312 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_items": 0, 2023-12-01T01:22:36.312 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_bytes": 0, 2023-12-01T01:22:36.312 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_items": 0, 2023-12-01T01:22:36.312 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-01T01:22:36.312 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_items": 0, 2023-12-01T01:22:36.312 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_bytes": 0, 2023-12-01T01:22:36.312 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_items": 0, 2023-12-01T01:22:36.313 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_bytes": 0, 2023-12-01T01:22:36.313 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_items": 0, 2023-12-01T01:22:36.313 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-01T01:22:36.313 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_items": 0, 2023-12-01T01:22:36.313 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-01T01:22:36.313 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_items": 0, 2023-12-01T01:22:36.313 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_bytes": 0, 2023-12-01T01:22:36.313 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_items": 0, 2023-12-01T01:22:36.313 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_bytes": 0, 2023-12-01T01:22:36.313 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_items": 0, 2023-12-01T01:22:36.313 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-01T01:22:36.313 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_items": 0, 2023-12-01T01:22:36.313 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_bytes": 0, 2023-12-01T01:22:36.314 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_items": 0, 2023-12-01T01:22:36.314 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_bytes": 0, 2023-12-01T01:22:36.314 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_items": 0, 2023-12-01T01:22:36.314 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_bytes": 0, 2023-12-01T01:22:36.314 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_items": 0, 2023-12-01T01:22:36.314 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_bytes": 0, 2023-12-01T01:22:36.314 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_items": 0, 2023-12-01T01:22:36.314 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_bytes": 307625, 2023-12-01T01:22:36.314 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_items": 133, 2023-12-01T01:22:36.314 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_bytes": 880, 2023-12-01T01:22:36.314 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_items": 10, 2023-12-01T01:22:36.314 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_bytes": 0, 2023-12-01T01:22:36.315 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_items": 0, 2023-12-01T01:22:36.315 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_bytes": 0, 2023-12-01T01:22:36.315 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_items": 0, 2023-12-01T01:22:36.315 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_bytes": 0, 2023-12-01T01:22:36.315 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_items": 0, 2023-12-01T01:22:36.315 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_bytes": 7336, 2023-12-01T01:22:36.315 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_items": 154, 2023-12-01T01:22:36.315 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_bytes": 0, 2023-12-01T01:22:36.315 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_items": 0, 2023-12-01T01:22:36.315 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_bytes": 0, 2023-12-01T01:22:36.315 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_items": 0, 2023-12-01T01:22:36.315 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_bytes": 349942, 2023-12-01T01:22:36.315 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_items": 3314, 2023-12-01T01:22:36.316 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_bytes": 0, 2023-12-01T01:22:36.316 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_items": 0, 2023-12-01T01:22:36.316 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_bytes": 0, 2023-12-01T01:22:36.316 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_items": 0 2023-12-01T01:22:36.316 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.316 INFO:teuthology.orchestra.run.smithi077.stdout: "objecter": { 2023-12-01T01:22:36.316 INFO:teuthology.orchestra.run.smithi077.stdout: "op_active": 0, 2023-12-01T01:22:36.316 INFO:teuthology.orchestra.run.smithi077.stdout: "op_laggy": 0, 2023-12-01T01:22:36.316 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send": 1438, 2023-12-01T01:22:36.316 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send_bytes": 6038975, 2023-12-01T01:22:36.316 INFO:teuthology.orchestra.run.smithi077.stdout: "op_resend": 33, 2023-12-01T01:22:36.316 INFO:teuthology.orchestra.run.smithi077.stdout: "op_reply": 1438, 2023-12-01T01:22:36.316 INFO:teuthology.orchestra.run.smithi077.stdout: "op_latency": { 2023-12-01T01:22:36.316 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1438, 2023-12-01T01:22:36.317 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 4.891885788, 2023-12-01T01:22:36.317 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.003401867 2023-12-01T01:22:36.317 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.317 INFO:teuthology.orchestra.run.smithi077.stdout: "op_inflight": 0, 2023-12-01T01:22:36.317 INFO:teuthology.orchestra.run.smithi077.stdout: "oplen_avg": { 2023-12-01T01:22:36.317 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1438, 2023-12-01T01:22:36.317 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 1476 2023-12-01T01:22:36.317 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.317 INFO:teuthology.orchestra.run.smithi077.stdout: "op": 1438, 2023-12-01T01:22:36.317 INFO:teuthology.orchestra.run.smithi077.stdout: "op_r": 229, 2023-12-01T01:22:36.318 INFO:teuthology.orchestra.run.smithi077.stdout: "op_w": 1209, 2023-12-01T01:22:36.318 INFO:teuthology.orchestra.run.smithi077.stdout: "op_rmw": 0, 2023-12-01T01:22:36.318 INFO:teuthology.orchestra.run.smithi077.stdout: "op_pg": 0, 2023-12-01T01:22:36.318 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_stat": 0, 2023-12-01T01:22:36.318 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_create": 10, 2023-12-01T01:22:36.318 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_read": 18, 2023-12-01T01:22:36.318 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_write": 183, 2023-12-01T01:22:36.318 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writefull": 19, 2023-12-01T01:22:36.318 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writesame": 0, 2023-12-01T01:22:36.318 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_append": 0, 2023-12-01T01:22:36.318 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_zero": 0, 2023-12-01T01:22:36.318 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_truncate": 0, 2023-12-01T01:22:36.318 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_delete": 971, 2023-12-01T01:22:36.319 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_mapext": 0, 2023-12-01T01:22:36.319 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_sparse_read": 0, 2023-12-01T01:22:36.319 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_clonerange": 0, 2023-12-01T01:22:36.319 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_getxattr": 211, 2023-12-01T01:22:36.319 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_setxattr": 20, 2023-12-01T01:22:36.319 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_cmpxattr": 0, 2023-12-01T01:22:36.319 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_rmxattr": 0, 2023-12-01T01:22:36.319 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_resetxattrs": 0, 2023-12-01T01:22:36.319 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_call": 0, 2023-12-01T01:22:36.319 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_watch": 0, 2023-12-01T01:22:36.319 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_notify": 0, 2023-12-01T01:22:36.319 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_src_cmpxattr": 0, 2023-12-01T01:22:36.319 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls": 0, 2023-12-01T01:22:36.319 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls_filter": 0, 2023-12-01T01:22:36.320 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_other": 0, 2023-12-01T01:22:36.320 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_active": 0, 2023-12-01T01:22:36.320 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_send": 0, 2023-12-01T01:22:36.320 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_resend": 0, 2023-12-01T01:22:36.320 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_ping": 0, 2023-12-01T01:22:36.320 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_active": 0, 2023-12-01T01:22:36.320 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_send": 0, 2023-12-01T01:22:36.320 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_resend": 0, 2023-12-01T01:22:36.320 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_active": 0, 2023-12-01T01:22:36.320 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_send": 0, 2023-12-01T01:22:36.320 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_resend": 0, 2023-12-01T01:22:36.320 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_active": 0, 2023-12-01T01:22:36.320 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_send": 0, 2023-12-01T01:22:36.320 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_resend": 0, 2023-12-01T01:22:36.321 INFO:teuthology.orchestra.run.smithi077.stdout: "command_active": 0, 2023-12-01T01:22:36.321 INFO:teuthology.orchestra.run.smithi077.stdout: "command_send": 0, 2023-12-01T01:22:36.321 INFO:teuthology.orchestra.run.smithi077.stdout: "command_resend": 0, 2023-12-01T01:22:36.321 INFO:teuthology.orchestra.run.smithi077.stdout: "map_epoch": 84, 2023-12-01T01:22:36.321 INFO:teuthology.orchestra.run.smithi077.stdout: "map_full": 0, 2023-12-01T01:22:36.321 INFO:teuthology.orchestra.run.smithi077.stdout: "map_inc": 2, 2023-12-01T01:22:36.321 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_sessions": 12, 2023-12-01T01:22:36.321 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_open": 12, 2023-12-01T01:22:36.321 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_close": 0, 2023-12-01T01:22:36.321 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_laggy": 0, 2023-12-01T01:22:36.321 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_wr": 35, 2023-12-01T01:22:36.321 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_rd": 0, 2023-12-01T01:22:36.321 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_del": 9 2023-12-01T01:22:36.321 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.322 INFO:teuthology.orchestra.run.smithi077.stdout: "oft": { 2023-12-01T01:22:36.322 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_objs": 1, 2023-12-01T01:22:36.322 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_kv_pairs": 0, 2023-12-01T01:22:36.322 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_updates": 0, 2023-12-01T01:22:36.322 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_removes": 0 2023-12-01T01:22:36.322 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.322 INFO:teuthology.orchestra.run.smithi077.stdout: "osd-slow-ops": { 2023-12-01T01:22:36.322 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_ops_count": 2311 2023-12-01T01:22:36.322 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.322 INFO:teuthology.orchestra.run.smithi077.stdout: "purge_queue": { 2023-12-01T01:22:36.323 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops": 0, 2023-12-01T01:22:36.323 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops_high_water": 6, 2023-12-01T01:22:36.323 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing": 0, 2023-12-01T01:22:36.323 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_high_water": 6, 2023-12-01T01:22:36.323 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executed": 958, 2023-12-01T01:22:36.323 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_item_in_journal": 0 2023-12-01T01:22:36.323 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.323 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-01T01:22:36.323 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2023-12-01T01:22:36.323 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2023-12-01T01:22:36.323 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2023-12-01T01:22:36.323 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 850191, 2023-12-01T01:22:36.323 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 71160684, 2023-12-01T01:22:36.323 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:36.323 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 850191, 2023-12-01T01:22:36.324 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2023-12-01T01:22:36.324 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2023-12-01T01:22:36.324 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 850191, 2023-12-01T01:22:36.324 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 71160684, 2023-12-01T01:22:36.324 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2023-12-01T01:22:36.324 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:36.324 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:36.324 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:36.324 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:36.324 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.324 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_bytes": { 2023-12-01T01:22:36.324 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2023-12-01T01:22:36.324 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2023-12-01T01:22:36.324 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2023-12-01T01:22:36.324 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2023-12-01T01:22:36.325 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2023-12-01T01:22:36.325 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:36.325 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2023-12-01T01:22:36.325 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 1438, 2023-12-01T01:22:36.325 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 6135733, 2023-12-01T01:22:36.325 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 1438, 2023-12-01T01:22:36.325 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 6135733, 2023-12-01T01:22:36.325 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2023-12-01T01:22:36.325 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:36.325 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:36.325 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:36.325 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:36.325 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.325 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_ops": { 2023-12-01T01:22:36.326 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2023-12-01T01:22:36.326 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 1024, 2023-12-01T01:22:36.326 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2023-12-01T01:22:36.326 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2023-12-01T01:22:36.326 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2023-12-01T01:22:36.326 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:36.326 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2023-12-01T01:22:36.326 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 1438, 2023-12-01T01:22:36.326 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 1438, 2023-12-01T01:22:36.326 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 1438, 2023-12-01T01:22:36.326 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 1438, 2023-12-01T01:22:36.326 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2023-12-01T01:22:36.326 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:36.326 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:36.327 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:36.327 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:36.327 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.327 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle": { 2023-12-01T01:22:36.327 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2023-12-01T01:22:36.327 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2023-12-01T01:22:36.327 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2023-12-01T01:22:36.327 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 958, 2023-12-01T01:22:36.327 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 96758, 2023-12-01T01:22:36.327 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:36.327 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 958, 2023-12-01T01:22:36.327 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2023-12-01T01:22:36.327 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2023-12-01T01:22:36.328 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 18, 2023-12-01T01:22:36.328 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 96758, 2023-12-01T01:22:36.328 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2023-12-01T01:22:36.328 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:36.328 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:36.328 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:36.328 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:36.328 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:36.328 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle-0x55b7b910e0a0": { 2023-12-01T01:22:36.328 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2023-12-01T01:22:36.328 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2023-12-01T01:22:36.328 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2023-12-01T01:22:36.328 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 3439, 2023-12-01T01:22:36.329 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 5857233, 2023-12-01T01:22:36.329 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:36.329 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 3439, 2023-12-01T01:22:36.329 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2023-12-01T01:22:36.329 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2023-12-01T01:22:36.329 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 165, 2023-12-01T01:22:36.329 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 5857233, 2023-12-01T01:22:36.329 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2023-12-01T01:22:36.329 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:36.329 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:36.329 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:36.329 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:36.329 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:36.329 INFO:teuthology.orchestra.run.smithi077.stdout:} 2023-12-01T01:22:36.701 INFO:tasks.check_counter:Daemon mds.a mds.exported=50 2023-12-01T01:22:36.701 INFO:tasks.check_counter:Daemon mds.a mds.imported=50 2023-12-01T01:22:36.702 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=158 2023-12-01T01:22:36.702 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=497 2023-12-01T01:22:36.702 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2023-12-01T01:22:36.702 DEBUG:tasks.check_counter:Getting stats from d 2023-12-01T01:22:36.702 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph daemon mds.d perf dump 2023-12-01T01:22:37.257 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:22:37.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:37 smithi203 ceph-mon[103581]: pgmap v440: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 11 op/s 2023-12-01T01:22:37.767 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:37 smithi077 ceph-mon[98867]: pgmap v440: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 11 op/s 2023-12-01T01:22:37.778 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:37 smithi137 ceph-mon[103312]: pgmap v440: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 11 op/s 2023-12-01T01:22:38.758 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2023-12-01T01:22:38.758 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-0": { 2023-12-01T01:22:38.759 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 4, 2023-12-01T01:22:38.759 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 4, 2023-12-01T01:22:38.759 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 2992, 2023-12-01T01:22:38.759 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 448, 2023-12-01T01:22:38.759 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 4, 2023-12-01T01:22:38.759 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 0, 2023-12-01T01:22:38.759 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.001559742, 2023-12-01T01:22:38.759 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.000034733, 2023-12-01T01:22:38.759 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.000612118, 2023-12-01T01:22:38.760 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-01T01:22:38.760 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2023-12-01T01:22:38.760 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 3, 2023-12-01T01:22:38.760 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000659156, 2023-12-01T01:22:38.760 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000219718 2023-12-01T01:22:38.760 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:38.760 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2023-12-01T01:22:38.760 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:38.760 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:38.760 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:38.761 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:38.761 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-12-01T01:22:38.761 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 448 2023-12-01T01:22:38.761 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:38.761 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-1": { 2023-12-01T01:22:38.761 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 113, 2023-12-01T01:22:38.761 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 110, 2023-12-01T01:22:38.761 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 439552, 2023-12-01T01:22:38.762 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 61776, 2023-12-01T01:22:38.762 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 2, 2023-12-01T01:22:38.762 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 1, 2023-12-01T01:22:38.762 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.015521045, 2023-12-01T01:22:38.762 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.005274994, 2023-12-01T01:22:38.762 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.009877965, 2023-12-01T01:22:38.762 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.002212622, 2023-12-01T01:22:38.763 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2023-12-01T01:22:38.763 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 109, 2023-12-01T01:22:38.763 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.003414912, 2023-12-01T01:22:38.763 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000031329 2023-12-01T01:22:38.763 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:38.763 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2023-12-01T01:22:38.763 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:38.764 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:38.764 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:38.764 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:38.764 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 439552, 2023-12-01T01:22:38.765 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 61776 2023-12-01T01:22:38.765 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:38.765 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-2": { 2023-12-01T01:22:38.765 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 1, 2023-12-01T01:22:38.765 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 83, 2023-12-01T01:22:38.765 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 160, 2023-12-01T01:22:38.765 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 90576, 2023-12-01T01:22:38.766 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 3, 2023-12-01T01:22:38.766 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 2, 2023-12-01T01:22:38.766 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.005337233, 2023-12-01T01:22:38.766 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.003501505, 2023-12-01T01:22:38.766 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.001181702, 2023-12-01T01:22:38.766 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-01T01:22:38.767 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2023-12-01T01:22:38.767 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 81, 2023-12-01T01:22:38.767 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.002675941, 2023-12-01T01:22:38.767 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000033036 2023-12-01T01:22:38.767 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:38.767 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2023-12-01T01:22:38.767 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:38.767 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:38.768 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:38.768 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:38.768 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 160, 2023-12-01T01:22:38.768 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 90576 2023-12-01T01:22:38.768 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:38.768 INFO:teuthology.orchestra.run.smithi077.stdout: "cct": { 2023-12-01T01:22:38.768 INFO:teuthology.orchestra.run.smithi077.stdout: "total_workers": 0, 2023-12-01T01:22:38.769 INFO:teuthology.orchestra.run.smithi077.stdout: "unhealthy_workers": 0 2023-12-01T01:22:38.769 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:38.769 INFO:teuthology.orchestra.run.smithi077.stdout: "mempool": { 2023-12-01T01:22:38.769 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_bytes": 0, 2023-12-01T01:22:38.769 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_items": 0, 2023-12-01T01:22:38.769 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_bytes": 0, 2023-12-01T01:22:38.769 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_items": 0, 2023-12-01T01:22:38.769 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_bytes": 0, 2023-12-01T01:22:38.770 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_items": 0, 2023-12-01T01:22:38.770 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-01T01:22:38.770 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_items": 0, 2023-12-01T01:22:38.770 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-01T01:22:38.770 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_items": 0, 2023-12-01T01:22:38.770 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_bytes": 0, 2023-12-01T01:22:38.770 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_items": 0, 2023-12-01T01:22:38.770 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-01T01:22:38.771 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_items": 0, 2023-12-01T01:22:38.771 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_bytes": 0, 2023-12-01T01:22:38.771 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_items": 0, 2023-12-01T01:22:38.771 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_bytes": 0, 2023-12-01T01:22:38.771 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_items": 0, 2023-12-01T01:22:38.771 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-01T01:22:38.771 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_items": 0, 2023-12-01T01:22:38.771 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-01T01:22:38.771 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_items": 0, 2023-12-01T01:22:38.772 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_bytes": 0, 2023-12-01T01:22:38.772 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_items": 0, 2023-12-01T01:22:38.772 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_bytes": 0, 2023-12-01T01:22:38.772 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_items": 0, 2023-12-01T01:22:38.772 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-01T01:22:38.772 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_items": 0, 2023-12-01T01:22:38.772 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_bytes": 0, 2023-12-01T01:22:38.772 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_items": 0, 2023-12-01T01:22:38.772 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_bytes": 0, 2023-12-01T01:22:38.772 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_items": 0, 2023-12-01T01:22:38.772 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_bytes": 0, 2023-12-01T01:22:38.772 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_items": 0, 2023-12-01T01:22:38.772 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_bytes": 0, 2023-12-01T01:22:38.772 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_items": 0, 2023-12-01T01:22:38.773 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_bytes": 148863, 2023-12-01T01:22:38.773 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_items": 28, 2023-12-01T01:22:38.773 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_bytes": 0, 2023-12-01T01:22:38.773 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_items": 0, 2023-12-01T01:22:38.773 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_bytes": 0, 2023-12-01T01:22:38.773 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_items": 0, 2023-12-01T01:22:38.773 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_bytes": 0, 2023-12-01T01:22:38.773 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_items": 0, 2023-12-01T01:22:38.773 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_bytes": 0, 2023-12-01T01:22:38.773 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_items": 0, 2023-12-01T01:22:38.773 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_bytes": 0, 2023-12-01T01:22:38.773 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_items": 0, 2023-12-01T01:22:38.773 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_bytes": 0, 2023-12-01T01:22:38.774 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_items": 0, 2023-12-01T01:22:38.774 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_bytes": 0, 2023-12-01T01:22:38.774 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_items": 0, 2023-12-01T01:22:38.774 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_bytes": 928, 2023-12-01T01:22:38.774 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_items": 2, 2023-12-01T01:22:38.774 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_bytes": 0, 2023-12-01T01:22:38.775 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_items": 0, 2023-12-01T01:22:38.775 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_bytes": 0, 2023-12-01T01:22:38.775 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_items": 0 2023-12-01T01:22:38.775 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:38.775 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-01T01:22:38.775 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2023-12-01T01:22:38.775 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2023-12-01T01:22:38.775 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2023-12-01T01:22:38.775 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 114, 2023-12-01T01:22:38.775 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 423622, 2023-12-01T01:22:38.776 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:38.776 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 114, 2023-12-01T01:22:38.776 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2023-12-01T01:22:38.776 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2023-12-01T01:22:38.776 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 114, 2023-12-01T01:22:38.776 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 423622, 2023-12-01T01:22:38.777 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2023-12-01T01:22:38.777 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:38.777 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:38.777 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:38.777 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:38.777 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:38.777 INFO:teuthology.orchestra.run.smithi077.stdout:} 2023-12-01T01:22:38.778 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:38 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:22:38.778 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:38 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:22:38.778 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:38 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:22:38.778 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:38 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:22:38.778 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:38 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:22:38.778 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:38 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:22:38.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:38 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:22:38.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:38 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:22:38.788 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:38 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-01T01:22:38.788 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:38 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-01T01:22:38.788 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:38 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-01T01:22:38.788 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:38 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' 2023-12-01T01:22:39.212 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2023-12-01T01:22:39.212 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2023-12-01T01:22:39.212 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2023-12-01T01:22:39.212 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2023-12-01T01:22:39.212 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2023-12-01T01:22:39.213 DEBUG:tasks.check_counter:Getting stats from g 2023-12-01T01:22:39.213 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph daemon mds.g perf dump 2023-12-01T01:22:39.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:39 smithi203 ceph-mon[103581]: pgmap v441: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 426 B/s wr, 12 op/s 2023-12-01T01:22:39.768 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:22:39.778 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:39 smithi137 ceph-mon[103312]: pgmap v441: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 426 B/s wr, 12 op/s 2023-12-01T01:22:39.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:39 smithi077 ceph-mon[98867]: pgmap v441: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 426 B/s wr, 12 op/s 2023-12-01T01:22:40.952 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2023-12-01T01:22:40.953 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-0": { 2023-12-01T01:22:40.953 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 369, 2023-12-01T01:22:40.953 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 371, 2023-12-01T01:22:40.953 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 123929, 2023-12-01T01:22:40.953 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 114760, 2023-12-01T01:22:40.953 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 6, 2023-12-01T01:22:40.953 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 2, 2023-12-01T01:22:40.953 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.037577082, 2023-12-01T01:22:40.953 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.013909215, 2023-12-01T01:22:40.953 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.008074341, 2023-12-01T01:22:40.953 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.011522580, 2023-12-01T01:22:40.953 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2023-12-01T01:22:40.953 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 369, 2023-12-01T01:22:40.953 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.014770127, 2023-12-01T01:22:40.953 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000040027 2023-12-01T01:22:40.954 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.954 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2023-12-01T01:22:40.954 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:40.954 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:40.954 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:40.954 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.954 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 0, 2023-12-01T01:22:40.954 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 0 2023-12-01T01:22:40.954 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.954 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-1": { 2023-12-01T01:22:40.954 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 686, 2023-12-01T01:22:40.954 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 764, 2023-12-01T01:22:40.955 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 6035135, 2023-12-01T01:22:40.955 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 427448, 2023-12-01T01:22:40.955 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 5, 2023-12-01T01:22:40.955 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 4, 2023-12-01T01:22:40.955 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.047281225, 2023-12-01T01:22:40.955 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.013771901, 2023-12-01T01:22:40.955 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.014019468, 2023-12-01T01:22:40.955 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.015978223, 2023-12-01T01:22:40.955 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2023-12-01T01:22:40.955 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 763, 2023-12-01T01:22:40.955 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.021896026, 2023-12-01T01:22:40.955 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000028697 2023-12-01T01:22:40.955 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.955 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2023-12-01T01:22:40.955 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:40.956 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:40.956 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:40.956 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.956 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 3152, 2023-12-01T01:22:40.956 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 215600 2023-12-01T01:22:40.956 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.956 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-2": { 2023-12-01T01:22:40.956 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 134, 2023-12-01T01:22:40.956 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 117, 2023-12-01T01:22:40.956 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 540484, 2023-12-01T01:22:40.956 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 62128, 2023-12-01T01:22:40.956 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 4, 2023-12-01T01:22:40.956 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 3, 2023-12-01T01:22:40.956 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.016973071, 2023-12-01T01:22:40.956 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.005432631, 2023-12-01T01:22:40.957 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.011740632, 2023-12-01T01:22:40.957 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.002281945, 2023-12-01T01:22:40.957 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2023-12-01T01:22:40.957 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 116, 2023-12-01T01:22:40.957 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.009162483, 2023-12-01T01:22:40.957 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000078986 2023-12-01T01:22:40.957 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.957 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2023-12-01T01:22:40.957 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:40.957 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:40.957 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:40.957 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.957 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 539136, 2023-12-01T01:22:40.957 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 61184 2023-12-01T01:22:40.957 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.958 INFO:teuthology.orchestra.run.smithi077.stdout: "cct": { 2023-12-01T01:22:40.958 INFO:teuthology.orchestra.run.smithi077.stdout: "total_workers": 1, 2023-12-01T01:22:40.958 INFO:teuthology.orchestra.run.smithi077.stdout: "unhealthy_workers": 0 2023-12-01T01:22:40.958 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.958 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-MDSRank": { 2023-12-01T01:22:40.958 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2023-12-01T01:22:40.958 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2023-12-01T01:22:40.958 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1803, 2023-12-01T01:22:40.958 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.043975913, 2023-12-01T01:22:40.958 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000024390 2023-12-01T01:22:40.958 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:40.958 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.958 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-PurgeQueue": { 2023-12-01T01:22:40.958 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2023-12-01T01:22:40.958 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2023-12-01T01:22:40.958 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:40.959 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:40.959 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:40.959 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:40.959 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.959 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2023-12-01T01:22:40.959 INFO:teuthology.orchestra.run.smithi077.stdout: "request": 0, 2023-12-01T01:22:40.959 INFO:teuthology.orchestra.run.smithi077.stdout: "reply": 0, 2023-12-01T01:22:40.959 INFO:teuthology.orchestra.run.smithi077.stdout: "reply_latency": { 2023-12-01T01:22:40.959 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:40.959 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:40.959 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:40.959 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.959 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_reply": 0, 2023-12-01T01:22:40.960 INFO:teuthology.orchestra.run.smithi077.stdout: "forward": 0, 2023-12-01T01:22:40.960 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_complete": 0, 2023-12-01T01:22:40.960 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_keys": 0, 2023-12-01T01:22:40.960 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_commit": 0, 2023-12-01T01:22:40.960 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_split": 0, 2023-12-01T01:22:40.960 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_merge": 0, 2023-12-01T01:22:40.960 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes": 962, 2023-12-01T01:22:40.960 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_top": 0, 2023-12-01T01:22:40.960 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_bottom": 0, 2023-12-01T01:22:40.960 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pin_tail": 962, 2023-12-01T01:22:40.960 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pinned": 962, 2023-12-01T01:22:40.960 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_expired": 0, 2023-12-01T01:22:40.960 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_with_caps": 0, 2023-12-01T01:22:40.960 INFO:teuthology.orchestra.run.smithi077.stdout: "caps": 0, 2023-12-01T01:22:40.960 INFO:teuthology.orchestra.run.smithi077.stdout: "subtrees": 1, 2023-12-01T01:22:40.961 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse": 0, 2023-12-01T01:22:40.961 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_hit": 0, 2023-12-01T01:22:40.961 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_forward": 0, 2023-12-01T01:22:40.961 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_discover": 0, 2023-12-01T01:22:40.961 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_dir_fetch": 0, 2023-12-01T01:22:40.961 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_remote_ino": 0, 2023-12-01T01:22:40.961 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_lock": 0, 2023-12-01T01:22:40.961 INFO:teuthology.orchestra.run.smithi077.stdout: "load_cent": 0, 2023-12-01T01:22:40.961 INFO:teuthology.orchestra.run.smithi077.stdout: "q": 0, 2023-12-01T01:22:40.961 INFO:teuthology.orchestra.run.smithi077.stdout: "exported": 0, 2023-12-01T01:22:40.961 INFO:teuthology.orchestra.run.smithi077.stdout: "exported_inodes": 0, 2023-12-01T01:22:40.961 INFO:teuthology.orchestra.run.smithi077.stdout: "imported": 0, 2023-12-01T01:22:40.961 INFO:teuthology.orchestra.run.smithi077.stdout: "imported_inodes": 0, 2023-12-01T01:22:40.961 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_dir_fetch": 0, 2023-12-01T01:22:40.961 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_backtrace_fetch": 0, 2023-12-01T01:22:40.962 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_peer_discover": 0, 2023-12-01T01:22:40.962 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rfiles": 0, 2023-12-01T01:22:40.962 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rbytes": 0, 2023-12-01T01:22:40.962 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rsnaps": 0, 2023-12-01T01:22:40.962 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_fetch": 0, 2023-12-01T01:22:40.962 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_set_tag": 0, 2023-12-01T01:22:40.962 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_repaired": 0, 2023-12-01T01:22:40.962 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_inotable_repaired": 0, 2023-12-01T01:22:40.962 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_inodes": 0, 2023-12-01T01:22:40.962 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_base_inodes": 0, 2023-12-01T01:22:40.962 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dirfrag_rstats": 0, 2023-12-01T01:22:40.962 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_file_inodes": 0, 2023-12-01T01:22:40.962 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_inode_file_caps": 0, 2023-12-01T01:22:40.963 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_revoke": 0, 2023-12-01T01:22:40.963 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_grant": 0, 2023-12-01T01:22:40.963 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_trunc": 0, 2023-12-01T01:22:40.963 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-01T01:22:40.963 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-01T01:22:40.963 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps": 0, 2023-12-01T01:22:40.963 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps_dirty": 0, 2023-12-01T01:22:40.963 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_cap_release": 0, 2023-12-01T01:22:40.963 INFO:teuthology.orchestra.run.smithi077.stdout: "process_request_cap_release": 0 2023-12-01T01:22:40.963 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.963 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_cache": { 2023-12-01T01:22:40.963 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update": 0, 2023-12-01T01:22:40.963 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update_receipt": 0, 2023-12-01T01:22:40.963 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_try_discover": 0, 2023-12-01T01:22:40.963 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_send_discover": 0, 2023-12-01T01:22:40.964 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_handle_discover": 0, 2023-12-01T01:22:40.964 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays": 0, 2023-12-01T01:22:40.964 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_delayed": 0, 2023-12-01T01:22:40.964 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_enqueuing": 0, 2023-12-01T01:22:40.964 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_created": 0, 2023-12-01T01:22:40.964 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_enqueued": 0, 2023-12-01T01:22:40.964 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_reintegrated": 0, 2023-12-01T01:22:40.964 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_migrated": 0, 2023-12-01T01:22:40.964 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_processing": 0, 2023-12-01T01:22:40.964 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_enqueued": 0, 2023-12-01T01:22:40.964 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_prioritized": 0, 2023-12-01T01:22:40.964 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_started": 0, 2023-12-01T01:22:40.964 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_completed": 0, 2023-12-01T01:22:40.964 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_enqueue_scrub": 0, 2023-12-01T01:22:40.964 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_exportdir": 0, 2023-12-01T01:22:40.965 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_flush": 0, 2023-12-01T01:22:40.965 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragmentdir": 0, 2023-12-01T01:22:40.965 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragstats": 0, 2023-12-01T01:22:40.965 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_inodestats": 0 2023-12-01T01:22:40.965 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.965 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_log": { 2023-12-01T01:22:40.965 INFO:teuthology.orchestra.run.smithi077.stdout: "evadd": 0, 2023-12-01T01:22:40.965 INFO:teuthology.orchestra.run.smithi077.stdout: "evex": 0, 2023-12-01T01:22:40.965 INFO:teuthology.orchestra.run.smithi077.stdout: "evtrm": 0, 2023-12-01T01:22:40.965 INFO:teuthology.orchestra.run.smithi077.stdout: "ev": 3439, 2023-12-01T01:22:40.965 INFO:teuthology.orchestra.run.smithi077.stdout: "evexg": 0, 2023-12-01T01:22:40.965 INFO:teuthology.orchestra.run.smithi077.stdout: "evexd": 0, 2023-12-01T01:22:40.965 INFO:teuthology.orchestra.run.smithi077.stdout: "segadd": 0, 2023-12-01T01:22:40.966 INFO:teuthology.orchestra.run.smithi077.stdout: "segex": 0, 2023-12-01T01:22:40.966 INFO:teuthology.orchestra.run.smithi077.stdout: "segtrm": 0, 2023-12-01T01:22:40.966 INFO:teuthology.orchestra.run.smithi077.stdout: "seg": 5, 2023-12-01T01:22:40.966 INFO:teuthology.orchestra.run.smithi077.stdout: "segexg": 0, 2023-12-01T01:22:40.966 INFO:teuthology.orchestra.run.smithi077.stdout: "segexd": 0, 2023-12-01T01:22:40.966 INFO:teuthology.orchestra.run.smithi077.stdout: "expos": 4194304, 2023-12-01T01:22:40.966 INFO:teuthology.orchestra.run.smithi077.stdout: "wrpos": 10051537, 2023-12-01T01:22:40.966 INFO:teuthology.orchestra.run.smithi077.stdout: "rdpos": 10050399, 2023-12-01T01:22:40.966 INFO:teuthology.orchestra.run.smithi077.stdout: "jlat": { 2023-12-01T01:22:40.966 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:40.966 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:40.966 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:40.967 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.967 INFO:teuthology.orchestra.run.smithi077.stdout: "replayed": 3439 2023-12-01T01:22:40.967 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.967 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_mem": { 2023-12-01T01:22:40.967 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 7, 2023-12-01T01:22:40.967 INFO:teuthology.orchestra.run.smithi077.stdout: "ino+": 1053, 2023-12-01T01:22:40.967 INFO:teuthology.orchestra.run.smithi077.stdout: "ino-": 1046, 2023-12-01T01:22:40.967 INFO:teuthology.orchestra.run.smithi077.stdout: "dir": 5, 2023-12-01T01:22:40.967 INFO:teuthology.orchestra.run.smithi077.stdout: "dir+": 68, 2023-12-01T01:22:40.967 INFO:teuthology.orchestra.run.smithi077.stdout: "dir-": 63, 2023-12-01T01:22:40.967 INFO:teuthology.orchestra.run.smithi077.stdout: "dn": 962, 2023-12-01T01:22:40.967 INFO:teuthology.orchestra.run.smithi077.stdout: "dn+": 2015, 2023-12-01T01:22:40.967 INFO:teuthology.orchestra.run.smithi077.stdout: "dn-": 1053, 2023-12-01T01:22:40.967 INFO:teuthology.orchestra.run.smithi077.stdout: "cap": 0, 2023-12-01T01:22:40.967 INFO:teuthology.orchestra.run.smithi077.stdout: "cap+": 0, 2023-12-01T01:22:40.967 INFO:teuthology.orchestra.run.smithi077.stdout: "cap-": 0, 2023-12-01T01:22:40.968 INFO:teuthology.orchestra.run.smithi077.stdout: "rss": 48028, 2023-12-01T01:22:40.968 INFO:teuthology.orchestra.run.smithi077.stdout: "heap": 207156 2023-12-01T01:22:40.968 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.968 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_server": { 2023-12-01T01:22:40.968 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_client_request": 0, 2023-12-01T01:22:40.968 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_server_request": 0, 2023-12-01T01:22:40.968 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_request": 0, 2023-12-01T01:22:40.968 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_session": 0, 2023-12-01T01:22:40.968 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_peer_request": 0, 2023-12-01T01:22:40.968 INFO:teuthology.orchestra.run.smithi077.stdout: "req_create_latency": { 2023-12-01T01:22:40.968 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:40.968 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:40.968 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:40.968 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.969 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getattr_latency": { 2023-12-01T01:22:40.969 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:40.969 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:40.969 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:40.969 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.969 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getfilelock_latency": { 2023-12-01T01:22:40.969 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:40.969 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:40.969 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:40.969 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.969 INFO:teuthology.orchestra.run.smithi077.stdout: "req_link_latency": { 2023-12-01T01:22:40.969 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:40.969 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:40.969 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:40.969 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.970 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookup_latency": { 2023-12-01T01:22:40.970 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:40.970 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:40.970 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:40.970 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.970 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookuphash_latency": { 2023-12-01T01:22:40.970 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:40.970 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:40.970 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:40.970 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.970 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupino_latency": { 2023-12-01T01:22:40.970 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:40.970 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:40.970 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:40.971 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.971 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupname_latency": { 2023-12-01T01:22:40.971 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:40.971 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:40.971 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:40.971 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.971 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupparent_latency": { 2023-12-01T01:22:40.971 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:40.971 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:40.971 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:40.971 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.971 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupsnap_latency": { 2023-12-01T01:22:40.971 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:40.971 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:40.971 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:40.972 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.972 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lssnap_latency": { 2023-12-01T01:22:40.972 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:40.972 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:40.972 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:40.972 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.972 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mkdir_latency": { 2023-12-01T01:22:40.972 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:40.972 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:40.972 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:40.972 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.972 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mknod_latency": { 2023-12-01T01:22:40.972 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:40.972 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:40.973 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:40.973 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.973 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mksnap_latency": { 2023-12-01T01:22:40.973 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:40.973 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:40.973 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:40.973 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.973 INFO:teuthology.orchestra.run.smithi077.stdout: "req_open_latency": { 2023-12-01T01:22:40.973 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:40.973 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:40.973 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:40.973 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.973 INFO:teuthology.orchestra.run.smithi077.stdout: "req_readdir_latency": { 2023-12-01T01:22:40.973 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:40.973 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:40.974 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:40.974 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.974 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rename_latency": { 2023-12-01T01:22:40.974 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:40.974 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:40.974 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:40.974 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.974 INFO:teuthology.orchestra.run.smithi077.stdout: "req_renamesnap_latency": { 2023-12-01T01:22:40.974 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:40.974 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:40.974 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:40.974 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.974 INFO:teuthology.orchestra.run.smithi077.stdout: "req_snapdiff_latency": { 2023-12-01T01:22:40.974 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:40.975 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:40.975 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:40.975 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.975 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmdir_latency": { 2023-12-01T01:22:40.975 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:40.975 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:40.975 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:40.975 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.975 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmsnap_latency": { 2023-12-01T01:22:40.975 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:40.975 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:40.975 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:40.975 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.975 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmxattr_latency": { 2023-12-01T01:22:40.976 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:40.976 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:40.976 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:40.976 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.976 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setattr_latency": { 2023-12-01T01:22:40.976 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:40.976 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:40.976 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:40.976 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.976 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setdirlayout_latency": { 2023-12-01T01:22:40.976 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:40.976 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:40.976 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:40.976 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.976 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setfilelock_latency": { 2023-12-01T01:22:40.976 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:40.977 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:40.977 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:40.977 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.977 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setlayout_latency": { 2023-12-01T01:22:40.977 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:40.977 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:40.977 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:40.977 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.977 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setxattr_latency": { 2023-12-01T01:22:40.977 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:40.977 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:40.977 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:40.977 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.977 INFO:teuthology.orchestra.run.smithi077.stdout: "req_symlink_latency": { 2023-12-01T01:22:40.977 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:40.978 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:40.978 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:40.978 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.978 INFO:teuthology.orchestra.run.smithi077.stdout: "req_unlink_latency": { 2023-12-01T01:22:40.978 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:40.978 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:40.978 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:40.978 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.978 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_revoke_eviction": 0, 2023-12-01T01:22:40.978 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_acquisition_throttle": 0, 2023-12-01T01:22:40.978 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getvxattr_latency": { 2023-12-01T01:22:40.978 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:40.978 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:40.978 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:40.978 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:40.979 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.979 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_sessions": { 2023-12-01T01:22:40.979 INFO:teuthology.orchestra.run.smithi077.stdout: "session_count": 1, 2023-12-01T01:22:40.979 INFO:teuthology.orchestra.run.smithi077.stdout: "session_add": 1, 2023-12-01T01:22:40.979 INFO:teuthology.orchestra.run.smithi077.stdout: "session_remove": 0, 2023-12-01T01:22:40.979 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_open": 1, 2023-12-01T01:22:40.979 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_stale": 0, 2023-12-01T01:22:40.979 INFO:teuthology.orchestra.run.smithi077.stdout: "total_load": 0, 2023-12-01T01:22:40.979 INFO:teuthology.orchestra.run.smithi077.stdout: "average_load": 0, 2023-12-01T01:22:40.979 INFO:teuthology.orchestra.run.smithi077.stdout: "avg_session_uptime": 1650, 2023-12-01T01:22:40.979 INFO:teuthology.orchestra.run.smithi077.stdout: "mdthresh_evicted": 0 2023-12-01T01:22:40.979 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.979 INFO:teuthology.orchestra.run.smithi077.stdout: "mempool": { 2023-12-01T01:22:40.980 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_bytes": 0, 2023-12-01T01:22:40.980 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_items": 0, 2023-12-01T01:22:40.980 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_bytes": 0, 2023-12-01T01:22:40.980 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_items": 0, 2023-12-01T01:22:40.980 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_bytes": 0, 2023-12-01T01:22:40.980 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_items": 0, 2023-12-01T01:22:40.980 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-01T01:22:40.980 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_items": 0, 2023-12-01T01:22:40.980 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-01T01:22:40.980 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_items": 0, 2023-12-01T01:22:40.980 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_bytes": 0, 2023-12-01T01:22:40.980 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_items": 0, 2023-12-01T01:22:40.980 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-01T01:22:40.980 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_items": 0, 2023-12-01T01:22:40.980 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_bytes": 0, 2023-12-01T01:22:40.980 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_items": 0, 2023-12-01T01:22:40.981 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_bytes": 0, 2023-12-01T01:22:40.981 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_items": 0, 2023-12-01T01:22:40.981 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-01T01:22:40.981 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_items": 0, 2023-12-01T01:22:40.981 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-01T01:22:40.981 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_items": 0, 2023-12-01T01:22:40.981 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_bytes": 0, 2023-12-01T01:22:40.981 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_items": 0, 2023-12-01T01:22:40.981 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_bytes": 0, 2023-12-01T01:22:40.981 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_items": 0, 2023-12-01T01:22:40.981 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-01T01:22:40.981 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_items": 0, 2023-12-01T01:22:40.981 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_bytes": 0, 2023-12-01T01:22:40.981 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_items": 0, 2023-12-01T01:22:40.982 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_bytes": 0, 2023-12-01T01:22:40.982 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_items": 0, 2023-12-01T01:22:40.982 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_bytes": 0, 2023-12-01T01:22:40.982 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_items": 0, 2023-12-01T01:22:40.982 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_bytes": 0, 2023-12-01T01:22:40.982 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_items": 0, 2023-12-01T01:22:40.982 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_bytes": 197319, 2023-12-01T01:22:40.982 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_items": 52, 2023-12-01T01:22:40.982 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_bytes": 0, 2023-12-01T01:22:40.982 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_items": 0, 2023-12-01T01:22:40.982 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_bytes": 0, 2023-12-01T01:22:40.982 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_items": 0, 2023-12-01T01:22:40.982 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_bytes": 0, 2023-12-01T01:22:40.982 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_items": 0, 2023-12-01T01:22:40.983 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_bytes": 0, 2023-12-01T01:22:40.983 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_items": 0, 2023-12-01T01:22:40.983 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_bytes": 7336, 2023-12-01T01:22:40.983 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_items": 154, 2023-12-01T01:22:40.983 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_bytes": 0, 2023-12-01T01:22:40.983 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_items": 0, 2023-12-01T01:22:40.983 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_bytes": 0, 2023-12-01T01:22:40.983 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_items": 0, 2023-12-01T01:22:40.983 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_bytes": 691672, 2023-12-01T01:22:40.983 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_items": 3910, 2023-12-01T01:22:40.983 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_bytes": 0, 2023-12-01T01:22:40.983 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_items": 0, 2023-12-01T01:22:40.983 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_bytes": 0, 2023-12-01T01:22:40.983 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_items": 0 2023-12-01T01:22:40.983 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.984 INFO:teuthology.orchestra.run.smithi077.stdout: "objecter": { 2023-12-01T01:22:40.984 INFO:teuthology.orchestra.run.smithi077.stdout: "op_active": 0, 2023-12-01T01:22:40.984 INFO:teuthology.orchestra.run.smithi077.stdout: "op_laggy": 0, 2023-12-01T01:22:40.984 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send": 1053, 2023-12-01T01:22:40.984 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send_bytes": 21, 2023-12-01T01:22:40.984 INFO:teuthology.orchestra.run.smithi077.stdout: "op_resend": 3, 2023-12-01T01:22:40.984 INFO:teuthology.orchestra.run.smithi077.stdout: "op_reply": 1053, 2023-12-01T01:22:40.984 INFO:teuthology.orchestra.run.smithi077.stdout: "op_latency": { 2023-12-01T01:22:40.984 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1053, 2023-12-01T01:22:40.984 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.656984678, 2023-12-01T01:22:40.984 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000623917 2023-12-01T01:22:40.984 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.984 INFO:teuthology.orchestra.run.smithi077.stdout: "op_inflight": 0, 2023-12-01T01:22:40.984 INFO:teuthology.orchestra.run.smithi077.stdout: "oplen_avg": { 2023-12-01T01:22:40.984 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1053, 2023-12-01T01:22:40.984 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 1054 2023-12-01T01:22:40.985 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.985 INFO:teuthology.orchestra.run.smithi077.stdout: "op": 1053, 2023-12-01T01:22:40.985 INFO:teuthology.orchestra.run.smithi077.stdout: "op_r": 1053, 2023-12-01T01:22:40.985 INFO:teuthology.orchestra.run.smithi077.stdout: "op_w": 0, 2023-12-01T01:22:40.985 INFO:teuthology.orchestra.run.smithi077.stdout: "op_rmw": 0, 2023-12-01T01:22:40.985 INFO:teuthology.orchestra.run.smithi077.stdout: "op_pg": 0, 2023-12-01T01:22:40.985 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_stat": 657, 2023-12-01T01:22:40.985 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_create": 0, 2023-12-01T01:22:40.985 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_read": 394, 2023-12-01T01:22:40.985 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_write": 0, 2023-12-01T01:22:40.985 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writefull": 0, 2023-12-01T01:22:40.985 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writesame": 0, 2023-12-01T01:22:40.985 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_append": 0, 2023-12-01T01:22:40.986 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_zero": 0, 2023-12-01T01:22:40.986 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_truncate": 0, 2023-12-01T01:22:40.986 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_delete": 0, 2023-12-01T01:22:40.986 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_mapext": 0, 2023-12-01T01:22:40.986 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_sparse_read": 0, 2023-12-01T01:22:40.986 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_clonerange": 0, 2023-12-01T01:22:40.986 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_getxattr": 1, 2023-12-01T01:22:40.986 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_setxattr": 0, 2023-12-01T01:22:40.986 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_cmpxattr": 0, 2023-12-01T01:22:40.986 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_rmxattr": 0, 2023-12-01T01:22:40.986 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_resetxattrs": 0, 2023-12-01T01:22:40.986 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_call": 0, 2023-12-01T01:22:40.986 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_watch": 0, 2023-12-01T01:22:40.986 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_notify": 0, 2023-12-01T01:22:40.987 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_src_cmpxattr": 0, 2023-12-01T01:22:40.987 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls": 0, 2023-12-01T01:22:40.987 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls_filter": 0, 2023-12-01T01:22:40.987 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_other": 0, 2023-12-01T01:22:40.987 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_active": 0, 2023-12-01T01:22:40.987 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_send": 0, 2023-12-01T01:22:40.987 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_resend": 0, 2023-12-01T01:22:40.987 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_ping": 0, 2023-12-01T01:22:40.987 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_active": 0, 2023-12-01T01:22:40.987 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_send": 0, 2023-12-01T01:22:40.987 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_resend": 0, 2023-12-01T01:22:40.987 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_active": 0, 2023-12-01T01:22:40.987 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_send": 0, 2023-12-01T01:22:40.987 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_resend": 0, 2023-12-01T01:22:40.988 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_active": 0, 2023-12-01T01:22:40.988 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_send": 0, 2023-12-01T01:22:40.988 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_resend": 0, 2023-12-01T01:22:40.988 INFO:teuthology.orchestra.run.smithi077.stdout: "command_active": 0, 2023-12-01T01:22:40.988 INFO:teuthology.orchestra.run.smithi077.stdout: "command_send": 0, 2023-12-01T01:22:40.988 INFO:teuthology.orchestra.run.smithi077.stdout: "command_resend": 0, 2023-12-01T01:22:40.988 INFO:teuthology.orchestra.run.smithi077.stdout: "map_epoch": 84, 2023-12-01T01:22:40.988 INFO:teuthology.orchestra.run.smithi077.stdout: "map_full": 0, 2023-12-01T01:22:40.988 INFO:teuthology.orchestra.run.smithi077.stdout: "map_inc": 2, 2023-12-01T01:22:40.988 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_sessions": 6, 2023-12-01T01:22:40.988 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_open": 6, 2023-12-01T01:22:40.988 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_close": 0, 2023-12-01T01:22:40.988 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_laggy": 0, 2023-12-01T01:22:40.988 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_wr": 0, 2023-12-01T01:22:40.989 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_rd": 2, 2023-12-01T01:22:40.989 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_del": 0 2023-12-01T01:22:40.989 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.989 INFO:teuthology.orchestra.run.smithi077.stdout: "oft": { 2023-12-01T01:22:40.989 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_objs": 0, 2023-12-01T01:22:40.989 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_kv_pairs": 0, 2023-12-01T01:22:40.989 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_updates": 0, 2023-12-01T01:22:40.989 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_removes": 0 2023-12-01T01:22:40.989 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.989 INFO:teuthology.orchestra.run.smithi077.stdout: "osd-slow-ops": { 2023-12-01T01:22:40.989 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_ops_count": 0 2023-12-01T01:22:40.989 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.989 INFO:teuthology.orchestra.run.smithi077.stdout: "purge_queue": { 2023-12-01T01:22:40.989 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops": 0, 2023-12-01T01:22:40.990 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops_high_water": 0, 2023-12-01T01:22:40.990 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing": 0, 2023-12-01T01:22:40.990 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_high_water": 0, 2023-12-01T01:22:40.990 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executed": 0, 2023-12-01T01:22:40.990 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_item_in_journal": 0 2023-12-01T01:22:40.990 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.990 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-01T01:22:40.990 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2023-12-01T01:22:40.990 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2023-12-01T01:22:40.990 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2023-12-01T01:22:40.990 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 1185, 2023-12-01T01:22:40.990 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 6583246, 2023-12-01T01:22:40.990 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:40.990 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 1185, 2023-12-01T01:22:40.991 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2023-12-01T01:22:40.991 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2023-12-01T01:22:40.991 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 1185, 2023-12-01T01:22:40.991 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 6583246, 2023-12-01T01:22:40.991 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2023-12-01T01:22:40.991 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:40.991 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:40.991 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:40.991 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:40.991 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.991 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_bytes": { 2023-12-01T01:22:40.991 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2023-12-01T01:22:40.991 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2023-12-01T01:22:40.992 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2023-12-01T01:22:40.992 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2023-12-01T01:22:40.992 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2023-12-01T01:22:40.992 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:40.992 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2023-12-01T01:22:40.992 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 1053, 2023-12-01T01:22:40.992 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 5857902, 2023-12-01T01:22:40.992 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 1053, 2023-12-01T01:22:40.992 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 5857902, 2023-12-01T01:22:40.992 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2023-12-01T01:22:40.992 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:40.992 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:40.993 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:40.993 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:40.993 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.993 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_ops": { 2023-12-01T01:22:40.993 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2023-12-01T01:22:40.993 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 1024, 2023-12-01T01:22:40.993 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2023-12-01T01:22:40.993 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2023-12-01T01:22:40.993 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2023-12-01T01:22:40.993 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:40.993 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2023-12-01T01:22:40.993 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 1053, 2023-12-01T01:22:40.994 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 1053, 2023-12-01T01:22:40.994 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 1053, 2023-12-01T01:22:40.994 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 1053, 2023-12-01T01:22:40.994 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2023-12-01T01:22:40.994 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:40.994 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:40.994 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:40.994 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:40.994 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.994 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle": { 2023-12-01T01:22:40.994 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2023-12-01T01:22:40.994 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2023-12-01T01:22:40.995 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2023-12-01T01:22:40.995 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2023-12-01T01:22:40.995 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2023-12-01T01:22:40.995 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:40.995 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2023-12-01T01:22:40.995 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2023-12-01T01:22:40.995 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2023-12-01T01:22:40.995 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 0, 2023-12-01T01:22:40.995 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 0, 2023-12-01T01:22:40.995 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2023-12-01T01:22:40.995 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:40.995 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:40.995 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:40.996 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:40.996 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:40.996 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle-0x55f60d836500": { 2023-12-01T01:22:40.996 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2023-12-01T01:22:40.996 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2023-12-01T01:22:40.996 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2023-12-01T01:22:40.996 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2023-12-01T01:22:40.996 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2023-12-01T01:22:40.996 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:40.996 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2023-12-01T01:22:40.996 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2023-12-01T01:22:40.996 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2023-12-01T01:22:40.996 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 0, 2023-12-01T01:22:40.996 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 0, 2023-12-01T01:22:40.997 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2023-12-01T01:22:40.997 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:40.997 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:40.997 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:40.997 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:40.997 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:40.997 INFO:teuthology.orchestra.run.smithi077.stdout:} 2023-12-01T01:22:41.314 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2023-12-01T01:22:41.315 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2023-12-01T01:22:41.315 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2023-12-01T01:22:41.315 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2023-12-01T01:22:41.315 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=0 2023-12-01T01:22:41.315 DEBUG:tasks.check_counter:Getting stats from j 2023-12-01T01:22:41.315 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph daemon mds.j perf dump 2023-12-01T01:22:41.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:41 smithi077 ceph-mon[98867]: pgmap v442: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 511 B/s wr, 12 op/s 2023-12-01T01:22:41.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:41 smithi203 ceph-mon[103581]: pgmap v442: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 511 B/s wr, 12 op/s 2023-12-01T01:22:41.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:41 smithi137 ceph-mon[103312]: pgmap v442: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 511 B/s wr, 12 op/s 2023-12-01T01:22:41.869 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.a/config 2023-12-01T01:22:43.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:42 smithi203 ceph-mon[103581]: pgmap v443: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 597 B/s wr, 12 op/s 2023-12-01T01:22:43.023 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2023-12-01T01:22:43.023 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-0": { 2023-12-01T01:22:43.023 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 114, 2023-12-01T01:22:43.023 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 111, 2023-12-01T01:22:43.024 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 439824, 2023-12-01T01:22:43.024 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 62368, 2023-12-01T01:22:43.024 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 4, 2023-12-01T01:22:43.024 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 0, 2023-12-01T01:22:43.024 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.017927631, 2023-12-01T01:22:43.024 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.006156734, 2023-12-01T01:22:43.024 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.010574146, 2023-12-01T01:22:43.024 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.002314592, 2023-12-01T01:22:43.024 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2023-12-01T01:22:43.024 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 110, 2023-12-01T01:22:43.024 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.004307184, 2023-12-01T01:22:43.024 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000039156 2023-12-01T01:22:43.024 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:43.024 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2023-12-01T01:22:43.024 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:43.025 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:43.025 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:43.025 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:43.025 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 439824, 2023-12-01T01:22:43.025 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 62368 2023-12-01T01:22:43.025 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:43.025 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-1": { 2023-12-01T01:22:43.025 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 5, 2023-12-01T01:22:43.025 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 86, 2023-12-01T01:22:43.025 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 3152, 2023-12-01T01:22:43.025 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 91200, 2023-12-01T01:22:43.025 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 3, 2023-12-01T01:22:43.025 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 2, 2023-12-01T01:22:43.025 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.005964750, 2023-12-01T01:22:43.026 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.003728807, 2023-12-01T01:22:43.026 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.001537299, 2023-12-01T01:22:43.026 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-01T01:22:43.026 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2023-12-01T01:22:43.026 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 85, 2023-12-01T01:22:43.026 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.003791191, 2023-12-01T01:22:43.026 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000044602 2023-12-01T01:22:43.026 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:43.026 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2023-12-01T01:22:43.026 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:43.026 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:43.026 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:43.026 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:43.026 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 3152, 2023-12-01T01:22:43.026 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 91200 2023-12-01T01:22:43.027 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:43.027 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-2": { 2023-12-01T01:22:43.027 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 0, 2023-12-01T01:22:43.027 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 2, 2023-12-01T01:22:43.029 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 0, 2023-12-01T01:22:43.029 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 0, 2023-12-01T01:22:43.029 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 2, 2023-12-01T01:22:43.029 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 1, 2023-12-01T01:22:43.029 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.000376073, 2023-12-01T01:22:43.030 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.000000000, 2023-12-01T01:22:43.030 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.000321150, 2023-12-01T01:22:43.030 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-01T01:22:43.030 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2023-12-01T01:22:43.030 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:43.030 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:43.030 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:43.030 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:43.030 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2023-12-01T01:22:43.030 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:43.030 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:43.030 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:43.030 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:43.030 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 0, 2023-12-01T01:22:43.030 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 0 2023-12-01T01:22:43.031 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:43.031 INFO:teuthology.orchestra.run.smithi077.stdout: "cct": { 2023-12-01T01:22:43.031 INFO:teuthology.orchestra.run.smithi077.stdout: "total_workers": 0, 2023-12-01T01:22:43.031 INFO:teuthology.orchestra.run.smithi077.stdout: "unhealthy_workers": 0 2023-12-01T01:22:43.031 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:43.031 INFO:teuthology.orchestra.run.smithi077.stdout: "mempool": { 2023-12-01T01:22:43.031 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_bytes": 0, 2023-12-01T01:22:43.031 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_items": 0, 2023-12-01T01:22:43.031 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_bytes": 0, 2023-12-01T01:22:43.031 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_items": 0, 2023-12-01T01:22:43.031 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_bytes": 0, 2023-12-01T01:22:43.031 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_items": 0, 2023-12-01T01:22:43.031 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-01T01:22:43.031 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_items": 0, 2023-12-01T01:22:43.032 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-01T01:22:43.032 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_items": 0, 2023-12-01T01:22:43.032 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_bytes": 0, 2023-12-01T01:22:43.032 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_items": 0, 2023-12-01T01:22:43.032 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-01T01:22:43.032 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_items": 0, 2023-12-01T01:22:43.032 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_bytes": 0, 2023-12-01T01:22:43.032 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_items": 0, 2023-12-01T01:22:43.032 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_bytes": 0, 2023-12-01T01:22:43.032 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_items": 0, 2023-12-01T01:22:43.032 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-01T01:22:43.032 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_items": 0, 2023-12-01T01:22:43.032 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-01T01:22:43.032 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_items": 0, 2023-12-01T01:22:43.033 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_bytes": 0, 2023-12-01T01:22:43.033 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_items": 0, 2023-12-01T01:22:43.033 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_bytes": 0, 2023-12-01T01:22:43.033 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_items": 0, 2023-12-01T01:22:43.033 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-01T01:22:43.033 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_items": 0, 2023-12-01T01:22:43.033 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_bytes": 0, 2023-12-01T01:22:43.033 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_items": 0, 2023-12-01T01:22:43.033 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_bytes": 0, 2023-12-01T01:22:43.033 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_items": 0, 2023-12-01T01:22:43.033 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_bytes": 0, 2023-12-01T01:22:43.033 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_items": 0, 2023-12-01T01:22:43.033 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_bytes": 0, 2023-12-01T01:22:43.033 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_items": 0, 2023-12-01T01:22:43.033 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_bytes": 148823, 2023-12-01T01:22:43.034 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_items": 27, 2023-12-01T01:22:43.034 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_bytes": 0, 2023-12-01T01:22:43.034 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_items": 0, 2023-12-01T01:22:43.034 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_bytes": 0, 2023-12-01T01:22:43.034 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_items": 0, 2023-12-01T01:22:43.034 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_bytes": 0, 2023-12-01T01:22:43.034 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_items": 0, 2023-12-01T01:22:43.034 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_bytes": 0, 2023-12-01T01:22:43.034 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_items": 0, 2023-12-01T01:22:43.034 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_bytes": 0, 2023-12-01T01:22:43.034 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_items": 0, 2023-12-01T01:22:43.034 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_bytes": 0, 2023-12-01T01:22:43.034 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_items": 0, 2023-12-01T01:22:43.034 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_bytes": 0, 2023-12-01T01:22:43.034 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_items": 0, 2023-12-01T01:22:43.035 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_bytes": 928, 2023-12-01T01:22:43.035 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_items": 2, 2023-12-01T01:22:43.035 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_bytes": 0, 2023-12-01T01:22:43.035 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_items": 0, 2023-12-01T01:22:43.035 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_bytes": 0, 2023-12-01T01:22:43.035 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_items": 0 2023-12-01T01:22:43.035 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2023-12-01T01:22:43.035 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-01T01:22:43.035 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2023-12-01T01:22:43.035 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2023-12-01T01:22:43.035 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2023-12-01T01:22:43.035 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 115, 2023-12-01T01:22:43.035 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 423752, 2023-12-01T01:22:43.035 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:43.035 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 115, 2023-12-01T01:22:43.036 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2023-12-01T01:22:43.036 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2023-12-01T01:22:43.036 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 115, 2023-12-01T01:22:43.036 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 423752, 2023-12-01T01:22:43.036 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2023-12-01T01:22:43.036 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2023-12-01T01:22:43.036 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2023-12-01T01:22:43.036 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2023-12-01T01:22:43.036 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:43.036 INFO:teuthology.orchestra.run.smithi077.stdout: } 2023-12-01T01:22:43.036 INFO:teuthology.orchestra.run.smithi077.stdout:} 2023-12-01T01:22:43.037 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:42 smithi137 ceph-mon[103312]: pgmap v443: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 597 B/s wr, 12 op/s 2023-12-01T01:22:43.048 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:42 smithi077 ceph-mon[98867]: pgmap v443: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 597 B/s wr, 12 op/s 2023-12-01T01:22:43.318 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-12-01T01:22:43.319 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-12-01T01:22:43.319 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2023-12-01T01:22:43.319 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2023-12-01T01:22:43.319 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2023-12-01T01:22:43.319 DEBUG:tasks.check_counter:Getting stats from b 2023-12-01T01:22:43.319 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph daemon mds.b perf dump 2023-12-01T01:22:43.891 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.b/config 2023-12-01T01:22:45.207 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2023-12-01T01:22:45.208 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2023-12-01T01:22:45.208 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 361761, 2023-12-01T01:22:45.208 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 361931, 2023-12-01T01:22:45.208 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 61440189, 2023-12-01T01:22:45.208 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 111660678, 2023-12-01T01:22:45.208 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 10, 2023-12-01T01:22:45.208 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 3, 2023-12-01T01:22:45.208 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 9.803703389, 2023-12-01T01:22:45.208 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 3.835905840, 2023-12-01T01:22:45.208 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 5.821013960, 2023-12-01T01:22:45.208 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.686978959, 2023-12-01T01:22:45.208 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2023-12-01T01:22:45.208 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 361933, 2023-12-01T01:22:45.208 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 7.142400533, 2023-12-01T01:22:45.209 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000019734 2023-12-01T01:22:45.209 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.209 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2023-12-01T01:22:45.209 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 662256, 2023-12-01T01:22:45.209 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.054072808, 2023-12-01T01:22:45.209 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000081 2023-12-01T01:22:45.209 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.209 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 0, 2023-12-01T01:22:45.209 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 0 2023-12-01T01:22:45.209 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.209 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2023-12-01T01:22:45.209 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 66916, 2023-12-01T01:22:45.209 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 120758, 2023-12-01T01:22:45.209 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 21812183, 2023-12-01T01:22:45.209 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 83190563, 2023-12-01T01:22:45.210 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 19, 2023-12-01T01:22:45.210 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 10, 2023-12-01T01:22:45.210 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 4.343212680, 2023-12-01T01:22:45.210 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 1.750394549, 2023-12-01T01:22:45.210 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 2.776953778, 2023-12-01T01:22:45.210 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.711481117, 2023-12-01T01:22:45.210 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2023-12-01T01:22:45.210 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 120754, 2023-12-01T01:22:45.210 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 11.643562396, 2023-12-01T01:22:45.210 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000096423 2023-12-01T01:22:45.210 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.210 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2023-12-01T01:22:45.210 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 97096, 2023-12-01T01:22:45.210 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.014209373, 2023-12-01T01:22:45.211 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000146 2023-12-01T01:22:45.211 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.211 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 12371744, 2023-12-01T01:22:45.211 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 38170816 2023-12-01T01:22:45.211 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.211 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2023-12-01T01:22:45.211 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 168575, 2023-12-01T01:22:45.211 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 168276, 2023-12-01T01:22:45.211 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 29158813, 2023-12-01T01:22:45.211 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 40812823, 2023-12-01T01:22:45.211 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 80, 2023-12-01T01:22:45.211 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 5, 2023-12-01T01:22:45.211 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 5.103157119, 2023-12-01T01:22:45.211 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 1.967691633, 2023-12-01T01:22:45.212 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 3.241655182, 2023-12-01T01:22:45.212 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.243350224, 2023-12-01T01:22:45.212 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2023-12-01T01:22:45.212 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 168274, 2023-12-01T01:22:45.212 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 5.066929038, 2023-12-01T01:22:45.212 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000030111 2023-12-01T01:22:45.212 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.212 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2023-12-01T01:22:45.212 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 315915, 2023-12-01T01:22:45.212 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.030798603, 2023-12-01T01:22:45.212 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000097 2023-12-01T01:22:45.212 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.212 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 6960, 2023-12-01T01:22:45.213 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 223440 2023-12-01T01:22:45.213 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.213 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2023-12-01T01:22:45.213 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 1, 2023-12-01T01:22:45.213 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2023-12-01T01:22:45.213 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.213 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-MDSRank": { 2023-12-01T01:22:45.213 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2023-12-01T01:22:45.213 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2023-12-01T01:22:45.213 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 93978, 2023-12-01T01:22:45.213 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 25.067017150, 2023-12-01T01:22:45.213 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000266732 2023-12-01T01:22:45.213 INFO:teuthology.orchestra.run.smithi137.stdout: } 2023-12-01T01:22:45.213 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.214 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-PurgeQueue": { 2023-12-01T01:22:45.214 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2023-12-01T01:22:45.214 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2023-12-01T01:22:45.214 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 3804, 2023-12-01T01:22:45.214 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.608369911, 2023-12-01T01:22:45.214 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000159928 2023-12-01T01:22:45.214 INFO:teuthology.orchestra.run.smithi137.stdout: } 2023-12-01T01:22:45.214 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.214 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2023-12-01T01:22:45.214 INFO:teuthology.orchestra.run.smithi137.stdout: "request": 19410, 2023-12-01T01:22:45.214 INFO:teuthology.orchestra.run.smithi137.stdout: "reply": 19091, 2023-12-01T01:22:45.214 INFO:teuthology.orchestra.run.smithi137.stdout: "reply_latency": { 2023-12-01T01:22:45.214 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 19091, 2023-12-01T01:22:45.214 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 61.772566856, 2023-12-01T01:22:45.215 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.003235690 2023-12-01T01:22:45.215 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.215 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_reply": 0, 2023-12-01T01:22:45.215 INFO:teuthology.orchestra.run.smithi137.stdout: "forward": 319, 2023-12-01T01:22:45.215 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_complete": 4, 2023-12-01T01:22:45.215 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_keys": 1, 2023-12-01T01:22:45.215 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_commit": 629, 2023-12-01T01:22:45.215 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_split": 22, 2023-12-01T01:22:45.215 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_merge": 0, 2023-12-01T01:22:45.215 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes": 33, 2023-12-01T01:22:45.215 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_top": 9, 2023-12-01T01:22:45.215 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_bottom": 13, 2023-12-01T01:22:45.215 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pin_tail": 11, 2023-12-01T01:22:45.215 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pinned": 20, 2023-12-01T01:22:45.215 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_expired": 5239, 2023-12-01T01:22:45.215 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_with_caps": 1, 2023-12-01T01:22:45.216 INFO:teuthology.orchestra.run.smithi137.stdout: "caps": 1, 2023-12-01T01:22:45.216 INFO:teuthology.orchestra.run.smithi137.stdout: "subtrees": 7, 2023-12-01T01:22:45.216 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse": 21376, 2023-12-01T01:22:45.216 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_hit": 20478, 2023-12-01T01:22:45.216 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_forward": 283, 2023-12-01T01:22:45.216 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_discover": 54, 2023-12-01T01:22:45.216 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_dir_fetch": 0, 2023-12-01T01:22:45.216 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_remote_ino": 0, 2023-12-01T01:22:45.216 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_lock": 19, 2023-12-01T01:22:45.216 INFO:teuthology.orchestra.run.smithi137.stdout: "load_cent": 17, 2023-12-01T01:22:45.216 INFO:teuthology.orchestra.run.smithi137.stdout: "q": 0, 2023-12-01T01:22:45.216 INFO:teuthology.orchestra.run.smithi137.stdout: "exported": 167, 2023-12-01T01:22:45.216 INFO:teuthology.orchestra.run.smithi137.stdout: "exported_inodes": 4619, 2023-12-01T01:22:45.216 INFO:teuthology.orchestra.run.smithi137.stdout: "imported": 1, 2023-12-01T01:22:45.216 INFO:teuthology.orchestra.run.smithi137.stdout: "imported_inodes": 6, 2023-12-01T01:22:45.217 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_dir_fetch": 0, 2023-12-01T01:22:45.217 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_backtrace_fetch": 2576, 2023-12-01T01:22:45.217 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_peer_discover": 0, 2023-12-01T01:22:45.217 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rfiles": 4, 2023-12-01T01:22:45.217 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rbytes": 246, 2023-12-01T01:22:45.217 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rsnaps": 0, 2023-12-01T01:22:45.217 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_fetch": 2576, 2023-12-01T01:22:45.217 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_set_tag": 0, 2023-12-01T01:22:45.217 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_repaired": 0, 2023-12-01T01:22:45.217 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_inotable_repaired": 0, 2023-12-01T01:22:45.217 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_inodes": 58, 2023-12-01T01:22:45.217 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_base_inodes": 3, 2023-12-01T01:22:45.217 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dirfrag_rstats": 55, 2023-12-01T01:22:45.217 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_file_inodes": 2518, 2023-12-01T01:22:45.218 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_inode_file_caps": 17, 2023-12-01T01:22:45.218 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_revoke": 8837, 2023-12-01T01:22:45.218 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_grant": 17074, 2023-12-01T01:22:45.218 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_trunc": 8290, 2023-12-01T01:22:45.218 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-01T01:22:45.218 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-01T01:22:45.218 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps": 12664, 2023-12-01T01:22:45.218 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps_dirty": 11232, 2023-12-01T01:22:45.218 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_cap_release": 97, 2023-12-01T01:22:45.218 INFO:teuthology.orchestra.run.smithi137.stdout: "process_request_cap_release": 12244 2023-12-01T01:22:45.218 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.218 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_cache": { 2023-12-01T01:22:45.218 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update": 114, 2023-12-01T01:22:45.218 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update_receipt": 431, 2023-12-01T01:22:45.219 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_try_discover": 242, 2023-12-01T01:22:45.219 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_send_discover": 374, 2023-12-01T01:22:45.219 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_handle_discover": 567, 2023-12-01T01:22:45.219 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays": 1, 2023-12-01T01:22:45.219 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_delayed": 0, 2023-12-01T01:22:45.219 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_enqueuing": 0, 2023-12-01T01:22:45.219 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_created": 2099, 2023-12-01T01:22:45.219 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_enqueued": 2096, 2023-12-01T01:22:45.219 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_reintegrated": 9, 2023-12-01T01:22:45.219 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_migrated": 0, 2023-12-01T01:22:45.219 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_processing": 0, 2023-12-01T01:22:45.219 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_enqueued": 0, 2023-12-01T01:22:45.219 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_prioritized": 0, 2023-12-01T01:22:45.219 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_started": 0, 2023-12-01T01:22:45.219 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_completed": 0, 2023-12-01T01:22:45.220 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_enqueue_scrub": 3, 2023-12-01T01:22:45.220 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_exportdir": 167, 2023-12-01T01:22:45.220 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_flush": 0, 2023-12-01T01:22:45.220 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragmentdir": 22, 2023-12-01T01:22:45.220 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragstats": 0, 2023-12-01T01:22:45.220 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_inodestats": 0 2023-12-01T01:22:45.220 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.220 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_log": { 2023-12-01T01:22:45.220 INFO:teuthology.orchestra.run.smithi137.stdout: "evadd": 40920, 2023-12-01T01:22:45.220 INFO:teuthology.orchestra.run.smithi137.stdout: "evex": 40876, 2023-12-01T01:22:45.220 INFO:teuthology.orchestra.run.smithi137.stdout: "evtrm": 40876, 2023-12-01T01:22:45.220 INFO:teuthology.orchestra.run.smithi137.stdout: "ev": 44, 2023-12-01T01:22:45.220 INFO:teuthology.orchestra.run.smithi137.stdout: "evexg": 0, 2023-12-01T01:22:45.220 INFO:teuthology.orchestra.run.smithi137.stdout: "evexd": 1211, 2023-12-01T01:22:45.220 INFO:teuthology.orchestra.run.smithi137.stdout: "segadd": 61, 2023-12-01T01:22:45.221 INFO:teuthology.orchestra.run.smithi137.stdout: "segex": 60, 2023-12-01T01:22:45.221 INFO:teuthology.orchestra.run.smithi137.stdout: "segtrm": 60, 2023-12-01T01:22:45.221 INFO:teuthology.orchestra.run.smithi137.stdout: "seg": 3, 2023-12-01T01:22:45.221 INFO:teuthology.orchestra.run.smithi137.stdout: "segexg": 0, 2023-12-01T01:22:45.221 INFO:teuthology.orchestra.run.smithi137.stdout: "segexd": 2, 2023-12-01T01:22:45.221 INFO:teuthology.orchestra.run.smithi137.stdout: "expos": 92037451, 2023-12-01T01:22:45.221 INFO:teuthology.orchestra.run.smithi137.stdout: "wrpos": 92133384, 2023-12-01T01:22:45.221 INFO:teuthology.orchestra.run.smithi137.stdout: "rdpos": 0, 2023-12-01T01:22:45.221 INFO:teuthology.orchestra.run.smithi137.stdout: "jlat": { 2023-12-01T01:22:45.221 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 26510, 2023-12-01T01:22:45.221 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 54.522312319, 2023-12-01T01:22:45.221 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.002056669 2023-12-01T01:22:45.221 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.221 INFO:teuthology.orchestra.run.smithi137.stdout: "replayed": 0 2023-12-01T01:22:45.222 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.222 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_mem": { 2023-12-01T01:22:45.222 INFO:teuthology.orchestra.run.smithi137.stdout: "ino": 35, 2023-12-01T01:22:45.222 INFO:teuthology.orchestra.run.smithi137.stdout: "ino+": 6048, 2023-12-01T01:22:45.222 INFO:teuthology.orchestra.run.smithi137.stdout: "ino-": 6013, 2023-12-01T01:22:45.222 INFO:teuthology.orchestra.run.smithi137.stdout: "dir": 34, 2023-12-01T01:22:45.222 INFO:teuthology.orchestra.run.smithi137.stdout: "dir+": 476, 2023-12-01T01:22:45.222 INFO:teuthology.orchestra.run.smithi137.stdout: "dir-": 442, 2023-12-01T01:22:45.222 INFO:teuthology.orchestra.run.smithi137.stdout: "dn": 33, 2023-12-01T01:22:45.222 INFO:teuthology.orchestra.run.smithi137.stdout: "dn+": 7497, 2023-12-01T01:22:45.222 INFO:teuthology.orchestra.run.smithi137.stdout: "dn-": 7464, 2023-12-01T01:22:45.222 INFO:teuthology.orchestra.run.smithi137.stdout: "cap": 1, 2023-12-01T01:22:45.222 INFO:teuthology.orchestra.run.smithi137.stdout: "cap+": 5977, 2023-12-01T01:22:45.222 INFO:teuthology.orchestra.run.smithi137.stdout: "cap-": 5976, 2023-12-01T01:22:45.222 INFO:teuthology.orchestra.run.smithi137.stdout: "rss": 92848, 2023-12-01T01:22:45.223 INFO:teuthology.orchestra.run.smithi137.stdout: "heap": 207156 2023-12-01T01:22:45.223 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.223 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_server": { 2023-12-01T01:22:45.223 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_client_request": 38484, 2023-12-01T01:22:45.223 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_server_request": 188, 2023-12-01T01:22:45.223 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_request": 19410, 2023-12-01T01:22:45.223 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_session": 179, 2023-12-01T01:22:45.223 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_peer_request": 351, 2023-12-01T01:22:45.223 INFO:teuthology.orchestra.run.smithi137.stdout: "req_create_latency": { 2023-12-01T01:22:45.223 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 5885, 2023-12-01T01:22:45.223 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 17.401482697, 2023-12-01T01:22:45.223 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.002956921 2023-12-01T01:22:45.223 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.223 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getattr_latency": { 2023-12-01T01:22:45.224 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 420, 2023-12-01T01:22:45.224 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 2.319200251, 2023-12-01T01:22:45.224 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.005521905 2023-12-01T01:22:45.224 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.224 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getfilelock_latency": { 2023-12-01T01:22:45.224 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:45.224 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:45.224 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:45.224 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.224 INFO:teuthology.orchestra.run.smithi137.stdout: "req_link_latency": { 2023-12-01T01:22:45.224 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2, 2023-12-01T01:22:45.224 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.002242979, 2023-12-01T01:22:45.224 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001121489 2023-12-01T01:22:45.224 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.224 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookup_latency": { 2023-12-01T01:22:45.225 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 529, 2023-12-01T01:22:45.225 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.272755160, 2023-12-01T01:22:45.225 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000515605 2023-12-01T01:22:45.225 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.225 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookuphash_latency": { 2023-12-01T01:22:45.225 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:45.225 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:45.225 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:45.225 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.225 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupino_latency": { 2023-12-01T01:22:45.225 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:45.225 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:45.225 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:45.225 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.225 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupname_latency": { 2023-12-01T01:22:45.226 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:45.226 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:45.226 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:45.226 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.226 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupparent_latency": { 2023-12-01T01:22:45.226 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:45.226 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:45.226 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:45.226 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.226 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupsnap_latency": { 2023-12-01T01:22:45.226 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:45.226 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:45.226 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:45.227 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.227 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lssnap_latency": { 2023-12-01T01:22:45.227 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:45.227 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:45.227 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:45.227 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.227 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mkdir_latency": { 2023-12-01T01:22:45.227 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 82, 2023-12-01T01:22:45.227 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.067701392, 2023-12-01T01:22:45.227 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000825626 2023-12-01T01:22:45.227 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.227 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mknod_latency": { 2023-12-01T01:22:45.227 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:45.227 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:45.227 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:45.228 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.228 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mksnap_latency": { 2023-12-01T01:22:45.228 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:45.228 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:45.228 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:45.228 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.228 INFO:teuthology.orchestra.run.smithi137.stdout: "req_open_latency": { 2023-12-01T01:22:45.228 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 321, 2023-12-01T01:22:45.228 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.079946740, 2023-12-01T01:22:45.228 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000249055 2023-12-01T01:22:45.228 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.228 INFO:teuthology.orchestra.run.smithi137.stdout: "req_readdir_latency": { 2023-12-01T01:22:45.228 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 76, 2023-12-01T01:22:45.228 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.595610681, 2023-12-01T01:22:45.228 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.007836982 2023-12-01T01:22:45.229 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.229 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rename_latency": { 2023-12-01T01:22:45.229 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 68, 2023-12-01T01:22:45.229 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.242200342, 2023-12-01T01:22:45.229 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.003561769 2023-12-01T01:22:45.229 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.229 INFO:teuthology.orchestra.run.smithi137.stdout: "req_renamesnap_latency": { 2023-12-01T01:22:45.229 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:45.229 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:45.229 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:45.229 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.229 INFO:teuthology.orchestra.run.smithi137.stdout: "req_snapdiff_latency": { 2023-12-01T01:22:45.229 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:45.229 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:45.230 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:45.230 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.230 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmdir_latency": { 2023-12-01T01:22:45.230 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 25, 2023-12-01T01:22:45.230 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 1.210414573, 2023-12-01T01:22:45.230 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.048416582 2023-12-01T01:22:45.230 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.230 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmsnap_latency": { 2023-12-01T01:22:45.230 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:45.230 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:45.230 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:45.230 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.230 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmxattr_latency": { 2023-12-01T01:22:45.230 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:45.231 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:45.231 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:45.231 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.231 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setattr_latency": { 2023-12-01T01:22:45.231 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 9628, 2023-12-01T01:22:45.231 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 34.443433760, 2023-12-01T01:22:45.231 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.003577423 2023-12-01T01:22:45.231 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.231 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setdirlayout_latency": { 2023-12-01T01:22:45.231 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:45.231 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:45.231 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:45.231 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.231 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setfilelock_latency": { 2023-12-01T01:22:45.232 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 4, 2023-12-01T01:22:45.232 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.001542971, 2023-12-01T01:22:45.232 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000385742 2023-12-01T01:22:45.232 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.232 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setlayout_latency": { 2023-12-01T01:22:45.232 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:45.232 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:45.232 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:45.232 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.232 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setxattr_latency": { 2023-12-01T01:22:45.232 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 6, 2023-12-01T01:22:45.232 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.016848804, 2023-12-01T01:22:45.232 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.002808134 2023-12-01T01:22:45.232 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.232 INFO:teuthology.orchestra.run.smithi137.stdout: "req_symlink_latency": { 2023-12-01T01:22:45.233 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 6, 2023-12-01T01:22:45.233 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.004088925, 2023-12-01T01:22:45.233 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000681487 2023-12-01T01:22:45.233 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.233 INFO:teuthology.orchestra.run.smithi137.stdout: "req_unlink_latency": { 2023-12-01T01:22:45.233 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2039, 2023-12-01T01:22:45.233 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 5.115097581, 2023-12-01T01:22:45.233 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.002508630 2023-12-01T01:22:45.233 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.233 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_revoke_eviction": 0, 2023-12-01T01:22:45.233 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_acquisition_throttle": 0, 2023-12-01T01:22:45.233 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getvxattr_latency": { 2023-12-01T01:22:45.233 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:45.233 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:45.234 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:45.234 INFO:teuthology.orchestra.run.smithi137.stdout: } 2023-12-01T01:22:45.234 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.234 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_sessions": { 2023-12-01T01:22:45.234 INFO:teuthology.orchestra.run.smithi137.stdout: "session_count": 1, 2023-12-01T01:22:45.234 INFO:teuthology.orchestra.run.smithi137.stdout: "session_add": 2, 2023-12-01T01:22:45.234 INFO:teuthology.orchestra.run.smithi137.stdout: "session_remove": 1, 2023-12-01T01:22:45.234 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_open": 1, 2023-12-01T01:22:45.234 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_stale": 0, 2023-12-01T01:22:45.234 INFO:teuthology.orchestra.run.smithi137.stdout: "total_load": 4921, 2023-12-01T01:22:45.234 INFO:teuthology.orchestra.run.smithi137.stdout: "average_load": 4921, 2023-12-01T01:22:45.234 INFO:teuthology.orchestra.run.smithi137.stdout: "avg_session_uptime": 300, 2023-12-01T01:22:45.234 INFO:teuthology.orchestra.run.smithi137.stdout: "mdthresh_evicted": 0 2023-12-01T01:22:45.234 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.235 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2023-12-01T01:22:45.235 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 240, 2023-12-01T01:22:45.235 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 240, 2023-12-01T01:22:45.235 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2023-12-01T01:22:45.235 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2023-12-01T01:22:45.235 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2023-12-01T01:22:45.235 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2023-12-01T01:22:45.235 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-01T01:22:45.235 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2023-12-01T01:22:45.235 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-01T01:22:45.235 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2023-12-01T01:22:45.235 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2023-12-01T01:22:45.235 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2023-12-01T01:22:45.235 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-01T01:22:45.236 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2023-12-01T01:22:45.236 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2023-12-01T01:22:45.236 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2023-12-01T01:22:45.236 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2023-12-01T01:22:45.236 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2023-12-01T01:22:45.236 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-01T01:22:45.236 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2023-12-01T01:22:45.236 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-01T01:22:45.236 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2023-12-01T01:22:45.236 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2023-12-01T01:22:45.236 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2023-12-01T01:22:45.236 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2023-12-01T01:22:45.236 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2023-12-01T01:22:45.236 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-01T01:22:45.237 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2023-12-01T01:22:45.237 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2023-12-01T01:22:45.237 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2023-12-01T01:22:45.237 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2023-12-01T01:22:45.237 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2023-12-01T01:22:45.237 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2023-12-01T01:22:45.237 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2023-12-01T01:22:45.237 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2023-12-01T01:22:45.237 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2023-12-01T01:22:45.237 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 298585, 2023-12-01T01:22:45.237 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 132, 2023-12-01T01:22:45.237 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 1584, 2023-12-01T01:22:45.237 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 18, 2023-12-01T01:22:45.237 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2023-12-01T01:22:45.237 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2023-12-01T01:22:45.238 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2023-12-01T01:22:45.238 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2023-12-01T01:22:45.238 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2023-12-01T01:22:45.238 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2023-12-01T01:22:45.238 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 7336, 2023-12-01T01:22:45.238 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 154, 2023-12-01T01:22:45.238 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2023-12-01T01:22:45.238 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2023-12-01T01:22:45.238 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2023-12-01T01:22:45.238 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2023-12-01T01:22:45.238 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 148079, 2023-12-01T01:22:45.238 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 903, 2023-12-01T01:22:45.238 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2023-12-01T01:22:45.238 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2023-12-01T01:22:45.239 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2023-12-01T01:22:45.239 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2023-12-01T01:22:45.239 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.239 INFO:teuthology.orchestra.run.smithi137.stdout: "objecter": { 2023-12-01T01:22:45.239 INFO:teuthology.orchestra.run.smithi137.stdout: "op_active": 0, 2023-12-01T01:22:45.239 INFO:teuthology.orchestra.run.smithi137.stdout: "op_laggy": 0, 2023-12-01T01:22:45.239 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send": 74821, 2023-12-01T01:22:45.239 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send_bytes": 93939532, 2023-12-01T01:22:45.239 INFO:teuthology.orchestra.run.smithi137.stdout: "op_resend": 37, 2023-12-01T01:22:45.239 INFO:teuthology.orchestra.run.smithi137.stdout: "op_reply": 74821, 2023-12-01T01:22:45.239 INFO:teuthology.orchestra.run.smithi137.stdout: "op_latency": { 2023-12-01T01:22:45.239 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 74821, 2023-12-01T01:22:45.239 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 579.918668655, 2023-12-01T01:22:45.239 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.007750747 2023-12-01T01:22:45.239 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.240 INFO:teuthology.orchestra.run.smithi137.stdout: "op_inflight": 0, 2023-12-01T01:22:45.240 INFO:teuthology.orchestra.run.smithi137.stdout: "oplen_avg": { 2023-12-01T01:22:45.240 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 74821, 2023-12-01T01:22:45.240 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 82897 2023-12-01T01:22:45.240 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.240 INFO:teuthology.orchestra.run.smithi137.stdout: "op": 74821, 2023-12-01T01:22:45.240 INFO:teuthology.orchestra.run.smithi137.stdout: "op_r": 2630, 2023-12-01T01:22:45.240 INFO:teuthology.orchestra.run.smithi137.stdout: "op_w": 72191, 2023-12-01T01:22:45.240 INFO:teuthology.orchestra.run.smithi137.stdout: "op_rmw": 0, 2023-12-01T01:22:45.240 INFO:teuthology.orchestra.run.smithi137.stdout: "op_pg": 0, 2023-12-01T01:22:45.240 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_stat": 381, 2023-12-01T01:22:45.240 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_create": 3474, 2023-12-01T01:22:45.240 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_read": 46, 2023-12-01T01:22:45.240 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_write": 26553, 2023-12-01T01:22:45.241 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writefull": 101, 2023-12-01T01:22:45.241 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writesame": 0, 2023-12-01T01:22:45.241 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_append": 0, 2023-12-01T01:22:45.241 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_zero": 0, 2023-12-01T01:22:45.241 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_truncate": 6, 2023-12-01T01:22:45.241 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_delete": 2691, 2023-12-01T01:22:45.241 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_mapext": 0, 2023-12-01T01:22:45.241 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_sparse_read": 0, 2023-12-01T01:22:45.241 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_clonerange": 0, 2023-12-01T01:22:45.241 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_getxattr": 2584, 2023-12-01T01:22:45.241 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_setxattr": 6950, 2023-12-01T01:22:45.241 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_cmpxattr": 0, 2023-12-01T01:22:45.241 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_rmxattr": 0, 2023-12-01T01:22:45.241 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_resetxattrs": 0, 2023-12-01T01:22:45.241 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_call": 0, 2023-12-01T01:22:45.242 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_watch": 0, 2023-12-01T01:22:45.242 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_notify": 0, 2023-12-01T01:22:45.242 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_src_cmpxattr": 0, 2023-12-01T01:22:45.242 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls": 0, 2023-12-01T01:22:45.242 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls_filter": 0, 2023-12-01T01:22:45.242 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_other": 38661, 2023-12-01T01:22:45.242 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_active": 0, 2023-12-01T01:22:45.242 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_send": 0, 2023-12-01T01:22:45.242 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_resend": 0, 2023-12-01T01:22:45.242 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_ping": 0, 2023-12-01T01:22:45.242 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_active": 0, 2023-12-01T01:22:45.242 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_send": 0, 2023-12-01T01:22:45.242 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_resend": 0, 2023-12-01T01:22:45.242 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_active": 0, 2023-12-01T01:22:45.242 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_send": 0, 2023-12-01T01:22:45.243 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_resend": 0, 2023-12-01T01:22:45.243 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_active": 0, 2023-12-01T01:22:45.243 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_send": 0, 2023-12-01T01:22:45.243 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_resend": 0, 2023-12-01T01:22:45.243 INFO:teuthology.orchestra.run.smithi137.stdout: "command_active": 0, 2023-12-01T01:22:45.243 INFO:teuthology.orchestra.run.smithi137.stdout: "command_send": 0, 2023-12-01T01:22:45.243 INFO:teuthology.orchestra.run.smithi137.stdout: "command_resend": 0, 2023-12-01T01:22:45.243 INFO:teuthology.orchestra.run.smithi137.stdout: "map_epoch": 84, 2023-12-01T01:22:45.243 INFO:teuthology.orchestra.run.smithi137.stdout: "map_full": 0, 2023-12-01T01:22:45.243 INFO:teuthology.orchestra.run.smithi137.stdout: "map_inc": 3, 2023-12-01T01:22:45.243 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_sessions": 12, 2023-12-01T01:22:45.243 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_open": 12, 2023-12-01T01:22:45.243 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_close": 0, 2023-12-01T01:22:45.243 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_laggy": 0, 2023-12-01T01:22:45.243 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_wr": 1315, 2023-12-01T01:22:45.244 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_rd": 10, 2023-12-01T01:22:45.244 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_del": 125 2023-12-01T01:22:45.244 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.244 INFO:teuthology.orchestra.run.smithi137.stdout: "oft": { 2023-12-01T01:22:45.244 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_objs": 1, 2023-12-01T01:22:45.244 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_kv_pairs": 0, 2023-12-01T01:22:45.244 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_updates": 0, 2023-12-01T01:22:45.244 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_removes": 0 2023-12-01T01:22:45.244 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.244 INFO:teuthology.orchestra.run.smithi137.stdout: "osd-slow-ops": { 2023-12-01T01:22:45.244 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_ops_count": 19767 2023-12-01T01:22:45.244 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.244 INFO:teuthology.orchestra.run.smithi137.stdout: "purge_queue": { 2023-12-01T01:22:45.244 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops": 0, 2023-12-01T01:22:45.244 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops_high_water": 27, 2023-12-01T01:22:45.245 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing": 0, 2023-12-01T01:22:45.245 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_high_water": 6, 2023-12-01T01:22:45.245 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed": 2096, 2023-12-01T01:22:45.245 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_item_in_journal": 0 2023-12-01T01:22:45.245 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.245 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-01T01:22:45.245 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2023-12-01T01:22:45.245 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2023-12-01T01:22:45.245 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2023-12-01T01:22:45.245 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 597248, 2023-12-01T01:22:45.245 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 57170959, 2023-12-01T01:22:45.245 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:45.245 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 597248, 2023-12-01T01:22:45.245 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2023-12-01T01:22:45.245 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2023-12-01T01:22:45.246 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 597248, 2023-12-01T01:22:45.246 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 57170959, 2023-12-01T01:22:45.246 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2023-12-01T01:22:45.246 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:45.246 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:45.246 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:45.246 INFO:teuthology.orchestra.run.smithi137.stdout: } 2023-12-01T01:22:45.246 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.246 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_bytes": { 2023-12-01T01:22:45.246 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2023-12-01T01:22:45.246 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2023-12-01T01:22:45.246 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2023-12-01T01:22:45.246 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2023-12-01T01:22:45.246 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2023-12-01T01:22:45.247 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:45.247 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2023-12-01T01:22:45.247 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 74821, 2023-12-01T01:22:45.247 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 94151160, 2023-12-01T01:22:45.247 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 74821, 2023-12-01T01:22:45.247 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 94151160, 2023-12-01T01:22:45.247 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2023-12-01T01:22:45.247 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:45.247 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:45.247 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:45.247 INFO:teuthology.orchestra.run.smithi137.stdout: } 2023-12-01T01:22:45.247 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.247 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_ops": { 2023-12-01T01:22:45.247 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2023-12-01T01:22:45.247 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 1024, 2023-12-01T01:22:45.248 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2023-12-01T01:22:45.248 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2023-12-01T01:22:45.248 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2023-12-01T01:22:45.248 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:45.248 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2023-12-01T01:22:45.248 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 74821, 2023-12-01T01:22:45.248 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 74821, 2023-12-01T01:22:45.248 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 74821, 2023-12-01T01:22:45.248 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 74821, 2023-12-01T01:22:45.248 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2023-12-01T01:22:45.248 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:45.248 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:45.248 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:45.248 INFO:teuthology.orchestra.run.smithi137.stdout: } 2023-12-01T01:22:45.249 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.249 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle": { 2023-12-01T01:22:45.249 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2023-12-01T01:22:45.249 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2023-12-01T01:22:45.249 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2023-12-01T01:22:45.249 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 2096, 2023-12-01T01:22:45.249 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 211696, 2023-12-01T01:22:45.249 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:45.249 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 2096, 2023-12-01T01:22:45.249 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2023-12-01T01:22:45.249 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2023-12-01T01:22:45.249 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 43, 2023-12-01T01:22:45.249 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 211696, 2023-12-01T01:22:45.249 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2023-12-01T01:22:45.249 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:45.250 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:45.250 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:45.250 INFO:teuthology.orchestra.run.smithi137.stdout: } 2023-12-01T01:22:45.250 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:45.250 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle-0x5630f2d060a0": { 2023-12-01T01:22:45.250 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2023-12-01T01:22:45.250 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2023-12-01T01:22:45.250 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2023-12-01T01:22:45.250 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 40920, 2023-12-01T01:22:45.250 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 87939080, 2023-12-01T01:22:45.250 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:45.250 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 40920, 2023-12-01T01:22:45.250 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2023-12-01T01:22:45.250 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2023-12-01T01:22:45.251 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 26510, 2023-12-01T01:22:45.251 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 87939080, 2023-12-01T01:22:45.251 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2023-12-01T01:22:45.251 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:45.251 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:45.251 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:45.251 INFO:teuthology.orchestra.run.smithi137.stdout: } 2023-12-01T01:22:45.251 INFO:teuthology.orchestra.run.smithi137.stdout: } 2023-12-01T01:22:45.251 INFO:teuthology.orchestra.run.smithi137.stdout:} 2023-12-01T01:22:45.528 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:45 smithi137 ceph-mon[103312]: pgmap v444: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 13 op/s 2023-12-01T01:22:45.645 INFO:tasks.check_counter:Daemon mds.b mds.exported=167 2023-12-01T01:22:45.645 INFO:tasks.check_counter:Daemon mds.b mds.imported=1 2023-12-01T01:22:45.645 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=114 2023-12-01T01:22:45.645 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=431 2023-12-01T01:22:45.645 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=22 2023-12-01T01:22:45.646 DEBUG:tasks.check_counter:Getting stats from e 2023-12-01T01:22:45.646 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph daemon mds.e perf dump 2023-12-01T01:22:45.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:45 smithi203 ceph-mon[103581]: pgmap v444: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 13 op/s 2023-12-01T01:22:45.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:45 smithi077 ceph-mon[98867]: pgmap v444: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 682 B/s wr, 13 op/s 2023-12-01T01:22:46.199 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.b/config 2023-12-01T01:22:47.573 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2023-12-01T01:22:47.573 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2023-12-01T01:22:47.573 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 513, 2023-12-01T01:22:47.573 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 493, 2023-12-01T01:22:47.573 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 682144, 2023-12-01T01:22:47.573 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 180177, 2023-12-01T01:22:47.573 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 5, 2023-12-01T01:22:47.573 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 1, 2023-12-01T01:22:47.573 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.053944453, 2023-12-01T01:22:47.574 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.018854739, 2023-12-01T01:22:47.574 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.019990522, 2023-12-01T01:22:47.574 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.013425433, 2023-12-01T01:22:47.574 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2023-12-01T01:22:47.574 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 492, 2023-12-01T01:22:47.574 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.023693974, 2023-12-01T01:22:47.574 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000048158 2023-12-01T01:22:47.574 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.574 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2023-12-01T01:22:47.574 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:47.574 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:47.574 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:47.574 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.575 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 556144, 2023-12-01T01:22:47.575 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 63552 2023-12-01T01:22:47.575 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.575 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2023-12-01T01:22:47.575 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 37, 2023-12-01T01:22:47.575 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 120, 2023-12-01T01:22:47.575 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 9080, 2023-12-01T01:22:47.575 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 228506, 2023-12-01T01:22:47.575 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 5, 2023-12-01T01:22:47.575 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 4, 2023-12-01T01:22:47.575 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.008617450, 2023-12-01T01:22:47.575 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.003673626, 2023-12-01T01:22:47.575 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.002490209, 2023-12-01T01:22:47.575 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.001342776, 2023-12-01T01:22:47.575 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2023-12-01T01:22:47.576 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 118, 2023-12-01T01:22:47.576 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.006488999, 2023-12-01T01:22:47.576 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000054991 2023-12-01T01:22:47.576 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.576 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2023-12-01T01:22:47.576 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:47.576 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:47.576 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:47.576 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.576 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 160, 2023-12-01T01:22:47.576 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 217248 2023-12-01T01:22:47.576 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.576 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2023-12-01T01:22:47.576 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 697, 2023-12-01T01:22:47.577 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 697, 2023-12-01T01:22:47.577 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 10861319, 2023-12-01T01:22:47.577 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 215978, 2023-12-01T01:22:47.577 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 4, 2023-12-01T01:22:47.577 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 3, 2023-12-01T01:22:47.577 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.046079738, 2023-12-01T01:22:47.577 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.009397287, 2023-12-01T01:22:47.577 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.017372342, 2023-12-01T01:22:47.577 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.016532713, 2023-12-01T01:22:47.577 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2023-12-01T01:22:47.577 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 696, 2023-12-01T01:22:47.577 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.016032810, 2023-12-01T01:22:47.577 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000023035 2023-12-01T01:22:47.577 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.578 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2023-12-01T01:22:47.578 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:47.578 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:47.578 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:47.578 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.578 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-12-01T01:22:47.578 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 448 2023-12-01T01:22:47.578 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.578 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2023-12-01T01:22:47.578 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 1, 2023-12-01T01:22:47.578 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2023-12-01T01:22:47.578 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.578 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-MDSRank": { 2023-12-01T01:22:47.578 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2023-12-01T01:22:47.579 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2023-12-01T01:22:47.579 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1895, 2023-12-01T01:22:47.579 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.046448675, 2023-12-01T01:22:47.579 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000024511 2023-12-01T01:22:47.579 INFO:teuthology.orchestra.run.smithi137.stdout: } 2023-12-01T01:22:47.579 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.579 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-PurgeQueue": { 2023-12-01T01:22:47.579 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2023-12-01T01:22:47.579 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2023-12-01T01:22:47.579 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:47.579 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:47.579 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:47.579 INFO:teuthology.orchestra.run.smithi137.stdout: } 2023-12-01T01:22:47.579 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.580 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2023-12-01T01:22:47.580 INFO:teuthology.orchestra.run.smithi137.stdout: "request": 0, 2023-12-01T01:22:47.580 INFO:teuthology.orchestra.run.smithi137.stdout: "reply": 0, 2023-12-01T01:22:47.580 INFO:teuthology.orchestra.run.smithi137.stdout: "reply_latency": { 2023-12-01T01:22:47.580 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:47.580 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:47.580 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:47.580 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.580 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_reply": 0, 2023-12-01T01:22:47.580 INFO:teuthology.orchestra.run.smithi137.stdout: "forward": 0, 2023-12-01T01:22:47.580 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_complete": 0, 2023-12-01T01:22:47.580 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_keys": 0, 2023-12-01T01:22:47.580 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_commit": 0, 2023-12-01T01:22:47.580 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_split": 0, 2023-12-01T01:22:47.580 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_merge": 0, 2023-12-01T01:22:47.581 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes": 1935, 2023-12-01T01:22:47.581 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_top": 0, 2023-12-01T01:22:47.581 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_bottom": 1, 2023-12-01T01:22:47.581 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pin_tail": 1934, 2023-12-01T01:22:47.581 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pinned": 1934, 2023-12-01T01:22:47.581 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_expired": 85, 2023-12-01T01:22:47.581 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_with_caps": 0, 2023-12-01T01:22:47.581 INFO:teuthology.orchestra.run.smithi137.stdout: "caps": 0, 2023-12-01T01:22:47.582 INFO:teuthology.orchestra.run.smithi137.stdout: "subtrees": 3, 2023-12-01T01:22:47.582 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse": 0, 2023-12-01T01:22:47.582 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_hit": 0, 2023-12-01T01:22:47.582 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_forward": 0, 2023-12-01T01:22:47.582 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_discover": 0, 2023-12-01T01:22:47.582 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_dir_fetch": 0, 2023-12-01T01:22:47.582 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_remote_ino": 0, 2023-12-01T01:22:47.582 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_lock": 0, 2023-12-01T01:22:47.582 INFO:teuthology.orchestra.run.smithi137.stdout: "load_cent": 0, 2023-12-01T01:22:47.582 INFO:teuthology.orchestra.run.smithi137.stdout: "q": 0, 2023-12-01T01:22:47.582 INFO:teuthology.orchestra.run.smithi137.stdout: "exported": 0, 2023-12-01T01:22:47.582 INFO:teuthology.orchestra.run.smithi137.stdout: "exported_inodes": 0, 2023-12-01T01:22:47.582 INFO:teuthology.orchestra.run.smithi137.stdout: "imported": 0, 2023-12-01T01:22:47.582 INFO:teuthology.orchestra.run.smithi137.stdout: "imported_inodes": 0, 2023-12-01T01:22:47.583 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_dir_fetch": 0, 2023-12-01T01:22:47.583 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_backtrace_fetch": 0, 2023-12-01T01:22:47.583 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_peer_discover": 0, 2023-12-01T01:22:47.583 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rfiles": 0, 2023-12-01T01:22:47.583 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rbytes": 0, 2023-12-01T01:22:47.583 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rsnaps": 0, 2023-12-01T01:22:47.583 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_fetch": 0, 2023-12-01T01:22:47.583 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_set_tag": 0, 2023-12-01T01:22:47.583 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_repaired": 0, 2023-12-01T01:22:47.583 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_inotable_repaired": 0, 2023-12-01T01:22:47.583 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_inodes": 0, 2023-12-01T01:22:47.583 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_base_inodes": 0, 2023-12-01T01:22:47.583 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dirfrag_rstats": 0, 2023-12-01T01:22:47.583 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_file_inodes": 0, 2023-12-01T01:22:47.583 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_inode_file_caps": 0, 2023-12-01T01:22:47.584 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_revoke": 0, 2023-12-01T01:22:47.584 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_grant": 0, 2023-12-01T01:22:47.584 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_trunc": 0, 2023-12-01T01:22:47.584 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-01T01:22:47.584 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-01T01:22:47.584 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps": 0, 2023-12-01T01:22:47.584 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps_dirty": 0, 2023-12-01T01:22:47.584 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_cap_release": 0, 2023-12-01T01:22:47.584 INFO:teuthology.orchestra.run.smithi137.stdout: "process_request_cap_release": 0 2023-12-01T01:22:47.584 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.584 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_cache": { 2023-12-01T01:22:47.584 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update": 0, 2023-12-01T01:22:47.584 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update_receipt": 0, 2023-12-01T01:22:47.584 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_try_discover": 0, 2023-12-01T01:22:47.585 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_send_discover": 0, 2023-12-01T01:22:47.585 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_handle_discover": 0, 2023-12-01T01:22:47.585 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays": 0, 2023-12-01T01:22:47.585 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_delayed": 0, 2023-12-01T01:22:47.585 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_enqueuing": 0, 2023-12-01T01:22:47.585 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_created": 0, 2023-12-01T01:22:47.585 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_enqueued": 0, 2023-12-01T01:22:47.585 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_reintegrated": 0, 2023-12-01T01:22:47.585 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_migrated": 0, 2023-12-01T01:22:47.585 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_processing": 0, 2023-12-01T01:22:47.585 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_enqueued": 0, 2023-12-01T01:22:47.585 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_prioritized": 0, 2023-12-01T01:22:47.585 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_started": 0, 2023-12-01T01:22:47.585 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_completed": 0, 2023-12-01T01:22:47.585 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_enqueue_scrub": 0, 2023-12-01T01:22:47.585 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_exportdir": 0, 2023-12-01T01:22:47.586 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_flush": 0, 2023-12-01T01:22:47.586 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragmentdir": 0, 2023-12-01T01:22:47.586 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragstats": 0, 2023-12-01T01:22:47.586 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_inodestats": 0 2023-12-01T01:22:47.586 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.586 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_log": { 2023-12-01T01:22:47.586 INFO:teuthology.orchestra.run.smithi137.stdout: "evadd": 0, 2023-12-01T01:22:47.586 INFO:teuthology.orchestra.run.smithi137.stdout: "evex": 0, 2023-12-01T01:22:47.586 INFO:teuthology.orchestra.run.smithi137.stdout: "evtrm": 0, 2023-12-01T01:22:47.586 INFO:teuthology.orchestra.run.smithi137.stdout: "ev": 6562, 2023-12-01T01:22:47.586 INFO:teuthology.orchestra.run.smithi137.stdout: "evexg": 0, 2023-12-01T01:22:47.586 INFO:teuthology.orchestra.run.smithi137.stdout: "evexd": 0, 2023-12-01T01:22:47.586 INFO:teuthology.orchestra.run.smithi137.stdout: "segadd": 0, 2023-12-01T01:22:47.586 INFO:teuthology.orchestra.run.smithi137.stdout: "segex": 0, 2023-12-01T01:22:47.587 INFO:teuthology.orchestra.run.smithi137.stdout: "segtrm": 0, 2023-12-01T01:22:47.587 INFO:teuthology.orchestra.run.smithi137.stdout: "seg": 8, 2023-12-01T01:22:47.587 INFO:teuthology.orchestra.run.smithi137.stdout: "segexg": 0, 2023-12-01T01:22:47.587 INFO:teuthology.orchestra.run.smithi137.stdout: "segexd": 0, 2023-12-01T01:22:47.587 INFO:teuthology.orchestra.run.smithi137.stdout: "expos": 4194304, 2023-12-01T01:22:47.587 INFO:teuthology.orchestra.run.smithi137.stdout: "wrpos": 14875204, 2023-12-01T01:22:47.587 INFO:teuthology.orchestra.run.smithi137.stdout: "rdpos": 14874066, 2023-12-01T01:22:47.587 INFO:teuthology.orchestra.run.smithi137.stdout: "jlat": { 2023-12-01T01:22:47.587 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:47.587 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:47.587 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:47.587 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.587 INFO:teuthology.orchestra.run.smithi137.stdout: "replayed": 6562 2023-12-01T01:22:47.587 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.587 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_mem": { 2023-12-01T01:22:47.588 INFO:teuthology.orchestra.run.smithi137.stdout: "ino": 21, 2023-12-01T01:22:47.588 INFO:teuthology.orchestra.run.smithi137.stdout: "ino+": 1640, 2023-12-01T01:22:47.588 INFO:teuthology.orchestra.run.smithi137.stdout: "ino-": 1619, 2023-12-01T01:22:47.588 INFO:teuthology.orchestra.run.smithi137.stdout: "dir": 82, 2023-12-01T01:22:47.588 INFO:teuthology.orchestra.run.smithi137.stdout: "dir+": 252, 2023-12-01T01:22:47.588 INFO:teuthology.orchestra.run.smithi137.stdout: "dir-": 170, 2023-12-01T01:22:47.588 INFO:teuthology.orchestra.run.smithi137.stdout: "dn": 1935, 2023-12-01T01:22:47.588 INFO:teuthology.orchestra.run.smithi137.stdout: "dn+": 3833, 2023-12-01T01:22:47.588 INFO:teuthology.orchestra.run.smithi137.stdout: "dn-": 1898, 2023-12-01T01:22:47.588 INFO:teuthology.orchestra.run.smithi137.stdout: "cap": 0, 2023-12-01T01:22:47.588 INFO:teuthology.orchestra.run.smithi137.stdout: "cap+": 0, 2023-12-01T01:22:47.588 INFO:teuthology.orchestra.run.smithi137.stdout: "cap-": 0, 2023-12-01T01:22:47.588 INFO:teuthology.orchestra.run.smithi137.stdout: "rss": 48908, 2023-12-01T01:22:47.588 INFO:teuthology.orchestra.run.smithi137.stdout: "heap": 207156 2023-12-01T01:22:47.588 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.589 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_server": { 2023-12-01T01:22:47.589 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_client_request": 0, 2023-12-01T01:22:47.589 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_server_request": 0, 2023-12-01T01:22:47.589 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_request": 0, 2023-12-01T01:22:47.589 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_session": 0, 2023-12-01T01:22:47.589 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_peer_request": 0, 2023-12-01T01:22:47.589 INFO:teuthology.orchestra.run.smithi137.stdout: "req_create_latency": { 2023-12-01T01:22:47.589 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:47.589 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:47.589 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:47.589 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.589 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getattr_latency": { 2023-12-01T01:22:47.589 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:47.589 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:47.589 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:47.590 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.590 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getfilelock_latency": { 2023-12-01T01:22:47.590 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:47.590 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:47.590 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:47.590 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.590 INFO:teuthology.orchestra.run.smithi137.stdout: "req_link_latency": { 2023-12-01T01:22:47.590 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:47.590 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:47.590 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:47.590 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.590 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookup_latency": { 2023-12-01T01:22:47.590 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:47.590 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:47.590 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:47.591 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.591 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookuphash_latency": { 2023-12-01T01:22:47.591 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:47.591 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:47.591 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:47.591 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.591 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupino_latency": { 2023-12-01T01:22:47.591 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:47.591 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:47.591 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:47.591 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.591 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupname_latency": { 2023-12-01T01:22:47.591 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:47.591 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:47.592 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:47.592 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.592 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupparent_latency": { 2023-12-01T01:22:47.592 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:47.592 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:47.592 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:47.592 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.592 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupsnap_latency": { 2023-12-01T01:22:47.592 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:47.592 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:47.592 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:47.592 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.592 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lssnap_latency": { 2023-12-01T01:22:47.592 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:47.592 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:47.593 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:47.593 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.593 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mkdir_latency": { 2023-12-01T01:22:47.593 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:47.593 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:47.593 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:47.593 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.593 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mknod_latency": { 2023-12-01T01:22:47.593 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:47.593 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:47.593 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:47.593 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.593 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mksnap_latency": { 2023-12-01T01:22:47.593 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:47.594 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:47.594 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:47.594 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.594 INFO:teuthology.orchestra.run.smithi137.stdout: "req_open_latency": { 2023-12-01T01:22:47.594 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:47.594 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:47.594 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:47.594 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.594 INFO:teuthology.orchestra.run.smithi137.stdout: "req_readdir_latency": { 2023-12-01T01:22:47.594 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:47.594 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:47.594 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:47.594 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.594 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rename_latency": { 2023-12-01T01:22:47.594 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:47.595 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:47.595 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:47.595 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.595 INFO:teuthology.orchestra.run.smithi137.stdout: "req_renamesnap_latency": { 2023-12-01T01:22:47.595 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:47.595 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:47.595 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:47.595 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.595 INFO:teuthology.orchestra.run.smithi137.stdout: "req_snapdiff_latency": { 2023-12-01T01:22:47.595 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:47.595 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:47.595 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:47.595 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.595 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmdir_latency": { 2023-12-01T01:22:47.596 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:47.596 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:47.596 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:47.596 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.596 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmsnap_latency": { 2023-12-01T01:22:47.596 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:47.596 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:47.596 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:47.596 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.596 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmxattr_latency": { 2023-12-01T01:22:47.596 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:47.596 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:47.596 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:47.596 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.596 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setattr_latency": { 2023-12-01T01:22:47.597 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:47.597 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:47.597 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:47.597 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.597 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setdirlayout_latency": { 2023-12-01T01:22:47.597 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:47.597 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:47.597 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:47.597 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.597 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setfilelock_latency": { 2023-12-01T01:22:47.597 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:47.597 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:47.597 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:47.597 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.597 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setlayout_latency": { 2023-12-01T01:22:47.598 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:47.598 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:47.598 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:47.598 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.598 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setxattr_latency": { 2023-12-01T01:22:47.598 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:47.598 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:47.598 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:47.598 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.598 INFO:teuthology.orchestra.run.smithi137.stdout: "req_symlink_latency": { 2023-12-01T01:22:47.598 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:47.598 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:47.598 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:47.598 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.599 INFO:teuthology.orchestra.run.smithi137.stdout: "req_unlink_latency": { 2023-12-01T01:22:47.599 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:47.599 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:47.599 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:47.599 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.599 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_revoke_eviction": 0, 2023-12-01T01:22:47.599 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_acquisition_throttle": 0, 2023-12-01T01:22:47.599 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getvxattr_latency": { 2023-12-01T01:22:47.599 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:47.599 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:47.599 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:47.600 INFO:teuthology.orchestra.run.smithi137.stdout: } 2023-12-01T01:22:47.600 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.600 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_sessions": { 2023-12-01T01:22:47.600 INFO:teuthology.orchestra.run.smithi137.stdout: "session_count": 1, 2023-12-01T01:22:47.600 INFO:teuthology.orchestra.run.smithi137.stdout: "session_add": 2, 2023-12-01T01:22:47.600 INFO:teuthology.orchestra.run.smithi137.stdout: "session_remove": 1, 2023-12-01T01:22:47.600 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_open": 1, 2023-12-01T01:22:47.600 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_stale": 0, 2023-12-01T01:22:47.600 INFO:teuthology.orchestra.run.smithi137.stdout: "total_load": 0, 2023-12-01T01:22:47.600 INFO:teuthology.orchestra.run.smithi137.stdout: "average_load": 0, 2023-12-01T01:22:47.600 INFO:teuthology.orchestra.run.smithi137.stdout: "avg_session_uptime": 1664, 2023-12-01T01:22:47.600 INFO:teuthology.orchestra.run.smithi137.stdout: "mdthresh_evicted": 0 2023-12-01T01:22:47.600 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.600 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2023-12-01T01:22:47.600 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2023-12-01T01:22:47.601 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2023-12-01T01:22:47.601 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2023-12-01T01:22:47.601 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2023-12-01T01:22:47.601 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2023-12-01T01:22:47.601 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2023-12-01T01:22:47.601 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-01T01:22:47.601 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2023-12-01T01:22:47.601 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-01T01:22:47.601 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2023-12-01T01:22:47.601 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2023-12-01T01:22:47.601 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2023-12-01T01:22:47.601 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-01T01:22:47.601 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2023-12-01T01:22:47.601 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2023-12-01T01:22:47.601 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2023-12-01T01:22:47.602 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2023-12-01T01:22:47.602 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2023-12-01T01:22:47.602 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-01T01:22:47.602 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2023-12-01T01:22:47.602 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-01T01:22:47.602 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2023-12-01T01:22:47.602 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2023-12-01T01:22:47.602 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2023-12-01T01:22:47.602 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2023-12-01T01:22:47.602 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2023-12-01T01:22:47.602 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-01T01:22:47.602 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2023-12-01T01:22:47.602 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2023-12-01T01:22:47.602 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2023-12-01T01:22:47.602 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2023-12-01T01:22:47.603 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2023-12-01T01:22:47.603 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2023-12-01T01:22:47.603 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2023-12-01T01:22:47.603 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2023-12-01T01:22:47.603 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2023-12-01T01:22:47.603 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 189426, 2023-12-01T01:22:47.603 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 54, 2023-12-01T01:22:47.603 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 0, 2023-12-01T01:22:47.603 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 0, 2023-12-01T01:22:47.603 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2023-12-01T01:22:47.603 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2023-12-01T01:22:47.603 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2023-12-01T01:22:47.603 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2023-12-01T01:22:47.603 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2023-12-01T01:22:47.603 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2023-12-01T01:22:47.604 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 7336, 2023-12-01T01:22:47.604 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 154, 2023-12-01T01:22:47.604 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2023-12-01T01:22:47.604 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2023-12-01T01:22:47.604 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2023-12-01T01:22:47.604 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2023-12-01T01:22:47.604 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 1505409, 2023-12-01T01:22:47.604 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 8638, 2023-12-01T01:22:47.604 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2023-12-01T01:22:47.604 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2023-12-01T01:22:47.604 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2023-12-01T01:22:47.604 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2023-12-01T01:22:47.604 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.605 INFO:teuthology.orchestra.run.smithi137.stdout: "objecter": { 2023-12-01T01:22:47.605 INFO:teuthology.orchestra.run.smithi137.stdout: "op_active": 0, 2023-12-01T01:22:47.605 INFO:teuthology.orchestra.run.smithi137.stdout: "op_laggy": 0, 2023-12-01T01:22:47.605 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send": 1104, 2023-12-01T01:22:47.605 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send_bytes": 21, 2023-12-01T01:22:47.605 INFO:teuthology.orchestra.run.smithi137.stdout: "op_resend": 3, 2023-12-01T01:22:47.605 INFO:teuthology.orchestra.run.smithi137.stdout: "op_reply": 1104, 2023-12-01T01:22:47.605 INFO:teuthology.orchestra.run.smithi137.stdout: "op_latency": { 2023-12-01T01:22:47.605 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1104, 2023-12-01T01:22:47.605 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.731986985, 2023-12-01T01:22:47.605 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000663031 2023-12-01T01:22:47.605 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.605 INFO:teuthology.orchestra.run.smithi137.stdout: "op_inflight": 0, 2023-12-01T01:22:47.606 INFO:teuthology.orchestra.run.smithi137.stdout: "oplen_avg": { 2023-12-01T01:22:47.606 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1104, 2023-12-01T01:22:47.606 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 1105 2023-12-01T01:22:47.606 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.606 INFO:teuthology.orchestra.run.smithi137.stdout: "op": 1104, 2023-12-01T01:22:47.606 INFO:teuthology.orchestra.run.smithi137.stdout: "op_r": 1104, 2023-12-01T01:22:47.606 INFO:teuthology.orchestra.run.smithi137.stdout: "op_w": 0, 2023-12-01T01:22:47.606 INFO:teuthology.orchestra.run.smithi137.stdout: "op_rmw": 0, 2023-12-01T01:22:47.606 INFO:teuthology.orchestra.run.smithi137.stdout: "op_pg": 0, 2023-12-01T01:22:47.606 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_stat": 667, 2023-12-01T01:22:47.606 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_create": 0, 2023-12-01T01:22:47.606 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_read": 435, 2023-12-01T01:22:47.606 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_write": 0, 2023-12-01T01:22:47.607 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writefull": 0, 2023-12-01T01:22:47.607 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writesame": 0, 2023-12-01T01:22:47.607 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_append": 0, 2023-12-01T01:22:47.607 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_zero": 0, 2023-12-01T01:22:47.607 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_truncate": 0, 2023-12-01T01:22:47.607 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_delete": 0, 2023-12-01T01:22:47.607 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_mapext": 0, 2023-12-01T01:22:47.607 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_sparse_read": 0, 2023-12-01T01:22:47.607 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_clonerange": 0, 2023-12-01T01:22:47.607 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_getxattr": 1, 2023-12-01T01:22:47.607 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_setxattr": 0, 2023-12-01T01:22:47.607 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_cmpxattr": 0, 2023-12-01T01:22:47.607 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_rmxattr": 0, 2023-12-01T01:22:47.607 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_resetxattrs": 0, 2023-12-01T01:22:47.607 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_call": 0, 2023-12-01T01:22:47.608 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_watch": 0, 2023-12-01T01:22:47.608 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_notify": 0, 2023-12-01T01:22:47.608 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_src_cmpxattr": 0, 2023-12-01T01:22:47.608 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls": 0, 2023-12-01T01:22:47.608 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls_filter": 0, 2023-12-01T01:22:47.608 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_other": 0, 2023-12-01T01:22:47.608 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_active": 0, 2023-12-01T01:22:47.608 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_send": 0, 2023-12-01T01:22:47.608 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_resend": 0, 2023-12-01T01:22:47.608 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_ping": 0, 2023-12-01T01:22:47.608 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_active": 0, 2023-12-01T01:22:47.608 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_send": 0, 2023-12-01T01:22:47.608 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_resend": 0, 2023-12-01T01:22:47.608 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_active": 0, 2023-12-01T01:22:47.608 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_send": 0, 2023-12-01T01:22:47.609 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_resend": 0, 2023-12-01T01:22:47.609 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_active": 0, 2023-12-01T01:22:47.609 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_send": 0, 2023-12-01T01:22:47.609 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_resend": 0, 2023-12-01T01:22:47.609 INFO:teuthology.orchestra.run.smithi137.stdout: "command_active": 0, 2023-12-01T01:22:47.609 INFO:teuthology.orchestra.run.smithi137.stdout: "command_send": 0, 2023-12-01T01:22:47.609 INFO:teuthology.orchestra.run.smithi137.stdout: "command_resend": 0, 2023-12-01T01:22:47.609 INFO:teuthology.orchestra.run.smithi137.stdout: "map_epoch": 84, 2023-12-01T01:22:47.609 INFO:teuthology.orchestra.run.smithi137.stdout: "map_full": 0, 2023-12-01T01:22:47.609 INFO:teuthology.orchestra.run.smithi137.stdout: "map_inc": 2, 2023-12-01T01:22:47.609 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_sessions": 5, 2023-12-01T01:22:47.609 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_open": 5, 2023-12-01T01:22:47.609 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_close": 0, 2023-12-01T01:22:47.609 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_laggy": 0, 2023-12-01T01:22:47.610 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_wr": 0, 2023-12-01T01:22:47.610 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_rd": 2, 2023-12-01T01:22:47.610 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_del": 0 2023-12-01T01:22:47.610 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.610 INFO:teuthology.orchestra.run.smithi137.stdout: "oft": { 2023-12-01T01:22:47.610 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_objs": 0, 2023-12-01T01:22:47.610 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_kv_pairs": 0, 2023-12-01T01:22:47.610 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_updates": 0, 2023-12-01T01:22:47.610 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_removes": 0 2023-12-01T01:22:47.610 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.610 INFO:teuthology.orchestra.run.smithi137.stdout: "osd-slow-ops": { 2023-12-01T01:22:47.610 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_ops_count": 0 2023-12-01T01:22:47.610 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.610 INFO:teuthology.orchestra.run.smithi137.stdout: "purge_queue": { 2023-12-01T01:22:47.611 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops": 0, 2023-12-01T01:22:47.611 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops_high_water": 0, 2023-12-01T01:22:47.611 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing": 0, 2023-12-01T01:22:47.611 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_high_water": 0, 2023-12-01T01:22:47.611 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed": 0, 2023-12-01T01:22:47.611 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_item_in_journal": 0 2023-12-01T01:22:47.611 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.611 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-01T01:22:47.611 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2023-12-01T01:22:47.611 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2023-12-01T01:22:47.611 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2023-12-01T01:22:47.611 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 1243, 2023-12-01T01:22:47.611 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 11430686, 2023-12-01T01:22:47.611 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:47.611 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 1243, 2023-12-01T01:22:47.612 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2023-12-01T01:22:47.612 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2023-12-01T01:22:47.612 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 1243, 2023-12-01T01:22:47.612 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 11430686, 2023-12-01T01:22:47.612 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2023-12-01T01:22:47.612 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:47.612 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:47.612 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:47.612 INFO:teuthology.orchestra.run.smithi137.stdout: } 2023-12-01T01:22:47.612 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.612 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_bytes": { 2023-12-01T01:22:47.612 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2023-12-01T01:22:47.613 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2023-12-01T01:22:47.613 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2023-12-01T01:22:47.613 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2023-12-01T01:22:47.613 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2023-12-01T01:22:47.613 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:47.613 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2023-12-01T01:22:47.613 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 1104, 2023-12-01T01:22:47.613 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 10680905, 2023-12-01T01:22:47.613 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 1104, 2023-12-01T01:22:47.613 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 10680905, 2023-12-01T01:22:47.613 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2023-12-01T01:22:47.613 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:47.613 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:47.613 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:47.613 INFO:teuthology.orchestra.run.smithi137.stdout: } 2023-12-01T01:22:47.614 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.614 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_ops": { 2023-12-01T01:22:47.614 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2023-12-01T01:22:47.614 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 1024, 2023-12-01T01:22:47.614 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2023-12-01T01:22:47.614 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2023-12-01T01:22:47.614 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2023-12-01T01:22:47.614 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:47.614 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2023-12-01T01:22:47.614 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 1104, 2023-12-01T01:22:47.614 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 1104, 2023-12-01T01:22:47.614 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 1104, 2023-12-01T01:22:47.614 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 1104, 2023-12-01T01:22:47.614 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2023-12-01T01:22:47.615 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:47.615 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:47.615 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:47.615 INFO:teuthology.orchestra.run.smithi137.stdout: } 2023-12-01T01:22:47.615 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.615 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle": { 2023-12-01T01:22:47.615 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2023-12-01T01:22:47.615 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2023-12-01T01:22:47.615 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2023-12-01T01:22:47.615 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2023-12-01T01:22:47.615 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2023-12-01T01:22:47.615 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:47.615 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2023-12-01T01:22:47.615 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2023-12-01T01:22:47.616 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2023-12-01T01:22:47.616 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 0, 2023-12-01T01:22:47.616 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 0, 2023-12-01T01:22:47.616 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2023-12-01T01:22:47.616 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:47.616 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:47.616 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:47.616 INFO:teuthology.orchestra.run.smithi137.stdout: } 2023-12-01T01:22:47.616 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:47.616 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle-0x5629e203e500": { 2023-12-01T01:22:47.616 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2023-12-01T01:22:47.616 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2023-12-01T01:22:47.616 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2023-12-01T01:22:47.616 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2023-12-01T01:22:47.616 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2023-12-01T01:22:47.617 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:47.617 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2023-12-01T01:22:47.617 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2023-12-01T01:22:47.617 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2023-12-01T01:22:47.617 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 0, 2023-12-01T01:22:47.617 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 0, 2023-12-01T01:22:47.617 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2023-12-01T01:22:47.617 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:47.617 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:47.617 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:47.617 INFO:teuthology.orchestra.run.smithi137.stdout: } 2023-12-01T01:22:47.617 INFO:teuthology.orchestra.run.smithi137.stdout: } 2023-12-01T01:22:47.617 INFO:teuthology.orchestra.run.smithi137.stdout:} 2023-12-01T01:22:47.618 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:47 smithi137 ceph-mon[103312]: pgmap v445: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 12 op/s 2023-12-01T01:22:47.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:47 smithi203 ceph-mon[103581]: pgmap v445: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 12 op/s 2023-12-01T01:22:47.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:47 smithi077 ceph-mon[98867]: pgmap v445: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 12 op/s 2023-12-01T01:22:47.954 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2023-12-01T01:22:47.954 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2023-12-01T01:22:47.954 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2023-12-01T01:22:47.955 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2023-12-01T01:22:47.955 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2023-12-01T01:22:47.955 DEBUG:tasks.check_counter:Getting stats from h 2023-12-01T01:22:47.955 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph daemon mds.h perf dump 2023-12-01T01:22:48.502 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.b/config 2023-12-01T01:22:49.719 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:49 smithi137 ceph-mon[103312]: pgmap v446: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 13 op/s 2023-12-01T01:22:49.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:49 smithi203 ceph-mon[103581]: pgmap v446: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 13 op/s 2023-12-01T01:22:49.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:49 smithi077 ceph-mon[98867]: pgmap v446: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 13 op/s 2023-12-01T01:22:49.938 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2023-12-01T01:22:49.938 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2023-12-01T01:22:49.938 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 1970, 2023-12-01T01:22:49.938 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 1580, 2023-12-01T01:22:49.938 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 877437, 2023-12-01T01:22:49.939 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 4075218, 2023-12-01T01:22:49.939 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 16, 2023-12-01T01:22:49.939 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 4, 2023-12-01T01:22:49.939 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.103926640, 2023-12-01T01:22:49.939 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.037688412, 2023-12-01T01:22:49.939 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.056897229, 2023-12-01T01:22:49.939 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.015928859, 2023-12-01T01:22:49.939 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2023-12-01T01:22:49.939 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1580, 2023-12-01T01:22:49.939 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.078410366, 2023-12-01T01:22:49.939 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000049626 2023-12-01T01:22:49.939 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.939 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2023-12-01T01:22:49.939 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2170, 2023-12-01T01:22:49.940 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000287943, 2023-12-01T01:22:49.940 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000132 2023-12-01T01:22:49.940 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.940 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 0, 2023-12-01T01:22:49.940 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 0 2023-12-01T01:22:49.940 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.940 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2023-12-01T01:22:49.940 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 4126, 2023-12-01T01:22:49.940 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 7253, 2023-12-01T01:22:49.940 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 1919188, 2023-12-01T01:22:49.940 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 4997914, 2023-12-01T01:22:49.940 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 14, 2023-12-01T01:22:49.940 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 8, 2023-12-01T01:22:49.940 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.272520462, 2023-12-01T01:22:49.940 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.120324360, 2023-12-01T01:22:49.941 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.175597324, 2023-12-01T01:22:49.941 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.014912171, 2023-12-01T01:22:49.941 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2023-12-01T01:22:49.941 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 7249, 2023-12-01T01:22:49.941 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.172053883, 2023-12-01T01:22:49.941 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000023734 2023-12-01T01:22:49.941 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.941 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2023-12-01T01:22:49.941 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 8731, 2023-12-01T01:22:49.941 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.001285697, 2023-12-01T01:22:49.941 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000147 2023-12-01T01:22:49.941 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.941 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 1672816, 2023-12-01T01:22:49.941 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 3878240 2023-12-01T01:22:49.942 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.942 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2023-12-01T01:22:49.942 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 3574, 2023-12-01T01:22:49.942 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 3538, 2023-12-01T01:22:49.942 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 2229687, 2023-12-01T01:22:49.942 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 1554640, 2023-12-01T01:22:49.942 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 11, 2023-12-01T01:22:49.942 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 5, 2023-12-01T01:22:49.942 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.198124616, 2023-12-01T01:22:49.942 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.084277322, 2023-12-01T01:22:49.942 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.124791262, 2023-12-01T01:22:49.942 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.010704949, 2023-12-01T01:22:49.942 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2023-12-01T01:22:49.943 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 3538, 2023-12-01T01:22:49.943 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.118349884, 2023-12-01T01:22:49.943 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000033451 2023-12-01T01:22:49.943 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.943 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2023-12-01T01:22:49.943 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 6036, 2023-12-01T01:22:49.943 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000903366, 2023-12-01T01:22:49.943 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000149 2023-12-01T01:22:49.943 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.943 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 160, 2023-12-01T01:22:49.943 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 219920 2023-12-01T01:22:49.943 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.943 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2023-12-01T01:22:49.943 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 1, 2023-12-01T01:22:49.943 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2023-12-01T01:22:49.944 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.944 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-MDSRank": { 2023-12-01T01:22:49.944 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2023-12-01T01:22:49.944 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2023-12-01T01:22:49.944 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 541, 2023-12-01T01:22:49.944 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 1.008066813, 2023-12-01T01:22:49.944 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001863339 2023-12-01T01:22:49.944 INFO:teuthology.orchestra.run.smithi137.stdout: } 2023-12-01T01:22:49.944 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.944 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-PurgeQueue": { 2023-12-01T01:22:49.944 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2023-12-01T01:22:49.944 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2023-12-01T01:22:49.945 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1859, 2023-12-01T01:22:49.945 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.146418770, 2023-12-01T01:22:49.945 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000078762 2023-12-01T01:22:49.945 INFO:teuthology.orchestra.run.smithi137.stdout: } 2023-12-01T01:22:49.945 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.945 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2023-12-01T01:22:49.945 INFO:teuthology.orchestra.run.smithi137.stdout: "request": 2453, 2023-12-01T01:22:49.945 INFO:teuthology.orchestra.run.smithi137.stdout: "reply": 2333, 2023-12-01T01:22:49.945 INFO:teuthology.orchestra.run.smithi137.stdout: "reply_latency": { 2023-12-01T01:22:49.945 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2333, 2023-12-01T01:22:49.945 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 1.780972720, 2023-12-01T01:22:49.945 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000763383 2023-12-01T01:22:49.945 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.946 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_reply": 0, 2023-12-01T01:22:49.946 INFO:teuthology.orchestra.run.smithi137.stdout: "forward": 120, 2023-12-01T01:22:49.946 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_complete": 0, 2023-12-01T01:22:49.946 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_keys": 0, 2023-12-01T01:22:49.946 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_commit": 11, 2023-12-01T01:22:49.946 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_split": 0, 2023-12-01T01:22:49.946 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_merge": 0, 2023-12-01T01:22:49.946 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes": 19, 2023-12-01T01:22:49.946 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_top": 2, 2023-12-01T01:22:49.946 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_bottom": 7, 2023-12-01T01:22:49.946 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pin_tail": 10, 2023-12-01T01:22:49.946 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pinned": 13, 2023-12-01T01:22:49.946 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_expired": 170, 2023-12-01T01:22:49.946 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_with_caps": 1, 2023-12-01T01:22:49.947 INFO:teuthology.orchestra.run.smithi137.stdout: "caps": 1, 2023-12-01T01:22:49.947 INFO:teuthology.orchestra.run.smithi137.stdout: "subtrees": 5, 2023-12-01T01:22:49.947 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse": 4062, 2023-12-01T01:22:49.947 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_hit": 3700, 2023-12-01T01:22:49.947 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_forward": 120, 2023-12-01T01:22:49.947 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_discover": 110, 2023-12-01T01:22:49.947 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_dir_fetch": 0, 2023-12-01T01:22:49.947 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_remote_ino": 0, 2023-12-01T01:22:49.947 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_lock": 2, 2023-12-01T01:22:49.947 INFO:teuthology.orchestra.run.smithi137.stdout: "load_cent": 0, 2023-12-01T01:22:49.947 INFO:teuthology.orchestra.run.smithi137.stdout: "q": 0, 2023-12-01T01:22:49.947 INFO:teuthology.orchestra.run.smithi137.stdout: "exported": 47, 2023-12-01T01:22:49.947 INFO:teuthology.orchestra.run.smithi137.stdout: "exported_inodes": 25, 2023-12-01T01:22:49.947 INFO:teuthology.orchestra.run.smithi137.stdout: "imported": 47, 2023-12-01T01:22:49.948 INFO:teuthology.orchestra.run.smithi137.stdout: "imported_inodes": 1039, 2023-12-01T01:22:49.948 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_dir_fetch": 0, 2023-12-01T01:22:49.948 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_backtrace_fetch": 335, 2023-12-01T01:22:49.948 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_peer_discover": 0, 2023-12-01T01:22:49.948 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rfiles": 3568, 2023-12-01T01:22:49.948 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rbytes": 50854799, 2023-12-01T01:22:49.948 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rsnaps": 0, 2023-12-01T01:22:49.948 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_fetch": 335, 2023-12-01T01:22:49.948 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_set_tag": 0, 2023-12-01T01:22:49.948 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_repaired": 0, 2023-12-01T01:22:49.948 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_inotable_repaired": 0, 2023-12-01T01:22:49.948 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_inodes": 0, 2023-12-01T01:22:49.948 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_base_inodes": 0, 2023-12-01T01:22:49.948 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dirfrag_rstats": 0, 2023-12-01T01:22:49.948 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_file_inodes": 335, 2023-12-01T01:22:49.949 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_inode_file_caps": 0, 2023-12-01T01:22:49.949 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_revoke": 13, 2023-12-01T01:22:49.949 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_grant": 1149, 2023-12-01T01:22:49.949 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_trunc": 0, 2023-12-01T01:22:49.949 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-01T01:22:49.949 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-01T01:22:49.949 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps": 650, 2023-12-01T01:22:49.949 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps_dirty": 638, 2023-12-01T01:22:49.949 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_cap_release": 19, 2023-12-01T01:22:49.949 INFO:teuthology.orchestra.run.smithi137.stdout: "process_request_cap_release": 2062 2023-12-01T01:22:49.949 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.949 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_cache": { 2023-12-01T01:22:49.949 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update": 135, 2023-12-01T01:22:49.949 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update_receipt": 501, 2023-12-01T01:22:49.949 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_try_discover": 293, 2023-12-01T01:22:49.950 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_send_discover": 417, 2023-12-01T01:22:49.950 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_handle_discover": 161, 2023-12-01T01:22:49.950 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays": 0, 2023-12-01T01:22:49.950 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_delayed": 0, 2023-12-01T01:22:49.950 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_enqueuing": 0, 2023-12-01T01:22:49.950 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_created": 1009, 2023-12-01T01:22:49.950 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_enqueued": 1009, 2023-12-01T01:22:49.950 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_reintegrated": 0, 2023-12-01T01:22:49.950 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_migrated": 0, 2023-12-01T01:22:49.950 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_processing": 0, 2023-12-01T01:22:49.950 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_enqueued": 0, 2023-12-01T01:22:49.950 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_prioritized": 0, 2023-12-01T01:22:49.950 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_started": 0, 2023-12-01T01:22:49.950 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_completed": 0, 2023-12-01T01:22:49.951 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_enqueue_scrub": 0, 2023-12-01T01:22:49.951 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_exportdir": 47, 2023-12-01T01:22:49.951 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_flush": 0, 2023-12-01T01:22:49.951 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragmentdir": 0, 2023-12-01T01:22:49.951 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragstats": 0, 2023-12-01T01:22:49.951 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_inodestats": 0 2023-12-01T01:22:49.951 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.951 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_log": { 2023-12-01T01:22:49.951 INFO:teuthology.orchestra.run.smithi137.stdout: "evadd": 3477, 2023-12-01T01:22:49.951 INFO:teuthology.orchestra.run.smithi137.stdout: "evex": 0, 2023-12-01T01:22:49.951 INFO:teuthology.orchestra.run.smithi137.stdout: "evtrm": 0, 2023-12-01T01:22:49.951 INFO:teuthology.orchestra.run.smithi137.stdout: "ev": 3477, 2023-12-01T01:22:49.951 INFO:teuthology.orchestra.run.smithi137.stdout: "evexg": 0, 2023-12-01T01:22:49.951 INFO:teuthology.orchestra.run.smithi137.stdout: "evexd": 0, 2023-12-01T01:22:49.952 INFO:teuthology.orchestra.run.smithi137.stdout: "segadd": 5, 2023-12-01T01:22:49.952 INFO:teuthology.orchestra.run.smithi137.stdout: "segex": 0, 2023-12-01T01:22:49.952 INFO:teuthology.orchestra.run.smithi137.stdout: "segtrm": 0, 2023-12-01T01:22:49.952 INFO:teuthology.orchestra.run.smithi137.stdout: "seg": 5, 2023-12-01T01:22:49.952 INFO:teuthology.orchestra.run.smithi137.stdout: "segexg": 0, 2023-12-01T01:22:49.952 INFO:teuthology.orchestra.run.smithi137.stdout: "segexd": 0, 2023-12-01T01:22:49.952 INFO:teuthology.orchestra.run.smithi137.stdout: "expos": 4194304, 2023-12-01T01:22:49.952 INFO:teuthology.orchestra.run.smithi137.stdout: "wrpos": 9946485, 2023-12-01T01:22:49.952 INFO:teuthology.orchestra.run.smithi137.stdout: "rdpos": 0, 2023-12-01T01:22:49.952 INFO:teuthology.orchestra.run.smithi137.stdout: "jlat": { 2023-12-01T01:22:49.952 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 144, 2023-12-01T01:22:49.952 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.957856066, 2023-12-01T01:22:49.952 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.006651778 2023-12-01T01:22:49.952 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.953 INFO:teuthology.orchestra.run.smithi137.stdout: "replayed": 0 2023-12-01T01:22:49.953 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.953 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_mem": { 2023-12-01T01:22:49.953 INFO:teuthology.orchestra.run.smithi137.stdout: "ino": 25, 2023-12-01T01:22:49.953 INFO:teuthology.orchestra.run.smithi137.stdout: "ino+": 1114, 2023-12-01T01:22:49.953 INFO:teuthology.orchestra.run.smithi137.stdout: "ino-": 1089, 2023-12-01T01:22:49.953 INFO:teuthology.orchestra.run.smithi137.stdout: "dir": 24, 2023-12-01T01:22:49.953 INFO:teuthology.orchestra.run.smithi137.stdout: "dir+": 292, 2023-12-01T01:22:49.953 INFO:teuthology.orchestra.run.smithi137.stdout: "dir-": 268, 2023-12-01T01:22:49.953 INFO:teuthology.orchestra.run.smithi137.stdout: "dn": 19, 2023-12-01T01:22:49.953 INFO:teuthology.orchestra.run.smithi137.stdout: "dn+": 2213, 2023-12-01T01:22:49.953 INFO:teuthology.orchestra.run.smithi137.stdout: "dn-": 2194, 2023-12-01T01:22:49.953 INFO:teuthology.orchestra.run.smithi137.stdout: "cap": 1, 2023-12-01T01:22:49.953 INFO:teuthology.orchestra.run.smithi137.stdout: "cap+": 1822, 2023-12-01T01:22:49.953 INFO:teuthology.orchestra.run.smithi137.stdout: "cap-": 1821, 2023-12-01T01:22:49.954 INFO:teuthology.orchestra.run.smithi137.stdout: "rss": 64944, 2023-12-01T01:22:49.954 INFO:teuthology.orchestra.run.smithi137.stdout: "heap": 198964 2023-12-01T01:22:49.954 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.954 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_server": { 2023-12-01T01:22:49.954 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_client_request": 2830, 2023-12-01T01:22:49.954 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_server_request": 20, 2023-12-01T01:22:49.954 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_request": 2453, 2023-12-01T01:22:49.954 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_session": 4, 2023-12-01T01:22:49.954 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_peer_request": 66, 2023-12-01T01:22:49.954 INFO:teuthology.orchestra.run.smithi137.stdout: "req_create_latency": { 2023-12-01T01:22:49.954 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 133, 2023-12-01T01:22:49.954 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.121206822, 2023-12-01T01:22:49.954 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000911329 2023-12-01T01:22:49.954 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.955 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getattr_latency": { 2023-12-01T01:22:49.955 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:49.955 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:49.955 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:49.955 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.955 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getfilelock_latency": { 2023-12-01T01:22:49.955 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:49.955 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:49.955 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:49.955 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.955 INFO:teuthology.orchestra.run.smithi137.stdout: "req_link_latency": { 2023-12-01T01:22:49.955 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:49.955 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:49.955 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:49.955 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.956 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookup_latency": { 2023-12-01T01:22:49.956 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1143, 2023-12-01T01:22:49.956 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.384436472, 2023-12-01T01:22:49.956 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000336339 2023-12-01T01:22:49.956 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.956 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookuphash_latency": { 2023-12-01T01:22:49.956 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:49.956 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:49.956 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:49.956 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.956 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupino_latency": { 2023-12-01T01:22:49.956 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1, 2023-12-01T01:22:49.956 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000116799, 2023-12-01T01:22:49.956 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000116799 2023-12-01T01:22:49.956 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.957 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupname_latency": { 2023-12-01T01:22:49.957 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:49.957 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:49.957 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:49.957 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.957 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupparent_latency": { 2023-12-01T01:22:49.957 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:49.957 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:49.957 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:49.957 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.957 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupsnap_latency": { 2023-12-01T01:22:49.957 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:49.957 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:49.958 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:49.958 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.958 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lssnap_latency": { 2023-12-01T01:22:49.958 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:49.958 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:49.958 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:49.958 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.958 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mkdir_latency": { 2023-12-01T01:22:49.958 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:49.958 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:49.958 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:49.958 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.958 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mknod_latency": { 2023-12-01T01:22:49.958 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:49.959 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:49.959 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:49.959 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.959 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mksnap_latency": { 2023-12-01T01:22:49.959 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:49.959 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:49.959 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:49.959 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.959 INFO:teuthology.orchestra.run.smithi137.stdout: "req_open_latency": { 2023-12-01T01:22:49.959 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:49.959 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:49.959 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:49.959 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.959 INFO:teuthology.orchestra.run.smithi137.stdout: "req_readdir_latency": { 2023-12-01T01:22:49.959 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 47, 2023-12-01T01:22:49.960 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.029330960, 2023-12-01T01:22:49.960 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000624062 2023-12-01T01:22:49.960 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.960 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rename_latency": { 2023-12-01T01:22:49.960 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:49.960 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:49.960 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:49.960 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.960 INFO:teuthology.orchestra.run.smithi137.stdout: "req_renamesnap_latency": { 2023-12-01T01:22:49.960 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:49.960 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:49.960 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:49.960 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.960 INFO:teuthology.orchestra.run.smithi137.stdout: "req_snapdiff_latency": { 2023-12-01T01:22:49.960 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:49.961 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:49.961 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:49.961 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.961 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmdir_latency": { 2023-12-01T01:22:49.961 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:49.961 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:49.961 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:49.961 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.961 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmsnap_latency": { 2023-12-01T01:22:49.961 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:49.961 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:49.961 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:49.961 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.961 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmxattr_latency": { 2023-12-01T01:22:49.961 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:49.962 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:49.962 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:49.962 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.962 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setattr_latency": { 2023-12-01T01:22:49.962 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:49.962 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:49.962 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:49.962 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.962 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setdirlayout_latency": { 2023-12-01T01:22:49.962 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:49.962 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:49.962 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:49.962 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.962 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setfilelock_latency": { 2023-12-01T01:22:49.963 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:49.963 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:49.963 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:49.963 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.963 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setlayout_latency": { 2023-12-01T01:22:49.963 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:49.963 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:49.963 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:49.963 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.963 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setxattr_latency": { 2023-12-01T01:22:49.963 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:49.963 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:49.963 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:49.963 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.963 INFO:teuthology.orchestra.run.smithi137.stdout: "req_symlink_latency": { 2023-12-01T01:22:49.964 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:49.964 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:49.964 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:49.964 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.964 INFO:teuthology.orchestra.run.smithi137.stdout: "req_unlink_latency": { 2023-12-01T01:22:49.964 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1009, 2023-12-01T01:22:49.964 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 1.245881667, 2023-12-01T01:22:49.964 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001234768 2023-12-01T01:22:49.964 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.964 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_revoke_eviction": 0, 2023-12-01T01:22:49.964 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_acquisition_throttle": 0, 2023-12-01T01:22:49.964 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getvxattr_latency": { 2023-12-01T01:22:49.964 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:49.964 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:49.964 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:49.965 INFO:teuthology.orchestra.run.smithi137.stdout: } 2023-12-01T01:22:49.965 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.965 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_sessions": { 2023-12-01T01:22:49.965 INFO:teuthology.orchestra.run.smithi137.stdout: "session_count": 1, 2023-12-01T01:22:49.965 INFO:teuthology.orchestra.run.smithi137.stdout: "session_add": 1, 2023-12-01T01:22:49.965 INFO:teuthology.orchestra.run.smithi137.stdout: "session_remove": 0, 2023-12-01T01:22:49.965 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_open": 1, 2023-12-01T01:22:49.965 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_stale": 0, 2023-12-01T01:22:49.965 INFO:teuthology.orchestra.run.smithi137.stdout: "total_load": 1660, 2023-12-01T01:22:49.965 INFO:teuthology.orchestra.run.smithi137.stdout: "average_load": 1660, 2023-12-01T01:22:49.965 INFO:teuthology.orchestra.run.smithi137.stdout: "avg_session_uptime": 301, 2023-12-01T01:22:49.965 INFO:teuthology.orchestra.run.smithi137.stdout: "mdthresh_evicted": 0 2023-12-01T01:22:49.965 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.965 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2023-12-01T01:22:49.965 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2023-12-01T01:22:49.966 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2023-12-01T01:22:49.966 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2023-12-01T01:22:49.966 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2023-12-01T01:22:49.966 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2023-12-01T01:22:49.966 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2023-12-01T01:22:49.966 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-01T01:22:49.966 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2023-12-01T01:22:49.966 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-01T01:22:49.966 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2023-12-01T01:22:49.966 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2023-12-01T01:22:49.966 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2023-12-01T01:22:49.966 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-01T01:22:49.966 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2023-12-01T01:22:49.966 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2023-12-01T01:22:49.966 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2023-12-01T01:22:49.967 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2023-12-01T01:22:49.967 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2023-12-01T01:22:49.967 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-01T01:22:49.967 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2023-12-01T01:22:49.967 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-01T01:22:49.967 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2023-12-01T01:22:49.967 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2023-12-01T01:22:49.967 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2023-12-01T01:22:49.967 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2023-12-01T01:22:49.967 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2023-12-01T01:22:49.967 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-01T01:22:49.967 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2023-12-01T01:22:49.967 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2023-12-01T01:22:49.967 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2023-12-01T01:22:49.967 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2023-12-01T01:22:49.968 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2023-12-01T01:22:49.968 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2023-12-01T01:22:49.968 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2023-12-01T01:22:49.968 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2023-12-01T01:22:49.968 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2023-12-01T01:22:49.968 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 294407, 2023-12-01T01:22:49.968 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 98, 2023-12-01T01:22:49.968 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 0, 2023-12-01T01:22:49.968 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 0, 2023-12-01T01:22:49.968 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2023-12-01T01:22:49.968 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2023-12-01T01:22:49.968 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2023-12-01T01:22:49.968 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2023-12-01T01:22:49.968 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2023-12-01T01:22:49.968 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2023-12-01T01:22:49.969 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 7336, 2023-12-01T01:22:49.969 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 154, 2023-12-01T01:22:49.969 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2023-12-01T01:22:49.969 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2023-12-01T01:22:49.969 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2023-12-01T01:22:49.969 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2023-12-01T01:22:49.969 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 99629, 2023-12-01T01:22:49.969 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 416, 2023-12-01T01:22:49.969 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2023-12-01T01:22:49.969 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2023-12-01T01:22:49.969 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2023-12-01T01:22:49.969 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2023-12-01T01:22:49.969 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.969 INFO:teuthology.orchestra.run.smithi137.stdout: "objecter": { 2023-12-01T01:22:49.969 INFO:teuthology.orchestra.run.smithi137.stdout: "op_active": 0, 2023-12-01T01:22:49.970 INFO:teuthology.orchestra.run.smithi137.stdout: "op_laggy": 0, 2023-12-01T01:22:49.970 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send": 1589, 2023-12-01T01:22:49.970 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send_bytes": 5935590, 2023-12-01T01:22:49.970 INFO:teuthology.orchestra.run.smithi137.stdout: "op_resend": 27, 2023-12-01T01:22:49.970 INFO:teuthology.orchestra.run.smithi137.stdout: "op_reply": 1589, 2023-12-01T01:22:49.970 INFO:teuthology.orchestra.run.smithi137.stdout: "op_latency": { 2023-12-01T01:22:49.970 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1589, 2023-12-01T01:22:49.970 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 5.028910280, 2023-12-01T01:22:49.970 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.003164827 2023-12-01T01:22:49.970 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.970 INFO:teuthology.orchestra.run.smithi137.stdout: "op_inflight": 0, 2023-12-01T01:22:49.970 INFO:teuthology.orchestra.run.smithi137.stdout: "oplen_avg": { 2023-12-01T01:22:49.970 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1589, 2023-12-01T01:22:49.970 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 1628 2023-12-01T01:22:49.970 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.971 INFO:teuthology.orchestra.run.smithi137.stdout: "op": 1589, 2023-12-01T01:22:49.971 INFO:teuthology.orchestra.run.smithi137.stdout: "op_r": 351, 2023-12-01T01:22:49.971 INFO:teuthology.orchestra.run.smithi137.stdout: "op_w": 1238, 2023-12-01T01:22:49.971 INFO:teuthology.orchestra.run.smithi137.stdout: "op_rmw": 0, 2023-12-01T01:22:49.971 INFO:teuthology.orchestra.run.smithi137.stdout: "op_pg": 0, 2023-12-01T01:22:49.971 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_stat": 0, 2023-12-01T01:22:49.971 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_create": 10, 2023-12-01T01:22:49.971 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_read": 16, 2023-12-01T01:22:49.971 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_write": 160, 2023-12-01T01:22:49.971 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writefull": 21, 2023-12-01T01:22:49.971 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writesame": 0, 2023-12-01T01:22:49.971 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_append": 0, 2023-12-01T01:22:49.971 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_zero": 0, 2023-12-01T01:22:49.971 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_truncate": 0, 2023-12-01T01:22:49.972 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_delete": 1022, 2023-12-01T01:22:49.972 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_mapext": 0, 2023-12-01T01:22:49.972 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_sparse_read": 0, 2023-12-01T01:22:49.972 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_clonerange": 0, 2023-12-01T01:22:49.972 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_getxattr": 335, 2023-12-01T01:22:49.972 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_setxattr": 20, 2023-12-01T01:22:49.972 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_cmpxattr": 0, 2023-12-01T01:22:49.972 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_rmxattr": 0, 2023-12-01T01:22:49.972 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_resetxattrs": 0, 2023-12-01T01:22:49.972 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_call": 0, 2023-12-01T01:22:49.972 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_watch": 0, 2023-12-01T01:22:49.972 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_notify": 0, 2023-12-01T01:22:49.972 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_src_cmpxattr": 0, 2023-12-01T01:22:49.972 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls": 0, 2023-12-01T01:22:49.972 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls_filter": 0, 2023-12-01T01:22:49.973 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_other": 0, 2023-12-01T01:22:49.973 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_active": 0, 2023-12-01T01:22:49.973 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_send": 0, 2023-12-01T01:22:49.973 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_resend": 0, 2023-12-01T01:22:49.973 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_ping": 0, 2023-12-01T01:22:49.973 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_active": 0, 2023-12-01T01:22:49.973 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_send": 0, 2023-12-01T01:22:49.973 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_resend": 0, 2023-12-01T01:22:49.973 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_active": 0, 2023-12-01T01:22:49.973 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_send": 0, 2023-12-01T01:22:49.973 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_resend": 0, 2023-12-01T01:22:49.973 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_active": 0, 2023-12-01T01:22:49.973 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_send": 0, 2023-12-01T01:22:49.973 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_resend": 0, 2023-12-01T01:22:49.973 INFO:teuthology.orchestra.run.smithi137.stdout: "command_active": 0, 2023-12-01T01:22:49.974 INFO:teuthology.orchestra.run.smithi137.stdout: "command_send": 0, 2023-12-01T01:22:49.974 INFO:teuthology.orchestra.run.smithi137.stdout: "command_resend": 0, 2023-12-01T01:22:49.974 INFO:teuthology.orchestra.run.smithi137.stdout: "map_epoch": 84, 2023-12-01T01:22:49.974 INFO:teuthology.orchestra.run.smithi137.stdout: "map_full": 0, 2023-12-01T01:22:49.974 INFO:teuthology.orchestra.run.smithi137.stdout: "map_inc": 2, 2023-12-01T01:22:49.974 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_sessions": 12, 2023-12-01T01:22:49.974 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_open": 12, 2023-12-01T01:22:49.974 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_close": 0, 2023-12-01T01:22:49.974 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_laggy": 0, 2023-12-01T01:22:49.974 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_wr": 35, 2023-12-01T01:22:49.974 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_rd": 0, 2023-12-01T01:22:49.975 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_del": 9 2023-12-01T01:22:49.975 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.975 INFO:teuthology.orchestra.run.smithi137.stdout: "oft": { 2023-12-01T01:22:49.975 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_objs": 1, 2023-12-01T01:22:49.975 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_kv_pairs": 0, 2023-12-01T01:22:49.975 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_updates": 0, 2023-12-01T01:22:49.975 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_removes": 0 2023-12-01T01:22:49.975 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.975 INFO:teuthology.orchestra.run.smithi137.stdout: "osd-slow-ops": { 2023-12-01T01:22:49.975 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_ops_count": 2520 2023-12-01T01:22:49.975 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.975 INFO:teuthology.orchestra.run.smithi137.stdout: "purge_queue": { 2023-12-01T01:22:49.975 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops": 0, 2023-12-01T01:22:49.975 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops_high_water": 6, 2023-12-01T01:22:49.975 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing": 0, 2023-12-01T01:22:49.976 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_high_water": 6, 2023-12-01T01:22:49.976 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed": 1009, 2023-12-01T01:22:49.976 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_item_in_journal": 0 2023-12-01T01:22:49.976 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.976 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-01T01:22:49.976 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2023-12-01T01:22:49.976 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2023-12-01T01:22:49.976 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2023-12-01T01:22:49.976 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 9666, 2023-12-01T01:22:49.976 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 4005325, 2023-12-01T01:22:49.976 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:49.976 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 9666, 2023-12-01T01:22:49.976 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2023-12-01T01:22:49.976 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2023-12-01T01:22:49.976 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 9666, 2023-12-01T01:22:49.977 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 4005325, 2023-12-01T01:22:49.977 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2023-12-01T01:22:49.977 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:49.977 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:49.977 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:49.977 INFO:teuthology.orchestra.run.smithi137.stdout: } 2023-12-01T01:22:49.977 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.977 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_bytes": { 2023-12-01T01:22:49.977 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2023-12-01T01:22:49.977 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2023-12-01T01:22:49.977 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2023-12-01T01:22:49.977 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2023-12-01T01:22:49.977 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2023-12-01T01:22:49.977 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:49.978 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2023-12-01T01:22:49.978 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 1589, 2023-12-01T01:22:49.978 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 6037499, 2023-12-01T01:22:49.978 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 1589, 2023-12-01T01:22:49.978 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 6037499, 2023-12-01T01:22:49.978 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2023-12-01T01:22:49.978 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:49.978 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:49.978 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:49.978 INFO:teuthology.orchestra.run.smithi137.stdout: } 2023-12-01T01:22:49.978 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.978 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_ops": { 2023-12-01T01:22:49.978 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2023-12-01T01:22:49.978 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 1024, 2023-12-01T01:22:49.979 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2023-12-01T01:22:49.979 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2023-12-01T01:22:49.979 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2023-12-01T01:22:49.979 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:49.979 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2023-12-01T01:22:49.979 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 1589, 2023-12-01T01:22:49.979 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 1589, 2023-12-01T01:22:49.979 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 1589, 2023-12-01T01:22:49.979 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 1589, 2023-12-01T01:22:49.979 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2023-12-01T01:22:49.979 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:49.979 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:49.979 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:49.979 INFO:teuthology.orchestra.run.smithi137.stdout: } 2023-12-01T01:22:49.980 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.980 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle": { 2023-12-01T01:22:49.980 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2023-12-01T01:22:49.980 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2023-12-01T01:22:49.980 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2023-12-01T01:22:49.980 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 1009, 2023-12-01T01:22:49.980 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 101909, 2023-12-01T01:22:49.980 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:49.980 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 1009, 2023-12-01T01:22:49.980 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2023-12-01T01:22:49.980 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2023-12-01T01:22:49.980 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 16, 2023-12-01T01:22:49.980 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 101909, 2023-12-01T01:22:49.980 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2023-12-01T01:22:49.980 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:49.981 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:49.981 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:49.981 INFO:teuthology.orchestra.run.smithi137.stdout: } 2023-12-01T01:22:49.981 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:49.981 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle-0x55b4ff3b60a0": { 2023-12-01T01:22:49.981 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2023-12-01T01:22:49.981 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2023-12-01T01:22:49.981 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2023-12-01T01:22:49.981 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 3477, 2023-12-01T01:22:49.981 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 5752181, 2023-12-01T01:22:49.981 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:49.981 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 3477, 2023-12-01T01:22:49.981 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2023-12-01T01:22:49.981 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2023-12-01T01:22:49.982 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 144, 2023-12-01T01:22:49.982 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 5752181, 2023-12-01T01:22:49.982 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2023-12-01T01:22:49.982 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:49.982 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:49.982 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:49.982 INFO:teuthology.orchestra.run.smithi137.stdout: } 2023-12-01T01:22:49.982 INFO:teuthology.orchestra.run.smithi137.stdout: } 2023-12-01T01:22:49.982 INFO:teuthology.orchestra.run.smithi137.stdout:} 2023-12-01T01:22:50.356 INFO:tasks.check_counter:Daemon mds.h mds.exported=47 2023-12-01T01:22:50.356 INFO:tasks.check_counter:Daemon mds.h mds.imported=47 2023-12-01T01:22:50.356 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=135 2023-12-01T01:22:50.356 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=501 2023-12-01T01:22:50.356 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2023-12-01T01:22:50.356 DEBUG:tasks.check_counter:Getting stats from k 2023-12-01T01:22:50.356 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph daemon mds.k perf dump 2023-12-01T01:22:50.909 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.b/config 2023-12-01T01:22:51.682 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:51 smithi137 ceph-mon[103312]: pgmap v447: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 12 op/s 2023-12-01T01:22:51.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:51 smithi203 ceph-mon[103581]: pgmap v447: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 12 op/s 2023-12-01T01:22:51.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:51 smithi077 ceph-mon[98867]: pgmap v447: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 12 op/s 2023-12-01T01:22:52.298 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2023-12-01T01:22:52.298 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2023-12-01T01:22:52.298 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 747, 2023-12-01T01:22:52.298 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 747, 2023-12-01T01:22:52.299 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 4419813, 2023-12-01T01:22:52.299 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 231522, 2023-12-01T01:22:52.299 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 6, 2023-12-01T01:22:52.299 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 2, 2023-12-01T01:22:52.299 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.062225554, 2023-12-01T01:22:52.299 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.019950568, 2023-12-01T01:22:52.299 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.016874798, 2023-12-01T01:22:52.299 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.019163181, 2023-12-01T01:22:52.299 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2023-12-01T01:22:52.299 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 746, 2023-12-01T01:22:52.299 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.025933544, 2023-12-01T01:22:52.299 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000034763 2023-12-01T01:22:52.300 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.300 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2023-12-01T01:22:52.300 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:52.300 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:52.300 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:52.300 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.300 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-12-01T01:22:52.300 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 448 2023-12-01T01:22:52.300 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.300 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2023-12-01T01:22:52.300 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 316, 2023-12-01T01:22:52.300 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 401, 2023-12-01T01:22:52.300 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 1636979, 2023-12-01T01:22:52.300 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 317201, 2023-12-01T01:22:52.300 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 5, 2023-12-01T01:22:52.301 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 4, 2023-12-01T01:22:52.301 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.028969344, 2023-12-01T01:22:52.301 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.008466632, 2023-12-01T01:22:52.301 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.007687826, 2023-12-01T01:22:52.301 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.010013589, 2023-12-01T01:22:52.301 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2023-12-01T01:22:52.301 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 399, 2023-12-01T01:22:52.301 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.020955521, 2023-12-01T01:22:52.301 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000052520 2023-12-01T01:22:52.301 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.301 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2023-12-01T01:22:52.301 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:52.302 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:52.302 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:52.302 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.302 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 160, 2023-12-01T01:22:52.302 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 219168 2023-12-01T01:22:52.302 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.302 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2023-12-01T01:22:52.302 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 178, 2023-12-01T01:22:52.302 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 158, 2023-12-01T01:22:52.302 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 566084, 2023-12-01T01:22:52.303 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 76835, 2023-12-01T01:22:52.303 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 4, 2023-12-01T01:22:52.303 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 3, 2023-12-01T01:22:52.303 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.023555254, 2023-12-01T01:22:52.303 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.006893645, 2023-12-01T01:22:52.303 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.014341530, 2023-12-01T01:22:52.303 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.004404636, 2023-12-01T01:22:52.303 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2023-12-01T01:22:52.303 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 157, 2023-12-01T01:22:52.303 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.009560466, 2023-12-01T01:22:52.303 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000060894 2023-12-01T01:22:52.303 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.303 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2023-12-01T01:22:52.303 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:52.304 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:52.304 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:52.304 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.304 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 556960, 2023-12-01T01:22:52.304 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 65328 2023-12-01T01:22:52.304 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.304 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2023-12-01T01:22:52.304 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 1, 2023-12-01T01:22:52.304 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2023-12-01T01:22:52.304 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.305 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-MDSRank": { 2023-12-01T01:22:52.305 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2023-12-01T01:22:52.305 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2023-12-01T01:22:52.305 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1840, 2023-12-01T01:22:52.305 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.047072464, 2023-12-01T01:22:52.305 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000025582 2023-12-01T01:22:52.305 INFO:teuthology.orchestra.run.smithi137.stdout: } 2023-12-01T01:22:52.305 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.305 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-PurgeQueue": { 2023-12-01T01:22:52.305 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2023-12-01T01:22:52.305 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2023-12-01T01:22:52.305 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:52.306 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:52.306 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:52.306 INFO:teuthology.orchestra.run.smithi137.stdout: } 2023-12-01T01:22:52.306 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.306 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2023-12-01T01:22:52.306 INFO:teuthology.orchestra.run.smithi137.stdout: "request": 0, 2023-12-01T01:22:52.306 INFO:teuthology.orchestra.run.smithi137.stdout: "reply": 0, 2023-12-01T01:22:52.306 INFO:teuthology.orchestra.run.smithi137.stdout: "reply_latency": { 2023-12-01T01:22:52.307 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:52.307 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:52.307 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:52.307 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.307 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_reply": 0, 2023-12-01T01:22:52.307 INFO:teuthology.orchestra.run.smithi137.stdout: "forward": 0, 2023-12-01T01:22:52.307 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_complete": 0, 2023-12-01T01:22:52.308 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_keys": 0, 2023-12-01T01:22:52.308 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_commit": 0, 2023-12-01T01:22:52.308 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_split": 0, 2023-12-01T01:22:52.308 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_merge": 0, 2023-12-01T01:22:52.308 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes": 1013, 2023-12-01T01:22:52.308 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_top": 0, 2023-12-01T01:22:52.308 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_bottom": 0, 2023-12-01T01:22:52.308 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pin_tail": 1013, 2023-12-01T01:22:52.308 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pinned": 1013, 2023-12-01T01:22:52.308 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_expired": 0, 2023-12-01T01:22:52.308 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_with_caps": 0, 2023-12-01T01:22:52.308 INFO:teuthology.orchestra.run.smithi137.stdout: "caps": 0, 2023-12-01T01:22:52.308 INFO:teuthology.orchestra.run.smithi137.stdout: "subtrees": 1, 2023-12-01T01:22:52.308 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse": 0, 2023-12-01T01:22:52.309 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_hit": 0, 2023-12-01T01:22:52.309 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_forward": 0, 2023-12-01T01:22:52.309 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_discover": 0, 2023-12-01T01:22:52.309 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_dir_fetch": 0, 2023-12-01T01:22:52.309 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_remote_ino": 0, 2023-12-01T01:22:52.309 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_lock": 0, 2023-12-01T01:22:52.309 INFO:teuthology.orchestra.run.smithi137.stdout: "load_cent": 0, 2023-12-01T01:22:52.309 INFO:teuthology.orchestra.run.smithi137.stdout: "q": 0, 2023-12-01T01:22:52.309 INFO:teuthology.orchestra.run.smithi137.stdout: "exported": 0, 2023-12-01T01:22:52.309 INFO:teuthology.orchestra.run.smithi137.stdout: "exported_inodes": 0, 2023-12-01T01:22:52.310 INFO:teuthology.orchestra.run.smithi137.stdout: "imported": 0, 2023-12-01T01:22:52.310 INFO:teuthology.orchestra.run.smithi137.stdout: "imported_inodes": 0, 2023-12-01T01:22:52.310 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_dir_fetch": 0, 2023-12-01T01:22:52.310 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_backtrace_fetch": 0, 2023-12-01T01:22:52.310 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_peer_discover": 0, 2023-12-01T01:22:52.310 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rfiles": 0, 2023-12-01T01:22:52.310 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rbytes": 0, 2023-12-01T01:22:52.310 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rsnaps": 0, 2023-12-01T01:22:52.310 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_fetch": 0, 2023-12-01T01:22:52.310 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_set_tag": 0, 2023-12-01T01:22:52.310 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_repaired": 0, 2023-12-01T01:22:52.310 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_inotable_repaired": 0, 2023-12-01T01:22:52.311 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_inodes": 0, 2023-12-01T01:22:52.311 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_base_inodes": 0, 2023-12-01T01:22:52.311 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dirfrag_rstats": 0, 2023-12-01T01:22:52.311 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_file_inodes": 0, 2023-12-01T01:22:52.311 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_inode_file_caps": 0, 2023-12-01T01:22:52.311 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_revoke": 0, 2023-12-01T01:22:52.311 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_grant": 0, 2023-12-01T01:22:52.311 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_trunc": 0, 2023-12-01T01:22:52.311 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-01T01:22:52.311 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-01T01:22:52.311 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps": 0, 2023-12-01T01:22:52.311 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps_dirty": 0, 2023-12-01T01:22:52.312 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_cap_release": 0, 2023-12-01T01:22:52.312 INFO:teuthology.orchestra.run.smithi137.stdout: "process_request_cap_release": 0 2023-12-01T01:22:52.312 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.312 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_cache": { 2023-12-01T01:22:52.312 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update": 0, 2023-12-01T01:22:52.312 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update_receipt": 0, 2023-12-01T01:22:52.312 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_try_discover": 0, 2023-12-01T01:22:52.312 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_send_discover": 0, 2023-12-01T01:22:52.312 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_handle_discover": 0, 2023-12-01T01:22:52.312 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays": 0, 2023-12-01T01:22:52.312 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_delayed": 0, 2023-12-01T01:22:52.312 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_enqueuing": 0, 2023-12-01T01:22:52.312 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_created": 0, 2023-12-01T01:22:52.312 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_enqueued": 0, 2023-12-01T01:22:52.313 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_reintegrated": 0, 2023-12-01T01:22:52.313 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_migrated": 0, 2023-12-01T01:22:52.313 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_processing": 0, 2023-12-01T01:22:52.313 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_enqueued": 0, 2023-12-01T01:22:52.313 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_prioritized": 0, 2023-12-01T01:22:52.313 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_started": 0, 2023-12-01T01:22:52.313 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_completed": 0, 2023-12-01T01:22:52.313 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_enqueue_scrub": 0, 2023-12-01T01:22:52.313 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_exportdir": 0, 2023-12-01T01:22:52.313 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_flush": 0, 2023-12-01T01:22:52.313 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragmentdir": 0, 2023-12-01T01:22:52.313 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragstats": 0, 2023-12-01T01:22:52.314 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_inodestats": 0 2023-12-01T01:22:52.314 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.314 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_log": { 2023-12-01T01:22:52.314 INFO:teuthology.orchestra.run.smithi137.stdout: "evadd": 0, 2023-12-01T01:22:52.314 INFO:teuthology.orchestra.run.smithi137.stdout: "evex": 0, 2023-12-01T01:22:52.314 INFO:teuthology.orchestra.run.smithi137.stdout: "evtrm": 0, 2023-12-01T01:22:52.314 INFO:teuthology.orchestra.run.smithi137.stdout: "ev": 3477, 2023-12-01T01:22:52.314 INFO:teuthology.orchestra.run.smithi137.stdout: "evexg": 0, 2023-12-01T01:22:52.314 INFO:teuthology.orchestra.run.smithi137.stdout: "evexd": 0, 2023-12-01T01:22:52.315 INFO:teuthology.orchestra.run.smithi137.stdout: "segadd": 0, 2023-12-01T01:22:52.315 INFO:teuthology.orchestra.run.smithi137.stdout: "segex": 0, 2023-12-01T01:22:52.315 INFO:teuthology.orchestra.run.smithi137.stdout: "segtrm": 0, 2023-12-01T01:22:52.315 INFO:teuthology.orchestra.run.smithi137.stdout: "seg": 5, 2023-12-01T01:22:52.315 INFO:teuthology.orchestra.run.smithi137.stdout: "segexg": 0, 2023-12-01T01:22:52.315 INFO:teuthology.orchestra.run.smithi137.stdout: "segexd": 0, 2023-12-01T01:22:52.315 INFO:teuthology.orchestra.run.smithi137.stdout: "expos": 4194304, 2023-12-01T01:22:52.315 INFO:teuthology.orchestra.run.smithi137.stdout: "wrpos": 9946485, 2023-12-01T01:22:52.315 INFO:teuthology.orchestra.run.smithi137.stdout: "rdpos": 9945347, 2023-12-01T01:22:52.315 INFO:teuthology.orchestra.run.smithi137.stdout: "jlat": { 2023-12-01T01:22:52.315 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:52.315 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:52.315 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:52.315 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.316 INFO:teuthology.orchestra.run.smithi137.stdout: "replayed": 3477 2023-12-01T01:22:52.316 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.316 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_mem": { 2023-12-01T01:22:52.316 INFO:teuthology.orchestra.run.smithi137.stdout: "ino": 7, 2023-12-01T01:22:52.316 INFO:teuthology.orchestra.run.smithi137.stdout: "ino+": 1031, 2023-12-01T01:22:52.316 INFO:teuthology.orchestra.run.smithi137.stdout: "ino-": 1024, 2023-12-01T01:22:52.316 INFO:teuthology.orchestra.run.smithi137.stdout: "dir": 5, 2023-12-01T01:22:52.316 INFO:teuthology.orchestra.run.smithi137.stdout: "dir+": 64, 2023-12-01T01:22:52.316 INFO:teuthology.orchestra.run.smithi137.stdout: "dir-": 59, 2023-12-01T01:22:52.316 INFO:teuthology.orchestra.run.smithi137.stdout: "dn": 1013, 2023-12-01T01:22:52.316 INFO:teuthology.orchestra.run.smithi137.stdout: "dn+": 2070, 2023-12-01T01:22:52.317 INFO:teuthology.orchestra.run.smithi137.stdout: "dn-": 1057, 2023-12-01T01:22:52.317 INFO:teuthology.orchestra.run.smithi137.stdout: "cap": 0, 2023-12-01T01:22:52.317 INFO:teuthology.orchestra.run.smithi137.stdout: "cap+": 0, 2023-12-01T01:22:52.317 INFO:teuthology.orchestra.run.smithi137.stdout: "cap-": 0, 2023-12-01T01:22:52.317 INFO:teuthology.orchestra.run.smithi137.stdout: "rss": 50732, 2023-12-01T01:22:52.317 INFO:teuthology.orchestra.run.smithi137.stdout: "heap": 207156 2023-12-01T01:22:52.317 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.317 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_server": { 2023-12-01T01:22:52.317 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_client_request": 0, 2023-12-01T01:22:52.317 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_server_request": 0, 2023-12-01T01:22:52.317 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_request": 0, 2023-12-01T01:22:52.317 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_session": 0, 2023-12-01T01:22:52.317 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_peer_request": 0, 2023-12-01T01:22:52.317 INFO:teuthology.orchestra.run.smithi137.stdout: "req_create_latency": { 2023-12-01T01:22:52.318 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:52.318 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:52.318 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:52.318 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.318 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getattr_latency": { 2023-12-01T01:22:52.318 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:52.318 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:52.318 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:52.318 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.318 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getfilelock_latency": { 2023-12-01T01:22:52.318 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:52.318 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:52.318 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:52.318 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.319 INFO:teuthology.orchestra.run.smithi137.stdout: "req_link_latency": { 2023-12-01T01:22:52.319 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:52.319 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:52.319 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:52.319 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.319 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookup_latency": { 2023-12-01T01:22:52.319 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:52.319 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:52.319 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:52.319 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.319 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookuphash_latency": { 2023-12-01T01:22:52.319 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:52.319 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:52.319 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:52.320 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.320 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupino_latency": { 2023-12-01T01:22:52.320 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:52.320 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:52.320 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:52.320 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.320 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupname_latency": { 2023-12-01T01:22:52.320 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:52.320 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:52.320 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:52.320 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.320 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupparent_latency": { 2023-12-01T01:22:52.320 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:52.320 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:52.321 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:52.321 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.321 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupsnap_latency": { 2023-12-01T01:22:52.321 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:52.321 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:52.321 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:52.321 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.321 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lssnap_latency": { 2023-12-01T01:22:52.321 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:52.321 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:52.321 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:52.321 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.321 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mkdir_latency": { 2023-12-01T01:22:52.321 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:52.322 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:52.322 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:52.322 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.322 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mknod_latency": { 2023-12-01T01:22:52.322 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:52.322 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:52.322 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:52.322 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.322 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mksnap_latency": { 2023-12-01T01:22:52.322 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:52.322 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:52.322 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:52.322 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.322 INFO:teuthology.orchestra.run.smithi137.stdout: "req_open_latency": { 2023-12-01T01:22:52.323 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:52.323 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:52.323 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:52.323 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.323 INFO:teuthology.orchestra.run.smithi137.stdout: "req_readdir_latency": { 2023-12-01T01:22:52.323 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:52.323 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:52.323 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:52.323 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.323 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rename_latency": { 2023-12-01T01:22:52.323 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:52.323 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:52.323 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:52.323 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.324 INFO:teuthology.orchestra.run.smithi137.stdout: "req_renamesnap_latency": { 2023-12-01T01:22:52.324 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:52.324 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:52.324 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:52.324 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.324 INFO:teuthology.orchestra.run.smithi137.stdout: "req_snapdiff_latency": { 2023-12-01T01:22:52.324 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:52.324 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:52.324 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:52.324 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.324 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmdir_latency": { 2023-12-01T01:22:52.324 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:52.324 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:52.324 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:52.324 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.325 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmsnap_latency": { 2023-12-01T01:22:52.325 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:52.325 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:52.325 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:52.325 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.325 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmxattr_latency": { 2023-12-01T01:22:52.325 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:52.325 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:52.325 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:52.325 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.325 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setattr_latency": { 2023-12-01T01:22:52.325 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:52.325 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:52.325 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:52.326 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.326 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setdirlayout_latency": { 2023-12-01T01:22:52.326 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:52.326 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:52.326 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:52.326 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.326 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setfilelock_latency": { 2023-12-01T01:22:52.326 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:52.326 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:52.326 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:52.327 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.327 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setlayout_latency": { 2023-12-01T01:22:52.327 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:52.327 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:52.327 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:52.327 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.327 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setxattr_latency": { 2023-12-01T01:22:52.327 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:52.327 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:52.327 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:52.327 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.327 INFO:teuthology.orchestra.run.smithi137.stdout: "req_symlink_latency": { 2023-12-01T01:22:52.327 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:52.327 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:52.328 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:52.328 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.328 INFO:teuthology.orchestra.run.smithi137.stdout: "req_unlink_latency": { 2023-12-01T01:22:52.328 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:52.328 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:52.328 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:52.328 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.328 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_revoke_eviction": 0, 2023-12-01T01:22:52.328 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_acquisition_throttle": 0, 2023-12-01T01:22:52.328 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getvxattr_latency": { 2023-12-01T01:22:52.328 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:52.328 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:52.328 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:52.328 INFO:teuthology.orchestra.run.smithi137.stdout: } 2023-12-01T01:22:52.328 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.329 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_sessions": { 2023-12-01T01:22:52.329 INFO:teuthology.orchestra.run.smithi137.stdout: "session_count": 1, 2023-12-01T01:22:52.329 INFO:teuthology.orchestra.run.smithi137.stdout: "session_add": 1, 2023-12-01T01:22:52.329 INFO:teuthology.orchestra.run.smithi137.stdout: "session_remove": 0, 2023-12-01T01:22:52.329 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_open": 1, 2023-12-01T01:22:52.329 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_stale": 0, 2023-12-01T01:22:52.329 INFO:teuthology.orchestra.run.smithi137.stdout: "total_load": 0, 2023-12-01T01:22:52.329 INFO:teuthology.orchestra.run.smithi137.stdout: "average_load": 0, 2023-12-01T01:22:52.329 INFO:teuthology.orchestra.run.smithi137.stdout: "avg_session_uptime": 1667, 2023-12-01T01:22:52.329 INFO:teuthology.orchestra.run.smithi137.stdout: "mdthresh_evicted": 0 2023-12-01T01:22:52.329 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.329 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2023-12-01T01:22:52.329 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2023-12-01T01:22:52.329 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2023-12-01T01:22:52.329 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2023-12-01T01:22:52.330 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2023-12-01T01:22:52.330 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2023-12-01T01:22:52.330 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2023-12-01T01:22:52.330 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-01T01:22:52.330 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2023-12-01T01:22:52.330 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-01T01:22:52.330 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2023-12-01T01:22:52.330 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2023-12-01T01:22:52.330 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2023-12-01T01:22:52.330 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-01T01:22:52.330 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2023-12-01T01:22:52.330 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2023-12-01T01:22:52.330 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2023-12-01T01:22:52.331 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2023-12-01T01:22:52.331 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2023-12-01T01:22:52.331 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-01T01:22:52.332 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2023-12-01T01:22:52.332 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-01T01:22:52.332 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2023-12-01T01:22:52.332 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2023-12-01T01:22:52.332 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2023-12-01T01:22:52.332 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2023-12-01T01:22:52.332 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2023-12-01T01:22:52.332 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-01T01:22:52.333 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2023-12-01T01:22:52.333 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2023-12-01T01:22:52.333 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2023-12-01T01:22:52.333 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2023-12-01T01:22:52.333 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2023-12-01T01:22:52.333 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2023-12-01T01:22:52.333 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2023-12-01T01:22:52.333 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2023-12-01T01:22:52.333 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2023-12-01T01:22:52.333 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 197319, 2023-12-01T01:22:52.333 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 52, 2023-12-01T01:22:52.333 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 0, 2023-12-01T01:22:52.334 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 0, 2023-12-01T01:22:52.334 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2023-12-01T01:22:52.334 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2023-12-01T01:22:52.334 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2023-12-01T01:22:52.334 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2023-12-01T01:22:52.334 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2023-12-01T01:22:52.334 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2023-12-01T01:22:52.334 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 7336, 2023-12-01T01:22:52.334 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 154, 2023-12-01T01:22:52.334 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2023-12-01T01:22:52.334 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2023-12-01T01:22:52.334 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2023-12-01T01:22:52.335 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2023-12-01T01:22:52.335 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 727168, 2023-12-01T01:22:52.335 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 4114, 2023-12-01T01:22:52.335 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2023-12-01T01:22:52.335 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2023-12-01T01:22:52.335 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2023-12-01T01:22:52.335 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2023-12-01T01:22:52.335 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.335 INFO:teuthology.orchestra.run.smithi137.stdout: "objecter": { 2023-12-01T01:22:52.335 INFO:teuthology.orchestra.run.smithi137.stdout: "op_active": 0, 2023-12-01T01:22:52.335 INFO:teuthology.orchestra.run.smithi137.stdout: "op_laggy": 0, 2023-12-01T01:22:52.336 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send": 1095, 2023-12-01T01:22:52.338 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send_bytes": 21, 2023-12-01T01:22:52.338 INFO:teuthology.orchestra.run.smithi137.stdout: "op_resend": 3, 2023-12-01T01:22:52.338 INFO:teuthology.orchestra.run.smithi137.stdout: "op_reply": 1095, 2023-12-01T01:22:52.338 INFO:teuthology.orchestra.run.smithi137.stdout: "op_latency": { 2023-12-01T01:22:52.338 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1095, 2023-12-01T01:22:52.338 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.624988870, 2023-12-01T01:22:52.338 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000570766 2023-12-01T01:22:52.338 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.339 INFO:teuthology.orchestra.run.smithi137.stdout: "op_inflight": 0, 2023-12-01T01:22:52.339 INFO:teuthology.orchestra.run.smithi137.stdout: "oplen_avg": { 2023-12-01T01:22:52.339 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1095, 2023-12-01T01:22:52.339 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 1096 2023-12-01T01:22:52.339 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.339 INFO:teuthology.orchestra.run.smithi137.stdout: "op": 1095, 2023-12-01T01:22:52.339 INFO:teuthology.orchestra.run.smithi137.stdout: "op_r": 1095, 2023-12-01T01:22:52.339 INFO:teuthology.orchestra.run.smithi137.stdout: "op_w": 0, 2023-12-01T01:22:52.339 INFO:teuthology.orchestra.run.smithi137.stdout: "op_rmw": 0, 2023-12-01T01:22:52.339 INFO:teuthology.orchestra.run.smithi137.stdout: "op_pg": 0, 2023-12-01T01:22:52.339 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_stat": 686, 2023-12-01T01:22:52.339 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_create": 0, 2023-12-01T01:22:52.339 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_read": 407, 2023-12-01T01:22:52.339 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_write": 0, 2023-12-01T01:22:52.339 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writefull": 0, 2023-12-01T01:22:52.340 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writesame": 0, 2023-12-01T01:22:52.340 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_append": 0, 2023-12-01T01:22:52.340 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_zero": 0, 2023-12-01T01:22:52.340 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_truncate": 0, 2023-12-01T01:22:52.340 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_delete": 0, 2023-12-01T01:22:52.340 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_mapext": 0, 2023-12-01T01:22:52.340 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_sparse_read": 0, 2023-12-01T01:22:52.340 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_clonerange": 0, 2023-12-01T01:22:52.340 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_getxattr": 1, 2023-12-01T01:22:52.342 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_setxattr": 0, 2023-12-01T01:22:52.342 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_cmpxattr": 0, 2023-12-01T01:22:52.342 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_rmxattr": 0, 2023-12-01T01:22:52.342 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_resetxattrs": 0, 2023-12-01T01:22:52.342 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_call": 0, 2023-12-01T01:22:52.342 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_watch": 0, 2023-12-01T01:22:52.342 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_notify": 0, 2023-12-01T01:22:52.342 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_src_cmpxattr": 0, 2023-12-01T01:22:52.342 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls": 0, 2023-12-01T01:22:52.343 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls_filter": 0, 2023-12-01T01:22:52.343 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_other": 0, 2023-12-01T01:22:52.343 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_active": 0, 2023-12-01T01:22:52.343 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_send": 0, 2023-12-01T01:22:52.343 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_resend": 0, 2023-12-01T01:22:52.343 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_ping": 0, 2023-12-01T01:22:52.343 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_active": 0, 2023-12-01T01:22:52.343 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_send": 0, 2023-12-01T01:22:52.343 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_resend": 0, 2023-12-01T01:22:52.343 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_active": 0, 2023-12-01T01:22:52.343 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_send": 0, 2023-12-01T01:22:52.343 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_resend": 0, 2023-12-01T01:22:52.344 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_active": 0, 2023-12-01T01:22:52.344 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_send": 0, 2023-12-01T01:22:52.344 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_resend": 0, 2023-12-01T01:22:52.344 INFO:teuthology.orchestra.run.smithi137.stdout: "command_active": 0, 2023-12-01T01:22:52.344 INFO:teuthology.orchestra.run.smithi137.stdout: "command_send": 0, 2023-12-01T01:22:52.344 INFO:teuthology.orchestra.run.smithi137.stdout: "command_resend": 0, 2023-12-01T01:22:52.344 INFO:teuthology.orchestra.run.smithi137.stdout: "map_epoch": 84, 2023-12-01T01:22:52.344 INFO:teuthology.orchestra.run.smithi137.stdout: "map_full": 0, 2023-12-01T01:22:52.344 INFO:teuthology.orchestra.run.smithi137.stdout: "map_inc": 2, 2023-12-01T01:22:52.344 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_sessions": 6, 2023-12-01T01:22:52.344 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_open": 6, 2023-12-01T01:22:52.344 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_close": 0, 2023-12-01T01:22:52.344 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_laggy": 0, 2023-12-01T01:22:52.345 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_wr": 0, 2023-12-01T01:22:52.345 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_rd": 2, 2023-12-01T01:22:52.345 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_del": 0 2023-12-01T01:22:52.345 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.345 INFO:teuthology.orchestra.run.smithi137.stdout: "oft": { 2023-12-01T01:22:52.345 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_objs": 0, 2023-12-01T01:22:52.345 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_kv_pairs": 0, 2023-12-01T01:22:52.345 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_updates": 0, 2023-12-01T01:22:52.345 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_removes": 0 2023-12-01T01:22:52.345 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.345 INFO:teuthology.orchestra.run.smithi137.stdout: "osd-slow-ops": { 2023-12-01T01:22:52.345 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_ops_count": 0 2023-12-01T01:22:52.346 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.346 INFO:teuthology.orchestra.run.smithi137.stdout: "purge_queue": { 2023-12-01T01:22:52.346 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops": 0, 2023-12-01T01:22:52.346 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops_high_water": 0, 2023-12-01T01:22:52.346 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing": 0, 2023-12-01T01:22:52.346 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_high_water": 0, 2023-12-01T01:22:52.346 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed": 0, 2023-12-01T01:22:52.346 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_item_in_journal": 0 2023-12-01T01:22:52.346 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.346 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-01T01:22:52.347 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2023-12-01T01:22:52.347 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2023-12-01T01:22:52.347 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2023-12-01T01:22:52.347 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 1237, 2023-12-01T01:22:52.347 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 6501403, 2023-12-01T01:22:52.347 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:52.347 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 1237, 2023-12-01T01:22:52.347 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2023-12-01T01:22:52.347 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2023-12-01T01:22:52.347 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 1237, 2023-12-01T01:22:52.347 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 6501403, 2023-12-01T01:22:52.347 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2023-12-01T01:22:52.347 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:52.348 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:52.348 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:52.348 INFO:teuthology.orchestra.run.smithi137.stdout: } 2023-12-01T01:22:52.348 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.348 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_bytes": { 2023-12-01T01:22:52.348 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2023-12-01T01:22:52.348 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2023-12-01T01:22:52.348 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2023-12-01T01:22:52.348 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2023-12-01T01:22:52.348 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2023-12-01T01:22:52.348 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:52.348 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2023-12-01T01:22:52.348 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 1095, 2023-12-01T01:22:52.349 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 5752186, 2023-12-01T01:22:52.349 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 1095, 2023-12-01T01:22:52.349 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 5752186, 2023-12-01T01:22:52.349 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2023-12-01T01:22:52.349 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:52.349 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:52.349 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:52.349 INFO:teuthology.orchestra.run.smithi137.stdout: } 2023-12-01T01:22:52.349 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.349 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_ops": { 2023-12-01T01:22:52.349 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2023-12-01T01:22:52.349 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 1024, 2023-12-01T01:22:52.350 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2023-12-01T01:22:52.350 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2023-12-01T01:22:52.350 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2023-12-01T01:22:52.350 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:52.350 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2023-12-01T01:22:52.350 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 1095, 2023-12-01T01:22:52.350 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 1095, 2023-12-01T01:22:52.350 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 1095, 2023-12-01T01:22:52.350 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 1095, 2023-12-01T01:22:52.350 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2023-12-01T01:22:52.350 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:52.351 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:52.351 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:52.351 INFO:teuthology.orchestra.run.smithi137.stdout: } 2023-12-01T01:22:52.351 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.351 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle": { 2023-12-01T01:22:52.351 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2023-12-01T01:22:52.351 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2023-12-01T01:22:52.351 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2023-12-01T01:22:52.351 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2023-12-01T01:22:52.351 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2023-12-01T01:22:52.351 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:52.351 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2023-12-01T01:22:52.351 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2023-12-01T01:22:52.352 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2023-12-01T01:22:52.352 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 0, 2023-12-01T01:22:52.352 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 0, 2023-12-01T01:22:52.352 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2023-12-01T01:22:52.352 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:52.352 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:52.352 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:52.353 INFO:teuthology.orchestra.run.smithi137.stdout: } 2023-12-01T01:22:52.353 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2023-12-01T01:22:52.353 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle-0x55da6f6ee5a0": { 2023-12-01T01:22:52.353 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2023-12-01T01:22:52.353 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2023-12-01T01:22:52.353 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2023-12-01T01:22:52.353 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2023-12-01T01:22:52.353 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2023-12-01T01:22:52.353 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:52.353 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2023-12-01T01:22:52.353 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2023-12-01T01:22:52.354 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2023-12-01T01:22:52.354 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 0, 2023-12-01T01:22:52.354 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 0, 2023-12-01T01:22:52.354 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2023-12-01T01:22:52.354 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2023-12-01T01:22:52.354 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2023-12-01T01:22:52.354 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2023-12-01T01:22:52.354 INFO:teuthology.orchestra.run.smithi137.stdout: } 2023-12-01T01:22:52.354 INFO:teuthology.orchestra.run.smithi137.stdout: } 2023-12-01T01:22:52.354 INFO:teuthology.orchestra.run.smithi137.stdout:} 2023-12-01T01:22:52.747 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2023-12-01T01:22:52.747 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2023-12-01T01:22:52.747 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2023-12-01T01:22:52.747 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2023-12-01T01:22:52.747 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2023-12-01T01:22:52.747 DEBUG:tasks.check_counter:Getting stats from c 2023-12-01T01:22:52.748 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph daemon mds.c perf dump 2023-12-01T01:22:53.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:52 smithi203 ceph-mon[103581]: pgmap v448: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 11 op/s 2023-12-01T01:22:53.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:52 smithi137 ceph-mon[103312]: pgmap v448: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 11 op/s 2023-12-01T01:22:53.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:52 smithi077 ceph-mon[98867]: pgmap v448: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 11 op/s 2023-12-01T01:22:53.323 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.c/config 2023-12-01T01:22:54.672 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2023-12-01T01:22:54.672 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-0": { 2023-12-01T01:22:54.672 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 703577, 2023-12-01T01:22:54.672 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 706814, 2023-12-01T01:22:54.672 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 115160839, 2023-12-01T01:22:54.672 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 119194839, 2023-12-01T01:22:54.672 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 17, 2023-12-01T01:22:54.672 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 4, 2023-12-01T01:22:54.672 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 27.441053840, 2023-12-01T01:22:54.673 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 11.477642777, 2023-12-01T01:22:54.673 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 20.169399223, 2023-12-01T01:22:54.673 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.013191058, 2023-12-01T01:22:54.673 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2023-12-01T01:22:54.673 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 706813, 2023-12-01T01:22:54.673 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 7.028675147, 2023-12-01T01:22:54.673 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000009944 2023-12-01T01:22:54.673 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.673 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2023-12-01T01:22:54.673 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1407691, 2023-12-01T01:22:54.673 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.149476422, 2023-12-01T01:22:54.673 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000106 2023-12-01T01:22:54.673 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.673 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 1317648, 2023-12-01T01:22:54.674 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 4389232 2023-12-01T01:22:54.674 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.674 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-1": { 2023-12-01T01:22:54.674 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 1393, 2023-12-01T01:22:54.674 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 1508, 2023-12-01T01:22:54.674 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 948121, 2023-12-01T01:22:54.674 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 2188420, 2023-12-01T01:22:54.674 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 12, 2023-12-01T01:22:54.674 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 5, 2023-12-01T01:22:54.674 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.096538539, 2023-12-01T01:22:54.674 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.034008420, 2023-12-01T01:22:54.674 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.040633518, 2023-12-01T01:22:54.674 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.022983228, 2023-12-01T01:22:54.674 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2023-12-01T01:22:54.675 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1507, 2023-12-01T01:22:54.675 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.094813121, 2023-12-01T01:22:54.675 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000062915 2023-12-01T01:22:54.675 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.675 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2023-12-01T01:22:54.675 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 858, 2023-12-01T01:22:54.675 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000152081, 2023-12-01T01:22:54.675 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000177 2023-12-01T01:22:54.675 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.675 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 568352, 2023-12-01T01:22:54.675 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 84464 2023-12-01T01:22:54.675 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.675 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-2": { 2023-12-01T01:22:54.675 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 156225, 2023-12-01T01:22:54.676 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 156814, 2023-12-01T01:22:54.676 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 27139802, 2023-12-01T01:22:54.676 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 26103066, 2023-12-01T01:22:54.676 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 10, 2023-12-01T01:22:54.676 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 6, 2023-12-01T01:22:54.676 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 4.385298730, 2023-12-01T01:22:54.676 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 1.621864024, 2023-12-01T01:22:54.676 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 3.179133927, 2023-12-01T01:22:54.676 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.013504344, 2023-12-01T01:22:54.676 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2023-12-01T01:22:54.676 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 156812, 2023-12-01T01:22:54.676 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 1.483520731, 2023-12-01T01:22:54.676 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000009460 2023-12-01T01:22:54.676 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.676 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2023-12-01T01:22:54.677 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 311645, 2023-12-01T01:22:54.677 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.027688428, 2023-12-01T01:22:54.677 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000088 2023-12-01T01:22:54.677 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.677 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 160, 2023-12-01T01:22:54.677 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 221808 2023-12-01T01:22:54.677 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.677 INFO:teuthology.orchestra.run.smithi203.stdout: "cct": { 2023-12-01T01:22:54.677 INFO:teuthology.orchestra.run.smithi203.stdout: "total_workers": 1, 2023-12-01T01:22:54.677 INFO:teuthology.orchestra.run.smithi203.stdout: "unhealthy_workers": 0 2023-12-01T01:22:54.677 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.677 INFO:teuthology.orchestra.run.smithi203.stdout: "finisher-MDSRank": { 2023-12-01T01:22:54.677 INFO:teuthology.orchestra.run.smithi203.stdout: "queue_len": 0, 2023-12-01T01:22:54.677 INFO:teuthology.orchestra.run.smithi203.stdout: "complete_latency": { 2023-12-01T01:22:54.678 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 646, 2023-12-01T01:22:54.678 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 1.441992697, 2023-12-01T01:22:54.678 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.002232186 2023-12-01T01:22:54.678 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-12-01T01:22:54.678 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.678 INFO:teuthology.orchestra.run.smithi203.stdout: "finisher-PurgeQueue": { 2023-12-01T01:22:54.678 INFO:teuthology.orchestra.run.smithi203.stdout: "queue_len": 0, 2023-12-01T01:22:54.678 INFO:teuthology.orchestra.run.smithi203.stdout: "complete_latency": { 2023-12-01T01:22:54.678 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 2038, 2023-12-01T01:22:54.678 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.277469025, 2023-12-01T01:22:54.678 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000136147 2023-12-01T01:22:54.678 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-12-01T01:22:54.678 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.678 INFO:teuthology.orchestra.run.smithi203.stdout: "mds": { 2023-12-01T01:22:54.679 INFO:teuthology.orchestra.run.smithi203.stdout: "request": 2820, 2023-12-01T01:22:54.679 INFO:teuthology.orchestra.run.smithi203.stdout: "reply": 2678, 2023-12-01T01:22:54.679 INFO:teuthology.orchestra.run.smithi203.stdout: "reply_latency": { 2023-12-01T01:22:54.679 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 2678, 2023-12-01T01:22:54.679 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 3.779851871, 2023-12-01T01:22:54.679 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.001411445 2023-12-01T01:22:54.679 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.679 INFO:teuthology.orchestra.run.smithi203.stdout: "slow_reply": 0, 2023-12-01T01:22:54.679 INFO:teuthology.orchestra.run.smithi203.stdout: "forward": 142, 2023-12-01T01:22:54.679 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_fetch_complete": 1, 2023-12-01T01:22:54.679 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_fetch_keys": 0, 2023-12-01T01:22:54.680 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_commit": 11, 2023-12-01T01:22:54.680 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_split": 0, 2023-12-01T01:22:54.680 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_merge": 0, 2023-12-01T01:22:54.680 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes": 14, 2023-12-01T01:22:54.680 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_top": 1, 2023-12-01T01:22:54.680 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_bottom": 1, 2023-12-01T01:22:54.680 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_pin_tail": 12, 2023-12-01T01:22:54.680 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_pinned": 12, 2023-12-01T01:22:54.680 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_expired": 255, 2023-12-01T01:22:54.680 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_with_caps": 0, 2023-12-01T01:22:54.680 INFO:teuthology.orchestra.run.smithi203.stdout: "caps": 0, 2023-12-01T01:22:54.680 INFO:teuthology.orchestra.run.smithi203.stdout: "subtrees": 3, 2023-12-01T01:22:54.680 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse": 4606, 2023-12-01T01:22:54.680 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_hit": 4187, 2023-12-01T01:22:54.681 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_forward": 136, 2023-12-01T01:22:54.681 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_discover": 86, 2023-12-01T01:22:54.681 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_dir_fetch": 0, 2023-12-01T01:22:54.681 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_remote_ino": 0, 2023-12-01T01:22:54.681 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_lock": 3, 2023-12-01T01:22:54.681 INFO:teuthology.orchestra.run.smithi203.stdout: "load_cent": 0, 2023-12-01T01:22:54.681 INFO:teuthology.orchestra.run.smithi203.stdout: "q": 0, 2023-12-01T01:22:54.681 INFO:teuthology.orchestra.run.smithi203.stdout: "exported": 38, 2023-12-01T01:22:54.681 INFO:teuthology.orchestra.run.smithi203.stdout: "exported_inodes": 121, 2023-12-01T01:22:54.681 INFO:teuthology.orchestra.run.smithi203.stdout: "imported": 61, 2023-12-01T01:22:54.681 INFO:teuthology.orchestra.run.smithi203.stdout: "imported_inodes": 1079, 2023-12-01T01:22:54.681 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_dir_fetch": 0, 2023-12-01T01:22:54.681 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_backtrace_fetch": 389, 2023-12-01T01:22:54.681 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_peer_discover": 0, 2023-12-01T01:22:54.681 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rfiles": 3568, 2023-12-01T01:22:54.682 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rbytes": 50854799, 2023-12-01T01:22:54.682 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rsnaps": 0, 2023-12-01T01:22:54.682 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_backtrace_fetch": 389, 2023-12-01T01:22:54.682 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_set_tag": 0, 2023-12-01T01:22:54.682 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_backtrace_repaired": 0, 2023-12-01T01:22:54.682 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_inotable_repaired": 0, 2023-12-01T01:22:54.682 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dir_inodes": 0, 2023-12-01T01:22:54.682 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dir_base_inodes": 0, 2023-12-01T01:22:54.682 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dirfrag_rstats": 0, 2023-12-01T01:22:54.682 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_file_inodes": 389, 2023-12-01T01:22:54.682 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_inode_file_caps": 23, 2023-12-01T01:22:54.682 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_revoke": 37, 2023-12-01T01:22:54.682 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_grant": 1417, 2023-12-01T01:22:54.683 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_trunc": 0, 2023-12-01T01:22:54.683 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-01T01:22:54.683 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-01T01:22:54.683 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_caps": 840, 2023-12-01T01:22:54.683 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_caps_dirty": 805, 2023-12-01T01:22:54.683 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_cap_release": 67, 2023-12-01T01:22:54.683 INFO:teuthology.orchestra.run.smithi203.stdout: "process_request_cap_release": 2313 2023-12-01T01:22:54.683 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.683 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_cache": { 2023-12-01T01:22:54.683 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_update": 101, 2023-12-01T01:22:54.683 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_update_receipt": 458, 2023-12-01T01:22:54.683 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_try_discover": 263, 2023-12-01T01:22:54.684 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_send_discover": 421, 2023-12-01T01:22:54.684 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_handle_discover": 196, 2023-12-01T01:22:54.684 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays": 0, 2023-12-01T01:22:54.684 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays_delayed": 0, 2023-12-01T01:22:54.684 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays_enqueuing": 0, 2023-12-01T01:22:54.684 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_created": 1138, 2023-12-01T01:22:54.684 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_enqueued": 1138, 2023-12-01T01:22:54.684 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_reintegrated": 0, 2023-12-01T01:22:54.684 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_migrated": 0, 2023-12-01T01:22:54.684 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_processing": 0, 2023-12-01T01:22:54.684 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_enqueued": 0, 2023-12-01T01:22:54.684 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_prioritized": 0, 2023-12-01T01:22:54.684 INFO:teuthology.orchestra.run.smithi203.stdout: "recovery_started": 0, 2023-12-01T01:22:54.684 INFO:teuthology.orchestra.run.smithi203.stdout: "recovery_completed": 0, 2023-12-01T01:22:54.685 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_enqueue_scrub": 0, 2023-12-01T01:22:54.685 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_exportdir": 38, 2023-12-01T01:22:54.685 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_flush": 0, 2023-12-01T01:22:54.685 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_fragmentdir": 0, 2023-12-01T01:22:54.686 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_fragstats": 0, 2023-12-01T01:22:54.686 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_inodestats": 0 2023-12-01T01:22:54.686 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.686 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_log": { 2023-12-01T01:22:54.686 INFO:teuthology.orchestra.run.smithi203.stdout: "evadd": 3999, 2023-12-01T01:22:54.686 INFO:teuthology.orchestra.run.smithi203.stdout: "evex": 0, 2023-12-01T01:22:54.686 INFO:teuthology.orchestra.run.smithi203.stdout: "evtrm": 0, 2023-12-01T01:22:54.687 INFO:teuthology.orchestra.run.smithi203.stdout: "ev": 3999, 2023-12-01T01:22:54.687 INFO:teuthology.orchestra.run.smithi203.stdout: "evexg": 0, 2023-12-01T01:22:54.687 INFO:teuthology.orchestra.run.smithi203.stdout: "evexd": 0, 2023-12-01T01:22:54.687 INFO:teuthology.orchestra.run.smithi203.stdout: "segadd": 5, 2023-12-01T01:22:54.687 INFO:teuthology.orchestra.run.smithi203.stdout: "segex": 0, 2023-12-01T01:22:54.687 INFO:teuthology.orchestra.run.smithi203.stdout: "segtrm": 0, 2023-12-01T01:22:54.687 INFO:teuthology.orchestra.run.smithi203.stdout: "seg": 5, 2023-12-01T01:22:54.687 INFO:teuthology.orchestra.run.smithi203.stdout: "segexg": 0, 2023-12-01T01:22:54.687 INFO:teuthology.orchestra.run.smithi203.stdout: "segexd": 0, 2023-12-01T01:22:54.687 INFO:teuthology.orchestra.run.smithi203.stdout: "expos": 4194304, 2023-12-01T01:22:54.687 INFO:teuthology.orchestra.run.smithi203.stdout: "wrpos": 10640683, 2023-12-01T01:22:54.687 INFO:teuthology.orchestra.run.smithi203.stdout: "rdpos": 0, 2023-12-01T01:22:54.687 INFO:teuthology.orchestra.run.smithi203.stdout: "jlat": { 2023-12-01T01:22:54.688 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 195, 2023-12-01T01:22:54.688 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 2.071564974, 2023-12-01T01:22:54.688 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.010623410 2023-12-01T01:22:54.688 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.688 INFO:teuthology.orchestra.run.smithi203.stdout: "replayed": 0 2023-12-01T01:22:54.688 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.688 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_mem": { 2023-12-01T01:22:54.688 INFO:teuthology.orchestra.run.smithi203.stdout: "ino": 19, 2023-12-01T01:22:54.688 INFO:teuthology.orchestra.run.smithi203.stdout: "ino+": 1239, 2023-12-01T01:22:54.688 INFO:teuthology.orchestra.run.smithi203.stdout: "ino-": 1220, 2023-12-01T01:22:54.688 INFO:teuthology.orchestra.run.smithi203.stdout: "dir": 17, 2023-12-01T01:22:54.688 INFO:teuthology.orchestra.run.smithi203.stdout: "dir+": 275, 2023-12-01T01:22:54.688 INFO:teuthology.orchestra.run.smithi203.stdout: "dir-": 258, 2023-12-01T01:22:54.688 INFO:teuthology.orchestra.run.smithi203.stdout: "dn": 14, 2023-12-01T01:22:54.688 INFO:teuthology.orchestra.run.smithi203.stdout: "dn+": 2345, 2023-12-01T01:22:54.689 INFO:teuthology.orchestra.run.smithi203.stdout: "dn-": 2331, 2023-12-01T01:22:54.689 INFO:teuthology.orchestra.run.smithi203.stdout: "cap": 0, 2023-12-01T01:22:54.689 INFO:teuthology.orchestra.run.smithi203.stdout: "cap+": 1954, 2023-12-01T01:22:54.689 INFO:teuthology.orchestra.run.smithi203.stdout: "cap-": 1954, 2023-12-01T01:22:54.689 INFO:teuthology.orchestra.run.smithi203.stdout: "rss": 63176, 2023-12-01T01:22:54.689 INFO:teuthology.orchestra.run.smithi203.stdout: "heap": 198964 2023-12-01T01:22:54.689 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.689 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_server": { 2023-12-01T01:22:54.689 INFO:teuthology.orchestra.run.smithi203.stdout: "dispatch_client_request": 3421, 2023-12-01T01:22:54.689 INFO:teuthology.orchestra.run.smithi203.stdout: "dispatch_server_request": 44, 2023-12-01T01:22:54.689 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_request": 2820, 2023-12-01T01:22:54.689 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_session": 5, 2023-12-01T01:22:54.689 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_peer_request": 216, 2023-12-01T01:22:54.689 INFO:teuthology.orchestra.run.smithi203.stdout: "req_create_latency": { 2023-12-01T01:22:54.690 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 193, 2023-12-01T01:22:54.690 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.205970175, 2023-12-01T01:22:54.690 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.001067202 2023-12-01T01:22:54.690 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.690 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getattr_latency": { 2023-12-01T01:22:54.690 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 7, 2023-12-01T01:22:54.690 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.005462095, 2023-12-01T01:22:54.690 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000780299 2023-12-01T01:22:54.690 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.690 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getfilelock_latency": { 2023-12-01T01:22:54.690 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:54.690 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:54.690 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:54.690 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.691 INFO:teuthology.orchestra.run.smithi203.stdout: "req_link_latency": { 2023-12-01T01:22:54.691 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:54.691 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:54.691 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:54.691 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.691 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookup_latency": { 2023-12-01T01:22:54.691 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1302, 2023-12-01T01:22:54.691 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 1.580836547, 2023-12-01T01:22:54.691 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.001214160 2023-12-01T01:22:54.691 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.691 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookuphash_latency": { 2023-12-01T01:22:54.691 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:54.691 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:54.691 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:54.692 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.692 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupino_latency": { 2023-12-01T01:22:54.692 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:54.692 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:54.692 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:54.692 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.692 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupname_latency": { 2023-12-01T01:22:54.692 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:54.692 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:54.692 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:54.692 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.692 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupparent_latency": { 2023-12-01T01:22:54.692 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:54.692 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:54.693 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:54.693 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.693 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupsnap_latency": { 2023-12-01T01:22:54.693 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:54.693 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:54.693 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:54.693 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.693 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lssnap_latency": { 2023-12-01T01:22:54.693 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:54.693 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:54.693 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:54.693 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.693 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mkdir_latency": { 2023-12-01T01:22:54.693 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:54.694 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:54.694 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:54.694 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.694 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mknod_latency": { 2023-12-01T01:22:54.694 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:54.694 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:54.694 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:54.694 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.694 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mksnap_latency": { 2023-12-01T01:22:54.694 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:54.694 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:54.694 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:54.694 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.694 INFO:teuthology.orchestra.run.smithi203.stdout: "req_open_latency": { 2023-12-01T01:22:54.695 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:54.695 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:54.695 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:54.695 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.695 INFO:teuthology.orchestra.run.smithi203.stdout: "req_readdir_latency": { 2023-12-01T01:22:54.695 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 38, 2023-12-01T01:22:54.695 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.028890688, 2023-12-01T01:22:54.695 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000760281 2023-12-01T01:22:54.695 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.695 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rename_latency": { 2023-12-01T01:22:54.695 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:54.695 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:54.695 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:54.695 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.695 INFO:teuthology.orchestra.run.smithi203.stdout: "req_renamesnap_latency": { 2023-12-01T01:22:54.696 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:54.696 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:54.696 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:54.696 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.696 INFO:teuthology.orchestra.run.smithi203.stdout: "req_snapdiff_latency": { 2023-12-01T01:22:54.696 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:54.696 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:54.696 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:54.696 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.696 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmdir_latency": { 2023-12-01T01:22:54.696 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 16, 2023-12-01T01:22:54.696 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.388958947, 2023-12-01T01:22:54.696 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.024309934 2023-12-01T01:22:54.697 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.697 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmsnap_latency": { 2023-12-01T01:22:54.697 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:54.697 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:54.697 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:54.697 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.697 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmxattr_latency": { 2023-12-01T01:22:54.697 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:54.697 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:54.697 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:54.697 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.697 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setattr_latency": { 2023-12-01T01:22:54.697 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:54.697 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:54.697 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:54.698 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.698 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setdirlayout_latency": { 2023-12-01T01:22:54.698 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:54.698 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:54.698 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:54.698 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.698 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setfilelock_latency": { 2023-12-01T01:22:54.698 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:54.698 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:54.698 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:54.698 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.698 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setlayout_latency": { 2023-12-01T01:22:54.698 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:54.698 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:54.699 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:54.699 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.699 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setxattr_latency": { 2023-12-01T01:22:54.699 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:54.699 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:54.699 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:54.699 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.699 INFO:teuthology.orchestra.run.smithi203.stdout: "req_symlink_latency": { 2023-12-01T01:22:54.699 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:54.699 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:54.699 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:54.699 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.699 INFO:teuthology.orchestra.run.smithi203.stdout: "req_unlink_latency": { 2023-12-01T01:22:54.699 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1122, 2023-12-01T01:22:54.700 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 1.569733419, 2023-12-01T01:22:54.700 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.001399049 2023-12-01T01:22:54.700 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.700 INFO:teuthology.orchestra.run.smithi203.stdout: "cap_revoke_eviction": 0, 2023-12-01T01:22:54.700 INFO:teuthology.orchestra.run.smithi203.stdout: "cap_acquisition_throttle": 0, 2023-12-01T01:22:54.700 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getvxattr_latency": { 2023-12-01T01:22:54.700 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:54.700 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:54.700 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:54.700 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-12-01T01:22:54.700 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.700 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_sessions": { 2023-12-01T01:22:54.700 INFO:teuthology.orchestra.run.smithi203.stdout: "session_count": 1, 2023-12-01T01:22:54.700 INFO:teuthology.orchestra.run.smithi203.stdout: "session_add": 1, 2023-12-01T01:22:54.700 INFO:teuthology.orchestra.run.smithi203.stdout: "session_remove": 0, 2023-12-01T01:22:54.701 INFO:teuthology.orchestra.run.smithi203.stdout: "sessions_open": 1, 2023-12-01T01:22:54.701 INFO:teuthology.orchestra.run.smithi203.stdout: "sessions_stale": 0, 2023-12-01T01:22:54.701 INFO:teuthology.orchestra.run.smithi203.stdout: "total_load": 1662, 2023-12-01T01:22:54.701 INFO:teuthology.orchestra.run.smithi203.stdout: "average_load": 1662, 2023-12-01T01:22:54.701 INFO:teuthology.orchestra.run.smithi203.stdout: "avg_session_uptime": 255, 2023-12-01T01:22:54.701 INFO:teuthology.orchestra.run.smithi203.stdout: "mdthresh_evicted": 0 2023-12-01T01:22:54.701 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.701 INFO:teuthology.orchestra.run.smithi203.stdout: "mempool": { 2023-12-01T01:22:54.701 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_bytes": 0, 2023-12-01T01:22:54.701 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_items": 0, 2023-12-01T01:22:54.701 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_bytes": 0, 2023-12-01T01:22:54.701 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_items": 0, 2023-12-01T01:22:54.701 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_bytes": 0, 2023-12-01T01:22:54.701 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_items": 0, 2023-12-01T01:22:54.701 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-01T01:22:54.702 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_items": 0, 2023-12-01T01:22:54.702 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-01T01:22:54.702 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_items": 0, 2023-12-01T01:22:54.702 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_bytes": 0, 2023-12-01T01:22:54.702 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_items": 0, 2023-12-01T01:22:54.702 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-01T01:22:54.702 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_items": 0, 2023-12-01T01:22:54.702 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_bytes": 0, 2023-12-01T01:22:54.702 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_items": 0, 2023-12-01T01:22:54.702 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_bytes": 0, 2023-12-01T01:22:54.702 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_items": 0, 2023-12-01T01:22:54.702 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-01T01:22:54.702 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_items": 0, 2023-12-01T01:22:54.702 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-01T01:22:54.703 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_items": 0, 2023-12-01T01:22:54.703 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_bytes": 0, 2023-12-01T01:22:54.703 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_items": 0, 2023-12-01T01:22:54.703 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_bytes": 0, 2023-12-01T01:22:54.703 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_items": 0, 2023-12-01T01:22:54.703 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-01T01:22:54.703 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_items": 0, 2023-12-01T01:22:54.703 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_bytes": 0, 2023-12-01T01:22:54.703 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_items": 0, 2023-12-01T01:22:54.703 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_bytes": 0, 2023-12-01T01:22:54.703 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_items": 0, 2023-12-01T01:22:54.703 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_bytes": 0, 2023-12-01T01:22:54.703 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_items": 0, 2023-12-01T01:22:54.703 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_bytes": 0, 2023-12-01T01:22:54.704 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_items": 0, 2023-12-01T01:22:54.704 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_bytes": 307559, 2023-12-01T01:22:54.704 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_items": 131, 2023-12-01T01:22:54.704 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_bytes": 880, 2023-12-01T01:22:54.704 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_items": 10, 2023-12-01T01:22:54.704 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_bytes": 0, 2023-12-01T01:22:54.704 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_items": 0, 2023-12-01T01:22:54.704 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_bytes": 0, 2023-12-01T01:22:54.704 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_items": 0, 2023-12-01T01:22:54.704 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_bytes": 0, 2023-12-01T01:22:54.704 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_items": 0, 2023-12-01T01:22:54.704 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_bytes": 7336, 2023-12-01T01:22:54.704 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_items": 154, 2023-12-01T01:22:54.704 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_bytes": 0, 2023-12-01T01:22:54.705 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_items": 0, 2023-12-01T01:22:54.705 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_bytes": 0, 2023-12-01T01:22:54.705 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_items": 0, 2023-12-01T01:22:54.705 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_bytes": 74632, 2023-12-01T01:22:54.705 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_items": 332, 2023-12-01T01:22:54.705 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_bytes": 0, 2023-12-01T01:22:54.705 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_items": 0, 2023-12-01T01:22:54.705 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_bytes": 0, 2023-12-01T01:22:54.705 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_items": 0 2023-12-01T01:22:54.705 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.705 INFO:teuthology.orchestra.run.smithi203.stdout: "objecter": { 2023-12-01T01:22:54.705 INFO:teuthology.orchestra.run.smithi203.stdout: "op_active": 0, 2023-12-01T01:22:54.705 INFO:teuthology.orchestra.run.smithi203.stdout: "op_laggy": 0, 2023-12-01T01:22:54.705 INFO:teuthology.orchestra.run.smithi203.stdout: "op_send": 1898, 2023-12-01T01:22:54.705 INFO:teuthology.orchestra.run.smithi203.stdout: "op_send_bytes": 6615716, 2023-12-01T01:22:54.706 INFO:teuthology.orchestra.run.smithi203.stdout: "op_resend": 27, 2023-12-01T01:22:54.706 INFO:teuthology.orchestra.run.smithi203.stdout: "op_reply": 1898, 2023-12-01T01:22:54.706 INFO:teuthology.orchestra.run.smithi203.stdout: "op_latency": { 2023-12-01T01:22:54.706 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1898, 2023-12-01T01:22:54.706 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 5.522862876, 2023-12-01T01:22:54.706 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.002909832 2023-12-01T01:22:54.706 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.706 INFO:teuthology.orchestra.run.smithi203.stdout: "op_inflight": 0, 2023-12-01T01:22:54.706 INFO:teuthology.orchestra.run.smithi203.stdout: "oplen_avg": { 2023-12-01T01:22:54.706 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1898, 2023-12-01T01:22:54.706 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 1940 2023-12-01T01:22:54.706 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.706 INFO:teuthology.orchestra.run.smithi203.stdout: "op": 1898, 2023-12-01T01:22:54.707 INFO:teuthology.orchestra.run.smithi203.stdout: "op_r": 431, 2023-12-01T01:22:54.707 INFO:teuthology.orchestra.run.smithi203.stdout: "op_w": 1467, 2023-12-01T01:22:54.707 INFO:teuthology.orchestra.run.smithi203.stdout: "op_rmw": 0, 2023-12-01T01:22:54.707 INFO:teuthology.orchestra.run.smithi203.stdout: "op_pg": 0, 2023-12-01T01:22:54.707 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_stat": 0, 2023-12-01T01:22:54.707 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_create": 10, 2023-12-01T01:22:54.707 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_read": 41, 2023-12-01T01:22:54.707 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_write": 236, 2023-12-01T01:22:54.707 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_writefull": 33, 2023-12-01T01:22:54.707 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_writesame": 0, 2023-12-01T01:22:54.707 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_append": 0, 2023-12-01T01:22:54.707 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_zero": 0, 2023-12-01T01:22:54.707 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_truncate": 0, 2023-12-01T01:22:54.707 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_delete": 1159, 2023-12-01T01:22:54.708 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_mapext": 0, 2023-12-01T01:22:54.708 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_sparse_read": 0, 2023-12-01T01:22:54.708 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_clonerange": 0, 2023-12-01T01:22:54.708 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_getxattr": 390, 2023-12-01T01:22:54.708 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_setxattr": 20, 2023-12-01T01:22:54.708 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_cmpxattr": 0, 2023-12-01T01:22:54.708 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_rmxattr": 0, 2023-12-01T01:22:54.708 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_resetxattrs": 0, 2023-12-01T01:22:54.708 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_call": 0, 2023-12-01T01:22:54.708 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_watch": 0, 2023-12-01T01:22:54.708 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_notify": 0, 2023-12-01T01:22:54.708 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_src_cmpxattr": 0, 2023-12-01T01:22:54.708 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_pgls": 0, 2023-12-01T01:22:54.708 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_pgls_filter": 0, 2023-12-01T01:22:54.708 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_other": 0, 2023-12-01T01:22:54.709 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_active": 0, 2023-12-01T01:22:54.709 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_send": 0, 2023-12-01T01:22:54.709 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_resend": 0, 2023-12-01T01:22:54.709 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_ping": 0, 2023-12-01T01:22:54.709 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_active": 0, 2023-12-01T01:22:54.709 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_send": 0, 2023-12-01T01:22:54.709 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_resend": 0, 2023-12-01T01:22:54.709 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_active": 0, 2023-12-01T01:22:54.709 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_send": 0, 2023-12-01T01:22:54.709 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_resend": 0, 2023-12-01T01:22:54.709 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_active": 0, 2023-12-01T01:22:54.709 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_send": 0, 2023-12-01T01:22:54.709 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_resend": 0, 2023-12-01T01:22:54.709 INFO:teuthology.orchestra.run.smithi203.stdout: "command_active": 0, 2023-12-01T01:22:54.709 INFO:teuthology.orchestra.run.smithi203.stdout: "command_send": 0, 2023-12-01T01:22:54.710 INFO:teuthology.orchestra.run.smithi203.stdout: "command_resend": 0, 2023-12-01T01:22:54.710 INFO:teuthology.orchestra.run.smithi203.stdout: "map_epoch": 84, 2023-12-01T01:22:54.710 INFO:teuthology.orchestra.run.smithi203.stdout: "map_full": 0, 2023-12-01T01:22:54.710 INFO:teuthology.orchestra.run.smithi203.stdout: "map_inc": 2, 2023-12-01T01:22:54.710 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_sessions": 12, 2023-12-01T01:22:54.710 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_session_open": 12, 2023-12-01T01:22:54.710 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_session_close": 0, 2023-12-01T01:22:54.710 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_laggy": 0, 2023-12-01T01:22:54.710 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_wr": 39, 2023-12-01T01:22:54.710 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_rd": 2, 2023-12-01T01:22:54.710 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_del": 10 2023-12-01T01:22:54.710 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.710 INFO:teuthology.orchestra.run.smithi203.stdout: "oft": { 2023-12-01T01:22:54.710 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_objs": 1, 2023-12-01T01:22:54.711 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_kv_pairs": 0, 2023-12-01T01:22:54.711 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_updates": 0, 2023-12-01T01:22:54.711 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_removes": 0 2023-12-01T01:22:54.711 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.711 INFO:teuthology.orchestra.run.smithi203.stdout: "osd-slow-ops": { 2023-12-01T01:22:54.711 INFO:teuthology.orchestra.run.smithi203.stdout: "slow_ops_count": 2902 2023-12-01T01:22:54.711 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.711 INFO:teuthology.orchestra.run.smithi203.stdout: "purge_queue": { 2023-12-01T01:22:54.711 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_ops": 0, 2023-12-01T01:22:54.711 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_ops_high_water": 14, 2023-12-01T01:22:54.711 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing": 0, 2023-12-01T01:22:54.711 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_high_water": 6, 2023-12-01T01:22:54.711 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executed": 1138, 2023-12-01T01:22:54.711 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_item_in_journal": 0 2023-12-01T01:22:54.711 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.712 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-01T01:22:54.712 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2023-12-01T01:22:54.712 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2023-12-01T01:22:54.712 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2023-12-01T01:22:54.712 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 861191, 2023-12-01T01:22:54.712 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 65564702, 2023-12-01T01:22:54.712 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:54.712 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 861191, 2023-12-01T01:22:54.712 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2023-12-01T01:22:54.712 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2023-12-01T01:22:54.712 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 861191, 2023-12-01T01:22:54.712 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 65564702, 2023-12-01T01:22:54.712 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2023-12-01T01:22:54.712 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:54.713 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:54.713 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:54.713 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-12-01T01:22:54.713 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.713 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-objecter_bytes": { 2023-12-01T01:22:54.713 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2023-12-01T01:22:54.713 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2023-12-01T01:22:54.713 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2023-12-01T01:22:54.713 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2023-12-01T01:22:54.713 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2023-12-01T01:22:54.713 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:54.713 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2023-12-01T01:22:54.713 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 1898, 2023-12-01T01:22:54.713 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 6730646, 2023-12-01T01:22:54.714 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 1898, 2023-12-01T01:22:54.714 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 6730646, 2023-12-01T01:22:54.714 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2023-12-01T01:22:54.714 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:54.714 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:54.714 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:54.714 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-12-01T01:22:54.714 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.714 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-objecter_ops": { 2023-12-01T01:22:54.714 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2023-12-01T01:22:54.714 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 1024, 2023-12-01T01:22:54.714 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2023-12-01T01:22:54.714 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2023-12-01T01:22:54.714 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2023-12-01T01:22:54.715 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:54.715 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2023-12-01T01:22:54.715 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 1898, 2023-12-01T01:22:54.715 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 1898, 2023-12-01T01:22:54.715 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 1898, 2023-12-01T01:22:54.715 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 1898, 2023-12-01T01:22:54.715 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2023-12-01T01:22:54.715 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:54.715 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:54.715 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:54.715 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-12-01T01:22:54.715 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.715 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-write_buf_throttle": { 2023-12-01T01:22:54.715 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2023-12-01T01:22:54.715 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 3758096384, 2023-12-01T01:22:54.716 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2023-12-01T01:22:54.716 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 1138, 2023-12-01T01:22:54.716 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 114946, 2023-12-01T01:22:54.716 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:54.716 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 1138, 2023-12-01T01:22:54.716 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2023-12-01T01:22:54.716 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2023-12-01T01:22:54.716 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 41, 2023-12-01T01:22:54.716 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 114946, 2023-12-01T01:22:54.717 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2023-12-01T01:22:54.717 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:54.717 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:54.717 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:54.717 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-12-01T01:22:54.718 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:54.718 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-write_buf_throttle-0x55ea5ee220a0": { 2023-12-01T01:22:54.718 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2023-12-01T01:22:54.718 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 3758096384, 2023-12-01T01:22:54.718 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2023-12-01T01:22:54.718 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 3999, 2023-12-01T01:22:54.718 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 6446379, 2023-12-01T01:22:54.718 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:54.718 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 3999, 2023-12-01T01:22:54.718 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2023-12-01T01:22:54.718 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2023-12-01T01:22:54.718 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 195, 2023-12-01T01:22:54.718 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 6446379, 2023-12-01T01:22:54.718 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2023-12-01T01:22:54.719 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:54.719 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:54.719 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:54.719 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-12-01T01:22:54.719 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-12-01T01:22:54.719 INFO:teuthology.orchestra.run.smithi203.stdout:} 2023-12-01T01:22:55.083 INFO:tasks.check_counter:Daemon mds.c mds.exported=38 2023-12-01T01:22:55.083 INFO:tasks.check_counter:Daemon mds.c mds.imported=61 2023-12-01T01:22:55.083 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=101 2023-12-01T01:22:55.083 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=458 2023-12-01T01:22:55.083 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2023-12-01T01:22:55.083 DEBUG:tasks.check_counter:Getting stats from f 2023-12-01T01:22:55.084 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph daemon mds.f perf dump 2023-12-01T01:22:55.634 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.c/config 2023-12-01T01:22:55.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:55 smithi203 ceph-mon[103581]: pgmap v449: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 12 op/s 2023-12-01T01:22:55.778 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:55 smithi137 ceph-mon[103312]: pgmap v449: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 12 op/s 2023-12-01T01:22:55.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:55 smithi077 ceph-mon[98867]: pgmap v449: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 12 op/s 2023-12-01T01:22:56.968 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2023-12-01T01:22:56.968 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-0": { 2023-12-01T01:22:56.968 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 735, 2023-12-01T01:22:56.968 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 735, 2023-12-01T01:22:56.968 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 29583177, 2023-12-01T01:22:56.968 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 227799, 2023-12-01T01:22:56.968 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 8, 2023-12-01T01:22:56.969 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 4, 2023-12-01T01:22:56.969 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.080705241, 2023-12-01T01:22:56.969 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.016794768, 2023-12-01T01:22:56.969 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.033922128, 2023-12-01T01:22:56.969 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.023166180, 2023-12-01T01:22:56.969 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2023-12-01T01:22:56.969 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 734, 2023-12-01T01:22:56.969 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.028476048, 2023-12-01T01:22:56.969 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000038795 2023-12-01T01:22:56.969 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.969 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2023-12-01T01:22:56.969 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:56.969 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:56.969 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:56.970 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.970 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-12-01T01:22:56.970 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 448 2023-12-01T01:22:56.970 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.970 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-1": { 2023-12-01T01:22:56.970 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 395, 2023-12-01T01:22:56.970 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 479, 2023-12-01T01:22:56.970 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 33516918, 2023-12-01T01:22:56.970 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 343160, 2023-12-01T01:22:56.970 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 6, 2023-12-01T01:22:56.970 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 5, 2023-12-01T01:22:56.970 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.042337770, 2023-12-01T01:22:56.970 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.011059004, 2023-12-01T01:22:56.971 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.019101552, 2023-12-01T01:22:56.971 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.009651998, 2023-12-01T01:22:56.971 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2023-12-01T01:22:56.971 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 477, 2023-12-01T01:22:56.971 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.038390120, 2023-12-01T01:22:56.971 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000080482 2023-12-01T01:22:56.971 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.971 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2023-12-01T01:22:56.971 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:56.971 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:56.971 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:56.971 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.971 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 160, 2023-12-01T01:22:56.972 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 220560 2023-12-01T01:22:56.972 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.972 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-2": { 2023-12-01T01:22:56.972 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 404, 2023-12-01T01:22:56.972 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 384, 2023-12-01T01:22:56.972 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 25789586, 2023-12-01T01:22:56.972 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 146690, 2023-12-01T01:22:56.972 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 5, 2023-12-01T01:22:56.973 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 4, 2023-12-01T01:22:56.973 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.052309514, 2023-12-01T01:22:56.973 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.013745932, 2023-12-01T01:22:56.973 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.028268773, 2023-12-01T01:22:56.973 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.010712038, 2023-12-01T01:22:56.973 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2023-12-01T01:22:56.973 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 383, 2023-12-01T01:22:56.973 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.016306983, 2023-12-01T01:22:56.973 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000042576 2023-12-01T01:22:56.973 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.973 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2023-12-01T01:22:56.973 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:56.973 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:56.973 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:56.974 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.974 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 556848, 2023-12-01T01:22:56.974 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 64896 2023-12-01T01:22:56.974 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.974 INFO:teuthology.orchestra.run.smithi203.stdout: "cct": { 2023-12-01T01:22:56.974 INFO:teuthology.orchestra.run.smithi203.stdout: "total_workers": 1, 2023-12-01T01:22:56.974 INFO:teuthology.orchestra.run.smithi203.stdout: "unhealthy_workers": 0 2023-12-01T01:22:56.974 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.974 INFO:teuthology.orchestra.run.smithi203.stdout: "finisher-MDSRank": { 2023-12-01T01:22:56.974 INFO:teuthology.orchestra.run.smithi203.stdout: "queue_len": 0, 2023-12-01T01:22:56.974 INFO:teuthology.orchestra.run.smithi203.stdout: "complete_latency": { 2023-12-01T01:22:56.974 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 2344, 2023-12-01T01:22:56.974 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.118242782, 2023-12-01T01:22:56.974 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000050444 2023-12-01T01:22:56.975 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-12-01T01:22:56.975 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.975 INFO:teuthology.orchestra.run.smithi203.stdout: "finisher-PurgeQueue": { 2023-12-01T01:22:56.975 INFO:teuthology.orchestra.run.smithi203.stdout: "queue_len": 0, 2023-12-01T01:22:56.975 INFO:teuthology.orchestra.run.smithi203.stdout: "complete_latency": { 2023-12-01T01:22:56.975 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:56.975 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:56.975 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:56.975 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-12-01T01:22:56.975 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.975 INFO:teuthology.orchestra.run.smithi203.stdout: "mds": { 2023-12-01T01:22:56.975 INFO:teuthology.orchestra.run.smithi203.stdout: "request": 0, 2023-12-01T01:22:56.975 INFO:teuthology.orchestra.run.smithi203.stdout: "reply": 0, 2023-12-01T01:22:56.975 INFO:teuthology.orchestra.run.smithi203.stdout: "reply_latency": { 2023-12-01T01:22:56.976 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:56.976 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:56.976 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:56.976 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.976 INFO:teuthology.orchestra.run.smithi203.stdout: "slow_reply": 0, 2023-12-01T01:22:56.976 INFO:teuthology.orchestra.run.smithi203.stdout: "forward": 0, 2023-12-01T01:22:56.976 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_fetch_complete": 0, 2023-12-01T01:22:56.976 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_fetch_keys": 0, 2023-12-01T01:22:56.976 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_commit": 0, 2023-12-01T01:22:56.976 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_split": 0, 2023-12-01T01:22:56.976 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_merge": 0, 2023-12-01T01:22:56.976 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes": 729, 2023-12-01T01:22:56.976 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_top": 487, 2023-12-01T01:22:56.976 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_bottom": 212, 2023-12-01T01:22:56.977 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_pin_tail": 30, 2023-12-01T01:22:56.977 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_pinned": 32, 2023-12-01T01:22:56.977 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_expired": 1446, 2023-12-01T01:22:56.977 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_with_caps": 0, 2023-12-01T01:22:56.977 INFO:teuthology.orchestra.run.smithi203.stdout: "caps": 0, 2023-12-01T01:22:56.977 INFO:teuthology.orchestra.run.smithi203.stdout: "subtrees": 3, 2023-12-01T01:22:56.977 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse": 0, 2023-12-01T01:22:56.977 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_hit": 0, 2023-12-01T01:22:56.977 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_forward": 0, 2023-12-01T01:22:56.977 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_discover": 0, 2023-12-01T01:22:56.977 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_dir_fetch": 0, 2023-12-01T01:22:56.977 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_remote_ino": 0, 2023-12-01T01:22:56.977 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_lock": 0, 2023-12-01T01:22:56.977 INFO:teuthology.orchestra.run.smithi203.stdout: "load_cent": 0, 2023-12-01T01:22:56.977 INFO:teuthology.orchestra.run.smithi203.stdout: "q": 0, 2023-12-01T01:22:56.978 INFO:teuthology.orchestra.run.smithi203.stdout: "exported": 0, 2023-12-01T01:22:56.978 INFO:teuthology.orchestra.run.smithi203.stdout: "exported_inodes": 0, 2023-12-01T01:22:56.978 INFO:teuthology.orchestra.run.smithi203.stdout: "imported": 0, 2023-12-01T01:22:56.978 INFO:teuthology.orchestra.run.smithi203.stdout: "imported_inodes": 0, 2023-12-01T01:22:56.978 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_dir_fetch": 0, 2023-12-01T01:22:56.978 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_backtrace_fetch": 0, 2023-12-01T01:22:56.978 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_peer_discover": 0, 2023-12-01T01:22:56.978 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rfiles": 4, 2023-12-01T01:22:56.978 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rbytes": 246, 2023-12-01T01:22:56.978 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rsnaps": 0, 2023-12-01T01:22:56.978 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_backtrace_fetch": 0, 2023-12-01T01:22:56.978 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_set_tag": 0, 2023-12-01T01:22:56.978 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_backtrace_repaired": 0, 2023-12-01T01:22:56.978 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_inotable_repaired": 0, 2023-12-01T01:22:56.979 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dir_inodes": 0, 2023-12-01T01:22:56.979 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dir_base_inodes": 0, 2023-12-01T01:22:56.979 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dirfrag_rstats": 0, 2023-12-01T01:22:56.979 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_file_inodes": 0, 2023-12-01T01:22:56.979 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_inode_file_caps": 0, 2023-12-01T01:22:56.979 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_revoke": 0, 2023-12-01T01:22:56.979 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_grant": 0, 2023-12-01T01:22:56.979 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_trunc": 0, 2023-12-01T01:22:56.979 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-01T01:22:56.979 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-01T01:22:56.979 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_caps": 0, 2023-12-01T01:22:56.979 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_caps_dirty": 0, 2023-12-01T01:22:56.979 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_cap_release": 0, 2023-12-01T01:22:56.979 INFO:teuthology.orchestra.run.smithi203.stdout: "process_request_cap_release": 0 2023-12-01T01:22:56.979 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.980 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_cache": { 2023-12-01T01:22:56.980 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_update": 0, 2023-12-01T01:22:56.980 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_update_receipt": 0, 2023-12-01T01:22:56.980 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_try_discover": 0, 2023-12-01T01:22:56.980 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_send_discover": 0, 2023-12-01T01:22:56.980 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_handle_discover": 0, 2023-12-01T01:22:56.980 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays": 0, 2023-12-01T01:22:56.980 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays_delayed": 0, 2023-12-01T01:22:56.980 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays_enqueuing": 0, 2023-12-01T01:22:56.980 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_created": 0, 2023-12-01T01:22:56.980 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_enqueued": 0, 2023-12-01T01:22:56.980 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_reintegrated": 0, 2023-12-01T01:22:56.980 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_migrated": 0, 2023-12-01T01:22:56.980 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_processing": 0, 2023-12-01T01:22:56.981 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_enqueued": 0, 2023-12-01T01:22:56.981 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_prioritized": 0, 2023-12-01T01:22:56.981 INFO:teuthology.orchestra.run.smithi203.stdout: "recovery_started": 0, 2023-12-01T01:22:56.981 INFO:teuthology.orchestra.run.smithi203.stdout: "recovery_completed": 0, 2023-12-01T01:22:56.981 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_enqueue_scrub": 0, 2023-12-01T01:22:56.981 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_exportdir": 0, 2023-12-01T01:22:56.981 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_flush": 0, 2023-12-01T01:22:56.981 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_fragmentdir": 0, 2023-12-01T01:22:56.981 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_fragstats": 0, 2023-12-01T01:22:56.981 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_inodestats": 0 2023-12-01T01:22:56.981 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.981 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_log": { 2023-12-01T01:22:56.981 INFO:teuthology.orchestra.run.smithi203.stdout: "evadd": 0, 2023-12-01T01:22:56.981 INFO:teuthology.orchestra.run.smithi203.stdout: "evex": 0, 2023-12-01T01:22:56.981 INFO:teuthology.orchestra.run.smithi203.stdout: "evtrm": 0, 2023-12-01T01:22:56.982 INFO:teuthology.orchestra.run.smithi203.stdout: "ev": 40920, 2023-12-01T01:22:56.982 INFO:teuthology.orchestra.run.smithi203.stdout: "evexg": 0, 2023-12-01T01:22:56.982 INFO:teuthology.orchestra.run.smithi203.stdout: "evexd": 0, 2023-12-01T01:22:56.982 INFO:teuthology.orchestra.run.smithi203.stdout: "segadd": 0, 2023-12-01T01:22:56.982 INFO:teuthology.orchestra.run.smithi203.stdout: "segex": 0, 2023-12-01T01:22:56.982 INFO:teuthology.orchestra.run.smithi203.stdout: "segtrm": 0, 2023-12-01T01:22:56.982 INFO:teuthology.orchestra.run.smithi203.stdout: "seg": 2, 2023-12-01T01:22:56.982 INFO:teuthology.orchestra.run.smithi203.stdout: "segexg": 0, 2023-12-01T01:22:56.982 INFO:teuthology.orchestra.run.smithi203.stdout: "segexd": 0, 2023-12-01T01:22:56.982 INFO:teuthology.orchestra.run.smithi203.stdout: "expos": 92037451, 2023-12-01T01:22:56.982 INFO:teuthology.orchestra.run.smithi203.stdout: "wrpos": 92133384, 2023-12-01T01:22:56.982 INFO:teuthology.orchestra.run.smithi203.stdout: "rdpos": 92132785, 2023-12-01T01:22:56.982 INFO:teuthology.orchestra.run.smithi203.stdout: "jlat": { 2023-12-01T01:22:56.985 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:56.985 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:56.985 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:56.985 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.986 INFO:teuthology.orchestra.run.smithi203.stdout: "replayed": 40920 2023-12-01T01:22:56.986 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.986 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_mem": { 2023-12-01T01:22:56.986 INFO:teuthology.orchestra.run.smithi203.stdout: "ino": 22, 2023-12-01T01:22:56.986 INFO:teuthology.orchestra.run.smithi203.stdout: "ino+": 5986, 2023-12-01T01:22:56.986 INFO:teuthology.orchestra.run.smithi203.stdout: "ino-": 5964, 2023-12-01T01:22:56.986 INFO:teuthology.orchestra.run.smithi203.stdout: "dir": 18, 2023-12-01T01:22:56.986 INFO:teuthology.orchestra.run.smithi203.stdout: "dir+": 270, 2023-12-01T01:22:56.986 INFO:teuthology.orchestra.run.smithi203.stdout: "dir-": 252, 2023-12-01T01:22:56.986 INFO:teuthology.orchestra.run.smithi203.stdout: "dn": 729, 2023-12-01T01:22:56.986 INFO:teuthology.orchestra.run.smithi203.stdout: "dn+": 6679, 2023-12-01T01:22:56.986 INFO:teuthology.orchestra.run.smithi203.stdout: "dn-": 5950, 2023-12-01T01:22:56.986 INFO:teuthology.orchestra.run.smithi203.stdout: "cap": 0, 2023-12-01T01:22:56.987 INFO:teuthology.orchestra.run.smithi203.stdout: "cap+": 0, 2023-12-01T01:22:56.987 INFO:teuthology.orchestra.run.smithi203.stdout: "cap-": 0, 2023-12-01T01:22:56.987 INFO:teuthology.orchestra.run.smithi203.stdout: "rss": 58784, 2023-12-01T01:22:56.987 INFO:teuthology.orchestra.run.smithi203.stdout: "heap": 207156 2023-12-01T01:22:56.987 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.987 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_server": { 2023-12-01T01:22:56.987 INFO:teuthology.orchestra.run.smithi203.stdout: "dispatch_client_request": 0, 2023-12-01T01:22:56.987 INFO:teuthology.orchestra.run.smithi203.stdout: "dispatch_server_request": 0, 2023-12-01T01:22:56.987 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_request": 0, 2023-12-01T01:22:56.987 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_session": 0, 2023-12-01T01:22:56.987 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_peer_request": 0, 2023-12-01T01:22:56.987 INFO:teuthology.orchestra.run.smithi203.stdout: "req_create_latency": { 2023-12-01T01:22:56.987 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:56.987 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:56.988 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:56.988 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.988 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getattr_latency": { 2023-12-01T01:22:56.988 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:56.988 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:56.988 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:56.988 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.988 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getfilelock_latency": { 2023-12-01T01:22:56.988 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:56.988 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:56.988 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:56.988 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.988 INFO:teuthology.orchestra.run.smithi203.stdout: "req_link_latency": { 2023-12-01T01:22:56.988 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:56.989 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:56.989 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:56.989 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.989 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookup_latency": { 2023-12-01T01:22:56.989 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:56.989 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:56.989 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:56.989 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.989 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookuphash_latency": { 2023-12-01T01:22:56.989 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:56.989 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:56.989 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:56.989 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.989 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupino_latency": { 2023-12-01T01:22:56.990 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:56.990 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:56.990 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:56.990 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.990 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupname_latency": { 2023-12-01T01:22:56.990 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:56.990 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:56.990 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:56.990 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.990 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupparent_latency": { 2023-12-01T01:22:56.990 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:56.990 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:56.990 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:56.991 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.991 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupsnap_latency": { 2023-12-01T01:22:56.991 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:56.991 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:56.991 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:56.991 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.991 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lssnap_latency": { 2023-12-01T01:22:56.991 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:56.991 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:56.991 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:56.991 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.991 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mkdir_latency": { 2023-12-01T01:22:56.991 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:56.991 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:56.992 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:56.992 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.992 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mknod_latency": { 2023-12-01T01:22:56.992 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:56.992 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:56.992 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:56.992 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.992 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mksnap_latency": { 2023-12-01T01:22:56.992 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:56.992 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:56.992 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:56.992 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.992 INFO:teuthology.orchestra.run.smithi203.stdout: "req_open_latency": { 2023-12-01T01:22:56.993 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:56.993 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:56.993 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:56.993 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.993 INFO:teuthology.orchestra.run.smithi203.stdout: "req_readdir_latency": { 2023-12-01T01:22:56.993 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:56.993 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:56.993 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:56.993 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.993 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rename_latency": { 2023-12-01T01:22:56.993 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:56.993 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:56.993 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:56.993 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.993 INFO:teuthology.orchestra.run.smithi203.stdout: "req_renamesnap_latency": { 2023-12-01T01:22:56.994 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:56.994 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:56.994 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:56.994 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.994 INFO:teuthology.orchestra.run.smithi203.stdout: "req_snapdiff_latency": { 2023-12-01T01:22:56.994 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:56.994 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:56.994 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:56.994 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.994 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmdir_latency": { 2023-12-01T01:22:56.994 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:56.994 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:56.995 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:56.995 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.995 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmsnap_latency": { 2023-12-01T01:22:56.995 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:56.995 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:56.995 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:56.995 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.995 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmxattr_latency": { 2023-12-01T01:22:56.995 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:56.995 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:56.996 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:56.996 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.996 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setattr_latency": { 2023-12-01T01:22:56.997 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:56.997 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:56.997 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:56.997 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.997 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setdirlayout_latency": { 2023-12-01T01:22:56.997 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:56.997 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:56.997 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:56.997 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.997 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setfilelock_latency": { 2023-12-01T01:22:56.997 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:56.997 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:56.998 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:56.998 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.998 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setlayout_latency": { 2023-12-01T01:22:56.998 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:56.998 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:56.998 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:56.998 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.998 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setxattr_latency": { 2023-12-01T01:22:56.998 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:56.998 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:56.999 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:56.999 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.999 INFO:teuthology.orchestra.run.smithi203.stdout: "req_symlink_latency": { 2023-12-01T01:22:56.999 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:56.999 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:56.999 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:56.999 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.999 INFO:teuthology.orchestra.run.smithi203.stdout: "req_unlink_latency": { 2023-12-01T01:22:56.999 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:56.999 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:56.999 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:56.999 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:56.999 INFO:teuthology.orchestra.run.smithi203.stdout: "cap_revoke_eviction": 0, 2023-12-01T01:22:56.999 INFO:teuthology.orchestra.run.smithi203.stdout: "cap_acquisition_throttle": 0, 2023-12-01T01:22:57.000 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getvxattr_latency": { 2023-12-01T01:22:57.000 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:57.000 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:57.000 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:57.000 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-12-01T01:22:57.000 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:57.000 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_sessions": { 2023-12-01T01:22:57.000 INFO:teuthology.orchestra.run.smithi203.stdout: "session_count": 1, 2023-12-01T01:22:57.000 INFO:teuthology.orchestra.run.smithi203.stdout: "session_add": 2, 2023-12-01T01:22:57.000 INFO:teuthology.orchestra.run.smithi203.stdout: "session_remove": 1, 2023-12-01T01:22:57.000 INFO:teuthology.orchestra.run.smithi203.stdout: "sessions_open": 2, 2023-12-01T01:22:57.000 INFO:teuthology.orchestra.run.smithi203.stdout: "sessions_stale": 0, 2023-12-01T01:22:57.000 INFO:teuthology.orchestra.run.smithi203.stdout: "total_load": 0, 2023-12-01T01:22:57.000 INFO:teuthology.orchestra.run.smithi203.stdout: "average_load": 0, 2023-12-01T01:22:57.000 INFO:teuthology.orchestra.run.smithi203.stdout: "avg_session_uptime": 2915, 2023-12-01T01:22:57.001 INFO:teuthology.orchestra.run.smithi203.stdout: "mdthresh_evicted": 0 2023-12-01T01:22:57.001 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:57.001 INFO:teuthology.orchestra.run.smithi203.stdout: "mempool": { 2023-12-01T01:22:57.001 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_bytes": 0, 2023-12-01T01:22:57.001 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_items": 0, 2023-12-01T01:22:57.001 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_bytes": 0, 2023-12-01T01:22:57.001 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_items": 0, 2023-12-01T01:22:57.001 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_bytes": 0, 2023-12-01T01:22:57.001 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_items": 0, 2023-12-01T01:22:57.001 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-01T01:22:57.001 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_items": 0, 2023-12-01T01:22:57.001 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-01T01:22:57.001 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_items": 0, 2023-12-01T01:22:57.001 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_bytes": 0, 2023-12-01T01:22:57.002 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_items": 0, 2023-12-01T01:22:57.002 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-01T01:22:57.002 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_items": 0, 2023-12-01T01:22:57.002 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_bytes": 0, 2023-12-01T01:22:57.002 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_items": 0, 2023-12-01T01:22:57.002 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_bytes": 0, 2023-12-01T01:22:57.002 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_items": 0, 2023-12-01T01:22:57.002 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-01T01:22:57.002 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_items": 0, 2023-12-01T01:22:57.002 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-01T01:22:57.002 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_items": 0, 2023-12-01T01:22:57.002 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_bytes": 0, 2023-12-01T01:22:57.002 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_items": 0, 2023-12-01T01:22:57.002 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_bytes": 0, 2023-12-01T01:22:57.002 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_items": 0, 2023-12-01T01:22:57.003 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-01T01:22:57.003 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_items": 0, 2023-12-01T01:22:57.003 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_bytes": 0, 2023-12-01T01:22:57.003 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_items": 0, 2023-12-01T01:22:57.003 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_bytes": 0, 2023-12-01T01:22:57.003 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_items": 0, 2023-12-01T01:22:57.003 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_bytes": 0, 2023-12-01T01:22:57.003 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_items": 0, 2023-12-01T01:22:57.003 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_bytes": 0, 2023-12-01T01:22:57.003 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_items": 0, 2023-12-01T01:22:57.003 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_bytes": 230474, 2023-12-01T01:22:57.003 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_items": 72, 2023-12-01T01:22:57.003 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_bytes": 0, 2023-12-01T01:22:57.003 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_items": 0, 2023-12-01T01:22:57.003 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_bytes": 0, 2023-12-01T01:22:57.004 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_items": 0, 2023-12-01T01:22:57.004 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_bytes": 0, 2023-12-01T01:22:57.004 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_items": 0, 2023-12-01T01:22:57.004 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_bytes": 0, 2023-12-01T01:22:57.004 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_items": 0, 2023-12-01T01:22:57.004 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_bytes": 7336, 2023-12-01T01:22:57.004 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_items": 154, 2023-12-01T01:22:57.004 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_bytes": 0, 2023-12-01T01:22:57.004 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_items": 0, 2023-12-01T01:22:57.004 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_bytes": 0, 2023-12-01T01:22:57.004 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_items": 0, 2023-12-01T01:22:57.004 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_bytes": 578567, 2023-12-01T01:22:57.004 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_items": 3259, 2023-12-01T01:22:57.004 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_bytes": 0, 2023-12-01T01:22:57.005 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_items": 0, 2023-12-01T01:22:57.005 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_bytes": 0, 2023-12-01T01:22:57.005 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_items": 0 2023-12-01T01:22:57.005 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:57.005 INFO:teuthology.orchestra.run.smithi203.stdout: "objecter": { 2023-12-01T01:22:57.005 INFO:teuthology.orchestra.run.smithi203.stdout: "op_active": 0, 2023-12-01T01:22:57.005 INFO:teuthology.orchestra.run.smithi203.stdout: "op_laggy": 0, 2023-12-01T01:22:57.005 INFO:teuthology.orchestra.run.smithi203.stdout: "op_send": 1388, 2023-12-01T01:22:57.005 INFO:teuthology.orchestra.run.smithi203.stdout: "op_send_bytes": 26, 2023-12-01T01:22:57.005 INFO:teuthology.orchestra.run.smithi203.stdout: "op_resend": 4, 2023-12-01T01:22:57.005 INFO:teuthology.orchestra.run.smithi203.stdout: "op_reply": 1388, 2023-12-01T01:22:57.005 INFO:teuthology.orchestra.run.smithi203.stdout: "op_latency": { 2023-12-01T01:22:57.005 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1388, 2023-12-01T01:22:57.005 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 1.034974328, 2023-12-01T01:22:57.006 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000745658 2023-12-01T01:22:57.006 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:57.006 INFO:teuthology.orchestra.run.smithi203.stdout: "op_inflight": 0, 2023-12-01T01:22:57.006 INFO:teuthology.orchestra.run.smithi203.stdout: "oplen_avg": { 2023-12-01T01:22:57.006 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1388, 2023-12-01T01:22:57.006 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 1389 2023-12-01T01:22:57.007 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:57.007 INFO:teuthology.orchestra.run.smithi203.stdout: "op": 1388, 2023-12-01T01:22:57.007 INFO:teuthology.orchestra.run.smithi203.stdout: "op_r": 1388, 2023-12-01T01:22:57.007 INFO:teuthology.orchestra.run.smithi203.stdout: "op_w": 0, 2023-12-01T01:22:57.007 INFO:teuthology.orchestra.run.smithi203.stdout: "op_rmw": 0, 2023-12-01T01:22:57.007 INFO:teuthology.orchestra.run.smithi203.stdout: "op_pg": 0, 2023-12-01T01:22:57.007 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_stat": 768, 2023-12-01T01:22:57.007 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_create": 0, 2023-12-01T01:22:57.007 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_read": 617, 2023-12-01T01:22:57.007 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_write": 0, 2023-12-01T01:22:57.007 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_writefull": 0, 2023-12-01T01:22:57.007 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_writesame": 0, 2023-12-01T01:22:57.007 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_append": 0, 2023-12-01T01:22:57.007 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_zero": 0, 2023-12-01T01:22:57.007 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_truncate": 0, 2023-12-01T01:22:57.008 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_delete": 0, 2023-12-01T01:22:57.008 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_mapext": 0, 2023-12-01T01:22:57.008 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_sparse_read": 0, 2023-12-01T01:22:57.008 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_clonerange": 0, 2023-12-01T01:22:57.008 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_getxattr": 2, 2023-12-01T01:22:57.008 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_setxattr": 0, 2023-12-01T01:22:57.008 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_cmpxattr": 0, 2023-12-01T01:22:57.008 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_rmxattr": 0, 2023-12-01T01:22:57.008 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_resetxattrs": 0, 2023-12-01T01:22:57.008 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_call": 0, 2023-12-01T01:22:57.008 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_watch": 0, 2023-12-01T01:22:57.008 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_notify": 0, 2023-12-01T01:22:57.008 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_src_cmpxattr": 0, 2023-12-01T01:22:57.008 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_pgls": 0, 2023-12-01T01:22:57.009 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_pgls_filter": 0, 2023-12-01T01:22:57.009 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_other": 0, 2023-12-01T01:22:57.009 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_active": 0, 2023-12-01T01:22:57.009 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_send": 0, 2023-12-01T01:22:57.009 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_resend": 0, 2023-12-01T01:22:57.009 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_ping": 0, 2023-12-01T01:22:57.009 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_active": 0, 2023-12-01T01:22:57.009 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_send": 0, 2023-12-01T01:22:57.009 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_resend": 0, 2023-12-01T01:22:57.009 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_active": 0, 2023-12-01T01:22:57.009 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_send": 0, 2023-12-01T01:22:57.009 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_resend": 0, 2023-12-01T01:22:57.009 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_active": 0, 2023-12-01T01:22:57.009 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_send": 0, 2023-12-01T01:22:57.009 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_resend": 0, 2023-12-01T01:22:57.010 INFO:teuthology.orchestra.run.smithi203.stdout: "command_active": 0, 2023-12-01T01:22:57.010 INFO:teuthology.orchestra.run.smithi203.stdout: "command_send": 0, 2023-12-01T01:22:57.010 INFO:teuthology.orchestra.run.smithi203.stdout: "command_resend": 0, 2023-12-01T01:22:57.010 INFO:teuthology.orchestra.run.smithi203.stdout: "map_epoch": 84, 2023-12-01T01:22:57.010 INFO:teuthology.orchestra.run.smithi203.stdout: "map_full": 0, 2023-12-01T01:22:57.010 INFO:teuthology.orchestra.run.smithi203.stdout: "map_inc": 2, 2023-12-01T01:22:57.010 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_sessions": 10, 2023-12-01T01:22:57.010 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_session_open": 10, 2023-12-01T01:22:57.010 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_session_close": 0, 2023-12-01T01:22:57.010 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_laggy": 0, 2023-12-01T01:22:57.010 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_wr": 0, 2023-12-01T01:22:57.010 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_rd": 2, 2023-12-01T01:22:57.010 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_del": 0 2023-12-01T01:22:57.010 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:57.011 INFO:teuthology.orchestra.run.smithi203.stdout: "oft": { 2023-12-01T01:22:57.011 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_objs": 0, 2023-12-01T01:22:57.011 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_kv_pairs": 0, 2023-12-01T01:22:57.011 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_updates": 0, 2023-12-01T01:22:57.011 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_removes": 0 2023-12-01T01:22:57.011 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:57.011 INFO:teuthology.orchestra.run.smithi203.stdout: "osd-slow-ops": { 2023-12-01T01:22:57.011 INFO:teuthology.orchestra.run.smithi203.stdout: "slow_ops_count": 0 2023-12-01T01:22:57.011 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:57.011 INFO:teuthology.orchestra.run.smithi203.stdout: "purge_queue": { 2023-12-01T01:22:57.011 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_ops": 0, 2023-12-01T01:22:57.011 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_ops_high_water": 0, 2023-12-01T01:22:57.011 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing": 0, 2023-12-01T01:22:57.011 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_high_water": 0, 2023-12-01T01:22:57.011 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executed": 0, 2023-12-01T01:22:57.012 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_item_in_journal": 0 2023-12-01T01:22:57.012 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:57.012 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-01T01:22:57.012 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2023-12-01T01:22:57.012 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2023-12-01T01:22:57.012 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2023-12-01T01:22:57.012 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 1530, 2023-12-01T01:22:57.012 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 88741844, 2023-12-01T01:22:57.012 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:57.012 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 1530, 2023-12-01T01:22:57.012 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2023-12-01T01:22:57.012 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2023-12-01T01:22:57.012 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 1530, 2023-12-01T01:22:57.012 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 88741844, 2023-12-01T01:22:57.013 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2023-12-01T01:22:57.013 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:57.013 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:57.013 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:57.013 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-12-01T01:22:57.013 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:57.013 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-objecter_bytes": { 2023-12-01T01:22:57.013 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2023-12-01T01:22:57.013 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2023-12-01T01:22:57.013 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2023-12-01T01:22:57.013 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2023-12-01T01:22:57.013 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2023-12-01T01:22:57.013 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:57.013 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2023-12-01T01:22:57.013 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 1388, 2023-12-01T01:22:57.014 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 87944797, 2023-12-01T01:22:57.014 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 1388, 2023-12-01T01:22:57.014 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 87944797, 2023-12-01T01:22:57.014 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2023-12-01T01:22:57.014 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:57.014 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:57.014 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:57.014 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-12-01T01:22:57.014 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:57.014 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-objecter_ops": { 2023-12-01T01:22:57.014 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2023-12-01T01:22:57.014 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 1024, 2023-12-01T01:22:57.014 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2023-12-01T01:22:57.014 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2023-12-01T01:22:57.015 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2023-12-01T01:22:57.015 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:57.015 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2023-12-01T01:22:57.015 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 1388, 2023-12-01T01:22:57.015 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 1388, 2023-12-01T01:22:57.015 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 1388, 2023-12-01T01:22:57.015 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 1388, 2023-12-01T01:22:57.015 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2023-12-01T01:22:57.015 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:57.015 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:57.016 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:57.016 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-12-01T01:22:57.016 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:57.016 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-write_buf_throttle": { 2023-12-01T01:22:57.016 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2023-12-01T01:22:57.016 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 3758096384, 2023-12-01T01:22:57.016 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2023-12-01T01:22:57.016 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2023-12-01T01:22:57.016 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2023-12-01T01:22:57.016 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:57.016 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2023-12-01T01:22:57.017 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2023-12-01T01:22:57.017 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2023-12-01T01:22:57.017 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 0, 2023-12-01T01:22:57.017 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 0, 2023-12-01T01:22:57.017 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2023-12-01T01:22:57.017 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:57.017 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:57.017 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:57.017 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-12-01T01:22:57.017 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:57.018 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-write_buf_throttle-0x555b43dd25a0": { 2023-12-01T01:22:57.018 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2023-12-01T01:22:57.018 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 3758096384, 2023-12-01T01:22:57.018 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2023-12-01T01:22:57.018 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2023-12-01T01:22:57.018 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2023-12-01T01:22:57.018 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:57.018 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2023-12-01T01:22:57.018 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2023-12-01T01:22:57.018 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2023-12-01T01:22:57.018 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 0, 2023-12-01T01:22:57.018 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 0, 2023-12-01T01:22:57.018 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2023-12-01T01:22:57.018 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:57.019 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:57.019 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:57.019 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-12-01T01:22:57.019 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-12-01T01:22:57.019 INFO:teuthology.orchestra.run.smithi203.stdout:} 2023-12-01T01:22:57.394 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2023-12-01T01:22:57.394 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2023-12-01T01:22:57.394 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2023-12-01T01:22:57.394 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2023-12-01T01:22:57.394 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2023-12-01T01:22:57.394 DEBUG:tasks.check_counter:Getting stats from i 2023-12-01T01:22:57.395 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph daemon mds.i perf dump 2023-12-01T01:22:57.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:57 smithi203 ceph-mon[103581]: pgmap v450: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 11 op/s 2023-12-01T01:22:57.778 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:57 smithi137 ceph-mon[103312]: pgmap v450: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 11 op/s 2023-12-01T01:22:57.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:57 smithi077 ceph-mon[98867]: pgmap v450: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 11 op/s 2023-12-01T01:22:57.947 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.c/config 2023-12-01T01:22:59.231 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2023-12-01T01:22:59.231 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-0": { 2023-12-01T01:22:59.231 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 184214, 2023-12-01T01:22:59.231 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 189676, 2023-12-01T01:22:59.231 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 34177329, 2023-12-01T01:22:59.231 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 46820992, 2023-12-01T01:22:59.231 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 13, 2023-12-01T01:22:59.231 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 6, 2023-12-01T01:22:59.231 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 5.376745952, 2023-12-01T01:22:59.231 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 2.075489385, 2023-12-01T01:22:59.232 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 3.889485624, 2023-12-01T01:22:59.232 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.036319374, 2023-12-01T01:22:59.232 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2023-12-01T01:22:59.232 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 189675, 2023-12-01T01:22:59.232 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 2.814252161, 2023-12-01T01:22:59.232 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000014837 2023-12-01T01:22:59.232 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.232 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2023-12-01T01:22:59.232 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 368573, 2023-12-01T01:22:59.232 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.031739000, 2023-12-01T01:22:59.232 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000086 2023-12-01T01:22:59.233 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.233 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 2723264, 2023-12-01T01:22:59.233 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 7154944 2023-12-01T01:22:59.233 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.233 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-1": { 2023-12-01T01:22:59.233 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 1389399, 2023-12-01T01:22:59.233 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 1390255, 2023-12-01T01:22:59.233 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 219996665, 2023-12-01T01:22:59.233 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 231877679, 2023-12-01T01:22:59.233 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 15, 2023-12-01T01:22:59.233 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 5, 2023-12-01T01:22:59.233 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 43.335048931, 2023-12-01T01:22:59.233 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 18.522512190, 2023-12-01T01:22:59.233 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 30.113510264, 2023-12-01T01:22:59.233 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.016306754, 2023-12-01T01:22:59.234 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2023-12-01T01:22:59.234 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1390254, 2023-12-01T01:22:59.234 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 13.894465675, 2023-12-01T01:22:59.234 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000009994 2023-12-01T01:22:59.234 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.234 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2023-12-01T01:22:59.234 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 2777667, 2023-12-01T01:22:59.234 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.243621787, 2023-12-01T01:22:59.234 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000087 2023-12-01T01:22:59.234 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.234 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 3152, 2023-12-01T01:22:59.234 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 225392 2023-12-01T01:22:59.234 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.235 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-2": { 2023-12-01T01:22:59.235 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 1716, 2023-12-01T01:22:59.235 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 2109, 2023-12-01T01:22:59.235 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 705645, 2023-12-01T01:22:59.235 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 1166372, 2023-12-01T01:22:59.235 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 14, 2023-12-01T01:22:59.235 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 9, 2023-12-01T01:22:59.235 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.094851897, 2023-12-01T01:22:59.235 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.033021645, 2023-12-01T01:22:59.235 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.046493724, 2023-12-01T01:22:59.235 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.016360457, 2023-12-01T01:22:59.235 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2023-12-01T01:22:59.235 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 2107, 2023-12-01T01:22:59.235 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.158066696, 2023-12-01T01:22:59.236 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000075019 2023-12-01T01:22:59.236 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.236 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2023-12-01T01:22:59.236 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 2164, 2023-12-01T01:22:59.236 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000354907, 2023-12-01T01:22:59.236 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000164 2023-12-01T01:22:59.236 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.236 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 1920, 2023-12-01T01:22:59.236 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 1536 2023-12-01T01:22:59.236 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.236 INFO:teuthology.orchestra.run.smithi203.stdout: "cct": { 2023-12-01T01:22:59.236 INFO:teuthology.orchestra.run.smithi203.stdout: "total_workers": 1, 2023-12-01T01:22:59.236 INFO:teuthology.orchestra.run.smithi203.stdout: "unhealthy_workers": 0 2023-12-01T01:22:59.237 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.237 INFO:teuthology.orchestra.run.smithi203.stdout: "finisher-MDSRank": { 2023-12-01T01:22:59.237 INFO:teuthology.orchestra.run.smithi203.stdout: "queue_len": 0, 2023-12-01T01:22:59.237 INFO:teuthology.orchestra.run.smithi203.stdout: "complete_latency": { 2023-12-01T01:22:59.237 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1086, 2023-12-01T01:22:59.237 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 2.269017144, 2023-12-01T01:22:59.237 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.002089334 2023-12-01T01:22:59.237 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-12-01T01:22:59.237 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.238 INFO:teuthology.orchestra.run.smithi203.stdout: "finisher-PurgeQueue": { 2023-12-01T01:22:59.238 INFO:teuthology.orchestra.run.smithi203.stdout: "queue_len": 0, 2023-12-01T01:22:59.238 INFO:teuthology.orchestra.run.smithi203.stdout: "complete_latency": { 2023-12-01T01:22:59.238 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 2879, 2023-12-01T01:22:59.238 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.434007553, 2023-12-01T01:22:59.238 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000150749 2023-12-01T01:22:59.238 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-12-01T01:22:59.239 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.239 INFO:teuthology.orchestra.run.smithi203.stdout: "mds": { 2023-12-01T01:22:59.239 INFO:teuthology.orchestra.run.smithi203.stdout: "request": 4770, 2023-12-01T01:22:59.239 INFO:teuthology.orchestra.run.smithi203.stdout: "reply": 4653, 2023-12-01T01:22:59.241 INFO:teuthology.orchestra.run.smithi203.stdout: "reply_latency": { 2023-12-01T01:22:59.241 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 4653, 2023-12-01T01:22:59.241 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 11.826600790, 2023-12-01T01:22:59.241 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.002541715 2023-12-01T01:22:59.241 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.241 INFO:teuthology.orchestra.run.smithi203.stdout: "slow_reply": 0, 2023-12-01T01:22:59.241 INFO:teuthology.orchestra.run.smithi203.stdout: "forward": 117, 2023-12-01T01:22:59.241 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_fetch_complete": 7, 2023-12-01T01:22:59.241 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_fetch_keys": 0, 2023-12-01T01:22:59.241 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_commit": 11, 2023-12-01T01:22:59.241 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_split": 0, 2023-12-01T01:22:59.241 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_merge": 0, 2023-12-01T01:22:59.242 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes": 220, 2023-12-01T01:22:59.242 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_top": 140, 2023-12-01T01:22:59.242 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_bottom": 63, 2023-12-01T01:22:59.242 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_pin_tail": 17, 2023-12-01T01:22:59.242 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_pinned": 20, 2023-12-01T01:22:59.242 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_expired": 82, 2023-12-01T01:22:59.242 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_with_caps": 0, 2023-12-01T01:22:59.242 INFO:teuthology.orchestra.run.smithi203.stdout: "caps": 0, 2023-12-01T01:22:59.242 INFO:teuthology.orchestra.run.smithi203.stdout: "subtrees": 5, 2023-12-01T01:22:59.242 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse": 7933, 2023-12-01T01:22:59.242 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_hit": 6106, 2023-12-01T01:22:59.242 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_forward": 51, 2023-12-01T01:22:59.242 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_discover": 29, 2023-12-01T01:22:59.242 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_dir_fetch": 0, 2023-12-01T01:22:59.242 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_remote_ino": 0, 2023-12-01T01:22:59.243 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_lock": 4, 2023-12-01T01:22:59.243 INFO:teuthology.orchestra.run.smithi203.stdout: "load_cent": 1000, 2023-12-01T01:22:59.243 INFO:teuthology.orchestra.run.smithi203.stdout: "q": 1, 2023-12-01T01:22:59.243 INFO:teuthology.orchestra.run.smithi203.stdout: "exported": 22, 2023-12-01T01:22:59.243 INFO:teuthology.orchestra.run.smithi203.stdout: "exported_inodes": 17, 2023-12-01T01:22:59.243 INFO:teuthology.orchestra.run.smithi203.stdout: "imported": 165, 2023-12-01T01:22:59.243 INFO:teuthology.orchestra.run.smithi203.stdout: "imported_inodes": 1892, 2023-12-01T01:22:59.243 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_dir_fetch": 0, 2023-12-01T01:22:59.243 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_backtrace_fetch": 374, 2023-12-01T01:22:59.243 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_peer_discover": 0, 2023-12-01T01:22:59.243 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rfiles": 3568, 2023-12-01T01:22:59.243 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rbytes": 50854799, 2023-12-01T01:22:59.243 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rsnaps": 0, 2023-12-01T01:22:59.243 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_backtrace_fetch": 374, 2023-12-01T01:22:59.243 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_set_tag": 0, 2023-12-01T01:22:59.244 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_backtrace_repaired": 0, 2023-12-01T01:22:59.244 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_inotable_repaired": 0, 2023-12-01T01:22:59.244 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dir_inodes": 18, 2023-12-01T01:22:59.244 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dir_base_inodes": 0, 2023-12-01T01:22:59.244 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dirfrag_rstats": 18, 2023-12-01T01:22:59.244 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_file_inodes": 356, 2023-12-01T01:22:59.244 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_inode_file_caps": 18, 2023-12-01T01:22:59.244 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_revoke": 170, 2023-12-01T01:22:59.244 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_grant": 2365, 2023-12-01T01:22:59.244 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_trunc": 4, 2023-12-01T01:22:59.244 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-01T01:22:59.244 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-01T01:22:59.244 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_caps": 1271, 2023-12-01T01:22:59.244 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_caps_dirty": 1087, 2023-12-01T01:22:59.245 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_cap_release": 158, 2023-12-01T01:22:59.245 INFO:teuthology.orchestra.run.smithi203.stdout: "process_request_cap_release": 3313 2023-12-01T01:22:59.245 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.245 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_cache": { 2023-12-01T01:22:59.245 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_update": 363, 2023-12-01T01:22:59.245 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_update_receipt": 155, 2023-12-01T01:22:59.245 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_try_discover": 82, 2023-12-01T01:22:59.245 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_send_discover": 286, 2023-12-01T01:22:59.245 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_handle_discover": 811, 2023-12-01T01:22:59.245 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays": 5, 2023-12-01T01:22:59.245 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays_delayed": 0, 2023-12-01T01:22:59.245 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays_enqueuing": 0, 2023-12-01T01:22:59.245 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_created": 1592, 2023-12-01T01:22:59.245 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_enqueued": 1587, 2023-12-01T01:22:59.245 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_reintegrated": 0, 2023-12-01T01:22:59.246 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_migrated": 0, 2023-12-01T01:22:59.246 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_processing": 0, 2023-12-01T01:22:59.246 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_enqueued": 0, 2023-12-01T01:22:59.246 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_prioritized": 0, 2023-12-01T01:22:59.246 INFO:teuthology.orchestra.run.smithi203.stdout: "recovery_started": 0, 2023-12-01T01:22:59.246 INFO:teuthology.orchestra.run.smithi203.stdout: "recovery_completed": 0, 2023-12-01T01:22:59.246 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_enqueue_scrub": 0, 2023-12-01T01:22:59.246 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_exportdir": 22, 2023-12-01T01:22:59.246 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_flush": 0, 2023-12-01T01:22:59.246 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_fragmentdir": 0, 2023-12-01T01:22:59.246 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_fragstats": 0, 2023-12-01T01:22:59.246 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_inodestats": 0 2023-12-01T01:22:59.246 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.246 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_log": { 2023-12-01T01:22:59.246 INFO:teuthology.orchestra.run.smithi203.stdout: "evadd": 6562, 2023-12-01T01:22:59.247 INFO:teuthology.orchestra.run.smithi203.stdout: "evex": 0, 2023-12-01T01:22:59.247 INFO:teuthology.orchestra.run.smithi203.stdout: "evtrm": 0, 2023-12-01T01:22:59.247 INFO:teuthology.orchestra.run.smithi203.stdout: "ev": 6562, 2023-12-01T01:22:59.247 INFO:teuthology.orchestra.run.smithi203.stdout: "evexg": 0, 2023-12-01T01:22:59.247 INFO:teuthology.orchestra.run.smithi203.stdout: "evexd": 0, 2023-12-01T01:22:59.247 INFO:teuthology.orchestra.run.smithi203.stdout: "segadd": 8, 2023-12-01T01:22:59.247 INFO:teuthology.orchestra.run.smithi203.stdout: "segex": 0, 2023-12-01T01:22:59.247 INFO:teuthology.orchestra.run.smithi203.stdout: "segtrm": 0, 2023-12-01T01:22:59.247 INFO:teuthology.orchestra.run.smithi203.stdout: "seg": 8, 2023-12-01T01:22:59.247 INFO:teuthology.orchestra.run.smithi203.stdout: "segexg": 0, 2023-12-01T01:22:59.247 INFO:teuthology.orchestra.run.smithi203.stdout: "segexd": 0, 2023-12-01T01:22:59.247 INFO:teuthology.orchestra.run.smithi203.stdout: "expos": 4194304, 2023-12-01T01:22:59.247 INFO:teuthology.orchestra.run.smithi203.stdout: "wrpos": 14875204, 2023-12-01T01:22:59.247 INFO:teuthology.orchestra.run.smithi203.stdout: "rdpos": 0, 2023-12-01T01:22:59.247 INFO:teuthology.orchestra.run.smithi203.stdout: "jlat": { 2023-12-01T01:22:59.248 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 506, 2023-12-01T01:22:59.248 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 4.770646563, 2023-12-01T01:22:59.248 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.009428155 2023-12-01T01:22:59.248 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.248 INFO:teuthology.orchestra.run.smithi203.stdout: "replayed": 0 2023-12-01T01:22:59.248 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.248 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_mem": { 2023-12-01T01:22:59.248 INFO:teuthology.orchestra.run.smithi203.stdout: "ino": 28, 2023-12-01T01:22:59.248 INFO:teuthology.orchestra.run.smithi203.stdout: "ino+": 1644, 2023-12-01T01:22:59.248 INFO:teuthology.orchestra.run.smithi203.stdout: "ino-": 1616, 2023-12-01T01:22:59.249 INFO:teuthology.orchestra.run.smithi203.stdout: "dir": 90, 2023-12-01T01:22:59.249 INFO:teuthology.orchestra.run.smithi203.stdout: "dir+": 255, 2023-12-01T01:22:59.249 INFO:teuthology.orchestra.run.smithi203.stdout: "dir-": 165, 2023-12-01T01:22:59.249 INFO:teuthology.orchestra.run.smithi203.stdout: "dn": 220, 2023-12-01T01:22:59.249 INFO:teuthology.orchestra.run.smithi203.stdout: "dn+": 4139, 2023-12-01T01:22:59.249 INFO:teuthology.orchestra.run.smithi203.stdout: "dn-": 3919, 2023-12-01T01:22:59.249 INFO:teuthology.orchestra.run.smithi203.stdout: "cap": 0, 2023-12-01T01:22:59.249 INFO:teuthology.orchestra.run.smithi203.stdout: "cap+": 2371, 2023-12-01T01:22:59.249 INFO:teuthology.orchestra.run.smithi203.stdout: "cap-": 2371, 2023-12-01T01:22:59.249 INFO:teuthology.orchestra.run.smithi203.stdout: "rss": 67040, 2023-12-01T01:22:59.249 INFO:teuthology.orchestra.run.smithi203.stdout: "heap": 198964 2023-12-01T01:22:59.249 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.249 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_server": { 2023-12-01T01:22:59.249 INFO:teuthology.orchestra.run.smithi203.stdout: "dispatch_client_request": 5591, 2023-12-01T01:22:59.249 INFO:teuthology.orchestra.run.smithi203.stdout: "dispatch_server_request": 62, 2023-12-01T01:22:59.250 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_request": 4770, 2023-12-01T01:22:59.250 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_session": 9, 2023-12-01T01:22:59.250 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_peer_request": 353, 2023-12-01T01:22:59.250 INFO:teuthology.orchestra.run.smithi203.stdout: "req_create_latency": { 2023-12-01T01:22:59.250 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 352, 2023-12-01T01:22:59.250 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.553414787, 2023-12-01T01:22:59.250 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.001572201 2023-12-01T01:22:59.250 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.250 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getattr_latency": { 2023-12-01T01:22:59.250 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 6, 2023-12-01T01:22:59.250 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.009568944, 2023-12-01T01:22:59.250 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.001594824 2023-12-01T01:22:59.250 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.250 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getfilelock_latency": { 2023-12-01T01:22:59.251 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:59.251 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:59.251 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:59.251 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.251 INFO:teuthology.orchestra.run.smithi203.stdout: "req_link_latency": { 2023-12-01T01:22:59.251 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:59.251 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:59.251 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:59.251 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.251 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookup_latency": { 2023-12-01T01:22:59.251 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 2452, 2023-12-01T01:22:59.251 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 1.006770484, 2023-12-01T01:22:59.251 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000410591 2023-12-01T01:22:59.251 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.251 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookuphash_latency": { 2023-12-01T01:22:59.252 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:59.252 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:59.252 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:59.252 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.252 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupino_latency": { 2023-12-01T01:22:59.252 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:59.252 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:59.252 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:59.252 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.252 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupname_latency": { 2023-12-01T01:22:59.252 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:59.252 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:59.252 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:59.252 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.252 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupparent_latency": { 2023-12-01T01:22:59.253 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:59.253 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:59.253 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:59.253 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.253 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupsnap_latency": { 2023-12-01T01:22:59.253 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:59.253 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:59.253 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:59.253 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.253 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lssnap_latency": { 2023-12-01T01:22:59.253 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:59.253 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:59.253 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:59.253 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.254 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mkdir_latency": { 2023-12-01T01:22:59.254 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 5, 2023-12-01T01:22:59.254 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.006987916, 2023-12-01T01:22:59.254 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.001397583 2023-12-01T01:22:59.254 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.254 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mknod_latency": { 2023-12-01T01:22:59.254 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:59.254 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:59.254 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:59.254 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.254 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mksnap_latency": { 2023-12-01T01:22:59.254 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:59.254 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:59.254 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:59.255 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.255 INFO:teuthology.orchestra.run.smithi203.stdout: "req_open_latency": { 2023-12-01T01:22:59.255 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 106, 2023-12-01T01:22:59.255 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.026059110, 2023-12-01T01:22:59.255 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000245840 2023-12-01T01:22:59.255 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.255 INFO:teuthology.orchestra.run.smithi203.stdout: "req_readdir_latency": { 2023-12-01T01:22:59.255 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 113, 2023-12-01T01:22:59.255 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.121002323, 2023-12-01T01:22:59.255 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.001070817 2023-12-01T01:22:59.255 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.255 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rename_latency": { 2023-12-01T01:22:59.255 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 20, 2023-12-01T01:22:59.255 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 6.350990538, 2023-12-01T01:22:59.256 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.317549526 2023-12-01T01:22:59.256 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.256 INFO:teuthology.orchestra.run.smithi203.stdout: "req_renamesnap_latency": { 2023-12-01T01:22:59.256 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:59.256 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:59.256 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:59.256 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.256 INFO:teuthology.orchestra.run.smithi203.stdout: "req_snapdiff_latency": { 2023-12-01T01:22:59.256 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:59.256 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:59.256 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:59.256 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.257 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmdir_latency": { 2023-12-01T01:22:59.257 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 52, 2023-12-01T01:22:59.257 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 1.621211549, 2023-12-01T01:22:59.257 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.031177145 2023-12-01T01:22:59.257 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.257 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmsnap_latency": { 2023-12-01T01:22:59.257 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:59.257 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:59.257 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:59.257 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.257 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmxattr_latency": { 2023-12-01T01:22:59.257 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:59.257 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:59.257 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:59.258 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.258 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setattr_latency": { 2023-12-01T01:22:59.258 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 4, 2023-12-01T01:22:59.258 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.012842007, 2023-12-01T01:22:59.258 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.003210501 2023-12-01T01:22:59.258 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.258 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setdirlayout_latency": { 2023-12-01T01:22:59.258 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:59.258 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:59.258 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:59.258 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.258 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setfilelock_latency": { 2023-12-01T01:22:59.258 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 4, 2023-12-01T01:22:59.258 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.001496959, 2023-12-01T01:22:59.258 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000374239 2023-12-01T01:22:59.259 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.259 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setlayout_latency": { 2023-12-01T01:22:59.259 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:59.259 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:59.259 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:59.259 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.259 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setxattr_latency": { 2023-12-01T01:22:59.259 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:59.259 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:59.259 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:59.259 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.259 INFO:teuthology.orchestra.run.smithi203.stdout: "req_symlink_latency": { 2023-12-01T01:22:59.259 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1, 2023-12-01T01:22:59.259 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000760182, 2023-12-01T01:22:59.259 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000760182 2023-12-01T01:22:59.260 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.260 INFO:teuthology.orchestra.run.smithi203.stdout: "req_unlink_latency": { 2023-12-01T01:22:59.260 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1538, 2023-12-01T01:22:59.260 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 2.115495991, 2023-12-01T01:22:59.260 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.001375485 2023-12-01T01:22:59.260 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.260 INFO:teuthology.orchestra.run.smithi203.stdout: "cap_revoke_eviction": 0, 2023-12-01T01:22:59.260 INFO:teuthology.orchestra.run.smithi203.stdout: "cap_acquisition_throttle": 0, 2023-12-01T01:22:59.260 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getvxattr_latency": { 2023-12-01T01:22:59.260 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:59.260 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:59.260 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:59.260 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-12-01T01:22:59.260 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.261 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_sessions": { 2023-12-01T01:22:59.261 INFO:teuthology.orchestra.run.smithi203.stdout: "session_count": 1, 2023-12-01T01:22:59.261 INFO:teuthology.orchestra.run.smithi203.stdout: "session_add": 2, 2023-12-01T01:22:59.261 INFO:teuthology.orchestra.run.smithi203.stdout: "session_remove": 1, 2023-12-01T01:22:59.261 INFO:teuthology.orchestra.run.smithi203.stdout: "sessions_open": 1, 2023-12-01T01:22:59.261 INFO:teuthology.orchestra.run.smithi203.stdout: "sessions_stale": 0, 2023-12-01T01:22:59.261 INFO:teuthology.orchestra.run.smithi203.stdout: "total_load": 2686, 2023-12-01T01:22:59.261 INFO:teuthology.orchestra.run.smithi203.stdout: "average_load": 2686, 2023-12-01T01:22:59.261 INFO:teuthology.orchestra.run.smithi203.stdout: "avg_session_uptime": 259, 2023-12-01T01:22:59.261 INFO:teuthology.orchestra.run.smithi203.stdout: "mdthresh_evicted": 0 2023-12-01T01:22:59.261 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.261 INFO:teuthology.orchestra.run.smithi203.stdout: "mempool": { 2023-12-01T01:22:59.261 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_bytes": 120, 2023-12-01T01:22:59.261 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_items": 120, 2023-12-01T01:22:59.261 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_bytes": 0, 2023-12-01T01:22:59.262 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_items": 0, 2023-12-01T01:22:59.262 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_bytes": 0, 2023-12-01T01:22:59.262 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_items": 0, 2023-12-01T01:22:59.262 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-01T01:22:59.262 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_items": 0, 2023-12-01T01:22:59.262 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-01T01:22:59.262 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_items": 0, 2023-12-01T01:22:59.262 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_bytes": 0, 2023-12-01T01:22:59.262 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_items": 0, 2023-12-01T01:22:59.262 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-01T01:22:59.262 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_items": 0, 2023-12-01T01:22:59.262 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_bytes": 0, 2023-12-01T01:22:59.262 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_items": 0, 2023-12-01T01:22:59.262 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_bytes": 0, 2023-12-01T01:22:59.262 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_items": 0, 2023-12-01T01:22:59.263 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-01T01:22:59.263 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_items": 0, 2023-12-01T01:22:59.263 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-01T01:22:59.263 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_items": 0, 2023-12-01T01:22:59.263 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_bytes": 0, 2023-12-01T01:22:59.263 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_items": 0, 2023-12-01T01:22:59.263 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_bytes": 0, 2023-12-01T01:22:59.263 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_items": 0, 2023-12-01T01:22:59.263 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-01T01:22:59.263 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_items": 0, 2023-12-01T01:22:59.263 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_bytes": 0, 2023-12-01T01:22:59.263 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_items": 0, 2023-12-01T01:22:59.263 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_bytes": 0, 2023-12-01T01:22:59.263 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_items": 0, 2023-12-01T01:22:59.263 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_bytes": 0, 2023-12-01T01:22:59.264 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_items": 0, 2023-12-01T01:22:59.264 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_bytes": 0, 2023-12-01T01:22:59.264 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_items": 0, 2023-12-01T01:22:59.264 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_bytes": 289995, 2023-12-01T01:22:59.264 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_items": 128, 2023-12-01T01:22:59.264 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_bytes": 968, 2023-12-01T01:22:59.264 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_items": 11, 2023-12-01T01:22:59.264 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_bytes": 0, 2023-12-01T01:22:59.264 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_items": 0, 2023-12-01T01:22:59.264 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_bytes": 0, 2023-12-01T01:22:59.264 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_items": 0, 2023-12-01T01:22:59.264 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_bytes": 0, 2023-12-01T01:22:59.264 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_items": 0, 2023-12-01T01:22:59.264 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_bytes": 7336, 2023-12-01T01:22:59.264 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_items": 154, 2023-12-01T01:22:59.265 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_bytes": 0, 2023-12-01T01:22:59.265 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_items": 0, 2023-12-01T01:22:59.265 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_bytes": 0, 2023-12-01T01:22:59.265 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_items": 0, 2023-12-01T01:22:59.265 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_bytes": 347357, 2023-12-01T01:22:59.265 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_items": 2942, 2023-12-01T01:22:59.265 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_bytes": 0, 2023-12-01T01:22:59.265 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_items": 0, 2023-12-01T01:22:59.265 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_bytes": 0, 2023-12-01T01:22:59.265 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_items": 0 2023-12-01T01:22:59.265 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.265 INFO:teuthology.orchestra.run.smithi203.stdout: "objecter": { 2023-12-01T01:22:59.265 INFO:teuthology.orchestra.run.smithi203.stdout: "op_active": 0, 2023-12-01T01:22:59.265 INFO:teuthology.orchestra.run.smithi203.stdout: "op_laggy": 0, 2023-12-01T01:22:59.265 INFO:teuthology.orchestra.run.smithi203.stdout: "op_send": 2764, 2023-12-01T01:22:59.266 INFO:teuthology.orchestra.run.smithi203.stdout: "op_send_bytes": 10944423, 2023-12-01T01:22:59.266 INFO:teuthology.orchestra.run.smithi203.stdout: "op_resend": 27, 2023-12-01T01:22:59.266 INFO:teuthology.orchestra.run.smithi203.stdout: "op_reply": 2764, 2023-12-01T01:22:59.266 INFO:teuthology.orchestra.run.smithi203.stdout: "op_latency": { 2023-12-01T01:22:59.266 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 2764, 2023-12-01T01:22:59.266 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 7.896803943, 2023-12-01T01:22:59.266 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.002857020 2023-12-01T01:22:59.266 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.266 INFO:teuthology.orchestra.run.smithi203.stdout: "op_inflight": 0, 2023-12-01T01:22:59.266 INFO:teuthology.orchestra.run.smithi203.stdout: "oplen_avg": { 2023-12-01T01:22:59.266 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 2764, 2023-12-01T01:22:59.266 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 2826 2023-12-01T01:22:59.266 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.266 INFO:teuthology.orchestra.run.smithi203.stdout: "op": 2764, 2023-12-01T01:22:59.266 INFO:teuthology.orchestra.run.smithi203.stdout: "op_r": 431, 2023-12-01T01:22:59.267 INFO:teuthology.orchestra.run.smithi203.stdout: "op_w": 2333, 2023-12-01T01:22:59.267 INFO:teuthology.orchestra.run.smithi203.stdout: "op_rmw": 0, 2023-12-01T01:22:59.267 INFO:teuthology.orchestra.run.smithi203.stdout: "op_pg": 0, 2023-12-01T01:22:59.267 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_stat": 0, 2023-12-01T01:22:59.267 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_create": 10, 2023-12-01T01:22:59.267 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_read": 50, 2023-12-01T01:22:59.267 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_write": 556, 2023-12-01T01:22:59.267 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_writefull": 37, 2023-12-01T01:22:59.267 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_writesame": 0, 2023-12-01T01:22:59.267 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_append": 0, 2023-12-01T01:22:59.267 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_zero": 0, 2023-12-01T01:22:59.267 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_truncate": 0, 2023-12-01T01:22:59.267 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_delete": 1691, 2023-12-01T01:22:59.267 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_mapext": 0, 2023-12-01T01:22:59.267 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_sparse_read": 0, 2023-12-01T01:22:59.268 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_clonerange": 0, 2023-12-01T01:22:59.268 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_getxattr": 375, 2023-12-01T01:22:59.268 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_setxattr": 20, 2023-12-01T01:22:59.268 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_cmpxattr": 0, 2023-12-01T01:22:59.268 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_rmxattr": 0, 2023-12-01T01:22:59.268 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_resetxattrs": 0, 2023-12-01T01:22:59.268 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_call": 0, 2023-12-01T01:22:59.268 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_watch": 0, 2023-12-01T01:22:59.268 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_notify": 0, 2023-12-01T01:22:59.268 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_src_cmpxattr": 0, 2023-12-01T01:22:59.269 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_pgls": 0, 2023-12-01T01:22:59.269 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_pgls_filter": 0, 2023-12-01T01:22:59.269 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_other": 4, 2023-12-01T01:22:59.269 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_active": 0, 2023-12-01T01:22:59.269 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_send": 0, 2023-12-01T01:22:59.269 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_resend": 0, 2023-12-01T01:22:59.269 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_ping": 0, 2023-12-01T01:22:59.269 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_active": 0, 2023-12-01T01:22:59.269 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_send": 0, 2023-12-01T01:22:59.269 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_resend": 0, 2023-12-01T01:22:59.269 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_active": 0, 2023-12-01T01:22:59.269 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_send": 0, 2023-12-01T01:22:59.269 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_resend": 0, 2023-12-01T01:22:59.269 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_active": 0, 2023-12-01T01:22:59.269 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_send": 0, 2023-12-01T01:22:59.270 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_resend": 0, 2023-12-01T01:22:59.270 INFO:teuthology.orchestra.run.smithi203.stdout: "command_active": 0, 2023-12-01T01:22:59.270 INFO:teuthology.orchestra.run.smithi203.stdout: "command_send": 0, 2023-12-01T01:22:59.270 INFO:teuthology.orchestra.run.smithi203.stdout: "command_resend": 0, 2023-12-01T01:22:59.270 INFO:teuthology.orchestra.run.smithi203.stdout: "map_epoch": 84, 2023-12-01T01:22:59.270 INFO:teuthology.orchestra.run.smithi203.stdout: "map_full": 0, 2023-12-01T01:22:59.270 INFO:teuthology.orchestra.run.smithi203.stdout: "map_inc": 2, 2023-12-01T01:22:59.270 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_sessions": 12, 2023-12-01T01:22:59.270 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_session_open": 12, 2023-12-01T01:22:59.270 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_session_close": 0, 2023-12-01T01:22:59.270 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_laggy": 0, 2023-12-01T01:22:59.270 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_wr": 52, 2023-12-01T01:22:59.270 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_rd": 14, 2023-12-01T01:22:59.270 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_del": 17 2023-12-01T01:22:59.270 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.271 INFO:teuthology.orchestra.run.smithi203.stdout: "oft": { 2023-12-01T01:22:59.271 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_objs": 1, 2023-12-01T01:22:59.271 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_kv_pairs": 0, 2023-12-01T01:22:59.271 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_updates": 0, 2023-12-01T01:22:59.271 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_removes": 0 2023-12-01T01:22:59.271 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.271 INFO:teuthology.orchestra.run.smithi203.stdout: "osd-slow-ops": { 2023-12-01T01:22:59.271 INFO:teuthology.orchestra.run.smithi203.stdout: "slow_ops_count": 4856 2023-12-01T01:22:59.271 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.271 INFO:teuthology.orchestra.run.smithi203.stdout: "purge_queue": { 2023-12-01T01:22:59.271 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_ops": 0, 2023-12-01T01:22:59.271 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_ops_high_water": 65, 2023-12-01T01:22:59.271 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing": 0, 2023-12-01T01:22:59.272 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_high_water": 6, 2023-12-01T01:22:59.272 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executed": 1587, 2023-12-01T01:22:59.272 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_item_in_journal": 0 2023-12-01T01:22:59.272 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.272 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-01T01:22:59.272 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2023-12-01T01:22:59.272 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2023-12-01T01:22:59.272 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2023-12-01T01:22:59.272 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 1575325, 2023-12-01T01:22:59.272 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 112812550, 2023-12-01T01:22:59.272 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:59.272 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 1575325, 2023-12-01T01:22:59.272 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2023-12-01T01:22:59.272 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2023-12-01T01:22:59.273 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 1575325, 2023-12-01T01:22:59.273 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 112812550, 2023-12-01T01:22:59.273 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2023-12-01T01:22:59.273 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:59.273 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:59.273 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:59.273 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-12-01T01:22:59.273 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.273 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-objecter_bytes": { 2023-12-01T01:22:59.273 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2023-12-01T01:22:59.273 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2023-12-01T01:22:59.273 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2023-12-01T01:22:59.273 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2023-12-01T01:22:59.273 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2023-12-01T01:22:59.273 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:59.273 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2023-12-01T01:22:59.274 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 2764, 2023-12-01T01:22:59.274 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 11104630, 2023-12-01T01:22:59.274 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 2764, 2023-12-01T01:22:59.274 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 11104630, 2023-12-01T01:22:59.274 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2023-12-01T01:22:59.274 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:59.274 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:59.274 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:59.274 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-12-01T01:22:59.274 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.274 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-objecter_ops": { 2023-12-01T01:22:59.274 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2023-12-01T01:22:59.274 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 1024, 2023-12-01T01:22:59.274 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2023-12-01T01:22:59.275 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2023-12-01T01:22:59.275 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2023-12-01T01:22:59.275 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:59.275 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2023-12-01T01:22:59.275 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 2764, 2023-12-01T01:22:59.275 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 2764, 2023-12-01T01:22:59.275 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 2764, 2023-12-01T01:22:59.275 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 2764, 2023-12-01T01:22:59.275 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2023-12-01T01:22:59.275 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:59.275 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:59.275 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:59.275 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-12-01T01:22:59.275 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.275 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-write_buf_throttle": { 2023-12-01T01:22:59.276 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2023-12-01T01:22:59.276 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 3758096384, 2023-12-01T01:22:59.276 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2023-12-01T01:22:59.276 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 1587, 2023-12-01T01:22:59.276 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 160319, 2023-12-01T01:22:59.276 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:59.276 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 1587, 2023-12-01T01:22:59.276 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2023-12-01T01:22:59.276 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2023-12-01T01:22:59.276 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 50, 2023-12-01T01:22:59.276 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 160319, 2023-12-01T01:22:59.276 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2023-12-01T01:22:59.276 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:59.276 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:59.276 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:59.277 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-12-01T01:22:59.277 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:22:59.277 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-write_buf_throttle-0x563635cd20a0": { 2023-12-01T01:22:59.277 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2023-12-01T01:22:59.277 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 3758096384, 2023-12-01T01:22:59.277 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2023-12-01T01:22:59.277 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 6562, 2023-12-01T01:22:59.277 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 10680900, 2023-12-01T01:22:59.277 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2023-12-01T01:22:59.277 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 6562, 2023-12-01T01:22:59.277 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2023-12-01T01:22:59.277 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2023-12-01T01:22:59.277 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 506, 2023-12-01T01:22:59.277 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 10680900, 2023-12-01T01:22:59.277 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2023-12-01T01:22:59.278 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:22:59.278 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:22:59.278 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:22:59.278 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-12-01T01:22:59.278 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-12-01T01:22:59.278 INFO:teuthology.orchestra.run.smithi203.stdout:} 2023-12-01T01:22:59.604 INFO:tasks.check_counter:Daemon mds.i mds.exported=22 2023-12-01T01:22:59.604 INFO:tasks.check_counter:Daemon mds.i mds.imported=165 2023-12-01T01:22:59.604 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=363 2023-12-01T01:22:59.604 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=155 2023-12-01T01:22:59.604 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2023-12-01T01:22:59.604 DEBUG:tasks.check_counter:Getting stats from l 2023-12-01T01:22:59.604 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a6f82997ac57b70a895455dfed6256360a1e4c32 shell --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 -- ceph daemon mds.l perf dump 2023-12-01T01:22:59.630 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:22:59 smithi203 ceph-mon[103581]: pgmap v451: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-12-01T01:22:59.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:22:59 smithi137 ceph-mon[103312]: pgmap v451: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-12-01T01:22:59.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:22:59 smithi077 ceph-mon[98867]: pgmap v451: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-12-01T01:23:00.159 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/mon.c/config 2023-12-01T01:23:01.504 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2023-12-01T01:23:01.504 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-0": { 2023-12-01T01:23:01.504 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 425, 2023-12-01T01:23:01.504 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 425, 2023-12-01T01:23:01.505 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 4307868, 2023-12-01T01:23:01.505 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 131443, 2023-12-01T01:23:01.505 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 6, 2023-12-01T01:23:01.505 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 2, 2023-12-01T01:23:01.505 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.038976870, 2023-12-01T01:23:01.505 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.009139834, 2023-12-01T01:23:01.505 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.013301391, 2023-12-01T01:23:01.505 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.012628719, 2023-12-01T01:23:01.505 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2023-12-01T01:23:01.505 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 424, 2023-12-01T01:23:01.505 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.017600654, 2023-12-01T01:23:01.505 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000041510 2023-12-01T01:23:01.505 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.505 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2023-12-01T01:23:01.506 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:23:01.506 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:23:01.506 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:23:01.506 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.506 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-12-01T01:23:01.506 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 448 2023-12-01T01:23:01.506 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.506 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-1": { 2023-12-01T01:23:01.507 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 274, 2023-12-01T01:23:01.507 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 362, 2023-12-01T01:23:01.507 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 2320359, 2023-12-01T01:23:01.507 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 307740, 2023-12-01T01:23:01.507 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 5, 2023-12-01T01:23:01.507 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 4, 2023-12-01T01:23:01.507 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.026764107, 2023-12-01T01:23:01.507 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.007887316, 2023-12-01T01:23:01.507 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.007558966, 2023-12-01T01:23:01.507 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.008864794, 2023-12-01T01:23:01.507 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2023-12-01T01:23:01.507 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 360, 2023-12-01T01:23:01.507 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.011991382, 2023-12-01T01:23:01.507 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000033309 2023-12-01T01:23:01.508 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.508 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2023-12-01T01:23:01.508 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:23:01.508 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:23:01.508 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:23:01.508 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.508 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 160, 2023-12-01T01:23:01.508 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 222832 2023-12-01T01:23:01.509 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.509 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-2": { 2023-12-01T01:23:01.509 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 533, 2023-12-01T01:23:01.509 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 513, 2023-12-01T01:23:01.509 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 688480, 2023-12-01T01:23:01.509 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 188083, 2023-12-01T01:23:01.509 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 3, 2023-12-01T01:23:01.509 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 2, 2023-12-01T01:23:01.509 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.057731872, 2023-12-01T01:23:01.509 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.016938215, 2023-12-01T01:23:01.509 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.021919231, 2023-12-01T01:23:01.509 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.017555824, 2023-12-01T01:23:01.509 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2023-12-01T01:23:01.509 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 512, 2023-12-01T01:23:01.510 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.021805669, 2023-12-01T01:23:01.510 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000042589 2023-12-01T01:23:01.510 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.510 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2023-12-01T01:23:01.510 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:23:01.510 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:23:01.510 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:23:01.510 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.510 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 557776, 2023-12-01T01:23:01.510 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 67104 2023-12-01T01:23:01.510 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.510 INFO:teuthology.orchestra.run.smithi203.stdout: "cct": { 2023-12-01T01:23:01.510 INFO:teuthology.orchestra.run.smithi203.stdout: "total_workers": 1, 2023-12-01T01:23:01.510 INFO:teuthology.orchestra.run.smithi203.stdout: "unhealthy_workers": 0 2023-12-01T01:23:01.510 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.511 INFO:teuthology.orchestra.run.smithi203.stdout: "finisher-MDSRank": { 2023-12-01T01:23:01.511 INFO:teuthology.orchestra.run.smithi203.stdout: "queue_len": 0, 2023-12-01T01:23:01.511 INFO:teuthology.orchestra.run.smithi203.stdout: "complete_latency": { 2023-12-01T01:23:01.511 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1872, 2023-12-01T01:23:01.511 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.055050851, 2023-12-01T01:23:01.511 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000029407 2023-12-01T01:23:01.511 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-12-01T01:23:01.511 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.511 INFO:teuthology.orchestra.run.smithi203.stdout: "finisher-PurgeQueue": { 2023-12-01T01:23:01.512 INFO:teuthology.orchestra.run.smithi203.stdout: "queue_len": 0, 2023-12-01T01:23:01.512 INFO:teuthology.orchestra.run.smithi203.stdout: "complete_latency": { 2023-12-01T01:23:01.512 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:23:01.512 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:23:01.512 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:23:01.512 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-12-01T01:23:01.512 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.512 INFO:teuthology.orchestra.run.smithi203.stdout: "mds": { 2023-12-01T01:23:01.512 INFO:teuthology.orchestra.run.smithi203.stdout: "request": 0, 2023-12-01T01:23:01.512 INFO:teuthology.orchestra.run.smithi203.stdout: "reply": 0, 2023-12-01T01:23:01.512 INFO:teuthology.orchestra.run.smithi203.stdout: "reply_latency": { 2023-12-01T01:23:01.512 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:23:01.512 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:23:01.512 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:23:01.512 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.513 INFO:teuthology.orchestra.run.smithi203.stdout: "slow_reply": 0, 2023-12-01T01:23:01.513 INFO:teuthology.orchestra.run.smithi203.stdout: "forward": 0, 2023-12-01T01:23:01.513 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_fetch_complete": 0, 2023-12-01T01:23:01.513 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_fetch_keys": 0, 2023-12-01T01:23:01.513 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_commit": 0, 2023-12-01T01:23:01.513 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_split": 0, 2023-12-01T01:23:01.513 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_merge": 0, 2023-12-01T01:23:01.513 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes": 1143, 2023-12-01T01:23:01.513 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_top": 0, 2023-12-01T01:23:01.513 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_bottom": 0, 2023-12-01T01:23:01.513 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_pin_tail": 1143, 2023-12-01T01:23:01.513 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_pinned": 1143, 2023-12-01T01:23:01.513 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_expired": 4, 2023-12-01T01:23:01.513 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_with_caps": 0, 2023-12-01T01:23:01.514 INFO:teuthology.orchestra.run.smithi203.stdout: "caps": 0, 2023-12-01T01:23:01.514 INFO:teuthology.orchestra.run.smithi203.stdout: "subtrees": 1, 2023-12-01T01:23:01.514 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse": 0, 2023-12-01T01:23:01.514 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_hit": 0, 2023-12-01T01:23:01.514 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_forward": 0, 2023-12-01T01:23:01.514 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_discover": 0, 2023-12-01T01:23:01.514 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_dir_fetch": 0, 2023-12-01T01:23:01.514 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_remote_ino": 0, 2023-12-01T01:23:01.514 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_lock": 0, 2023-12-01T01:23:01.514 INFO:teuthology.orchestra.run.smithi203.stdout: "load_cent": 0, 2023-12-01T01:23:01.514 INFO:teuthology.orchestra.run.smithi203.stdout: "q": 0, 2023-12-01T01:23:01.514 INFO:teuthology.orchestra.run.smithi203.stdout: "exported": 0, 2023-12-01T01:23:01.514 INFO:teuthology.orchestra.run.smithi203.stdout: "exported_inodes": 0, 2023-12-01T01:23:01.514 INFO:teuthology.orchestra.run.smithi203.stdout: "imported": 0, 2023-12-01T01:23:01.514 INFO:teuthology.orchestra.run.smithi203.stdout: "imported_inodes": 0, 2023-12-01T01:23:01.515 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_dir_fetch": 0, 2023-12-01T01:23:01.515 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_backtrace_fetch": 0, 2023-12-01T01:23:01.515 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_peer_discover": 0, 2023-12-01T01:23:01.515 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rfiles": 0, 2023-12-01T01:23:01.515 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rbytes": 0, 2023-12-01T01:23:01.515 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rsnaps": 0, 2023-12-01T01:23:01.515 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_backtrace_fetch": 0, 2023-12-01T01:23:01.515 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_set_tag": 0, 2023-12-01T01:23:01.515 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_backtrace_repaired": 0, 2023-12-01T01:23:01.515 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_inotable_repaired": 0, 2023-12-01T01:23:01.515 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dir_inodes": 0, 2023-12-01T01:23:01.515 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dir_base_inodes": 0, 2023-12-01T01:23:01.515 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dirfrag_rstats": 0, 2023-12-01T01:23:01.515 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_file_inodes": 0, 2023-12-01T01:23:01.515 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_inode_file_caps": 0, 2023-12-01T01:23:01.516 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_revoke": 0, 2023-12-01T01:23:01.516 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_grant": 0, 2023-12-01T01:23:01.516 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_trunc": 0, 2023-12-01T01:23:01.516 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-01T01:23:01.516 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-01T01:23:01.516 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_caps": 0, 2023-12-01T01:23:01.516 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_caps_dirty": 0, 2023-12-01T01:23:01.516 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_cap_release": 0, 2023-12-01T01:23:01.516 INFO:teuthology.orchestra.run.smithi203.stdout: "process_request_cap_release": 0 2023-12-01T01:23:01.516 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.516 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_cache": { 2023-12-01T01:23:01.517 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_update": 0, 2023-12-01T01:23:01.517 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_update_receipt": 0, 2023-12-01T01:23:01.517 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_try_discover": 0, 2023-12-01T01:23:01.517 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_send_discover": 0, 2023-12-01T01:23:01.517 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_handle_discover": 0, 2023-12-01T01:23:01.517 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays": 0, 2023-12-01T01:23:01.517 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays_delayed": 0, 2023-12-01T01:23:01.517 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays_enqueuing": 0, 2023-12-01T01:23:01.517 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_created": 0, 2023-12-01T01:23:01.517 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_enqueued": 0, 2023-12-01T01:23:01.517 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_reintegrated": 0, 2023-12-01T01:23:01.517 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_migrated": 0, 2023-12-01T01:23:01.517 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_processing": 0, 2023-12-01T01:23:01.518 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_enqueued": 0, 2023-12-01T01:23:01.518 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_prioritized": 0, 2023-12-01T01:23:01.518 INFO:teuthology.orchestra.run.smithi203.stdout: "recovery_started": 0, 2023-12-01T01:23:01.518 INFO:teuthology.orchestra.run.smithi203.stdout: "recovery_completed": 0, 2023-12-01T01:23:01.518 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_enqueue_scrub": 0, 2023-12-01T01:23:01.518 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_exportdir": 0, 2023-12-01T01:23:01.518 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_flush": 0, 2023-12-01T01:23:01.518 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_fragmentdir": 0, 2023-12-01T01:23:01.518 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_fragstats": 0, 2023-12-01T01:23:01.518 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_inodestats": 0 2023-12-01T01:23:01.518 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.518 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_log": { 2023-12-01T01:23:01.518 INFO:teuthology.orchestra.run.smithi203.stdout: "evadd": 0, 2023-12-01T01:23:01.519 INFO:teuthology.orchestra.run.smithi203.stdout: "evex": 0, 2023-12-01T01:23:01.519 INFO:teuthology.orchestra.run.smithi203.stdout: "evtrm": 0, 2023-12-01T01:23:01.519 INFO:teuthology.orchestra.run.smithi203.stdout: "ev": 3999, 2023-12-01T01:23:01.519 INFO:teuthology.orchestra.run.smithi203.stdout: "evexg": 0, 2023-12-01T01:23:01.519 INFO:teuthology.orchestra.run.smithi203.stdout: "evexd": 0, 2023-12-01T01:23:01.519 INFO:teuthology.orchestra.run.smithi203.stdout: "segadd": 0, 2023-12-01T01:23:01.519 INFO:teuthology.orchestra.run.smithi203.stdout: "segex": 0, 2023-12-01T01:23:01.519 INFO:teuthology.orchestra.run.smithi203.stdout: "segtrm": 0, 2023-12-01T01:23:01.519 INFO:teuthology.orchestra.run.smithi203.stdout: "seg": 5, 2023-12-01T01:23:01.520 INFO:teuthology.orchestra.run.smithi203.stdout: "segexg": 0, 2023-12-01T01:23:01.520 INFO:teuthology.orchestra.run.smithi203.stdout: "segexd": 0, 2023-12-01T01:23:01.520 INFO:teuthology.orchestra.run.smithi203.stdout: "expos": 4194304, 2023-12-01T01:23:01.520 INFO:teuthology.orchestra.run.smithi203.stdout: "wrpos": 10640683, 2023-12-01T01:23:01.520 INFO:teuthology.orchestra.run.smithi203.stdout: "rdpos": 10640084, 2023-12-01T01:23:01.520 INFO:teuthology.orchestra.run.smithi203.stdout: "jlat": { 2023-12-01T01:23:01.520 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:23:01.520 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:23:01.520 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:23:01.520 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.520 INFO:teuthology.orchestra.run.smithi203.stdout: "replayed": 3999 2023-12-01T01:23:01.520 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.520 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_mem": { 2023-12-01T01:23:01.520 INFO:teuthology.orchestra.run.smithi203.stdout: "ino": 8, 2023-12-01T01:23:01.520 INFO:teuthology.orchestra.run.smithi203.stdout: "ino+": 1182, 2023-12-01T01:23:01.521 INFO:teuthology.orchestra.run.smithi203.stdout: "ino-": 1174, 2023-12-01T01:23:01.521 INFO:teuthology.orchestra.run.smithi203.stdout: "dir": 6, 2023-12-01T01:23:01.521 INFO:teuthology.orchestra.run.smithi203.stdout: "dir+": 81, 2023-12-01T01:23:01.521 INFO:teuthology.orchestra.run.smithi203.stdout: "dir-": 75, 2023-12-01T01:23:01.521 INFO:teuthology.orchestra.run.smithi203.stdout: "dn": 1143, 2023-12-01T01:23:01.521 INFO:teuthology.orchestra.run.smithi203.stdout: "dn+": 2243, 2023-12-01T01:23:01.521 INFO:teuthology.orchestra.run.smithi203.stdout: "dn-": 1100, 2023-12-01T01:23:01.521 INFO:teuthology.orchestra.run.smithi203.stdout: "cap": 0, 2023-12-01T01:23:01.521 INFO:teuthology.orchestra.run.smithi203.stdout: "cap+": 0, 2023-12-01T01:23:01.521 INFO:teuthology.orchestra.run.smithi203.stdout: "cap-": 0, 2023-12-01T01:23:01.521 INFO:teuthology.orchestra.run.smithi203.stdout: "rss": 50028, 2023-12-01T01:23:01.521 INFO:teuthology.orchestra.run.smithi203.stdout: "heap": 207156 2023-12-01T01:23:01.521 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.522 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_server": { 2023-12-01T01:23:01.522 INFO:teuthology.orchestra.run.smithi203.stdout: "dispatch_client_request": 0, 2023-12-01T01:23:01.522 INFO:teuthology.orchestra.run.smithi203.stdout: "dispatch_server_request": 0, 2023-12-01T01:23:01.522 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_request": 0, 2023-12-01T01:23:01.522 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_session": 0, 2023-12-01T01:23:01.523 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_peer_request": 0, 2023-12-01T01:23:01.523 INFO:teuthology.orchestra.run.smithi203.stdout: "req_create_latency": { 2023-12-01T01:23:01.523 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:23:01.523 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:23:01.523 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:23:01.523 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.523 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getattr_latency": { 2023-12-01T01:23:01.523 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:23:01.523 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:23:01.523 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:23:01.523 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.523 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getfilelock_latency": { 2023-12-01T01:23:01.523 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:23:01.523 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:23:01.523 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:23:01.524 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.524 INFO:teuthology.orchestra.run.smithi203.stdout: "req_link_latency": { 2023-12-01T01:23:01.524 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:23:01.524 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:23:01.524 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:23:01.524 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.524 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookup_latency": { 2023-12-01T01:23:01.524 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:23:01.524 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:23:01.524 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:23:01.524 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.524 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookuphash_latency": { 2023-12-01T01:23:01.524 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:23:01.524 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:23:01.524 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:23:01.525 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.525 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupino_latency": { 2023-12-01T01:23:01.525 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:23:01.525 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:23:01.525 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:23:01.525 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.525 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupname_latency": { 2023-12-01T01:23:01.525 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:23:01.525 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:23:01.525 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:23:01.525 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.525 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupparent_latency": { 2023-12-01T01:23:01.525 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:23:01.525 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:23:01.525 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:23:01.526 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.526 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupsnap_latency": { 2023-12-01T01:23:01.526 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:23:01.526 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:23:01.526 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:23:01.526 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.526 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lssnap_latency": { 2023-12-01T01:23:01.526 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:23:01.526 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:23:01.526 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:23:01.526 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.526 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mkdir_latency": { 2023-12-01T01:23:01.526 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:23:01.526 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:23:01.526 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:23:01.527 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.527 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mknod_latency": { 2023-12-01T01:23:01.527 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:23:01.527 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:23:01.527 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:23:01.527 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.527 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mksnap_latency": { 2023-12-01T01:23:01.527 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:23:01.528 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:23:01.528 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:23:01.528 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.528 INFO:teuthology.orchestra.run.smithi203.stdout: "req_open_latency": { 2023-12-01T01:23:01.528 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:23:01.528 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:23:01.528 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:23:01.528 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.528 INFO:teuthology.orchestra.run.smithi203.stdout: "req_readdir_latency": { 2023-12-01T01:23:01.528 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:23:01.529 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:23:01.529 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:23:01.529 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.529 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rename_latency": { 2023-12-01T01:23:01.530 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:23:01.530 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:23:01.530 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:23:01.530 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.530 INFO:teuthology.orchestra.run.smithi203.stdout: "req_renamesnap_latency": { 2023-12-01T01:23:01.530 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:23:01.530 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:23:01.530 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:23:01.530 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.530 INFO:teuthology.orchestra.run.smithi203.stdout: "req_snapdiff_latency": { 2023-12-01T01:23:01.530 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:23:01.530 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:23:01.530 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:23:01.530 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.530 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmdir_latency": { 2023-12-01T01:23:01.531 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:23:01.531 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:23:01.531 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:23:01.531 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.531 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmsnap_latency": { 2023-12-01T01:23:01.531 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:23:01.531 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:23:01.531 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:23:01.531 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.531 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmxattr_latency": { 2023-12-01T01:23:01.531 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:23:01.532 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:23:01.532 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:23:01.532 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.532 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setattr_latency": { 2023-12-01T01:23:01.532 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:23:01.532 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:23:01.532 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:23:01.532 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.532 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setdirlayout_latency": { 2023-12-01T01:23:01.533 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:23:01.533 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:23:01.533 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:23:01.533 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.533 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setfilelock_latency": { 2023-12-01T01:23:01.533 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:23:01.533 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:23:01.533 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:23:01.533 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.533 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setlayout_latency": { 2023-12-01T01:23:01.533 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:23:01.533 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:23:01.533 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:23:01.533 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.534 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setxattr_latency": { 2023-12-01T01:23:01.534 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:23:01.534 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:23:01.534 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:23:01.534 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.534 INFO:teuthology.orchestra.run.smithi203.stdout: "req_symlink_latency": { 2023-12-01T01:23:01.534 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:23:01.534 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:23:01.534 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:23:01.534 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.534 INFO:teuthology.orchestra.run.smithi203.stdout: "req_unlink_latency": { 2023-12-01T01:23:01.534 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:23:01.534 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:23:01.534 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:23:01.535 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.535 INFO:teuthology.orchestra.run.smithi203.stdout: "cap_revoke_eviction": 0, 2023-12-01T01:23:01.535 INFO:teuthology.orchestra.run.smithi203.stdout: "cap_acquisition_throttle": 0, 2023-12-01T01:23:01.535 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getvxattr_latency": { 2023-12-01T01:23:01.535 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:23:01.535 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:23:01.535 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:23:01.535 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-12-01T01:23:01.535 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.535 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_sessions": { 2023-12-01T01:23:01.535 INFO:teuthology.orchestra.run.smithi203.stdout: "session_count": 1, 2023-12-01T01:23:01.535 INFO:teuthology.orchestra.run.smithi203.stdout: "session_add": 1, 2023-12-01T01:23:01.535 INFO:teuthology.orchestra.run.smithi203.stdout: "session_remove": 0, 2023-12-01T01:23:01.535 INFO:teuthology.orchestra.run.smithi203.stdout: "sessions_open": 1, 2023-12-01T01:23:01.535 INFO:teuthology.orchestra.run.smithi203.stdout: "sessions_stale": 0, 2023-12-01T01:23:01.535 INFO:teuthology.orchestra.run.smithi203.stdout: "total_load": 0, 2023-12-01T01:23:01.536 INFO:teuthology.orchestra.run.smithi203.stdout: "average_load": 0, 2023-12-01T01:23:01.536 INFO:teuthology.orchestra.run.smithi203.stdout: "avg_session_uptime": 1654, 2023-12-01T01:23:01.536 INFO:teuthology.orchestra.run.smithi203.stdout: "mdthresh_evicted": 0 2023-12-01T01:23:01.536 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.536 INFO:teuthology.orchestra.run.smithi203.stdout: "mempool": { 2023-12-01T01:23:01.536 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_bytes": 0, 2023-12-01T01:23:01.536 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_items": 0, 2023-12-01T01:23:01.536 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_bytes": 0, 2023-12-01T01:23:01.536 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_items": 0, 2023-12-01T01:23:01.536 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_bytes": 0, 2023-12-01T01:23:01.536 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_items": 0, 2023-12-01T01:23:01.536 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-01T01:23:01.536 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_items": 0, 2023-12-01T01:23:01.536 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-01T01:23:01.536 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_items": 0, 2023-12-01T01:23:01.537 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_bytes": 0, 2023-12-01T01:23:01.537 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_items": 0, 2023-12-01T01:23:01.537 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-01T01:23:01.537 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_items": 0, 2023-12-01T01:23:01.537 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_bytes": 0, 2023-12-01T01:23:01.537 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_items": 0, 2023-12-01T01:23:01.537 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_bytes": 0, 2023-12-01T01:23:01.537 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_items": 0, 2023-12-01T01:23:01.537 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-01T01:23:01.537 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_items": 0, 2023-12-01T01:23:01.537 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-01T01:23:01.537 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_items": 0, 2023-12-01T01:23:01.537 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_bytes": 0, 2023-12-01T01:23:01.537 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_items": 0, 2023-12-01T01:23:01.537 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_bytes": 0, 2023-12-01T01:23:01.538 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_items": 0, 2023-12-01T01:23:01.538 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-01T01:23:01.538 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_items": 0, 2023-12-01T01:23:01.538 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_bytes": 0, 2023-12-01T01:23:01.538 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_items": 0, 2023-12-01T01:23:01.538 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_bytes": 0, 2023-12-01T01:23:01.538 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_items": 0, 2023-12-01T01:23:01.538 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_bytes": 0, 2023-12-01T01:23:01.538 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_items": 0, 2023-12-01T01:23:01.538 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_bytes": 0, 2023-12-01T01:23:01.538 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_items": 0, 2023-12-01T01:23:01.538 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_bytes": 189256, 2023-12-01T01:23:01.538 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_items": 49, 2023-12-01T01:23:01.538 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_bytes": 0, 2023-12-01T01:23:01.539 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_items": 0, 2023-12-01T01:23:01.539 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_bytes": 0, 2023-12-01T01:23:01.539 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_items": 0, 2023-12-01T01:23:01.539 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_bytes": 0, 2023-12-01T01:23:01.539 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_items": 0, 2023-12-01T01:23:01.539 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_bytes": 0, 2023-12-01T01:23:01.539 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_items": 0, 2023-12-01T01:23:01.539 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_bytes": 7336, 2023-12-01T01:23:01.539 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_items": 154, 2023-12-01T01:23:01.539 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_bytes": 0, 2023-12-01T01:23:01.539 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_items": 0, 2023-12-01T01:23:01.539 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_bytes": 0, 2023-12-01T01:23:01.539 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_items": 0, 2023-12-01T01:23:01.539 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_bytes": 821088, 2023-12-01T01:23:01.539 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_items": 4645, 2023-12-01T01:23:01.540 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_bytes": 0, 2023-12-01T01:23:01.540 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_items": 0, 2023-12-01T01:23:01.540 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_bytes": 0, 2023-12-01T01:23:01.540 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_items": 0 2023-12-01T01:23:01.540 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.540 INFO:teuthology.orchestra.run.smithi203.stdout: "objecter": { 2023-12-01T01:23:01.540 INFO:teuthology.orchestra.run.smithi203.stdout: "op_active": 0, 2023-12-01T01:23:01.540 INFO:teuthology.orchestra.run.smithi203.stdout: "op_laggy": 0, 2023-12-01T01:23:01.540 INFO:teuthology.orchestra.run.smithi203.stdout: "op_send": 1083, 2023-12-01T01:23:01.540 INFO:teuthology.orchestra.run.smithi203.stdout: "op_send_bytes": 21, 2023-12-01T01:23:01.540 INFO:teuthology.orchestra.run.smithi203.stdout: "op_resend": 3, 2023-12-01T01:23:01.540 INFO:teuthology.orchestra.run.smithi203.stdout: "op_reply": 1083, 2023-12-01T01:23:01.540 INFO:teuthology.orchestra.run.smithi203.stdout: "op_latency": { 2023-12-01T01:23:01.540 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1083, 2023-12-01T01:23:01.540 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.653983794, 2023-12-01T01:23:01.540 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000603863 2023-12-01T01:23:01.541 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.541 INFO:teuthology.orchestra.run.smithi203.stdout: "op_inflight": 0, 2023-12-01T01:23:01.541 INFO:teuthology.orchestra.run.smithi203.stdout: "oplen_avg": { 2023-12-01T01:23:01.541 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1083, 2023-12-01T01:23:01.541 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 1084 2023-12-01T01:23:01.541 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.541 INFO:teuthology.orchestra.run.smithi203.stdout: "op": 1083, 2023-12-01T01:23:01.541 INFO:teuthology.orchestra.run.smithi203.stdout: "op_r": 1083, 2023-12-01T01:23:01.541 INFO:teuthology.orchestra.run.smithi203.stdout: "op_w": 0, 2023-12-01T01:23:01.541 INFO:teuthology.orchestra.run.smithi203.stdout: "op_rmw": 0, 2023-12-01T01:23:01.541 INFO:teuthology.orchestra.run.smithi203.stdout: "op_pg": 0, 2023-12-01T01:23:01.541 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_stat": 651, 2023-12-01T01:23:01.541 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_create": 0, 2023-12-01T01:23:01.541 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_read": 430, 2023-12-01T01:23:01.541 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_write": 0, 2023-12-01T01:23:01.542 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_writefull": 0, 2023-12-01T01:23:01.542 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_writesame": 0, 2023-12-01T01:23:01.542 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_append": 0, 2023-12-01T01:23:01.542 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_zero": 0, 2023-12-01T01:23:01.542 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_truncate": 0, 2023-12-01T01:23:01.542 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_delete": 0, 2023-12-01T01:23:01.542 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_mapext": 0, 2023-12-01T01:23:01.542 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_sparse_read": 0, 2023-12-01T01:23:01.542 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_clonerange": 0, 2023-12-01T01:23:01.542 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_getxattr": 1, 2023-12-01T01:23:01.542 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_setxattr": 0, 2023-12-01T01:23:01.542 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_cmpxattr": 0, 2023-12-01T01:23:01.542 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_rmxattr": 0, 2023-12-01T01:23:01.542 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_resetxattrs": 0, 2023-12-01T01:23:01.543 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_call": 0, 2023-12-01T01:23:01.543 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_watch": 0, 2023-12-01T01:23:01.543 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_notify": 0, 2023-12-01T01:23:01.543 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_src_cmpxattr": 0, 2023-12-01T01:23:01.543 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_pgls": 0, 2023-12-01T01:23:01.543 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_pgls_filter": 0, 2023-12-01T01:23:01.543 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_other": 0, 2023-12-01T01:23:01.543 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_active": 0, 2023-12-01T01:23:01.543 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_send": 0, 2023-12-01T01:23:01.543 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_resend": 0, 2023-12-01T01:23:01.543 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_ping": 0, 2023-12-01T01:23:01.543 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_active": 0, 2023-12-01T01:23:01.543 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_send": 0, 2023-12-01T01:23:01.543 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_resend": 0, 2023-12-01T01:23:01.543 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_active": 0, 2023-12-01T01:23:01.544 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_send": 0, 2023-12-01T01:23:01.544 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_resend": 0, 2023-12-01T01:23:01.544 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_active": 0, 2023-12-01T01:23:01.544 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_send": 0, 2023-12-01T01:23:01.544 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_resend": 0, 2023-12-01T01:23:01.544 INFO:teuthology.orchestra.run.smithi203.stdout: "command_active": 0, 2023-12-01T01:23:01.544 INFO:teuthology.orchestra.run.smithi203.stdout: "command_send": 0, 2023-12-01T01:23:01.544 INFO:teuthology.orchestra.run.smithi203.stdout: "command_resend": 0, 2023-12-01T01:23:01.544 INFO:teuthology.orchestra.run.smithi203.stdout: "map_epoch": 84, 2023-12-01T01:23:01.544 INFO:teuthology.orchestra.run.smithi203.stdout: "map_full": 0, 2023-12-01T01:23:01.544 INFO:teuthology.orchestra.run.smithi203.stdout: "map_inc": 2, 2023-12-01T01:23:01.544 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_sessions": 5, 2023-12-01T01:23:01.544 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_session_open": 5, 2023-12-01T01:23:01.544 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_session_close": 0, 2023-12-01T01:23:01.544 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_laggy": 0, 2023-12-01T01:23:01.545 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_wr": 0, 2023-12-01T01:23:01.545 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_rd": 2, 2023-12-01T01:23:01.545 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_del": 0 2023-12-01T01:23:01.545 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.545 INFO:teuthology.orchestra.run.smithi203.stdout: "oft": { 2023-12-01T01:23:01.545 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_objs": 0, 2023-12-01T01:23:01.545 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_kv_pairs": 0, 2023-12-01T01:23:01.545 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_updates": 0, 2023-12-01T01:23:01.545 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_removes": 0 2023-12-01T01:23:01.545 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.545 INFO:teuthology.orchestra.run.smithi203.stdout: "osd-slow-ops": { 2023-12-01T01:23:01.545 INFO:teuthology.orchestra.run.smithi203.stdout: "slow_ops_count": 0 2023-12-01T01:23:01.545 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.545 INFO:teuthology.orchestra.run.smithi203.stdout: "purge_queue": { 2023-12-01T01:23:01.545 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_ops": 0, 2023-12-01T01:23:01.545 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_ops_high_water": 0, 2023-12-01T01:23:01.546 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing": 0, 2023-12-01T01:23:01.546 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_high_water": 0, 2023-12-01T01:23:01.546 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executed": 0, 2023-12-01T01:23:01.546 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_item_in_journal": 0 2023-12-01T01:23:01.546 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.546 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-01T01:23:01.546 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2023-12-01T01:23:01.546 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2023-12-01T01:23:01.546 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2023-12-01T01:23:01.546 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 1228, 2023-12-01T01:23:01.546 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 7195888, 2023-12-01T01:23:01.546 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2023-12-01T01:23:01.546 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 1228, 2023-12-01T01:23:01.546 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2023-12-01T01:23:01.546 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2023-12-01T01:23:01.547 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 1228, 2023-12-01T01:23:01.547 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 7195888, 2023-12-01T01:23:01.547 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2023-12-01T01:23:01.547 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:23:01.547 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:23:01.547 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:23:01.547 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-12-01T01:23:01.547 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.547 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-objecter_bytes": { 2023-12-01T01:23:01.547 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2023-12-01T01:23:01.547 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2023-12-01T01:23:01.547 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2023-12-01T01:23:01.547 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2023-12-01T01:23:01.547 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2023-12-01T01:23:01.547 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2023-12-01T01:23:01.548 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2023-12-01T01:23:01.548 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 1083, 2023-12-01T01:23:01.548 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 6446479, 2023-12-01T01:23:01.548 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 1083, 2023-12-01T01:23:01.548 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 6446479, 2023-12-01T01:23:01.548 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2023-12-01T01:23:01.548 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:23:01.548 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:23:01.548 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:23:01.548 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-12-01T01:23:01.548 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.548 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-objecter_ops": { 2023-12-01T01:23:01.548 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2023-12-01T01:23:01.548 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 1024, 2023-12-01T01:23:01.548 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2023-12-01T01:23:01.549 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2023-12-01T01:23:01.549 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2023-12-01T01:23:01.549 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2023-12-01T01:23:01.549 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2023-12-01T01:23:01.549 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 1083, 2023-12-01T01:23:01.549 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 1083, 2023-12-01T01:23:01.549 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 1083, 2023-12-01T01:23:01.549 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 1083, 2023-12-01T01:23:01.549 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2023-12-01T01:23:01.549 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:23:01.549 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:23:01.549 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:23:01.549 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-12-01T01:23:01.549 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.549 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-write_buf_throttle": { 2023-12-01T01:23:01.550 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2023-12-01T01:23:01.550 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 3758096384, 2023-12-01T01:23:01.550 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2023-12-01T01:23:01.550 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2023-12-01T01:23:01.550 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2023-12-01T01:23:01.550 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2023-12-01T01:23:01.550 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2023-12-01T01:23:01.550 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2023-12-01T01:23:01.550 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2023-12-01T01:23:01.550 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 0, 2023-12-01T01:23:01.550 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 0, 2023-12-01T01:23:01.550 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2023-12-01T01:23:01.550 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:23:01.550 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:23:01.550 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:23:01.551 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-12-01T01:23:01.551 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2023-12-01T01:23:01.551 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-write_buf_throttle-0x55ff4fc90500": { 2023-12-01T01:23:01.551 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2023-12-01T01:23:01.551 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 3758096384, 2023-12-01T01:23:01.551 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2023-12-01T01:23:01.551 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2023-12-01T01:23:01.551 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2023-12-01T01:23:01.551 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2023-12-01T01:23:01.551 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2023-12-01T01:23:01.551 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2023-12-01T01:23:01.551 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2023-12-01T01:23:01.551 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 0, 2023-12-01T01:23:01.551 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 0, 2023-12-01T01:23:01.551 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2023-12-01T01:23:01.551 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2023-12-01T01:23:01.552 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2023-12-01T01:23:01.552 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2023-12-01T01:23:01.552 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-12-01T01:23:01.552 INFO:teuthology.orchestra.run.smithi203.stdout: } 2023-12-01T01:23:01.552 INFO:teuthology.orchestra.run.smithi203.stdout:} 2023-12-01T01:23:01.697 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:01 smithi203 ceph-mon[103581]: pgmap v452: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 11 op/s 2023-12-01T01:23:01.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:01 smithi137 ceph-mon[103312]: pgmap v452: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 11 op/s 2023-12-01T01:23:01.904 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2023-12-01T01:23:01.904 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2023-12-01T01:23:01.904 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2023-12-01T01:23:01.904 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2023-12-01T01:23:01.905 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2023-12-01T01:23:01.905 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-12-01T01:23:01.916 INFO:tasks.kclient:Unmounting kernel clients... 2023-12-01T01:23:01.916 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-12-01T01:23:01.916 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:23:01.917 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:01 smithi077 ceph-mon[98867]: pgmap v452: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 11 op/s 2023-12-01T01:23:01.940 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-12-01T01:23:01.941 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:23:02.002 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:23:02.177 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.176+0000 7f7ec2b72700 1 -- 172.21.15.77:0/1075652393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ebc101260 msgr2=0x7f7ebc101680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:02.177 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.176+0000 7f7ec2b72700 1 --2- 172.21.15.77:0/1075652393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ebc101260 0x7f7ebc101680 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0009990 tx=0x7f7eb0009ca0 comp rx=0 tx=0).stop 2023-12-01T01:23:02.178 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.176+0000 7f7ec2b72700 1 -- 172.21.15.77:0/1075652393 shutdown_connections 2023-12-01T01:23:02.178 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.176+0000 7f7ec2b72700 1 --2- 172.21.15.77:0/1075652393 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ebc1008a0 0x7f7ebc108160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:02.178 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.176+0000 7f7ec2b72700 1 --2- 172.21.15.77:0/1075652393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ebc101260 0x7f7ebc101680 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:02.178 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.176+0000 7f7ec2b72700 1 --2- 172.21.15.77:0/1075652393 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ebc0fff00 0x7f7ebc100360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:02.178 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.176+0000 7f7ec2b72700 1 -- 172.21.15.77:0/1075652393 >> 172.21.15.77:0/1075652393 conn(0x7f7ebc0fa950 msgr2=0x7f7ebc0fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:23:02.178 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.176+0000 7f7ec2b72700 1 -- 172.21.15.77:0/1075652393 shutdown_connections 2023-12-01T01:23:02.178 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.176+0000 7f7ec2b72700 1 -- 172.21.15.77:0/1075652393 wait complete. 2023-12-01T01:23:02.180 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.177+0000 7f7ec2b72700 1 Processor -- start 2023-12-01T01:23:02.180 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.177+0000 7f7ec2b72700 1 -- start start 2023-12-01T01:23:02.180 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.177+0000 7f7ec2b72700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ebc0fff00 0x7f7ebc195910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:02.180 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.177+0000 7f7ec2b72700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ebc1008a0 0x7f7ebc195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:02.180 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.177+0000 7f7ec2b72700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ebc101260 0x7f7ebc19cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:02.180 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.177+0000 7f7ec2b72700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc19d490 con 0x7f7ebc101260 2023-12-01T01:23:02.180 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.177+0000 7f7ec2b72700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc19d5d0 con 0x7f7ebc0fff00 2023-12-01T01:23:02.180 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.177+0000 7f7ec2b72700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc19d760 con 0x7f7ebc1008a0 2023-12-01T01:23:02.181 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.177+0000 7f7ec110f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ebc101260 0x7f7ebc19cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:02.181 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.177+0000 7f7ec110f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ebc101260 0x7f7ebc19cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33198/0 (socket says 172.21.15.77:33198) 2023-12-01T01:23:02.181 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.177+0000 7f7ec110f700 1 -- 172.21.15.77:0/3159835069 learned_addr learned my addr 172.21.15.77:0/3159835069 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:23:02.181 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.177+0000 7f7ec090e700 1 --2- 172.21.15.77:0/3159835069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ebc0fff00 0x7f7ebc195910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:02.181 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.177+0000 7f7ebbfff700 1 --2- 172.21.15.77:0/3159835069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ebc1008a0 0x7f7ebc195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:02.181 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.177+0000 7f7ec110f700 1 -- 172.21.15.77:0/3159835069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ebc1008a0 msgr2=0x7f7ebc195e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:02.181 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.177+0000 7f7ec110f700 1 --2- 172.21.15.77:0/3159835069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ebc1008a0 0x7f7ebc195e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:02.181 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.177+0000 7f7ec110f700 1 -- 172.21.15.77:0/3159835069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ebc0fff00 msgr2=0x7f7ebc195910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:02.181 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.177+0000 7f7ec110f700 1 --2- 172.21.15.77:0/3159835069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ebc0fff00 0x7f7ebc195910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:02.181 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.177+0000 7f7ec110f700 1 -- 172.21.15.77:0/3159835069 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb00095f0 con 0x7f7ebc101260 2023-12-01T01:23:02.181 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.178+0000 7f7ec110f700 1 --2- 172.21.15.77:0/3159835069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ebc101260 0x7f7ebc19cec0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f7eb400ef30 tx=0x7f7eb400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:23:02.181 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.178+0000 7f7eb9ffb700 1 -- 172.21.15.77:0/3159835069 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb4018070 con 0x7f7ebc101260 2023-12-01T01:23:02.181 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.178+0000 7f7eb9ffb700 1 -- 172.21.15.77:0/3159835069 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7eb400f040 con 0x7f7ebc101260 2023-12-01T01:23:02.181 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.178+0000 7f7ec2b72700 1 -- 172.21.15.77:0/3159835069 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ebc19d9e0 con 0x7f7ebc101260 2023-12-01T01:23:02.181 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.178+0000 7f7eb9ffb700 1 -- 172.21.15.77:0/3159835069 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb400cdb0 con 0x7f7ebc101260 2023-12-01T01:23:02.182 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.178+0000 7f7ec2b72700 1 -- 172.21.15.77:0/3159835069 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ebc19dea0 con 0x7f7ebc101260 2023-12-01T01:23:02.182 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.179+0000 7f7ec2b72700 1 -- 172.21.15.77:0/3159835069 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ebc1072b0 con 0x7f7ebc101260 2023-12-01T01:23:02.182 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.180+0000 7f7eb9ffb700 1 -- 172.21.15.77:0/3159835069 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f7eb4003a00 con 0x7f7ebc101260 2023-12-01T01:23:02.182 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.180+0000 7f7eb9ffb700 1 --2- 172.21.15.77:0/3159835069 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7eac0a0570 0x7f7eac0a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:02.182 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.181+0000 7f7eb9ffb700 1 -- 172.21.15.77:0/3159835069 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f7eb40cb4d0 con 0x7f7ebc101260 2023-12-01T01:23:02.183 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.181+0000 7f7ec090e700 1 --2- 172.21.15.77:0/3159835069 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7eac0a0570 0x7f7eac0a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:02.183 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.181+0000 7f7ec090e700 1 --2- 172.21.15.77:0/3159835069 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7eac0a0570 0x7f7eac0a2a30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7eb000b2e0 tx=0x7f7eb0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:23:02.186 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.185+0000 7f7eb9ffb700 1 -- 172.21.15.77:0/3159835069 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7eb40951a0 con 0x7f7ebc101260 2023-12-01T01:23:02.379 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.378+0000 7f7ec2b72700 1 -- 172.21.15.77:0/3159835069 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ebc02cec0 con 0x7f7ebc101260 2023-12-01T01:23:02.381 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.380+0000 7f7eb9ffb700 1 -- 172.21.15.77:0/3159835069 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22472 (secure 0 0 0) 0x7f7eb40987c0 con 0x7f7ebc101260 2023-12-01T01:23:02.381 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:23:02.381 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":84,"fsid":"2a06e29a-8fe6-11ee-95a2-87774f69a715","created":"2023-12-01T01:09:30.210398+0000","modified":"2023-12-01T01:17:25.609766+0000","last_up_change":"2023-12-01T01:15:44.586092+0000","last_in_change":"2023-12-01T01:15:26.305439+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:12:30.514229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:16:23.568860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-01T01:16:24.531137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fc651e62-a103-4ab8-a76d-d3708ea6aeec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6803","nonce":2244799094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6805","nonce":2244799094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6809","nonce":2244799094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6807","nonce":2244799094}]},"public_addr":"172.21.15.77:6803/2244799094","cluster_addr":"172.21.15.77:6805/2244799094","heartbeat_back_addr":"172.21.15.77:6809/2244799094","heartbeat_front_addr":"172.21.15.77:6807/2244799094","state":["exists","up"]},{"osd":1,"uuid":"0adf8c54-49ec-49a0-8bf1-af1f20e60729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6801","nonce":224194890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6803","nonce":224194890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6807","nonce":224194890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6805","nonce":224194890}]},"public_addr":"172.21.15.137:6801/224194890","cluster_addr":"172.21.15.137:6803/224194890","heartbeat_back_addr":"172.21.15.137:6807/224194890","heartbeat_front_addr":"172.21.15.137:6805/224194890","state":["exists","up"]},{"osd":2,"uuid":"5a69802c-dede-4544-9a12-436f5c1c0038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6801","nonce":4078459110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6803","nonce":4078459110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6807","nonce":4078459110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6805","nonce":4078459110}]},"public_addr":"172.21.15.203:6801/4078459110","cluster_addr":"172.21.15.203:6803/4078459110","heartbeat_back_addr":"172.21.15.203:6807/4078459110","heartbeat_front_addr":"172.21.15.203:6805/4078459110","state":["exists","up"]},{"osd":3,"uuid":"ad325a97-580b-4c99-bf12-cbb0d92a6d0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6811","nonce":1454440477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6813","nonce":1454440477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6817","nonce":1454440477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6815","nonce":1454440477}]},"public_addr":"172.21.15.77:6811/1454440477","cluster_addr":"172.21.15.77:6813/1454440477","heartbeat_back_addr":"172.21.15.77:6817/1454440477","heartbeat_front_addr":"172.21.15.77:6815/1454440477","state":["exists","up"]},{"osd":4,"uuid":"1e3ac1ba-98bb-4b8b-8e26-50064c66bf2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6809","nonce":1103241451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6811","nonce":1103241451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6815","nonce":1103241451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6813","nonce":1103241451}]},"public_addr":"172.21.15.137:6809/1103241451","cluster_addr":"172.21.15.137:6811/1103241451","heartbeat_back_addr":"172.21.15.137:6815/1103241451","heartbeat_front_addr":"172.21.15.137:6813/1103241451","state":["exists","up"]},{"osd":5,"uuid":"eb32c016-abeb-44ab-abf0-19dd09235736","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6809","nonce":2648569482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6811","nonce":2648569482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6815","nonce":2648569482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6813","nonce":2648569482}]},"public_addr":"172.21.15.203:6809/2648569482","cluster_addr":"172.21.15.203:6811/2648569482","heartbeat_back_addr":"172.21.15.203:6815/2648569482","heartbeat_front_addr":"172.21.15.203:6813/2648569482","state":["exists","up"]},{"osd":6,"uuid":"0e3fa97e-2ca6-418c-9e8d-a6fe3c56adb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6819","nonce":2463414651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6821","nonce":2463414651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6825","nonce":2463414651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6823","nonce":2463414651}]},"public_addr":"172.21.15.77:6819/2463414651","cluster_addr":"172.21.15.77:6821/2463414651","heartbeat_back_addr":"172.21.15.77:6825/2463414651","heartbeat_front_addr":"172.21.15.77:6823/2463414651","state":["exists","up"]},{"osd":7,"uuid":"d7398c01-0f98-4138-8a49-c3ff0aae7e88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6817","nonce":3108330766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6819","nonce":3108330766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6823","nonce":3108330766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6821","nonce":3108330766}]},"public_addr":"172.21.15.137:6817/3108330766","cluster_addr":"172.21.15.137:6819/3108330766","heartbeat_back_addr":"172.21.15.137:6823/3108330766","heartbeat_front_addr":"172.21.15.137:6821/3108330766","state":["exists","up"]},{"osd":8,"uuid":"0c09050d-3e43-410c-8cd7-bcb891af56b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6817","nonce":1161158776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6819","nonce":1161158776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6823","nonce":1161158776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6821","nonce":1161158776}]},"public_addr":"172.21.15.203:6817/1161158776","cluster_addr":"172.21.15.203:6819/1161158776","heartbeat_back_addr":"172.21.15.203:6823/1161158776","heartbeat_front_addr":"172.21.15.203:6821/1161158776","state":["exists","up"]},{"osd":9,"uuid":"527ecb49-95ec-48bd-9872-3b07fae5c5ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6827","nonce":2409880685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6829","nonce":2409880685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6833","nonce":2409880685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6831","nonce":2409880685}]},"public_addr":"172.21.15.77:6827/2409880685","cluster_addr":"172.21.15.77:6829/2409880685","heartbeat_back_addr":"172.21.15.77:6833/2409880685","heartbeat_front_addr":"172.21.15.77:6831/2409880685","state":["exists","up"]},{"osd":10,"uuid":"1680bb3e-a3c8-427b-af68-e2214f32bed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6825","nonce":374920146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6827","nonce":374920146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6831","nonce":374920146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6829","nonce":374920146}]},"public_addr":"172.21.15.137:6825/374920146","cluster_addr":"172.21.15.137:6827/374920146","heartbeat_back_addr":"172.21.15.137:6831/374920146","heartbeat_front_addr":"172.21.15.137:6829/374920146","state":["exists","up"]},{"osd":11,"uuid":"8a533466-023d-4803-ae36-9a9f1715e49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6825","nonce":2161588962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6827","nonce":2161588962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6831","nonce":2161588962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6829","nonce":2161588962}]},"public_addr":"172.21.15.203:6825/2161588962","cluster_addr":"172.21.15.203:6827/2161588962","heartbeat_back_addr":"172.21.15.203:6831/2161588962","heartbeat_front_addr":"172.21.15.203:6829/2161588962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:11:45.420509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:06.210115+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:28.191307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:49.594116+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:11.261689+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:33.376541+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:54.968336+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:16.775310+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:38.733792+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:59.456699+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:15:20.924059+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:15:43.094773+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/2260035254":"2023-12-02T01:10:24.191835+0000","172.21.15.77:6800/1456621747":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/219838828":"2023-12-02T01:10:24.191835+0000","172.21.15.77:6801/1456621747":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/2937149266":"2023-12-02T01:10:02.232948+0000","172.21.15.77:0/807520018":"2023-12-02T01:10:02.232948+0000","172.21.15.77:6801/1661788026":"2023-12-02T01:10:02.232948+0000","172.21.15.77:6800/1661788026":"2023-12-02T01:10:02.232948+0000","172.21.15.77:0/3551282473":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/1202222558":"2023-12-02T01:10:02.232948+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:23:02.383 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.382+0000 7f7ec2b72700 1 -- 172.21.15.77:0/3159835069 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7eac0a0570 msgr2=0x7f7eac0a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:02.383 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.382+0000 7f7ec2b72700 1 --2- 172.21.15.77:0/3159835069 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7eac0a0570 0x7f7eac0a2a30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7eb000b2e0 tx=0x7f7eb0005fb0 comp rx=0 tx=0).stop 2023-12-01T01:23:02.383 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.382+0000 7f7ec2b72700 1 -- 172.21.15.77:0/3159835069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ebc101260 msgr2=0x7f7ebc19cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:02.383 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.382+0000 7f7ec2b72700 1 --2- 172.21.15.77:0/3159835069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ebc101260 0x7f7ebc19cec0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f7eb400ef30 tx=0x7f7eb400c3a0 comp rx=0 tx=0).stop 2023-12-01T01:23:02.384 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.382+0000 7f7ec2b72700 1 -- 172.21.15.77:0/3159835069 shutdown_connections 2023-12-01T01:23:02.384 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.382+0000 7f7ec2b72700 1 --2- 172.21.15.77:0/3159835069 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f7eac0a0570 0x7f7eac0a2a30 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:02.384 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.382+0000 7f7ec2b72700 1 --2- 172.21.15.77:0/3159835069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ebc101260 0x7f7ebc19cec0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:02.384 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.382+0000 7f7ec2b72700 1 --2- 172.21.15.77:0/3159835069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ebc0fff00 0x7f7ebc195910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:02.384 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.382+0000 7f7ec2b72700 1 --2- 172.21.15.77:0/3159835069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ebc1008a0 0x7f7ebc195e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:02.384 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.382+0000 7f7ec2b72700 1 -- 172.21.15.77:0/3159835069 >> 172.21.15.77:0/3159835069 conn(0x7f7ebc0fa950 msgr2=0x7f7ebc1043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:23:02.384 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.382+0000 7f7ec2b72700 1 -- 172.21.15.77:0/3159835069 shutdown_connections 2023-12-01T01:23:02.384 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.382+0000 7f7ec2b72700 1 -- 172.21.15.77:0/3159835069 wait complete. 2023-12-01T01:23:02.397 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-01T01:12:30.514229+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-01T01:16:23.568860+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-01T01:16:24.531137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-01T01:23:02.397 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-01T01:23:02.524 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.520+0000 7f2a0f25e700 1 -- 172.21.15.77:0/3795531464 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a08102940 msgr2=0x7f2a081091f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:02.524 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.520+0000 7f2a0f25e700 1 --2- 172.21.15.77:0/3795531464 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a08102940 0x7f2a081091f0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f2a0000b0d0 tx=0x7f2a0000b3e0 comp rx=0 tx=0).stop 2023-12-01T01:23:02.524 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.520+0000 7f2a0f25e700 1 -- 172.21.15.77:0/3795531464 shutdown_connections 2023-12-01T01:23:02.524 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.520+0000 7f2a0f25e700 1 --2- 172.21.15.77:0/3795531464 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a08102940 0x7f2a081091f0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:02.524 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.520+0000 7f2a0f25e700 1 --2- 172.21.15.77:0/3795531464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a08100db0 0x7f2a081011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:02.524 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.520+0000 7f2a0f25e700 1 --2- 172.21.15.77:0/3795531464 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a08101fa0 0x7f2a08102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:02.525 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.520+0000 7f2a0f25e700 1 -- 172.21.15.77:0/3795531464 >> 172.21.15.77:0/3795531464 conn(0x7f2a080fc300 msgr2=0x7f2a080fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:23:02.525 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.520+0000 7f2a0f25e700 1 -- 172.21.15.77:0/3795531464 shutdown_connections 2023-12-01T01:23:02.525 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.520+0000 7f2a0f25e700 1 -- 172.21.15.77:0/3795531464 wait complete. 2023-12-01T01:23:02.525 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.521+0000 7f2a0f25e700 1 Processor -- start 2023-12-01T01:23:02.525 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.521+0000 7f2a0f25e700 1 -- start start 2023-12-01T01:23:02.525 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.521+0000 7f2a0f25e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a08100db0 0x7f2a08196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:02.525 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.521+0000 7f2a0f25e700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a08101fa0 0x7f2a08197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:02.525 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.521+0000 7f2a0f25e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a08102940 0x7f2a0819e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:02.525 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.521+0000 7f2a0f25e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0819e6b0 con 0x7f2a08102940 2023-12-01T01:23:02.525 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.521+0000 7f2a0f25e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0819e7f0 con 0x7f2a08100db0 2023-12-01T01:23:02.525 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.521+0000 7f2a0f25e700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0819e980 con 0x7f2a08101fa0 2023-12-01T01:23:02.525 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.521+0000 7f2a0d7fb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a08102940 0x7f2a0819e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:02.525 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.521+0000 7f2a0d7fb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a08102940 0x7f2a0819e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33212/0 (socket says 172.21.15.77:33212) 2023-12-01T01:23:02.525 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.521+0000 7f2a0d7fb700 1 -- 172.21.15.77:0/555283100 learned_addr learned my addr 172.21.15.77:0/555283100 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:23:02.525 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.521+0000 7f2a07fff700 1 --2- 172.21.15.77:0/555283100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a08101fa0 0x7f2a08197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:02.526 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.521+0000 7f2a0cffa700 1 --2- 172.21.15.77:0/555283100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a08100db0 0x7f2a08196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:02.526 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.521+0000 7f2a0d7fb700 1 -- 172.21.15.77:0/555283100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a08101fa0 msgr2=0x7f2a08197070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:02.526 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.521+0000 7f2a0d7fb700 1 --2- 172.21.15.77:0/555283100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a08101fa0 0x7f2a08197070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:02.526 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.521+0000 7f2a0d7fb700 1 -- 172.21.15.77:0/555283100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a08100db0 msgr2=0x7f2a08196b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:02.526 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.521+0000 7f2a0d7fb700 1 --2- 172.21.15.77:0/555283100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a08100db0 0x7f2a08196b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:02.526 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.521+0000 7f2a0d7fb700 1 -- 172.21.15.77:0/555283100 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a00009d00 con 0x7f2a08102940 2023-12-01T01:23:02.526 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.521+0000 7f2a0cffa700 1 --2- 172.21.15.77:0/555283100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a08100db0 0x7f2a08196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:23:02.526 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.522+0000 7f2a0d7fb700 1 --2- 172.21.15.77:0/555283100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a08102940 0x7f2a0819e0e0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f2a00008ec0 tx=0x7f2a00008ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:23:02.526 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.522+0000 7f2a05ffb700 1 -- 172.21.15.77:0/555283100 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0000e040 con 0x7f2a08102940 2023-12-01T01:23:02.526 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.522+0000 7f2a05ffb700 1 -- 172.21.15.77:0/555283100 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a0000be30 con 0x7f2a08102940 2023-12-01T01:23:02.526 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.522+0000 7f2a0f25e700 1 -- 172.21.15.77:0/555283100 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a0819ec00 con 0x7f2a08102940 2023-12-01T01:23:02.526 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.522+0000 7f2a05ffb700 1 -- 172.21.15.77:0/555283100 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0001b6c0 con 0x7f2a08102940 2023-12-01T01:23:02.526 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.522+0000 7f2a0f25e700 1 -- 172.21.15.77:0/555283100 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a0819efd0 con 0x7f2a08102940 2023-12-01T01:23:02.526 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.523+0000 7f2a0f25e700 1 -- 172.21.15.77:0/555283100 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a081082d0 con 0x7f2a08102940 2023-12-01T01:23:02.528 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.524+0000 7f2a05ffb700 1 -- 172.21.15.77:0/555283100 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f2a00005090 con 0x7f2a08102940 2023-12-01T01:23:02.528 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.525+0000 7f2a05ffb700 1 --2- 172.21.15.77:0/555283100 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f29f80a0520 0x7f29f80a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:02.528 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.525+0000 7f2a05ffb700 1 -- 172.21.15.77:0/555283100 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f2a000ccef0 con 0x7f2a08102940 2023-12-01T01:23:02.528 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.525+0000 7f2a0cffa700 1 --2- 172.21.15.77:0/555283100 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f29f80a0520 0x7f29f80a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:02.528 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.525+0000 7f2a0cffa700 1 --2- 172.21.15.77:0/555283100 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f29f80a0520 0x7f29f80a29e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2a08101e00 tx=0x7f29fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:23:02.529 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.528+0000 7f2a05ffb700 1 -- 172.21.15.77:0/555283100 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2a00096bc0 con 0x7f2a08102940 2023-12-01T01:23:02.724 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.722+0000 7f2a0f25e700 1 -- 172.21.15.77:0/555283100 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2a0810c880 con 0x7f2a08102940 2023-12-01T01:23:02.724 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.722+0000 7f2a05ffb700 1 -- 172.21.15.77:0/555283100 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2a0009a1e0 con 0x7f2a08102940 2023-12-01T01:23:02.724 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 1 2023-12-01T01:23:02.727 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.725+0000 7f2a0f25e700 1 -- 172.21.15.77:0/555283100 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f29f80a0520 msgr2=0x7f29f80a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:02.727 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.725+0000 7f2a0f25e700 1 --2- 172.21.15.77:0/555283100 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f29f80a0520 0x7f29f80a29e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2a08101e00 tx=0x7f29fc009260 comp rx=0 tx=0).stop 2023-12-01T01:23:02.727 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.725+0000 7f2a0f25e700 1 -- 172.21.15.77:0/555283100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a08102940 msgr2=0x7f2a0819e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:02.727 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.725+0000 7f2a0f25e700 1 --2- 172.21.15.77:0/555283100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a08102940 0x7f2a0819e0e0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f2a00008ec0 tx=0x7f2a00008ef0 comp rx=0 tx=0).stop 2023-12-01T01:23:02.727 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.725+0000 7f2a0f25e700 1 -- 172.21.15.77:0/555283100 shutdown_connections 2023-12-01T01:23:02.727 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.725+0000 7f2a0f25e700 1 --2- 172.21.15.77:0/555283100 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f29f80a0520 0x7f29f80a29e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:02.727 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.725+0000 7f2a0f25e700 1 --2- 172.21.15.77:0/555283100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a08102940 0x7f2a0819e0e0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:02.727 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.725+0000 7f2a0f25e700 1 --2- 172.21.15.77:0/555283100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a08100db0 0x7f2a08196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:02.727 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.725+0000 7f2a0f25e700 1 --2- 172.21.15.77:0/555283100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a08101fa0 0x7f2a08197070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:02.727 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.725+0000 7f2a0f25e700 1 -- 172.21.15.77:0/555283100 >> 172.21.15.77:0/555283100 conn(0x7f2a080fc300 msgr2=0x7f2a08105410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:23:02.728 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.725+0000 7f2a0f25e700 1 -- 172.21.15.77:0/555283100 shutdown_connections 2023-12-01T01:23:02.728 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.725+0000 7f2a0f25e700 1 -- 172.21.15.77:0/555283100 wait complete. 2023-12-01T01:23:02.739 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-01T01:23:02.865 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.861+0000 7ff26232c700 1 -- 172.21.15.77:0/2860742267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff25c1018a0 msgr2=0x7ff25c103c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:02.865 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.861+0000 7ff26232c700 1 --2- 172.21.15.77:0/2860742267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff25c1018a0 0x7ff25c103c90 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff24c009990 tx=0x7ff24c009ca0 comp rx=0 tx=0).stop 2023-12-01T01:23:02.865 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.861+0000 7ff26232c700 1 -- 172.21.15.77:0/2860742267 shutdown_connections 2023-12-01T01:23:02.865 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.861+0000 7ff26232c700 1 --2- 172.21.15.77:0/2860742267 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff25c106b00 0x7ff25c108ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:02.865 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.861+0000 7ff26232c700 1 --2- 172.21.15.77:0/2860742267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff25c1018a0 0x7ff25c103c90 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:02.865 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.861+0000 7ff26232c700 1 --2- 172.21.15.77:0/2860742267 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff25c1041d0 0x7ff25c1065c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:02.865 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.862+0000 7ff26232c700 1 -- 172.21.15.77:0/2860742267 >> 172.21.15.77:0/2860742267 conn(0x7ff25c0fb1f0 msgr2=0x7ff25c0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:23:02.865 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.862+0000 7ff26232c700 1 -- 172.21.15.77:0/2860742267 shutdown_connections 2023-12-01T01:23:02.866 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.862+0000 7ff26232c700 1 -- 172.21.15.77:0/2860742267 wait complete. 2023-12-01T01:23:02.866 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.862+0000 7ff26232c700 1 Processor -- start 2023-12-01T01:23:02.866 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.862+0000 7ff26232c700 1 -- start start 2023-12-01T01:23:02.866 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.862+0000 7ff26232c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff25c1018a0 0x7ff25c196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:02.866 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.862+0000 7ff26232c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff25c1041d0 0x7ff25c197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:02.866 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.862+0000 7ff26232c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff25c106b00 0x7ff25c19e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:02.866 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.862+0000 7ff26232c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c19e640 con 0x7ff25c1041d0 2023-12-01T01:23:02.866 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.863+0000 7ff26232c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c19e780 con 0x7ff25c106b00 2023-12-01T01:23:02.866 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.863+0000 7ff26232c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c19e910 con 0x7ff25c1018a0 2023-12-01T01:23:02.866 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.863+0000 7ff25b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff25c1041d0 0x7ff25c197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:02.866 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.863+0000 7ff25b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff25c1041d0 0x7ff25c197000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33224/0 (socket says 172.21.15.77:33224) 2023-12-01T01:23:02.866 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.863+0000 7ff25b7fe700 1 -- 172.21.15.77:0/3951053885 learned_addr learned my addr 172.21.15.77:0/3951053885 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:23:02.866 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.863+0000 7ff2608c9700 1 --2- 172.21.15.77:0/3951053885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff25c106b00 0x7ff25c19e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:02.866 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.863+0000 7ff25bfff700 1 --2- 172.21.15.77:0/3951053885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff25c1018a0 0x7ff25c196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:02.867 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.863+0000 7ff25b7fe700 1 -- 172.21.15.77:0/3951053885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff25c1018a0 msgr2=0x7ff25c196ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:02.867 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.863+0000 7ff25b7fe700 1 --2- 172.21.15.77:0/3951053885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff25c1018a0 0x7ff25c196ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:02.867 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.863+0000 7ff25b7fe700 1 -- 172.21.15.77:0/3951053885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff25c106b00 msgr2=0x7ff25c19e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:02.867 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.863+0000 7ff25b7fe700 1 --2- 172.21.15.77:0/3951053885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff25c106b00 0x7ff25c19e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:02.867 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.863+0000 7ff25b7fe700 1 -- 172.21.15.77:0/3951053885 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff24c0095f0 con 0x7ff25c1041d0 2023-12-01T01:23:02.867 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.863+0000 7ff25b7fe700 1 --2- 172.21.15.77:0/3951053885 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff25c1041d0 0x7ff25c197000 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff25000e9a0 tx=0x7ff25000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:23:02.867 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.863+0000 7ff2597fa700 1 -- 172.21.15.77:0/3951053885 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff250009790 con 0x7ff25c1041d0 2023-12-01T01:23:02.867 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.863+0000 7ff2597fa700 1 -- 172.21.15.77:0/3951053885 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff250010430 con 0x7ff25c1041d0 2023-12-01T01:23:02.867 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.863+0000 7ff26232c700 1 -- 172.21.15.77:0/3951053885 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff25c19ebf0 con 0x7ff25c1041d0 2023-12-01T01:23:02.867 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.863+0000 7ff2597fa700 1 -- 172.21.15.77:0/3951053885 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff250018680 con 0x7ff25c1041d0 2023-12-01T01:23:02.867 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.863+0000 7ff26232c700 1 -- 172.21.15.77:0/3951053885 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff25c19f100 con 0x7ff25c1041d0 2023-12-01T01:23:02.868 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.864+0000 7ff26232c700 1 -- 172.21.15.77:0/3951053885 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff25c0fcd70 con 0x7ff25c1041d0 2023-12-01T01:23:02.868 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.865+0000 7ff2597fa700 1 -- 172.21.15.77:0/3951053885 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7ff2500105a0 con 0x7ff25c1041d0 2023-12-01T01:23:02.868 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.866+0000 7ff2597fa700 1 --2- 172.21.15.77:0/3951053885 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff2440a0520 0x7ff2440a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:02.868 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.866+0000 7ff2597fa700 1 -- 172.21.15.77:0/3951053885 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7ff250014070 con 0x7ff25c1041d0 2023-12-01T01:23:02.868 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.866+0000 7ff25bfff700 1 --2- 172.21.15.77:0/3951053885 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff2440a0520 0x7ff2440a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:02.868 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.866+0000 7ff25bfff700 1 --2- 172.21.15.77:0/3951053885 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff2440a0520 0x7ff2440a29e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff24c00b2e0 tx=0x7ff24c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:23:02.870 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:02.869+0000 7ff2597fa700 1 -- 172.21.15.77:0/3951053885 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff2500958b0 con 0x7ff25c1041d0 2023-12-01T01:23:03.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:02 smithi203 ceph-mon[103581]: pgmap v453: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 11 op/s 2023-12-01T01:23:03.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:02 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/3159835069' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:23:03.028 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:02 smithi137 ceph-mon[103312]: pgmap v453: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 11 op/s 2023-12-01T01:23:03.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:02 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/3159835069' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:23:03.060 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.058+0000 7ff26232c700 1 -- 172.21.15.77:0/3951053885 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff25c02ce00 con 0x7ff25c1041d0 2023-12-01T01:23:03.060 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.059+0000 7ff2597fa700 1 -- 172.21.15.77:0/3951053885 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff250098ed0 con 0x7ff25c1041d0 2023-12-01T01:23:03.060 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 32 2023-12-01T01:23:03.063 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.061+0000 7ff26232c700 1 -- 172.21.15.77:0/3951053885 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff2440a0520 msgr2=0x7ff2440a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:03.063 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.061+0000 7ff26232c700 1 --2- 172.21.15.77:0/3951053885 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff2440a0520 0x7ff2440a29e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff24c00b2e0 tx=0x7ff24c005fb0 comp rx=0 tx=0).stop 2023-12-01T01:23:03.063 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.061+0000 7ff26232c700 1 -- 172.21.15.77:0/3951053885 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff25c1041d0 msgr2=0x7ff25c197000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:03.063 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.061+0000 7ff26232c700 1 --2- 172.21.15.77:0/3951053885 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff25c1041d0 0x7ff25c197000 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff25000e9a0 tx=0x7ff25000ecb0 comp rx=0 tx=0).stop 2023-12-01T01:23:03.063 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.061+0000 7ff26232c700 1 -- 172.21.15.77:0/3951053885 shutdown_connections 2023-12-01T01:23:03.063 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.061+0000 7ff26232c700 1 --2- 172.21.15.77:0/3951053885 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff2440a0520 0x7ff2440a29e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:03.063 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.061+0000 7ff26232c700 1 --2- 172.21.15.77:0/3951053885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff25c106b00 0x7ff25c19e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:03.063 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.061+0000 7ff26232c700 1 --2- 172.21.15.77:0/3951053885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff25c1018a0 0x7ff25c196ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:03.063 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.061+0000 7ff26232c700 1 --2- 172.21.15.77:0/3951053885 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff25c1041d0 0x7ff25c197000 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:03.063 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.061+0000 7ff26232c700 1 -- 172.21.15.77:0/3951053885 >> 172.21.15.77:0/3951053885 conn(0x7ff25c0fb1f0 msgr2=0x7ff25c102510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:23:03.063 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.061+0000 7ff26232c700 1 -- 172.21.15.77:0/3951053885 shutdown_connections 2023-12-01T01:23:03.064 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.061+0000 7ff26232c700 1 -- 172.21.15.77:0/3951053885 wait complete. 2023-12-01T01:23:03.074 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:02 smithi077 ceph-mon[98867]: pgmap v453: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 11 op/s 2023-12-01T01:23:03.074 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:02 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/3159835069' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:23:03.075 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-01T01:23:03.199 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.198+0000 7f584e678700 1 -- 172.21.15.77:0/1675394874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5848101200 msgr2=0x7f5848101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:03.200 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.198+0000 7f584e678700 1 --2- 172.21.15.77:0/1675394874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5848101200 0x7f5848101620 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f5838009990 tx=0x7f5838009ca0 comp rx=0 tx=0).stop 2023-12-01T01:23:03.200 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.198+0000 7f584e678700 1 -- 172.21.15.77:0/1675394874 shutdown_connections 2023-12-01T01:23:03.200 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.198+0000 7f584e678700 1 --2- 172.21.15.77:0/1675394874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58481008d0 0x7f5848108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:03.200 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.198+0000 7f584e678700 1 --2- 172.21.15.77:0/1675394874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5848101200 0x7f5848101620 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:03.200 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.198+0000 7f584e678700 1 --2- 172.21.15.77:0/1675394874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58480fff30 0x7f5848100390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:03.200 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.198+0000 7f584e678700 1 -- 172.21.15.77:0/1675394874 >> 172.21.15.77:0/1675394874 conn(0x7f58480fa960 msgr2=0x7f58480fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:23:03.200 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.198+0000 7f584e678700 1 -- 172.21.15.77:0/1675394874 shutdown_connections 2023-12-01T01:23:03.200 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.198+0000 7f584e678700 1 -- 172.21.15.77:0/1675394874 wait complete. 2023-12-01T01:23:03.203 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.199+0000 7f584e678700 1 Processor -- start 2023-12-01T01:23:03.203 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.199+0000 7f584e678700 1 -- start start 2023-12-01T01:23:03.203 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.199+0000 7f584e678700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58480fff30 0x7f58481958d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:03.203 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.199+0000 7f584e678700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58481008d0 0x7f5848195e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:03.203 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.199+0000 7f584e678700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5848101200 0x7f584819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:03.203 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.199+0000 7f584e678700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584819d450 con 0x7f58481008d0 2023-12-01T01:23:03.204 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.199+0000 7f584e678700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584819d590 con 0x7f58480fff30 2023-12-01T01:23:03.204 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.199+0000 7f584e678700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584819d720 con 0x7f5848101200 2023-12-01T01:23:03.204 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.199+0000 7f583ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58481008d0 0x7f5848195e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:03.204 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.199+0000 7f583ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58481008d0 0x7f5848195e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33238/0 (socket says 172.21.15.77:33238) 2023-12-01T01:23:03.204 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.199+0000 7f583ffff700 1 -- 172.21.15.77:0/560533084 learned_addr learned my addr 172.21.15.77:0/560533084 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:23:03.205 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.199+0000 7f5847fff700 1 --2- 172.21.15.77:0/560533084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58480fff30 0x7f58481958d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:03.205 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.199+0000 7f584cc15700 1 --2- 172.21.15.77:0/560533084 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5848101200 0x7f584819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:03.205 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.199+0000 7f583ffff700 1 -- 172.21.15.77:0/560533084 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5848101200 msgr2=0x7f584819ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:03.205 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.199+0000 7f583ffff700 1 --2- 172.21.15.77:0/560533084 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5848101200 0x7f584819ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:03.205 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.199+0000 7f583ffff700 1 -- 172.21.15.77:0/560533084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58480fff30 msgr2=0x7f58481958d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:03.205 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.199+0000 7f583ffff700 1 --2- 172.21.15.77:0/560533084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58480fff30 0x7f58481958d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:03.205 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.199+0000 7f583ffff700 1 -- 172.21.15.77:0/560533084 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58380095f0 con 0x7f58481008d0 2023-12-01T01:23:03.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.200+0000 7f583ffff700 1 --2- 172.21.15.77:0/560533084 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58481008d0 0x7f5848195e10 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f583000e9a0 tx=0x7f583000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:23:03.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.200+0000 7f5845ffb700 1 -- 172.21.15.77:0/560533084 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5830009790 con 0x7f58481008d0 2023-12-01T01:23:03.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.200+0000 7f5845ffb700 1 -- 172.21.15.77:0/560533084 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f583000cbd0 con 0x7f58481008d0 2023-12-01T01:23:03.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.200+0000 7f584e678700 1 -- 172.21.15.77:0/560533084 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f584819da00 con 0x7f58481008d0 2023-12-01T01:23:03.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.200+0000 7f5845ffb700 1 -- 172.21.15.77:0/560533084 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5830018680 con 0x7f58481008d0 2023-12-01T01:23:03.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.200+0000 7f584e678700 1 -- 172.21.15.77:0/560533084 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f584819df10 con 0x7f58481008d0 2023-12-01T01:23:03.206 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.201+0000 7f584e678700 1 -- 172.21.15.77:0/560533084 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5848106cd0 con 0x7f58481008d0 2023-12-01T01:23:03.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.202+0000 7f5845ffb700 1 -- 172.21.15.77:0/560533084 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f583000cd40 con 0x7f58481008d0 2023-12-01T01:23:03.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.203+0000 7f5845ffb700 1 --2- 172.21.15.77:0/560533084 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f58340a0520 0x7f58340a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:03.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.203+0000 7f5847fff700 1 --2- 172.21.15.77:0/560533084 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f58340a0520 0x7f58340a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:03.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.203+0000 7f5845ffb700 1 -- 172.21.15.77:0/560533084 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f5830014070 con 0x7f58481008d0 2023-12-01T01:23:03.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.203+0000 7f5847fff700 1 --2- 172.21.15.77:0/560533084 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f58340a0520 0x7f58340a29e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f583800b680 tx=0x7f5838005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:23:03.207 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.207+0000 7f5845ffb700 1 -- 172.21.15.77:0/560533084 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5830095ad0 con 0x7f58481008d0 2023-12-01T01:23:03.399 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.397+0000 7f584e678700 1 -- 172.21.15.77:0/560533084 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f584802cec0 con 0x7f58481008d0 2023-12-01T01:23:03.399 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.398+0000 7f5845ffb700 1 -- 172.21.15.77:0/560533084 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7f58300990f0 con 0x7f58481008d0 2023-12-01T01:23:03.399 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 64 2023-12-01T01:23:03.401 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.400+0000 7f584e678700 1 -- 172.21.15.77:0/560533084 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f58340a0520 msgr2=0x7f58340a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:03.402 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.400+0000 7f584e678700 1 --2- 172.21.15.77:0/560533084 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f58340a0520 0x7f58340a29e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f583800b680 tx=0x7f5838005fd0 comp rx=0 tx=0).stop 2023-12-01T01:23:03.402 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.400+0000 7f584e678700 1 -- 172.21.15.77:0/560533084 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58481008d0 msgr2=0x7f5848195e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:03.402 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.400+0000 7f584e678700 1 --2- 172.21.15.77:0/560533084 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58481008d0 0x7f5848195e10 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f583000e9a0 tx=0x7f583000ecb0 comp rx=0 tx=0).stop 2023-12-01T01:23:03.402 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.400+0000 7f584e678700 1 -- 172.21.15.77:0/560533084 shutdown_connections 2023-12-01T01:23:03.402 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.400+0000 7f584e678700 1 --2- 172.21.15.77:0/560533084 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f58340a0520 0x7f58340a29e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:03.402 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.400+0000 7f584e678700 1 --2- 172.21.15.77:0/560533084 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5848101200 0x7f584819ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:03.402 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.400+0000 7f584e678700 1 --2- 172.21.15.77:0/560533084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58480fff30 0x7f58481958d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:03.402 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.400+0000 7f584e678700 1 --2- 172.21.15.77:0/560533084 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58481008d0 0x7f5848195e10 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:03.402 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.400+0000 7f584e678700 1 -- 172.21.15.77:0/560533084 >> 172.21.15.77:0/560533084 conn(0x7f58480fa960 msgr2=0x7f58480fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:23:03.402 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.400+0000 7f584e678700 1 -- 172.21.15.77:0/560533084 shutdown_connections 2023-12-01T01:23:03.402 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.400+0000 7f584e678700 1 -- 172.21.15.77:0/560533084 wait complete. 2023-12-01T01:23:03.414 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-01T01:23:03.536 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.534+0000 7f918da9d700 1 -- 172.21.15.77:0/2963834989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9188102040 msgr2=0x7f91881024a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:03.536 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.534+0000 7f918da9d700 1 --2- 172.21.15.77:0/2963834989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9188102040 0x7f91881024a0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f9170009990 tx=0x7f9170009ca0 comp rx=0 tx=0).stop 2023-12-01T01:23:03.536 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.534+0000 7f918da9d700 1 -- 172.21.15.77:0/2963834989 shutdown_connections 2023-12-01T01:23:03.536 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.534+0000 7f918da9d700 1 --2- 172.21.15.77:0/2963834989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91881029e0 0x7f9188109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:03.536 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.534+0000 7f918da9d700 1 --2- 172.21.15.77:0/2963834989 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91880ff070 0x7f91880ff490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:03.536 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.534+0000 7f918da9d700 1 --2- 172.21.15.77:0/2963834989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9188102040 0x7f91881024a0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:03.536 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.534+0000 7f918da9d700 1 -- 172.21.15.77:0/2963834989 >> 172.21.15.77:0/2963834989 conn(0x7f91880fa980 msgr2=0x7f91880fcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:23:03.536 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.534+0000 7f918da9d700 1 -- 172.21.15.77:0/2963834989 shutdown_connections 2023-12-01T01:23:03.536 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.534+0000 7f918da9d700 1 -- 172.21.15.77:0/2963834989 wait complete. 2023-12-01T01:23:03.536 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.535+0000 7f918da9d700 1 Processor -- start 2023-12-01T01:23:03.538 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.535+0000 7f918da9d700 1 -- start start 2023-12-01T01:23:03.538 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.535+0000 7f918da9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91880ff070 0x7f9188196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:03.539 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.535+0000 7f918da9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9188102040 0x7f9188196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:03.539 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.535+0000 7f918da9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91881029e0 0x7f918819e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:03.539 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.535+0000 7f918da9d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918819e5d0 con 0x7f9188102040 2023-12-01T01:23:03.539 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.535+0000 7f918da9d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918819e710 con 0x7f91881029e0 2023-12-01T01:23:03.539 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.535+0000 7f918da9d700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918819e8a0 con 0x7f91880ff070 2023-12-01T01:23:03.539 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.535+0000 7f9186ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9188102040 0x7f9188196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:03.539 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.535+0000 7f9186ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9188102040 0x7f9188196f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33252/0 (socket says 172.21.15.77:33252) 2023-12-01T01:23:03.539 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.535+0000 7f9186ffd700 1 -- 172.21.15.77:0/2370796192 learned_addr learned my addr 172.21.15.77:0/2370796192 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:23:03.539 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.535+0000 7f91877fe700 1 --2- 172.21.15.77:0/2370796192 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91880ff070 0x7f9188196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:03.539 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.535+0000 7f9187fff700 1 --2- 172.21.15.77:0/2370796192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91881029e0 0x7f918819e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:03.539 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.535+0000 7f9186ffd700 1 -- 172.21.15.77:0/2370796192 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91880ff070 msgr2=0x7f9188196a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:03.539 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.535+0000 7f9186ffd700 1 --2- 172.21.15.77:0/2370796192 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91880ff070 0x7f9188196a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:03.539 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.535+0000 7f9186ffd700 1 -- 172.21.15.77:0/2370796192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91881029e0 msgr2=0x7f918819e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:03.540 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.535+0000 7f9186ffd700 1 --2- 172.21.15.77:0/2370796192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91881029e0 0x7f918819e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:03.540 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.535+0000 7f9186ffd700 1 -- 172.21.15.77:0/2370796192 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91700095f0 con 0x7f9188102040 2023-12-01T01:23:03.540 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.535+0000 7f9187fff700 1 --2- 172.21.15.77:0/2370796192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91881029e0 0x7f918819e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-01T01:23:03.540 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.536+0000 7f9186ffd700 1 --2- 172.21.15.77:0/2370796192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9188102040 0x7f9188196f90 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f9170000c00 tx=0x7f917000f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:23:03.540 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.536+0000 7f9184ff9700 1 -- 172.21.15.77:0/2370796192 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917001d070 con 0x7f9188102040 2023-12-01T01:23:03.540 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.536+0000 7f918da9d700 1 -- 172.21.15.77:0/2370796192 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f918819eb20 con 0x7f9188102040 2023-12-01T01:23:03.540 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.536+0000 7f9184ff9700 1 -- 172.21.15.77:0/2370796192 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f917000fb50 con 0x7f9188102040 2023-12-01T01:23:03.540 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.536+0000 7f9184ff9700 1 -- 172.21.15.77:0/2370796192 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9170017650 con 0x7f9188102040 2023-12-01T01:23:03.540 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.536+0000 7f918da9d700 1 -- 172.21.15.77:0/2370796192 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f918819f020 con 0x7f9188102040 2023-12-01T01:23:03.540 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.537+0000 7f918da9d700 1 -- 172.21.15.77:0/2370796192 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9188107dd0 con 0x7f9188102040 2023-12-01T01:23:03.541 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.538+0000 7f9184ff9700 1 -- 172.21.15.77:0/2370796192 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f917000fcc0 con 0x7f9188102040 2023-12-01T01:23:03.541 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.538+0000 7f9184ff9700 1 --2- 172.21.15.77:0/2370796192 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f91740a0520 0x7f91740a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:03.541 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.538+0000 7f91877fe700 1 --2- 172.21.15.77:0/2370796192 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f91740a0520 0x7f91740a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:03.541 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.538+0000 7f9184ff9700 1 -- 172.21.15.77:0/2370796192 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f91700ccc70 con 0x7f9188102040 2023-12-01T01:23:03.541 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.539+0000 7f91877fe700 1 --2- 172.21.15.77:0/2370796192 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f91740a0520 0x7f91740a29e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9188068ee0 tx=0x7f917800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:23:03.544 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.542+0000 7f9184ff9700 1 -- 172.21.15.77:0/2370796192 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9170096940 con 0x7f9188102040 2023-12-01T01:23:03.693 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:03 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/555283100' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-01T01:23:03.694 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:03 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/3951053885' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-01T01:23:03.754 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.751+0000 7f918da9d700 1 -- 172.21.15.77:0/2370796192 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f918802cde0 con 0x7f9188102040 2023-12-01T01:23:03.754 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.752+0000 7f9184ff9700 1 -- 172.21.15.77:0/2370796192 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v84) v1 ==== 81+0+590 (secure 0 0 0) 0x7f91700a0070 con 0x7f9188102040 2023-12-01T01:23:03.754 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/2260035254 2023-12-02T01:10:24.191835+0000 2023-12-01T01:23:03.754 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6800/1456621747 2023-12-02T01:10:24.191835+0000 2023-12-01T01:23:03.754 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/219838828 2023-12-02T01:10:24.191835+0000 2023-12-01T01:23:03.754 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6801/1456621747 2023-12-02T01:10:24.191835+0000 2023-12-01T01:23:03.754 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/2937149266 2023-12-02T01:10:02.232948+0000 2023-12-01T01:23:03.754 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/807520018 2023-12-02T01:10:02.232948+0000 2023-12-01T01:23:03.755 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6801/1661788026 2023-12-02T01:10:02.232948+0000 2023-12-01T01:23:03.755 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6800/1661788026 2023-12-02T01:10:02.232948+0000 2023-12-01T01:23:03.755 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/3551282473 2023-12-02T01:10:24.191835+0000 2023-12-01T01:23:03.755 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/1202222558 2023-12-02T01:10:02.232948+0000 2023-12-01T01:23:03.756 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.754+0000 7f918da9d700 1 -- 172.21.15.77:0/2370796192 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f91740a0520 msgr2=0x7f91740a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:03.756 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.754+0000 7f918da9d700 1 --2- 172.21.15.77:0/2370796192 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f91740a0520 0x7f91740a29e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9188068ee0 tx=0x7f917800a380 comp rx=0 tx=0).stop 2023-12-01T01:23:03.756 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.754+0000 7f918da9d700 1 -- 172.21.15.77:0/2370796192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9188102040 msgr2=0x7f9188196f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:03.757 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.754+0000 7f918da9d700 1 --2- 172.21.15.77:0/2370796192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9188102040 0x7f9188196f90 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f9170000c00 tx=0x7f917000f710 comp rx=0 tx=0).stop 2023-12-01T01:23:03.757 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.754+0000 7f918da9d700 1 -- 172.21.15.77:0/2370796192 shutdown_connections 2023-12-01T01:23:03.757 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.754+0000 7f918da9d700 1 --2- 172.21.15.77:0/2370796192 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f91740a0520 0x7f91740a29e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:03.757 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.754+0000 7f918da9d700 1 --2- 172.21.15.77:0/2370796192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f91881029e0 0x7f918819e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:03.757 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.754+0000 7f918da9d700 1 --2- 172.21.15.77:0/2370796192 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91880ff070 0x7f9188196a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:03.757 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.754+0000 7f918da9d700 1 --2- 172.21.15.77:0/2370796192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9188102040 0x7f9188196f90 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:03.757 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.754+0000 7f918da9d700 1 -- 172.21.15.77:0/2370796192 >> 172.21.15.77:0/2370796192 conn(0x7f91880fa980 msgr2=0x7f91880fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:23:03.757 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.754+0000 7f918da9d700 1 -- 172.21.15.77:0/2370796192 shutdown_connections 2023-12-01T01:23:03.758 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:03.754+0000 7f918da9d700 1 -- 172.21.15.77:0/2370796192 wait complete. 2023-12-01T01:23:03.758 INFO:teuthology.orchestra.run.smithi077.stderr:listed 10 entries 2023-12-01T01:23:03.768 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-01T01:23:03.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:23:03.768 DEBUG:teuthology.orchestra.run.smithi077:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-01T01:23:03.899 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi077.front.sepia.ceph.com 2023-12-01T01:23:03.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:23:03.900 DEBUG:teuthology.orchestra.run.smithi077:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-01T01:23:04.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:23:04.008 DEBUG:teuthology.orchestra.run.smithi077:> ip netns list 2023-12-01T01:23:04.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:03 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/555283100' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-01T01:23:04.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:03 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/3951053885' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-01T01:23:04.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:03 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/560533084' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-01T01:23:04.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:03 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/555283100' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-01T01:23:04.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:03 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/3951053885' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-01T01:23:04.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:03 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/560533084' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-01T01:23:04.055 INFO:teuthology.orchestra.run.smithi077.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-01T01:23:04.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:23:04.055 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-01T01:23:04.140 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:03 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/560533084' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-01T01:23:04.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:23:04.163 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link delete ceph-brx 2023-12-01T01:23:04.271 INFO:tasks.kclient:Unmounting kernel clients... 2023-12-01T01:23:04.271 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2023-12-01T01:23:04.271 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-01T01:23:04.289 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=14783 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/d73725ed462f9eeafa2f0d689dd984d6f8efafce404561639bf2c0422bc77209/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/VNIEINUIRPRRPY6TCXJWPBK2WY:/var/lib/containers/storage/overlay/l/V7ICVUB6I4K7FNRKZ73QNOP6DG,upperdir=/var/lib/containers/storage/overlay/d73725ed462f9eeafa2f0d689dd984d6f8efafce404561639bf2c0422bc77209/diff,workdir=/var/lib/containers/storage/overlay/d73725ed462f9eeafa2f0d689dd984d6f8efafce404561639bf2c0422bc77209/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/9b41f417be59e2bff2485c9a4c56aea4d515ce979fe6eb85849672f4eb84d605/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/VNIEINUIRPRRPY6TCXJWPBK2WY:/var/lib/containers/storage/overlay/l/V7ICVUB6I4K7FNRKZ73QNOP6DG,upperdir=/var/lib/containers/storage/overlay/9b41f417be59e2bff2485c9a4c56aea4d515ce979fe6eb85849672f4eb84d605/diff,workdir=/var/lib/containers/storage/overlay/9b41f417be59e2bff2485c9a4c56aea4d515ce979fe6eb85849672f4eb84d605/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0 overlay /var/lib/containers/storage/overlay/c2d8c4a8e33cd3971621ddd0a1426f7be459f81e57858f04030d511690130535/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/VNIEINUIRPRRPY6TCXJWPBK2WY:/var/lib/containers/storage/overlay/l/V7ICVUB6I4K7FNRKZ73QNOP6DG,upperdir=/var/lib/containers/storage/overlay/c2d8c4a8e33cd3971621ddd0a1426f7be459f81e57858f04030d511690130535/diff,workdir=/var/lib/containers/storage/overlay/c2d8c4a8e33cd3971621ddd0a1426f7be459f81e57858f04030d511690130535/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/d787a0b0073064130d29d02e825220d5bedf5dd7107938eddb46208ec02ec7ef/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/VNIEINUIRPRRPY6TCXJWPBK2WY:/var/lib/containers/storage/overlay/l/V7ICVUB6I4K7FNRKZ73QNOP6DG,upperdir=/var/lib/containers/storage/overlay/d787a0b0073064130d29d02e825220d5bedf5dd7107938eddb46208ec02ec7ef/diff,workdir=/var/lib/containers/storage/overlay/d787a0b0073064130d29d02e825220d5bedf5dd7107938eddb46208ec02ec7ef/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/9ac09f72ef4c3e21fbdecfa06524e14506ce491233094c3aa1314151369e8397/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/VNIEINUIRPRRPY6TCXJWPBK2WY:/var/lib/containers/storage/overlay/l/V7ICVUB6I4K7FNRKZ73QNOP6DG,upperdir=/var/lib/containers/storage/overlay/9ac09f72ef4c3e21fbdecfa06524e14506ce491233094c3aa1314151369e8397/diff,workdir=/var/lib/containers/storage/overlay/9ac09f72ef4c3e21fbdecfa06524e14506ce491233094c3aa1314151369e8397/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/cc29f6e02edf88e08a88d3b17f267a17d74fdd11ef09167619b819083e65dcc5/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/VNIEINUIRPRRPY6TCXJWPBK2WY:/var/lib/containers/storage/overlay/l/V7ICVUB6I4K7FNRKZ73QNOP6DG,upperdir=/var/lib/containers/storage/overlay/cc29f6e02edf88e08a88d3b17f267a17d74fdd11ef09167619b819083e65dcc5/diff,workdir=/var/lib/containers/storage/overlay/cc29f6e02edf88e08a88d3b17f267a17d74fdd11ef09167619b819083e65dcc5/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/a71985e79d2592a2a051540d0abfa9487b7a2d666bd2d9b22b536d7637d2e816/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/VNIEINUIRPRRPY6TCXJWPBK2WY:/var/lib/containers/storage/overlay/l/V7ICVUB6I4K7FNRKZ73QNOP6DG,upperdir=/var/lib/containers/storage/overlay/a71985e79d2592a2a051540d0abfa9487b7a2d666bd2d9b22b536d7637d2e816/diff,workdir=/var/lib/containers/storage/overlay/a71985e79d2592a2a051540d0abfa9487b7a2d666bd2d9b22b536d7637d2e816/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/e01cf77158cb465e4fd057d21d6dda74d471d8de91a47fdf5c6ad4104f7a49b3/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/VNIEINUIRPRRPY6TCXJWPBK2WY:/var/lib/containers/storage/overlay/l/V7ICVUB6I4K7FNRKZ73QNOP6DG,upperdir=/var/lib/containers/storage/overlay/e01cf77158cb465e4fd057d21d6dda74d471d8de91a47fdf5c6ad4104f7a49b3/diff,workdir=/var/lib/containers/storage/overlay/e01cf77158cb465e4fd057d21d6dda74d471d8de91a47fdf5c6ad4104f7a49b3/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/09d07b7979f26819988e7609d97511ca9b1564577e8a8b71084a466c89e91acd/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/VNIEINUIRPRRPY6TCXJWPBK2WY:/var/lib/containers/storage/overlay/l/V7ICVUB6I4K7FNRKZ73QNOP6DG,upperdir=/var/lib/containers/storage/overlay/09d07b7979f26819988e7609d97511ca9b1564577e8a8b71084a466c89e91acd/diff,workdir=/var/lib/containers/storage/overlay/09d07b7979f26819988e7609d97511ca9b1564577e8a8b71084a466c89e91acd/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/be436d8b742c80a5234ea162549f4ea812ef1437bb16ee4ce82da6c83ec133e3/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/VNIEINUIRPRRPY6TCXJWPBK2WY:/var/lib/containers/storage/overlay/l/V7ICVUB6I4K7FNRKZ73QNOP6DG,upperdir=/var/lib/containers/storage/overlay/be436d8b742c80a5234ea162549f4ea812ef1437bb16ee4ce82da6c83ec133e3/diff,workdir=/var/lib/containers/storage/overlay/be436d8b742c80a5234ea162549f4ea812ef1437bb16ee4ce82da6c83ec133e3/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 2023-12-01T01:23:04.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:23:04.289 DEBUG:teuthology.orchestra.run.smithi077:> ip netns list 2023-12-01T01:23:04.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-01T01:23:04.347 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link delete ceph-brx 2023-12-01T01:23:04.417 INFO:teuthology.orchestra.run.smithi077.stderr:Cannot find device "ceph-brx" 2023-12-01T01:23:04.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-01T01:23:04.419 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-12-01T01:23:04.429 INFO:tasks.cephadm:Teardown begin 2023-12-01T01:23:04.429 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-01T01:23:04.430 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:23:04.584 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.581+0000 7f73c57e9700 1 -- 172.21.15.77:0/1468253676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f73c0106aa0 msgr2=0x7f73c0108f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:04.584 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.581+0000 7f73c57e9700 1 --2- 172.21.15.77:0/1468253676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f73c0106aa0 0x7f73c0108f10 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f73b800b0a0 tx=0x7f73b800b3b0 comp rx=0 tx=0).stop 2023-12-01T01:23:04.584 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.581+0000 7f73c57e9700 1 -- 172.21.15.77:0/1468253676 shutdown_connections 2023-12-01T01:23:04.585 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.581+0000 7f73c57e9700 1 --2- 172.21.15.77:0/1468253676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f73c0106aa0 0x7f73c0108f10 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:04.585 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.581+0000 7f73c57e9700 1 --2- 172.21.15.77:0/1468253676 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73c0101840 0x7f73c0103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:04.585 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.581+0000 7f73c57e9700 1 --2- 172.21.15.77:0/1468253676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f73c0104170 0x7f73c0106560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:04.585 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.581+0000 7f73c57e9700 1 -- 172.21.15.77:0/1468253676 >> 172.21.15.77:0/1468253676 conn(0x7f73c00fb210 msgr2=0x7f73c00fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:23:04.585 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.581+0000 7f73c57e9700 1 -- 172.21.15.77:0/1468253676 shutdown_connections 2023-12-01T01:23:04.585 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.581+0000 7f73c57e9700 1 -- 172.21.15.77:0/1468253676 wait complete. 2023-12-01T01:23:04.585 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.581+0000 7f73c57e9700 1 Processor -- start 2023-12-01T01:23:04.585 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.582+0000 7f73c57e9700 1 -- start start 2023-12-01T01:23:04.585 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.582+0000 7f73c57e9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73c0101840 0x7f73c01969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:04.585 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.582+0000 7f73c57e9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f73c0104170 0x7f73c0196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:04.585 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.582+0000 7f73c57e9700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f73c0106aa0 0x7f73c019dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:04.585 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.582+0000 7f73c57e9700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c019e570 con 0x7f73c0104170 2023-12-01T01:23:04.585 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.582+0000 7f73c57e9700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c019e6b0 con 0x7f73c0106aa0 2023-12-01T01:23:04.585 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.582+0000 7f73c57e9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c019e840 con 0x7f73c0101840 2023-12-01T01:23:04.586 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.582+0000 7f73be7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f73c0104170 0x7f73c0196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:04.586 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.582+0000 7f73be7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f73c0104170 0x7f73c0196f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33266/0 (socket says 172.21.15.77:33266) 2023-12-01T01:23:04.586 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.582+0000 7f73be7fc700 1 -- 172.21.15.77:0/3021104230 learned_addr learned my addr 172.21.15.77:0/3021104230 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:23:04.586 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.582+0000 7f73beffd700 1 --2- 172.21.15.77:0/3021104230 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73c0101840 0x7f73c01969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:04.586 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.582+0000 7f73bf7fe700 1 --2- 172.21.15.77:0/3021104230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f73c0106aa0 0x7f73c019dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:04.586 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.582+0000 7f73be7fc700 1 -- 172.21.15.77:0/3021104230 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73c0101840 msgr2=0x7f73c01969f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:04.586 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.582+0000 7f73be7fc700 1 --2- 172.21.15.77:0/3021104230 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73c0101840 0x7f73c01969f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:04.586 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.582+0000 7f73be7fc700 1 -- 172.21.15.77:0/3021104230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f73c0106aa0 msgr2=0x7f73c019dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:04.586 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.582+0000 7f73be7fc700 1 --2- 172.21.15.77:0/3021104230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f73c0106aa0 0x7f73c019dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:04.586 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.582+0000 7f73be7fc700 1 -- 172.21.15.77:0/3021104230 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b8009d00 con 0x7f73c0104170 2023-12-01T01:23:04.586 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.582+0000 7f73be7fc700 1 --2- 172.21.15.77:0/3021104230 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f73c0104170 0x7f73c0196f30 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f73b000d950 tx=0x7f73b000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:23:04.586 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.582+0000 7f73b7fff700 1 -- 172.21.15.77:0/3021104230 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b000f890 con 0x7f73c0104170 2023-12-01T01:23:04.586 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.583+0000 7f73b7fff700 1 -- 172.21.15.77:0/3021104230 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73b0010460 con 0x7f73c0104170 2023-12-01T01:23:04.586 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.583+0000 7f73c57e9700 1 -- 172.21.15.77:0/3021104230 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73c019eb20 con 0x7f73c0104170 2023-12-01T01:23:04.587 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.583+0000 7f73b7fff700 1 -- 172.21.15.77:0/3021104230 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b000e610 con 0x7f73c0104170 2023-12-01T01:23:04.587 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.583+0000 7f73c57e9700 1 -- 172.21.15.77:0/3021104230 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73c019f030 con 0x7f73c0104170 2023-12-01T01:23:04.587 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.583+0000 7f73c57e9700 1 -- 172.21.15.77:0/3021104230 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73c00fcd90 con 0x7f73c0104170 2023-12-01T01:23:04.588 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.584+0000 7f73b7fff700 1 -- 172.21.15.77:0/3021104230 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f73b00108d0 con 0x7f73c0104170 2023-12-01T01:23:04.588 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.585+0000 7f73b7fff700 1 --2- 172.21.15.77:0/3021104230 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f73ac0a0570 0x7f73ac0a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:04.588 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.585+0000 7f73beffd700 1 --2- 172.21.15.77:0/3021104230 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f73ac0a0570 0x7f73ac0a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:04.588 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.585+0000 7f73b7fff700 1 -- 172.21.15.77:0/3021104230 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f73b00cb740 con 0x7f73c0104170 2023-12-01T01:23:04.588 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.586+0000 7f73beffd700 1 --2- 172.21.15.77:0/3021104230 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f73ac0a0570 0x7f73ac0a2a30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f73a800ac50 tx=0x7f73a800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:23:04.590 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.589+0000 7f73b7fff700 1 -- 172.21.15.77:0/3021104230 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f73b00952e0 con 0x7f73c0104170 2023-12-01T01:23:04.700 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:04 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/2370796192' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-01T01:23:04.700 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:04 smithi077 ceph-mon[98867]: pgmap v454: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2023-12-01T01:23:04.829 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.828+0000 7f73c57e9700 1 -- 172.21.15.77:0/3021104230 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f73c002cc70 con 0x7f73c0104170 2023-12-01T01:23:04.830 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.829+0000 7f73b7fff700 1 -- 172.21.15.77:0/3021104230 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+10866 (secure 0 0 0) 0x7f73b0016090 con 0x7f73c0104170 2023-12-01T01:23:04.830 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:23:04.830 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1435145620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1435145620},{"type":"v1","addr":"172.21.15.77:6835","nonce":1435145620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/841838908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":841838908},{"type":"v1","addr":"172.21.15.77:6837","nonce":841838908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-01T01:16:25.345276+0000","modified":"2023-12-01T01:22:25.411862+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24496,"mds_2":24475,"mds_3":24473,"mds_4":14649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/3726913446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3726913446},{"type":"v1","addr":"172.21.15.77:6839","nonce":3726913446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14649":{"gid":14649,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":5,"addr":"172.21.15.77:6841/3023184965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3023184965},{"type":"v1","addr":"172.21.15.77:6841","nonce":3023184965}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/2249638445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2249638445},{"type":"v1","addr":"172.21.15.137:6833","nonce":2249638445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6833/1201599774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1201599774},{"type":"v1","addr":"172.21.15.203:6833","nonce":1201599774}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3935354878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3935354878},{"type":"v1","addr":"172.21.15.137:6835","nonce":3935354878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3237538176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3237538176},{"type":"v1","addr":"172.21.15.203:6835","nonce":3237538176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3518881867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3518881867},{"type":"v1","addr":"172.21.15.137:6837","nonce":3518881867}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":46,"addr":"172.21.15.203:6837/2887080977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2887080977},{"type":"v1","addr":"172.21.15.203:6837","nonce":2887080977}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":44,"addr":"172.21.15.203:6839/2888221613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2888221613},{"type":"v1","addr":"172.21.15.203:6839","nonce":2888221613}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/1099966660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1099966660},{"type":"v1","addr":"172.21.15.137:6839","nonce":1099966660}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-01T01:23:04.832 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.831+0000 7f73c57e9700 1 -- 172.21.15.77:0/3021104230 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f73ac0a0570 msgr2=0x7f73ac0a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:04.832 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.831+0000 7f73c57e9700 1 --2- 172.21.15.77:0/3021104230 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f73ac0a0570 0x7f73ac0a2a30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f73a800ac50 tx=0x7f73a800a380 comp rx=0 tx=0).stop 2023-12-01T01:23:04.832 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.831+0000 7f73c57e9700 1 -- 172.21.15.77:0/3021104230 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f73c0104170 msgr2=0x7f73c0196f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:04.832 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.831+0000 7f73c57e9700 1 --2- 172.21.15.77:0/3021104230 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f73c0104170 0x7f73c0196f30 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f73b000d950 tx=0x7f73b000dc60 comp rx=0 tx=0).stop 2023-12-01T01:23:04.833 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.831+0000 7f73c57e9700 1 -- 172.21.15.77:0/3021104230 shutdown_connections 2023-12-01T01:23:04.833 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.831+0000 7f73c57e9700 1 --2- 172.21.15.77:0/3021104230 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f73ac0a0570 0x7f73ac0a2a30 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:04.833 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.831+0000 7f73c57e9700 1 --2- 172.21.15.77:0/3021104230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f73c0106aa0 0x7f73c019dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:04.833 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.831+0000 7f73c57e9700 1 --2- 172.21.15.77:0/3021104230 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73c0101840 0x7f73c01969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:04.833 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.831+0000 7f73c57e9700 1 --2- 172.21.15.77:0/3021104230 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f73c0104170 0x7f73c0196f30 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:04.833 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.831+0000 7f73c57e9700 1 -- 172.21.15.77:0/3021104230 >> 172.21.15.77:0/3021104230 conn(0x7f73c00fb210 msgr2=0x7f73c01024b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:23:04.833 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.831+0000 7f73c57e9700 1 -- 172.21.15.77:0/3021104230 shutdown_connections 2023-12-01T01:23:04.833 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.831+0000 7f73c57e9700 1 -- 172.21.15.77:0/3021104230 wait complete. 2023-12-01T01:23:04.833 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 41 2023-12-01T01:23:04.844 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-01T01:23:04.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.967+0000 7fa1f56f9700 1 -- 172.21.15.77:0/842626687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1f0101840 msgr2=0x7fa1f0103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:04.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.967+0000 7fa1f56f9700 1 --2- 172.21.15.77:0/842626687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1f0101840 0x7fa1f0103c30 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8009990 tx=0x7fa1d8009ca0 comp rx=0 tx=0).stop 2023-12-01T01:23:04.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.967+0000 7fa1f56f9700 1 -- 172.21.15.77:0/842626687 shutdown_connections 2023-12-01T01:23:04.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.967+0000 7fa1f56f9700 1 --2- 172.21.15.77:0/842626687 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1f0106aa0 0x7fa1f0108f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:04.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.967+0000 7fa1f56f9700 1 --2- 172.21.15.77:0/842626687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1f0101840 0x7fa1f0103c30 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:04.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.967+0000 7fa1f56f9700 1 --2- 172.21.15.77:0/842626687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1f0104170 0x7fa1f0106560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:04.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.967+0000 7fa1f56f9700 1 -- 172.21.15.77:0/842626687 >> 172.21.15.77:0/842626687 conn(0x7fa1f00fb210 msgr2=0x7fa1f00fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:23:04.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.967+0000 7fa1f56f9700 1 -- 172.21.15.77:0/842626687 shutdown_connections 2023-12-01T01:23:04.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.967+0000 7fa1f56f9700 1 -- 172.21.15.77:0/842626687 wait complete. 2023-12-01T01:23:04.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.967+0000 7fa1f56f9700 1 Processor -- start 2023-12-01T01:23:04.969 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.968+0000 7fa1f56f9700 1 -- start start 2023-12-01T01:23:04.971 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.968+0000 7fa1f56f9700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1f0101840 0x7fa1f01969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:04.971 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.968+0000 7fa1f56f9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1f0104170 0x7fa1f0196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:04.971 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.968+0000 7fa1f56f9700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1f0106aa0 0x7fa1f019df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:04.972 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.968+0000 7fa1f56f9700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f019e540 con 0x7fa1f0104170 2023-12-01T01:23:04.972 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.968+0000 7fa1f56f9700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f019e680 con 0x7fa1f0101840 2023-12-01T01:23:04.972 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.968+0000 7fa1f56f9700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f019e810 con 0x7fa1f0106aa0 2023-12-01T01:23:04.972 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.968+0000 7fa1ee7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1f0104170 0x7fa1f0196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:04.972 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.968+0000 7fa1ee7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1f0104170 0x7fa1f0196f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33278/0 (socket says 172.21.15.77:33278) 2023-12-01T01:23:04.972 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.968+0000 7fa1ee7fc700 1 -- 172.21.15.77:0/1690362736 learned_addr learned my addr 172.21.15.77:0/1690362736 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:23:04.972 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.968+0000 7fa1eeffd700 1 --2- 172.21.15.77:0/1690362736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1f0101840 0x7fa1f01969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:04.972 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.968+0000 7fa1ef7fe700 1 --2- 172.21.15.77:0/1690362736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1f0106aa0 0x7fa1f019df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:04.972 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.968+0000 7fa1ee7fc700 1 -- 172.21.15.77:0/1690362736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1f0106aa0 msgr2=0x7fa1f019df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:04.972 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.968+0000 7fa1ee7fc700 1 --2- 172.21.15.77:0/1690362736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1f0106aa0 0x7fa1f019df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:04.972 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.968+0000 7fa1ee7fc700 1 -- 172.21.15.77:0/1690362736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1f0101840 msgr2=0x7fa1f01969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:04.972 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.968+0000 7fa1ee7fc700 1 --2- 172.21.15.77:0/1690362736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1f0101840 0x7fa1f01969c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:04.972 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.968+0000 7fa1ee7fc700 1 -- 172.21.15.77:0/1690362736 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d80095f0 con 0x7fa1f0104170 2023-12-01T01:23:04.972 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.968+0000 7fa1ee7fc700 1 --2- 172.21.15.77:0/1690362736 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1f0104170 0x7fa1f0196f00 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fa1e000d950 tx=0x7fa1e000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:23:04.973 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.969+0000 7fa1e7fff700 1 -- 172.21.15.77:0/1690362736 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e000f890 con 0x7fa1f0104170 2023-12-01T01:23:04.973 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.969+0000 7fa1e7fff700 1 -- 172.21.15.77:0/1690362736 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1e0010460 con 0x7fa1f0104170 2023-12-01T01:23:04.973 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.969+0000 7fa1f56f9700 1 -- 172.21.15.77:0/1690362736 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1f019eaf0 con 0x7fa1f0104170 2023-12-01T01:23:04.973 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.969+0000 7fa1e7fff700 1 -- 172.21.15.77:0/1690362736 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e000e610 con 0x7fa1f0104170 2023-12-01T01:23:04.973 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.969+0000 7fa1f56f9700 1 -- 172.21.15.77:0/1690362736 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1f019f000 con 0x7fa1f0104170 2023-12-01T01:23:04.973 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.970+0000 7fa1f56f9700 1 -- 172.21.15.77:0/1690362736 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1f00fcd90 con 0x7fa1f0104170 2023-12-01T01:23:04.973 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.971+0000 7fa1e7fff700 1 -- 172.21.15.77:0/1690362736 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fa1e00108d0 con 0x7fa1f0104170 2023-12-01T01:23:04.974 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.971+0000 7fa1e7fff700 1 --2- 172.21.15.77:0/1690362736 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa1dc0a0570 0x7fa1dc0a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:04.974 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.971+0000 7fa1e7fff700 1 -- 172.21.15.77:0/1690362736 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fa1e00cb6a0 con 0x7fa1f0104170 2023-12-01T01:23:04.974 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.971+0000 7fa1eeffd700 1 --2- 172.21.15.77:0/1690362736 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa1dc0a0570 0x7fa1dc0a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:04.974 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.972+0000 7fa1eeffd700 1 --2- 172.21.15.77:0/1690362736 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa1dc0a0570 0x7fa1dc0a2a30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800b5c0 tx=0x7fa1d8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:23:04.976 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:04.976+0000 7fa1e7fff700 1 -- 172.21.15.77:0/1690362736 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa1e00952e0 con 0x7fa1f0104170 2023-12-01T01:23:05.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:04 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/2370796192' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-01T01:23:05.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:04 smithi203 ceph-mon[103581]: pgmap v454: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2023-12-01T01:23:05.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:04 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/2370796192' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-01T01:23:05.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:04 smithi137 ceph-mon[103312]: pgmap v454: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2023-12-01T01:23:05.211 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.209+0000 7fa1f56f9700 1 -- 172.21.15.77:0/1690362736 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1f002cec0 con 0x7fa1f0104170 2023-12-01T01:23:05.211 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.210+0000 7fa1e7fff700 1 -- 172.21.15.77:0/1690362736 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+10866 (secure 0 0 0) 0x7fa1e0016090 con 0x7fa1f0104170 2023-12-01T01:23:05.212 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:23:05.212 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14574,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/1435145620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":1435145620},{"type":"v1","addr":"172.21.15.77:6835","nonce":1435145620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14580,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/841838908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":841838908},{"type":"v1","addr":"172.21.15.77:6837","nonce":841838908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-01T01:16:25.345276+0000","modified":"2023-12-01T01:22:25.411862+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24496,"mds_2":24475,"mds_3":24473,"mds_4":14649},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14637":{"gid":14637,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/3726913446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3726913446},{"type":"v1","addr":"172.21.15.77:6839","nonce":3726913446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14649":{"gid":14649,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":5,"addr":"172.21.15.77:6841/3023184965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":3023184965},{"type":"v1","addr":"172.21.15.77:6841","nonce":3023184965}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24428":{"gid":24428,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/2249638445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2249638445},{"type":"v1","addr":"172.21.15.137:6833","nonce":2249638445}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24430":{"gid":24430,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6833/1201599774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":1201599774},{"type":"v1","addr":"172.21.15.203:6833","nonce":1201599774}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24449":{"gid":24449,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3935354878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3935354878},{"type":"v1","addr":"172.21.15.137:6835","nonce":3935354878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24463":{"gid":24463,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/3237538176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3237538176},{"type":"v1","addr":"172.21.15.203:6835","nonce":3237538176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24473":{"gid":24473,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":6,"addr":"172.21.15.137:6837/3518881867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":3518881867},{"type":"v1","addr":"172.21.15.137:6837","nonce":3518881867}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24475":{"gid":24475,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":46,"addr":"172.21.15.203:6837/2887080977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2887080977},{"type":"v1","addr":"172.21.15.203:6837","nonce":2887080977}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":44,"addr":"172.21.15.203:6839/2888221613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2888221613},{"type":"v1","addr":"172.21.15.203:6839","nonce":2888221613}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/1099966660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1099966660},{"type":"v1","addr":"172.21.15.137:6839","nonce":1099966660}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-01T01:23:05.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.213+0000 7fa1f56f9700 1 -- 172.21.15.77:0/1690362736 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa1dc0a0570 msgr2=0x7fa1dc0a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:05.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.213+0000 7fa1f56f9700 1 --2- 172.21.15.77:0/1690362736 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa1dc0a0570 0x7fa1dc0a2a30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800b5c0 tx=0x7fa1d8005fd0 comp rx=0 tx=0).stop 2023-12-01T01:23:05.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.213+0000 7fa1f56f9700 1 -- 172.21.15.77:0/1690362736 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1f0104170 msgr2=0x7fa1f0196f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:05.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.213+0000 7fa1f56f9700 1 --2- 172.21.15.77:0/1690362736 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1f0104170 0x7fa1f0196f00 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fa1e000d950 tx=0x7fa1e000dc60 comp rx=0 tx=0).stop 2023-12-01T01:23:05.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.213+0000 7fa1f56f9700 1 -- 172.21.15.77:0/1690362736 shutdown_connections 2023-12-01T01:23:05.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.213+0000 7fa1f56f9700 1 --2- 172.21.15.77:0/1690362736 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa1dc0a0570 0x7fa1dc0a2a30 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:05.214 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.213+0000 7fa1f56f9700 1 --2- 172.21.15.77:0/1690362736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1f0106aa0 0x7fa1f019df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:05.215 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.213+0000 7fa1f56f9700 1 --2- 172.21.15.77:0/1690362736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1f0101840 0x7fa1f01969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:05.215 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.213+0000 7fa1f56f9700 1 --2- 172.21.15.77:0/1690362736 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1f0104170 0x7fa1f0196f00 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:05.215 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.213+0000 7fa1f56f9700 1 -- 172.21.15.77:0/1690362736 >> 172.21.15.77:0/1690362736 conn(0x7fa1f00fb210 msgr2=0x7fa1f0102570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:23:05.215 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.213+0000 7fa1f56f9700 1 -- 172.21.15.77:0/1690362736 shutdown_connections 2023-12-01T01:23:05.215 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.213+0000 7fa1f56f9700 1 -- 172.21.15.77:0/1690362736 wait complete. 2023-12-01T01:23:05.215 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 41 2023-12-01T01:23:05.226 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-01T01:23:05.351 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.347+0000 7f7787852700 1 -- 172.21.15.77:0/619565133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7780100da0 msgr2=0x7f77801011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:05.351 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.347+0000 7f7787852700 1 --2- 172.21.15.77:0/619565133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7780100da0 0x7f77801011c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f77700099c0 tx=0x7f7770009cd0 comp rx=0 tx=0).stop 2023-12-01T01:23:05.351 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.347+0000 7f7787852700 1 -- 172.21.15.77:0/619565133 shutdown_connections 2023-12-01T01:23:05.351 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.347+0000 7f7787852700 1 --2- 172.21.15.77:0/619565133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7780102930 0x7f77801091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:05.352 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.347+0000 7f7787852700 1 --2- 172.21.15.77:0/619565133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7780100da0 0x7f77801011c0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:05.352 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.347+0000 7f7787852700 1 --2- 172.21.15.77:0/619565133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7780101f90 0x7f77801023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:05.352 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.347+0000 7f7787852700 1 -- 172.21.15.77:0/619565133 >> 172.21.15.77:0/619565133 conn(0x7f77800fc310 msgr2=0x7f77800fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:23:05.352 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.347+0000 7f7787852700 1 -- 172.21.15.77:0/619565133 shutdown_connections 2023-12-01T01:23:05.352 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.347+0000 7f7787852700 1 -- 172.21.15.77:0/619565133 wait complete. 2023-12-01T01:23:05.352 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.348+0000 7f7787852700 1 Processor -- start 2023-12-01T01:23:05.352 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.348+0000 7f7787852700 1 -- start start 2023-12-01T01:23:05.352 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.348+0000 7f7787852700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7780100da0 0x7f7780196930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:05.352 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.348+0000 7f7787852700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7780101f90 0x7f7780196e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:05.352 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.348+0000 7f7787852700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7780102930 0x7f778019dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:05.352 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.348+0000 7f7787852700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778019e4b0 con 0x7f7780100da0 2023-12-01T01:23:05.352 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.348+0000 7f7787852700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778019e5f0 con 0x7f7780102930 2023-12-01T01:23:05.352 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.348+0000 7f7787852700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778019e780 con 0x7f7780101f90 2023-12-01T01:23:05.352 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.348+0000 7f77855ee700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7780100da0 0x7f7780196930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:05.352 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.348+0000 7f77855ee700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7780100da0 0x7f7780196930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33288/0 (socket says 172.21.15.77:33288) 2023-12-01T01:23:05.353 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.348+0000 7f77855ee700 1 -- 172.21.15.77:0/2926588795 learned_addr learned my addr 172.21.15.77:0/2926588795 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:23:05.353 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.348+0000 7f7784ded700 1 --2- 172.21.15.77:0/2926588795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7780101f90 0x7f7780196e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:05.353 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.349+0000 7f7785def700 1 --2- 172.21.15.77:0/2926588795 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7780102930 0x7f778019dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:05.353 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.349+0000 7f77855ee700 1 -- 172.21.15.77:0/2926588795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7780101f90 msgr2=0x7f7780196e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:05.353 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.349+0000 7f77855ee700 1 --2- 172.21.15.77:0/2926588795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7780101f90 0x7f7780196e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:05.353 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.349+0000 7f77855ee700 1 -- 172.21.15.77:0/2926588795 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7780102930 msgr2=0x7f778019dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:05.353 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.349+0000 7f77855ee700 1 --2- 172.21.15.77:0/2926588795 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7780102930 0x7f778019dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:05.353 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.349+0000 7f77855ee700 1 -- 172.21.15.77:0/2926588795 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77700095f0 con 0x7f7780100da0 2023-12-01T01:23:05.353 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.349+0000 7f7785def700 1 --2- 172.21.15.77:0/2926588795 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7780102930 0x7f778019dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:23:05.353 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.349+0000 7f77855ee700 1 --2- 172.21.15.77:0/2926588795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7780100da0 0x7f7780196930 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f77700036f0 tx=0x7f77700038c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:23:05.353 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.349+0000 7f77767fc700 1 -- 172.21.15.77:0/2926588795 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777001d070 con 0x7f7780100da0 2023-12-01T01:23:05.353 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.349+0000 7f77767fc700 1 -- 172.21.15.77:0/2926588795 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7770022470 con 0x7f7780100da0 2023-12-01T01:23:05.353 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.349+0000 7f7787852700 1 -- 172.21.15.77:0/2926588795 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f778019ea00 con 0x7f7780100da0 2023-12-01T01:23:05.353 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.349+0000 7f77767fc700 1 -- 172.21.15.77:0/2926588795 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7770004230 con 0x7f7780100da0 2023-12-01T01:23:05.353 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.349+0000 7f7787852700 1 -- 172.21.15.77:0/2926588795 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f778019ee20 con 0x7f7780100da0 2023-12-01T01:23:05.354 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.350+0000 7f7787852700 1 -- 172.21.15.77:0/2926588795 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7780107b90 con 0x7f7780100da0 2023-12-01T01:23:05.354 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.351+0000 7f77767fc700 1 -- 172.21.15.77:0/2926588795 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f77700225e0 con 0x7f7780100da0 2023-12-01T01:23:05.354 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.352+0000 7f77767fc700 1 --2- 172.21.15.77:0/2926588795 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f776c0a0570 0x7f776c0a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:05.355 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.352+0000 7f7784ded700 1 --2- 172.21.15.77:0/2926588795 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f776c0a0570 0x7f776c0a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:05.355 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.352+0000 7f77767fc700 1 -- 172.21.15.77:0/2926588795 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7f77700cd690 con 0x7f7780100da0 2023-12-01T01:23:05.355 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.352+0000 7f7784ded700 1 --2- 172.21.15.77:0/2926588795 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f776c0a0570 0x7f776c0a2a30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f777c005ea0 tx=0x7f777c005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:23:05.356 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.356+0000 7f77767fc700 1 -- 172.21.15.77:0/2926588795 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7770097360 con 0x7f7780100da0 2023-12-01T01:23:05.555 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.554+0000 7f7787852700 1 -- 172.21.15.77:0/2926588795 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f778002cc70 con 0x7f7780100da0 2023-12-01T01:23:05.557 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.556+0000 7f77767fc700 1 -- 172.21.15.77:0/2926588795 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22472 (secure 0 0 0) 0x7f77700ba020 con 0x7f7780100da0 2023-12-01T01:23:05.557 INFO:teuthology.orchestra.run.smithi077.stdout: 2023-12-01T01:23:05.557 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":84,"fsid":"2a06e29a-8fe6-11ee-95a2-87774f69a715","created":"2023-12-01T01:09:30.210398+0000","modified":"2023-12-01T01:17:25.609766+0000","last_up_change":"2023-12-01T01:15:44.586092+0000","last_in_change":"2023-12-01T01:15:26.305439+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-01T01:12:30.514229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-01T01:16:23.568860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-01T01:16:24.531137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fc651e62-a103-4ab8-a76d-d3708ea6aeec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6803","nonce":2244799094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6805","nonce":2244799094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6809","nonce":2244799094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":2244799094},{"type":"v1","addr":"172.21.15.77:6807","nonce":2244799094}]},"public_addr":"172.21.15.77:6803/2244799094","cluster_addr":"172.21.15.77:6805/2244799094","heartbeat_back_addr":"172.21.15.77:6809/2244799094","heartbeat_front_addr":"172.21.15.77:6807/2244799094","state":["exists","up"]},{"osd":1,"uuid":"0adf8c54-49ec-49a0-8bf1-af1f20e60729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6801","nonce":224194890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6803","nonce":224194890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6807","nonce":224194890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":224194890},{"type":"v1","addr":"172.21.15.137:6805","nonce":224194890}]},"public_addr":"172.21.15.137:6801/224194890","cluster_addr":"172.21.15.137:6803/224194890","heartbeat_back_addr":"172.21.15.137:6807/224194890","heartbeat_front_addr":"172.21.15.137:6805/224194890","state":["exists","up"]},{"osd":2,"uuid":"5a69802c-dede-4544-9a12-436f5c1c0038","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6801","nonce":4078459110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6803","nonce":4078459110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6807","nonce":4078459110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4078459110},{"type":"v1","addr":"172.21.15.203:6805","nonce":4078459110}]},"public_addr":"172.21.15.203:6801/4078459110","cluster_addr":"172.21.15.203:6803/4078459110","heartbeat_back_addr":"172.21.15.203:6807/4078459110","heartbeat_front_addr":"172.21.15.203:6805/4078459110","state":["exists","up"]},{"osd":3,"uuid":"ad325a97-580b-4c99-bf12-cbb0d92a6d0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6811","nonce":1454440477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6813","nonce":1454440477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6817","nonce":1454440477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":1454440477},{"type":"v1","addr":"172.21.15.77:6815","nonce":1454440477}]},"public_addr":"172.21.15.77:6811/1454440477","cluster_addr":"172.21.15.77:6813/1454440477","heartbeat_back_addr":"172.21.15.77:6817/1454440477","heartbeat_front_addr":"172.21.15.77:6815/1454440477","state":["exists","up"]},{"osd":4,"uuid":"1e3ac1ba-98bb-4b8b-8e26-50064c66bf2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6809","nonce":1103241451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6811","nonce":1103241451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6815","nonce":1103241451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1103241451},{"type":"v1","addr":"172.21.15.137:6813","nonce":1103241451}]},"public_addr":"172.21.15.137:6809/1103241451","cluster_addr":"172.21.15.137:6811/1103241451","heartbeat_back_addr":"172.21.15.137:6815/1103241451","heartbeat_front_addr":"172.21.15.137:6813/1103241451","state":["exists","up"]},{"osd":5,"uuid":"eb32c016-abeb-44ab-abf0-19dd09235736","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6809","nonce":2648569482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6811","nonce":2648569482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6815","nonce":2648569482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":2648569482},{"type":"v1","addr":"172.21.15.203:6813","nonce":2648569482}]},"public_addr":"172.21.15.203:6809/2648569482","cluster_addr":"172.21.15.203:6811/2648569482","heartbeat_back_addr":"172.21.15.203:6815/2648569482","heartbeat_front_addr":"172.21.15.203:6813/2648569482","state":["exists","up"]},{"osd":6,"uuid":"0e3fa97e-2ca6-418c-9e8d-a6fe3c56adb3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6819","nonce":2463414651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6821","nonce":2463414651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6825","nonce":2463414651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2463414651},{"type":"v1","addr":"172.21.15.77:6823","nonce":2463414651}]},"public_addr":"172.21.15.77:6819/2463414651","cluster_addr":"172.21.15.77:6821/2463414651","heartbeat_back_addr":"172.21.15.77:6825/2463414651","heartbeat_front_addr":"172.21.15.77:6823/2463414651","state":["exists","up"]},{"osd":7,"uuid":"d7398c01-0f98-4138-8a49-c3ff0aae7e88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6817","nonce":3108330766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6819","nonce":3108330766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6823","nonce":3108330766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":3108330766},{"type":"v1","addr":"172.21.15.137:6821","nonce":3108330766}]},"public_addr":"172.21.15.137:6817/3108330766","cluster_addr":"172.21.15.137:6819/3108330766","heartbeat_back_addr":"172.21.15.137:6823/3108330766","heartbeat_front_addr":"172.21.15.137:6821/3108330766","state":["exists","up"]},{"osd":8,"uuid":"0c09050d-3e43-410c-8cd7-bcb891af56b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6817","nonce":1161158776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6819","nonce":1161158776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6823","nonce":1161158776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1161158776},{"type":"v1","addr":"172.21.15.203:6821","nonce":1161158776}]},"public_addr":"172.21.15.203:6817/1161158776","cluster_addr":"172.21.15.203:6819/1161158776","heartbeat_back_addr":"172.21.15.203:6823/1161158776","heartbeat_front_addr":"172.21.15.203:6821/1161158776","state":["exists","up"]},{"osd":9,"uuid":"527ecb49-95ec-48bd-9872-3b07fae5c5ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6827","nonce":2409880685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6829","nonce":2409880685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6833","nonce":2409880685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":2409880685},{"type":"v1","addr":"172.21.15.77:6831","nonce":2409880685}]},"public_addr":"172.21.15.77:6827/2409880685","cluster_addr":"172.21.15.77:6829/2409880685","heartbeat_back_addr":"172.21.15.77:6833/2409880685","heartbeat_front_addr":"172.21.15.77:6831/2409880685","state":["exists","up"]},{"osd":10,"uuid":"1680bb3e-a3c8-427b-af68-e2214f32bed3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6825","nonce":374920146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6827","nonce":374920146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6831","nonce":374920146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":374920146},{"type":"v1","addr":"172.21.15.137:6829","nonce":374920146}]},"public_addr":"172.21.15.137:6825/374920146","cluster_addr":"172.21.15.137:6827/374920146","heartbeat_back_addr":"172.21.15.137:6831/374920146","heartbeat_front_addr":"172.21.15.137:6829/374920146","state":["exists","up"]},{"osd":11,"uuid":"8a533466-023d-4803-ae36-9a9f1715e49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6825","nonce":2161588962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6827","nonce":2161588962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6831","nonce":2161588962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2161588962},{"type":"v1","addr":"172.21.15.203:6829","nonce":2161588962}]},"public_addr":"172.21.15.203:6825/2161588962","cluster_addr":"172.21.15.203:6827/2161588962","heartbeat_back_addr":"172.21.15.203:6831/2161588962","heartbeat_front_addr":"172.21.15.203:6829/2161588962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:11:45.420509+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:06.210115+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:28.191307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:12:49.594116+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:11.261689+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:33.376541+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:13:54.968336+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:16.775310+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:38.733792+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:14:59.456699+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:15:20.924059+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-01T01:15:43.094773+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/2260035254":"2023-12-02T01:10:24.191835+0000","172.21.15.77:6800/1456621747":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/219838828":"2023-12-02T01:10:24.191835+0000","172.21.15.77:6801/1456621747":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/2937149266":"2023-12-02T01:10:02.232948+0000","172.21.15.77:0/807520018":"2023-12-02T01:10:02.232948+0000","172.21.15.77:6801/1661788026":"2023-12-02T01:10:02.232948+0000","172.21.15.77:6800/1661788026":"2023-12-02T01:10:02.232948+0000","172.21.15.77:0/3551282473":"2023-12-02T01:10:24.191835+0000","172.21.15.77:0/1202222558":"2023-12-02T01:10:02.232948+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-01T01:23:05.559 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.559+0000 7f7787852700 1 -- 172.21.15.77:0/2926588795 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f776c0a0570 msgr2=0x7f776c0a2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:05.559 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.559+0000 7f7787852700 1 --2- 172.21.15.77:0/2926588795 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f776c0a0570 0x7f776c0a2a30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f777c005ea0 tx=0x7f777c005e30 comp rx=0 tx=0).stop 2023-12-01T01:23:05.560 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.559+0000 7f7787852700 1 -- 172.21.15.77:0/2926588795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7780100da0 msgr2=0x7f7780196930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:05.560 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.559+0000 7f7787852700 1 --2- 172.21.15.77:0/2926588795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7780100da0 0x7f7780196930 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f77700036f0 tx=0x7f77700038c0 comp rx=0 tx=0).stop 2023-12-01T01:23:05.560 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.559+0000 7f7787852700 1 -- 172.21.15.77:0/2926588795 shutdown_connections 2023-12-01T01:23:05.560 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.559+0000 7f7787852700 1 --2- 172.21.15.77:0/2926588795 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f776c0a0570 0x7f776c0a2a30 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:05.560 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.559+0000 7f7787852700 1 --2- 172.21.15.77:0/2926588795 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7780102930 0x7f778019dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:05.560 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.559+0000 7f7787852700 1 --2- 172.21.15.77:0/2926588795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7780100da0 0x7f7780196930 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:05.560 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.559+0000 7f7787852700 1 --2- 172.21.15.77:0/2926588795 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7780101f90 0x7f7780196e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:05.560 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.559+0000 7f7787852700 1 -- 172.21.15.77:0/2926588795 >> 172.21.15.77:0/2926588795 conn(0x7f77800fc310 msgr2=0x7f77800fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:23:05.560 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.559+0000 7f7787852700 1 -- 172.21.15.77:0/2926588795 shutdown_connections 2023-12-01T01:23:05.560 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.559+0000 7f7787852700 1 -- 172.21.15.77:0/2926588795 wait complete. 2023-12-01T01:23:05.572 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-01T01:23:05.696 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.694+0000 7fa8e944b700 1 -- 172.21.15.77:0/3835724197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8e4104190 msgr2=0x7fa8e4106580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:05.696 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.694+0000 7fa8e944b700 1 --2- 172.21.15.77:0/3835724197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8e4104190 0x7fa8e4106580 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4009990 tx=0x7fa8d4009ca0 comp rx=0 tx=0).stop 2023-12-01T01:23:05.697 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.694+0000 7fa8e944b700 1 -- 172.21.15.77:0/3835724197 shutdown_connections 2023-12-01T01:23:05.697 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.694+0000 7fa8e944b700 1 --2- 172.21.15.77:0/3835724197 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8e4106ac0 0x7fa8e4108ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:05.697 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.694+0000 7fa8e944b700 1 --2- 172.21.15.77:0/3835724197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8e4101860 0x7fa8e4103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:05.697 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.694+0000 7fa8e944b700 1 --2- 172.21.15.77:0/3835724197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8e4104190 0x7fa8e4106580 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:05.697 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.694+0000 7fa8e944b700 1 -- 172.21.15.77:0/3835724197 >> 172.21.15.77:0/3835724197 conn(0x7fa8e40fb1f0 msgr2=0x7fa8e40fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:23:05.697 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.694+0000 7fa8e944b700 1 -- 172.21.15.77:0/3835724197 shutdown_connections 2023-12-01T01:23:05.697 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.694+0000 7fa8e944b700 1 -- 172.21.15.77:0/3835724197 wait complete. 2023-12-01T01:23:05.697 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.694+0000 7fa8e944b700 1 Processor -- start 2023-12-01T01:23:05.697 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.695+0000 7fa8e944b700 1 -- start start 2023-12-01T01:23:05.697 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.695+0000 7fa8e944b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8e4101860 0x7fa8e41011d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:05.697 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.695+0000 7fa8e944b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8e4104190 0x7fa8e40ff810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:05.697 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.695+0000 7fa8e944b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8e4106ac0 0x7fa8e40ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:05.697 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.695+0000 7fa8e944b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e41004f0 con 0x7fa8e4106ac0 2023-12-01T01:23:05.697 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.695+0000 7fa8e944b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e4100660 con 0x7fa8e4104190 2023-12-01T01:23:05.698 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.695+0000 7fa8e944b700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e41a0a50 con 0x7fa8e4101860 2023-12-01T01:23:05.698 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.695+0000 7fa8e37fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8e4106ac0 0x7fa8e40ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:05.698 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.695+0000 7fa8e37fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8e4106ac0 0x7fa8e40ffd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33306/0 (socket says 172.21.15.77:33306) 2023-12-01T01:23:05.698 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.695+0000 7fa8e37fe700 1 -- 172.21.15.77:0/315670640 learned_addr learned my addr 172.21.15.77:0/315670640 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:23:05.698 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.695+0000 7fa8e2ffd700 1 --2- 172.21.15.77:0/315670640 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8e4101860 0x7fa8e41011d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:05.698 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.695+0000 7fa8e27fc700 1 --2- 172.21.15.77:0/315670640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8e4104190 0x7fa8e40ff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:05.698 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.695+0000 7fa8e37fe700 1 -- 172.21.15.77:0/315670640 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8e4101860 msgr2=0x7fa8e41011d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:05.698 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.695+0000 7fa8e37fe700 1 --2- 172.21.15.77:0/315670640 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8e4101860 0x7fa8e41011d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:05.698 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.695+0000 7fa8e37fe700 1 -- 172.21.15.77:0/315670640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8e4104190 msgr2=0x7fa8e40ff810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:05.698 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.695+0000 7fa8e37fe700 1 --2- 172.21.15.77:0/315670640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8e4104190 0x7fa8e40ff810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:05.698 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.695+0000 7fa8e37fe700 1 -- 172.21.15.77:0/315670640 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d40095f0 con 0x7fa8e4106ac0 2023-12-01T01:23:05.698 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.695+0000 7fa8e37fe700 1 --2- 172.21.15.77:0/315670640 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8e4106ac0 0x7fa8e40ffd50 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc007d70 tx=0x7fa8dc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:23:05.698 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.695+0000 7fa8dbfff700 1 -- 172.21.15.77:0/315670640 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc00ad40 con 0x7fa8e4106ac0 2023-12-01T01:23:05.698 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.696+0000 7fa8dbfff700 1 -- 172.21.15.77:0/315670640 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8dc0128e0 con 0x7fa8e4106ac0 2023-12-01T01:23:05.699 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.696+0000 7fa8e944b700 1 -- 172.21.15.77:0/315670640 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8e41a0d30 con 0x7fa8e4106ac0 2023-12-01T01:23:05.699 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.696+0000 7fa8dbfff700 1 -- 172.21.15.77:0/315670640 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc0116b0 con 0x7fa8e4106ac0 2023-12-01T01:23:05.699 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.696+0000 7fa8e944b700 1 -- 172.21.15.77:0/315670640 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8e41a11e0 con 0x7fa8e4106ac0 2023-12-01T01:23:05.700 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.697+0000 7fa8e944b700 1 -- 172.21.15.77:0/315670640 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8e40fcd70 con 0x7fa8e4106ac0 2023-12-01T01:23:05.700 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.697+0000 7fa8dbfff700 1 -- 172.21.15.77:0/315670640 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7fa8dc012420 con 0x7fa8e4106ac0 2023-12-01T01:23:05.700 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.698+0000 7fa8dbfff700 1 --2- 172.21.15.77:0/315670640 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa8d00a05a0 0x7fa8d00a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:05.700 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.698+0000 7fa8e2ffd700 1 --2- 172.21.15.77:0/315670640 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa8d00a05a0 0x7fa8d00a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:05.700 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.698+0000 7fa8dbfff700 1 -- 172.21.15.77:0/315670640 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8153+0+0 (secure 0 0 0) 0x7fa8dc0cbb10 con 0x7fa8e4106ac0 2023-12-01T01:23:05.700 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.699+0000 7fa8e2ffd700 1 --2- 172.21.15.77:0/315670640 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa8d00a05a0 0x7fa8d00a2a60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc005950 tx=0x7fa8cc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:23:05.703 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.703+0000 7fa8dbfff700 1 -- 172.21.15.77:0/315670640 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa8dc095750 con 0x7fa8e4106ac0 2023-12-01T01:23:05.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:05 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/3021104230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:23:05.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:05 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/1690362736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:23:05.904 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:05 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/2926588795' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:23:05.937 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.937+0000 7fa8e944b700 1 -- 172.21.15.77:0/315670640 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa8e402cde0 con 0x7fa8e4106ac0 2023-12-01T01:23:05.954 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.954+0000 7fa8dbfff700 1 -- 172.21.15.77:0/315670640 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v42) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa8dc098d70 con 0x7fa8e4106ac0 2023-12-01T01:23:05.956 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.956+0000 7fa8e944b700 1 -- 172.21.15.77:0/315670640 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa8d00a05a0 msgr2=0x7fa8d00a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:05.957 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.956+0000 7fa8e944b700 1 --2- 172.21.15.77:0/315670640 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa8d00a05a0 0x7fa8d00a2a60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc005950 tx=0x7fa8cc00b410 comp rx=0 tx=0).stop 2023-12-01T01:23:05.957 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.956+0000 7fa8e944b700 1 -- 172.21.15.77:0/315670640 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8e4106ac0 msgr2=0x7fa8e40ffd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:05.957 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.956+0000 7fa8e944b700 1 --2- 172.21.15.77:0/315670640 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8e4106ac0 0x7fa8e40ffd50 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc007d70 tx=0x7fa8dc00a340 comp rx=0 tx=0).stop 2023-12-01T01:23:05.957 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.956+0000 7fa8e944b700 1 -- 172.21.15.77:0/315670640 shutdown_connections 2023-12-01T01:23:05.957 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.956+0000 7fa8e944b700 1 --2- 172.21.15.77:0/315670640 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7fa8d00a05a0 0x7fa8d00a2a60 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:05.957 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.956+0000 7fa8e944b700 1 --2- 172.21.15.77:0/315670640 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8e4106ac0 0x7fa8e40ffd50 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:05.957 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.956+0000 7fa8e944b700 1 --2- 172.21.15.77:0/315670640 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8e4101860 0x7fa8e41011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:05.957 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.956+0000 7fa8e944b700 1 --2- 172.21.15.77:0/315670640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8e4104190 0x7fa8e40ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:05.957 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.956+0000 7fa8e944b700 1 -- 172.21.15.77:0/315670640 >> 172.21.15.77:0/315670640 conn(0x7fa8e40fb1f0 msgr2=0x7fa8e41024f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:23:05.957 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.956+0000 7fa8e944b700 1 -- 172.21.15.77:0/315670640 shutdown_connections 2023-12-01T01:23:05.958 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:05.956+0000 7fa8e944b700 1 -- 172.21.15.77:0/315670640 wait complete. 2023-12-01T01:23:05.958 INFO:teuthology.orchestra.run.smithi077.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-01T01:23:05.970 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-01T01:23:05.996 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:05 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/3021104230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:23:05.996 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:05 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/1690362736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:23:05.996 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:05 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/2926588795' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:23:05.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:05 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/3021104230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:23:05.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:05 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/1690362736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-01T01:23:05.997 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:05 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/2926588795' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-01T01:23:05.997 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:23:05 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: ignoring --setuser ceph since I am not root 2023-12-01T01:23:05.997 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:23:05 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: ignoring --setgroup ceph since I am not root 2023-12-01T01:23:05.997 INFO:journalctl@ceph.mds.f.smithi203.stdout:Dec 01 01:23:05 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-f[140711]: ignoring --setuser ceph since I am not root 2023-12-01T01:23:05.998 INFO:journalctl@ceph.mds.f.smithi203.stdout:Dec 01 01:23:05 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-f[140711]: ignoring --setgroup ceph since I am not root 2023-12-01T01:23:06.100 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.097+0000 7f8ceb8f3700 1 -- 172.21.15.77:0/4071736304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ce4100e70 msgr2=0x7f8ce41012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:06.100 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.097+0000 7f8ceb8f3700 1 --2- 172.21.15.77:0/4071736304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ce4100e70 0x7f8ce41012d0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f8ce00099c0 tx=0x7f8ce0009cd0 comp rx=0 tx=0).stop 2023-12-01T01:23:06.100 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.097+0000 7f8ceb8f3700 1 -- 172.21.15.77:0/4071736304 shutdown_connections 2023-12-01T01:23:06.100 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.097+0000 7f8ceb8f3700 1 --2- 172.21.15.77:0/4071736304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ce4101810 0x7f8ce41080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:06.100 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.097+0000 7f8ceb8f3700 1 --2- 172.21.15.77:0/4071736304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ce40ffc80 0x7f8ce41000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:06.100 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.097+0000 7f8ceb8f3700 1 --2- 172.21.15.77:0/4071736304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ce4100e70 0x7f8ce41012d0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:06.101 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.097+0000 7f8ceb8f3700 1 -- 172.21.15.77:0/4071736304 >> 172.21.15.77:0/4071736304 conn(0x7f8ce40fb1f0 msgr2=0x7f8ce40fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:23:06.101 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.097+0000 7f8ceb8f3700 1 -- 172.21.15.77:0/4071736304 shutdown_connections 2023-12-01T01:23:06.101 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.097+0000 7f8ceb8f3700 1 -- 172.21.15.77:0/4071736304 wait complete. 2023-12-01T01:23:06.101 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.098+0000 7f8ceb8f3700 1 Processor -- start 2023-12-01T01:23:06.101 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.098+0000 7f8ceb8f3700 1 -- start start 2023-12-01T01:23:06.101 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.098+0000 7f8ceb8f3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ce40ffc80 0x7f8ce41969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:06.101 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.098+0000 7f8ceb8f3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ce4100e70 0x7f8ce4196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:06.101 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.098+0000 7f8ceb8f3700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ce4101810 0x7f8ce419dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:06.101 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.098+0000 7f8ceb8f3700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce419e570 con 0x7f8ce4100e70 2023-12-01T01:23:06.101 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.098+0000 7f8ceb8f3700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce419e6b0 con 0x7f8ce40ffc80 2023-12-01T01:23:06.101 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.098+0000 7f8ceb8f3700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce419e840 con 0x7f8ce4101810 2023-12-01T01:23:06.101 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.098+0000 7f8ce8e8e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ce4100e70 0x7f8ce4196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:06.102 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.098+0000 7f8ce8e8e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ce4100e70 0x7f8ce4196f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33342/0 (socket says 172.21.15.77:33342) 2023-12-01T01:23:06.102 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.098+0000 7f8ce8e8e700 1 -- 172.21.15.77:0/1835481065 learned_addr learned my addr 172.21.15.77:0/1835481065 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:23:06.102 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.098+0000 7f8ce9e90700 1 --2- 172.21.15.77:0/1835481065 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ce4101810 0x7f8ce419dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:06.102 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.098+0000 7f8ce968f700 1 --2- 172.21.15.77:0/1835481065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ce40ffc80 0x7f8ce41969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:06.102 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.098+0000 7f8ce8e8e700 1 -- 172.21.15.77:0/1835481065 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ce4101810 msgr2=0x7f8ce419dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:06.102 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.098+0000 7f8ce8e8e700 1 --2- 172.21.15.77:0/1835481065 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ce4101810 0x7f8ce419dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:06.102 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.098+0000 7f8ce8e8e700 1 -- 172.21.15.77:0/1835481065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ce40ffc80 msgr2=0x7f8ce41969f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:06.102 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.098+0000 7f8ce8e8e700 1 --2- 172.21.15.77:0/1835481065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ce40ffc80 0x7f8ce41969f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:06.102 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.098+0000 7f8ce8e8e700 1 -- 172.21.15.77:0/1835481065 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce00095f0 con 0x7f8ce4100e70 2023-12-01T01:23:06.102 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.098+0000 7f8ce8e8e700 1 --2- 172.21.15.77:0/1835481065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ce4100e70 0x7f8ce4196f30 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0009990 tx=0x7f8ce0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:23:06.102 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.098+0000 7f8cd67fc700 1 -- 172.21.15.77:0/1835481065 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce001d070 con 0x7f8ce4100e70 2023-12-01T01:23:06.102 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.099+0000 7f8cd67fc700 1 -- 172.21.15.77:0/1835481065 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ce0004090 con 0x7f8ce4100e70 2023-12-01T01:23:06.102 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.099+0000 7f8cd67fc700 1 -- 172.21.15.77:0/1835481065 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce0004a00 con 0x7f8ce4100e70 2023-12-01T01:23:06.103 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.099+0000 7f8ceb8f3700 1 -- 172.21.15.77:0/1835481065 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ce419eac0 con 0x7f8ce4100e70 2023-12-01T01:23:06.103 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.099+0000 7f8ceb8f3700 1 -- 172.21.15.77:0/1835481065 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ce419ef70 con 0x7f8ce4100e70 2023-12-01T01:23:06.103 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.099+0000 7f8ceb8f3700 1 -- 172.21.15.77:0/1835481065 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ce410b490 con 0x7f8ce4100e70 2023-12-01T01:23:06.103 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.100+0000 7f8cd67fc700 1 -- 172.21.15.77:0/1835481065 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f8ce0003bd0 con 0x7f8ce4100e70 2023-12-01T01:23:06.104 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.100+0000 7f8cd67fc700 1 --2- 172.21.15.77:0/1835481065 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8cd00a05a0 0x7f8cd00a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:06.104 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.100+0000 7f8ce968f700 1 --2- 172.21.15.77:0/1835481065 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8cd00a05a0 0x7f8cd00a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:06.104 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.100+0000 7f8cd67fc700 1 -- 172.21.15.77:0/1835481065 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8583+0+0 (secure 0 0 0) 0x7f8ce00ccd80 con 0x7f8ce4100e70 2023-12-01T01:23:06.104 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.101+0000 7f8ce968f700 1 --2- 172.21.15.77:0/1835481065 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8cd00a05a0 0x7f8cd00a2a60 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8005fd0 tx=0x7f8cd8005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:23:06.105 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.104+0000 7f8cd67fc700 1 -- 172.21.15.77:0/1835481065 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8ce0096850 con 0x7f8ce4100e70 2023-12-01T01:23:06.262 INFO:journalctl@ceph.mds.f.smithi203.stdout:Dec 01 01:23:05 smithi203 ceph-mds[140736]: -- 172.21.15.203:0/739992562 <== mon.0 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ee956011e0 con 0x55ee955d6c00 2023-12-01T01:23:06.262 INFO:journalctl@ceph.mds.f.smithi203.stdout:Dec 01 01:23:05 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-f[140711]: starting mds.f at 2023-12-01T01:23:06.263 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:23:06 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: ignoring --setuser ceph since I am not root 2023-12-01T01:23:06.263 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:23:06 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: ignoring --setgroup ceph since I am not root 2023-12-01T01:23:06.263 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:23:06 smithi203 ceph-mds[141073]: -- 172.21.15.203:0/355056472 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5573211ad1e0 con 0x557321182800 2023-12-01T01:23:06.263 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:23:06 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: starting mds.c at 2023-12-01T01:23:06.264 INFO:journalctl@ceph.mds.l.smithi203.stdout:Dec 01 01:23:06 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-l[140370]: ignoring --setuser ceph since I am not root 2023-12-01T01:23:06.264 INFO:journalctl@ceph.mds.l.smithi203.stdout:Dec 01 01:23:06 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-l[140370]: ignoring --setgroup ceph since I am not root 2023-12-01T01:23:06.264 INFO:journalctl@ceph.mds.l.smithi203.stdout:Dec 01 01:23:06 smithi203 ceph-mds[140394]: -- 172.21.15.203:0/70210603 <== mon.0 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ab1d36d1e0 con 0x55ab1d342800 2023-12-01T01:23:06.264 INFO:journalctl@ceph.mds.l.smithi203.stdout:Dec 01 01:23:06 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-l[140370]: starting mds.l at 2023-12-01T01:23:06.264 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:23:05 smithi203 ceph-mds[141410]: -- 172.21.15.203:0/3171111410 <== mon.0 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56196a95f1e0 con 0x56196a934c00 2023-12-01T01:23:06.264 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:23:05 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: starting mds.i at 2023-12-01T01:23:06.279 INFO:journalctl@ceph.mds.b.smithi137.stdout:Dec 01 01:23:06 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-b[141375]: ignoring --setuser ceph since I am not root 2023-12-01T01:23:06.279 INFO:journalctl@ceph.mds.b.smithi137.stdout:Dec 01 01:23:06 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-b[141375]: ignoring --setgroup ceph since I am not root 2023-12-01T01:23:06.279 INFO:journalctl@ceph.mds.b.smithi137.stdout:Dec 01 01:23:06 smithi137 ceph-mds[141399]: -- 172.21.15.137:0/334153939 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ee3e6f91e0 con 0x55ee3e6cec00 2023-12-01T01:23:06.279 INFO:journalctl@ceph.mds.b.smithi137.stdout:Dec 01 01:23:06 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-b[141375]: starting mds.b at 2023-12-01T01:23:06.279 INFO:journalctl@ceph.mds.k.smithi137.stdout:Dec 01 01:23:06 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-k[140344]: ignoring --setuser ceph since I am not root 2023-12-01T01:23:06.279 INFO:journalctl@ceph.mds.k.smithi137.stdout:Dec 01 01:23:06 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-k[140344]: ignoring --setgroup ceph since I am not root 2023-12-01T01:23:06.280 INFO:journalctl@ceph.mds.k.smithi137.stdout:Dec 01 01:23:06 smithi137 ceph-mds[140369]: -- 172.21.15.137:0/731053617 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556e16f891e0 con 0x556e16f5e800 2023-12-01T01:23:06.280 INFO:journalctl@ceph.mds.k.smithi137.stdout:Dec 01 01:23:06 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-k[140344]: starting mds.k at 2023-12-01T01:23:06.280 INFO:journalctl@ceph.mds.e.smithi137.stdout:Dec 01 01:23:05 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-e[140682]: ignoring --setuser ceph since I am not root 2023-12-01T01:23:06.280 INFO:journalctl@ceph.mds.e.smithi137.stdout:Dec 01 01:23:05 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-e[140682]: ignoring --setgroup ceph since I am not root 2023-12-01T01:23:06.280 INFO:journalctl@ceph.mds.e.smithi137.stdout:Dec 01 01:23:05 smithi137 ceph-mds[140706]: -- 172.21.15.137:0/1082278413 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557421dcd1e0 con 0x557421da2c00 2023-12-01T01:23:06.280 INFO:journalctl@ceph.mds.e.smithi137.stdout:Dec 01 01:23:05 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-e[140682]: starting mds.e at 2023-12-01T01:23:06.280 INFO:journalctl@ceph.mds.h.smithi137.stdout:Dec 01 01:23:05 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-h[141032]: ignoring --setuser ceph since I am not root 2023-12-01T01:23:06.281 INFO:journalctl@ceph.mds.h.smithi137.stdout:Dec 01 01:23:05 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-h[141032]: ignoring --setgroup ceph since I am not root 2023-12-01T01:23:06.281 INFO:journalctl@ceph.mds.h.smithi137.stdout:Dec 01 01:23:05 smithi137 ceph-mds[141056]: -- 172.21.15.137:0/1701644957 <== mon.0 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5564a2be31e0 con 0x5564a2bb9000 2023-12-01T01:23:06.281 INFO:journalctl@ceph.mds.h.smithi137.stdout:Dec 01 01:23:05 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-h[141032]: starting mds.h at 2023-12-01T01:23:06.281 INFO:journalctl@ceph.mds.h.smithi137.stdout:Dec 01 01:23:06 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-h[141032]: 2023-12-01T01:23:05.999+0000 7fdd4f481700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-12-01T01:23:06.342 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.340+0000 7f8ceb8f3700 1 -- 172.21.15.77:0/1835481065 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8ce419f520 con 0x7f8ce4100e70 2023-12-01T01:23:06.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:05 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mon-a[98843]: 2023-12-01T01:23:05.938+0000 7f9887e07700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-01T01:23:06.404 INFO:journalctl@ceph.mds.a.smithi077.stdout:Dec 01 01:23:05 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-a[150304]: ignoring --setuser ceph since I am not root 2023-12-01T01:23:06.405 INFO:journalctl@ceph.mds.a.smithi077.stdout:Dec 01 01:23:05 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-a[150304]: ignoring --setgroup ceph since I am not root 2023-12-01T01:23:06.405 INFO:journalctl@ceph.mds.a.smithi077.stdout:Dec 01 01:23:05 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-a[150304]: starting mds.a at 2023-12-01T01:23:06.405 INFO:journalctl@ceph.mds.g.smithi077.stdout:Dec 01 01:23:05 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-g[149918]: ignoring --setuser ceph since I am not root 2023-12-01T01:23:06.405 INFO:journalctl@ceph.mds.g.smithi077.stdout:Dec 01 01:23:05 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-g[149918]: ignoring --setgroup ceph since I am not root 2023-12-01T01:23:06.405 INFO:journalctl@ceph.mds.g.smithi077.stdout:Dec 01 01:23:05 smithi077 ceph-mds[149942]: -- 172.21.15.77:0/3941090088 <== mon.0 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560f82d871e0 con 0x560f82d5d000 2023-12-01T01:23:06.405 INFO:journalctl@ceph.mds.g.smithi077.stdout:Dec 01 01:23:05 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-g[149918]: starting mds.g at 2023-12-01T01:23:06.958 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.957+0000 7f8cd67fc700 1 -- 172.21.15.77:0/1835481065 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v43) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8ce0099e70 con 0x7f8ce4100e70 2023-12-01T01:23:06.962 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.960+0000 7f8ceb8f3700 1 -- 172.21.15.77:0/1835481065 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8cd00a05a0 msgr2=0x7f8cd00a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:06.962 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.960+0000 7f8ceb8f3700 1 --2- 172.21.15.77:0/1835481065 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8cd00a05a0 0x7f8cd00a2a60 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8005fd0 tx=0x7f8cd8005e50 comp rx=0 tx=0).stop 2023-12-01T01:23:06.962 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.960+0000 7f8ceb8f3700 1 -- 172.21.15.77:0/1835481065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ce4100e70 msgr2=0x7f8ce4196f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:06.962 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.960+0000 7f8ceb8f3700 1 --2- 172.21.15.77:0/1835481065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ce4100e70 0x7f8ce4196f30 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0009990 tx=0x7f8ce0004730 comp rx=0 tx=0).stop 2023-12-01T01:23:06.963 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.960+0000 7f8ceb8f3700 1 -- 172.21.15.77:0/1835481065 shutdown_connections 2023-12-01T01:23:06.963 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.960+0000 7f8ceb8f3700 1 --2- 172.21.15.77:0/1835481065 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f8cd00a05a0 0x7f8cd00a2a60 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:06.963 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.960+0000 7f8ceb8f3700 1 --2- 172.21.15.77:0/1835481065 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ce4101810 0x7f8ce419dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:06.963 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.960+0000 7f8ceb8f3700 1 --2- 172.21.15.77:0/1835481065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ce40ffc80 0x7f8ce41969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:06.963 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.960+0000 7f8ceb8f3700 1 --2- 172.21.15.77:0/1835481065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ce4100e70 0x7f8ce4196f30 secure :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0009990 tx=0x7f8ce0004730 comp rx=0 tx=0).stop 2023-12-01T01:23:06.963 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.960+0000 7f8ceb8f3700 1 -- 172.21.15.77:0/1835481065 >> 172.21.15.77:0/1835481065 conn(0x7f8ce40fb1f0 msgr2=0x7f8ce4068f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:23:06.963 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.961+0000 7f8ceb8f3700 1 -- 172.21.15.77:0/1835481065 shutdown_connections 2023-12-01T01:23:06.963 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:06.961+0000 7f8ceb8f3700 1 -- 172.21.15.77:0/1835481065 wait complete. 2023-12-01T01:23:06.976 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:06 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/315670640' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-12-01T01:23:06.976 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:06 smithi077 ceph-mon[98867]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-12-01T01:23:06.976 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:06 smithi077 ceph-mon[98867]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-01T01:23:06.976 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:06 smithi077 ceph-mon[98867]: Health check cleared: MDS_DAMAGE (was: 2 MDSs report damaged metadata) 2023-12-01T01:23:06.976 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:06 smithi077 ceph-mon[98867]: osdmap e85: 12 total, 12 up, 12 in 2023-12-01T01:23:06.976 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:06 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/315670640' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-12-01T01:23:06.976 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:06 smithi077 ceph-mon[98867]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-12-01T01:23:06.976 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:06 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/1835481065' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-12-01T01:23:06.977 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:06 smithi077 ceph-mon[98867]: pgmap v456: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 818 B/s wr, 13 op/s 2023-12-01T01:23:06.978 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-01T01:23:07.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:06 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/315670640' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-12-01T01:23:07.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:06 smithi203 ceph-mon[103581]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-12-01T01:23:07.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:06 smithi203 ceph-mon[103581]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-01T01:23:07.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:06 smithi203 ceph-mon[103581]: Health check cleared: MDS_DAMAGE (was: 2 MDSs report damaged metadata) 2023-12-01T01:23:07.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:06 smithi203 ceph-mon[103581]: osdmap e85: 12 total, 12 up, 12 in 2023-12-01T01:23:07.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:06 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/315670640' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-12-01T01:23:07.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:06 smithi203 ceph-mon[103581]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-12-01T01:23:07.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:06 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/1835481065' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-12-01T01:23:07.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:06 smithi203 ceph-mon[103581]: pgmap v456: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 818 B/s wr, 13 op/s 2023-12-01T01:23:07.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:06 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/315670640' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-12-01T01:23:07.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:06 smithi137 ceph-mon[103312]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-12-01T01:23:07.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:06 smithi137 ceph-mon[103312]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-01T01:23:07.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:06 smithi137 ceph-mon[103312]: Health check cleared: MDS_DAMAGE (was: 2 MDSs report damaged metadata) 2023-12-01T01:23:07.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:06 smithi137 ceph-mon[103312]: osdmap e85: 12 total, 12 up, 12 in 2023-12-01T01:23:07.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:06 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/315670640' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-12-01T01:23:07.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:06 smithi137 ceph-mon[103312]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-12-01T01:23:07.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:06 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/1835481065' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-12-01T01:23:07.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:06 smithi137 ceph-mon[103312]: pgmap v456: 97 pgs: 97 active+clean; 32 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 818 B/s wr, 13 op/s 2023-12-01T01:23:07.104 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.103+0000 7ff056d4c700 1 -- 172.21.15.77:0/488097925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff050101810 msgr2=0x7ff0501080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:07.105 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.103+0000 7ff056d4c700 1 --2- 172.21.15.77:0/488097925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff050101810 0x7ff0501080c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff04800b0d0 tx=0x7ff04800b3e0 comp rx=0 tx=0).stop 2023-12-01T01:23:07.105 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.103+0000 7ff056d4c700 1 -- 172.21.15.77:0/488097925 shutdown_connections 2023-12-01T01:23:07.105 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.103+0000 7ff056d4c700 1 --2- 172.21.15.77:0/488097925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff050101810 0x7ff0501080c0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:07.105 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.103+0000 7ff056d4c700 1 --2- 172.21.15.77:0/488097925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0500ffc80 0x7ff0501000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:07.105 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.103+0000 7ff056d4c700 1 --2- 172.21.15.77:0/488097925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff050100e70 0x7ff0501012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:07.105 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.103+0000 7ff056d4c700 1 -- 172.21.15.77:0/488097925 >> 172.21.15.77:0/488097925 conn(0x7ff0500fb1f0 msgr2=0x7ff0500fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:23:07.105 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.103+0000 7ff056d4c700 1 -- 172.21.15.77:0/488097925 shutdown_connections 2023-12-01T01:23:07.105 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.103+0000 7ff056d4c700 1 -- 172.21.15.77:0/488097925 wait complete. 2023-12-01T01:23:07.105 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.103+0000 7ff056d4c700 1 Processor -- start 2023-12-01T01:23:07.105 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.104+0000 7ff056d4c700 1 -- start start 2023-12-01T01:23:07.107 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.104+0000 7ff056d4c700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0500ffc80 0x7ff050196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:07.107 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.104+0000 7ff056d4c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff050100e70 0x7ff050197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:07.107 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.104+0000 7ff056d4c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff050101810 0x7ff05019e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:07.107 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.104+0000 7ff056d4c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05019e6b0 con 0x7ff050101810 2023-12-01T01:23:07.108 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.104+0000 7ff056d4c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05019e7f0 con 0x7ff050100e70 2023-12-01T01:23:07.108 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.104+0000 7ff056d4c700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05019e980 con 0x7ff0500ffc80 2023-12-01T01:23:07.108 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.104+0000 7ff0552e9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff050101810 0x7ff05019e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:07.108 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.104+0000 7ff0552e9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff050101810 0x7ff05019e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33362/0 (socket says 172.21.15.77:33362) 2023-12-01T01:23:07.108 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.104+0000 7ff0552e9700 1 -- 172.21.15.77:0/2896354645 learned_addr learned my addr 172.21.15.77:0/2896354645 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:23:07.108 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.104+0000 7ff054ae8700 1 --2- 172.21.15.77:0/2896354645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0500ffc80 0x7ff050196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:07.108 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.104+0000 7ff04ffff700 1 --2- 172.21.15.77:0/2896354645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff050100e70 0x7ff050197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:07.108 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.104+0000 7ff0552e9700 1 -- 172.21.15.77:0/2896354645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0500ffc80 msgr2=0x7ff050196b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:07.108 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.104+0000 7ff0552e9700 1 --2- 172.21.15.77:0/2896354645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0500ffc80 0x7ff050196b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:07.108 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.104+0000 7ff0552e9700 1 -- 172.21.15.77:0/2896354645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff050100e70 msgr2=0x7ff050197070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:07.108 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.104+0000 7ff0552e9700 1 --2- 172.21.15.77:0/2896354645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff050100e70 0x7ff050197070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:07.108 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.104+0000 7ff0552e9700 1 -- 172.21.15.77:0/2896354645 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff048009d00 con 0x7ff050101810 2023-12-01T01:23:07.108 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.104+0000 7ff04ffff700 1 --2- 172.21.15.77:0/2896354645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff050100e70 0x7ff050197070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:23:07.108 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.104+0000 7ff0552e9700 1 --2- 172.21.15.77:0/2896354645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff050101810 0x7ff05019e0e0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff048007a00 tx=0x7ff048009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:23:07.109 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.104+0000 7ff04dffb700 1 -- 172.21.15.77:0/2896354645 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04800e040 con 0x7ff050101810 2023-12-01T01:23:07.109 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.104+0000 7ff04dffb700 1 -- 172.21.15.77:0/2896354645 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff048003e10 con 0x7ff050101810 2023-12-01T01:23:07.109 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.104+0000 7ff056d4c700 1 -- 172.21.15.77:0/2896354645 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff05019ec00 con 0x7ff050101810 2023-12-01T01:23:07.109 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.104+0000 7ff04dffb700 1 -- 172.21.15.77:0/2896354645 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff048005610 con 0x7ff050101810 2023-12-01T01:23:07.109 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.105+0000 7ff056d4c700 1 -- 172.21.15.77:0/2896354645 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff05019f020 con 0x7ff050101810 2023-12-01T01:23:07.109 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.105+0000 7ff056d4c700 1 -- 172.21.15.77:0/2896354645 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff050106b00 con 0x7ff050101810 2023-12-01T01:23:07.110 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.106+0000 7ff04dffb700 1 -- 172.21.15.77:0/2896354645 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7ff048005770 con 0x7ff050101810 2023-12-01T01:23:07.110 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.107+0000 7ff04dffb700 1 --2- 172.21.15.77:0/2896354645 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff0400a0520 0x7ff0400a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:07.110 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.107+0000 7ff054ae8700 1 --2- 172.21.15.77:0/2896354645 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff0400a0520 0x7ff0400a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:07.110 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.107+0000 7ff04dffb700 1 -- 172.21.15.77:0/2896354645 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8583+0+0 (secure 0 0 0) 0x7ff0480cd260 con 0x7ff050101810 2023-12-01T01:23:07.110 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.107+0000 7ff054ae8700 1 --2- 172.21.15.77:0/2896354645 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff0400a0520 0x7ff0400a29e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff044005950 tx=0x7ff0440058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:23:07.112 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.111+0000 7ff04dffb700 1 -- 172.21.15.77:0/2896354645 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff048096d30 con 0x7ff050101810 2023-12-01T01:23:07.302 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.301+0000 7ff056d4c700 1 -- 172.21.15.77:0/2896354645 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff05002d090 con 0x7ff050101810 2023-12-01T01:23:07.960 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.959+0000 7ff04dffb700 1 -- 172.21.15.77:0/2896354645 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v86) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff04809a350 con 0x7ff050101810 2023-12-01T01:23:07.964 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.962+0000 7ff056d4c700 1 -- 172.21.15.77:0/2896354645 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff0400a0520 msgr2=0x7ff0400a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:07.964 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.962+0000 7ff056d4c700 1 --2- 172.21.15.77:0/2896354645 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff0400a0520 0x7ff0400a29e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff044005950 tx=0x7ff0440058e0 comp rx=0 tx=0).stop 2023-12-01T01:23:07.964 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.963+0000 7ff056d4c700 1 -- 172.21.15.77:0/2896354645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff050101810 msgr2=0x7ff05019e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:07.964 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.963+0000 7ff056d4c700 1 --2- 172.21.15.77:0/2896354645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff050101810 0x7ff05019e0e0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff048007a00 tx=0x7ff048009570 comp rx=0 tx=0).stop 2023-12-01T01:23:07.964 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.963+0000 7ff056d4c700 1 -- 172.21.15.77:0/2896354645 shutdown_connections 2023-12-01T01:23:07.964 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.963+0000 7ff056d4c700 1 --2- 172.21.15.77:0/2896354645 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7ff0400a0520 0x7ff0400a29e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:07.965 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.963+0000 7ff056d4c700 1 --2- 172.21.15.77:0/2896354645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff050101810 0x7ff05019e0e0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:07.965 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.963+0000 7ff056d4c700 1 --2- 172.21.15.77:0/2896354645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff0500ffc80 0x7ff050196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:07.965 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.963+0000 7ff056d4c700 1 --2- 172.21.15.77:0/2896354645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff050100e70 0x7ff050197070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:07.965 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.963+0000 7ff056d4c700 1 -- 172.21.15.77:0/2896354645 >> 172.21.15.77:0/2896354645 conn(0x7ff0500fb1f0 msgr2=0x7ff0500fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:23:07.965 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.963+0000 7ff056d4c700 1 -- 172.21.15.77:0/2896354645 shutdown_connections 2023-12-01T01:23:07.965 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:07.963+0000 7ff056d4c700 1 -- 172.21.15.77:0/2896354645 wait complete. 2023-12-01T01:23:07.966 INFO:teuthology.orchestra.run.smithi077.stderr:pool 'cephfs_metadata' removed 2023-12-01T01:23:07.978 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:07 smithi077 ceph-mon[98867]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-12-01T01:23:07.978 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:07 smithi077 ceph-mon[98867]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-12-01T01:23:07.978 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:07 smithi077 ceph-mon[98867]: Cluster is now healthy 2023-12-01T01:23:07.979 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:07 smithi077 ceph-mon[98867]: mds.? [v2:172.21.15.77:6839/3121699483,v1:172.21.15.77:6841/3121699483] up:boot 2023-12-01T01:23:07.979 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:07 smithi077 ceph-mon[98867]: mds.? [v2:172.21.15.77:6838/1488765335,v1:172.21.15.77:6840/1488765335] up:boot 2023-12-01T01:23:07.979 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:07 smithi077 ceph-mon[98867]: mds.? [v2:172.21.15.203:6836/153284111,v1:172.21.15.203:6838/153284111] up:boot 2023-12-01T01:23:07.979 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:07 smithi077 ceph-mon[98867]: mds.? [v2:172.21.15.203:6837/2935478232,v1:172.21.15.203:6839/2935478232] up:boot 2023-12-01T01:23:07.979 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:07 smithi077 ceph-mon[98867]: mds.? [v2:172.21.15.137:6832/1343281956,v1:172.21.15.137:6834/1343281956] up:boot 2023-12-01T01:23:07.979 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:07 smithi077 ceph-mon[98867]: mds.? [v2:172.21.15.137:6838/4263247165,v1:172.21.15.137:6839/4263247165] up:boot 2023-12-01T01:23:07.979 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:07 smithi077 ceph-mon[98867]: mds.? [v2:172.21.15.203:6832/2487184367,v1:172.21.15.203:6834/2487184367] up:boot 2023-12-01T01:23:07.979 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:07 smithi077 ceph-mon[98867]: mds.? [v2:172.21.15.203:6833/1403302386,v1:172.21.15.203:6835/1403302386] up:boot 2023-12-01T01:23:07.979 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:07 smithi077 ceph-mon[98867]: mds.? [v2:172.21.15.137:6833/515252503,v1:172.21.15.137:6835/515252503] up:boot 2023-12-01T01:23:07.979 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:07 smithi077 ceph-mon[98867]: mds.? [v2:172.21.15.137:6836/4278346276,v1:172.21.15.137:6837/4278346276] up:boot 2023-12-01T01:23:07.979 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:07 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/1835481065' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-12-01T01:23:07.979 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:07 smithi077 ceph-mon[98867]: fsmap 12 up:standby 2023-12-01T01:23:07.979 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:07 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-01T01:23:07.979 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:07 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-01T01:23:07.980 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:07 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-01T01:23:07.980 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:07 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-01T01:23:07.980 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:07 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-01T01:23:07.980 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:07 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-01T01:23:07.980 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:07 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-01T01:23:07.980 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:07 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-01T01:23:07.980 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:07 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-01T01:23:07.980 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:07 smithi077 ceph-mon[98867]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-01T01:23:07.980 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:07 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/2896354645' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-01T01:23:07.981 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-01T01:23:08.012 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:07 smithi203 ceph-mon[103581]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-12-01T01:23:08.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:07 smithi203 ceph-mon[103581]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-12-01T01:23:08.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:07 smithi203 ceph-mon[103581]: Cluster is now healthy 2023-12-01T01:23:08.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:07 smithi203 ceph-mon[103581]: mds.? [v2:172.21.15.77:6839/3121699483,v1:172.21.15.77:6841/3121699483] up:boot 2023-12-01T01:23:08.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:07 smithi203 ceph-mon[103581]: mds.? [v2:172.21.15.77:6838/1488765335,v1:172.21.15.77:6840/1488765335] up:boot 2023-12-01T01:23:08.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:07 smithi203 ceph-mon[103581]: mds.? [v2:172.21.15.203:6836/153284111,v1:172.21.15.203:6838/153284111] up:boot 2023-12-01T01:23:08.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:07 smithi203 ceph-mon[103581]: mds.? [v2:172.21.15.203:6837/2935478232,v1:172.21.15.203:6839/2935478232] up:boot 2023-12-01T01:23:08.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:07 smithi203 ceph-mon[103581]: mds.? [v2:172.21.15.137:6832/1343281956,v1:172.21.15.137:6834/1343281956] up:boot 2023-12-01T01:23:08.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:07 smithi203 ceph-mon[103581]: mds.? [v2:172.21.15.137:6838/4263247165,v1:172.21.15.137:6839/4263247165] up:boot 2023-12-01T01:23:08.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:07 smithi203 ceph-mon[103581]: mds.? [v2:172.21.15.203:6832/2487184367,v1:172.21.15.203:6834/2487184367] up:boot 2023-12-01T01:23:08.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:07 smithi203 ceph-mon[103581]: mds.? [v2:172.21.15.203:6833/1403302386,v1:172.21.15.203:6835/1403302386] up:boot 2023-12-01T01:23:08.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:07 smithi203 ceph-mon[103581]: mds.? [v2:172.21.15.137:6833/515252503,v1:172.21.15.137:6835/515252503] up:boot 2023-12-01T01:23:08.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:07 smithi203 ceph-mon[103581]: mds.? [v2:172.21.15.137:6836/4278346276,v1:172.21.15.137:6837/4278346276] up:boot 2023-12-01T01:23:08.013 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:07 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/1835481065' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-12-01T01:23:08.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:07 smithi203 ceph-mon[103581]: fsmap 12 up:standby 2023-12-01T01:23:08.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:07 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-01T01:23:08.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:07 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-01T01:23:08.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:07 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-01T01:23:08.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:07 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-01T01:23:08.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:07 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-01T01:23:08.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:07 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-01T01:23:08.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:07 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-01T01:23:08.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:07 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-01T01:23:08.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:07 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-01T01:23:08.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:07 smithi203 ceph-mon[103581]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-01T01:23:08.014 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:07 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/2896354645' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-01T01:23:08.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:07 smithi137 ceph-mon[103312]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-12-01T01:23:08.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:07 smithi137 ceph-mon[103312]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-12-01T01:23:08.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:07 smithi137 ceph-mon[103312]: Cluster is now healthy 2023-12-01T01:23:08.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:07 smithi137 ceph-mon[103312]: mds.? [v2:172.21.15.77:6839/3121699483,v1:172.21.15.77:6841/3121699483] up:boot 2023-12-01T01:23:08.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:07 smithi137 ceph-mon[103312]: mds.? [v2:172.21.15.77:6838/1488765335,v1:172.21.15.77:6840/1488765335] up:boot 2023-12-01T01:23:08.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:07 smithi137 ceph-mon[103312]: mds.? [v2:172.21.15.203:6836/153284111,v1:172.21.15.203:6838/153284111] up:boot 2023-12-01T01:23:08.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:07 smithi137 ceph-mon[103312]: mds.? [v2:172.21.15.203:6837/2935478232,v1:172.21.15.203:6839/2935478232] up:boot 2023-12-01T01:23:08.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:07 smithi137 ceph-mon[103312]: mds.? [v2:172.21.15.137:6832/1343281956,v1:172.21.15.137:6834/1343281956] up:boot 2023-12-01T01:23:08.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:07 smithi137 ceph-mon[103312]: mds.? [v2:172.21.15.137:6838/4263247165,v1:172.21.15.137:6839/4263247165] up:boot 2023-12-01T01:23:08.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:07 smithi137 ceph-mon[103312]: mds.? [v2:172.21.15.203:6832/2487184367,v1:172.21.15.203:6834/2487184367] up:boot 2023-12-01T01:23:08.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:07 smithi137 ceph-mon[103312]: mds.? [v2:172.21.15.203:6833/1403302386,v1:172.21.15.203:6835/1403302386] up:boot 2023-12-01T01:23:08.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:07 smithi137 ceph-mon[103312]: mds.? [v2:172.21.15.137:6833/515252503,v1:172.21.15.137:6835/515252503] up:boot 2023-12-01T01:23:08.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:07 smithi137 ceph-mon[103312]: mds.? [v2:172.21.15.137:6836/4278346276,v1:172.21.15.137:6837/4278346276] up:boot 2023-12-01T01:23:08.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:07 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/1835481065' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-12-01T01:23:08.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:07 smithi137 ceph-mon[103312]: fsmap 12 up:standby 2023-12-01T01:23:08.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:07 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-01T01:23:08.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:07 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-01T01:23:08.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:07 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-01T01:23:08.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:07 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-01T01:23:08.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:07 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-01T01:23:08.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:07 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-01T01:23:08.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:07 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-01T01:23:08.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:07 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-01T01:23:08.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:07 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-01T01:23:08.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:07 smithi137 ceph-mon[103312]: from='mgr.14152 172.21.15.77:0/3307081570' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-01T01:23:08.031 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:07 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/2896354645' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-01T01:23:08.110 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.108+0000 7f06647ef700 1 -- 172.21.15.77:0/1102154590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f065c101880 msgr2=0x7f065c108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:08.110 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.108+0000 7f06647ef700 1 --2- 172.21.15.77:0/1102154590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f065c101880 0x7f065c108130 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f065400b0d0 tx=0x7f065400b3e0 comp rx=0 tx=0).stop 2023-12-01T01:23:08.110 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.108+0000 7f06647ef700 1 -- 172.21.15.77:0/1102154590 shutdown_connections 2023-12-01T01:23:08.110 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.108+0000 7f06647ef700 1 --2- 172.21.15.77:0/1102154590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f065c101880 0x7f065c108130 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:08.110 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.108+0000 7f06647ef700 1 --2- 172.21.15.77:0/1102154590 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f065c0ffcf0 0x7f065c100110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:08.110 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.108+0000 7f06647ef700 1 --2- 172.21.15.77:0/1102154590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f065c100ee0 0x7f065c101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:08.111 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.108+0000 7f06647ef700 1 -- 172.21.15.77:0/1102154590 >> 172.21.15.77:0/1102154590 conn(0x7f065c0fb260 msgr2=0x7f065c0fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:23:08.111 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.108+0000 7f06647ef700 1 -- 172.21.15.77:0/1102154590 shutdown_connections 2023-12-01T01:23:08.111 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.108+0000 7f06647ef700 1 -- 172.21.15.77:0/1102154590 wait complete. 2023-12-01T01:23:08.111 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.109+0000 7f06647ef700 1 Processor -- start 2023-12-01T01:23:08.111 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.109+0000 7f06647ef700 1 -- start start 2023-12-01T01:23:08.113 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.109+0000 7f06647ef700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f065c0ffcf0 0x7f065c195a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:08.113 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.109+0000 7f06647ef700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f065c100ee0 0x7f065c195f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:08.113 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.109+0000 7f06647ef700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f065c101880 0x7f065c19cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:08.113 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.109+0000 7f06647ef700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065c19d5c0 con 0x7f065c100ee0 2023-12-01T01:23:08.113 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.109+0000 7f06647ef700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065c19d700 con 0x7f065c0ffcf0 2023-12-01T01:23:08.113 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.109+0000 7f06647ef700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065c19d890 con 0x7f065c101880 2023-12-01T01:23:08.113 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.109+0000 7f0661d8a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f065c100ee0 0x7f065c195f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:08.113 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.109+0000 7f0661d8a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f065c100ee0 0x7f065c195f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33376/0 (socket says 172.21.15.77:33376) 2023-12-01T01:23:08.113 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.109+0000 7f0661d8a700 1 -- 172.21.15.77:0/1410254034 learned_addr learned my addr 172.21.15.77:0/1410254034 (peer_addr_for_me v2:172.21.15.77:0/0) 2023-12-01T01:23:08.113 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.109+0000 7f066258b700 1 --2- 172.21.15.77:0/1410254034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f065c0ffcf0 0x7f065c195a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:08.114 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.110+0000 7f0662d8c700 1 --2- 172.21.15.77:0/1410254034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f065c101880 0x7f065c19cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:08.114 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.110+0000 7f0661d8a700 1 -- 172.21.15.77:0/1410254034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f065c101880 msgr2=0x7f065c19cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:08.114 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.110+0000 7f0661d8a700 1 --2- 172.21.15.77:0/1410254034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f065c101880 0x7f065c19cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:08.114 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.110+0000 7f0661d8a700 1 -- 172.21.15.77:0/1410254034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f065c0ffcf0 msgr2=0x7f065c195a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:08.114 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.110+0000 7f0661d8a700 1 --2- 172.21.15.77:0/1410254034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f065c0ffcf0 0x7f065c195a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:08.114 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.110+0000 7f0661d8a700 1 -- 172.21.15.77:0/1410254034 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0654009d00 con 0x7f065c100ee0 2023-12-01T01:23:08.114 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.110+0000 7f0662d8c700 1 --2- 172.21.15.77:0/1410254034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f065c101880 0x7f065c19cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-01T01:23:08.114 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.110+0000 7f0661d8a700 1 --2- 172.21.15.77:0/1410254034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f065c100ee0 0x7f065c195f80 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f065800d950 tx=0x7f065800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:23:08.114 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.110+0000 7f06537fe700 1 -- 172.21.15.77:0/1410254034 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0658009b40 con 0x7f065c100ee0 2023-12-01T01:23:08.114 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.110+0000 7f06537fe700 1 -- 172.21.15.77:0/1410254034 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0658009ca0 con 0x7f065c100ee0 2023-12-01T01:23:08.114 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.110+0000 7f06647ef700 1 -- 172.21.15.77:0/1410254034 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f065c19db70 con 0x7f065c100ee0 2023-12-01T01:23:08.114 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.110+0000 7f06537fe700 1 -- 172.21.15.77:0/1410254034 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065800f620 con 0x7f065c100ee0 2023-12-01T01:23:08.114 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.110+0000 7f06647ef700 1 -- 172.21.15.77:0/1410254034 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f065c19e080 con 0x7f065c100ee0 2023-12-01T01:23:08.115 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.111+0000 7f06647ef700 1 -- 172.21.15.77:0/1410254034 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f065c107240 con 0x7f065c100ee0 2023-12-01T01:23:08.115 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.112+0000 7f06537fe700 1 -- 172.21.15.77:0/1410254034 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) v1 ==== 135070+0+0 (secure 0 0 0) 0x7f0658010460 con 0x7f065c100ee0 2023-12-01T01:23:08.115 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.113+0000 7f06537fe700 1 --2- 172.21.15.77:0/1410254034 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f06480a0520 0x7f06480a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-01T01:23:08.116 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.113+0000 7f066258b700 1 --2- 172.21.15.77:0/1410254034 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f06480a0520 0x7f06480a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-01T01:23:08.116 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.113+0000 7f06537fe700 1 -- 172.21.15.77:0/1410254034 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8087+0+0 (secure 0 0 0) 0x7f06580cbcd0 con 0x7f065c100ee0 2023-12-01T01:23:08.116 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.113+0000 7f066258b700 1 --2- 172.21.15.77:0/1410254034 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f06480a0520 0x7f06480a29e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f064c005fd0 tx=0x7f064c005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-01T01:23:08.119 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.117+0000 7f06537fe700 1 -- 172.21.15.77:0/1410254034 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f06580958b0 con 0x7f065c100ee0 2023-12-01T01:23:08.323 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.322+0000 7f06647ef700 1 -- 172.21.15.77:0/1410254034 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f065c02cc70 con 0x7f065c100ee0 2023-12-01T01:23:08.967 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:08 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/2896354645' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-12-01T01:23:08.967 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:08 smithi077 ceph-mon[98867]: osdmap e86: 12 total, 12 up, 12 in 2023-12-01T01:23:08.967 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:08 smithi077 ceph-mon[98867]: from='client.? 172.21.15.77:0/1410254034' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-01T01:23:08.967 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:08 smithi077 ceph-mon[98867]: pgmap v458: 65 pgs: 65 active+clean; 577 KiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:23:08.967 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.965+0000 7f06537fe700 1 -- 172.21.15.77:0/1410254034 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v87) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0658098ed0 con 0x7f065c100ee0 2023-12-01T01:23:08.971 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.970+0000 7f06647ef700 1 -- 172.21.15.77:0/1410254034 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f06480a0520 msgr2=0x7f06480a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:08.971 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.970+0000 7f06647ef700 1 --2- 172.21.15.77:0/1410254034 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f06480a0520 0x7f06480a29e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f064c005fd0 tx=0x7f064c005dd0 comp rx=0 tx=0).stop 2023-12-01T01:23:08.971 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.970+0000 7f06647ef700 1 -- 172.21.15.77:0/1410254034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f065c100ee0 msgr2=0x7f065c195f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-01T01:23:08.971 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.970+0000 7f06647ef700 1 --2- 172.21.15.77:0/1410254034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f065c100ee0 0x7f065c195f80 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f065800d950 tx=0x7f065800dd10 comp rx=0 tx=0).stop 2023-12-01T01:23:08.973 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.972+0000 7f06647ef700 1 -- 172.21.15.77:0/1410254034 shutdown_connections 2023-12-01T01:23:08.973 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.972+0000 7f06647ef700 1 --2- 172.21.15.77:0/1410254034 >> [v2:172.21.15.77:6800/3838941679,v1:172.21.15.77:6801/3838941679] conn(0x7f06480a0520 0x7f06480a29e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:08.974 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.972+0000 7f06647ef700 1 --2- 172.21.15.77:0/1410254034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f065c101880 0x7f065c19cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:08.974 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.972+0000 7f06647ef700 1 --2- 172.21.15.77:0/1410254034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f065c0ffcf0 0x7f065c195a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:08.974 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.972+0000 7f06647ef700 1 --2- 172.21.15.77:0/1410254034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f065c100ee0 0x7f065c195f80 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-01T01:23:08.974 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.972+0000 7f06647ef700 1 -- 172.21.15.77:0/1410254034 >> 172.21.15.77:0/1410254034 conn(0x7f065c0fb260 msgr2=0x7f065c104380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-01T01:23:08.975 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.974+0000 7f06647ef700 1 -- 172.21.15.77:0/1410254034 shutdown_connections 2023-12-01T01:23:08.976 INFO:teuthology.orchestra.run.smithi077.stderr:2023-12-01T01:23:08.974+0000 7f06647ef700 1 -- 172.21.15.77:0/1410254034 wait complete. 2023-12-01T01:23:08.976 INFO:teuthology.orchestra.run.smithi077.stderr:pool 'cephfs_data' removed 2023-12-01T01:23:08.998 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-01T01:23:09.044 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-01T01:23:09.077 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-01T01:23:09.109 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-12-01T01:23:09.110 DEBUG:teuthology.orchestra.run.smithi077:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-12-01T01:23:09.127 DEBUG:teuthology.orchestra.run.smithi137:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-12-01T01:23:09.144 DEBUG:teuthology.orchestra.run.smithi203:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-12-01T01:23:09.166 INFO:tasks.cephadm:Stopping all daemons... 2023-12-01T01:23:09.167 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-12-01T01:23:09.167 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mon.a 2023-12-01T01:23:09.262 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:08 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/2896354645' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-12-01T01:23:09.262 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:08 smithi203 ceph-mon[103581]: osdmap e86: 12 total, 12 up, 12 in 2023-12-01T01:23:09.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:08 smithi203 ceph-mon[103581]: from='client.? 172.21.15.77:0/1410254034' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-01T01:23:09.263 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:08 smithi203 ceph-mon[103581]: pgmap v458: 65 pgs: 65 active+clean; 577 KiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:23:09.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:08 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/2896354645' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-12-01T01:23:09.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:08 smithi137 ceph-mon[103312]: osdmap e86: 12 total, 12 up, 12 in 2023-12-01T01:23:09.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:08 smithi137 ceph-mon[103312]: from='client.? 172.21.15.77:0/1410254034' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-01T01:23:09.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:08 smithi137 ceph-mon[103312]: pgmap v458: 65 pgs: 65 active+clean; 577 KiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail 2023-12-01T01:23:09.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:09 smithi077 systemd[1]: Stopping Ceph mon.a for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:23:10.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:09 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mon-a[98843]: 2023-12-01T01:23:09.881+0000 7f988d612700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-12-01T01:23:10.154 INFO:journalctl@ceph.mon.a.smithi077.stdout:Dec 01 01:23:09 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mon-a[98843]: 2023-12-01T01:23:09.881+0000 7f988d612700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-12-01T01:23:10.450 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mon.a.service' 2023-12-01T01:23:10.510 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-01T01:23:10.510 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-12-01T01:23:10.510 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-12-01T01:23:10.511 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mon.b 2023-12-01T01:23:10.877 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:10 smithi137 systemd[1]: Stopping Ceph mon.b for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:23:11.208 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:10 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mon-b[103288]: 2023-12-01T01:23:10.875+0000 7f6a3ff5a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-12-01T01:23:11.208 INFO:journalctl@ceph.mon.b.smithi137.stdout:Dec 01 01:23:10 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mon-b[103288]: 2023-12-01T01:23:10.875+0000 7f6a3ff5a700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-12-01T01:23:11.412 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mon.b.service' 2023-12-01T01:23:11.465 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-01T01:23:11.466 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-12-01T01:23:11.466 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-12-01T01:23:11.466 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mon.c 2023-12-01T01:23:11.762 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:11 smithi203 systemd[1]: Stopping Ceph mon.c for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:23:12.202 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:11 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mon-c[103529]: 2023-12-01T01:23:11.826+0000 7f71e4f47700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-12-01T01:23:12.202 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:11 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mon-c[103529]: 2023-12-01T01:23:11.826+0000 7f71e4f47700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-12-01T01:23:12.474 INFO:journalctl@ceph.mon.c.smithi203.stdout:Dec 01 01:23:12 smithi203 bash[145209]: ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mon-c 2023-12-01T01:23:12.481 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mon.c.service' 2023-12-01T01:23:12.540 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-01T01:23:12.540 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-12-01T01:23:12.540 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-12-01T01:23:12.540 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mgr.x 2023-12-01T01:23:12.888 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:23:12 smithi077 systemd[1]: Stopping Ceph mgr.x for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:23:13.154 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:23:12 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x[99117]: 2023-12-01T01:23:12.886+0000 7f747f7c0700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-01T01:23:13.428 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mgr.x.service' 2023-12-01T01:23:13.454 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:23:13 smithi077 bash[186949]: ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-x 2023-12-01T01:23:13.454 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:23:13 smithi077 systemd[1]: ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mgr.x.service: Succeeded. 2023-12-01T01:23:13.454 INFO:journalctl@ceph.mgr.x.smithi077.stdout:Dec 01 01:23:13 smithi077 systemd[1]: Stopped Ceph mgr.x for 2a06e29a-8fe6-11ee-95a2-87774f69a715. 2023-12-01T01:23:13.466 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-01T01:23:13.466 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-12-01T01:23:13.466 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-12-01T01:23:13.466 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mgr.y 2023-12-01T01:23:13.781 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Dec 01 01:23:13 smithi137 systemd[1]: Stopping Ceph mgr.y for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:23:14.175 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Dec 01 01:23:14 smithi137 bash[145205]: ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-y 2023-12-01T01:23:14.355 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Dec 01 01:23:14 smithi137 systemd[1]: ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-12-01T01:23:14.360 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mgr.y.service' 2023-12-01T01:23:14.397 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-01T01:23:14.397 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-12-01T01:23:14.397 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-12-01T01:23:14.397 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mgr.z 2023-12-01T01:23:14.762 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Dec 01 01:23:14 smithi203 systemd[1]: Stopping Ceph mgr.z for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:23:15.286 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Dec 01 01:23:15 smithi203 bash[145359]: ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mgr-z 2023-12-01T01:23:15.286 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Dec 01 01:23:15 smithi203 systemd[1]: ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-12-01T01:23:15.291 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mgr.z.service' 2023-12-01T01:23:15.328 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-01T01:23:15.328 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-12-01T01:23:15.328 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-12-01T01:23:15.328 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@osd.0 2023-12-01T01:23:15.654 INFO:journalctl@ceph.osd.0.smithi077.stdout:Dec 01 01:23:15 smithi077 systemd[1]: Stopping Ceph osd.0 for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:23:16.154 INFO:journalctl@ceph.osd.0.smithi077.stdout:Dec 01 01:23:15 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-0[117495]: 2023-12-01T01:23:15.686+0000 7fdc973a2700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-01T01:23:16.154 INFO:journalctl@ceph.osd.0.smithi077.stdout:Dec 01 01:23:15 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-0[117495]: 2023-12-01T01:23:15.686+0000 7fdc973a2700 -1 osd.0 87 *** Got signal Terminated *** 2023-12-01T01:23:16.154 INFO:journalctl@ceph.osd.0.smithi077.stdout:Dec 01 01:23:15 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-0[117495]: 2023-12-01T01:23:15.686+0000 7fdc973a2700 -1 osd.0 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-01T01:23:21.462 INFO:journalctl@ceph.osd.0.smithi077.stdout:Dec 01 01:23:21 smithi077 bash[187094]: ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-0 2023-12-01T01:23:22.726 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@osd.0.service' 2023-12-01T01:23:22.767 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-01T01:23:22.767 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-12-01T01:23:22.767 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-12-01T01:23:22.767 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@osd.1 2023-12-01T01:23:23.056 INFO:journalctl@ceph.osd.1.smithi137.stdout:Dec 01 01:23:22 smithi137 systemd[1]: Stopping Ceph osd.1 for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:23:23.529 INFO:journalctl@ceph.osd.1.smithi137.stdout:Dec 01 01:23:23 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-1[108281]: 2023-12-01T01:23:23.114+0000 7fc6196cb700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-01T01:23:23.529 INFO:journalctl@ceph.osd.1.smithi137.stdout:Dec 01 01:23:23 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-1[108281]: 2023-12-01T01:23:23.114+0000 7fc6196cb700 -1 osd.1 87 *** Got signal Terminated *** 2023-12-01T01:23:23.529 INFO:journalctl@ceph.osd.1.smithi137.stdout:Dec 01 01:23:23 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-1[108281]: 2023-12-01T01:23:23.114+0000 7fc6196cb700 -1 osd.1 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-01T01:23:29.029 INFO:journalctl@ceph.osd.1.smithi137.stdout:Dec 01 01:23:28 smithi137 bash[145346]: ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-1 2023-12-01T01:23:30.263 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@osd.1.service' 2023-12-01T01:23:30.303 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-01T01:23:30.303 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-12-01T01:23:30.303 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-12-01T01:23:30.303 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@osd.2 2023-12-01T01:23:30.668 INFO:journalctl@ceph.osd.2.smithi203.stdout:Dec 01 01:23:30 smithi203 systemd[1]: Stopping Ceph osd.2 for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:23:31.012 INFO:journalctl@ceph.osd.2.smithi203.stdout:Dec 01 01:23:30 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-2[109394]: 2023-12-01T01:23:30.666+0000 7f1a52fc8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-01T01:23:31.012 INFO:journalctl@ceph.osd.2.smithi203.stdout:Dec 01 01:23:30 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-2[109394]: 2023-12-01T01:23:30.666+0000 7f1a52fc8700 -1 osd.2 87 *** Got signal Terminated *** 2023-12-01T01:23:31.012 INFO:journalctl@ceph.osd.2.smithi203.stdout:Dec 01 01:23:30 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-2[109394]: 2023-12-01T01:23:30.666+0000 7f1a52fc8700 -1 osd.2 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-01T01:23:36.342 INFO:journalctl@ceph.osd.2.smithi203.stdout:Dec 01 01:23:36 smithi203 bash[145508]: ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-2 2023-12-01T01:23:37.709 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@osd.2.service' 2023-12-01T01:23:37.750 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-01T01:23:37.750 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-12-01T01:23:37.750 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-12-01T01:23:37.750 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@osd.3 2023-12-01T01:23:38.118 INFO:journalctl@ceph.osd.3.smithi077.stdout:Dec 01 01:23:37 smithi077 systemd[1]: Stopping Ceph osd.3 for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:23:38.404 INFO:journalctl@ceph.osd.3.smithi077.stdout:Dec 01 01:23:38 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-3[125398]: 2023-12-01T01:23:38.116+0000 7ff0d6fc2700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-01T01:23:38.404 INFO:journalctl@ceph.osd.3.smithi077.stdout:Dec 01 01:23:38 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-3[125398]: 2023-12-01T01:23:38.116+0000 7ff0d6fc2700 -1 osd.3 87 *** Got signal Terminated *** 2023-12-01T01:23:38.404 INFO:journalctl@ceph.osd.3.smithi077.stdout:Dec 01 01:23:38 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-3[125398]: 2023-12-01T01:23:38.116+0000 7ff0d6fc2700 -1 osd.3 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-01T01:23:42.654 INFO:journalctl@ceph.osd.3.smithi077.stdout:Dec 01 01:23:42 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-3[125398]: 2023-12-01T01:23:42.323+0000 7ff0d2db7700 -1 osd.3 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:18.804404+0000 front 2023-12-01T01:23:18.804421+0000 (oldest deadline 2023-12-01T01:23:41.704650+0000) 2023-12-01T01:23:43.262 INFO:journalctl@ceph.osd.5.smithi203.stdout:Dec 01 01:23:42 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-5[118084]: 2023-12-01T01:23:42.841+0000 7f5008ab5700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:17.231033+0000 front 2023-12-01T01:23:17.231063+0000 (oldest deadline 2023-12-01T01:23:41.931024+0000) 2023-12-01T01:23:43.791 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:23:43 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:23:43.537+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.288397+0000 front 2023-12-01T01:23:19.288373+0000 (oldest deadline 2023-12-01T01:23:43.388594+0000) 2023-12-01T01:23:44.154 INFO:journalctl@ceph.osd.3.smithi077.stdout:Dec 01 01:23:43 smithi077 bash[187368]: ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-3 2023-12-01T01:23:44.262 INFO:journalctl@ceph.osd.5.smithi203.stdout:Dec 01 01:23:43 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-5[118084]: 2023-12-01T01:23:43.795+0000 7f5008ab5700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:17.231033+0000 front 2023-12-01T01:23:17.231063+0000 (oldest deadline 2023-12-01T01:23:41.931024+0000) 2023-12-01T01:23:44.833 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:23:44 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:23:44.487+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.288397+0000 front 2023-12-01T01:23:19.288373+0000 (oldest deadline 2023-12-01T01:23:43.388594+0000) 2023-12-01T01:23:45.147 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:23:44 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:23:44.882+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:23:45.158 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@osd.3.service' 2023-12-01T01:23:45.179 INFO:journalctl@ceph.osd.5.smithi203.stdout:Dec 01 01:23:44 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-5[118084]: 2023-12-01T01:23:44.829+0000 7f5008ab5700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:17.231033+0000 front 2023-12-01T01:23:17.231063+0000 (oldest deadline 2023-12-01T01:23:41.931024+0000) 2023-12-01T01:23:45.200 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-01T01:23:45.200 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-12-01T01:23:45.200 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-12-01T01:23:45.200 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@osd.4 2023-12-01T01:23:45.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:23:45 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:23:45.177+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:23:45.528 INFO:journalctl@ceph.osd.4.smithi137.stdout:Dec 01 01:23:45 smithi137 systemd[1]: Stopping Ceph osd.4 for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:23:45.897 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:23:45 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:23:45.497+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.288397+0000 front 2023-12-01T01:23:19.288373+0000 (oldest deadline 2023-12-01T01:23:43.388594+0000) 2023-12-01T01:23:46.028 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:23:45 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:23:45.720+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:23:46.029 INFO:journalctl@ceph.osd.4.smithi137.stdout:Dec 01 01:23:45 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-4[116898]: 2023-12-01T01:23:45.552+0000 7f69420cc700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-01T01:23:46.029 INFO:journalctl@ceph.osd.4.smithi137.stdout:Dec 01 01:23:45 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-4[116898]: 2023-12-01T01:23:45.552+0000 7f69420cc700 -1 osd.4 87 *** Got signal Terminated *** 2023-12-01T01:23:46.029 INFO:journalctl@ceph.osd.4.smithi137.stdout:Dec 01 01:23:45 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-4[116898]: 2023-12-01T01:23:45.552+0000 7f69420cc700 -1 osd.4 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-01T01:23:46.154 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:23:45 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:23:45.896+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:23:46.181 INFO:journalctl@ceph.osd.5.smithi203.stdout:Dec 01 01:23:45 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-5[118084]: 2023-12-01T01:23:45.786+0000 7f5008ab5700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:17.231033+0000 front 2023-12-01T01:23:17.231063+0000 (oldest deadline 2023-12-01T01:23:41.931024+0000) 2023-12-01T01:23:46.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:23:46 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:23:46.180+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:23:46.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:23:46 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:23:46.456+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.288397+0000 front 2023-12-01T01:23:19.288373+0000 (oldest deadline 2023-12-01T01:23:43.388594+0000) 2023-12-01T01:23:47.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:23:46 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:23:46.754+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:23:47.164 INFO:journalctl@ceph.osd.5.smithi203.stdout:Dec 01 01:23:46 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-5[118084]: 2023-12-01T01:23:46.763+0000 7f5008ab5700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:17.231033+0000 front 2023-12-01T01:23:17.231063+0000 (oldest deadline 2023-12-01T01:23:41.931024+0000) 2023-12-01T01:23:47.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:23:46 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:23:46.903+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:23:47.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:23:47 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:23:47.162+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:23:47.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:23:47 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:23:47.436+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.288397+0000 front 2023-12-01T01:23:19.288373+0000 (oldest deadline 2023-12-01T01:23:43.388594+0000) 2023-12-01T01:23:48.167 INFO:journalctl@ceph.osd.5.smithi203.stdout:Dec 01 01:23:47 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-5[118084]: 2023-12-01T01:23:47.778+0000 7f5008ab5700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:17.231033+0000 front 2023-12-01T01:23:17.231063+0000 (oldest deadline 2023-12-01T01:23:41.931024+0000) 2023-12-01T01:23:48.278 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:23:47 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:23:47.801+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:23:48.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:23:47 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:23:47.935+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:23:48.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:23:48 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:23:48.165+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:23:48.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:23:48 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:23:48.484+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.288397+0000 front 2023-12-01T01:23:19.288373+0000 (oldest deadline 2023-12-01T01:23:43.388594+0000) 2023-12-01T01:23:49.143 INFO:journalctl@ceph.osd.5.smithi203.stdout:Dec 01 01:23:48 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-5[118084]: 2023-12-01T01:23:48.768+0000 7f5008ab5700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:17.231033+0000 front 2023-12-01T01:23:17.231063+0000 (oldest deadline 2023-12-01T01:23:41.931024+0000) 2023-12-01T01:23:49.278 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:23:48 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:23:48.804+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:23:49.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:23:48 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:23:48.983+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:23:49.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:23:49 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:23:49.141+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:23:49.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:23:49 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:23:49.491+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.288397+0000 front 2023-12-01T01:23:19.288373+0000 (oldest deadline 2023-12-01T01:23:43.388594+0000) 2023-12-01T01:23:49.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:23:49 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:23:49.491+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:23.389153+0000 front 2023-12-01T01:23:23.389160+0000 (oldest deadline 2023-12-01T01:23:48.689179+0000) 2023-12-01T01:23:50.162 INFO:journalctl@ceph.osd.5.smithi203.stdout:Dec 01 01:23:49 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-5[118084]: 2023-12-01T01:23:49.760+0000 7f5008ab5700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:17.231033+0000 front 2023-12-01T01:23:17.231063+0000 (oldest deadline 2023-12-01T01:23:41.931024+0000) 2023-12-01T01:23:50.279 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:23:49 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:23:49.800+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:23:50.279 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:23:49 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:23:49.800+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:23:50.279 INFO:journalctl@ceph.osd.4.smithi137.stdout:Dec 01 01:23:49 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-4[116898]: 2023-12-01T01:23:49.947+0000 7f693e6c2700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.023886+0000 front 2023-12-01T01:23:27.023858+0000 (oldest deadline 2023-12-01T01:23:49.824566+0000) 2023-12-01T01:23:50.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:23:49 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:23:49.990+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:23:50.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:23:50 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:23:50.160+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:23:50.778 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:23:50 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:23:50.391+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.648037+0000 front 2023-12-01T01:23:25.648097+0000 (oldest deadline 2023-12-01T01:23:49.748180+0000) 2023-12-01T01:23:50.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:23:50 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:23:50.540+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.288397+0000 front 2023-12-01T01:23:19.288373+0000 (oldest deadline 2023-12-01T01:23:43.388594+0000) 2023-12-01T01:23:50.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:23:50 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:23:50.540+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:23.389153+0000 front 2023-12-01T01:23:23.389160+0000 (oldest deadline 2023-12-01T01:23:48.689179+0000) 2023-12-01T01:23:51.157 INFO:journalctl@ceph.osd.5.smithi203.stdout:Dec 01 01:23:50 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-5[118084]: 2023-12-01T01:23:50.765+0000 7f5008ab5700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:17.231033+0000 front 2023-12-01T01:23:17.231063+0000 (oldest deadline 2023-12-01T01:23:41.931024+0000) 2023-12-01T01:23:51.157 INFO:journalctl@ceph.osd.5.smithi203.stdout:Dec 01 01:23:50 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-5[118084]: 2023-12-01T01:23:50.765+0000 7f5008ab5700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.832060+0000 front 2023-12-01T01:23:27.832188+0000 (oldest deadline 2023-12-01T01:23:50.131913+0000) 2023-12-01T01:23:51.158 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:23:50 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:23:50.815+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:23:51.158 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:23:50 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:23:50.815+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:23:51.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:23:50 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:23:50.990+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:23:51.438 INFO:journalctl@ceph.osd.4.smithi137.stdout:Dec 01 01:23:51 smithi137 bash[145610]: ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-4 2023-12-01T01:23:51.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:23:51 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:23:51.361+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.051505+0000 front 2023-12-01T01:23:25.051469+0000 (oldest deadline 2023-12-01T01:23:50.852139+0000) 2023-12-01T01:23:51.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:23:51 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:23:51.155+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:23:51.779 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:23:51 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:23:51.436+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.648037+0000 front 2023-12-01T01:23:25.648097+0000 (oldest deadline 2023-12-01T01:23:49.748180+0000) 2023-12-01T01:23:51.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:23:51 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:23:51.551+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.288397+0000 front 2023-12-01T01:23:19.288373+0000 (oldest deadline 2023-12-01T01:23:43.388594+0000) 2023-12-01T01:23:51.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:23:51 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:23:51.551+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:23.389153+0000 front 2023-12-01T01:23:23.389160+0000 (oldest deadline 2023-12-01T01:23:48.689179+0000) 2023-12-01T01:23:52.012 INFO:journalctl@ceph.osd.5.smithi203.stdout:Dec 01 01:23:51 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-5[118084]: 2023-12-01T01:23:51.719+0000 7f5008ab5700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:17.231033+0000 front 2023-12-01T01:23:17.231063+0000 (oldest deadline 2023-12-01T01:23:41.931024+0000) 2023-12-01T01:23:52.012 INFO:journalctl@ceph.osd.5.smithi203.stdout:Dec 01 01:23:51 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-5[118084]: 2023-12-01T01:23:51.719+0000 7f5008ab5700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.832060+0000 front 2023-12-01T01:23:27.832188+0000 (oldest deadline 2023-12-01T01:23:50.131913+0000) 2023-12-01T01:23:52.279 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:23:51 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:23:51.805+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:23:52.279 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:23:51 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:23:51.805+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:23:52.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:23:51 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:23:51.967+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:23:52.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:23:52 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:23:52.384+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.051505+0000 front 2023-12-01T01:23:25.051469+0000 (oldest deadline 2023-12-01T01:23:50.852139+0000) 2023-12-01T01:23:52.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:23:52 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:23:52.177+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:23:52.696 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:23:52 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:23:52.463+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.648037+0000 front 2023-12-01T01:23:25.648097+0000 (oldest deadline 2023-12-01T01:23:49.748180+0000) 2023-12-01T01:23:52.703 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@osd.4.service' 2023-12-01T01:23:52.742 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-01T01:23:52.743 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-12-01T01:23:52.743 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-12-01T01:23:52.743 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@osd.5 2023-12-01T01:23:52.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:23:52 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:23:52.559+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.288397+0000 front 2023-12-01T01:23:19.288373+0000 (oldest deadline 2023-12-01T01:23:43.388594+0000) 2023-12-01T01:23:52.905 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:23:52 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:23:52.559+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:23.389153+0000 front 2023-12-01T01:23:23.389160+0000 (oldest deadline 2023-12-01T01:23:48.689179+0000) 2023-12-01T01:23:53.028 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:23:52 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:23:52.757+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:23:53.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:23:52 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:23:52.757+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:23:53.114 INFO:journalctl@ceph.osd.5.smithi203.stdout:Dec 01 01:23:52 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-5[118084]: 2023-12-01T01:23:52.761+0000 7f5008ab5700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:17.231033+0000 front 2023-12-01T01:23:17.231063+0000 (oldest deadline 2023-12-01T01:23:41.931024+0000) 2023-12-01T01:23:53.114 INFO:journalctl@ceph.osd.5.smithi203.stdout:Dec 01 01:23:52 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-5[118084]: 2023-12-01T01:23:52.761+0000 7f5008ab5700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.832060+0000 front 2023-12-01T01:23:27.832188+0000 (oldest deadline 2023-12-01T01:23:50.131913+0000) 2023-12-01T01:23:53.114 INFO:journalctl@ceph.osd.5.smithi203.stdout:Dec 01 01:23:52 smithi203 systemd[1]: Stopping Ceph osd.5 for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:23:53.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:23:53 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:23:53.006+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:23:53.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:23:53 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:23:53.344+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.051505+0000 front 2023-12-01T01:23:25.051469+0000 (oldest deadline 2023-12-01T01:23:50.852139+0000) 2023-12-01T01:23:53.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:23:53 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:23:53.204+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:23:53.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:23:53 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:23:53.204+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:23:53.513 INFO:journalctl@ceph.osd.5.smithi203.stdout:Dec 01 01:23:53 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-5[118084]: 2023-12-01T01:23:53.111+0000 7f500e4c3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-01T01:23:53.513 INFO:journalctl@ceph.osd.5.smithi203.stdout:Dec 01 01:23:53 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-5[118084]: 2023-12-01T01:23:53.111+0000 7f500e4c3700 -1 osd.5 87 *** Got signal Terminated *** 2023-12-01T01:23:53.513 INFO:journalctl@ceph.osd.5.smithi203.stdout:Dec 01 01:23:53 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-5[118084]: 2023-12-01T01:23:53.111+0000 7f500e4c3700 -1 osd.5 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-01T01:23:53.779 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:23:53 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:23:53.457+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.648037+0000 front 2023-12-01T01:23:25.648097+0000 (oldest deadline 2023-12-01T01:23:49.748180+0000) 2023-12-01T01:23:53.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:23:53 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:23:53.529+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.288397+0000 front 2023-12-01T01:23:19.288373+0000 (oldest deadline 2023-12-01T01:23:43.388594+0000) 2023-12-01T01:23:53.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:23:53 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:23:53.529+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:23.389153+0000 front 2023-12-01T01:23:23.389160+0000 (oldest deadline 2023-12-01T01:23:48.689179+0000) 2023-12-01T01:23:54.012 INFO:journalctl@ceph.osd.5.smithi203.stdout:Dec 01 01:23:53 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-5[118084]: 2023-12-01T01:23:53.734+0000 7f5008ab5700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:17.231033+0000 front 2023-12-01T01:23:17.231063+0000 (oldest deadline 2023-12-01T01:23:41.931024+0000) 2023-12-01T01:23:54.012 INFO:journalctl@ceph.osd.5.smithi203.stdout:Dec 01 01:23:53 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-5[118084]: 2023-12-01T01:23:53.734+0000 7f5008ab5700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.832060+0000 front 2023-12-01T01:23:27.832188+0000 (oldest deadline 2023-12-01T01:23:50.131913+0000) 2023-12-01T01:23:54.278 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:23:53 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:23:53.777+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:23:54.279 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:23:53 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:23:53.777+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:23:54.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:23:53 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:23:53.969+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:23:54.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:23:53 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:23:53.969+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:24.870879+0000 front 2023-12-01T01:23:24.870799+0000 (oldest deadline 2023-12-01T01:23:53.071830+0000) 2023-12-01T01:23:54.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:23:54 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:23:54.297+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.051505+0000 front 2023-12-01T01:23:25.051469+0000 (oldest deadline 2023-12-01T01:23:50.852139+0000) 2023-12-01T01:23:54.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:23:54 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:23:54.225+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:23:54.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:23:54 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:23:54.225+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:23:54.738 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:23:54 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:23:54.478+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.648037+0000 front 2023-12-01T01:23:25.648097+0000 (oldest deadline 2023-12-01T01:23:49.748180+0000) 2023-12-01T01:23:54.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:23:54 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:23:54.544+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.288397+0000 front 2023-12-01T01:23:19.288373+0000 (oldest deadline 2023-12-01T01:23:43.388594+0000) 2023-12-01T01:23:54.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:23:54 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:23:54.544+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:23.389153+0000 front 2023-12-01T01:23:23.389160+0000 (oldest deadline 2023-12-01T01:23:48.689179+0000) 2023-12-01T01:23:55.028 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:23:54 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:23:54.737+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:23:55.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:23:54 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:23:54.737+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:23:55.183 INFO:journalctl@ceph.osd.5.smithi203.stdout:Dec 01 01:23:54 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-5[118084]: 2023-12-01T01:23:54.768+0000 7f5008ab5700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:17.231033+0000 front 2023-12-01T01:23:17.231063+0000 (oldest deadline 2023-12-01T01:23:41.931024+0000) 2023-12-01T01:23:55.183 INFO:journalctl@ceph.osd.5.smithi203.stdout:Dec 01 01:23:54 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-5[118084]: 2023-12-01T01:23:54.768+0000 7f5008ab5700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.832060+0000 front 2023-12-01T01:23:27.832188+0000 (oldest deadline 2023-12-01T01:23:50.131913+0000) 2023-12-01T01:23:55.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:23:55 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:23:55.009+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:23:55.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:23:55 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:23:55.009+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:24.870879+0000 front 2023-12-01T01:23:24.870799+0000 (oldest deadline 2023-12-01T01:23:53.071830+0000) 2023-12-01T01:23:55.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:23:55 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:23:55.269+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.051505+0000 front 2023-12-01T01:23:25.051469+0000 (oldest deadline 2023-12-01T01:23:50.852139+0000) 2023-12-01T01:23:55.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:23:55 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:23:55.182+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:23:55.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:23:55 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:23:55.182+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:23:55.730 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:23:55 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:23:55.478+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.648037+0000 front 2023-12-01T01:23:25.648097+0000 (oldest deadline 2023-12-01T01:23:49.748180+0000) 2023-12-01T01:23:55.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:23:55 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:23:55.584+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.288397+0000 front 2023-12-01T01:23:19.288373+0000 (oldest deadline 2023-12-01T01:23:43.388594+0000) 2023-12-01T01:23:55.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:23:55 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:23:55.584+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:23.389153+0000 front 2023-12-01T01:23:23.389160+0000 (oldest deadline 2023-12-01T01:23:48.689179+0000) 2023-12-01T01:23:56.012 INFO:journalctl@ceph.osd.5.smithi203.stdout:Dec 01 01:23:55 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-5[118084]: 2023-12-01T01:23:55.722+0000 7f5008ab5700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:17.231033+0000 front 2023-12-01T01:23:17.231063+0000 (oldest deadline 2023-12-01T01:23:41.931024+0000) 2023-12-01T01:23:56.012 INFO:journalctl@ceph.osd.5.smithi203.stdout:Dec 01 01:23:55 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-5[118084]: 2023-12-01T01:23:55.722+0000 7f5008ab5700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.832060+0000 front 2023-12-01T01:23:27.832188+0000 (oldest deadline 2023-12-01T01:23:50.131913+0000) 2023-12-01T01:23:56.028 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:23:55 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:23:55.729+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:23:56.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:23:55 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:23:55.729+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:23:56.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:23:55 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:23:55.995+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:23:56.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:23:55 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:23:55.995+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:24.870879+0000 front 2023-12-01T01:23:24.870799+0000 (oldest deadline 2023-12-01T01:23:53.071830+0000) 2023-12-01T01:23:56.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:23:56 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:23:56.258+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.051505+0000 front 2023-12-01T01:23:25.051469+0000 (oldest deadline 2023-12-01T01:23:50.852139+0000) 2023-12-01T01:23:56.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:23:56 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:23:56.175+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:23:56.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:23:56 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:23:56.175+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:23:56.690 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:23:56 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:23:56.430+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.648037+0000 front 2023-12-01T01:23:25.648097+0000 (oldest deadline 2023-12-01T01:23:49.748180+0000) 2023-12-01T01:23:56.690 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:23:56 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:23:56.430+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.049088+0000 front 2023-12-01T01:23:32.049212+0000 (oldest deadline 2023-12-01T01:23:56.149291+0000) 2023-12-01T01:23:56.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:23:56 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:23:56.570+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.288397+0000 front 2023-12-01T01:23:19.288373+0000 (oldest deadline 2023-12-01T01:23:43.388594+0000) 2023-12-01T01:23:56.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:23:56 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:23:56.570+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:23.389153+0000 front 2023-12-01T01:23:23.389160+0000 (oldest deadline 2023-12-01T01:23:48.689179+0000) 2023-12-01T01:23:56.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:23:56 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:23:56.570+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:33.191174+0000 front 2023-12-01T01:23:33.191244+0000 (oldest deadline 2023-12-01T01:23:56.091292+0000) 2023-12-01T01:23:57.012 INFO:journalctl@ceph.osd.5.smithi203.stdout:Dec 01 01:23:56 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-5[118084]: 2023-12-01T01:23:56.695+0000 7f5008ab5700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:17.231033+0000 front 2023-12-01T01:23:17.231063+0000 (oldest deadline 2023-12-01T01:23:41.931024+0000) 2023-12-01T01:23:57.012 INFO:journalctl@ceph.osd.5.smithi203.stdout:Dec 01 01:23:56 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-5[118084]: 2023-12-01T01:23:56.695+0000 7f5008ab5700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.832060+0000 front 2023-12-01T01:23:27.832188+0000 (oldest deadline 2023-12-01T01:23:50.131913+0000) 2023-12-01T01:23:57.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:23:56 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:23:56.688+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:23:57.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:23:56 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:23:56.689+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:23:57.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:23:57 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:23:57.005+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:23:57.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:23:57 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:23:57.005+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:24.870879+0000 front 2023-12-01T01:23:24.870799+0000 (oldest deadline 2023-12-01T01:23:53.071830+0000) 2023-12-01T01:23:57.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:23:57 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:23:57.005+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:34.173408+0000 front 2023-12-01T01:23:34.173306+0000 (oldest deadline 2023-12-01T01:23:56.473489+0000) 2023-12-01T01:23:57.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:23:57 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:23:57.298+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.051505+0000 front 2023-12-01T01:23:25.051469+0000 (oldest deadline 2023-12-01T01:23:50.852139+0000) 2023-12-01T01:23:57.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:23:57 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:23:57.298+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:31.953358+0000 front 2023-12-01T01:23:31.953386+0000 (oldest deadline 2023-12-01T01:23:56.653545+0000) 2023-12-01T01:23:57.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:23:57 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:23:57.140+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:23:57.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:23:57 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:23:57.140+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:23:57.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:23:57 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:23:57.140+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:23:57.779 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:23:57 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:23:57.446+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.648037+0000 front 2023-12-01T01:23:25.648097+0000 (oldest deadline 2023-12-01T01:23:49.748180+0000) 2023-12-01T01:23:57.779 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:23:57 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:23:57.446+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.049088+0000 front 2023-12-01T01:23:32.049212+0000 (oldest deadline 2023-12-01T01:23:56.149291+0000) 2023-12-01T01:23:57.779 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:23:57 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:23:57.681+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:23:57.779 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:23:57 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:23:57.681+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:23:57.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:23:57 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:23:57.553+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.288397+0000 front 2023-12-01T01:23:19.288373+0000 (oldest deadline 2023-12-01T01:23:43.388594+0000) 2023-12-01T01:23:57.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:23:57 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:23:57.553+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:23.389153+0000 front 2023-12-01T01:23:23.389160+0000 (oldest deadline 2023-12-01T01:23:48.689179+0000) 2023-12-01T01:23:57.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:23:57 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:23:57.553+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:33.191174+0000 front 2023-12-01T01:23:33.191244+0000 (oldest deadline 2023-12-01T01:23:56.091292+0000) 2023-12-01T01:23:58.012 INFO:journalctl@ceph.osd.5.smithi203.stdout:Dec 01 01:23:57 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-5[118084]: 2023-12-01T01:23:57.736+0000 7f5008ab5700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:17.231033+0000 front 2023-12-01T01:23:17.231063+0000 (oldest deadline 2023-12-01T01:23:41.931024+0000) 2023-12-01T01:23:58.012 INFO:journalctl@ceph.osd.5.smithi203.stdout:Dec 01 01:23:57 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-5[118084]: 2023-12-01T01:23:57.736+0000 7f5008ab5700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.832060+0000 front 2023-12-01T01:23:27.832188+0000 (oldest deadline 2023-12-01T01:23:50.131913+0000) 2023-12-01T01:23:58.012 INFO:journalctl@ceph.osd.5.smithi203.stdout:Dec 01 01:23:57 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-5[118084]: 2023-12-01T01:23:57.736+0000 7f5008ab5700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.432924+0000 front 2023-12-01T01:23:32.432957+0000 (oldest deadline 2023-12-01T01:23:57.733070+0000) 2023-12-01T01:23:58.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:23:57 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:23:57.991+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:23:58.405 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:23:57 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:23:57.991+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:24.870879+0000 front 2023-12-01T01:23:24.870799+0000 (oldest deadline 2023-12-01T01:23:53.071830+0000) 2023-12-01T01:23:58.405 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:23:57 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:23:57.991+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:34.173408+0000 front 2023-12-01T01:23:34.173306+0000 (oldest deadline 2023-12-01T01:23:56.473489+0000) 2023-12-01T01:23:58.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:23:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:23:58.317+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.051505+0000 front 2023-12-01T01:23:25.051469+0000 (oldest deadline 2023-12-01T01:23:50.852139+0000) 2023-12-01T01:23:58.513 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:23:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:23:58.317+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:31.953358+0000 front 2023-12-01T01:23:31.953386+0000 (oldest deadline 2023-12-01T01:23:56.653545+0000) 2023-12-01T01:23:58.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:23:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:23:58.094+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:23:58.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:23:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:23:58.094+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:23:58.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:23:58 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:23:58.094+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:23:58.779 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:23:58 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:23:58.474+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.648037+0000 front 2023-12-01T01:23:25.648097+0000 (oldest deadline 2023-12-01T01:23:49.748180+0000) 2023-12-01T01:23:58.779 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:23:58 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:23:58.474+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.049088+0000 front 2023-12-01T01:23:32.049212+0000 (oldest deadline 2023-12-01T01:23:56.149291+0000) 2023-12-01T01:23:58.779 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:23:58 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:23:58.680+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:23:58.779 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:23:58 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:23:58.680+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:23:58.781 INFO:journalctl@ceph.osd.5.smithi203.stdout:Dec 01 01:23:58 smithi203 bash[145780]: ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-5 2023-12-01T01:23:58.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:23:58 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:23:58.544+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.288397+0000 front 2023-12-01T01:23:19.288373+0000 (oldest deadline 2023-12-01T01:23:43.388594+0000) 2023-12-01T01:23:58.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:23:58 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:23:58.544+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:23.389153+0000 front 2023-12-01T01:23:23.389160+0000 (oldest deadline 2023-12-01T01:23:48.689179+0000) 2023-12-01T01:23:58.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:23:58 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:23:58.544+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:33.191174+0000 front 2023-12-01T01:23:33.191244+0000 (oldest deadline 2023-12-01T01:23:56.091292+0000) 2023-12-01T01:23:59.365 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:23:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:23:59.332+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.051505+0000 front 2023-12-01T01:23:25.051469+0000 (oldest deadline 2023-12-01T01:23:50.852139+0000) 2023-12-01T01:23:59.365 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:23:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:23:59.333+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:31.953358+0000 front 2023-12-01T01:23:31.953386+0000 (oldest deadline 2023-12-01T01:23:56.653545+0000) 2023-12-01T01:23:59.366 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:23:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:23:59.090+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:23:59.366 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:23:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:23:59.090+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:23:59.366 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:23:59 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:23:59.090+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:23:59.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:23:58 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:23:58.964+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:23:59.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:23:58 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:23:58.964+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:24.870879+0000 front 2023-12-01T01:23:24.870799+0000 (oldest deadline 2023-12-01T01:23:53.071830+0000) 2023-12-01T01:23:59.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:23:58 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:23:58.964+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:34.173408+0000 front 2023-12-01T01:23:34.173306+0000 (oldest deadline 2023-12-01T01:23:56.473489+0000) 2023-12-01T01:23:59.778 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:23:59 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:23:59.495+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.648037+0000 front 2023-12-01T01:23:25.648097+0000 (oldest deadline 2023-12-01T01:23:49.748180+0000) 2023-12-01T01:23:59.779 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:23:59 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:23:59.495+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.049088+0000 front 2023-12-01T01:23:32.049212+0000 (oldest deadline 2023-12-01T01:23:56.149291+0000) 2023-12-01T01:23:59.779 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:23:59 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:23:59.632+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:23:59.779 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:23:59 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:23:59.632+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:23:59.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:23:59 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:23:59.542+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.288397+0000 front 2023-12-01T01:23:19.288373+0000 (oldest deadline 2023-12-01T01:23:43.388594+0000) 2023-12-01T01:23:59.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:23:59 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:23:59.542+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:23.389153+0000 front 2023-12-01T01:23:23.389160+0000 (oldest deadline 2023-12-01T01:23:48.689179+0000) 2023-12-01T01:23:59.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:23:59 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:23:59.542+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:33.191174+0000 front 2023-12-01T01:23:33.191244+0000 (oldest deadline 2023-12-01T01:23:56.091292+0000) 2023-12-01T01:24:00.048 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@osd.5.service' 2023-12-01T01:24:00.088 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-01T01:24:00.088 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-12-01T01:24:00.088 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-12-01T01:24:00.088 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@osd.6 2023-12-01T01:24:00.301 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:00 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:00.040+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:00.301 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:00 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:00.040+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:00.301 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:00 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:00.040+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:00.404 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:24:00 smithi077 systemd[1]: Stopping Ceph osd.6 for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:24:00.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:23:59 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:23:59.938+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:24:00.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:23:59 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:23:59.938+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:24.870879+0000 front 2023-12-01T01:23:24.870799+0000 (oldest deadline 2023-12-01T01:23:53.071830+0000) 2023-12-01T01:24:00.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:23:59 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:23:59.938+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:34.173408+0000 front 2023-12-01T01:23:34.173306+0000 (oldest deadline 2023-12-01T01:23:56.473489+0000) 2023-12-01T01:24:00.762 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:00 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:00.300+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.051505+0000 front 2023-12-01T01:23:25.051469+0000 (oldest deadline 2023-12-01T01:23:50.852139+0000) 2023-12-01T01:24:00.762 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:00 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:00.300+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:31.953358+0000 front 2023-12-01T01:23:31.953386+0000 (oldest deadline 2023-12-01T01:23:56.653545+0000) 2023-12-01T01:24:00.779 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:24:00 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:24:00.525+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.648037+0000 front 2023-12-01T01:23:25.648097+0000 (oldest deadline 2023-12-01T01:23:49.748180+0000) 2023-12-01T01:24:00.779 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:24:00 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:24:00.525+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.049088+0000 front 2023-12-01T01:23:32.049212+0000 (oldest deadline 2023-12-01T01:23:56.149291+0000) 2023-12-01T01:24:00.779 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:00 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:00.662+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:24:00.779 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:00 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:00.662+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:24:00.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:24:00 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:24:00.464+0000 7efee3c40700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-01T01:24:00.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:24:00 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:24:00.464+0000 7efee3c40700 -1 osd.6 87 *** Got signal Terminated *** 2023-12-01T01:24:00.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:24:00 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:24:00.464+0000 7efee3c40700 -1 osd.6 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-01T01:24:00.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:24:00 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:24:00.587+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.288397+0000 front 2023-12-01T01:23:19.288373+0000 (oldest deadline 2023-12-01T01:23:43.388594+0000) 2023-12-01T01:24:00.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:24:00 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:24:00.587+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:23.389153+0000 front 2023-12-01T01:23:23.389160+0000 (oldest deadline 2023-12-01T01:23:48.689179+0000) 2023-12-01T01:24:00.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:24:00 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:24:00.587+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:33.191174+0000 front 2023-12-01T01:23:33.191244+0000 (oldest deadline 2023-12-01T01:23:56.091292+0000) 2023-12-01T01:24:01.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:00 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:00.945+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:24:01.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:00 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:00.945+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:24.870879+0000 front 2023-12-01T01:23:24.870799+0000 (oldest deadline 2023-12-01T01:23:53.071830+0000) 2023-12-01T01:24:01.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:00 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:00.945+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:34.173408+0000 front 2023-12-01T01:23:34.173306+0000 (oldest deadline 2023-12-01T01:23:56.473489+0000) 2023-12-01T01:24:01.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:01 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:01.273+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.051505+0000 front 2023-12-01T01:23:25.051469+0000 (oldest deadline 2023-12-01T01:23:50.852139+0000) 2023-12-01T01:24:01.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:01 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:01.273+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:31.953358+0000 front 2023-12-01T01:23:31.953386+0000 (oldest deadline 2023-12-01T01:23:56.653545+0000) 2023-12-01T01:24:01.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:01 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:01.049+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:01.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:01 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:01.049+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:01.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:01 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:01.049+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:01.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:24:01 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:24:01.544+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.288397+0000 front 2023-12-01T01:23:19.288373+0000 (oldest deadline 2023-12-01T01:23:43.388594+0000) 2023-12-01T01:24:01.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:24:01 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:24:01.544+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:23.389153+0000 front 2023-12-01T01:23:23.389160+0000 (oldest deadline 2023-12-01T01:23:48.689179+0000) 2023-12-01T01:24:01.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:24:01 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:24:01.544+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:33.191174+0000 front 2023-12-01T01:23:33.191244+0000 (oldest deadline 2023-12-01T01:23:56.091292+0000) 2023-12-01T01:24:02.029 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:24:01 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:24:01.555+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.648037+0000 front 2023-12-01T01:23:25.648097+0000 (oldest deadline 2023-12-01T01:23:49.748180+0000) 2023-12-01T01:24:02.029 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:24:01 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:24:01.555+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.049088+0000 front 2023-12-01T01:23:32.049212+0000 (oldest deadline 2023-12-01T01:23:56.149291+0000) 2023-12-01T01:24:02.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:01 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:01.624+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:24:02.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:01 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:01.624+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:24:02.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:01 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:01.624+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:35.218307+0000 front 2023-12-01T01:23:35.218482+0000 (oldest deadline 2023-12-01T01:24:01.118359+0000) 2023-12-01T01:24:02.405 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:01 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:01.982+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:24:02.405 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:01 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:01.982+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:24.870879+0000 front 2023-12-01T01:23:24.870799+0000 (oldest deadline 2023-12-01T01:23:53.071830+0000) 2023-12-01T01:24:02.405 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:01 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:01.982+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:34.173408+0000 front 2023-12-01T01:23:34.173306+0000 (oldest deadline 2023-12-01T01:23:56.473489+0000) 2023-12-01T01:24:02.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:02 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:02.274+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.051505+0000 front 2023-12-01T01:23:25.051469+0000 (oldest deadline 2023-12-01T01:23:50.852139+0000) 2023-12-01T01:24:02.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:02 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:02.274+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:31.953358+0000 front 2023-12-01T01:23:31.953386+0000 (oldest deadline 2023-12-01T01:23:56.653545+0000) 2023-12-01T01:24:02.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:02 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:02.026+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:02.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:02 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:02.026+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:02.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:02 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:02.026+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:02.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:24:02 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:24:02.502+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.288397+0000 front 2023-12-01T01:23:19.288373+0000 (oldest deadline 2023-12-01T01:23:43.388594+0000) 2023-12-01T01:24:02.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:24:02 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:24:02.502+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:23.389153+0000 front 2023-12-01T01:23:23.389160+0000 (oldest deadline 2023-12-01T01:23:48.689179+0000) 2023-12-01T01:24:02.905 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:24:02 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:24:02.502+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:33.191174+0000 front 2023-12-01T01:23:33.191244+0000 (oldest deadline 2023-12-01T01:23:56.091292+0000) 2023-12-01T01:24:03.029 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:24:02 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:24:02.532+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.648037+0000 front 2023-12-01T01:23:25.648097+0000 (oldest deadline 2023-12-01T01:23:49.748180+0000) 2023-12-01T01:24:03.029 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:24:02 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:24:02.532+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.049088+0000 front 2023-12-01T01:23:32.049212+0000 (oldest deadline 2023-12-01T01:23:56.149291+0000) 2023-12-01T01:24:03.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:02 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:02.579+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:24:03.030 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:02 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:02.579+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:24:03.030 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:02 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:02.579+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:35.218307+0000 front 2023-12-01T01:23:35.218482+0000 (oldest deadline 2023-12-01T01:24:01.118359+0000) 2023-12-01T01:24:03.262 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:03 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:03.236+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.051505+0000 front 2023-12-01T01:23:25.051469+0000 (oldest deadline 2023-12-01T01:23:50.852139+0000) 2023-12-01T01:24:03.262 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:03 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:03.236+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:31.953358+0000 front 2023-12-01T01:23:31.953386+0000 (oldest deadline 2023-12-01T01:23:56.653545+0000) 2023-12-01T01:24:03.263 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:02 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:02.986+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:03.263 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:02 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:02.986+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:03.263 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:02 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:02.986+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:03.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:03 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:03.018+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:24:03.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:03 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:03.018+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:24.870879+0000 front 2023-12-01T01:23:24.870799+0000 (oldest deadline 2023-12-01T01:23:53.071830+0000) 2023-12-01T01:24:03.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:03 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:03.018+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:34.173408+0000 front 2023-12-01T01:23:34.173306+0000 (oldest deadline 2023-12-01T01:23:56.473489+0000) 2023-12-01T01:24:03.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:24:03 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:24:03.547+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.288397+0000 front 2023-12-01T01:23:19.288373+0000 (oldest deadline 2023-12-01T01:23:43.388594+0000) 2023-12-01T01:24:03.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:24:03 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:24:03.547+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:23.389153+0000 front 2023-12-01T01:23:23.389160+0000 (oldest deadline 2023-12-01T01:23:48.689179+0000) 2023-12-01T01:24:03.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:24:03 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:24:03.547+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:33.191174+0000 front 2023-12-01T01:23:33.191244+0000 (oldest deadline 2023-12-01T01:23:56.091292+0000) 2023-12-01T01:24:04.029 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:24:03 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:24:03.539+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.648037+0000 front 2023-12-01T01:23:25.648097+0000 (oldest deadline 2023-12-01T01:23:49.748180+0000) 2023-12-01T01:24:04.029 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:24:03 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:24:03.539+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.049088+0000 front 2023-12-01T01:23:32.049212+0000 (oldest deadline 2023-12-01T01:23:56.149291+0000) 2023-12-01T01:24:04.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:03 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:03.555+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:24:04.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:03 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:03.555+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:24:04.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:03 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:03.555+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:35.218307+0000 front 2023-12-01T01:23:35.218482+0000 (oldest deadline 2023-12-01T01:24:01.118359+0000) 2023-12-01T01:24:04.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:04 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:04.046+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:24:04.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:04 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:04.046+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:24.870879+0000 front 2023-12-01T01:23:24.870799+0000 (oldest deadline 2023-12-01T01:23:53.071830+0000) 2023-12-01T01:24:04.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:04 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:04.046+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:34.173408+0000 front 2023-12-01T01:23:34.173306+0000 (oldest deadline 2023-12-01T01:23:56.473489+0000) 2023-12-01T01:24:04.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:04 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:04.205+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.051505+0000 front 2023-12-01T01:23:25.051469+0000 (oldest deadline 2023-12-01T01:23:50.852139+0000) 2023-12-01T01:24:04.513 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:04 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:04.205+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:31.953358+0000 front 2023-12-01T01:23:31.953386+0000 (oldest deadline 2023-12-01T01:23:56.653545+0000) 2023-12-01T01:24:04.513 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:04 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:04.205+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.354667+0000 front 2023-12-01T01:23:41.354653+0000 (oldest deadline 2023-12-01T01:24:03.654706+0000) 2023-12-01T01:24:04.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:04 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:04.025+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:04.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:04 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:04.025+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:04.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:04 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:04.025+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:04.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:24:04 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:24:04.597+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.288397+0000 front 2023-12-01T01:23:19.288373+0000 (oldest deadline 2023-12-01T01:23:43.388594+0000) 2023-12-01T01:24:04.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:24:04 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:24:04.597+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:23.389153+0000 front 2023-12-01T01:23:23.389160+0000 (oldest deadline 2023-12-01T01:23:48.689179+0000) 2023-12-01T01:24:04.904 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:24:04 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6[133534]: 2023-12-01T01:24:04.597+0000 7efedd230700 -1 osd.6 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:33.191174+0000 front 2023-12-01T01:23:33.191244+0000 (oldest deadline 2023-12-01T01:23:56.091292+0000) 2023-12-01T01:24:05.029 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:24:04 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:24:04.582+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.648037+0000 front 2023-12-01T01:23:25.648097+0000 (oldest deadline 2023-12-01T01:23:49.748180+0000) 2023-12-01T01:24:05.029 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:24:04 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:24:04.582+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.049088+0000 front 2023-12-01T01:23:32.049212+0000 (oldest deadline 2023-12-01T01:23:56.149291+0000) 2023-12-01T01:24:05.029 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:24:04 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:24:04.582+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.950793+0000 front 2023-12-01T01:23:41.950763+0000 (oldest deadline 2023-12-01T01:24:04.250870+0000) 2023-12-01T01:24:05.030 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:04 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:04.567+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:24:05.030 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:04 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:04.567+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:24:05.030 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:04 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:04.567+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:35.218307+0000 front 2023-12-01T01:23:35.218482+0000 (oldest deadline 2023-12-01T01:24:01.118359+0000) 2023-12-01T01:24:05.262 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:05 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:05.230+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.051505+0000 front 2023-12-01T01:23:25.051469+0000 (oldest deadline 2023-12-01T01:23:50.852139+0000) 2023-12-01T01:24:05.262 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:05 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:05.230+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:31.953358+0000 front 2023-12-01T01:23:31.953386+0000 (oldest deadline 2023-12-01T01:23:56.653545+0000) 2023-12-01T01:24:05.263 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:05 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:05.230+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.354667+0000 front 2023-12-01T01:23:41.354653+0000 (oldest deadline 2023-12-01T01:24:03.654706+0000) 2023-12-01T01:24:05.263 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:04 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:04.992+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:05.263 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:04 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:04.992+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:05.263 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:04 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:04.992+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:05.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:05 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:05.065+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:24:05.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:05 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:05.065+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:24.870879+0000 front 2023-12-01T01:23:24.870799+0000 (oldest deadline 2023-12-01T01:23:53.071830+0000) 2023-12-01T01:24:05.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:05 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:05.065+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:34.173408+0000 front 2023-12-01T01:23:34.173306+0000 (oldest deadline 2023-12-01T01:23:56.473489+0000) 2023-12-01T01:24:06.029 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:24:05 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:24:05.561+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.648037+0000 front 2023-12-01T01:23:25.648097+0000 (oldest deadline 2023-12-01T01:23:49.748180+0000) 2023-12-01T01:24:06.029 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:24:05 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:24:05.561+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.049088+0000 front 2023-12-01T01:23:32.049212+0000 (oldest deadline 2023-12-01T01:23:56.149291+0000) 2023-12-01T01:24:06.029 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:24:05 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:24:05.561+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.950793+0000 front 2023-12-01T01:23:41.950763+0000 (oldest deadline 2023-12-01T01:24:04.250870+0000) 2023-12-01T01:24:06.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:05 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:05.557+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:24:06.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:05 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:05.557+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:24:06.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:05 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:05.557+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:35.218307+0000 front 2023-12-01T01:23:35.218482+0000 (oldest deadline 2023-12-01T01:24:01.118359+0000) 2023-12-01T01:24:06.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:05 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:05.557+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.118898+0000 front 2023-12-01T01:23:41.118998+0000 (oldest deadline 2023-12-01T01:24:05.119565+0000) 2023-12-01T01:24:06.262 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:06 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:06.211+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.051505+0000 front 2023-12-01T01:23:25.051469+0000 (oldest deadline 2023-12-01T01:23:50.852139+0000) 2023-12-01T01:24:06.262 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:06 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:06.211+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:31.953358+0000 front 2023-12-01T01:23:31.953386+0000 (oldest deadline 2023-12-01T01:23:56.653545+0000) 2023-12-01T01:24:06.262 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:06 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:06.211+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.354667+0000 front 2023-12-01T01:23:41.354653+0000 (oldest deadline 2023-12-01T01:24:03.654706+0000) 2023-12-01T01:24:06.263 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:06 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:06.008+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:06.263 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:06 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:06.008+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:06.263 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:06 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:06.008+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:06.263 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:06 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:06.008+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:40.196046+0000 front 2023-12-01T01:23:40.196142+0000 (oldest deadline 2023-12-01T01:24:05.495965+0000) 2023-12-01T01:24:06.404 INFO:journalctl@ceph.osd.6.smithi077.stdout:Dec 01 01:24:06 smithi077 bash[187631]: ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-6 2023-12-01T01:24:06.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:06 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:06.044+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:24:06.405 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:06 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:06.044+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:24.870879+0000 front 2023-12-01T01:23:24.870799+0000 (oldest deadline 2023-12-01T01:23:53.071830+0000) 2023-12-01T01:24:06.405 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:06 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:06.044+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:34.173408+0000 front 2023-12-01T01:23:34.173306+0000 (oldest deadline 2023-12-01T01:23:56.473489+0000) 2023-12-01T01:24:07.029 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:24:06 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:24:06.585+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.648037+0000 front 2023-12-01T01:23:25.648097+0000 (oldest deadline 2023-12-01T01:23:49.748180+0000) 2023-12-01T01:24:07.029 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:24:06 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:24:06.585+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.049088+0000 front 2023-12-01T01:23:32.049212+0000 (oldest deadline 2023-12-01T01:23:56.149291+0000) 2023-12-01T01:24:07.029 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:24:06 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:24:06.585+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.950793+0000 front 2023-12-01T01:23:41.950763+0000 (oldest deadline 2023-12-01T01:24:04.250870+0000) 2023-12-01T01:24:07.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:06 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:06.582+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:24:07.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:06 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:06.582+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:24:07.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:06 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:06.582+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:35.218307+0000 front 2023-12-01T01:23:35.218482+0000 (oldest deadline 2023-12-01T01:24:01.118359+0000) 2023-12-01T01:24:07.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:06 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:06.582+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.118898+0000 front 2023-12-01T01:23:41.118998+0000 (oldest deadline 2023-12-01T01:24:05.119565+0000) 2023-12-01T01:24:07.134 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:07 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:07.003+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:24:07.134 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:07 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:07.003+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:24.870879+0000 front 2023-12-01T01:23:24.870799+0000 (oldest deadline 2023-12-01T01:23:53.071830+0000) 2023-12-01T01:24:07.134 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:07 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:07.003+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:34.173408+0000 front 2023-12-01T01:23:34.173306+0000 (oldest deadline 2023-12-01T01:23:56.473489+0000) 2023-12-01T01:24:07.503 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@osd.6.service' 2023-12-01T01:24:07.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:07 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:07.169+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.051505+0000 front 2023-12-01T01:23:25.051469+0000 (oldest deadline 2023-12-01T01:23:50.852139+0000) 2023-12-01T01:24:07.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:07 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:07.169+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:31.953358+0000 front 2023-12-01T01:23:31.953386+0000 (oldest deadline 2023-12-01T01:23:56.653545+0000) 2023-12-01T01:24:07.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:07 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:07.169+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.354667+0000 front 2023-12-01T01:23:41.354653+0000 (oldest deadline 2023-12-01T01:24:03.654706+0000) 2023-12-01T01:24:07.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:07 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:07.047+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:07.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:07 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:07.047+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:07.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:07 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:07.047+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:07.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:07 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:07.047+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:40.196046+0000 front 2023-12-01T01:23:40.196142+0000 (oldest deadline 2023-12-01T01:24:05.495965+0000) 2023-12-01T01:24:07.543 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-01T01:24:07.543 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-12-01T01:24:07.543 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-12-01T01:24:07.543 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@osd.7 2023-12-01T01:24:07.916 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:07 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:07.609+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:24:07.927 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:07 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:07.609+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:24:07.927 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:07 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:07.609+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:35.218307+0000 front 2023-12-01T01:23:35.218482+0000 (oldest deadline 2023-12-01T01:24:01.118359+0000) 2023-12-01T01:24:07.927 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:07 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:07.609+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.118898+0000 front 2023-12-01T01:23:41.118998+0000 (oldest deadline 2023-12-01T01:24:05.119565+0000) 2023-12-01T01:24:07.927 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:24:07 smithi137 systemd[1]: Stopping Ceph osd.7 for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:24:07.928 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:24:07 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:24:07.590+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.648037+0000 front 2023-12-01T01:23:25.648097+0000 (oldest deadline 2023-12-01T01:23:49.748180+0000) 2023-12-01T01:24:07.928 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:24:07 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:24:07.590+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.049088+0000 front 2023-12-01T01:23:32.049212+0000 (oldest deadline 2023-12-01T01:23:56.149291+0000) 2023-12-01T01:24:07.928 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:24:07 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:24:07.590+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.950793+0000 front 2023-12-01T01:23:41.950763+0000 (oldest deadline 2023-12-01T01:24:04.250870+0000) 2023-12-01T01:24:07.928 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:24:07 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:24:07.915+0000 7f49f3c38700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-01T01:24:08.279 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:24:07 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:24:07.915+0000 7f49f3c38700 -1 osd.7 87 *** Got signal Terminated *** 2023-12-01T01:24:08.279 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:24:07 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:24:07.915+0000 7f49f3c38700 -1 osd.7 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-01T01:24:08.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:08 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:08.050+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:24:08.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:08 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:08.050+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:24.870879+0000 front 2023-12-01T01:23:24.870799+0000 (oldest deadline 2023-12-01T01:23:53.071830+0000) 2023-12-01T01:24:08.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:08 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:08.050+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:34.173408+0000 front 2023-12-01T01:23:34.173306+0000 (oldest deadline 2023-12-01T01:23:56.473489+0000) 2023-12-01T01:24:08.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:08 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:08.180+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.051505+0000 front 2023-12-01T01:23:25.051469+0000 (oldest deadline 2023-12-01T01:23:50.852139+0000) 2023-12-01T01:24:08.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:08 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:08.180+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:31.953358+0000 front 2023-12-01T01:23:31.953386+0000 (oldest deadline 2023-12-01T01:23:56.653545+0000) 2023-12-01T01:24:08.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:08 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:08.180+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.354667+0000 front 2023-12-01T01:23:41.354653+0000 (oldest deadline 2023-12-01T01:24:03.654706+0000) 2023-12-01T01:24:08.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:08 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:08.068+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:08.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:08 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:08.068+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:08.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:08 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:08.068+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:08.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:08 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:08.068+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:40.196046+0000 front 2023-12-01T01:23:40.196142+0000 (oldest deadline 2023-12-01T01:24:05.495965+0000) 2023-12-01T01:24:09.029 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:24:08 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:24:08.618+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.648037+0000 front 2023-12-01T01:23:25.648097+0000 (oldest deadline 2023-12-01T01:23:49.748180+0000) 2023-12-01T01:24:09.029 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:24:08 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:24:08.618+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.049088+0000 front 2023-12-01T01:23:32.049212+0000 (oldest deadline 2023-12-01T01:23:56.149291+0000) 2023-12-01T01:24:09.029 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:24:08 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:24:08.618+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.950793+0000 front 2023-12-01T01:23:41.950763+0000 (oldest deadline 2023-12-01T01:24:04.250870+0000) 2023-12-01T01:24:09.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:08 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:08.636+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:24:09.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:08 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:08.636+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:24:09.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:08 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:08.636+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:35.218307+0000 front 2023-12-01T01:23:35.218482+0000 (oldest deadline 2023-12-01T01:24:01.118359+0000) 2023-12-01T01:24:09.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:08 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:08.636+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.118898+0000 front 2023-12-01T01:23:41.118998+0000 (oldest deadline 2023-12-01T01:24:05.119565+0000) 2023-12-01T01:24:09.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:09 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:09.022+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:24:09.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:09 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:09.022+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:24.870879+0000 front 2023-12-01T01:23:24.870799+0000 (oldest deadline 2023-12-01T01:23:53.071830+0000) 2023-12-01T01:24:09.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:09 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:09.022+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:34.173408+0000 front 2023-12-01T01:23:34.173306+0000 (oldest deadline 2023-12-01T01:23:56.473489+0000) 2023-12-01T01:24:09.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:09 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:09.022+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:42.875052+0000 front 2023-12-01T01:23:42.875201+0000 (oldest deadline 2023-12-01T01:24:08.175169+0000) 2023-12-01T01:24:09.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:09 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:09.141+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.051505+0000 front 2023-12-01T01:23:25.051469+0000 (oldest deadline 2023-12-01T01:23:50.852139+0000) 2023-12-01T01:24:09.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:09 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:09.141+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:31.953358+0000 front 2023-12-01T01:23:31.953386+0000 (oldest deadline 2023-12-01T01:23:56.653545+0000) 2023-12-01T01:24:09.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:09 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:09.141+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.354667+0000 front 2023-12-01T01:23:41.354653+0000 (oldest deadline 2023-12-01T01:24:03.654706+0000) 2023-12-01T01:24:09.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:09 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:09.101+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:09.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:09 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:09.101+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:09.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:09 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:09.101+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:09.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:09 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:09.101+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:40.196046+0000 front 2023-12-01T01:23:40.196142+0000 (oldest deadline 2023-12-01T01:24:05.495965+0000) 2023-12-01T01:24:10.029 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:24:09 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:24:09.587+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.648037+0000 front 2023-12-01T01:23:25.648097+0000 (oldest deadline 2023-12-01T01:23:49.748180+0000) 2023-12-01T01:24:10.029 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:24:09 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:24:09.587+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.049088+0000 front 2023-12-01T01:23:32.049212+0000 (oldest deadline 2023-12-01T01:23:56.149291+0000) 2023-12-01T01:24:10.029 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:24:09 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:24:09.587+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.950793+0000 front 2023-12-01T01:23:41.950763+0000 (oldest deadline 2023-12-01T01:24:04.250870+0000) 2023-12-01T01:24:10.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:09 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:09.628+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:24:10.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:09 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:09.628+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:24:10.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:09 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:09.628+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:35.218307+0000 front 2023-12-01T01:23:35.218482+0000 (oldest deadline 2023-12-01T01:24:01.118359+0000) 2023-12-01T01:24:10.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:09 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:09.628+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.118898+0000 front 2023-12-01T01:23:41.118998+0000 (oldest deadline 2023-12-01T01:24:05.119565+0000) 2023-12-01T01:24:10.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:10 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:10.023+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:24:10.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:10 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:10.023+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:24.870879+0000 front 2023-12-01T01:23:24.870799+0000 (oldest deadline 2023-12-01T01:23:53.071830+0000) 2023-12-01T01:24:10.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:10 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:10.023+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:34.173408+0000 front 2023-12-01T01:23:34.173306+0000 (oldest deadline 2023-12-01T01:23:56.473489+0000) 2023-12-01T01:24:10.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:10 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:10.023+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:42.875052+0000 front 2023-12-01T01:23:42.875201+0000 (oldest deadline 2023-12-01T01:24:08.175169+0000) 2023-12-01T01:24:10.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:10 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:10.179+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.051505+0000 front 2023-12-01T01:23:25.051469+0000 (oldest deadline 2023-12-01T01:23:50.852139+0000) 2023-12-01T01:24:10.513 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:10 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:10.179+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:31.953358+0000 front 2023-12-01T01:23:31.953386+0000 (oldest deadline 2023-12-01T01:23:56.653545+0000) 2023-12-01T01:24:10.513 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:10 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:10.179+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.354667+0000 front 2023-12-01T01:23:41.354653+0000 (oldest deadline 2023-12-01T01:24:03.654706+0000) 2023-12-01T01:24:10.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:10 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:10.066+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:10.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:10 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:10.066+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:10.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:10 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:10.066+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:10.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:10 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:10.066+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:40.196046+0000 front 2023-12-01T01:23:40.196142+0000 (oldest deadline 2023-12-01T01:24:05.495965+0000) 2023-12-01T01:24:11.029 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:24:10 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:24:10.579+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.648037+0000 front 2023-12-01T01:23:25.648097+0000 (oldest deadline 2023-12-01T01:23:49.748180+0000) 2023-12-01T01:24:11.029 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:24:10 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:24:10.579+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.049088+0000 front 2023-12-01T01:23:32.049212+0000 (oldest deadline 2023-12-01T01:23:56.149291+0000) 2023-12-01T01:24:11.029 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:24:10 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:24:10.579+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.950793+0000 front 2023-12-01T01:23:41.950763+0000 (oldest deadline 2023-12-01T01:24:04.250870+0000) 2023-12-01T01:24:11.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:10 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:10.622+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:24:11.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:10 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:10.622+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:24:11.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:10 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:10.622+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:35.218307+0000 front 2023-12-01T01:23:35.218482+0000 (oldest deadline 2023-12-01T01:24:01.118359+0000) 2023-12-01T01:24:11.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:10 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:10.622+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.118898+0000 front 2023-12-01T01:23:41.118998+0000 (oldest deadline 2023-12-01T01:24:05.119565+0000) 2023-12-01T01:24:11.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:11 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:11.007+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:24:11.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:11 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:11.007+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:24.870879+0000 front 2023-12-01T01:23:24.870799+0000 (oldest deadline 2023-12-01T01:23:53.071830+0000) 2023-12-01T01:24:11.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:11 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:11.007+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:34.173408+0000 front 2023-12-01T01:23:34.173306+0000 (oldest deadline 2023-12-01T01:23:56.473489+0000) 2023-12-01T01:24:11.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:11 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:11.007+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:42.875052+0000 front 2023-12-01T01:23:42.875201+0000 (oldest deadline 2023-12-01T01:24:08.175169+0000) 2023-12-01T01:24:11.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:11 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:11.184+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.051505+0000 front 2023-12-01T01:23:25.051469+0000 (oldest deadline 2023-12-01T01:23:50.852139+0000) 2023-12-01T01:24:11.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:11 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:11.184+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:31.953358+0000 front 2023-12-01T01:23:31.953386+0000 (oldest deadline 2023-12-01T01:23:56.653545+0000) 2023-12-01T01:24:11.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:11 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:11.184+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.354667+0000 front 2023-12-01T01:23:41.354653+0000 (oldest deadline 2023-12-01T01:24:03.654706+0000) 2023-12-01T01:24:11.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:11 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:11.072+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:11.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:11 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:11.072+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:11.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:11 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:11.072+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:11.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:11 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:11.072+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:40.196046+0000 front 2023-12-01T01:23:40.196142+0000 (oldest deadline 2023-12-01T01:24:05.495965+0000) 2023-12-01T01:24:12.029 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:24:11 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:24:11.623+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.648037+0000 front 2023-12-01T01:23:25.648097+0000 (oldest deadline 2023-12-01T01:23:49.748180+0000) 2023-12-01T01:24:12.029 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:24:11 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:24:11.623+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.049088+0000 front 2023-12-01T01:23:32.049212+0000 (oldest deadline 2023-12-01T01:23:56.149291+0000) 2023-12-01T01:24:12.029 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:24:11 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:24:11.623+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.950793+0000 front 2023-12-01T01:23:41.950763+0000 (oldest deadline 2023-12-01T01:24:04.250870+0000) 2023-12-01T01:24:12.029 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:24:11 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:24:11.623+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.352070+0000 front 2023-12-01T01:23:48.352123+0000 (oldest deadline 2023-12-01T01:24:11.252349+0000) 2023-12-01T01:24:12.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:11 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:11.657+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:24:12.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:11 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:11.657+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:24:12.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:11 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:11.657+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:35.218307+0000 front 2023-12-01T01:23:35.218482+0000 (oldest deadline 2023-12-01T01:24:01.118359+0000) 2023-12-01T01:24:12.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:11 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:11.657+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.118898+0000 front 2023-12-01T01:23:41.118998+0000 (oldest deadline 2023-12-01T01:24:05.119565+0000) 2023-12-01T01:24:12.030 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:11 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:11.657+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.020498+0000 front 2023-12-01T01:23:48.020518+0000 (oldest deadline 2023-12-01T01:24:11.520657+0000) 2023-12-01T01:24:12.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:12 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:12.031+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:24:12.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:12 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:12.031+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:24.870879+0000 front 2023-12-01T01:23:24.870799+0000 (oldest deadline 2023-12-01T01:23:53.071830+0000) 2023-12-01T01:24:12.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:12 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:12.031+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:34.173408+0000 front 2023-12-01T01:23:34.173306+0000 (oldest deadline 2023-12-01T01:23:56.473489+0000) 2023-12-01T01:24:12.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:12 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:12.031+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:42.875052+0000 front 2023-12-01T01:23:42.875201+0000 (oldest deadline 2023-12-01T01:24:08.175169+0000) 2023-12-01T01:24:12.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:12 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:12.204+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.051505+0000 front 2023-12-01T01:23:25.051469+0000 (oldest deadline 2023-12-01T01:23:50.852139+0000) 2023-12-01T01:24:12.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:12 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:12.204+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:31.953358+0000 front 2023-12-01T01:23:31.953386+0000 (oldest deadline 2023-12-01T01:23:56.653545+0000) 2023-12-01T01:24:12.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:12 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:12.204+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.354667+0000 front 2023-12-01T01:23:41.354653+0000 (oldest deadline 2023-12-01T01:24:03.654706+0000) 2023-12-01T01:24:12.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:12 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:12.090+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:12.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:12 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:12.090+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:12.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:12 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:12.090+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:12.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:12 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:12.090+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:40.196046+0000 front 2023-12-01T01:23:40.196142+0000 (oldest deadline 2023-12-01T01:24:05.495965+0000) 2023-12-01T01:24:13.029 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:24:12 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:24:12.665+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.648037+0000 front 2023-12-01T01:23:25.648097+0000 (oldest deadline 2023-12-01T01:23:49.748180+0000) 2023-12-01T01:24:13.029 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:24:12 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:24:12.665+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.049088+0000 front 2023-12-01T01:23:32.049212+0000 (oldest deadline 2023-12-01T01:23:56.149291+0000) 2023-12-01T01:24:13.029 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:24:12 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:24:12.665+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.950793+0000 front 2023-12-01T01:23:41.950763+0000 (oldest deadline 2023-12-01T01:24:04.250870+0000) 2023-12-01T01:24:13.029 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:24:12 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7[125541]: 2023-12-01T01:24:12.665+0000 7f49eea2b700 -1 osd.7 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.352070+0000 front 2023-12-01T01:23:48.352123+0000 (oldest deadline 2023-12-01T01:24:11.252349+0000) 2023-12-01T01:24:13.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:12 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:12.625+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:24:13.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:12 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:12.625+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:24:13.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:12 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:12.625+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:35.218307+0000 front 2023-12-01T01:23:35.218482+0000 (oldest deadline 2023-12-01T01:24:01.118359+0000) 2023-12-01T01:24:13.030 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:12 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:12.625+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.118898+0000 front 2023-12-01T01:23:41.118998+0000 (oldest deadline 2023-12-01T01:24:05.119565+0000) 2023-12-01T01:24:13.030 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:12 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:12.625+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.020498+0000 front 2023-12-01T01:23:48.020518+0000 (oldest deadline 2023-12-01T01:24:11.520657+0000) 2023-12-01T01:24:13.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:13 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:13.014+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:24:13.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:13 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:13.014+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:24.870879+0000 front 2023-12-01T01:23:24.870799+0000 (oldest deadline 2023-12-01T01:23:53.071830+0000) 2023-12-01T01:24:13.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:13 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:13.014+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:34.173408+0000 front 2023-12-01T01:23:34.173306+0000 (oldest deadline 2023-12-01T01:23:56.473489+0000) 2023-12-01T01:24:13.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:13 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:13.014+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:42.875052+0000 front 2023-12-01T01:23:42.875201+0000 (oldest deadline 2023-12-01T01:24:08.175169+0000) 2023-12-01T01:24:13.405 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:13 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:13.014+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:49.276219+0000 front 2023-12-01T01:23:49.276259+0000 (oldest deadline 2023-12-01T01:24:12.776201+0000) 2023-12-01T01:24:13.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:13 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:13.170+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.051505+0000 front 2023-12-01T01:23:25.051469+0000 (oldest deadline 2023-12-01T01:23:50.852139+0000) 2023-12-01T01:24:13.513 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:13 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:13.170+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:31.953358+0000 front 2023-12-01T01:23:31.953386+0000 (oldest deadline 2023-12-01T01:23:56.653545+0000) 2023-12-01T01:24:13.513 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:13 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:13.170+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.354667+0000 front 2023-12-01T01:23:41.354653+0000 (oldest deadline 2023-12-01T01:24:03.654706+0000) 2023-12-01T01:24:13.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:13 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:13.102+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:13.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:13 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:13.102+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:13.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:13 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:13.102+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:13.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:13 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:13.102+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:40.196046+0000 front 2023-12-01T01:23:40.196142+0000 (oldest deadline 2023-12-01T01:24:05.495965+0000) 2023-12-01T01:24:13.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:13 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:13.102+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.897485+0000 front 2023-12-01T01:23:48.897591+0000 (oldest deadline 2023-12-01T01:24:12.997468+0000) 2023-12-01T01:24:14.029 INFO:journalctl@ceph.osd.7.smithi137.stdout:Dec 01 01:24:13 smithi137 bash[145875]: ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-7 2023-12-01T01:24:14.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:13 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:13.614+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:24:14.030 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:13 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:13.614+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:24:14.030 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:13 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:13.614+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:35.218307+0000 front 2023-12-01T01:23:35.218482+0000 (oldest deadline 2023-12-01T01:24:01.118359+0000) 2023-12-01T01:24:14.030 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:13 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:13.614+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.118898+0000 front 2023-12-01T01:23:41.118998+0000 (oldest deadline 2023-12-01T01:24:05.119565+0000) 2023-12-01T01:24:14.030 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:13 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:13.614+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.020498+0000 front 2023-12-01T01:23:48.020518+0000 (oldest deadline 2023-12-01T01:24:11.520657+0000) 2023-12-01T01:24:14.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:14 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:14.034+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:24:14.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:14 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:14.034+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:24.870879+0000 front 2023-12-01T01:23:24.870799+0000 (oldest deadline 2023-12-01T01:23:53.071830+0000) 2023-12-01T01:24:14.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:14 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:14.034+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:34.173408+0000 front 2023-12-01T01:23:34.173306+0000 (oldest deadline 2023-12-01T01:23:56.473489+0000) 2023-12-01T01:24:14.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:14 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:14.034+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:42.875052+0000 front 2023-12-01T01:23:42.875201+0000 (oldest deadline 2023-12-01T01:24:08.175169+0000) 2023-12-01T01:24:14.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:14 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:14.034+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:49.276219+0000 front 2023-12-01T01:23:49.276259+0000 (oldest deadline 2023-12-01T01:24:12.776201+0000) 2023-12-01T01:24:14.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:14 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:14.150+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.051505+0000 front 2023-12-01T01:23:25.051469+0000 (oldest deadline 2023-12-01T01:23:50.852139+0000) 2023-12-01T01:24:14.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:14 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:14.150+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:31.953358+0000 front 2023-12-01T01:23:31.953386+0000 (oldest deadline 2023-12-01T01:23:56.653545+0000) 2023-12-01T01:24:14.513 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:14 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:14.150+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.354667+0000 front 2023-12-01T01:23:41.354653+0000 (oldest deadline 2023-12-01T01:24:03.654706+0000) 2023-12-01T01:24:14.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:14 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:14.121+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:14.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:14 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:14.121+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:14.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:14 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:14.121+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:14.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:14 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:14.121+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:40.196046+0000 front 2023-12-01T01:23:40.196142+0000 (oldest deadline 2023-12-01T01:24:05.495965+0000) 2023-12-01T01:24:14.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:14 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:14.121+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.897485+0000 front 2023-12-01T01:23:48.897591+0000 (oldest deadline 2023-12-01T01:24:12.997468+0000) 2023-12-01T01:24:14.938 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:14 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:14.648+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:24:14.938 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:14 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:14.648+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:24:14.938 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:14 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:14.648+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:35.218307+0000 front 2023-12-01T01:23:35.218482+0000 (oldest deadline 2023-12-01T01:24:01.118359+0000) 2023-12-01T01:24:14.938 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:14 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:14.648+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.118898+0000 front 2023-12-01T01:23:41.118998+0000 (oldest deadline 2023-12-01T01:24:05.119565+0000) 2023-12-01T01:24:14.938 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:14 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:14.648+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.020498+0000 front 2023-12-01T01:23:48.020518+0000 (oldest deadline 2023-12-01T01:24:11.520657+0000) 2023-12-01T01:24:14.946 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@osd.7.service' 2023-12-01T01:24:14.985 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-01T01:24:14.986 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-12-01T01:24:14.986 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-12-01T01:24:14.986 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@osd.8 2023-12-01T01:24:15.351 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:15 smithi203 systemd[1]: Stopping Ceph osd.8 for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:24:15.351 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:15 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:15.114+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.051505+0000 front 2023-12-01T01:23:25.051469+0000 (oldest deadline 2023-12-01T01:23:50.852139+0000) 2023-12-01T01:24:15.352 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:15 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:15.114+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:31.953358+0000 front 2023-12-01T01:23:31.953386+0000 (oldest deadline 2023-12-01T01:23:56.653545+0000) 2023-12-01T01:24:15.352 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:15 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:15.114+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.354667+0000 front 2023-12-01T01:23:41.354653+0000 (oldest deadline 2023-12-01T01:24:03.654706+0000) 2023-12-01T01:24:15.352 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:15 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:15.092+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:15.352 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:15 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:15.092+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:15.352 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:15 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:15.092+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:15.352 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:15 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:15.092+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:40.196046+0000 front 2023-12-01T01:23:40.196142+0000 (oldest deadline 2023-12-01T01:24:05.495965+0000) 2023-12-01T01:24:15.352 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:15 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:15.092+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.897485+0000 front 2023-12-01T01:23:48.897591+0000 (oldest deadline 2023-12-01T01:24:12.997468+0000) 2023-12-01T01:24:15.405 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:15 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:15.011+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:24:15.405 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:15 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:15.011+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:24.870879+0000 front 2023-12-01T01:23:24.870799+0000 (oldest deadline 2023-12-01T01:23:53.071830+0000) 2023-12-01T01:24:15.405 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:15 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:15.011+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:34.173408+0000 front 2023-12-01T01:23:34.173306+0000 (oldest deadline 2023-12-01T01:23:56.473489+0000) 2023-12-01T01:24:15.405 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:15 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:15.011+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:42.875052+0000 front 2023-12-01T01:23:42.875201+0000 (oldest deadline 2023-12-01T01:24:08.175169+0000) 2023-12-01T01:24:15.405 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:15 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:15.011+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:49.276219+0000 front 2023-12-01T01:23:49.276259+0000 (oldest deadline 2023-12-01T01:24:12.776201+0000) 2023-12-01T01:24:15.762 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:15 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:15.350+0000 7f49c2b8e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-01T01:24:15.762 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:15 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:15.350+0000 7f49c2b8e700 -1 osd.8 87 *** Got signal Terminated *** 2023-12-01T01:24:15.762 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:15 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:15.350+0000 7f49c2b8e700 -1 osd.8 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-01T01:24:16.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:15 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:15.674+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:24:16.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:15 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:15.674+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:24:16.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:15 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:15.674+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:35.218307+0000 front 2023-12-01T01:23:35.218482+0000 (oldest deadline 2023-12-01T01:24:01.118359+0000) 2023-12-01T01:24:16.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:15 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:15.674+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.118898+0000 front 2023-12-01T01:23:41.118998+0000 (oldest deadline 2023-12-01T01:24:05.119565+0000) 2023-12-01T01:24:16.030 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:15 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:15.674+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.020498+0000 front 2023-12-01T01:23:48.020518+0000 (oldest deadline 2023-12-01T01:24:11.520657+0000) 2023-12-01T01:24:16.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:16 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:16.000+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:24:16.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:16 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:16.000+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:24.870879+0000 front 2023-12-01T01:23:24.870799+0000 (oldest deadline 2023-12-01T01:23:53.071830+0000) 2023-12-01T01:24:16.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:16 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:16.000+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:34.173408+0000 front 2023-12-01T01:23:34.173306+0000 (oldest deadline 2023-12-01T01:23:56.473489+0000) 2023-12-01T01:24:16.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:16 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:16.000+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:42.875052+0000 front 2023-12-01T01:23:42.875201+0000 (oldest deadline 2023-12-01T01:24:08.175169+0000) 2023-12-01T01:24:16.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:16 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:16.000+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:49.276219+0000 front 2023-12-01T01:23:49.276259+0000 (oldest deadline 2023-12-01T01:24:12.776201+0000) 2023-12-01T01:24:16.513 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:16 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:16.135+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.051505+0000 front 2023-12-01T01:23:25.051469+0000 (oldest deadline 2023-12-01T01:23:50.852139+0000) 2023-12-01T01:24:16.513 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:16 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:16.135+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:31.953358+0000 front 2023-12-01T01:23:31.953386+0000 (oldest deadline 2023-12-01T01:23:56.653545+0000) 2023-12-01T01:24:16.513 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:16 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:16.135+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.354667+0000 front 2023-12-01T01:23:41.354653+0000 (oldest deadline 2023-12-01T01:24:03.654706+0000) 2023-12-01T01:24:16.514 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:16 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:16.135+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:50.056461+0000 front 2023-12-01T01:23:50.056507+0000 (oldest deadline 2023-12-01T01:24:15.956647+0000) 2023-12-01T01:24:16.514 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:16 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:16.121+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:16.514 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:16 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:16.121+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:16.514 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:16 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:16.121+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:16.514 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:16 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:16.121+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:40.196046+0000 front 2023-12-01T01:23:40.196142+0000 (oldest deadline 2023-12-01T01:24:05.495965+0000) 2023-12-01T01:24:16.514 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:16 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:16.121+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.897485+0000 front 2023-12-01T01:23:48.897591+0000 (oldest deadline 2023-12-01T01:24:12.997468+0000) 2023-12-01T01:24:17.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:16 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:16.698+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:24:17.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:16 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:16.698+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:24:17.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:16 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:16.698+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:35.218307+0000 front 2023-12-01T01:23:35.218482+0000 (oldest deadline 2023-12-01T01:24:01.118359+0000) 2023-12-01T01:24:17.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:16 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:16.698+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.118898+0000 front 2023-12-01T01:23:41.118998+0000 (oldest deadline 2023-12-01T01:24:05.119565+0000) 2023-12-01T01:24:17.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:16 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:16.698+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.020498+0000 front 2023-12-01T01:23:48.020518+0000 (oldest deadline 2023-12-01T01:24:11.520657+0000) 2023-12-01T01:24:17.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:17 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:17.020+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:24:17.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:17 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:17.020+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:24.870879+0000 front 2023-12-01T01:23:24.870799+0000 (oldest deadline 2023-12-01T01:23:53.071830+0000) 2023-12-01T01:24:17.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:17 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:17.020+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:34.173408+0000 front 2023-12-01T01:23:34.173306+0000 (oldest deadline 2023-12-01T01:23:56.473489+0000) 2023-12-01T01:24:17.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:17 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:17.020+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:42.875052+0000 front 2023-12-01T01:23:42.875201+0000 (oldest deadline 2023-12-01T01:24:08.175169+0000) 2023-12-01T01:24:17.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:17 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:17.020+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:49.276219+0000 front 2023-12-01T01:23:49.276259+0000 (oldest deadline 2023-12-01T01:24:12.776201+0000) 2023-12-01T01:24:17.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:17 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:17.176+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.051505+0000 front 2023-12-01T01:23:25.051469+0000 (oldest deadline 2023-12-01T01:23:50.852139+0000) 2023-12-01T01:24:17.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:17 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:17.176+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:31.953358+0000 front 2023-12-01T01:23:31.953386+0000 (oldest deadline 2023-12-01T01:23:56.653545+0000) 2023-12-01T01:24:17.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:17 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:17.176+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.354667+0000 front 2023-12-01T01:23:41.354653+0000 (oldest deadline 2023-12-01T01:24:03.654706+0000) 2023-12-01T01:24:17.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:17 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:17.176+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:50.056461+0000 front 2023-12-01T01:23:50.056507+0000 (oldest deadline 2023-12-01T01:24:15.956647+0000) 2023-12-01T01:24:17.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:17 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:17.136+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:17.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:17 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:17.136+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:17.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:17 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:17.136+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:17.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:17 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:17.136+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:40.196046+0000 front 2023-12-01T01:23:40.196142+0000 (oldest deadline 2023-12-01T01:24:05.495965+0000) 2023-12-01T01:24:17.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:17 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:17.136+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.897485+0000 front 2023-12-01T01:23:48.897591+0000 (oldest deadline 2023-12-01T01:24:12.997468+0000) 2023-12-01T01:24:18.028 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:17 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:17.742+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:24:18.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:17 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:17.742+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:24:18.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:17 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:17.742+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:35.218307+0000 front 2023-12-01T01:23:35.218482+0000 (oldest deadline 2023-12-01T01:24:01.118359+0000) 2023-12-01T01:24:18.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:17 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:17.742+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.118898+0000 front 2023-12-01T01:23:41.118998+0000 (oldest deadline 2023-12-01T01:24:05.119565+0000) 2023-12-01T01:24:18.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:17 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:17.742+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.020498+0000 front 2023-12-01T01:23:48.020518+0000 (oldest deadline 2023-12-01T01:24:11.520657+0000) 2023-12-01T01:24:18.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:18 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:18.063+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:24:18.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:18 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:18.063+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:24.870879+0000 front 2023-12-01T01:23:24.870799+0000 (oldest deadline 2023-12-01T01:23:53.071830+0000) 2023-12-01T01:24:18.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:18 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:18.063+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:34.173408+0000 front 2023-12-01T01:23:34.173306+0000 (oldest deadline 2023-12-01T01:23:56.473489+0000) 2023-12-01T01:24:18.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:18 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:18.063+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:42.875052+0000 front 2023-12-01T01:23:42.875201+0000 (oldest deadline 2023-12-01T01:24:08.175169+0000) 2023-12-01T01:24:18.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:18 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:18.063+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:49.276219+0000 front 2023-12-01T01:23:49.276259+0000 (oldest deadline 2023-12-01T01:24:12.776201+0000) 2023-12-01T01:24:18.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:18 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:18.221+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.051505+0000 front 2023-12-01T01:23:25.051469+0000 (oldest deadline 2023-12-01T01:23:50.852139+0000) 2023-12-01T01:24:18.513 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:18 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:18.221+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:31.953358+0000 front 2023-12-01T01:23:31.953386+0000 (oldest deadline 2023-12-01T01:23:56.653545+0000) 2023-12-01T01:24:18.513 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:18 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:18.221+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.354667+0000 front 2023-12-01T01:23:41.354653+0000 (oldest deadline 2023-12-01T01:24:03.654706+0000) 2023-12-01T01:24:18.513 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:18 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:18.221+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:50.056461+0000 front 2023-12-01T01:23:50.056507+0000 (oldest deadline 2023-12-01T01:24:15.956647+0000) 2023-12-01T01:24:18.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:18 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:18.106+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:18.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:18 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:18.106+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:18.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:18 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:18.106+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:18.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:18 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:18.106+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:40.196046+0000 front 2023-12-01T01:23:40.196142+0000 (oldest deadline 2023-12-01T01:24:05.495965+0000) 2023-12-01T01:24:18.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:18 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:18.106+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.897485+0000 front 2023-12-01T01:23:48.897591+0000 (oldest deadline 2023-12-01T01:24:12.997468+0000) 2023-12-01T01:24:19.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:18 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:18.702+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:24:19.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:18 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:18.702+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:24:19.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:18 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:18.702+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:35.218307+0000 front 2023-12-01T01:23:35.218482+0000 (oldest deadline 2023-12-01T01:24:01.118359+0000) 2023-12-01T01:24:19.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:18 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:18.702+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.118898+0000 front 2023-12-01T01:23:41.118998+0000 (oldest deadline 2023-12-01T01:24:05.119565+0000) 2023-12-01T01:24:19.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:18 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:18.702+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.020498+0000 front 2023-12-01T01:23:48.020518+0000 (oldest deadline 2023-12-01T01:24:11.520657+0000) 2023-12-01T01:24:19.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:19 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:19.038+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:24:19.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:19 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:19.038+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:24.870879+0000 front 2023-12-01T01:23:24.870799+0000 (oldest deadline 2023-12-01T01:23:53.071830+0000) 2023-12-01T01:24:19.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:19 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:19.038+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:34.173408+0000 front 2023-12-01T01:23:34.173306+0000 (oldest deadline 2023-12-01T01:23:56.473489+0000) 2023-12-01T01:24:19.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:19 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:19.038+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:42.875052+0000 front 2023-12-01T01:23:42.875201+0000 (oldest deadline 2023-12-01T01:24:08.175169+0000) 2023-12-01T01:24:19.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:19 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:19.038+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:49.276219+0000 front 2023-12-01T01:23:49.276259+0000 (oldest deadline 2023-12-01T01:24:12.776201+0000) 2023-12-01T01:24:19.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:19 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:19.191+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.051505+0000 front 2023-12-01T01:23:25.051469+0000 (oldest deadline 2023-12-01T01:23:50.852139+0000) 2023-12-01T01:24:19.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:19 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:19.191+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:31.953358+0000 front 2023-12-01T01:23:31.953386+0000 (oldest deadline 2023-12-01T01:23:56.653545+0000) 2023-12-01T01:24:19.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:19 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:19.191+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.354667+0000 front 2023-12-01T01:23:41.354653+0000 (oldest deadline 2023-12-01T01:24:03.654706+0000) 2023-12-01T01:24:19.513 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:19 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:19.191+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:50.056461+0000 front 2023-12-01T01:23:50.056507+0000 (oldest deadline 2023-12-01T01:24:15.956647+0000) 2023-12-01T01:24:19.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:19 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:19.137+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:19.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:19 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:19.137+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:19.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:19 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:19.137+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:19.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:19 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:19.137+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:40.196046+0000 front 2023-12-01T01:23:40.196142+0000 (oldest deadline 2023-12-01T01:24:05.495965+0000) 2023-12-01T01:24:19.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:19 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:19.137+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.897485+0000 front 2023-12-01T01:23:48.897591+0000 (oldest deadline 2023-12-01T01:24:12.997468+0000) 2023-12-01T01:24:20.028 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:19 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:19.699+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:24:20.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:19 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:19.699+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:24:20.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:19 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:19.699+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:35.218307+0000 front 2023-12-01T01:23:35.218482+0000 (oldest deadline 2023-12-01T01:24:01.118359+0000) 2023-12-01T01:24:20.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:19 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:19.699+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.118898+0000 front 2023-12-01T01:23:41.118998+0000 (oldest deadline 2023-12-01T01:24:05.119565+0000) 2023-12-01T01:24:20.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:19 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:19.699+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.020498+0000 front 2023-12-01T01:23:48.020518+0000 (oldest deadline 2023-12-01T01:24:11.520657+0000) 2023-12-01T01:24:20.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:20 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:20.036+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:24:20.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:20 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:20.036+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:24.870879+0000 front 2023-12-01T01:23:24.870799+0000 (oldest deadline 2023-12-01T01:23:53.071830+0000) 2023-12-01T01:24:20.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:20 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:20.036+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:34.173408+0000 front 2023-12-01T01:23:34.173306+0000 (oldest deadline 2023-12-01T01:23:56.473489+0000) 2023-12-01T01:24:20.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:20 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:20.036+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:42.875052+0000 front 2023-12-01T01:23:42.875201+0000 (oldest deadline 2023-12-01T01:24:08.175169+0000) 2023-12-01T01:24:20.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:20 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:20.036+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:49.276219+0000 front 2023-12-01T01:23:49.276259+0000 (oldest deadline 2023-12-01T01:24:12.776201+0000) 2023-12-01T01:24:20.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:20 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:20.199+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.051505+0000 front 2023-12-01T01:23:25.051469+0000 (oldest deadline 2023-12-01T01:23:50.852139+0000) 2023-12-01T01:24:20.512 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:20 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:20.199+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:31.953358+0000 front 2023-12-01T01:23:31.953386+0000 (oldest deadline 2023-12-01T01:23:56.653545+0000) 2023-12-01T01:24:20.513 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:20 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:20.199+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.354667+0000 front 2023-12-01T01:23:41.354653+0000 (oldest deadline 2023-12-01T01:24:03.654706+0000) 2023-12-01T01:24:20.513 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:20 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:20.199+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:50.056461+0000 front 2023-12-01T01:23:50.056507+0000 (oldest deadline 2023-12-01T01:24:15.956647+0000) 2023-12-01T01:24:20.513 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:20 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8[126144]: 2023-12-01T01:24:20.199+0000 7f49bd981700 -1 osd.8 87 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-12-01T01:23:55.957159+0000 front 2023-12-01T01:23:55.957100+0000 (oldest deadline 2023-12-01T01:24:19.457262+0000) 2023-12-01T01:24:20.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:20 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:20.182+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:20.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:20 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:20.182+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:20.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:20 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:20.182+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:20.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:20 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:20.182+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:40.196046+0000 front 2023-12-01T01:23:40.196142+0000 (oldest deadline 2023-12-01T01:24:05.495965+0000) 2023-12-01T01:24:20.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:20 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:20.182+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.897485+0000 front 2023-12-01T01:23:48.897591+0000 (oldest deadline 2023-12-01T01:24:12.997468+0000) 2023-12-01T01:24:21.028 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:20 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:20.703+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:24:21.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:20 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:20.703+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:24:21.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:20 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:20.703+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:35.218307+0000 front 2023-12-01T01:23:35.218482+0000 (oldest deadline 2023-12-01T01:24:01.118359+0000) 2023-12-01T01:24:21.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:20 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:20.703+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.118898+0000 front 2023-12-01T01:23:41.118998+0000 (oldest deadline 2023-12-01T01:24:05.119565+0000) 2023-12-01T01:24:21.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:20 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:20.703+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.020498+0000 front 2023-12-01T01:23:48.020518+0000 (oldest deadline 2023-12-01T01:24:11.520657+0000) 2023-12-01T01:24:21.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:20 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:20.703+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-12-01T01:23:53.822008+0000 front 2023-12-01T01:23:53.821905+0000 (oldest deadline 2023-12-01T01:24:19.721792+0000) 2023-12-01T01:24:21.220 INFO:journalctl@ceph.osd.8.smithi203.stdout:Dec 01 01:24:21 smithi203 bash[146044]: ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-8 2023-12-01T01:24:21.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:21 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:21.162+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:21.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:21 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:21.162+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:21.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:21 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:21.162+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:21.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:21 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:21.162+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:40.196046+0000 front 2023-12-01T01:23:40.196142+0000 (oldest deadline 2023-12-01T01:24:05.495965+0000) 2023-12-01T01:24:21.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:21 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:21.162+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.897485+0000 front 2023-12-01T01:23:48.897591+0000 (oldest deadline 2023-12-01T01:24:12.997468+0000) 2023-12-01T01:24:21.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:20 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:20.992+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:24:21.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:20 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:20.992+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:24.870879+0000 front 2023-12-01T01:23:24.870799+0000 (oldest deadline 2023-12-01T01:23:53.071830+0000) 2023-12-01T01:24:21.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:20 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:20.992+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:34.173408+0000 front 2023-12-01T01:23:34.173306+0000 (oldest deadline 2023-12-01T01:23:56.473489+0000) 2023-12-01T01:24:21.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:20 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:20.992+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:42.875052+0000 front 2023-12-01T01:23:42.875201+0000 (oldest deadline 2023-12-01T01:24:08.175169+0000) 2023-12-01T01:24:21.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:20 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:20.992+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:49.276219+0000 front 2023-12-01T01:23:49.276259+0000 (oldest deadline 2023-12-01T01:24:12.776201+0000) 2023-12-01T01:24:22.028 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:21 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:21.667+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:24:22.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:21 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:21.667+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:24:22.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:21 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:21.667+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:35.218307+0000 front 2023-12-01T01:23:35.218482+0000 (oldest deadline 2023-12-01T01:24:01.118359+0000) 2023-12-01T01:24:22.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:21 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:21.667+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.118898+0000 front 2023-12-01T01:23:41.118998+0000 (oldest deadline 2023-12-01T01:24:05.119565+0000) 2023-12-01T01:24:22.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:21 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:21.667+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.020498+0000 front 2023-12-01T01:23:48.020518+0000 (oldest deadline 2023-12-01T01:24:11.520657+0000) 2023-12-01T01:24:22.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:21 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:21.667+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-12-01T01:23:53.822008+0000 front 2023-12-01T01:23:53.821905+0000 (oldest deadline 2023-12-01T01:24:19.721792+0000) 2023-12-01T01:24:22.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:22 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:22.042+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:24:22.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:22 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:22.042+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:24.870879+0000 front 2023-12-01T01:23:24.870799+0000 (oldest deadline 2023-12-01T01:23:53.071830+0000) 2023-12-01T01:24:22.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:22 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:22.042+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:34.173408+0000 front 2023-12-01T01:23:34.173306+0000 (oldest deadline 2023-12-01T01:23:56.473489+0000) 2023-12-01T01:24:22.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:22 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:22.042+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:42.875052+0000 front 2023-12-01T01:23:42.875201+0000 (oldest deadline 2023-12-01T01:24:08.175169+0000) 2023-12-01T01:24:22.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:22 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:22.042+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:49.276219+0000 front 2023-12-01T01:23:49.276259+0000 (oldest deadline 2023-12-01T01:24:12.776201+0000) 2023-12-01T01:24:22.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:22 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:22.144+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:22.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:22 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:22.144+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:22.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:22 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:22.144+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:22.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:22 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:22.144+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:40.196046+0000 front 2023-12-01T01:23:40.196142+0000 (oldest deadline 2023-12-01T01:24:05.495965+0000) 2023-12-01T01:24:22.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:22 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:22.144+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.897485+0000 front 2023-12-01T01:23:48.897591+0000 (oldest deadline 2023-12-01T01:24:12.997468+0000) 2023-12-01T01:24:22.585 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@osd.8.service' 2023-12-01T01:24:22.624 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-01T01:24:22.624 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-12-01T01:24:22.625 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-12-01T01:24:22.625 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@osd.9 2023-12-01T01:24:22.987 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:22 smithi077 systemd[1]: Stopping Ceph osd.9 for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:24:23.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:22 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:22.715+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:24:23.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:22 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:22.715+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:24:23.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:22 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:22.715+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:35.218307+0000 front 2023-12-01T01:23:35.218482+0000 (oldest deadline 2023-12-01T01:24:01.118359+0000) 2023-12-01T01:24:23.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:22 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:22.715+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.118898+0000 front 2023-12-01T01:23:41.118998+0000 (oldest deadline 2023-12-01T01:24:05.119565+0000) 2023-12-01T01:24:23.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:22 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:22.715+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.020498+0000 front 2023-12-01T01:23:48.020518+0000 (oldest deadline 2023-12-01T01:24:11.520657+0000) 2023-12-01T01:24:23.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:22 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:22.715+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-12-01T01:23:53.822008+0000 front 2023-12-01T01:23:53.821905+0000 (oldest deadline 2023-12-01T01:24:19.721792+0000) 2023-12-01T01:24:23.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:22 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:22.986+0000 7fe6add69700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-01T01:24:23.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:22 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:22.986+0000 7fe6add69700 -1 osd.9 87 *** Got signal Terminated *** 2023-12-01T01:24:23.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:22 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:22.986+0000 7fe6add69700 -1 osd.9 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-01T01:24:23.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:23 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:23.055+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:24:23.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:23 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:23.055+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:24.870879+0000 front 2023-12-01T01:23:24.870799+0000 (oldest deadline 2023-12-01T01:23:53.071830+0000) 2023-12-01T01:24:23.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:23 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:23.055+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:34.173408+0000 front 2023-12-01T01:23:34.173306+0000 (oldest deadline 2023-12-01T01:23:56.473489+0000) 2023-12-01T01:24:23.405 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:23 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:23.055+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:42.875052+0000 front 2023-12-01T01:23:42.875201+0000 (oldest deadline 2023-12-01T01:24:08.175169+0000) 2023-12-01T01:24:23.405 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:23 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:23.055+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:49.276219+0000 front 2023-12-01T01:23:49.276259+0000 (oldest deadline 2023-12-01T01:24:12.776201+0000) 2023-12-01T01:24:23.405 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:23 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:23.055+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-12-01T01:23:56.877412+0000 front 2023-12-01T01:23:56.877445+0000 (oldest deadline 2023-12-01T01:24:22.177438+0000) 2023-12-01T01:24:23.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:23 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:23.170+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:23.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:23 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:23.170+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:23.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:23 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:23.170+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:23.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:23 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:23.170+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:40.196046+0000 front 2023-12-01T01:23:40.196142+0000 (oldest deadline 2023-12-01T01:24:05.495965+0000) 2023-12-01T01:24:23.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:23 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:23.170+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.897485+0000 front 2023-12-01T01:23:48.897591+0000 (oldest deadline 2023-12-01T01:24:12.997468+0000) 2023-12-01T01:24:23.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:23 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:23.170+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-12-01T01:23:57.098319+0000 front 2023-12-01T01:23:57.098373+0000 (oldest deadline 2023-12-01T01:24:22.998452+0000) 2023-12-01T01:24:24.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:23 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:23.718+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:24:24.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:23 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:23.718+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:24:24.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:23 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:23.718+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:35.218307+0000 front 2023-12-01T01:23:35.218482+0000 (oldest deadline 2023-12-01T01:24:01.118359+0000) 2023-12-01T01:24:24.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:23 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:23.718+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.118898+0000 front 2023-12-01T01:23:41.118998+0000 (oldest deadline 2023-12-01T01:24:05.119565+0000) 2023-12-01T01:24:24.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:23 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:23.718+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.020498+0000 front 2023-12-01T01:23:48.020518+0000 (oldest deadline 2023-12-01T01:24:11.520657+0000) 2023-12-01T01:24:24.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:23 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:23.718+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-12-01T01:23:53.822008+0000 front 2023-12-01T01:23:53.821905+0000 (oldest deadline 2023-12-01T01:24:19.721792+0000) 2023-12-01T01:24:24.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:24 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:24.076+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:24:24.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:24 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:24.076+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:24.870879+0000 front 2023-12-01T01:23:24.870799+0000 (oldest deadline 2023-12-01T01:23:53.071830+0000) 2023-12-01T01:24:24.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:24 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:24.076+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:34.173408+0000 front 2023-12-01T01:23:34.173306+0000 (oldest deadline 2023-12-01T01:23:56.473489+0000) 2023-12-01T01:24:24.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:24 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:24.076+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:42.875052+0000 front 2023-12-01T01:23:42.875201+0000 (oldest deadline 2023-12-01T01:24:08.175169+0000) 2023-12-01T01:24:24.405 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:24 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:24.076+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:49.276219+0000 front 2023-12-01T01:23:49.276259+0000 (oldest deadline 2023-12-01T01:24:12.776201+0000) 2023-12-01T01:24:24.405 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:24 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:24.076+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-12-01T01:23:56.877412+0000 front 2023-12-01T01:23:56.877445+0000 (oldest deadline 2023-12-01T01:24:22.177438+0000) 2023-12-01T01:24:24.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:24 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:24.192+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:24.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:24 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:24.192+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:24.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:24 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:24.192+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:24.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:24 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:24.192+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:40.196046+0000 front 2023-12-01T01:23:40.196142+0000 (oldest deadline 2023-12-01T01:24:05.495965+0000) 2023-12-01T01:24:24.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:24 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:24.192+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.897485+0000 front 2023-12-01T01:23:48.897591+0000 (oldest deadline 2023-12-01T01:24:12.997468+0000) 2023-12-01T01:24:24.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:24 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:24.192+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-12-01T01:23:57.098319+0000 front 2023-12-01T01:23:57.098373+0000 (oldest deadline 2023-12-01T01:24:22.998452+0000) 2023-12-01T01:24:25.028 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:24 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:24.715+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:24:25.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:24 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:24.715+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:24:25.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:24 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:24.715+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:35.218307+0000 front 2023-12-01T01:23:35.218482+0000 (oldest deadline 2023-12-01T01:24:01.118359+0000) 2023-12-01T01:24:25.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:24 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:24.715+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.118898+0000 front 2023-12-01T01:23:41.118998+0000 (oldest deadline 2023-12-01T01:24:05.119565+0000) 2023-12-01T01:24:25.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:24 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:24.715+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.020498+0000 front 2023-12-01T01:23:48.020518+0000 (oldest deadline 2023-12-01T01:24:11.520657+0000) 2023-12-01T01:24:25.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:24 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:24.715+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-12-01T01:23:53.822008+0000 front 2023-12-01T01:23:53.821905+0000 (oldest deadline 2023-12-01T01:24:19.721792+0000) 2023-12-01T01:24:25.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:25 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:25.040+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:24:25.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:25 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:25.040+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:24.870879+0000 front 2023-12-01T01:23:24.870799+0000 (oldest deadline 2023-12-01T01:23:53.071830+0000) 2023-12-01T01:24:25.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:25 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:25.040+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:34.173408+0000 front 2023-12-01T01:23:34.173306+0000 (oldest deadline 2023-12-01T01:23:56.473489+0000) 2023-12-01T01:24:25.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:25 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:25.040+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:42.875052+0000 front 2023-12-01T01:23:42.875201+0000 (oldest deadline 2023-12-01T01:24:08.175169+0000) 2023-12-01T01:24:25.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:25 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:25.040+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:49.276219+0000 front 2023-12-01T01:23:49.276259+0000 (oldest deadline 2023-12-01T01:24:12.776201+0000) 2023-12-01T01:24:25.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:25 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:25.040+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-12-01T01:23:56.877412+0000 front 2023-12-01T01:23:56.877445+0000 (oldest deadline 2023-12-01T01:24:22.177438+0000) 2023-12-01T01:24:25.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:25 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:25.216+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:25.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:25 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:25.216+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:25.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:25 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:25.216+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:25.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:25 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:25.216+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:40.196046+0000 front 2023-12-01T01:23:40.196142+0000 (oldest deadline 2023-12-01T01:24:05.495965+0000) 2023-12-01T01:24:25.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:25 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:25.216+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.897485+0000 front 2023-12-01T01:23:48.897591+0000 (oldest deadline 2023-12-01T01:24:12.997468+0000) 2023-12-01T01:24:25.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:25 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:25.216+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-12-01T01:23:57.098319+0000 front 2023-12-01T01:23:57.098373+0000 (oldest deadline 2023-12-01T01:24:22.998452+0000) 2023-12-01T01:24:26.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:25 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:25.734+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:24:26.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:25 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:25.734+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:24:26.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:25 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:25.734+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:35.218307+0000 front 2023-12-01T01:23:35.218482+0000 (oldest deadline 2023-12-01T01:24:01.118359+0000) 2023-12-01T01:24:26.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:25 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:25.734+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.118898+0000 front 2023-12-01T01:23:41.118998+0000 (oldest deadline 2023-12-01T01:24:05.119565+0000) 2023-12-01T01:24:26.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:25 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:25.734+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.020498+0000 front 2023-12-01T01:23:48.020518+0000 (oldest deadline 2023-12-01T01:24:11.520657+0000) 2023-12-01T01:24:26.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:25 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:25.734+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-12-01T01:23:53.822008+0000 front 2023-12-01T01:23:53.821905+0000 (oldest deadline 2023-12-01T01:24:19.721792+0000) 2023-12-01T01:24:26.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:26 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:26.013+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:24:26.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:26 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:26.013+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:24.870879+0000 front 2023-12-01T01:23:24.870799+0000 (oldest deadline 2023-12-01T01:23:53.071830+0000) 2023-12-01T01:24:26.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:26 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:26.013+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:34.173408+0000 front 2023-12-01T01:23:34.173306+0000 (oldest deadline 2023-12-01T01:23:56.473489+0000) 2023-12-01T01:24:26.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:26 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:26.013+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:42.875052+0000 front 2023-12-01T01:23:42.875201+0000 (oldest deadline 2023-12-01T01:24:08.175169+0000) 2023-12-01T01:24:26.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:26 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:26.013+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:49.276219+0000 front 2023-12-01T01:23:49.276259+0000 (oldest deadline 2023-12-01T01:24:12.776201+0000) 2023-12-01T01:24:26.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:26 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:26.013+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-12-01T01:23:56.877412+0000 front 2023-12-01T01:23:56.877445+0000 (oldest deadline 2023-12-01T01:24:22.177438+0000) 2023-12-01T01:24:26.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:26 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:26.216+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:26.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:26 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:26.216+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:26.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:26 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:26.216+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:26.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:26 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:26.216+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:40.196046+0000 front 2023-12-01T01:23:40.196142+0000 (oldest deadline 2023-12-01T01:24:05.495965+0000) 2023-12-01T01:24:26.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:26 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:26.216+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.897485+0000 front 2023-12-01T01:23:48.897591+0000 (oldest deadline 2023-12-01T01:24:12.997468+0000) 2023-12-01T01:24:26.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:26 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:26.216+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-12-01T01:23:57.098319+0000 front 2023-12-01T01:23:57.098373+0000 (oldest deadline 2023-12-01T01:24:22.998452+0000) 2023-12-01T01:24:27.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:26 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:26.724+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:24:27.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:26 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:26.724+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:24:27.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:26 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:26.724+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:35.218307+0000 front 2023-12-01T01:23:35.218482+0000 (oldest deadline 2023-12-01T01:24:01.118359+0000) 2023-12-01T01:24:27.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:26 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:26.724+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.118898+0000 front 2023-12-01T01:23:41.118998+0000 (oldest deadline 2023-12-01T01:24:05.119565+0000) 2023-12-01T01:24:27.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:26 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:26.724+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.020498+0000 front 2023-12-01T01:23:48.020518+0000 (oldest deadline 2023-12-01T01:24:11.520657+0000) 2023-12-01T01:24:27.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:26 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:26.724+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-12-01T01:23:53.822008+0000 front 2023-12-01T01:23:53.821905+0000 (oldest deadline 2023-12-01T01:24:19.721792+0000) 2023-12-01T01:24:27.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:26 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:26.985+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.170205+0000 front 2023-12-01T01:23:20.170238+0000 (oldest deadline 2023-12-01T01:23:44.870346+0000) 2023-12-01T01:24:27.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:26 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:26.985+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:24.870879+0000 front 2023-12-01T01:23:24.870799+0000 (oldest deadline 2023-12-01T01:23:53.071830+0000) 2023-12-01T01:24:27.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:26 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:26.985+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:34.173408+0000 front 2023-12-01T01:23:34.173306+0000 (oldest deadline 2023-12-01T01:23:56.473489+0000) 2023-12-01T01:24:27.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:26 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:26.985+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:42.875052+0000 front 2023-12-01T01:23:42.875201+0000 (oldest deadline 2023-12-01T01:24:08.175169+0000) 2023-12-01T01:24:27.404 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:26 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:26.985+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:49.276219+0000 front 2023-12-01T01:23:49.276259+0000 (oldest deadline 2023-12-01T01:24:12.776201+0000) 2023-12-01T01:24:27.405 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:26 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9[141694]: 2023-12-01T01:24:26.985+0000 7fe6a9b5e700 -1 osd.9 87 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-12-01T01:23:56.877412+0000 front 2023-12-01T01:23:56.877445+0000 (oldest deadline 2023-12-01T01:24:22.177438+0000) 2023-12-01T01:24:27.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:27 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:27.184+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:27.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:27 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:27.184+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:27.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:27 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:27.184+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:27.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:27 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:27.184+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:40.196046+0000 front 2023-12-01T01:23:40.196142+0000 (oldest deadline 2023-12-01T01:24:05.495965+0000) 2023-12-01T01:24:27.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:27 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:27.184+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.897485+0000 front 2023-12-01T01:23:48.897591+0000 (oldest deadline 2023-12-01T01:24:12.997468+0000) 2023-12-01T01:24:27.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:27 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:27.184+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-12-01T01:23:57.098319+0000 front 2023-12-01T01:23:57.098373+0000 (oldest deadline 2023-12-01T01:24:22.998452+0000) 2023-12-01T01:24:28.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:27 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:27.746+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:24:28.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:27 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:27.746+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:24:28.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:27 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:27.746+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:35.218307+0000 front 2023-12-01T01:23:35.218482+0000 (oldest deadline 2023-12-01T01:24:01.118359+0000) 2023-12-01T01:24:28.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:27 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:27.746+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.118898+0000 front 2023-12-01T01:23:41.118998+0000 (oldest deadline 2023-12-01T01:24:05.119565+0000) 2023-12-01T01:24:28.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:27 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:27.746+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.020498+0000 front 2023-12-01T01:23:48.020518+0000 (oldest deadline 2023-12-01T01:24:11.520657+0000) 2023-12-01T01:24:28.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:27 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:27.746+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-12-01T01:23:53.822008+0000 front 2023-12-01T01:23:53.821905+0000 (oldest deadline 2023-12-01T01:24:19.721792+0000) 2023-12-01T01:24:28.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:27 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:27.746+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2023-12-01T01:24:03.222974+0000 front 2023-12-01T01:24:03.222942+0000 (oldest deadline 2023-12-01T01:24:27.322887+0000) 2023-12-01T01:24:28.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:28 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:28.232+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:28.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:28 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:28.232+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:28.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:28 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:28.232+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:28.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:28 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:28.232+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:40.196046+0000 front 2023-12-01T01:23:40.196142+0000 (oldest deadline 2023-12-01T01:24:05.495965+0000) 2023-12-01T01:24:28.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:28 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:28.232+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.897485+0000 front 2023-12-01T01:23:48.897591+0000 (oldest deadline 2023-12-01T01:24:12.997468+0000) 2023-12-01T01:24:28.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:28 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:28.232+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-12-01T01:23:57.098319+0000 front 2023-12-01T01:23:57.098373+0000 (oldest deadline 2023-12-01T01:24:22.998452+0000) 2023-12-01T01:24:29.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:28 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:28.758+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:24:29.030 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:28 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:28.758+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:24:29.030 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:28 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:28.758+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:35.218307+0000 front 2023-12-01T01:23:35.218482+0000 (oldest deadline 2023-12-01T01:24:01.118359+0000) 2023-12-01T01:24:29.030 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:28 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:28.758+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.118898+0000 front 2023-12-01T01:23:41.118998+0000 (oldest deadline 2023-12-01T01:24:05.119565+0000) 2023-12-01T01:24:29.030 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:28 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:28.758+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.020498+0000 front 2023-12-01T01:23:48.020518+0000 (oldest deadline 2023-12-01T01:24:11.520657+0000) 2023-12-01T01:24:29.030 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:28 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:28.758+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-12-01T01:23:53.822008+0000 front 2023-12-01T01:23:53.821905+0000 (oldest deadline 2023-12-01T01:24:19.721792+0000) 2023-12-01T01:24:29.030 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:28 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:28.758+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2023-12-01T01:24:03.222974+0000 front 2023-12-01T01:24:03.222942+0000 (oldest deadline 2023-12-01T01:24:27.322887+0000) 2023-12-01T01:24:29.156 INFO:journalctl@ceph.osd.9.smithi077.stdout:Dec 01 01:24:28 smithi077 bash[187903]: ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-9 2023-12-01T01:24:29.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:29 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:29.222+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:29.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:29 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:29.222+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:29.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:29 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:29.222+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:29.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:29 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:29.222+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:40.196046+0000 front 2023-12-01T01:23:40.196142+0000 (oldest deadline 2023-12-01T01:24:05.495965+0000) 2023-12-01T01:24:29.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:29 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:29.222+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.897485+0000 front 2023-12-01T01:23:48.897591+0000 (oldest deadline 2023-12-01T01:24:12.997468+0000) 2023-12-01T01:24:29.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:29 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:29.222+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-12-01T01:23:57.098319+0000 front 2023-12-01T01:23:57.098373+0000 (oldest deadline 2023-12-01T01:24:22.998452+0000) 2023-12-01T01:24:30.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:29 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:29.767+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:24:30.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:29 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:29.767+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:24:30.030 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:29 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:29.767+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:35.218307+0000 front 2023-12-01T01:23:35.218482+0000 (oldest deadline 2023-12-01T01:24:01.118359+0000) 2023-12-01T01:24:30.030 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:29 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:29.767+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.118898+0000 front 2023-12-01T01:23:41.118998+0000 (oldest deadline 2023-12-01T01:24:05.119565+0000) 2023-12-01T01:24:30.030 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:29 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:29.767+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.020498+0000 front 2023-12-01T01:23:48.020518+0000 (oldest deadline 2023-12-01T01:24:11.520657+0000) 2023-12-01T01:24:30.030 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:29 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:29.767+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-12-01T01:23:53.822008+0000 front 2023-12-01T01:23:53.821905+0000 (oldest deadline 2023-12-01T01:24:19.721792+0000) 2023-12-01T01:24:30.030 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:29 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:29.767+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2023-12-01T01:24:03.222974+0000 front 2023-12-01T01:24:03.222942+0000 (oldest deadline 2023-12-01T01:24:27.322887+0000) 2023-12-01T01:24:30.102 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@osd.9.service' 2023-12-01T01:24:30.142 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-01T01:24:30.142 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-12-01T01:24:30.142 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-12-01T01:24:30.143 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@osd.10 2023-12-01T01:24:30.497 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:30 smithi137 systemd[1]: Stopping Ceph osd.10 for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:24:30.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:30 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:30.188+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:30.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:30 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:30.188+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:30.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:30 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:30.188+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:30.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:30 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:30.188+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:40.196046+0000 front 2023-12-01T01:23:40.196142+0000 (oldest deadline 2023-12-01T01:24:05.495965+0000) 2023-12-01T01:24:30.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:30 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:30.188+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.897485+0000 front 2023-12-01T01:23:48.897591+0000 (oldest deadline 2023-12-01T01:24:12.997468+0000) 2023-12-01T01:24:30.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:30 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:30.188+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-12-01T01:23:57.098319+0000 front 2023-12-01T01:23:57.098373+0000 (oldest deadline 2023-12-01T01:24:22.998452+0000) 2023-12-01T01:24:30.779 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:30 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:30.495+0000 7f09cec6f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-01T01:24:30.779 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:30 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:30.495+0000 7f09cec6f700 -1 osd.10 87 *** Got signal Terminated *** 2023-12-01T01:24:30.779 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:30 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:30.495+0000 7f09cec6f700 -1 osd.10 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-01T01:24:30.779 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:30 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:30.724+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:24:30.779 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:30 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:30.724+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:24:30.779 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:30 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:30.724+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:35.218307+0000 front 2023-12-01T01:23:35.218482+0000 (oldest deadline 2023-12-01T01:24:01.118359+0000) 2023-12-01T01:24:30.779 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:30 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:30.724+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.118898+0000 front 2023-12-01T01:23:41.118998+0000 (oldest deadline 2023-12-01T01:24:05.119565+0000) 2023-12-01T01:24:30.779 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:30 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:30.724+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.020498+0000 front 2023-12-01T01:23:48.020518+0000 (oldest deadline 2023-12-01T01:24:11.520657+0000) 2023-12-01T01:24:30.780 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:30 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:30.724+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-12-01T01:23:53.822008+0000 front 2023-12-01T01:23:53.821905+0000 (oldest deadline 2023-12-01T01:24:19.721792+0000) 2023-12-01T01:24:30.780 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:30 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:30.724+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2023-12-01T01:24:03.222974+0000 front 2023-12-01T01:24:03.222942+0000 (oldest deadline 2023-12-01T01:24:27.322887+0000) 2023-12-01T01:24:31.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:31 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:31.211+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:31.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:31 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:31.211+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:31.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:31 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:31.211+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:31.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:31 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:31.211+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:40.196046+0000 front 2023-12-01T01:23:40.196142+0000 (oldest deadline 2023-12-01T01:24:05.495965+0000) 2023-12-01T01:24:31.514 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:31 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:31.211+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.897485+0000 front 2023-12-01T01:23:48.897591+0000 (oldest deadline 2023-12-01T01:24:12.997468+0000) 2023-12-01T01:24:31.514 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:31 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:31.211+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-12-01T01:23:57.098319+0000 front 2023-12-01T01:23:57.098373+0000 (oldest deadline 2023-12-01T01:24:22.998452+0000) 2023-12-01T01:24:32.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:31 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:31.768+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:24:32.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:31 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:31.768+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:24:32.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:31 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:31.768+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:35.218307+0000 front 2023-12-01T01:23:35.218482+0000 (oldest deadline 2023-12-01T01:24:01.118359+0000) 2023-12-01T01:24:32.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:31 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:31.768+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.118898+0000 front 2023-12-01T01:23:41.118998+0000 (oldest deadline 2023-12-01T01:24:05.119565+0000) 2023-12-01T01:24:32.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:31 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:31.768+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.020498+0000 front 2023-12-01T01:23:48.020518+0000 (oldest deadline 2023-12-01T01:24:11.520657+0000) 2023-12-01T01:24:32.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:31 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:31.768+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-12-01T01:23:53.822008+0000 front 2023-12-01T01:23:53.821905+0000 (oldest deadline 2023-12-01T01:24:19.721792+0000) 2023-12-01T01:24:32.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:31 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:31.768+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2023-12-01T01:24:03.222974+0000 front 2023-12-01T01:24:03.222942+0000 (oldest deadline 2023-12-01T01:24:27.322887+0000) 2023-12-01T01:24:32.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:32 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:32.230+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:32.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:32 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:32.230+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:32.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:32 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:32.230+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:32.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:32 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:32.230+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:40.196046+0000 front 2023-12-01T01:23:40.196142+0000 (oldest deadline 2023-12-01T01:24:05.495965+0000) 2023-12-01T01:24:32.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:32 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:32.230+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.897485+0000 front 2023-12-01T01:23:48.897591+0000 (oldest deadline 2023-12-01T01:24:12.997468+0000) 2023-12-01T01:24:32.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:32 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:32.230+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-12-01T01:23:57.098319+0000 front 2023-12-01T01:23:57.098373+0000 (oldest deadline 2023-12-01T01:24:22.998452+0000) 2023-12-01T01:24:33.028 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:32 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:32.727+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:24:33.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:32 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:32.727+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:24:33.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:32 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:32.727+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:35.218307+0000 front 2023-12-01T01:23:35.218482+0000 (oldest deadline 2023-12-01T01:24:01.118359+0000) 2023-12-01T01:24:33.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:32 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:32.727+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.118898+0000 front 2023-12-01T01:23:41.118998+0000 (oldest deadline 2023-12-01T01:24:05.119565+0000) 2023-12-01T01:24:33.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:32 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:32.727+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.020498+0000 front 2023-12-01T01:23:48.020518+0000 (oldest deadline 2023-12-01T01:24:11.520657+0000) 2023-12-01T01:24:33.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:32 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:32.727+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-12-01T01:23:53.822008+0000 front 2023-12-01T01:23:53.821905+0000 (oldest deadline 2023-12-01T01:24:19.721792+0000) 2023-12-01T01:24:33.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:32 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:32.727+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2023-12-01T01:24:03.222974+0000 front 2023-12-01T01:24:03.222942+0000 (oldest deadline 2023-12-01T01:24:27.322887+0000) 2023-12-01T01:24:33.762 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:33 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:33.262+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:33.762 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:33 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:33.262+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:33.762 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:33 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:33.262+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:33.763 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:33 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:33.262+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:40.196046+0000 front 2023-12-01T01:23:40.196142+0000 (oldest deadline 2023-12-01T01:24:05.495965+0000) 2023-12-01T01:24:33.763 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:33 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:33.262+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.897485+0000 front 2023-12-01T01:23:48.897591+0000 (oldest deadline 2023-12-01T01:24:12.997468+0000) 2023-12-01T01:24:33.763 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:33 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:33.262+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-12-01T01:23:57.098319+0000 front 2023-12-01T01:23:57.098373+0000 (oldest deadline 2023-12-01T01:24:22.998452+0000) 2023-12-01T01:24:34.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:33 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:33.733+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:24:34.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:33 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:33.733+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:24:34.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:33 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:33.733+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:35.218307+0000 front 2023-12-01T01:23:35.218482+0000 (oldest deadline 2023-12-01T01:24:01.118359+0000) 2023-12-01T01:24:34.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:33 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:33.733+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.118898+0000 front 2023-12-01T01:23:41.118998+0000 (oldest deadline 2023-12-01T01:24:05.119565+0000) 2023-12-01T01:24:34.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:33 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:33.733+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.020498+0000 front 2023-12-01T01:23:48.020518+0000 (oldest deadline 2023-12-01T01:24:11.520657+0000) 2023-12-01T01:24:34.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:33 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:33.733+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-12-01T01:23:53.822008+0000 front 2023-12-01T01:23:53.821905+0000 (oldest deadline 2023-12-01T01:24:19.721792+0000) 2023-12-01T01:24:34.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:33 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:33.733+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2023-12-01T01:24:03.222974+0000 front 2023-12-01T01:24:03.222942+0000 (oldest deadline 2023-12-01T01:24:27.322887+0000) 2023-12-01T01:24:34.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:34 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:34.245+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:34.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:34 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:34.245+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:34.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:34 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:34.245+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:34.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:34 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:34.245+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:40.196046+0000 front 2023-12-01T01:23:40.196142+0000 (oldest deadline 2023-12-01T01:24:05.495965+0000) 2023-12-01T01:24:34.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:34 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:34.245+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.897485+0000 front 2023-12-01T01:23:48.897591+0000 (oldest deadline 2023-12-01T01:24:12.997468+0000) 2023-12-01T01:24:34.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:34 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:34.245+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-12-01T01:23:57.098319+0000 front 2023-12-01T01:23:57.098373+0000 (oldest deadline 2023-12-01T01:24:22.998452+0000) 2023-12-01T01:24:35.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:34 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:34.710+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:19.515503+0000 front 2023-12-01T01:23:19.515453+0000 (oldest deadline 2023-12-01T01:23:45.415587+0000) 2023-12-01T01:24:35.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:34 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:34.710+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:25.415993+0000 front 2023-12-01T01:23:25.416080+0000 (oldest deadline 2023-12-01T01:23:49.516295+0000) 2023-12-01T01:24:35.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:34 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:34.710+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:35.218307+0000 front 2023-12-01T01:23:35.218482+0000 (oldest deadline 2023-12-01T01:24:01.118359+0000) 2023-12-01T01:24:35.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:34 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:34.710+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:41.118898+0000 front 2023-12-01T01:23:41.118998+0000 (oldest deadline 2023-12-01T01:24:05.119565+0000) 2023-12-01T01:24:35.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:34 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:34.710+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.020498+0000 front 2023-12-01T01:23:48.020518+0000 (oldest deadline 2023-12-01T01:24:11.520657+0000) 2023-12-01T01:24:35.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:34 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:34.710+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-12-01T01:23:53.822008+0000 front 2023-12-01T01:23:53.821905+0000 (oldest deadline 2023-12-01T01:24:19.721792+0000) 2023-12-01T01:24:35.029 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:34 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10[134381]: 2023-12-01T01:24:34.710+0000 7f09cba66700 -1 osd.10 87 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2023-12-01T01:24:03.222974+0000 front 2023-12-01T01:24:03.222942+0000 (oldest deadline 2023-12-01T01:24:27.322887+0000) 2023-12-01T01:24:35.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:35 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:35.241+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:35.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:35 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:35.241+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:35.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:35 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:35.241+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:35.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:35 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:35.241+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:40.196046+0000 front 2023-12-01T01:23:40.196142+0000 (oldest deadline 2023-12-01T01:24:05.495965+0000) 2023-12-01T01:24:35.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:35 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:35.241+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.897485+0000 front 2023-12-01T01:23:48.897591+0000 (oldest deadline 2023-12-01T01:24:12.997468+0000) 2023-12-01T01:24:35.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:35 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:35.241+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-12-01T01:23:57.098319+0000 front 2023-12-01T01:23:57.098373+0000 (oldest deadline 2023-12-01T01:24:22.998452+0000) 2023-12-01T01:24:36.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:36 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:36.202+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:36.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:36 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:36.202+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:36.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:36 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:36.202+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:36.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:36 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:36.202+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:40.196046+0000 front 2023-12-01T01:23:40.196142+0000 (oldest deadline 2023-12-01T01:24:05.495965+0000) 2023-12-01T01:24:36.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:36 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:36.202+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.897485+0000 front 2023-12-01T01:23:48.897591+0000 (oldest deadline 2023-12-01T01:24:12.997468+0000) 2023-12-01T01:24:36.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:36 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:36.202+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-12-01T01:23:57.098319+0000 front 2023-12-01T01:23:57.098373+0000 (oldest deadline 2023-12-01T01:24:22.998452+0000) 2023-12-01T01:24:36.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:36 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:36.202+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2023-12-01T01:24:11.799946+0000 front 2023-12-01T01:24:11.799846+0000 (oldest deadline 2023-12-01T01:24:35.299826+0000) 2023-12-01T01:24:36.528 INFO:journalctl@ceph.osd.10.smithi137.stdout:Dec 01 01:24:36 smithi137 bash[146146]: ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-10 2023-12-01T01:24:37.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:37 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:37.197+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:37.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:37 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:37.197+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:37.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:37 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:37.197+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:37.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:37 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:37.197+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:40.196046+0000 front 2023-12-01T01:23:40.196142+0000 (oldest deadline 2023-12-01T01:24:05.495965+0000) 2023-12-01T01:24:37.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:37 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:37.197+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.897485+0000 front 2023-12-01T01:23:48.897591+0000 (oldest deadline 2023-12-01T01:24:12.997468+0000) 2023-12-01T01:24:37.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:37 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:37.197+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-12-01T01:23:57.098319+0000 front 2023-12-01T01:23:57.098373+0000 (oldest deadline 2023-12-01T01:24:22.998452+0000) 2023-12-01T01:24:37.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:37 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:37.197+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2023-12-01T01:24:11.799946+0000 front 2023-12-01T01:24:11.799846+0000 (oldest deadline 2023-12-01T01:24:35.299826+0000) 2023-12-01T01:24:37.706 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@osd.10.service' 2023-12-01T01:24:37.748 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-01T01:24:37.748 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-12-01T01:24:37.748 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-12-01T01:24:37.748 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@osd.11 2023-12-01T01:24:38.139 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:37 smithi203 systemd[1]: Stopping Ceph osd.11 for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:24:38.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:38 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:38.138+0000 7f9f2ca88700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-01T01:24:38.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:38 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:38.138+0000 7f9f2ca88700 -1 osd.11 87 *** Got signal Terminated *** 2023-12-01T01:24:38.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:38 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:38.138+0000 7f9f2ca88700 -1 osd.11 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-01T01:24:38.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:38 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:38.244+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:38.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:38 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:38.244+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:38.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:38 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:38.244+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:38.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:38 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:38.244+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:40.196046+0000 front 2023-12-01T01:23:40.196142+0000 (oldest deadline 2023-12-01T01:24:05.495965+0000) 2023-12-01T01:24:38.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:38 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:38.244+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.897485+0000 front 2023-12-01T01:23:48.897591+0000 (oldest deadline 2023-12-01T01:24:12.997468+0000) 2023-12-01T01:24:38.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:38 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:38.244+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-12-01T01:23:57.098319+0000 front 2023-12-01T01:23:57.098373+0000 (oldest deadline 2023-12-01T01:24:22.998452+0000) 2023-12-01T01:24:38.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:38 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:38.244+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2023-12-01T01:24:11.799946+0000 front 2023-12-01T01:24:11.799846+0000 (oldest deadline 2023-12-01T01:24:35.299826+0000) 2023-12-01T01:24:39.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:39 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:39.226+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:39.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:39 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:39.226+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:39.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:39 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:39.226+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:39.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:39 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:39.226+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:40.196046+0000 front 2023-12-01T01:23:40.196142+0000 (oldest deadline 2023-12-01T01:24:05.495965+0000) 2023-12-01T01:24:39.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:39 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:39.226+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.897485+0000 front 2023-12-01T01:23:48.897591+0000 (oldest deadline 2023-12-01T01:24:12.997468+0000) 2023-12-01T01:24:39.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:39 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:39.226+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-12-01T01:23:57.098319+0000 front 2023-12-01T01:23:57.098373+0000 (oldest deadline 2023-12-01T01:24:22.998452+0000) 2023-12-01T01:24:39.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:39 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:39.226+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2023-12-01T01:24:11.799946+0000 front 2023-12-01T01:24:11.799846+0000 (oldest deadline 2023-12-01T01:24:35.299826+0000) 2023-12-01T01:24:40.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:40 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:40.224+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:40.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:40 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:40.224+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:40.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:40 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:40.224+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:40.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:40 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:40.224+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:40.196046+0000 front 2023-12-01T01:23:40.196142+0000 (oldest deadline 2023-12-01T01:24:05.495965+0000) 2023-12-01T01:24:40.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:40 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:40.224+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.897485+0000 front 2023-12-01T01:23:48.897591+0000 (oldest deadline 2023-12-01T01:24:12.997468+0000) 2023-12-01T01:24:40.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:40 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:40.224+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-12-01T01:23:57.098319+0000 front 2023-12-01T01:23:57.098373+0000 (oldest deadline 2023-12-01T01:24:22.998452+0000) 2023-12-01T01:24:40.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:40 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:40.224+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2023-12-01T01:24:11.799946+0000 front 2023-12-01T01:24:11.799846+0000 (oldest deadline 2023-12-01T01:24:35.299826+0000) 2023-12-01T01:24:41.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:41 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:41.254+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:41.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:41 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:41.254+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:41.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:41 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:41.254+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:41.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:41 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:41.254+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:40.196046+0000 front 2023-12-01T01:23:40.196142+0000 (oldest deadline 2023-12-01T01:24:05.495965+0000) 2023-12-01T01:24:41.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:41 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:41.254+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.897485+0000 front 2023-12-01T01:23:48.897591+0000 (oldest deadline 2023-12-01T01:24:12.997468+0000) 2023-12-01T01:24:41.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:41 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:41.254+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-12-01T01:23:57.098319+0000 front 2023-12-01T01:23:57.098373+0000 (oldest deadline 2023-12-01T01:24:22.998452+0000) 2023-12-01T01:24:41.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:41 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:41.254+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2023-12-01T01:24:11.799946+0000 front 2023-12-01T01:24:11.799846+0000 (oldest deadline 2023-12-01T01:24:35.299826+0000) 2023-12-01T01:24:42.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:42 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:42.227+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2023-12-01T01:23:20.492092+0000 front 2023-12-01T01:23:20.491994+0000 (oldest deadline 2023-12-01T01:23:45.092450+0000) 2023-12-01T01:24:42.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:42 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:42.227+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2023-12-01T01:23:27.393675+0000 front 2023-12-01T01:23:27.393771+0000 (oldest deadline 2023-12-01T01:23:52.693810+0000) 2023-12-01T01:24:42.512 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:42 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:42.227+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2023-12-01T01:23:32.694199+0000 front 2023-12-01T01:23:32.694215+0000 (oldest deadline 2023-12-01T01:23:56.794311+0000) 2023-12-01T01:24:42.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:42 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:42.227+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2023-12-01T01:23:40.196046+0000 front 2023-12-01T01:23:40.196142+0000 (oldest deadline 2023-12-01T01:24:05.495965+0000) 2023-12-01T01:24:42.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:42 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:42.227+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2023-12-01T01:23:48.897485+0000 front 2023-12-01T01:23:48.897591+0000 (oldest deadline 2023-12-01T01:24:12.997468+0000) 2023-12-01T01:24:42.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:42 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:42.227+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.203:6812 osd.5 since back 2023-12-01T01:23:57.098319+0000 front 2023-12-01T01:23:57.098373+0000 (oldest deadline 2023-12-01T01:24:22.998452+0000) 2023-12-01T01:24:42.513 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:42 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11[134964]: 2023-12-01T01:24:42.227+0000 7f9f2987f700 -1 osd.11 87 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2023-12-01T01:24:11.799946+0000 front 2023-12-01T01:24:11.799846+0000 (oldest deadline 2023-12-01T01:24:35.299826+0000) 2023-12-01T01:24:44.012 INFO:journalctl@ceph.osd.11.smithi203.stdout:Dec 01 01:24:43 smithi203 bash[146315]: ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-osd-11 2023-12-01T01:24:45.253 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@osd.11.service' 2023-12-01T01:24:45.292 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-01T01:24:45.292 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-12-01T01:24:45.293 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-12-01T01:24:45.293 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mds.a 2023-12-01T01:24:45.654 INFO:journalctl@ceph.mds.a.smithi077.stdout:Dec 01 01:24:45 smithi077 systemd[1]: Stopping Ceph mds.a for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:24:46.404 INFO:journalctl@ceph.mds.a.smithi077.stdout:Dec 01 01:24:45 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-a[150304]: 2023-12-01T01:24:45.908+0000 7f9515a7c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-01T01:24:46.404 INFO:journalctl@ceph.mds.a.smithi077.stdout:Dec 01 01:24:45 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-a[150304]: 2023-12-01T01:24:45.908+0000 7f9515a7c700 -1 mds.a *** got signal Terminated *** 2023-12-01T01:24:50.404 INFO:journalctl@ceph.mds.a.smithi077.stdout:Dec 01 01:24:50 smithi077 bash[188180]: ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-a 2023-12-01T01:24:50.435 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mds.a.service' 2023-12-01T01:24:50.471 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-01T01:24:50.471 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-12-01T01:24:50.472 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-12-01T01:24:50.472 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mds.d 2023-12-01T01:24:50.851 INFO:journalctl@ceph.mds.d.smithi077.stdout:Dec 01 01:24:50 smithi077 systemd[1]: Stopping Ceph mds.d for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:24:51.154 INFO:journalctl@ceph.mds.d.smithi077.stdout:Dec 01 01:24:50 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-d[149309]: 2023-12-01T01:24:50.849+0000 7f113e9f3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-01T01:24:51.154 INFO:journalctl@ceph.mds.d.smithi077.stdout:Dec 01 01:24:50 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-d[149309]: 2023-12-01T01:24:50.849+0000 7f113e9f3700 -1 mds.d *** got signal Terminated *** 2023-12-01T01:24:54.654 INFO:journalctl@ceph.mds.d.smithi077.stdout:Dec 01 01:24:54 smithi077 bash[188326]: ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-d 2023-12-01T01:24:54.706 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mds.d.service' 2023-12-01T01:24:54.742 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-01T01:24:54.742 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-12-01T01:24:54.742 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-12-01T01:24:54.742 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mds.g 2023-12-01T01:24:55.124 INFO:journalctl@ceph.mds.g.smithi077.stdout:Dec 01 01:24:54 smithi077 systemd[1]: Stopping Ceph mds.g for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:24:55.404 INFO:journalctl@ceph.mds.g.smithi077.stdout:Dec 01 01:24:55 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-g[149918]: 2023-12-01T01:24:55.122+0000 7ff50ab9f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-01T01:24:55.404 INFO:journalctl@ceph.mds.g.smithi077.stdout:Dec 01 01:24:55 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-g[149918]: 2023-12-01T01:24:55.122+0000 7ff50ab9f700 -1 mds.g *** got signal Terminated *** 2023-12-01T01:24:58.548 INFO:journalctl@ceph.mds.g.smithi077.stdout:Dec 01 01:24:58 smithi077 bash[188465]: ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-g 2023-12-01T01:24:58.549 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mds.g.service' 2023-12-01T01:24:58.588 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-01T01:24:58.589 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-12-01T01:24:58.589 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-12-01T01:24:58.589 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mds.j 2023-12-01T01:24:58.904 INFO:journalctl@ceph.mds.j.smithi077.stdout:Dec 01 01:24:58 smithi077 systemd[1]: Stopping Ceph mds.j for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:24:59.404 INFO:journalctl@ceph.mds.j.smithi077.stdout:Dec 01 01:24:58 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-j[148946]: 2023-12-01T01:24:58.963+0000 7fbb30851700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-01T01:24:59.406 INFO:journalctl@ceph.mds.j.smithi077.stdout:Dec 01 01:24:58 smithi077 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-j[148946]: 2023-12-01T01:24:58.964+0000 7fbb30851700 -1 mds.j *** got signal Terminated *** 2023-12-01T01:25:00.874 INFO:journalctl@ceph.mds.j.smithi077.stdout:Dec 01 01:25:00 smithi077 bash[188612]: ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-j 2023-12-01T01:25:00.879 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mds.j.service' 2023-12-01T01:25:00.911 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-01T01:25:00.911 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-12-01T01:25:00.912 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-12-01T01:25:00.912 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mds.b 2023-12-01T01:25:01.267 INFO:journalctl@ceph.mds.b.smithi137.stdout:Dec 01 01:25:00 smithi137 systemd[1]: Stopping Ceph mds.b for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:25:01.529 INFO:journalctl@ceph.mds.b.smithi137.stdout:Dec 01 01:25:01 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-b[141375]: 2023-12-01T01:25:01.265+0000 7f3e8de0f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-01T01:25:01.529 INFO:journalctl@ceph.mds.b.smithi137.stdout:Dec 01 01:25:01 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-b[141375]: 2023-12-01T01:25:01.265+0000 7f3e8de0f700 -1 mds.b *** got signal Terminated *** 2023-12-01T01:25:05.774 INFO:journalctl@ceph.mds.b.smithi137.stdout:Dec 01 01:25:05 smithi137 bash[146416]: ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-b 2023-12-01T01:25:05.780 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mds.b.service' 2023-12-01T01:25:05.817 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-01T01:25:05.818 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-12-01T01:25:05.818 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-12-01T01:25:05.818 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mds.e 2023-12-01T01:25:06.029 INFO:journalctl@ceph.mds.e.smithi137.stdout:Dec 01 01:25:05 smithi137 systemd[1]: Stopping Ceph mds.e for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:25:06.529 INFO:journalctl@ceph.mds.e.smithi137.stdout:Dec 01 01:25:06 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-e[140682]: 2023-12-01T01:25:06.221+0000 7fa0e54f3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-01T01:25:06.529 INFO:journalctl@ceph.mds.e.smithi137.stdout:Dec 01 01:25:06 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-e[140682]: 2023-12-01T01:25:06.221+0000 7fa0e54f3700 -1 mds.e *** got signal Terminated *** 2023-12-01T01:25:10.529 INFO:journalctl@ceph.mds.e.smithi137.stdout:Dec 01 01:25:10 smithi137 bash[146571]: ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-e 2023-12-01T01:25:10.633 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mds.e.service' 2023-12-01T01:25:10.667 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-01T01:25:10.667 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-12-01T01:25:10.668 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-12-01T01:25:10.668 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mds.h 2023-12-01T01:25:11.029 INFO:journalctl@ceph.mds.h.smithi137.stdout:Dec 01 01:25:10 smithi137 systemd[1]: Stopping Ceph mds.h for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:25:11.529 INFO:journalctl@ceph.mds.h.smithi137.stdout:Dec 01 01:25:11 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-h[141032]: 2023-12-01T01:25:11.046+0000 7fdd4d47d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-01T01:25:11.529 INFO:journalctl@ceph.mds.h.smithi137.stdout:Dec 01 01:25:11 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-h[141032]: 2023-12-01T01:25:11.046+0000 7fdd4d47d700 -1 mds.h *** got signal Terminated *** 2023-12-01T01:25:14.528 INFO:journalctl@ceph.mds.h.smithi137.stdout:Dec 01 01:25:14 smithi137 bash[146717]: ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-h 2023-12-01T01:25:14.586 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mds.h.service' 2023-12-01T01:25:14.624 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-01T01:25:14.624 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-12-01T01:25:14.624 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-12-01T01:25:14.624 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mds.k 2023-12-01T01:25:15.006 INFO:journalctl@ceph.mds.k.smithi137.stdout:Dec 01 01:25:14 smithi137 systemd[1]: Stopping Ceph mds.k for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:25:15.279 INFO:journalctl@ceph.mds.k.smithi137.stdout:Dec 01 01:25:15 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-k[140344]: 2023-12-01T01:25:15.004+0000 7fb24b561700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-01T01:25:15.280 INFO:journalctl@ceph.mds.k.smithi137.stdout:Dec 01 01:25:15 smithi137 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-k[140344]: 2023-12-01T01:25:15.004+0000 7fb24b561700 -1 mds.k *** got signal Terminated *** 2023-12-01T01:25:18.372 INFO:journalctl@ceph.mds.k.smithi137.stdout:Dec 01 01:25:18 smithi137 bash[146873]: ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-k 2023-12-01T01:25:18.566 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mds.k.service' 2023-12-01T01:25:18.600 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-01T01:25:18.600 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-12-01T01:25:18.600 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-12-01T01:25:18.600 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mds.c 2023-12-01T01:25:18.999 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:25:18 smithi203 systemd[1]: Stopping Ceph mds.c for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:25:19.262 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:25:18 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: 2023-12-01T01:25:18.998+0000 7fa48cac4700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-01T01:25:19.262 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:25:18 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c[141049]: 2023-12-01T01:25:18.998+0000 7fa48cac4700 -1 mds.c *** got signal Terminated *** 2023-12-01T01:25:22.510 INFO:journalctl@ceph.mds.c.smithi203.stdout:Dec 01 01:25:22 smithi203 bash[146583]: ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-c 2023-12-01T01:25:22.517 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mds.c.service' 2023-12-01T01:25:22.552 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-01T01:25:22.552 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-12-01T01:25:22.552 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-12-01T01:25:22.553 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mds.f 2023-12-01T01:25:22.932 INFO:journalctl@ceph.mds.f.smithi203.stdout:Dec 01 01:25:22 smithi203 systemd[1]: Stopping Ceph mds.f for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:25:23.262 INFO:journalctl@ceph.mds.f.smithi203.stdout:Dec 01 01:25:22 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-f[140711]: 2023-12-01T01:25:22.931+0000 7f9143ccf700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-01T01:25:23.262 INFO:journalctl@ceph.mds.f.smithi203.stdout:Dec 01 01:25:22 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-f[140711]: 2023-12-01T01:25:22.931+0000 7f9143ccf700 -1 mds.f *** got signal Terminated *** 2023-12-01T01:25:26.512 INFO:journalctl@ceph.mds.f.smithi203.stdout:Dec 01 01:25:26 smithi203 bash[146723]: ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-f 2023-12-01T01:25:26.591 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mds.f.service' 2023-12-01T01:25:26.625 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-01T01:25:26.625 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-12-01T01:25:26.625 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-12-01T01:25:26.626 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mds.i 2023-12-01T01:25:27.006 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:25:26 smithi203 systemd[1]: Stopping Ceph mds.i for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:25:27.262 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:25:27 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:25:27.004+0000 7f854a1fb700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-01T01:25:27.262 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:25:27 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i[141386]: 2023-12-01T01:25:27.004+0000 7f854a1fb700 -1 mds.i *** got signal Terminated *** 2023-12-01T01:25:30.370 INFO:journalctl@ceph.mds.i.smithi203.stdout:Dec 01 01:25:30 smithi203 bash[146873]: ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-i 2023-12-01T01:25:30.557 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mds.i.service' 2023-12-01T01:25:30.591 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-01T01:25:30.591 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-12-01T01:25:30.591 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-12-01T01:25:30.591 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mds.l 2023-12-01T01:25:30.972 INFO:journalctl@ceph.mds.l.smithi203.stdout:Dec 01 01:25:30 smithi203 systemd[1]: Stopping Ceph mds.l for 2a06e29a-8fe6-11ee-95a2-87774f69a715... 2023-12-01T01:25:31.262 INFO:journalctl@ceph.mds.l.smithi203.stdout:Dec 01 01:25:30 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-l[140370]: 2023-12-01T01:25:30.971+0000 7ff2f09d2700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-01T01:25:31.262 INFO:journalctl@ceph.mds.l.smithi203.stdout:Dec 01 01:25:30 smithi203 ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-l[140370]: 2023-12-01T01:25:30.971+0000 7ff2f09d2700 -1 mds.l *** got signal Terminated *** 2023-12-01T01:25:34.334 INFO:journalctl@ceph.mds.l.smithi203.stdout:Dec 01 01:25:34 smithi203 bash[147013]: ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715-mds-l 2023-12-01T01:25:34.538 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2a06e29a-8fe6-11ee-95a2-87774f69a715@mds.l.service' 2023-12-01T01:25:34.574 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-01T01:25:34.574 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-12-01T01:25:34.574 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 --force --keep-logs 2023-12-01T01:25:34.873 INFO:teuthology.orchestra.run.smithi077.stdout:Deleting cluster with fsid: 2a06e29a-8fe6-11ee-95a2-87774f69a715 2023-12-01T01:25:36.427 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 --force --keep-logs 2023-12-01T01:25:36.728 INFO:teuthology.orchestra.run.smithi137.stdout:Deleting cluster with fsid: 2a06e29a-8fe6-11ee-95a2-87774f69a715 2023-12-01T01:25:38.276 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 --force --keep-logs 2023-12-01T01:25:38.590 INFO:teuthology.orchestra.run.smithi203.stdout:Deleting cluster with fsid: 2a06e29a-8fe6-11ee-95a2-87774f69a715 2023-12-01T01:25:40.167 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-01T01:25:40.203 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-01T01:25:40.239 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-01T01:25:40.269 INFO:tasks.cephadm:Archiving crash dumps... 2023-12-01T01:25:40.271 DEBUG:teuthology.misc:Transferring archived files from smithi077:/var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/crash to /home/teuthworker/archive/yuriw-2023-11-30_22:47:08-fs-reef-release-distro-default-smithi/7473168/remote/smithi077/crash 2023-12-01T01:25:40.272 DEBUG:teuthology.orchestra.run.smithi077:> sudo tar c -f - -C /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/crash -- . 2023-12-01T01:25:40.310 INFO:teuthology.orchestra.run.smithi077.stderr:tar: /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/crash: Cannot open: No such file or directory 2023-12-01T01:25:40.310 INFO:teuthology.orchestra.run.smithi077.stderr:tar: Error is not recoverable: exiting now 2023-12-01T01:25:40.312 DEBUG:teuthology.misc:Transferring archived files from smithi137:/var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/crash to /home/teuthworker/archive/yuriw-2023-11-30_22:47:08-fs-reef-release-distro-default-smithi/7473168/remote/smithi137/crash 2023-12-01T01:25:40.313 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/crash -- . 2023-12-01T01:25:40.362 INFO:teuthology.orchestra.run.smithi137.stderr:tar: /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/crash: Cannot open: No such file or directory 2023-12-01T01:25:40.362 INFO:teuthology.orchestra.run.smithi137.stderr:tar: Error is not recoverable: exiting now 2023-12-01T01:25:40.364 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/crash to /home/teuthworker/archive/yuriw-2023-11-30_22:47:08-fs-reef-release-distro-default-smithi/7473168/remote/smithi203/crash 2023-12-01T01:25:40.365 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/crash -- . 2023-12-01T01:25:40.401 INFO:teuthology.orchestra.run.smithi203.stderr:tar: /var/lib/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/crash: Cannot open: No such file or directory 2023-12-01T01:25:40.401 INFO:teuthology.orchestra.run.smithi203.stderr:tar: Error is not recoverable: exiting now 2023-12-01T01:25:40.402 INFO:tasks.cephadm:Checking cluster log for badness... 2023-12-01T01:25:40.403 DEBUG:teuthology.orchestra.run.smithi077:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-12-01T01:25:40.436 INFO:teuthology.orchestra.run.smithi077.stderr:grep: /var/log/ceph/2a06e29a-8fe6-11ee-95a2-87774f69a715/ceph.log: No such file or directory 2023-12-01T01:25:40.438 INFO:tasks.cephadm:Compressing logs... 2023-12-01T01:25:40.438 DEBUG:teuthology.orchestra.run.smithi077:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-01T01:25:40.481 DEBUG:teuthology.orchestra.run.smithi137:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-01T01:25:40.483 DEBUG:teuthology.orchestra.run.smithi203:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-01T01:25:40.508 INFO:teuthology.orchestra.run.smithi077.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-12-01T01:25:40.510 INFO:teuthology.orchestra.run.smithi203.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-12-01T01:25:40.511 INFO:teuthology.orchestra.run.smithi137.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-12-01T01:26:33.940 INFO:tasks.cephadm:Archiving logs... 2023-12-01T01:26:33.941 DEBUG:teuthology.misc:Transferring archived files from smithi077:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-11-30_22:47:08-fs-reef-release-distro-default-smithi/7473168/remote/smithi077/log 2023-12-01T01:26:33.942 DEBUG:teuthology.orchestra.run.smithi077:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-01T01:26:36.880 DEBUG:teuthology.misc:Transferring archived files from smithi137:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-11-30_22:47:08-fs-reef-release-distro-default-smithi/7473168/remote/smithi137/log 2023-12-01T01:26:36.881 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-01T01:26:42.626 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-11-30_22:47:08-fs-reef-release-distro-default-smithi/7473168/remote/smithi203/log 2023-12-01T01:26:42.627 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-01T01:26:46.431 INFO:tasks.cephadm:Removing cluster... 2023-12-01T01:26:46.431 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 --force 2023-12-01T01:26:46.730 INFO:teuthology.orchestra.run.smithi077.stdout:Deleting cluster with fsid: 2a06e29a-8fe6-11ee-95a2-87774f69a715 2023-12-01T01:26:47.205 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 --force 2023-12-01T01:26:47.505 INFO:teuthology.orchestra.run.smithi137.stdout:Deleting cluster with fsid: 2a06e29a-8fe6-11ee-95a2-87774f69a715 2023-12-01T01:26:48.010 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2a06e29a-8fe6-11ee-95a2-87774f69a715 --force 2023-12-01T01:26:48.309 INFO:teuthology.orchestra.run.smithi203.stdout:Deleting cluster with fsid: 2a06e29a-8fe6-11ee-95a2-87774f69a715 2023-12-01T01:26:48.793 INFO:tasks.cephadm:Removing cephadm ... 2023-12-01T01:26:48.793 DEBUG:teuthology.orchestra.run.smithi077:> rm -rf /home/ubuntu/cephtest/cephadm 2023-12-01T01:26:48.816 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf /home/ubuntu/cephtest/cephadm 2023-12-01T01:26:48.834 DEBUG:teuthology.orchestra.run.smithi203:> rm -rf /home/ubuntu/cephtest/cephadm 2023-12-01T01:26:48.852 INFO:tasks.cephadm:Teardown complete 2023-12-01T01:26:48.853 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-12-01T01:26:48.863 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-12-01T01:26:48.863 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-01T01:26:48.866 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-01T01:26:48.878 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-01T01:26:48.990 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-12-01T01:26:49.001 INFO:teuthology.task.clock:Checking final clock skew... 2023-12-01T01:26:49.001 DEBUG:teuthology.orchestra.run.smithi077:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-01T01:26:49.004 DEBUG:teuthology.orchestra.run.smithi137:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-01T01:26:49.007 DEBUG:teuthology.orchestra.run.smithi203:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-01T01:26:49.020 INFO:teuthology.orchestra.run.smithi077.stderr:bash: ntpq: command not found 2023-12-01T01:26:49.020 INFO:teuthology.orchestra.run.smithi137.stderr:bash: ntpq: command not found 2023-12-01T01:26:49.022 INFO:teuthology.orchestra.run.smithi203.stderr:bash: ntpq: command not found 2023-12-01T01:26:49.024 INFO:teuthology.orchestra.run.smithi077.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-01T01:26:49.024 INFO:teuthology.orchestra.run.smithi077.stdout:=============================================================================== 2023-12-01T01:26:49.024 INFO:teuthology.orchestra.run.smithi077.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 25 +245us[ +245us] +/- 83ms 2023-12-01T01:26:49.024 INFO:teuthology.orchestra.run.smithi077.stdout:^* hv02.front.sepia.ceph.com 3 6 377 26 -224us[ -227us] +/- 62ms 2023-12-01T01:26:49.024 INFO:teuthology.orchestra.run.smithi077.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 26 +180us[ +180us] +/- 79ms 2023-12-01T01:26:49.024 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-12-01T01:26:49.026 INFO:teuthology.orchestra.run.smithi137.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-01T01:26:49.026 INFO:teuthology.orchestra.run.smithi137.stdout:=============================================================================== 2023-12-01T01:26:49.026 INFO:teuthology.orchestra.run.smithi137.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 26 +208us[ +208us] +/- 83ms 2023-12-01T01:26:49.026 INFO:teuthology.orchestra.run.smithi137.stdout:^* hv02.front.sepia.ceph.com 3 6 377 27 -251us[ -258us] +/- 62ms 2023-12-01T01:26:49.026 INFO:teuthology.orchestra.run.smithi137.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 28 +138us[ +131us] +/- 79ms 2023-12-01T01:26:49.026 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-12-01T01:26:49.027 INFO:teuthology.orchestra.run.smithi203.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-01T01:26:49.027 INFO:teuthology.orchestra.run.smithi203.stdout:=============================================================================== 2023-12-01T01:26:49.028 INFO:teuthology.orchestra.run.smithi203.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 27 +215us[ +216us] +/- 83ms 2023-12-01T01:26:49.028 INFO:teuthology.orchestra.run.smithi203.stdout:^* hv02.front.sepia.ceph.com 3 6 377 25 -252us[ -251us] +/- 62ms 2023-12-01T01:26:49.028 INFO:teuthology.orchestra.run.smithi203.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 27 +149us[ +151us] +/- 79ms 2023-12-01T01:26:49.028 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-12-01T01:26:49.028 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-12-01T01:26:49.042 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-12-01T01:26:49.043 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-12-01T01:26:49.053 DEBUG:teuthology.orchestra.run.smithi077:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-12-01T01:26:49.168 DEBUG:teuthology.orchestra.run.smithi137:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-12-01T01:26:49.261 DEBUG:teuthology.orchestra.run.smithi203:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-12-01T01:26:49.355 DEBUG:teuthology.orchestra.run.smithi077:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-01T01:26:49.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-01T01:26:49.388 DEBUG:teuthology.orchestra.run.smithi137:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-01T01:26:49.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-01T01:26:49.420 DEBUG:teuthology.orchestra.run.smithi203:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-01T01:26:49.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-01T01:26:49.452 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-12-01T01:26:49.467 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-12-01T01:26:49.481 INFO:teuthology.task.internal:Duration was 1840.556765 seconds 2023-12-01T01:26:49.482 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-12-01T01:26:49.493 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-12-01T01:26:49.493 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-01T01:26:49.495 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-01T01:26:49.497 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-01T01:26:49.542 INFO:teuthology.orchestra.run.smithi137.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-01T01:26:49.542 INFO:teuthology.orchestra.run.smithi077.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-01T01:26:49.544 INFO:teuthology.orchestra.run.smithi203.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-01T01:26:49.790 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-12-01T01:26:49.790 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi077.front.sepia.ceph.com 2023-12-01T01:26:49.790 DEBUG:teuthology.orchestra.run.smithi077:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-01T01:26:49.813 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi137.front.sepia.ceph.com 2023-12-01T01:26:49.813 DEBUG:teuthology.orchestra.run.smithi137:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-01T01:26:49.835 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi203.front.sepia.ceph.com 2023-12-01T01:26:49.835 DEBUG:teuthology.orchestra.run.smithi203:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-01T01:26:49.857 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-12-01T01:26:49.858 DEBUG:teuthology.orchestra.run.smithi077:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-01T01:26:49.860 DEBUG:teuthology.orchestra.run.smithi137:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-01T01:26:49.877 DEBUG:teuthology.orchestra.run.smithi203:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-01T01:26:49.948 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-12-01T01:26:49.948 DEBUG:teuthology.orchestra.run.smithi077:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-01T01:26:49.951 DEBUG:teuthology.orchestra.run.smithi137:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-01T01:26:49.953 DEBUG:teuthology.orchestra.run.smithi203:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-01T01:26:50.052 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-12-01T01:26:50.065 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-12-01T01:26:50.065 DEBUG:teuthology.orchestra.run.smithi077:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-01T01:26:50.093 DEBUG:teuthology.orchestra.run.smithi137:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-01T01:26:50.120 DEBUG:teuthology.orchestra.run.smithi203:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-01T01:26:50.149 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-12-01T01:26:50.165 DEBUG:teuthology.orchestra.run.smithi077:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-01T01:26:50.168 DEBUG:teuthology.orchestra.run.smithi137:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-01T01:26:50.171 DEBUG:teuthology.orchestra.run.smithi203:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-01T01:26:50.191 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern = core 2023-12-01T01:26:50.194 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern = core 2023-12-01T01:26:50.215 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern = core 2023-12-01T01:26:50.241 DEBUG:teuthology.orchestra.run.smithi077:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-01T01:26:50.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-01T01:26:50.261 DEBUG:teuthology.orchestra.run.smithi137:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-01T01:26:50.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-01T01:26:50.279 DEBUG:teuthology.orchestra.run.smithi203:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-01T01:26:50.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-01T01:26:50.297 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-12-01T01:26:50.309 INFO:teuthology.task.internal:Transferring archived files... 2023-12-01T01:26:50.309 DEBUG:teuthology.misc:Transferring archived files from smithi077:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-11-30_22:47:08-fs-reef-release-distro-default-smithi/7473168/remote/smithi077 2023-12-01T01:26:50.310 DEBUG:teuthology.orchestra.run.smithi077:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-01T01:26:50.356 DEBUG:teuthology.misc:Transferring archived files from smithi137:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-11-30_22:47:08-fs-reef-release-distro-default-smithi/7473168/remote/smithi137 2023-12-01T01:26:50.356 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-01T01:26:50.395 DEBUG:teuthology.misc:Transferring archived files from smithi203:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-11-30_22:47:08-fs-reef-release-distro-default-smithi/7473168/remote/smithi203 2023-12-01T01:26:50.395 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-01T01:26:50.439 INFO:teuthology.task.internal:Removing archive directory... 2023-12-01T01:26:50.439 DEBUG:teuthology.orchestra.run.smithi077:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-01T01:26:50.443 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-01T01:26:50.445 DEBUG:teuthology.orchestra.run.smithi203:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-01T01:26:50.488 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-12-01T01:26:50.504 INFO:teuthology.task.internal:Not uploading archives. 2023-12-01T01:26:50.504 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-12-01T01:26:50.516 INFO:teuthology.task.internal:Tidying up after the test... 2023-12-01T01:26:50.517 DEBUG:teuthology.orchestra.run.smithi077:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-01T01:26:50.521 DEBUG:teuthology.orchestra.run.smithi137:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-01T01:26:50.525 DEBUG:teuthology.orchestra.run.smithi203:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-01T01:26:50.535 INFO:teuthology.orchestra.run.smithi077.stdout: 262320 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 1 01:26 /home/ubuntu/cephtest 2023-12-01T01:26:50.537 INFO:teuthology.orchestra.run.smithi137.stdout: 262181 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 1 01:26 /home/ubuntu/cephtest 2023-12-01T01:26:50.545 INFO:teuthology.orchestra.run.smithi203.stdout: 262181 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 1 01:26 /home/ubuntu/cephtest 2023-12-01T01:26:50.546 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-12-01T01:26:50.558 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-12-01T01:26:50.587 INFO:teuthology.nuke:Checking targets against current locks 2023-12-01T01:26:50.615 DEBUG:teuthology.nuke:shortname: smithi077 2023-12-01T01:26:50.615 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-01T01:26:50.640 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi077.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-30_22:47:08-fs-reef-release-distro-default-smithi/7473168', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-01 00:50:54.330090', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-12-01T01:26:50.644 DEBUG:teuthology.nuke:shortname: smithi137 2023-12-01T01:26:50.645 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-01T01:26:50.671 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi137.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-30_22:47:08-fs-reef-release-distro-default-smithi/7473168', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-01 00:50:54.328310', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-12-01T01:26:50.673 DEBUG:teuthology.nuke:shortname: smithi203 2023-12-01T01:26:50.673 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-01T01:26:50.707 INFO:teuthology.orchestra.console.smithi077:Power off 2023-12-01T01:26:50.707 DEBUG:teuthology.orchestra.console.smithi077:pexpect command: ipmitool -H smithi077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-01T01:26:50.718 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi203.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-30_22:47:08-fs-reef-release-distro-default-smithi/7473168', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-01 00:50:54.331166', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-12-01T01:26:50.734 DEBUG:teuthology.orchestra.console.smithi077:power off output: Chassis Power Control: Down/Off 2023-12-01T01:26:50.734 DEBUG:teuthology.orchestra.console.smithi077:pexpect command: ipmitool -H smithi077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-01T01:26:50.760 INFO:teuthology.orchestra.console.smithi137:Power off 2023-12-01T01:26:50.760 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-01T01:26:50.780 DEBUG:teuthology.orchestra.console.smithi077:check power output: Chassis Power is on 2023-12-01T01:26:50.793 DEBUG:teuthology.orchestra.console.smithi137:power off output: Chassis Power Control: Down/Off 2023-12-01T01:26:50.794 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-01T01:26:50.817 INFO:teuthology.orchestra.console.smithi203:Power off 2023-12-01T01:26:50.817 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-01T01:26:50.829 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is on 2023-12-01T01:26:50.842 DEBUG:teuthology.orchestra.console.smithi203:power off output: Chassis Power Control: Down/Off 2023-12-01T01:26:50.842 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-01T01:26:50.869 DEBUG:teuthology.orchestra.console.smithi203:check power output: Chassis Power is on 2023-12-01T01:26:54.782 DEBUG:teuthology.orchestra.console.smithi077:pexpect command: ipmitool -H smithi077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-01T01:26:54.829 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-01T01:26:54.870 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-01T01:26:54.895 DEBUG:teuthology.orchestra.console.smithi077:check power output: Chassis Power is on 2023-12-01T01:26:54.942 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is on 2023-12-01T01:26:54.980 DEBUG:teuthology.orchestra.console.smithi203:check power output: Chassis Power is on 2023-12-01T01:26:58.898 DEBUG:teuthology.orchestra.console.smithi077:pexpect command: ipmitool -H smithi077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-01T01:26:58.943 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-01T01:26:58.981 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-01T01:26:59.010 DEBUG:teuthology.orchestra.console.smithi077:check power output: Chassis Power is on 2023-12-01T01:26:59.053 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is off 2023-12-01T01:26:59.092 DEBUG:teuthology.orchestra.console.smithi203:check power output: Chassis Power is on 2023-12-01T01:26:59.154 INFO:teuthology.orchestra.console.smithi137:Power off completed 2023-12-01T01:27:03.014 DEBUG:teuthology.orchestra.console.smithi077:pexpect command: ipmitool -H smithi077.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-01T01:27:03.093 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-01T01:27:03.126 DEBUG:teuthology.orchestra.console.smithi077:check power output: Chassis Power is off 2023-12-01T01:27:03.205 DEBUG:teuthology.orchestra.console.smithi203:check power output: Chassis Power is off 2023-12-01T01:27:03.227 INFO:teuthology.orchestra.console.smithi077:Power off completed 2023-12-01T01:27:03.306 INFO:teuthology.orchestra.console.smithi203:Power off completed 2023-12-01T01:27:03.411 INFO:teuthology.run:Summary data: description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/fsx}} duration: 1840.5567648410797 failure_reason: 'error during scrub thrashing: rank damage found: {''backtrace''}' flavor: default owner: scheduled_yuriw@teuthology status: fail success: false 2023-12-01T01:27:03.411 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-01T01:27:03.489 INFO:teuthology.run:FAIL